ClamWin Free Antivirus :: View topic

Posted: Fri Jul 06, 2007 3:15 pm

kcbrown wrote:

Whoever wrote the code in libclamav/readdb.c has now officially failed data structures and/or algorithms. Smile

Supposedly this was fixed in the devel branch of clamav. You might want to check their svn to see if that change is in place, see if you can contact someone to push the patch through.

Posted: Fri Jul 06, 2007 10:31 pm

b0ne wrote:

kcbrown wrote:

Whoever wrote the code in libclamav/readdb.c has now officially failed data structures and/or algorithms. Smile

Supposedly this was fixed in the devel branch of clamav. You might want to check their svn to see if that change is in place, see if you can contact someone to push the patch through.

I looked over the source in the trunk. The bug is still there, though the code itself is somewhat different. It may be faster but it's still O(n^2).

So they still fail the class. Wink

Posted: Fri Jul 06, 2007 10:41 pm

Give Clam a while and see if they use the improved code. If they don't by the next ClamWin version, why not make the improved code a user option in ClamWin until Clam adopts it? Someone needs to check it occasionally.

Regards,

Posted: Fri Jul 06, 2007 11:01 pm

GuitarBob wrote:

Give Clam a while and see if they use the improved code. If they don't by the next ClamWin version, why not make the improved code a user option in ClamWin until Clam adopts it? Someone needs to check it occasionally.

Regards,

That would require either compiling two separate versions of libclamav, or putting a lot of conditional code in the backend. In other words, that's probably a lot more trouble than it's worth, especially since the currently released code is obviously very, very broken. Writing the code such that the database load behaviour were selectable would also be a great deal of trouble and would complicate the code more than it needs to be (the simpler the code, the more reliable it is, though it obviously needs to be simple in an intelligent way, as this bug illustrates so well).

I'd be more inclined to agree with you on this if it were a frontend feature or something, but this is the backend we're talking about here...

Posted: Sat Jul 07, 2007 1:29 am

GuitarBob wrote:

Thanks for your work. I hope they can get it into ClamWin soon. Is this also something that can be used by ClamAV as well? They are really separate projects, and giving it to one project might not necessarily get it to the other project. In fact, ClamAV would probably be the one to give it to, since, at this point, ClamWin essentially provides a Windows GUI to ClamAV, with specific Windows coding only as needed.

The fix is to the general ClamAV code (libclamav in particular), so it applies universally to all platforms that ClamAV runs on.

Posted: Sat Jul 07, 2007 3:16 pm

ClamAV is very much aware of this problem (they've added a FAQ specifically for this, recommending the use of clamd/clamscand in stead of clamscan). I hope they will give this issue some priority.

Posted: Mon Jul 09, 2007 8:12 am

kcbrown wrote:

Whoever wrote the code in libclamav/readdb.c has now officially failed data structures and/or algorithms. Smile

Trying to maintain an ordered single-linked list at insertion time? For shame... Very Happy

The problem is the MD5 section linked list insertion in cli_loadhdb(). For every MD5 section that it encounters in the virus database, it scans the entire list looking for the right place to put the new node. This is an O(n^2) algorithm.

Like I said, for shame! Very Happy

Anyway, I fixed it, and it's now just a question of how I'm supposed to get the fix to someone who can apply it.

The fix takes the entire scan runtime against an almost-empty text file on my Athlon64 linux box from 33 seconds to 2 seconds. I can't tell how much of that time is spent reading the database versus other things.

Let's just say this is a major win and leave it at that. Depending on how that list is used, more performance might be had by changing the data structure being used from a linked list to a variant of a tree, but this seems to take care of the lowest-hanging fruit.

I'd attach the source patch here but I don't see any way of attaching anything...

Edit: I've submitted bug 1749001 against this issue and supplied the patch there. Here's the link to the bug: https://sourceforge.net/tracker/index.php?func=detail&aid=1749001&group_id=105508&atid=641462 https://sourceforge.net/tracker/index.php?func=detail&aid=1749001&group_id=105508&atid=641462

Are you sure this patch also applies to clamav 0.91rc2? The problem of db loading time should already be fixed in this version.

Posted: Mon Jul 09, 2007 1:26 pm

I believe he's talking about 0.90.2, which is the latest stable version available under ClamWin.

Regards,

Posted: Mon Jul 09, 2007 1:56 pm

kcbrown wrote:

b0ne wrote:

kcbrown wrote:

Whoever wrote the code in libclamav/readdb.c has now officially failed data structures and/or algorithms. Smile

Supposedly this was fixed in the devel branch of clamav. You might want to check their svn to see if that change is in place, see if you can contact someone to push the patch through.

I looked over the source in the trunk. The bug is still there, though the code itself is somewhat different. It may be faster but it's still O(n^2).

So they still fail the class. Wink

It would be really great if you can provide a patch against current svn of clamav:
https://svn.clamav.net/svn/clamav-devel/trunk/

and send it to the clamav developers list:
https://lists.clamav.net/mailman/listinfo/clamav-devel

or clamav bugzilla:
https://wwws.clamav.net/bugzilla/

Posted: Mon Jul 09, 2007 3:32 pm

In my opinion, the ClamWin team should have some sort of expedited "pipeline" (and vice-versa) to ClamAV for cases like this. It should not be difficult for users to effect a needed change. Users are not only the "customers" of the product, they can be a valuable developmental resource as well. This is what Open Source is supposed to be about.

Regards,

Posted: Mon Jul 09, 2007 7:39 pm

>> or clamav bugzilla:
>> https://wwws.clamav.net/bugzilla/ https://wwws.clamav.net/bugzilla/

Bug report 565 has been filled on clamav bugzilla. It should be visible for others soon.
https://wwws.clamav.net/bugzilla/show_bug.cgi?id=565 https://wwws.clamav.net/bugzilla/show_bug.cgi?id=565

Best regard[/url]