You might be interested to know that Clam/ClamWin's detection rate of new malware has increased again. It is now rated at 16% in detecting new malware at http://winnow.oitc.com/avmalwarestats.php in its ongoing study. This is a better rate than 10 of the commercial antivirus companies, and it is within a point or two of several others, including F-Prot. When I started monitoring this, Clam detected 14% of new malware. The potentially unwanted application signatures (or PUA), when fully implemented, should help improve it a bit more.

Regards,

TrendMicro detected zero new viruses even with heuristics?? Right.......... This test is a highly suspect. The fact that they are calling "HEUR/Packed" a detection is pretty foolish. Those heuristics pickup all packed files, malware or not.

The test conductoer, Oak Island Trading Company, says the test consists of 100% malware which they consider to be all new because each malware isn't spotted by very many AVs. I don't know about Trend Micro, but some of the other AVs like Bitdefender and Kaspersky are close to the same results they had on the AV-Comparatives Retrospective Tests, but NOD32 seems low. Trend seems to stay away from tests, so there's not much to compare it to. I notice Trend is one of the sponsors listed on the OITC summary page.

If an AV has real good dynamic heirustics/behavior blocking, it probably won't show up on this test if all they are doing is just scanning files and not allowing malware to actually run. That's probably why NOD32 is so low, and maybe Trend also. Clam is doing better than many might think it should--with practically no heuristics but with well-maintained signatures.

Regards,

NOD32's heuristics do not require the sample to physically run on the box as they run the malware inside an emulation environment.

How they calculate their detection percentage is pretty bad too:


3887 samples vs 4458 samples... these are obviously not identical test sets since there are more in some tests than others... who is to say that any of them are the same at all?? They are comparing apples to oranges. Not a lot about this data makes sense.

Yeah--I noticed the totals are different for each AV. And it's hard to reconcile what they show for NOD32 with other tests that show it to be much better. This emphasizes the need to fully explain the testing methodology if you want people to understand/believe it. I guess they are looking at a very small malware subset that is somewhat kind to Clam and unkind to Trend/NOD32.

Regards,

The online multi-scanners use Linux versions of the antivirus scanners. Jotti tells me that they set all scanners for maximum detection except one, which is de-tuned to reduce its number of false positives. Their scanner versions don't work with any live data, which negates behavior blocking or intrusion detection. Heuristics and sandbox capabilities are used, however, if the scanner has them. To sum it up, on Jotti the AVs are used as file scanners with advanced heuristics and sandbox enabled.

Other services like the OITC ongoing test in question (which is based on results from the Virus Total multi-scanner project) probably operate in a similar manner, although Virus Total doesn't have Clam set up to detect Potentially Unwanted Applications yet.

NOD32 does so poorly on the multi-scanners compared to its results on the AV Comparatives tests, however, that I have to believe that its full capacity is not being utilized on the multi-scanner services, and Trend Micro may be in a similar position as well.

It's apples and oranges again, isn't it? It's hard to find a good believable/reliable AV test. I guess AV Comparatives is the best available AV test. They use Windows versions of scanners on Windows boxes.

Regards,

None of the AVs on the OITC test that show zero detections, including Trend, are actually participating in it or are not included in the Virus Total scanners from which the test data is derived. Here's what OITC says:

"Trend was originally listed as going to participate at virustotal.com
along with Arcavir, KingSoft and mks_vi. To date they have not and we
use data from virustotal.com in out test as described in our
methodology. However, our database still has placeholders just in
case the reevaluate and decide to participate. Its a shame really."


I think this is a serious test, but, again, it's hard to compare an AV version used on Linux with the same version used on Windows. It appears that the functionality differs. I'm sure the AV companies devote more time to Windows in developing/testing their products, since that is where the large number of customers are.

I'll be quiet now!

Regards,