Avast WEBforum

Consumer Products => Avast Free Antivirus / Premium Security (legacy Pro Antivirus, Internet Security, Premier) => Topic started by: Kobra on June 15, 2004, 04:37:11 PM

Title: Kobra's AV test on 6-14-04
Post by: Kobra on June 15, 2004, 04:37:11 PM
Kobra's 6-14-04 AV Test.

Testbed consisted of 321 Viruses, Trojans and Worms, all for the Windows32 environment, and all reasonably new samples.  I don't have any data on whether some of these are zoo, or ITW, but they are all real threats I feel someone is likely to encounter, since I got them off the internet (and i've verified they are real as each sample must be detected by at least 4 AV's for me to consider it).  All scanners were installed on a clean system, without any traces of other anti-virus softwares - between each test the system and directories were cleaned, and the registry was sweeped.  Each AV product was treated with a double-reboot, one before, and one after installation.   Each scanner was set at its highest possible settings, and was triple checked for proper options and configuration.  Most products were the full registered version when possible, others were fully functional unrestricted trials.  All products were tested with the current version as of 6-14-04, and the latest definitions for that date.  Each product was run through the test set a minimum of 3 times to establish proper settings and reliability, the only product to exhibit some variance on this was F-Secure, which had one scan come up less than the other two without any settings changes indicating a possible stability issue.

The final standings:

1)   eXtendia AVK
2)   Kaspersky 5.0/4.5
2)   McAfee VirusScan 8.0
3)   F-Secure
4)   GData AVK
5)   RAV + Norton (2 way tie)
6)   Dr.Web
7)   CommandAV + F-Prot + BitDefender (3 Way Tie)
8)   ETrust
9)   Trend
10) Avast! Pro
11) Panda AV
12) KingSoft
13) NOD32
14) AVG Pro
15) AntiVIR
16) ClamWIN
17) UNA
18) Norman
19) Solo
20) Proland
21) Sophos
22) Hauri
23) CAT Quickheal
24) Ikarus

Heuristics seemed to play some of a roll in this test, as no AV had every virus in my test in their definitions, and products with stronger heuristics were able to hold their position towards the top of the test. Double/Multi engined products put up strong showings as well, proving to me that the redundacy method works, and I think more AV companies should considering double-engines. The strongest heurisitical AV I noticed was F-Prot/Command, picking up only 247 samples with definitions but they were able to power through 67 additional hits on "Possible Virus" indicators - very strong!  Norton with BloodHound activated had 30 Heuristical pickups, and DrWeb rounded up the pack with 20 heuristical pickups.  eXtendia AVK grabs the number one slot with double engine scanning, anything the KAV engine missed, the RAV engine picked up with great redundancy on the double engine/definition system.  McAfee actually missed only 2 samples with its definitions, but picked those 2 up as "Suspicious File", and therefore, scores nearly perfect as well.

The biggest dissapointments for me were Norman and Nod32.  Even with Advanced-Heuristics enabled, NOD32 failed to pick up a large portion of the samples.  Norman, while finding some of the toughest samples, managed to completely miss a large portion of them!  Showing that their sandbox-emulation system has great potetential, but its far from complete.

Actual test numbers were:

Total Samples/Found Samples (321 total possible) + Number Missed + Detection Percentage

1)  eXtendia AVK - 321/321 0 Missed - 100%
2)  Kaspersky 5.0 - 320/321 1 Missed - 99.70% (with Extended Database ON)
 2)  McAfee VirusScan 8.0 - 319/321 + 2 (2 found as joke programs - heuristically) - 99%
3)  F-Secure - 319/321 2 Missed - 99.37%
4)  GData AVK - 317/321 4 Missed - 98.75%
5)  RAV + Norton (2 way tie) - 315/321 6 Missed - 98.13%
6)  Dr.Web - 310/321 11 Missed - 96.57%
7)  CommandAV + F-Prot + BitDefender (3 Way Tie) - 309/321 12 Missed - 96.26%
8)  ETrust - 301/321 20 Missed - 93.76%
9)  Trend - 300/321 21 Missed - 93.45%
10) Avast! Pro - 299/321 22 Missed - 93.14%
11) Panda - 298/321 23 Missed - 92.83%
12) KingSoft - 288/321 33 Missed - 89.71%
13) NOD32 - 285/321 36 Missed  (results identical with or without advanced heuristics) - 88.78%
14) AVG Pro - 275/321 46 Missed - 85.66%
15) AntiVIR - 268/321 53 Missed - 83.48%
16) ClamWIN - 247/321 74 Missed - 76.94%
17) UNA - 222/321 99 Missed - 69.15%
18) Norman - 215/321 106 Missed - 66.97%
19) Solo - 182/321 139 Missed - 56.69%
20) Proland - 73/321 248 Missed - 22.74%
21) Sophos - 50/321 271 Missed - 15.57%
22) Hauri - 49/321 272 Missed - 15.26%
23) CAT Quickheal - 21/321 300 Missed - 6%
24) Ikarus - Crashed on first virus. - 0%

Interesting also to note, is the detection level of the US AVK version with KAV+RAV engines was higher than the German version with KAV+BitDefender engines.  Several vendors have free versions of their for purchase AV's, we didn't test the free versions, as it would serve no purpose for this test, but based on the results, none of the free versions would have been very impressive anyway. The term "Heuristics" seems like it should be taken very liberally, as some products that claim to be loaded with Heuristics scored miserably on items they clearly didn't have definitions for.  Scanning speed was not measured, as it was totally irrelevant to my testing, and on-access scanners were not tested, as it would have been too time consuming, but considering most products have similar on-access engines as on-demand, and use the same database, results most likely, would be very similar.

Cut through the hype, cut through the marketing schemes, this was a real test, with real samples, and none of these samples were provided to the antivirus software vendors in advance.  This is real world, and these are likely badguys you'll encounter, since I got them in my real encounters, and all were aquired on the internet in daily activities which anyone out there might be involved in. (Installing shareware, filesharing, surfing, etc).  Keep in mind that with ITW tests the AV vendors have full disclosure of what they will be tested on in advance, not so here, so heuristics and real detection algorithms will play a big part, as well as the depth and scope of their definition database.

Title: Re:Kobra's AV test on 6-14-04
Post by: RejZoR on June 15, 2004, 04:52:33 PM
Ooooooo and where is NOD32's excelent heuristic part :P
Second one is that you didn't include ligitim samples. Those would provide much higher false positive number for heuristic based antivirus programs :P
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 15, 2004, 05:02:25 PM
These were all real virus/trojans/worms as indicated.  No false samples, no cleaned samples, and no fake samples.  So i'm not sure what you mean by "ligitim" samples?  ???

NOD32 results were completely the same whether I used norman scan, or the shell extension /AH scan.  No different.
Title: Re:Kobra's AV test on 6-14-04
Post by: Tipton on June 15, 2004, 05:03:41 PM
So are you going to submit the threats that Avast missed, to the Avast team for consideration in their virus def updates?

Douglas
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 15, 2004, 05:21:25 PM
Honestly, I was *HOPING* to be surprised by a ton of things in this test, and really all I did was re-enforce many of the other testing sites on their results, mine are very close to theres, which actually shocked me, because i'm sure my samples aren't the same.  This tells me overall, I think this might be a great guage of these products.

Also, I wanted to test the multi-engined products against the others, since most testers seem to not like testing them.  Strong showings by F-Secure, and the AVK' brothers proved this idea works, and works incredibly well.  The strength of the KAV engine cannot be denied as well, since all but one of the top 5 products use the KAV engine.  :P  I forgot to add, one product I tested was called V-Catch, and turned out to be a trojan downloader and spyware application masking as a AV product.. LOL!  Thankfully it was the last product I tested, and I just reformatted, I think it downloaded 30 trojans to my system. 8-)

I did NOT test any Dos viruses, as this is completely retarded to test these in a windows based environment, it tells us nothing.  I cannot understand why Clementi at AV-Comparatives bothers to test them, all they do is skew his test results badly.  For example on his test, NOD32 scored 95.51%, but without DOS or other OS samples, NOD32 scored only 87.71%.  Which amazingly enough, is within 1% variance of *MY* results.  So i'm oblivious as to why he skews his own results for no real purpose?  Who the hell cares what a product scores on DOS?!?   ???
Title: Re:Kobra's AV test on 6-14-04
Post by: .: Mac :. on June 15, 2004, 05:55:08 PM
Kobra, I admire the effort you put into these tests. But I find this
Quote
24) CAT Quickheal - 21/321 300 Missed - 6%
hard to belive! Quick Heal Has the Check-Mark Level one certification (meaning it detects all ITW viruses) and it had the VB100% award.

The Last time QuickHeal was reviewed by the VB guys on XP Pro this is what they had to say

Quote
     Summary
          o ItW Overall - 100.00%
          o ItW Overall (o/a) - 100.00%
          o ItW File - 100.00%
          o Macro - 97.54%
          o Standard - 80.67%
          o Polymorphic - 91.08%

      Quick Heal has a tendency towards better detection of more recent viruses or those which are currently in the wild. This selectivity is commonly associated with a fast throughput rate for clean files, as was indeed the case for Quick Heal. With such selectivity the chance of false positives is reduced - Quick Heal generated none. With complete detection of viruses in the ItW test set, a VB 100% is netted by CAT.
Dont get me wrong Im not saying anything bad about your tests im just saying how can CAT get all those awards and miss 300 of your viruses?
Title: Re:Kobra's AV test on 6-14-04
Post by: RejZoR on June 15, 2004, 06:08:45 PM
Maybe those were not exactly ITW :P Non-ITW samples are also very important.
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 15, 2004, 06:09:54 PM
Mac, people need to understand what "ITW" means..  Of course any AV should score 100% on ITW tests, because ITW Viruses are provided to the AV companies in ADVANCE of the test!  Personally, I think ITW testing is virtually meaningless to real world users.  *ALL* the ITW testing tells me, is how well a company is at maintaining definitions for the ITW institute test sample-set.  Nothing more, nothing less.

I used to think "OMG, it scores 100% on ITW tests!" and started basing my usage of a AV off that, and let me tell you, it was a sad misconception on my part.  People see ITW and automagically assume that it means 100% from everything thats out there.  Hardly.. LOL..  I mean, look at NOD32, a product that on virtually every REAL test, scores in the 80 percentile range, but scores 100% on ITW.  Why?  Becuase they make sure they have all ITW definitions in their database, and check them extensively to avoid ITW false positives.

Now in addition, you'd be HORRIBLY mistaken to think that ITW covers Trojans, Worms, Malicious downloaders/droppers, and other things. It doesn't... ITW covers exactly what it says, VIRUSES.  You'd be further mistaken to assume ITW covers all known circulating viruses, it doesn't, it just covers what ONE organization of people considers to be the most prevelant circulating threats out there. In fact, I personally no longer use VB or Checkmark to make my AV decisions, becuase they are so limited in their scope comparative to whats actually out there.

PS: Remember, my test bed included Viruses, Trojans, Droppers and Worms.  Theres bigger threats out there than typical annoying viruses, and an AV that ignores those threats, is a poor AV in my opinion.
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 15, 2004, 07:09:11 PM
Side note: Someone recommended I test KAV4.5, and I did.  It missed only 1 sample, and scored 99.68%.  Considering i'd put the margin of error at 1% either way, thats a 100% product.

On recommendation from KAV5 users, i'm retesting KAV5 with the extended database download.  Which should make it 100%, or very close to it, according to the people i've talked to that deal with KAV.  KAV5 apparently defaults to the non-extended DB.

Edit: KAV5.0 now tested with extended DB option on, and it scores the same as 4.5, moving KAV5 up to second place along with KAV4.5.
Title: Re:Kobra's AV test on 6-14-04
Post by: .: Mac :. on June 15, 2004, 10:03:55 PM
Kobra still QuickHeal detects 80% of standard viruses. BTW did you turn on QuickHeals herustics they are not on by default
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 15, 2004, 11:21:33 PM
Just for you Mac, I retested, double checked every setting, and re-checked my testing setup.  Same results.   Check the time/date stamp on the Quickheal interface.

(http://home.comcast.net/~prolawn00/cat.JPG)

Personally, I put *ZERO* stock in what Virus Bulletin says.  Remember, these are the same guys that say NOD32 scores 100% in *ALL* catagories, and thats just flat out *BS* with capitol letters...
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 16, 2004, 12:22:27 AM
Side note I reinstalled Avast again, went through settings MANY times, re-checked, checked again, and changed some things, and got Avast to detect 299 Viruses, up from 292.  Nothing I can do will increase this further.  I will adjust my rating of it in the review. =)  Avast! Moves ahead of Panda in my test now.  Also, someone requested a testing of Ahn's V3 Pro.. Man, it has a great interface, and tons of options, but sure missed the detections!

I'm preparing to zip up and submit these missed samples to Avast as well. Heres the updated results:

1)  eXtendia AVK - 321/321 0 Missed - 100%
2)  Kaspersky 5.0 - 320/321 1 Missed - 99.70% (with Extended Database ON)
2)  McAfee VirusScan 8.0 - 319/321 + 2 (2 found as joke programs - heuristically) - 99%
3)  F-Secure - 319/321 2 Missed - 99.37%
4)  GData AVK - 317/321 4 Missed - 98.75%
5)  RAV + Norton (2 way tie) - 315/321 6 Missed - 98.13%
6)  Dr.Web - 310/321 11 Missed - 96.57%
7)  CommandAV + F-Prot + BitDefender (3 Way Tie) - 309/321 12 Missed - 96.26%
8)  ETrust - 301/321 20 Missed - 93.76%
9)  Trend - 300/321 21 Missed - 93.45%
10) Avast! Pro - 299/321 22 Missed - 93.14%
11) Panda - 298/321 23 Missed - 92.83%
12) KingSoft - 288/321 33 Missed - 89.71%
13) NOD32 - 285/321 36 Missed  (results identical with or without advanced heuristics) - 88.78%
14) AVG Pro - 275/321 46 Missed - 85.66%
15) AntiVIR - 268/321 53 Missed - 83.48%
16) ClamWIN - 247/321 74 Missed - 76.94%
17) UNA - 222/321 99 Missed - 69.15%
18) Norman - 215/321 106 Missed - 66.97%
19) Solo - 182/321 139 Missed - 56.69%
20) V3 Pro - 109/321 212 Missed - 33.95%
21) Proland - 73/321 248 Missed - 22.74%
22) Sophos - 50/321 271 Missed - 15.57%
23) Hauri - 49/321 272 Missed - 15.26%
24) CAT Quickheal - 21/321 300 Missed - 6%
25) Ikarus - Crashed on first virus. - 0%




Title: Re:Kobra's AV test on 6-14-04
Post by: KezzerDrix on June 16, 2004, 12:51:05 AM
I didn't think it was that easy to simply go and download MAL-Ware.   :-\  

Did you find them all at one place? Can you supply a link\links?  

Should the FBI be notified?  
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 16, 2004, 12:57:56 AM
I think the issue becomes an issue when you start "Distributing" the viruses with malicious intent.  Professionals use viruses to test all of the time, I use them to analyze and examine, and test, as I run into them.  Since I write for several technical sites, i'm well within my legal right, especially considering my samples are read-only marked.  ;D  Remember, AV developers gotta gets there samples from somewhere, and most of the time, they get them from hobbiests or users that send them in after finding them.

On a side note, i've gone through all the logs again, and have found some issues with Avast "Skipping" files I don't want it to skip.  I'll have to take these up with the Avast guys, because if thats the cast, it would dramatically effect Avasts scores.  I've checked all my settings on my end, and theres nothing I can do i've not already done.
Title: Re:Kobra's AV test on 6-14-04
Post by: Dwarden on June 16, 2004, 01:51:43 AM
i wonder about this AV

http://www.v-buster.com/

 ;D :D 8) ??? :o
Title: Re:Kobra's AV test on 6-14-04
Post by: .: Mac :. on June 16, 2004, 02:16:11 AM
Kobra I have notified Quick Heal support Staff (Useing my Dad's Quick Heal registration). I gve them the link to this thread. Reply is as follows:



Dear Kyle,

     We are working on more advanced heuristics. This new engine will be introduced in Quick Heal 7.02. Please tell Kobra to retest Quickheal once the new engine is released. We really do not see how Quickheal missed that many of his samples. If he would like have him send the samples to you for submission (Dont forget to include your Registration code in the email).


Sincerely,
The Quick Heal Team
Http://www.QuickHeal.com
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 16, 2004, 02:35:21 AM
Kobra I have notified Quick Heal support Staff (Useing my Dad's Quick Heal registration). I gve them the link to this thread. Reply is as follows:

Dear Kyle,

     We are working on more advanced heuristics. This new engine will be introduced in Quick Heal 7.02. Please tell Kobra to retest Quickheal once the new engine is released. We really do not see how Quickheal missed that many of his samples. If he would like have him send the samples to you for submission (Dont forget to include your Registration code in the email).


Sincerely,
The Quick Heal Team
Http://www.QuickHeal.com

Any date/time on the new engine?  The biggest thing I noticed with QuickHeal, was it claims to have heuristics, yet I witnessed *NO* Heuristics in action.  I've re-installed it twice now and re-tested twice to make sure.

Secondly, how big is their database?  I'm guessing its pretty small, and with or without heuristics, a new AV company is at a severe disadvantage because signatures take time to build, unless they arrange to buy or rent definitions from another company. One product, Ahn's V3 Pro, has a EXCEPTIONAL interface and layout, and incredible options.  But their definition base is so small, its just not a viable product for most people.

I've got one more test i'm going to try with Quickheal, and thats going to be on a Win98 machine and see if it behaves any differently, then i'm done with it for now.
Title: Re:Kobra's AV test on 6-14-04
Post by: .: Mac :. on June 16, 2004, 03:07:22 AM
oh it has herustics(Weak) However it does have a herustic-like worm detector called the Quick Heal Sensor which runs at startup to check for suspicious changes in the registry and also looks in real time for methods common for worms spreading.

Quote
SENSOR FOR NEW WORMS, TROJANS AND BACKDOORS

This new sensational technology is designed to fight the threats posed by new Trojans, Worms and BackDoors.

    * Checks most sensitive areas of the system
    * Traps and captivates any new Trojan, Worm, Backdoors and any other malicious code
    * Powerful Protection from Internet Threats.
    * Proactive Technology kills the malicious code before it can act.


The database is a decent sized one, however alot of the old DOS viruses are ommitted. (Most are extinct anyway)


BTW Quickheal has a Personal Firewall that is in BETA form right now. it will be $28 when finished. Mabye you could do a firewall roundup next?
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 16, 2004, 03:34:19 AM
i wonder about this AV

http://www.v-buster.com/

 ;D :D 8) ??? :o

You guys ready for this?

Ok, the bad first:
This is probably the worst layout and implemented AV i've ever seen in my life, its horrible to install, horrible to run, and really is a DOS program overlayed with a really bad WindowsGUI.   This appears to have NO archival/packer support, and cannot detect archived baddies whatsoever. (well, it is a dos program after all).  Overall, its a gross looking and old school operating program.

Now the good:
This thing, without a doubt, the coolest and neatest little thing i've seen in awhile.  It is definition less, but manages to to detect 265 out of 321 baddies, and considering *MANY* are packed/archived, thats probably 100% score - right out of the box without any connection to the internet and no ability to update.  Its scoring 82.55% without the ability to unpack/unarchive?  Ironically, most of the baddies its finding, its finding with some type of Heuristics or code emulation and its very fast. Alerting me with "Definately a unknown Trojan" or "Strange Acting File, Probably Virus".

Nobody could possibly like this program I don't think, as far as running it, and using it, its rather a pain in the rear, but I cannot argue with its detection/heuristics and ability to find new stuff.  Maybe the guys at Avast should contact this dude, and try to license his technology?   :o
Title: Re:Kobra's AV test on 6-14-04
Post by: .: Mac :. on June 16, 2004, 06:14:55 AM
another email from CAT on your any date/time for new engine question


Hello Kyle,

    Thanks for contacting Quick Heal support!  The new engine is in the 4th Beta stage. We run them through 5 Beta versions to work out most of the bugs. The Beta engine is only given to the public upon request. late June to mid July you should see the public release.


Sincerely,
The Quick Heal Team
Http://www.QuickHeal.com
Title: Re:Kobra's AV test on 6-14-04
Post by: Dwarden on June 16, 2004, 02:57:49 PM
Cobra, thanks that exactly fit what i "heard" about that AV. I wonder if author is preparing for full w32/w64 version as it will be bad to throw away such qualite heuristic engine (the only one comparable which come to my mind is Dr. Web ).
Title: Re:Kobra's AV test on 6-14-04
Post by: Vlk on June 16, 2004, 04:26:22 PM
About V-Buster: really funny looking tiny program but certainly worth looking at...
Title: Re:Kobra's AV test on 6-14-04
Post by: Technodrome on June 16, 2004, 05:00:14 PM
Inexperienced users should clearly steer away from this AV. V-Buster is purely heuristics scanner and it will probably give you a lots of FPs.

There is a similar (freeware) av scanner (ROSE SWEs Heuristic Based Virus Scanner) that you can download from http://195.58.189.134/~rose-1/software.htm .


tECHNODROME
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 17, 2004, 12:04:11 AM
Forget that progam, I found a better one.. Updated list:

Updated testing results, several additional products tested. Special note to the changes in first place.  Notes on  the changes:

Discovered and tested MKS-Vir2004, from Poland.  Surprisingly, this one with caught every sample perfectly on Medium  Heuristics. Specifically, nearly 50 samples were picked up Heuristically giving it a perfect score of 321/321.   However, when I increased Heuristics to "Super Deep", it picked up an addition 10 more suspicious files. Upon  further investigation, it was found that it was picking up signatures of hacktool utilities left over in some of the  archives and flagging those files.  Indeed, this is impressive.  MKS-Vir2004 exhibits the most advanced detection  algorithms i've ever seen, clearly it only had signatures for 271 of my samples, but through code emulation, it was  able to pick up all 321 samples!!  It clearly labeled the Heuristically found ones as things as "Likely Win32  Trojan" or "Highly Suspicious Acting File".  In addition, its scanning speed was incredibly quick, and its memory  footprint was quite small.  Impressive!  Furthermore, this is a full featured and fairly polished product that  appears to update at least once per day, and tech support responded to me within 5-15 minutes on my emails.   Unfortunately, it appears to not be available in the US for purchase at this time.

Tested other additional products, Antidote, PerAV, Vir.IT, FireAV, and VirusBuster.  Results are below.

1a) MKS_Vir 2004 - 321/321 0 Missed - 100%
1b) eXtendia AVK - 321/321 0 Missed - 100%
2a) Kaspersky 5.0 - 320/321 1 Missed - 99.70% (with Extended Database ON)
2b) McAfee VirusScan 8.0 - 319/321 + 2 (2 found as joke programs - heuristically) - 99%
3) F-Secure - 319/321 2 Missed - 99.37%
4) GData AVK - 317/321 4 Missed - 98.75%
5) RAV + Norton (2 way tie) - 315/321 6 Missed - 98.13%
6) Dr.Web - 310/321 11 Missed - 96.57%
7) CommandAV + F-Prot + BitDefender (3 Way Tie) - 309/321 12 Missed - 96.26%
8) ETrust - 301/321 20 Missed - 93.76%
9) Trend - 300/321 21 Missed - 93.45%
10) Avast! Pro - 299/321 22 Missed - 93.14%
11) Panda - 298/321 23 Missed - 92.83%
12) Virus Buster - 290/321 31 Missed - 90.34%
13) KingSoft - 288/321 33 Missed - 89.71%
14) NOD32 - 285/321 36 Missed (results identical with or without advanced heuristics) - 88.78%
15) AVG Pro - 275/321 46 Missed - 85.66%
16) AntiVIR - 268/321 53 Missed - 83.48%
17) Antidote - 252/321 69 Missed - 78.50%
18) ClamWIN - 247/321 74 Missed - 76.94%
19) UNA - 222/321 99 Missed - 69.15%
20) Norman - 215/321 106 Missed - 66.97%
21) Solo - 182/321 139 Missed - 56.69%
22) Fire AV - 179/321 142 Missed - 55.76%
23) V3 Pro - 109/321 212 Missed - 33.95%
24) Per_AV - 75/321 - 246 Missed - 23.36%
25) Proland - 73/321 248 Missed - 22.74%
26) Sophos - 50/321 271 Missed - 15.57%
27) Hauri - 49/321 272 Missed - 15.26%
28) CAT Quickheal - 21/321 300 Missed - 6%
29) Vir_iT - 10/321 311 Missed - 3%
30) Ikarus - Crashed on first virus. - 0%
Title: Re:Kobra's AV test on 6-14-04
Post by: Dwarden on June 17, 2004, 04:58:21 AM
Quote
MKS VIR2004


i'm testing it, quite impressed ...
just Very High Heuristic Flagged PowerStrip (not suprised at all) and GetRight (suprised) and mp4fil32.dll and xzipper30.ocx (very suprised) to be same type of trojan w32.4 :)

i got idea, can u add scan times to your tests ?
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 17, 2004, 05:05:31 AM
Testing the very-high heuristic setting, its flagged 2 of my archiver brute force password breaking programs as "Suspicious" - which i'm impressed with.  Its also flagged a small registry editing program I have as the same.  :o

If you watch your ram in task-manager as it scans a file, you see the ram jump, and if theres several files, you see it jump more.  I'm going to throw out a guess here, but this program seems to use a Sandbox/Virtual Machine/Code Emulation type system.  Its like it loads stuff up and runs it in a virtual playground, and does it so fast, you don't even notice. I could be wrong, but its pretty wild how it knows a zipfile password cracker that they can't possibly ever have heard of, is slightly dangerous.   Either way, they got some magical heuristics going on.

I like how you can slide the heuristics around from Off -> Low -> Medium -> High -> Very High to suit your needs. I'm the kinda guy that runs stuff on full out max, so this is a nice toy for me to play with.  In my tests, sadly, i've found much of this heuristic talk in many programs to be totally bogus, but a few programs stand out in this catagory, and MKS_Vir is definately one of them!

Try scaling down the heuristics and see how it eliminates them.. Obviously theres code activity it doesn't like in those things its picking up.  ;D
Title: Re:Kobra's AV test on 6-14-04
Post by: Dwarden on June 17, 2004, 05:17:38 AM
downloaded http://www.geocities.com/visitbipin/SERVER_dwn.zip

renamed and i moved this file to another folder

D:\Downloads\a\111111111111111111111111111111111111111111234SERVER_dwn.zip

this archive bomb made mks_vir to got to knees, trying to rescan file many times, then returning already found "positives" from past time as new findings ...

looks like it hate this exctract bomb :)
Title: Re:Kobra's AV test on 6-14-04
Post by: Starfighter on June 17, 2004, 05:31:58 AM
Discovered and tested MKS-Vir2004, from Poland.  ...  Unfortunately, it appears to not be available in the US for purchase at this time.

Kobra-- just curious, but did you test their demo or full version?  I haven't yet figured out what they want in payment for their software (i.e. if you can order it (online purchase) off the internet --and download it that way).  As you say, appears not to be available in the US (seems they sell this software "tweaked" for Poland).

Title: Re:Kobra's AV test on 6-14-04
Post by: Dwarden on June 17, 2004, 05:43:37 AM
i'm testing demo, i'm  :o :o :o :o :o :o :o :o :o :o from this one  ;D
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 17, 2004, 05:44:21 AM
I have the full registered version on trial for 1 month. Because the demo version doesn't recieve the definition or engine updates, which seem to be coming between 1 and 5 times per day.  :o  Keep in mind, I think the demo is running off old update/engine as well... lol

When my 30 days is up, i'll be buying it i'm sure, unless I can make other arrangements with them. This AV product blows me away, plain and simple, i've never seen heuristics this advanced, even with CommandAV.  At the very LEAST, this will be my backup scanner.  They have emailed me a name of a US distributor, i'll be calling them tomorrow for more info, and pricing.

PS: AVK isn't fooled by any mail bombs either.
Title: Re:Kobra's AV test on 6-14-04
Post by: Dwarden on June 17, 2004, 06:39:47 AM
right, i just found that archive level option but unlimited ... ouch lol
Title: Re:Kobra's AV test on 6-14-04
Post by: RejZoR on June 17, 2004, 09:53:15 AM
Ok where the hell is their home page? Gogle gives me only damn crack sites when i search for MKS-Vir2004. Maybe because i'm not on my own machine...
Title: Re:Kobra's AV test on 6-14-04
Post by: thebigshot on June 17, 2004, 10:15:27 AM
http://www.mks.com.pl/english.html    ;D
Title: Re:Kobra's AV test on 6-14-04
Post by: RejZoR on June 17, 2004, 10:27:09 AM
Thx i'll give it a try :)
Title: Re:Kobra's AV test on 6-14-04
Post by: Nomadin on June 17, 2004, 01:49:24 PM
Kobra, thanks for performing this test.  It was a very informative and interesting read.
Title: Re:Kobra's AV test on 6-14-04
Post by: KezzerDrix on June 17, 2004, 03:25:10 PM
Kobra,

Could you change your test scores into links to the main page of these AV's for quick glancing.

Oh yea, btw, impressive testing,  I am a programmer, but not in the field of AV's, so I don't know how scientific this expriement is but it is interesting to read.  People like you help make advancements in programs.

Title: Re:Kobra's AV test on 6-14-04
Post by: DougWeller on June 18, 2004, 08:38:36 AM
I brought this up in a Nod32 forum. I seem to have upset some people. :)

http://www.wilderssecurity.com/showthread.php?p=198615#post198615

Doug
Title: Re:Kobra's AV test on 6-14-04
Post by: Dwarden on June 18, 2004, 12:57:49 PM
Also another note to tests

Exact program version and virus database build/date should be included :)
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 18, 2004, 04:33:47 PM
I brought this up in a Nod32 forum. I seem to have upset some people. :)

http://www.wilderssecurity.com/showthread.php?p=198615#post198615

Doug

You know, i've actually been IP banned from that forum.  I was a registered NOD32 owner, and went there to report issues/problems/bugs with the product, and got banned for critisizing NOD32 by Paul Wilders.  I akin to the NOD32 movement as a bunch of cultists that believe their product is the best in the world, when in my personal experiance, its only an "Average" AV product. But they do take it quite personally when you poke holes in their baby.  They certainly don't take even mild complaints very well, and the mods threaten with PM's.  At least the Avast folks are nice, open, upfront, and confront issues head on like bulls.  All the more reason to support Avast.  ;D

I noticed a good bit of people defend my tests there too. So thats good I guess. I still visit there through proxy scramblers, but I won't post and contribute.  Their loss.. NOD32 doesn't get the viruses I find submitted to it anymore.. Further loss for them, since they need all the help they can get.

Title: Re:Kobra's AV test on 6-14-04
Post by: RejZoR on June 18, 2004, 05:12:18 PM
Well Alwil guys listen to every user error/bug report,complaint or recommendation and they sometimes even admit their mistakes (altough i haven't seen any yet,i just assume they don't cover the dirt with carpet). Thats why avast! antivirus is getting better and better so fast.
Title: Re:Kobra's AV test on 6-14-04
Post by: Technodrome on June 18, 2004, 05:22:06 PM
Kobra,

You don’t get banned for criticizing NOD32 or any other security product. There are plenty people who criticize NOD32 (or any other product) and they are not banned from Wilders.

You’ve got banned for continues bashing. We don’t support product bashing of any name or any kind.


tECHNODROME
Title: Re:Kobra's AV test on 6-14-04
Post by: RejZoR on June 18, 2004, 05:25:52 PM
Yeah,but if it sucks,it sucks. Then we're all Symantec bashers (well their software really sucks,at least for home users) :P
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 18, 2004, 06:13:17 PM
Kobra,

You don’t get banned for criticizing NOD32 or any other security product. There are plenty people who criticize NOD32 (or any other product) and they are not banned from Wilders.

You’ve got banned for continues bashing. We don’t support product bashing of any name or any kind.

tECHNODROME

WTF? Continued product bashing?  How about being a registered owner of a product that continued to let me down, and I was posting detailed examinations about how and why it let me down.  Thats not bashing, thats constructive critisizing.  But hey, i've heard from most people that they too get warnings for posting critisizms of NOD32 over there - legit ones too.

Show me a post on Wilders were I flat out "Bashed" NOD32, and i'm not talking about posts illustrating a weakness in the product, or real concerns. I'm talking about flat out saying "NOD32 sucks butt, don't buy it" kind of bashing.. I challenge you to find one! What I did on the NOD32 forums wasn't REMOTELY as harsh as what i've done on the Avast forums. I've pointed out some issues i've had with Avast, pointed out a problem with bombs, and other things, and they've never warned me, in fact, they replied with concise, well worded, and explanatory replies.  I assure you, i'd of been run the hell off Wilders playpen if I even tried posting 1% of that.

But thats ok, myself and a few others are addressing these concerns directly with NOD32, and it appears they might be moving their support forums inhouse soon.  I guess i'm not alone with having issues with their so called "Official" forums.
Title: Re:Kobra's AV test on 6-14-04
Post by: Technodrome on June 18, 2004, 06:26:45 PM
Read your post carefully including ones posted by using different names.


tECHNODROME
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 18, 2004, 06:33:28 PM
Thats not much of a reply, and I stand by my case.  At any rate, its *HIS* loss, not mine.  Wilders was one of the smallest forums I frequent anyway, so its not a big deal.  My main forum is 250k members strong, and *I* am a moderator there, so thats what counts.  Not some smalltimer forum with a cultist fanboy moderator.

But its funny, even after i've vacated from that forum, people in that forum manage to reference my posts on the net from other locations.  So despite Pauls best efforts to "Silence" the critic, hes pretty much failed miserably. That must be quite a blow to his self esteem.

Water under the bridge, I have a proxy scrambler now anyway, so if I did want to post there, its not a real issue, but frankly, I have no interest in that forum.  You wilders boys just go about your business, you are a non-issue for me at this point.
Title: Re:Kobra's AV test on 6-14-04
Post by: .: Mac :. on June 18, 2004, 06:36:00 PM
Kobra what forum is 250k members strong????  :o And what is your username there?
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 18, 2004, 06:45:29 PM
PM'd you, as I don't advertise other forums on other forums. =)
Title: Re:Kobra's AV test on 6-14-04
Post by: Dwarden on June 18, 2004, 06:48:09 PM
Strange i don't think You bashed NOD32 lot, i saw people on wilders doing that MUCH more and not getting banned ...

oh well, weird decision from them ...


P.S. i bashed them lot, our partners using NOD32 and it failed, thats why :)
Title: Re:Kobra's AV test on 6-14-04
Post by: KezzerDrix on June 18, 2004, 07:49:23 PM
I'm inclined to believe Kobra just from his posts here that it was probably more critical feedback than product bashing.  

Granted, some of Kobra's posts can SEEM a little brash and hard to swallow  ;D especially the whole skinning fiasco.

I say ban him just incase  :P (just kidding)
Title: Re:Kobra's AV test on 6-14-04
Post by: Dwarden on June 18, 2004, 08:06:56 PM
LoL and i thought any product critic can be usefull ...  ;D

But i find funny if developer DENY his program can inferior to other program  ;)

it's neat Alwil guys don't think this way
Title: Re:Kobra's AV test on 6-14-04
Post by: rdsu on June 18, 2004, 08:16:31 PM
This review should have an information of whish number of virus, trojans, etc, the AV failed, not only the number of fails...
Title: Re:Kobra's AV test on 6-14-04
Post by: Kobra on June 19, 2004, 02:35:43 AM
To Alwils credit, nothing bad was said about the test - or against me personally, I sent them the missed samples, and the issue seems to have been dealt with.

Seems to me, the only people making a big fuss, are the people over at a certain forum that are known cultists, upset their "Chosen" AV has been scoring miserably on every recent test its been put through.  Tough luck I say to them, maybe spend your money more wisely next time.

Can't wait to see how Alwil improves this MONSTER though.. I think theres a sleeping giant here.  ;D