Author Topic: MRG Flash Test Update  (Read 5054 times)

0 Members and 1 Guest are viewing this topic.

Tgell

  • Guest
MRG Flash Test Update
« on: October 19, 2011, 12:45:58 AM »
avast! hits 100% again. Congratulations avast!

http://malwareresearchgroup.com/2011/10/18/mrg-flash-test-18102011/

Offline Lisandro

  • Avast team
  • Certainly Bot
  • *
  • Posts: 67195
Re: MRG Flash Test Update
« Reply #1 on: October 19, 2011, 01:53:46 AM »
Impressive, specially if we think on the free ones...
All other failed, Microsoft, Avira and AVG.
The best things in life are free.

alpha1

  • Guest
Re: MRG Flash Test Update
« Reply #2 on: October 19, 2011, 02:54:13 AM »
this testing site doesnt seem too credible,nor is a few samples an indication of anything.

im not saying avast isnt good,but rigorous testing needs be done by competent authorities.

Offline Lisandro

  • Avast team
  • Certainly Bot
  • *
  • Posts: 67195
Re: MRG Flash Test Update
« Reply #3 on: October 19, 2011, 12:36:58 PM »
but rigorous testing needs be done by competent authorities.
Fully agree.
The best things in life are free.

spg SCOTT

  • Guest
Re: MRG Flash Test Update
« Reply #4 on: October 19, 2011, 04:43:30 PM »
Ignoring my personal views on tests of pretty much any form...

I honestly don't get these tests.

I get the idea of testing so called zero day malware, so that the paranoid people can feel good (or go find another AV... ::) )

But I don't get these ones. Are there only four samples?
Drop in the ocean anyone?

IMO these tests, would be more informative to me (if I hypothetically formed my decisions on such tests) if they showed the time until they were detected. So see which ones detect already, and those that don't keep testing until they do.

That would give a more appropriate representation of how quickly an AV picks up on new malware.




Tgell

  • Guest
Re: MRG Flash Test Update
« Reply #5 on: October 19, 2011, 09:59:34 PM »
This is what the Flash Tests are about according to MRG.

http://malwareresearchgroup.com/2011/08/17/mrg-effitas-flash-tests-%e2%80%93-update/

alpha1

  • Guest
Re: MRG Flash Test Update
« Reply #6 on: October 19, 2011, 10:15:48 PM »
Ignoring my personal views on tests of pretty much any form...

I honestly don't get these tests.

I get the idea of testing so called zero day malware, so that the paranoid people can feel good (or go find another AV... ::) )


i dont think it has anything to do with paranoia,how can the average user possibly know how competent an av is without some sort of test,review,comparison,etc. by professionals?

ady4um

  • Guest
Re: MRG Flash Test Update
« Reply #7 on: October 19, 2011, 10:54:20 PM »
how can the average user possibly know how competent an av is without some sort of test,review,comparison,etc. by professionals?

Well, if the user wants to make some kind of conclusion from tests, or if some AV test is going to help him/her decide which AV to use, then at least should be a "professional" test.

Even for the most simple statistical / probabilistic analysis you need a "serious" amount of data.

If I am going to go after some "random" (read as any "non-professional") test, then my own test should be good enough for me and the use I personally have of my system (since I have no malware and Avast keeps stopping the very infrequent I found).

In my view, the percentage of virus recognitions from 1 test means almost nothing. A specific AV could have 95% one day, and 85% the next. If the security tool can make it OK in the long run, *then* I can consider it worth trying it on my system.

Other considerations are also valid, such as needed resources, OS, (ease of) configuration, and even looks (and more).

DonZ63

  • Guest
Re: MRG Flash Test Update
« Reply #8 on: October 19, 2011, 10:56:27 PM »
I don't believe MRG is a certified test lab?

spg SCOTT

  • Guest
Re: MRG Flash Test Update
« Reply #9 on: October 20, 2011, 12:15:01 AM »
Tgell, thanks for the link, that clears up a question or two.

alpha1, ok. I may have been a little judgmental there...

Essentially what I mean is that basically people come to the forum etc, either happy or sad depending on the results and, following that, people will decide on their av based on one test. (like they do with those youtube videos)

These tests don't make sense to me, since by the time the results are posted, they are redundant...

And testing 4 samples?


Anyway...I am probably not the best person to ask since I don't particularly like any testing, since they very often don't depict the real world (IMHO)

alpha1

  • Guest
Re: MRG Flash Test Update
« Reply #10 on: October 20, 2011, 12:16:41 AM »
how can the average user possibly know how competent an av is without some sort of test,review,comparison,etc. by professionals?

Well, if the user wants to make some kind of conclusion from tests, or if some AV test is going to help him/her decide which AV to use, then at least should be a "professional" test.

Even for the most simple statistical / probabilistic analysis you need a "serious" amount of data.

If I am going to go after some "random" (read as any "non-professional") test, then my own test should be good enough for me and the use I personally have of my system (since I have no malware and Avast keeps stopping the very infrequent I found).

In my view, the percentage of virus recognitions from 1 test means almost nothing. A specific AV could have 95% one day, and 85% the next. If the security tool can make it OK in the long run, *then* I can consider it worth trying it on my system.

Other considerations are also valid, such as needed resources, OS, (ease of) configuration, and even looks (and more).

yes,consistency is important,however,an individual's personal experiences(satisfaction/dissatisfaction) in detection dont mean much for others.

« Last Edit: October 20, 2011, 01:49:57 AM by alpha1 »

alpha1

  • Guest
Re: MRG Flash Test Update
« Reply #11 on: October 20, 2011, 12:19:23 AM »
I don't believe MRG is a certified test lab?

no,i dont believe its a certified anything.

alpha1

  • Guest
Re: MRG Flash Test Update
« Reply #12 on: October 20, 2011, 12:34:15 AM »
Essentially what I mean is that basically people come to the forum etc, either happy or sad depending on the results and, following that, people will decide on their av based on one test. (like they do with those youtube videos)

These tests don't make sense to me, since by the time the results are posted, they are redundant...

And testing 4 samples?


Anyway...I am probably not the best person to ask since I don't particularly like any testing, since they very often don't depict the real world (IMHO)

no doubt i agree with you on the 4 samples issue,the funnier thing is,they were actually trying to be generous with 4 samples,since they used to use only 1 sample before.

but i disagree that results are redundant by the time they're released,especially if an av is consistent,multiple tests will give the consumer a good idea about a product.
even inconsistent results for a particular av will give a good idea of that av.

i also disagree with the "real world" statement,since the real world probably differs greatly from 1 individual to another; thats where large scale testing holds value.

btw scott,actually you (and others with knowledge and experience)are the right person to ask,because you actually have the ability to think independently,even though we may disagree on certain things.

spg SCOTT

  • Guest
Re: MRG Flash Test Update
« Reply #13 on: October 20, 2011, 01:04:15 AM »
Quote
no doubt i agree with you on the 4 samples issue,the funnier thing is,they were actually trying to be generous with 4 samples,since they used to use only 1 sample before.
Yeah...and we moan about youtube tests...usually they have about 15 ;D

Quote
but i disagree that results are redundant by the time they're released,especially if an av is consistent,multiple tests will give the consumer a good idea about a product.
even inconsistent results for a particular av will give a good idea of that av.
The problem for me is, that one detection is simply that, so it they are basically a snapshot of that specific moment in time. And in simply collecting all of your results, the conditions may have changed.
That is why I said initially that I thought they would be better if those that failed were re-tested until the detections are added. That would give a better idea of how the av tackles the newer malware. (i.e. How quick they pick it up, not do they detect it when we gat a sample in?)

Quote
i also disagree with the "real world" statement,since the real world probably differs greatly from 1 individual to another; thats where large scale testing holds value.
I would agree there that each user is different, but when you look at tests, all people care about is detection.
All other factors that would be thought about normally go out the window as far as I can tell. People seem to say "right well that one has the highest detection this week, I'll pick that one." (or they condemn their current av because it wasn't as quick to catch something as another one)
Without considering other things like whether there are multiple updates (which would theoretically help with the zero day stuff) or whether it is actually easy to use/set up.
If you can't actually use it, then how can you protect yourself from it.


Quote
btw scott,actually you (and others with knowledge and experience)are the right person to ask,because you actually have the ability to think independently,even though we may disagree on certain things.
Well, I think I may be a little biased against tests...I do try and look at both sides, and I can see the value that some people get from tests, but (repeating myself again :P) it detracts from the actual use of the av.




alpha1

  • Guest
Re: MRG Flash Test Update
« Reply #14 on: October 20, 2011, 02:03:55 AM »
scot,you might have a point there,about the retesting.