Author Topic: using heuristics  (Read 7461 times)

0 Members and 1 Guest are viewing this topic.

Offline MWassef

  • Avast Evangelist
  • Super Poster
  • ***
  • Posts: 1315
using heuristics
« on: October 09, 2003, 08:56:59 AM »
I know that Avast4 uses heusristics in its mail provider..
but, is there any plans to add heuristics to both on-access and on-demand scanners of Avast4 as in the mail one?  ::) ::)
if not, why?   ??? ???
MW

Offline MWassef

  • Avast Evangelist
  • Super Poster
  • ***
  • Posts: 1315
Re:using heuristics
« Reply #1 on: October 09, 2003, 06:02:12 PM »
any comment from Alwil team ?  ??? :'(
MW

Offline Lisandro

  • Avast team
  • Certainly Bot
  • *
  • Posts: 67195
Re:using heuristics
« Reply #2 on: October 09, 2003, 11:01:24 PM »
I know that Avast4 uses heusristics in its mail provider..
but, is there any plans to add heuristics to both on-access and on-demand scanners of Avast4 as in the mail one?  ::) ::)
if not, why?   ??? ???

If you are talking about OLE Documents (Heuristics for macro in Word and Excell), there is an especific option for this in the configurations...
If not, what is Heuristic analysis in on-access and on-demand? I´m a little bit stupid on Heuristics but I think no make sense test them for all kind of files...
Maybe avast! team could say something and we will learn about it  ;D
The best things in life are free.

Hornus Continuum

  • Guest
Re:using heuristics
« Reply #3 on: October 14, 2003, 04:32:52 PM »
Technical,

Normally, when a scanner searches a file for the presence of a virus, it looks for one or more signatures, unique byte sequences that are always present in that virus.  This may be as simple as a text string or as complex as a code sequence.  However, when a scanner uses heuristics, it also checks whether the program attempts to perform a potentially malicious activity, for example low-level disk writes, perhaps to overwrite the Master Book Record and destroy the computer's ability to access the hard drive.

Heuristics allows a scanner to detect new viruses, and also some moldy oldies, whose signatures are not in its database, although it can't identify them.  There's a downside though, an increase in false positive detections; a disk editor, to continue the example, might be reported as containing a virus even though it does not.

Regards,
Hornus

techie101

  • Guest
Re:using heuristics
« Reply #4 on: October 14, 2003, 04:48:09 PM »
hornus,

Thank you for the excellent explanation.
I like to explain heuristics this way:

If it walks like a duck, looks like a duck and quakes like a duck then it is a duck!

Heuristics takes this approach when examining files.

Heuristics tend to be effective with polymorphic viruses which tend to "change" their appearance to anti-virus software to avoid detection.

Using heuristics does require a bit more discipline before deleting files!  False positives are an unfortunate side effect.

Hornus.....thanks again.

techie

Offline Lisandro

  • Avast team
  • Certainly Bot
  • *
  • Posts: 67195
Re:using heuristics
« Reply #5 on: October 15, 2003, 12:30:03 AM »
Technical,

Normally, when a scanner searches a file for the presence of a virus, it looks for one or more signatures, unique byte sequences that are always present in that virus.  This may be as simple as a text string or as complex as a code sequence.  However, when a scanner uses heuristics, it also checks whether the program attempts to perform a potentially malicious activity, for example low-level disk writes, perhaps to overwrite the Master Book Record and destroy the computer's ability to access the hard drive.

Heuristics allows a scanner to detect new viruses, and also some moldy oldies, whose signatures are not in its database, although it can't identify them.  There's a downside though, an increase in false positive detections; a disk editor, to continue the example, might be reported as containing a virus even though it does not.

Regards,
Hornus

Thanks Hornus. You give us a lesson of Heuristics. We give you a karma!  ;)
So, maybe in the future, avast! will have both on-access and on-demand Heuristic scanners...  8)
The best things in life are free.

techie101

  • Guest
Re:using heuristics
« Reply #6 on: October 15, 2003, 05:10:15 AM »
So, maybe in the future, avast! will have both on-access and on-demand Heuristic scanners...  8)

Not a bad suggestion!  I would like to see heuristics as part of the scanning engine, not just for some of the On Access modules.

AVAST TEAM ARE YOU LISTENING???

Ok, Technical...that one earns you a "K"
 :D
« Last Edit: October 15, 2003, 05:11:59 AM by techie101 »

Waldo

  • Guest
Re:using heuristics
« Reply #7 on: October 15, 2003, 08:24:29 AM »
Hi,

Heuristics is all about detecting virus-like behaviour, rather than looking for specific signatures. Eg, if a program tries to write to the boot sector of a drive, or amend an executable file, or send hundreds of identical emails, it's probably a virus. Though of course it might not be, and that's the big problem. Real viruses might get ignored, because the user assumes that it's yet another false alarm.

In the early days of the technology, heuristic scanners failed to detect many viruses and also generated too many false alarms. But the technology is improving rapidly, and heuristic scanners are now very good. Many leading antivirus packages offer signature-based and heuristic scanning in combination, and in my opinion that's the best way to go. With heuristics and signatures, you get the best of both worlds. Although having some common sense about not opening unexpected attachments offers even better protection than just about any antivirus program!

As for why the major AV companies aren't moving to a totally heuristics-based model, maybe it has something to do with how much annual subscription revenue they'd lose if no one needed to update their AV software ever again.
 
Despite the efforts of the marketing folks at AV companies, the reality is that heuristics will only find some of the newer viruses and other malware that appear. Heuristic analysis is based on the approach of looking for bad guys who look a lot like other previously known bad guys. So, if the code in a program contains actions which are identical or sufficiently similar to that of some previously seen malware, then it will be flagged as suspicious.

The best heuristics in AV scanners will catch about 90% of the new malware - but that still leaves us with the other 10% to contend with. As well, the quality of heuristics varies depending on the AV scanner, so it is not a safe bet to assume that any heuristics will provide good protection.
 
Actually, heuristic analysis is far from being a solution. Antiviral technology is based on pattern-matching algorithms that search for known strings (segments of code) of known viruses in your storage devices (HDD, diskettes, etc.) or in RAM. That is why if your antivirus software is outdated, it won't recognize new viruses.

Heuristic analysis is an attempt to solve this pattern dependency. It is a technology remotely resembling artificial intelligence (actually, it uses some techniques from AI), to analyse code, and interpret it, and make an assumption about it (whether it does harm or not).

Try to imagine it as a program that tries to resemble a human programmer, that analyses the source code of the program you want to check out, and tells you if he thinks it's regular code, or if he thinks there's something malicious about it.... the problem is, heuristic analysis is nowhere near the analysis capability of a human being.

Heuristic analysis is supposed to detect ANY abnormal or harmful code, even if it doesn't resemble any known viral patterns (so it should detect new viruses).

The problem is, heuristic analysis makes assumptions, and making assumptions about the malicious purpose of code is hardly accurate, so what you usually see from heuristic analysis is lots of false positives, and you can certainly assume there's lots of false negatives too... its results are totally inconclusive and extremely unreliable, therefore, not usable in the real world.

Probably, in the future, as AI technology advances, heuristic analysis will be a useful tool, not only in antivirus technology, but in IDS (intrusion Detection Systems) and several other security related applications as well.
 
In many ways it probably should be the answer but circumstances dictate against this. If done well Heuristic software could catch a lot, if not most, viruses. But there are several obstacles.

It is more computationally intense so more powerful engines are needed to run it, hence greater expense. It is far harder for the AV vendors to produce and maintain good Heuristic software than it is to simply identify patterns and put them in a pattern file. They get zillions of new viruses to publish patterns for so they are busy as it is. The general public don't really want good software, they want cheap software. AV is normally a commodity item for many companies and the most talked about item when negotiating is the cost per seat. Also unlike other software, pattern matching AV is only as good as its recent pattern files, so many customers renegotiate and possibly change vendors every year, this is why the vendors will go far lower on price to get a three year than a one year licence deal. They know that this does away with two negotiations which they are more likely to lose than win.

Short version... :)

Heuristic Analysis:
 
The ability of a virus scanner to identify a potential virus by analysing the behavior of the program, rather than looking for a known virus signature.
In general, heuristic analysis is not as reliable as signature-based virus scanning as it is not possible to predict precisely what a program will do when executed. However, heuristic scanning is a useful addition to any anti-virus policy.

The main disadvantage of heuristic scanning is that the product often produces false alarms when perfectly innocent code is suspected of behaving as a virus might. The main danger with anti-virus software that produces multiple false alarms is that users will eventually start to take no notice of the false alarms, providing the possibility that a genuine virus outbreak will be missed.

Kind regards,

Waldo

Offline Lisandro

  • Avast team
  • Certainly Bot
  • *
  • Posts: 67195
Re:using heuristics
« Reply #8 on: October 15, 2003, 01:45:53 PM »
Hi,
Actually, heuristic analysis is far from being a solution. Antiviral technology is based on pattern-matching algorithms that search for known strings (segments of code) of known viruses in your storage devices (HDD, diskettes, etc.) or in RAM. That is why if your antivirus software is outdated, it won't recognize new viruses.

Heuristic analysis is an attempt to solve this pattern dependency. It is a technology remotely resembling artificial intelligence (actually, it uses some techniques from AI), to analyse code, and interpret it, and make an assumption about it (whether it does harm or not).


The ability of a virus scanner to identify a potential virus by analysing the behavior of the program, rather than looking for a known virus signature.
In general, heuristic analysis is not as reliable as signature-based virus scanning as it is not possible to predict precisely what a program will do when executed. However, heuristic scanning is a useful addition to any anti-virus policy.

The main disadvantage of heuristic scanning is that the product often produces false alarms when perfectly innocent code is suspected of behaving as a virus might. The main danger with anti-virus software that produces multiple false alarms is that users will eventually start to take no notice of the false alarms, providing the possibility that a genuine virus outbreak will be missed.

Kind regards,
Waldo

Thanks Waldo (you rule!). The false positives and false negatives will always be present at Heuristic analysis... Maybe I have to change my mind a little... Maybe minacross will think a little bit different... Thanks, for your review. You earn a karma.  ;)
The best things in life are free.

Offline MWassef

  • Avast Evangelist
  • Super Poster
  • ***
  • Posts: 1315
Re:using heuristics
« Reply #9 on: October 15, 2003, 02:09:55 PM »
may be  ::)
MW