Protecting over 200 million PCs, Macs, & Mobiles – more than any other antivirus

September 6th, 2010

Spring cleaning in our virus database

We would reach 3 millions of detections in our virus database (VPS)  this week, but … this huge number means that when you put all the detections together, there is no difference between sophistical algorithmic detection and “temporary” machine generated detection.

When a new undetected malware sample comes to our viruslab, we have a certain handling procedure. There is some preliminary automatic  analysis and grouping with other malware samples we already have. If the group contains few samples, no human analysis is needed. The detection is machine generated and released in the next virus database update. Grouping malware together works in a relatively small time window. Bigger groups are handled by human analyst.

Initially, each group began as small, as just one sample. As this grows, we deploy polymorphic detections which cover hundreds to thousands of previously machine generated detections.

Now it is time to clean up and reduce the number of unnecessary detections. The main reason for this is to decrease the size of our virus database updates sent to users around the globe. This will reduce the amount of transferred data, subsequently reducing the amount of needed energy and helping, of course, our forests. :-)

Categories: lab, Virus Lab Tags: , ,
  • AriPerwiraCom

    like this and keep up our security… I’ve been use avast since 2005

  • Llanziel

    Thanks for this blog. Now I understand the time required for new samples. Because I submitted 20+ samples since last week. Some of these are now detected by avast! and others still without results. But I wait for this, no rush.

  • Ardiva
  • Giu Santana

    Não seria perigosa essa necessidade de limpar e reduzir o número de detecções desnecessárias? poderíamos ser atacados por vírus obsoletos, falo como mero usuário do Avast! Não tenho nenhum conhecimento aprofundado, mas fiquei um pouco receoso.

    It would be dangerous to the need to clean and reduce the number of detections that are unnecessary? could be attacked by viruses obsolete, I speak as a mere user of Avast! I have no detailed knowledge, but I was a little afraid.

  • Michal Trs

    @Giu Santana
    Please don’t worry. Unnecessary detection means 100% covered by other more complex detection.

  • RejZoR

    Don’t worry, this just means they will convert (not remove) those 1000 detections into 1 detection that can still cover all those 1000 while be much more friendly to your PC (in terms of updating, resource usage etc).

    @Michal, when will this cleaning and consolidation happen?

  • Michal Trs

    We started on Sunday and will continue till next week

  • Giu Santana

    @Michal Trs
    Thank you. Congratulations on the blog.

  • 12-es_csaj

    With VPS 100906-0, avast! had about 2.970.000 definitions.
    Few minutes ago, it updated itself to 100906-1, and now it has only 2.677.191.
    Is it sure, that avast! will have as good detection as before the “spring cleaning”?

    • igor

      Yes, the detection capabilities remain the same; some malware may now be detected under a different name, but detected nevertheless.

  • moogie

    Michal and the rest of the Avast team, thanks and keep up the great work!
    Maybe this will also lead to, as a consequence, even faster scanning times?

    • igor

      It’s unlikely to affect the scan speed in any measurable way, I’m afraid.
      The memory usage might get slightly lower though.

  • bob3160

    Thanks for constantly improving what’s already the best antivirus program.
    avast!5 rocks.

  • Pingback: Spring Photography – Nikon D90 Pictures - Nikon F70 Digital Camera - Nikon F70

  • Marc

    Koool, more business for resellers that will propose more and more AVAST to make business to propose a reformating hard drive when they will be infected by old viruses

    Thank you to keep our business grows up ;-)

    Continue in this way!

  • Laperuz

    I have a sample which is detected by the real-time shield. Few days ago i’ve scanned the file in the virus chest and got VBS:Malware-gen detection. But now the virus chest scan doesn’t find anything. Is this related with the cleaning of your database?

    Keep up your excellent work :)

  • Michal Trs

    All viruses are still detected! Short example what the “spring cleaning” means: There was new undetected sample -> become detected as Win32:Trojan-gen, few days later was deployed Win32:Fraudo [Trj] detection, which covered this and many others samples. File is detected by both detection, but in Avast scan report is displayed the generic one – Win32:Fraudo [Trj]. And now we are removing unnecessary Win32:Trojan-gen… this is just example, I had used random detection names from my mind :-)

  • Michal Trs

    Hi, this cleaning affected only MZ-PE files (binary execution files for MS Windows). It could be a false positive alert…

  • Laperuz

    @Michal Trs
    The file got 27 of 43 on virustotal:

    And detection is VBS:AutoRun-gen, not VBS:Malware-gen,sorry. But still there isn’t detection in the virus chest scan.

    P.S I use Avast Free 5.0.673

  • // DrHartmutFeucht

    when will be a cloud version ready – no signature updates at all necessary – for both avast and users would be the best / most efficient solution

  • Lyn

    I just started using Avast 3 days ago. Today I used it to do a scan, and it found the JS:Pdfka-AMM trojan in one of my files which it quarantined. What I’d like to know is, since that is a known trojan and an old one, how did it get in? Why didn’t Avast protect my system? I have since purchased and registered Avast today, and am using the updated product. But still, this should not have happened. Right?

  • Michal Krejdl

    Actually, it is an expected behavior. JS:Pdfka-AMM detection was released on September 1. (so that’s a week ago) and it’s a pretty recent threat. How did you conclude that it is an “old one”? If the mentioned scan was the very first since installing Avast, there’s no surprise that Avast did not detect it earlier. It would detect it if you attempted to open the infected pdf. Simply – no scan = no detections. :-)

  • smit_7648

    i came to know what is faisal.exe ?
    is it virus ?
    if yes, how it’s work ?

  • Michal Krejdl

    Not really. Cloud only changes the way of distributing most recent detections, but a good base of local detections will be always necessary.

  • Milos

    post this issues in our, please.

  • Niko70

    Thanks for giving us such details which I find very interesting.
    Thank you on behalf of forests!
    Have a nice day!

  • kasper5150

    Since the release of 5.0.677, I have to thank the Avast crew for a good update and I love the new 2 click install :)

    I do on average about 50-60 avast free installs a day(I work for a small IT business in Oklahoma) and the streamlined installs make my job more efficient and our customers are very pleased with the behavior of Avast. Thanks again for another great release and keep up the good work!!

  • takprosto

    like this and keep up our security…

  • Aethec

    From nearly 3 million to less than 2 million ? Impressive. Doesn’t this also mean that you should do something to prevent duplicate entries from happening in the first place ?

  • vlk

    Aethec :
    From nearly 3 million to less than 2 million ? Impressive. Doesn’t this also mean that you should do something to prevent duplicate entries from happening in the first place ?

    I don’t think so. It is not regular duplicates. It is generic detections being added after the less generic detections.

    Say detection A detects 3 samples. Subsequently (after a few days), a new, ingenious detection is released that detets 17,259 samples we have seen plus an unspecified number of samples we have not seen.

    This detection also covers the 3 samples for which we have initially created detection A. So A is now redundant and can be removed.

  • michaltrs ignored laperuz

    @Michal Trs

    And detection is VBS:AutoRun-gen, not VBS:Malware-gen,sorry. But still there isn’t detection in the virus chest scan.

    P.S I use Avast Free 5.0.673

    Laperuz said that before the spring cleaning, Avast had quarantined VBS: AutoRun-gen, but after spring cleaning, there was no detection for it.

    27/43 vendors detect it, Avast is one of the 16 that doesn’t. Please explain.

  • Milos

    @michaltrs ignored laperuz
    as I see from virustotal report “Avast5″ detects it, so what’s the problem?

  • Laperuz

    Do you use different bases for the virus chest scan and right-click scan? I want to understand why the virus chest scan doesn’t detect the file, which is detected by the right-click scan.

  • Milos

    no, virus bases are same, but maybe there are different settings for on-demand scan and resident shield.

  • Aethec

    @vlk >> Understood. Thanks for the explanation.

  • R

    Was the installer file size reduction due to this cleaning?
    setup_av_free.exe from Sept 07 52.5MB
    setup_av_free.exe from Sept 13 47.9MB

  • Michal Trs

    Hi “R”, reducing size of our VPS (and setup file) was the main reason for the “Spring cleaning”.

  • sheina

    @Michal Krejdl
    It is not a recent threat it was discovered back in june of 2009. Avast is just late on adding it to their threats.

  • Michal Krejdl

    Are you kidding? This particular exploit could only be discovered back in june 2009 if you had a time machine.