Protecting over 200 million PCs, Macs, & Mobiles – more than any other antivirus

September 6th, 2010

Spring cleaning in our virus database

We would reach 3 millions of detections in our virus database (VPS)  this week, but … this huge number means that when you put all the detections together, there is no difference between sophistical algorithmic detection and “temporary” machine generated detection.

When a new undetected malware sample comes to our viruslab, we have a certain handling procedure. There is some preliminary automatic  analysis and grouping with other malware samples we already have. If the group contains few samples, no human analysis is needed. The detection is machine generated and released in the next virus database update. Grouping malware together works in a relatively small time window. Bigger groups are handled by human analyst.

Initially, each group began as small, as just one sample. As this grows, we deploy polymorphic detections which cover hundreds to thousands of previously machine generated detections.

Now it is time to clean up and reduce the number of unnecessary detections. The main reason for this is to decrease the size of our virus database updates sent to users around the globe. This will reduce the amount of transferred data, subsequently reducing the amount of needed energy and helping, of course, our forests. :-)

Categories: lab, Virus Lab Tags: , ,
  1. September 6th, 2010 at 19:03 | #1

    like this and keep up our security… I’ve been use avast since 2005

  2. Llanziel
    September 6th, 2010 at 19:06 | #2

    Thanks for this blog. Now I understand the time required for new samples. Because I submitted 20+ samples since last week. Some of these are now detected by avast! and others still without results. But I wait for this, no rush.

  3. Ardiva
    September 6th, 2010 at 19:15 | #3
  4. September 6th, 2010 at 19:24 | #4

    Não seria perigosa essa necessidade de limpar e reduzir o número de detecções desnecessárias? poderíamos ser atacados por vírus obsoletos, falo como mero usuário do Avast! Não tenho nenhum conhecimento aprofundado, mas fiquei um pouco receoso.

    It would be dangerous to the need to clean and reduce the number of detections that are unnecessary? could be attacked by viruses obsolete, I speak as a mere user of Avast! I have no detailed knowledge, but I was a little afraid.

  5. September 6th, 2010 at 19:50 | #5

    @Giu Santana
    Please don’t worry. Unnecessary detection means 100% covered by other more complex detection.

  6. September 6th, 2010 at 19:57 | #6

    Don’t worry, this just means they will convert (not remove) those 1000 detections into 1 detection that can still cover all those 1000 while be much more friendly to your PC (in terms of updating, resource usage etc).

    @Michal, when will this cleaning and consolidation happen?

  7. September 6th, 2010 at 20:00 | #7

    @RejZoR
    We started on Sunday and will continue till next week

  8. September 6th, 2010 at 21:25 | #8

    @Michal Trs
    Thank you. Congratulations on the blog.

  9. September 6th, 2010 at 21:40 | #9

    With VPS 100906-0, avast! had about 2.970.000 definitions.
    Few minutes ago, it updated itself to 100906-1, and now it has only 2.677.191.
    Is it sure, that avast! will have as good detection as before the “spring cleaning”?

    • igor
      September 7th, 2010 at 00:30 | #10

      Yes, the detection capabilities remain the same; some malware may now be detected under a different name, but detected nevertheless.

  10. moogie
    September 6th, 2010 at 22:12 | #11

    Michal and the rest of the Avast team, thanks and keep up the great work!
    Maybe this will also lead to, as a consequence, even faster scanning times?

    • igor
      September 7th, 2010 at 00:33 | #12

      It’s unlikely to affect the scan speed in any measurable way, I’m afraid.
      The memory usage might get slightly lower though.

  11. September 7th, 2010 at 01:05 | #13

    Thanks for constantly improving what’s already the best antivirus program.
    avast!5 rocks.

  12. Marc
    September 7th, 2010 at 11:58 | #14

    Koool, more business for resellers that will propose more and more AVAST to make business to propose a reformating hard drive when they will be infected by old viruses

    Thank you to keep our business grows up ;-)

    Continue in this way!

  13. Laperuz
    September 7th, 2010 at 12:16 | #15

    I have a sample which is detected by the real-time shield. Few days ago i’ve scanned the file in the virus chest and got VBS:Malware-gen detection. But now the virus chest scan doesn’t find anything. Is this related with the cleaning of your database?

    Keep up your excellent work :)

  14. September 7th, 2010 at 12:52 | #16

    @Marc
    All viruses are still detected! Short example what the “spring cleaning” means: There was new undetected sample -> become detected as Win32:Trojan-gen, few days later was deployed Win32:Fraudo [Trj] detection, which covered this and many others samples. File is detected by both detection, but in Avast scan report is displayed the generic one – Win32:Fraudo [Trj]. And now we are removing unnecessary Win32:Trojan-gen… this is just example, I had used random detection names from my mind :-)

  15. September 7th, 2010 at 12:55 | #17

    @Laperuz
    Hi, this cleaning affected only MZ-PE files (binary execution files for MS Windows). It could be a false positive alert…

  16. Laperuz
    September 7th, 2010 at 13:08 | #18

    @Michal Trs
    The file got 27 of 43 on virustotal:
    http://www.virustotal.com/file-scan/report.html?id=eccf71163145d2df15b252c6f7579522f7a4ca4d2327a809010f48238c41f88d-1283857334

    And detection is VBS:AutoRun-gen, not VBS:Malware-gen,sorry. But still there isn’t detection in the virus chest scan.

    P.S I use Avast Free 5.0.673

  17. September 7th, 2010 at 16:16 | #19

    when will be a cloud version ready – no signature updates at all necessary – for both avast and users would be the best / most efficient solution

  18. Lyn
    September 7th, 2010 at 23:49 | #20

    I just started using Avast 3 days ago. Today I used it to do a scan, and it found the JS:Pdfka-AMM trojan in one of my files which it quarantined. What I’d like to know is, since that is a known trojan and an old one, how did it get in? Why didn’t Avast protect my system? I have since purchased and registered Avast today, and am using the updated product. But still, this should not have happened. Right?

  19. September 8th, 2010 at 11:22 | #21

    @Lyn
    Actually, it is an expected behavior. JS:Pdfka-AMM detection was released on September 1. (so that’s a week ago) and it’s a pretty recent threat. How did you conclude that it is an “old one”? If the mentioned scan was the very first since installing Avast, there’s no surprise that Avast did not detect it earlier. It would detect it if you attempted to open the infected pdf. Simply – no scan = no detections. :-)

  20. smit_7648
    September 8th, 2010 at 11:24 | #22

    i came to know what is faisal.exe ?
    is it virus ?
    if yes, how it’s work ?

  21. September 8th, 2010 at 11:27 | #23

    @DrHartmutFeucht
    Not really. Cloud only changes the way of distributing most recent detections, but a good base of local detections will be always necessary.

  22. Milos
    September 8th, 2010 at 11:31 | #24

    @smit_7648
    Hello,
    post this issues in our forum.avast.com, please.

  23. Niko70
    September 8th, 2010 at 11:51 | #25

    Thanks for giving us such details which I find very interesting.
    Thank you on behalf of forests!
    Have a nice day!
    N.

  24. September 10th, 2010 at 21:11 | #26

    Since the release of 5.0.677, I have to thank the Avast crew for a good update and I love the new 2 click install :)

    I do on average about 50-60 avast free installs a day(I work for a small IT business in Oklahoma) and the streamlined installs make my job more efficient and our customers are very pleased with the behavior of Avast. Thanks again for another great release and keep up the good work!!

  25. September 11th, 2010 at 08:57 | #27

    like this and keep up our security…

  26. Aethec
    September 13th, 2010 at 16:49 | #28

    From nearly 3 million to less than 2 million ? Impressive. Doesn’t this also mean that you should do something to prevent duplicate entries from happening in the first place ?

  27. September 13th, 2010 at 17:37 | #29

    Aethec :
    From nearly 3 million to less than 2 million ? Impressive. Doesn’t this also mean that you should do something to prevent duplicate entries from happening in the first place ?

    I don’t think so. It is not regular duplicates. It is generic detections being added after the less generic detections.

    Say detection A detects 3 samples. Subsequently (after a few days), a new, ingenious detection is released that detets 17,259 samples we have seen plus an unspecified number of samples we have not seen.

    This detection also covers the 3 samples for which we have initially created detection A. So A is now redundant and can be removed.

  28. michaltrs ignored laperuz
    September 14th, 2010 at 05:23 | #30

    @Michal Trs

    And detection is VBS:AutoRun-gen, not VBS:Malware-gen,sorry. But still there isn’t detection in the virus chest scan.

    P.S I use Avast Free 5.0.673

    Laperuz said that before the spring cleaning, Avast had quarantined VBS: AutoRun-gen, but after spring cleaning, there was no detection for it.

    http://www.virustotal.com/file-scan/report.html?id=eccf71163145d2df15b252c6f7579522f7a4ca4d2327a809010f48238c41f88d-1283857334

    27/43 vendors detect it, Avast is one of the 16 that doesn’t. Please explain.

  29. Milos
    September 14th, 2010 at 09:21 | #31

    @michaltrs ignored laperuz
    Hello,
    as I see from virustotal report “Avast5″ detects it, so what’s the problem?

  30. Laperuz
    September 14th, 2010 at 16:01 | #32

    @Milos
    Do you use different bases for the virus chest scan and right-click scan? I want to understand why the virus chest scan doesn’t detect the file, which is detected by the right-click scan.

  31. Milos
    September 14th, 2010 at 16:07 | #33

    @Laperuz
    Hello,
    no, virus bases are same, but maybe there are different settings for on-demand scan and resident shield.

  32. Aethec
    September 14th, 2010 at 17:32 | #34

    @vlk >> Understood. Thanks for the explanation.

  33. R
    September 21st, 2010 at 06:08 | #35

    Was the installer file size reduction due to this cleaning?
    setup_av_free.exe from Sept 07 52.5MB
    setup_av_free.exe from Sept 13 47.9MB

  34. September 21st, 2010 at 09:46 | #36

    @R
    Hi “R”, reducing size of our VPS (and setup file) was the main reason for the “Spring cleaning”.

  35. sheina
    October 2nd, 2010 at 06:10 | #37

    @Michal Krejdl
    It is not a recent threat it was discovered back in june of 2009. Avast is just late on adding it to their threats.

  36. October 4th, 2010 at 20:56 | #38

    @sheina
    Are you kidding? This particular exploit could only be discovered back in june 2009 if you had a time machine.

Comments are closed.