Avast WEBforum

Consumer Products => Avast Free Antivirus / Premium Security (legacy Pro Antivirus, Internet Security, Premier) => Topic started by: LorenzoC on May 07, 2008, 08:43:30 AM

Title: Huge temp files are created and not removed
Post by: LorenzoC on May 07, 2008, 08:43:30 AM
Upon scanning AVAST creates these files:
C:\Documents and Settings\Administrator\Impostazioni locali\Temp\_avast4_\unp35430682.tmp - 118.637 KB
C:\Documents and Settings\Administrator\Impostazioni locali\Temp\_avast4_\unp234548821.tmp - 118.638 KB

I guess they are made during the unpacking of the Thunderbird mailboxes.
Problem is they are not removed when the scanning is interrupted.
Title: Re: Huge temp files are created and not removed
Post by: rassel on May 07, 2008, 11:20:07 AM
Can you delete it? if no than try this steps

Thats a quite huge files on your temp :o. Can you try run any cleaner that you have?e.g http://www.ccleaner.com/ (http://www.ccleaner.com/) or you can run it by going(Im using window XP but im not sure if yours is same as me) START>All Programs>Accessories>System Tools>Disk Cleanup
if you still see the files than you download an unlocker or go here for more information http://ccollomb.free.fr/unlocker/#download

Im not sure if i did it the right way but hope there is someone who can help you the better on your problem than using my ways.

What im helping here is to help you to delete the file.
Title: Re: Huge temp files are created and not removed
Post by: LorenzoC on May 07, 2008, 11:35:21 AM
Thanks for your help but you haven't got the point. :)

Those files are created "dinamically" by AVAST when you are scanning mailboxes, which are seen as "archives" actually.
I guess AVAST creates those files in order to look inside the archive and they are huge because the mailbox itself is huge.

Note: AVAST should either skip the mailboxes or have an option for it because scanning those huge archives is demanding and useless once the incoming mail is already scanned by the TSR provider.

Then AVAST should remove the temporary files when the scanning is done, and so it does, UNLESS you stop the scanning before it comes to the end.
In that case the files are left in the "_avast_" folder and that is not good.

Yes, I can manually remove the files but it would be better if AVAST took care of it upon stopping the scanner.
I guess I am not the only one with those leftovers in the HD.




Title: Re: Huge temp files are created and not removed
Post by: igor on May 07, 2008, 11:50:20 AM
avast! normally removes the temporary files even when the scan is interrupted... must be some very time-consuming operation going on here.
What exactly are you scanning?
Title: Re: Huge temp files are created and not removed
Post by: rassel on May 07, 2008, 11:52:45 AM
Quote
you haven't got the point.
Sorry for not understanding your problem.

Quote
avast! normally removes the temporary files even when the scan is interrupted...

Yea my _avast4_ folder was empty even i did/interrupt a scan.
Title: Re: Huge temp files are created and not removed
Post by: LorenzoC on May 07, 2008, 12:13:07 PM
I don't do anything special, I either run the "standard scan" on C: or the "scan" via Explorer extension (don't know if they are exactly the same).
I haven't seen what happens with the AVAST screen server scanning enabled.
At some point of the scanning AVAST meets my Thunderbird mailboxes, that are inside a "mail" folder of about 170MB. Not that big considering mail archives can be easily some Giga large.
It happened to me to stop the scanner and then I got those files in the "_avast_" , that apparently aren't removed upon scanner stopping.

The first time I manually removed the files thinking of a glitch but I can reproduce the same behaviour any time so I thought to post it here.

I don't have any problem in manually removing the files. If I leave the files in place, AVAST creates other files the next scanning passage and removes them when it finishes, leaving the "leftovers" there.

Oh, of course I don't remember what is in the mailboxes, I don't know if it could be a particular attachment or something. I remove all the spam anyway and the mailboxes should be "clean" of viruses.

On a side note, sorry for repeating, the scanning of mailboxes should be only optional. I don't see any point in scanning them each time.
Title: Re: Huge temp files are created and not removed
Post by: LorenzoC on May 07, 2008, 05:52:17 PM
To work around the issue I've set in "exclusions" this path:
C:\Documents and Settings\Administrator\Dati applicazioni\Thunderbird\Profiles\mc135pum.default\Mail\*

It is something that should be addressed in a smarter way then, first because not all users are able to work around, then because, even if I know my PC is slow (PIII 500), it doens't sound that good AVAST put on its knees by a mailbox about 120MB large...

Title: Re: Huge temp files are created and not removed
Post by: igor on May 07, 2008, 07:55:35 PM
Try to enable the creation of the report file in avast! settings and let everything, even "OK files", be included there. Then, scan the mail folder again.
I'd like to know what's found there - because there wouldn't be any temporary files for ordinary scanning, it must be some "archive extraction". Maybe the MIME unpacker is seeing the mailbox as one message and a big attachment, don't know...
Title: Re: Huge temp files are created and not removed
Post by: LorenzoC on May 07, 2008, 08:50:08 PM
Ok, I've got the report file, what should I do with it?

BTW there's nothing in it, here is the block of one of the mailboxes, the largest, the others are identical:
(I've shortened the path to \mail)

\Mail\pop.gmail.com\Inbox\PartNo_0#599384397\PartNo_0#38486210\PartNo_0#3100651197 [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Inbox\PartNo_0#599384397\PartNo_0#38486210\PartNo_1#3325552817 [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Inbox\PartNo_0#599384397\PartNo_0#38486210 [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Inbox\PartNo_0#599384397 [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Inbox [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Templates
\Mail\pop.gmail.com\msgFilterRules.dat [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\popstate.dat [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Inbox.msf [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Trash\PartNo_0#3517757697 [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Trash\PartNo_1#2142653006 [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Trash [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Trash.msf [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Drafts
\Mail\pop.gmail.com\Drafts.msf [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Junk
\Mail\pop.gmail.com\Junk.msf [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Templates.msf [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Sent\PartNo_0#1731396481 [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Sent [E] File was skipped because of scanner settings. (42016)
\Mail\pop.gmail.com\Sent.msf [E] File was skipped because of scanner settings. (42016)

Now tell me AVAST creates the huge temporary files only to skip content :)
Title: Re: Huge temp files are created and not removed
Post by: alanrf on May 07, 2008, 11:59:44 PM
The problem here is, in part, Thunderbird itself. 

Thunderbird mail files have name likes "Inbox" and "Sent".  They have no file types on mail folders at all.  Thunderbird mail files are huge long text files containing all the messages in clear text concatenated one message after the other.  They look to avast as though each folder is one long .eml file.  So avast can, at least it has only ever been able so far, scan the very first message in each Thunderbird mail folder only.

The problem is that avast does not know that these files are Thunderbird mail files it has standard way of detecting them as such.

avast has been scanning my Thunderbird mail folders for years and I have some Thunderbird archive folders very much larger than yours.  However in recent times I have arranged my Thunderbird settings so that the archive files are held in a different folder that I exclude from scanning by avast - because it saves time in the scan and because avast does not understand them anyway.

I have not seen the problem of large temporary files being left around by avast but I do not usually interrupt my scans either.

I have just removed the exclusion and had the current version of avast scan my large Thunderbird archives.  It scanned them (1.8Gb of them) and it did complain - in the report - about one part being too large to scan, otherwise it completed without leaving any temporary files around.

I suspect that any abnormal termination of the avast scan might well leave one or more temporary files around that will not get cleaned up. 
Title: Re: Huge temp files are created and not removed
Post by: alanrf on May 08, 2008, 12:27:53 AM
I am not sure what is happening with your exclusion setting.  I just tried the same setting om my system and here is part of  the report:

As in your case I have shortened the path information.

Quote
Profiles\default.4p3\key3.db
  • is OK

Profiles\default.4p3\localstore.rdf
  • is OK

Profiles\default.4p3\Mail\Gmail1 [E] Skipped due to exclusions settings. (42019)
Profiles\default.4p3\Mail\Hotmail1 [E] Skipped due to exclusions settings. (42019)
Profiles\default.4p3\Mail\Hotmail2 [E] Skipped due to exclusions settings. (42019)
Profiles\default.4p3\Mail\Hotmail3 [E] Skipped due to exclusions settings. (42019)
Profiles\default.4p3\Mail\hotmail3-1 [E] Skipped due to exclusions settings. (42019)
Profiles\default.4p3\Mail\Local Folders [E] Skipped due to exclusions settings. (42019)
Profiles\default.4p3\Mail\localhost [E] Skipped due to exclusions settings. (42019)
Profiles\default.4p3\Mail\mail.comcast-1.net [E] Skipped due to exclusions settings. (42019)
Profiles\default.4p3\Mail\mail.comcast-2.net [E] Skipped due to exclusions settings. (42019)
Profiles\default.4p3\Mail\mail.comcast.net [E] Skipped due to exclusions settings. (42019)
Profiles\default.4p3\Mail\MSN [E] Skipped due to exclusions settings. (42019)
Profiles\default.4p3\Mail\pop.gmail-1.com [E] Skipped due to exclusions settings. (42019)
Profiles\default.4p3\Mail\Yahoo1 [E] Skipped due to exclusions settings. (42019)
Profiles\default.4p3\Mail\Yahoo2 [E] Skipped due to exclusions settings. (42019)
Profiles\default.4p3\mailViews.dat
  • is OK
Title: Re: Huge temp files are created and not removed
Post by: igor on May 08, 2008, 12:35:04 AM
Well, as I said - the MIME unpacker is trying to extract something from the mailboxes here (it's the PartNo_* lines; maybe it misunderstood the structure of appended e-mails, somehow). I guess the resulting "extracted" files are rather huge... but I'm afraid it's hard to say much more without seeing the actual files (mailboxes)  :-\
Title: Re: Huge temp files are created and not removed
Post by: alanrf on May 08, 2008, 12:41:19 AM
In further checking ...

I am suspecting that your exclusion setting may end "Mail\ *" with a space before the asterisk.  If I have that setting my results  look similar to yours.
Title: Re: Huge temp files are created and not removed
Post by: LorenzoC on May 08, 2008, 08:51:28 AM
@alanrf: if you are referring to me, I don't have any problem with exclusions, thunderbird mail is not scanned now. Like I said in the first post, I wanted to point out that if I cancel the scanning using the proper button before it ends (both standard and explorer extension) big temporary files are left in the _avast_ folder.
No other issues, besides the scanning being obviously slow when scanning the mail.

@igor: yes, I understand the mechanics of the thing. But see:
1. I guess Thunderbird uses a standard MBOX format for storing mail, not a proprietary one. File names in the folders should not be an issue. Everybody using Thunderbird has the same file format and naming.
2. My mailboxes aren't particularly huge (the whole "mail" is 170MB) and there isn't anything strange inside. There can be some images or PDF / Office documents attached, nothing more. Mailboxes are also "compacted".
3. My point is the way AVAST handles the mail scanning MUST be verified and no matter what it does upon scanning, it should be able to stop the scanning removing temporary files, expecially because by default it scans the mailboxes it EVERY TIME you scan your disk (of course unless you manually exclude).
4. If AVAST opens the mailboxes only to skip the content that is a joke.
Title: Re: Huge temp files are created and not removed
Post by: alanrf on May 08, 2008, 09:46:11 AM
LorenzoC

my posts were directed to you.  Igor is familiar with the format of Thunderbird files ... he and I have discussed them a  number of times over the years in this forum. 

I was explaining to you that I have very much larger files in my Thunderbird and I have had since before I started using avast.   Everyone of the folks I support using avast also use Thunderbird.  None of us are experiencing any problems with avast scanning Thunderbird files - nor are we seeing any significant complaints from other users about avast and Thunderbird.

Quote
I don't have any problem with exclusions, thunderbird mail is not scanned now.

The post of your avast report shows that clearly it is still being scanned. 

It seems likely you do have a problem with your exclusion setting.  I used the exclusion setting correctly and it produced the results I posted showing avast skipping the avast folders completely.

Quote
4. If AVAST opens the mailboxes only to skip the content that is a joke.

If there is a joke ... then it is not by avast and certainly not appearing on my system. As I clearly posted ... that does not occur with the exclusions set properly. 

If the exclusion is not entered properly, as I also described, then I get results from the avast scan very similar to the report you posted.

Compacted just means that the deleted mails have been removed from the folder.  I do notice that you do seem to have a large trash folder - Thunderbird provides options to clear that out for you (and reduce avast scan time) - unless you deliberately need to keep the trash.

I guess I have to repeat again that avast does not understand the format of the mail folders of Thunderbird.  It only can understand the first email message stored in the folder - the all the rest of the messages in the folder is just text to it and so scanning it is not valuable.

I have spent quite some time today in scanning my large Thunderbird files to try to reproduce the "left over files" that you experienced.  I can see the large files being created but every attempt I have made to interrupt then I can see avast cleaning up the files before it terminates.  I do not doubt that you experienced those files but reproducing the condition does not appear to be easy.       

Title: Re: Huge temp files are created and not removed
Post by: LorenzoC on May 08, 2008, 10:43:11 AM
Uhm... let me explain:

1.  AVAST scanning mailboxes, interrupted, I saw the temp files not removed.
2. I manually deleted the temp files.
3. In order to not keep on manually deleting those files I changed AVAST options so now the "mail" folder is excluded.
4. No more orphan temp files.

I say it again: mail excluded, no more files left in _avast_.

Now you may wonder why I am posting here if my local issue is "solved".
Simply because if I am having the issue it means other people can have it as well and IMHO it should be addressed.

I am happy you are not having the same issue but, you see, that is not particularly meaningful. Even if there were other 100 people who come here and say they aren't having any problem with it, still it doens't mean much.

I have a relatively small "mail" folder, the boxes are compacted, spam deleted, trash deleted, I've got only incoming and sent folders. In those folders there are my mail and some messages do have attachments. So what, can't I receive attachments? Those attachment are relatively small documents and images, some zipped, not a whole DVD content.

About skipping files, as I said above it would be much smarter to NOT scan MBOX files at all instead of decompressing, extracting, whatever it is done and then skip the contents. Incoming mail is already scanned, even if an attachement is executed the "standard shield" should stop it if infected. So it seems to me the scanning of the mailboxes is just a waste of resources/time. Expecially given the difficulties I am seeing.

Last thing: it is a little annoying when you are told between the lines that the issue you are reporting is not a flaw in the software but something wrong in you. Ok, I am not paying for AVAST but in the same time I am not payed to testing, bug finding, reporting and such.

I am seeing temp files more than 100MB left in _avast_.
I can reproduce the issue at will.
You say the issue doesn't exist? Ok, fine.

Title: Re: Huge temp files are created and not removed
Post by: alanrf on May 08, 2008, 10:51:56 AM
I will say once more and then leave you to your belief.

You can disbelieve the clear information I posted that shows you an exclusion - correctly set - causes avast to skip the Thunderbird mail files - it does not unpack them; it does not scan them. 

If you want to go on unpacking them - as your post above clearly shows you are - it is your system and your choice.  avast gives you a way to stop that - I showed you the evidence above.

There is no such thing as a standard for an MBOX file - it is just a plain text file.

     
 
Title: Re: Huge temp files are created and not removed
Post by: LorenzoC on May 08, 2008, 11:05:12 AM
Ok, I am stupid

Now let people fill their disks with "_avast4_\unp*.tmp files".

When somebody complains of it you tell them you aren't having any problem.

Case closed.

On a side note, I may be stupid but  it seemed obvious to me that when I was asked to generate the report file I DID NOT exclude the mailbox, otherwise I could not generate any report on it. The exclusion I set is on the whole "mail" folder so nothing is scanned inside. The files "skipped" are reported with NO EXCLUSION set (and NO blocker either). Otherwise I would not say it is a joke (the irony to generate huge temp file to skip their content) either suggest to exclude them as default.

a. no exclusion -> scanning cancelled -> temp files left
b. exclusion -> no scanning -> no temp files left
c. testing to generate the report -> no exclusion -> scanning completed -> no temp files left -> skipped contents (due to scanner settings) in the report.

My current status b.
That translates in "better if AVAST doesn't see Thunderbird mailboxes".

Edit: speaking of me being stupid, I find a little difficult to understand the idea of a software that can detect rootkits but can't recognize a directory from a file (see blocker blocking directories because their names) or recognize a Thuderbird MBOX file.
Title: Re: Huge temp files are created and not removed
Post by: alanrf on May 08, 2008, 11:44:30 AM
I said:

Quote
I suspect that any abnormal termination of the avast scan might well leave one or more temporary files around that will not get cleaned up.

Quote
I do not doubt that you experienced those files but reproducing the condition does not appear to be easy.

I have not said that you did not experience this problem. I have not said that the problem does not exist.

The stopping the scans of the Thunderbird mail is a separate issue. 

In the end I was simply trying to assist you to really stop scanning the Thunderbird files and asking you to take a look at the avast4.ini file in Programs File\Alwil Software\Avast4\Data.

When you look at the exclusion to avoid scanning the Thunderbird Mail accounts in the Mail folder then please make sure that the exclusion setting ends  ....\Mail\* and that there is no space between the \ and the *.  I believe you will find that if you then create a report of your avast scan the part_..... files will not be created and no longer appear in the report.
Title: Re: Huge temp files are created and not removed
Post by: alanrf on May 08, 2008, 12:05:49 PM
Just some parting thoughts ...

I think that you misunderstand.  I was trying to recreate your problem because I support a bunch of people using Thunderbird and avast too.  If you can have the problem then so could they, if I could recreate the problem it would help me and them should they ever experience the problem too.

I could probably point you to the post long ago where I said something very similar about avast being able to recognize a Thunderbird MBOX in this forum.  I thought it would be easy ... but then I remembered that it is just a plain text file of email messages and that those messages can be in every language on the planet. 

I have actually written code to manipulate Thunderbird mail files for myself and I know it is a Thunderbird MBOX.  I would hate to have to write the code to recognize a Thunderbird MBOX in every language possible - since there is nothing in it to say it is a Thunderbird MBOX.  Thunderbird mail files do not need to have "Thunderbird" in their folder names anywhere either if you care to use the settings available in the Thunderbird client. 

 
Title: Re: Huge temp files are created and not removed
Post by: LorenzoC on May 08, 2008, 12:24:34 PM
Ok, I put aside all the misunderstandings and summarize:

I was requested to create a report showing AVAST accessing the mailboxes during normal operations.
That is what I tried to do, with default settings, no exclusions or anything.

Pressing the button "cancel" in the scanner window is not what I would call "abnormal termination".

I guess "abnormal termination" is when you kill/terminate the process via task manager or such, which I never tried.

This kind of line:
\Mail\pop.gmail.com\Inbox\PartNo_0#599384397\PartNo_0#38486210\PartNo_0#3100651197 [E] File was skipped because of scanner settings. (42016)

Seems to show AVAST finds the "Inbox" MBOX file, extract/decompress something that is somehow nested, then decides to skip what it finds, my guess is because of the "standard scan" settings.

To extract the "PartNo*" things it creates the huge temp files in _avast_ and for some reason when you abort the scanning (pressing "cancel" button in the scanner window) before it comes to the true end AVAST isnt' able to properly terminate all operations and leaves the temp files in place.

My reasoning is:
1. I don't see anything strange in my mailboxes so I don't see any reason why AVAST should not be able to handle it, expecially if it is supposed to scan the mailboxes every scan as default.
2. It is not "good/normal" that, whatever AVAST is doing, it can't properly close the scanning task leaving the temps files behind (regardless the size).

Now, if the same issue or other issues have been reported about mail scanning and they had not been addressed, I guess it makes even more sense to exclude those dir/files from scanning.
I belive AVAST should be able to detect the mailboxes, but in my poor understanding you could simply add a procedure during install that scans the HD, finds mail client profiles and writes exclusion rules as part of the default settings. It is basic, you find "\Thunderbird\Profiles\" and exclude (with all the refining), you don't need to really detect the MBOX files.
Title: Re: Huge temp files are created and not removed
Post by: alanrf on May 08, 2008, 12:51:40 PM
Would you be willing please to post here the complete exclusions line from your avast4.ini file?

Then we will have a better idea of what is happening in scanning of your Thunderbird files, why it is creating the part_ files and then not scanning them.  I am fairly sure that you had other exclusions set when you posted the test results above, like excluding .msf files and part_ files. Not all the files were excluded in your report above (the Junk and Drafts folders were reported as scanned OK).

Again I wish I understood how you can recreate the orphaned avast files in the temp folder just by canceling the scan. I do not have a Win2K system or any FAT partitions to test on with my systems but perhaps this is a difference of avast installations in different operating systems 

avast knows nothing at all about any mail clients (and there are a lot of them) - with the exception of MS Outlook where it works with an interface provided specially for that product.  There are no standards in existence for email file structures within mail clients and no standards for the actual mail files themselves.  The only standards that every mail client must conform to are the POP and SMTP standards and it is those that the avast Internet Mail provider works with to ensure that the mail is clean before it is committed to whatever mail file format by whatever mail client you use.  avast simply does not know or care that these files belong to Thunderbird and it does not know Thunderbird is a mail client.     
Title: Re: Huge temp files are created and not removed
Post by: LorenzoC on May 08, 2008, 01:10:16 PM
Not it is becoming funny... I made the report WITHOUT ANY exclusion, how many times should I tell? :)

Currently in the "ini" file there is only this line:
Exclude=C:\Documents and Settings\Administrator\Dati applicazioni\Thunderbird\Profiles\mc135pum.default\Mail\*

That is what I've already said, intended to exclude the whole "mail" directory and downwards.

I added the exclusion after I noticed the issue with temp files and reproduced it some times, that means the issue happens WITHOUT ANY exclusion, in default condition.

Now, "default" means the "standard scan" and it should detect file types and scan against the appropriate virus types. I don't know if this can make the scanner "skip" during mail scanning.

About AVAST not knowing anything, well, but we know something more. I understand you can't provide coverage for any possibile mail client but you could at least exclude Thunderbird or other KNOWN troubled clients. Like I said, I would simply write the exclusion rules during the installing of AVAST, simply looking to the users profiles.

Edit: the junk, draft, etc "folders", that are OK, are obviously EMPTY.
Title: Re: Huge temp files are created and not removed
Post by: LorenzoC on May 08, 2008, 09:06:45 PM
Ok, I made another round test with the reporting ON and NO EXCLUSION, this time I switched on "thorough" instead "standard", here is the report (I shortened the path):

* Task 'Simple user interface' used
* Started on giovedì 8 maggio 2008 20.56.50
* VPS: 080507-0, 07/05/2008
*

\Mail\pop.gmail.com\Inbox\PartNo_0#599384397\PartNo_0#38486210\PartNo_0#3100651197 is OK
\Mail\pop.gmail.com\Inbox\PartNo_0#599384397\PartNo_0#38486210\PartNo_1#3325552817 is OK
\Mail\pop.gmail.com\Inbox\PartNo_0#599384397\PartNo_0#38486210 is OK
\Mail\pop.gmail.com\Inbox\PartNo_0#599384397 is OK
\Mail\pop.gmail.com\Inbox is OK
\Mail\pop.gmail.com\Templates is OK
\Mail\pop.gmail.com\msgFilterRules.dat is OK
\Mail\pop.gmail.com\popstate.dat is OK
\Mail\pop.gmail.com\Inbox.msf is OK
\Mail\pop.gmail.com\Trash is OK
\Mail\pop.gmail.com\Trash.msf is OK
\Mail\pop.gmail.com\Drafts is OK
\Mail\pop.gmail.com\Drafts.msf is OK
\Mail\pop.gmail.com\Junk is OK
\Mail\pop.gmail.com\Junk.msf is OK
\Mail\pop.gmail.com\Templates.msf is OK
\Mail\pop.gmail.com\Sent\PartNo_0#1731396481 is OK
\Mail\pop.gmail.com\Sent is OK
\Mail\pop.gmail.com\Sent.msf is OK

Infected files: 0
Total files: 19
Total folders: 1
Total size: 421,0 MB

*
* Task stopped: giovedì 8 maggio 2008 20.59.32
* Run-time was 2 minute(s), 42 second(s)
*

Compare with the previous report line:
\Mail\pop.gmail.com\Inbox\PartNo_0#599384397\PartNo_0#38486210\PartNo_0#3100651197 [E] File was skipped because of scanner settings. (42016)

The only difference is the different scanning method.

In this test I made AVAST complete the scanning.

Then I tried stopping it before it completes and the two huge temp files are still left behind in the _avast_ folder, so there isn't any change about the main issue of this thread.

So now I see 2 issues:

1. in "standard scanning mode" it seems the mailboxes aren't actually scanned because it ends in the "skipping" of every file. In "thorough scanning mode" it seems the mailboxes are scanned instead.
2. in what ever mode, cancelling the scanning makes AVAST leaving the temp files in _avast_ folder.


Title: Re: Huge temp files are created and not removed
Post by: igor on May 08, 2008, 09:19:23 PM
Igor is familiar with the format of Thunderbird files ... he and I have discussed them a  number of times over the years in this forum.

I'm not very familiar with the format of Thunderbird files... but as far as I remember, there's actually nothing to be familiar about - cause there's no "format" here.
The mailboxes are just a concatenation of the messages (pretty stupid, if you ask me) - so if they contain some strange messages (I mean messages with strange structure of parts, attachments, ...) - so it's possible that the MIME unpacker, responsible for extraction of one message, gets confused and crosses the border between the messages, and extracts something strange.

If it extracts a huge file, it's an operation hard to interrupt. So, if you cancel the scan and the on-going operation doesn't stop within some time limit, it's terminated in a "hard way"... and that's when the temporary files are kept in the TEMP folder, I'm afraid.

The limit could be enlarged, but it doesn't really solve the problem. Maybe the MIME unpacker could be improved somehow (not to extract what it shouldn't - though I'm not 100% sure it's the case here) - but not without inspecting the particular Inbox file.
Title: Re: Huge temp files are created and not removed
Post by: LorenzoC on May 08, 2008, 09:29:31 PM
Well, sorry but at this point I don't see what scanning the MBOX format is good for. Besides the fact the MBOX can't be properly handled, even if something wrong is detected It seems AVAST could not do anything about it in any case, besides trashing the whole file, so why even bother?

On a side note, I am not an expert but the "compacting" of the MBOX in Thunderbird should make sure the file is "cleaned" by any "strangeness", that is why I told I regularly compact the mailboxes.
For example long time ago I was using an antivirus who alerted me about viruses when the original message was  deleted and it did not once the mailbox was compacted, meaning before compacting the deleted messages were still present inside the actual MBOX file.
Title: Re: Huge temp files are created and not removed
Post by: alanrf on May 09, 2008, 03:24:20 AM
Two main sections to this thread:

1) the scanning issue.

We who have been here a long time and have our own exclusion list built can sometimes forget what it is to be a new user of avast. Based on that we can also find our long held assumptions about the differences in the levels of scanning in avast challenged, as mine have been by LorenzoC.  So I must express my thanks to LorenzoC for making re-learn some of those differences.

I believe that, with the exclusion you posted earlier, you will find that none of the files or folders (ie for Thunderbird in this case the accounts) will be scanned by avast at all (in any on demand scan).

I hope we can agree on the basic issue in the scanning function - that avast has no value in scanning our Thunderbird mail folders in its on demand scans.

2) the "orphaned" temporary files

I think Igor gave us a very clear pointer in this:

Quote
If it extracts a huge file, it's an operation hard to interrupt. So, if you cancel the scan and the on-going operation doesn't stop within some time limit, it's terminated in a "hard way"... and that's when the temporary files are kept in the TEMP folder, I'm afraid.

All of my testing to try to reproduce your concern was done on a relatively new multi-processor system with fast disks. Even with this system when I interrupted the scan of the Thunderbird files I noticed a very distinct lag before avast cleaned up the temporary files and I heard the sound associated with the end of the avast scan.  Given the information from Igor I can believe that on an older system with slower hardware there could be a timeout condition in which the scan would terminate before all of the disk activity (including cleanup) could occur.  In such a case since the slowness of the hardware would be a constant then the problem would be easily reproducible - as you have reported.

In support of Igor's comment on the MBOX issue.  When I started to write my own code to manipulate Thunderbird's MBOX mail folders I was surprised by the lack of consistency in the way messages are concatenated in the MBOX file.  Sometimes there is a blank line between messages, sometimes not and I can easily understand how the avast unpacker can be confused by this inconsistency.  It takes real hard work to separate the messages properly especially when plain text messages are simply run together.  I think Igor is referring to this problem rather than any issue of compacting which simply removes the deleted messages from the file.

There is a constant tussle among the Thunderbird developers many of whom believe MBOX was a bad decision and want to replace MBOX with MailDir.  We will have to see how this plays in the development of Thunderbird now that this very small player in the email world has itself been orphaned by Mozilla casting it out from the organization. 
Title: Re: Huge temp files are created and not removed
Post by: LorenzoC on May 09, 2008, 09:11:19 AM
Ok, understood.

Anyway the point here it is not how much Thunderbird sucks :) but how to make it live together with AVAST without issues and possibly without destroying your mail.

Inform users, work around limitations.
Title: Re: Huge temp files are created and not removed
Post by: alanrf on May 09, 2008, 09:38:16 AM
I have made my comments in the other thread you posted in.

Let's get one thing clear absolutely clear. avast has not come anywhere close to destroying your mail.

In this thread and now in others you appear determined to distort comments to fit your concerns.  I am saddened by your stance in this and I suspect that it may affect the level of assistance you may expect.
Title: Re: Huge temp files are created and not removed
Post by: LorenzoC on May 09, 2008, 10:10:47 AM
"you appear determined to distort comments"
This is a false accusation, you are simply being TECHNICALLY wrong in this thread and elsewere. Mail is not destroyed once the "Inbox" is moved to the "chest" but for the user it is not accessible until you move the "Inbox" back in place. Is that fun?

"I suspect that it may affect"
This is some sort of menace but I could care less. Infact your help wasn't either useful or requested. My post were not meant to ask for assistance, they were only ISSUEs reporting. I don't need any assistance, thanks.
Title: Re: Huge temp files are created and not removed
Post by: alanrf on May 09, 2008, 10:17:20 AM
Quote
I don't need any assistance, thanks.

I shall comply with your wishes - with the exception on rebutting false information.
Title: Re: Huge temp files are created and not removed
Post by: LorenzoC on May 09, 2008, 10:18:25 AM
Please show me the false statement(s).