Double traffic when using web shield

[mike_r]

Hi.

Does Avast WebShield downloads files in parallel with download managers? The thing is, some days ago I downloaded several PDF files using download manager (Download Master). After that my provider billed me exactly the double traffic volume - this had not happen anytime until I installed Avast WebShield and used it with "intelligent stream scanning" checked. I'm sure that those PDFs were only files I downloaded.

Is it a bug or a feature or...? Which settings do I have to use to avoid such behaviour?

Thanks.

[mike_r]

The next thought - it seems that I'm wrong about webshield. It collects IP packets and then scans the whole file, right?

[DavidR]

The Web Shield is a localhost proxy that scans http port 80 traffic, when your browser wants to view a web page it is retrieved and scanned by web shield and if of then displayed in your browser, so there should be no double scanning, web shield intercepts locally before arriving in your temporary internet files, browser cache.

I don't know how your download manager works, if it is using the http protocol on port 80 then that traffic will also be scanned in the same way, so again it shouldn't cause double traffic. Are you viewing the pdf files on-line as well as downloading them.

The provider if it has a bandwidth limit it will be logging every item downloaded, e.g. pages viewed, images, etc. they all have to be downloaded to your browser cache before they can be viewed. So you have to consider this as well as any files you download as it all contributes to the bandwidth total.

[RejZoR]

It's a simple explanation. Downloads were counted twice. Once when they were transfered to avast! Web shield proxy and once after the proxy.
It was the same data, just counted twice. Though i have no idea why would that happen.

[lukor]

Mike,
as Rejzor already explained, many download meters running on your local PC might count the traffic twice - the one between browser and WebShield and the one between WebShield and the web server. It can usually be solved by ignoring the local (or localhost) traffic from the counter.

If you were indeed billed for double traffic (and those must have been really large PDFs I assume) this should not be the case - the provider may actually see just the traffic leaving your PC - or am I wrong? Does the provider uses some kind of application on your system to count the transfered bytes?

Lukas.

[lukor]

Mike,
how come that the download with Download Master goes through WebShield in your installation? (have you configured DM to use WebShield as a localhost proxy (localhost:12080) ) ? The actual download is performed by the dmaster.exe process and is not scanned by WebShield under default settings.

Can you try to enable "Show detailed information on performed action" on the "Advanced" tab of WebShield configuration and confirm, that downloads are really redirected via webshield? (if this is true, you should see the information line as soon as the download is initiated)

[mike_r]

when your browser wants to view a web page it is retrieved and scanned by web shield and if of then displayed in your browser

That's clear. The only question here is whether the file is downloaded every time it's accessed or only once. Download managers can operate with several download streams - how does WebShield behave in this case?

P.S. I've checked the options - there was a single stream for each file, so the result may depend on how DM initiates downloading etc etc.

[mike_r]

If you were indeed billed for double traffic (and those must have been really large PDFs I assume) this should not be the case - the provider may actually see just the traffic leaving your PC - or am I wrong? Does the provider uses some kind of application on your system to count the transfered bytes?

Lukas.

Neither me nor my provider uses download meters on my machine (well, I had one some time ago, but billing was correct so I removed it), so that was the real traffic volume counted by the provider on his side.

[DavidR]

when your browser wants to view a web page it is retrieved and scanned by web shield and if of then displayed in your browser

That's clear. The only question here is whether the file is downloaded every time it's accessed or only once. Download managers can operate with several download streams - how does WebShield behave in this case?

P.S. I've checked the options - there was a single stream for each file, so the result may depend on how DM initiates downloading etc etc.

A file/web page in the browser cache should in theory depending on your browser, settings and the web page refresh setting. Once in the cache the next time you access it it should use the cached file, this is designed to compensate with slow connections and it being faster to access your HDD than the web. Some people with high speed broadband don't cache web pages so each time they access the web page it would be direct so it would be effectively downloaded many times.

When a download manager splits it onto multiple streams (if set), each one only downloads a small part of the file so it shouldn't increase the overall size. However, as Lukor mentioned "how come that the download with Download Master goes through WebShield" I too would have though it wouldn't have been using http protocol to download the file, but I know nothing about how Download Master works

[mike_r]

Mike,
how come that the download with Download Master goes through WebShield in your installation? (have you configured DM to use WebShield as a localhost proxy (localhost:12080) ) ? The actual download is performed by the dmaster.exe process and is not scanned by WebShield under default settings.

Can you try to enable "Show detailed information on performed action" on the "Advanced" tab of WebShield configuration and confirm, that downloads are really redirected via webshield? (if this is true, you should see the information line as soon as the download is initiated)

WebShield has been configured to scan the traffic coming through the port 80 and nothing more. AFAIK DM used the same port to download files.

Unfortunately, logging was disabled so I cannot say what happened exactly. Seems that somehow I initiated downloads twice (so that's not a WebShield's issue). I tried to download some files later and no one is catched by WebShield.

[mike_r]

When a download manager splits it onto multiple streams (if set), each one only downloads a small part of the file so it shouldn't increase the overall size. However, as Lukor mentioned "how come that the download with Download Master goes through WebShield" I too would have though it wouldn't have been using http protocol to download the file, but I know nothing about how Download Master works

DM as well as many other download managers uses some features of HTTP protocol (such as partial downloading). As I already posted it seems that partial downloading is not even monitored by WebShield in "intelligent stream scanning" mode.

[DavidR]

Well using firefox's inbuilt downloader with web shield enabled and intelligent streaming enabled (plus ignore local communication) the web shield still scanned a downloaded iso file (ultimate boot disk iso) before it was completely downloaded web shield alerted on a system tool contained in the iso file. So in my case it does scan partial downloads and the only way to download it was to pause web shield.

[lukor]

Mike,
how come that the download with Download Master goes through WebShield in your installation? (have you configured DM to use WebShield as a localhost proxy (localhost:12080) ) ? The actual download is performed by the dmaster.exe process and is not scanned by WebShield under default settings.

Can you try to enable "Show detailed information on performed action" on the "Advanced" tab of WebShield configuration and confirm, that downloads are really redirected via webshield? (if this is true, you should see the information line as soon as the download is initiated)

WebShield has been configured to scan the traffic coming through the port 80 and nothing more. AFAIK DM used the same port to download files.

Unfortunately, logging was disabled so I cannot say what happened exactly. Seems that somehow I initiated downloads twice (so that's not a WebShield's issue). I tried to download some files later and no one is cought by WebShield.

WebShield does scan the port 80 under default configuration but only for certain applications (known to us as supported browsers). Unless you have tweaked WebShield to scan DM traffic (which is indeed possible), WebShield does not scan DM at all (on port 80 or any other you can think of).

For the sake of objectiveness I have tried to install the download manager and it really does behave as it should. Secondly, partial downloads are no problem for WebShield. They come as separated HTTP requests and as thus they are processed by WebShield (if WebShield is used for that particular app). Of course, the possibility that WebShield will miss a virus (spread across different download parts) is higher.

Lukas.

[Lisandro]

Unless you have tweaked WebShield to scan DM traffic (which is indeed possible), WebShield does not scan DM at all (on port 80 or any other you can think of).

Lukas, how can we do it without losing the protection of the default 'supported browsers'?

[DavidR]

If DM isn't supported would the optin process not work Tech.

@ Lukas
Since we are on the supported browsers issue again, how are users to know exactly what is a supported browser ?

Since avast took the decision not to have web shield monitor every application using the http port, there is no reference anywhere of the supported applications. I understand that this information is contained within the VPS file so it can be updated, unfortunately users can't access that information. Perhaps that information can be extracted to the avast GUI under the Web Shield interface, Supported Browser (button/link) or something like that ?

This information would not only be usefull the avast users but also those on the forums trying to help.