Upload Issues (Web Shield) [Fix inbound]

[Parabell]

I had the problems with lots of small files (jpgs 2-5mb) and
also with files about 500GB, as well on Dropbox as with some
http/https uploads (not all of them, some Sites work, some not).

My first problems I had with dropbox, uploading the latest
Wedding pics to my boss, which worked always for one or
two pictures after reebooting the computer. After three days
and only some files uploaded, I transferred the files via
network to my Win 7 professional 64Bit, also with Avast on
it, from which i was able to upload the files without probs.

At that time I had no clue and tried all the way down to
deinstalling dropbox, reinstalling, deleting caches and so on.

Since I found out Avast being the cause, I figured out that
after a reboot of my computer dropbox was somewhat quicker
launched and online than Avast, so always a few pics got
out, so that phenomenon made sense.

On my computer it worked few days to exclude all the dropbox
directories from Avast in the settings - but now that doesn't work
either. No idea, why.

It's kind of maddening...

Has anybody also a Win7 Home and Prof computer and can confirm
if this problem is existent on both versions or only on the home edition?

[OliPicard]

I can confirm the issue is on Windows 7 Pro and Ultimate versions. Not sure about the Home one.

[Parabell]

Ok, then it is just luck that my win7 prof had none of these issues,
while my Win7 home keeps me swearing.. 

[wefixit]

Just out of interest have you noticed this issue only on larger files? Can you Sync small files or is it just all files? I noticed when uploading to websites using HTTP4 and HTML5 that small uploads (less than 10mb) uploaded OK however anything over this amount resulted in Upload issues as described in the original post.

I haven't really noticed, the only thing I noticed is that if I left avast with shields on (before I realise that this was causing problems), was that when I exited dropbox and then re opened it, it manage to upload 4-5 files that were around 2MB each, then everything stalled even other files that were that small...
I hope this resolves soon...

[OliPicard]

Ah Ok, Was just checking as I have found smaller files with the flash uploader would upload ok. With normal HTTP and HTML5 uploads however the issue is still active even with smaller files.

[drake127]

Guys, is it a progress bar issue or do you have actual problem with your upload? If you disable avast! and upload some file, does it take the same time as with avast! regardless of the progress bar? Just don't look at it.

[Parabell]

Funny Question.

No, the Problem is (as mentioned many times before from
different people) that uploads SEEM to work, but no data
at all is transferred.

You can try and wait DAYS, it is not possible to upload
even 60 small jpgs into dropbox with Avast webshield
active.

And it is no Dropbox problem, as other upload possibilties
are not working, too.

[drake127]

You are right, I am experiencing the same issue. However I was able to upload whole CD to Dropbox at my standard speed (~ 5 Mbps). It was just Dropbox progress bar that acted funny - "uploaded" few MB instantly, then got stuck for minute or so and then did the same until the upload completed.

From what you are saying, you experience a little different behaviour which prevents you from finishing the upload. Am I right? Can I do anything to reproduce it? How much RAM do you have available? How fast your internet connection is?

[OliPicard]

Guys, is it a progress bar issue or do you have actual problem with your upload? If you disable avast! and upload some file, does it take the same time as with avast! regardless of the progress bar? Just don't look at it.

Hi Drake, My personal experience has been that the file has claimed it has "uploaded" its self along side setting a super unrealistic upload speed. Then when the upload "finishes" the upload has only been 1% complete in comparison to the "completed" version the avast shield provides. (I included screenshots with my logs sent to your email address ) If i disable the shield everything goes back to normal but if I re-enable the shield the problems come back not just on sproutvideo but also on dmcloud (using a flash uploader) This bug isnt affecting YouTube or any Google services. It seems to be down to upsetting a JQ/Json Lib. This is the error i found when debugging dmcloud. 

19:36:54.451 SyntaxError: JSON.parse: unexpected end of data jquery.json.js:38 so something is interfering with the data stream. Json/JQ thinks the file is still going and then panics. the json shows the actual speed while the scripts have been modified when the web shield gets between them.

[OliPicard]

Just reading what Lukas has said to the dropbox thread. Seems like a very hard problem to re-create. If theirs any data you need do let us know!

"Hello Guys,
it seems this is really caused by the nature of the WebShield implementation. The super-fast upload phase is caused by the fact, the until avast buffers are filled, the data is consumed by WebShield, scanned and then passed onto the network. There the speed should be appropriate to your network speed, however since the application (dropbox) has already sent quite a lot of bytes, it would seem as stalled for a while. In total, the speed shouldn't be affected, the total time should be the same, however you could see these super-fast bursts. We haven't experienced any complete stalls or aborted uploads though. We will look into this and post you soon.


Sorry guys,
Lukas."

[lukor]

Just retried that with the beta. Uploading several files to DropBox and I can say the progress bar is really confusing - at my current location, the file says: 3 minutes left (with WebShield off), with WebShield on, it went straight to 75% and said 5 secs left. However, it still took 3 minutes to complete the upload and the files were uploaded correctly.


Lukas.

[OliPicard]

Just retried that with the beta. Uploading several files to DropBox and I can say the progress bar is really confusing - at my current location, the file says: 3 minutes left (with WebShield off), with WebShield on, it went straight to 75% and said 5 secs left. However, it still took 3 minutes to complete the upload and the files were uploaded correctly.


Lukas.

Application uploads will go through web transfers (http and HTML5 web apps) won't as they are able to compensate for bad/strange uploads. Dropbox is built on S3 and ive found no issues using the AWS S3 uploader myself. However other applications that rely on a linear latency system will act up upon uploading using HTML5 uploaders and Flash Player uploaders which often interface directly into the API backend

Uploads failing are web app based using server side scripts to confirm the upload being successful are resulting in quicker than normal termination of uploads as the Avast web shield is providing an illusion of a fast connection by gathering information of the hardware spec to use as a bandwidth throttle. 

[lukor]

We were trying hard to optimize the webshield's operations to consume the data as fast as possible that we forgot sometimes too fast is also a problem.

This is clearly a bug and now, when demystified by other fixes inside beta, it's cause is also very clear. We will come up with some solution, even though it's not an easy problem. It's somewhat complicated to persuade the client to slow down.

Thanks. Lukas

[Parabell]

Lukor, please try some more files with dropbox.

Try to copy a whole dir with perhaps about 60 files.
5 files are the amount which I sometimes get up
by luck, but more I have never got completed.
As mentioned, not in days.

In some webuploads I geht the same problem,
with unrealistic speeds, the Progress Bar jumps,
but after "completing" upload (get the message
from the site) there is nothing on the server.
 
I had that on two sites with about 20 big zips.

[Parabell]

Found something out:

It is Dropbox (trying to synchronize) which causes Avast to
freeze, when I try to deactivate the Web Shield.

When Dropbox synchronization is paused, Avast is able to
deactivate the Webshield without freeze.

But SOMETIMES Avast seems to hinder Dropbox from getting
into "pause synchronization"-mode. In that cases it seems
that Dropbox is freezing (but is not, the task is still active,
only needs minutes to get in pause mode).

If Avast webshield is paused, Dropbox will behave completely
normal and responsive.

They really dont like each other anymore... 

Cheers,

Monica