Avast WEBforum

Consumer Products => Avast Free Antivirus / Premium Security (legacy Pro Antivirus, Internet Security, Premier) => Topic started by: Rundvleeskroket on June 26, 2016, 05:29:04 AM

Title: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 26, 2016, 05:29:04 AM
Hi. It seems that URLs that Avast deems dangerous are blocked. But when I uncheck 'Block malware URLs' under Web Shield Settings, they remain blocked. Is this normal?

The Avast notification does change though. From: 'Infection URL:Mal', to 'Infection URL:Mal2'. Note the '2'.

Disabling Script blocking also does not fix this. I have to entirely uncheck 'Enable Web Scanning' to access the URL. I'd rather not do that of course.

The URL I'm trying to visit is http://sceper.ws/page/2 and every subsequent number. The main site itself (sceper.ws) does load fine, without warnings. The content on the main page is on the numbered pages after a few hours, so I do want to be able to view those without disabling (large parts of) my AV. And there isn't an option as far as I can tell that lets me add an exclusion to URL blocking for a specific domain. I can add a URL exclusion to Web Shield, but that excludes it from all the other scans in there as well. I don't want to do that.

Is there currently actual malware on those numbered pages? It doesn't seem so. I am however running adblock, mixed content blocking, and privacy badger. In Firefox. If not, perhaps those pages should be removed from the blacklist. And perhaps disabling the malware URL blocking option should actually disable said blocking.

Hopefully I can get some clarification, and maybe even an update to the blacklist. Thanks  :)
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: mchain on June 26, 2016, 06:38:25 AM
By disabling Web Scanning, as you know, there is a risk in unforeseen actions on your system:
http://zulu.zscaler.com/ (http://zulu.zscaler.com/)
Use this site to scan for both hxxp://sceper.ws/page/2 and hxxp://sceper.ws/  (urls disabled for user safety, to restore/remove x's with t's)
You should get this:  http://zulu.zscaler.com/submission/show/d800eb206ba179527a7bed8785f383d3-1466914402 (http://zulu.zscaler.com/submission/show/d800eb206ba179527a7bed8785f383d3-1466914402)
Elevated Phish risk
Main site sceper.ws not infected.

Avast WebBlock attached.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 26, 2016, 06:58:40 AM
I'm not getting that alert about a malicious JavaScript. One of my blockers may perhaps have already filtered it out.

All I get is the "Infection URL:Mal" warning. So, blocked because the URL has been serving something it shouldn't in the past. Or still is?

I never ever download anything from there. I just use it as a TV-guide of sorts. See what's new. Nothing else.

I don't want to disable Web Scanning. But there doesn't seem to be a way to add a specific domain exclusion for the URL-blocking feature only. So I have to choose between disabling Web Scanning entirely, or not seeing the pages at all.

As I understand it from the zScaler site, it is a known site with an elevated phishing risk. Fine. I understand the risks. It doesn't mean the site is dangerous when just viewed without clicking on things, right?

I'd still like to know how come when I uncheck the malware URL block option, it remains blocked, and with another 'Infection URL:Mal2' alert. Didn't I just explicitly disable that check? How is Mal2 different from Mal?

I'd be happy with an option to proceed with caution.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: mchain on June 26, 2016, 07:21:54 AM
Avast block posted in attached .png was from a different site:  www.scanurl.net, a different web scanning site, scanning the same website you cannot visit, from a third-party point of view. 

Note the avast block is:  JS:ScriptPE-inf [Trj] -- avast Web Shield.

Phish warnings by independent third-parties should not be ignored as your web site is likely compromised.

Only other possible setting for Web Shield is 'Ask'.  See attached below: 
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Eddy on June 26, 2016, 07:25:24 AM
URL:Mal = Domain and/or IP is blocked.
If you get to see URL: Mal or URL:Mal2 depens on what part of avast detects it.

Suspicious scripts and links to blacklisted sites :
http://www.web-malware-removal.com/website-malware-virus-scanner/?url=sceper.ws

Blacklisted domain :
https://www.virustotal.com/en/url/c4adbccb19fab3e0a2cf1b1d1e0902e7750ff866ca5e2814282d58398eb68b14/analysis/1466917838/

Malicious, link to blacklisted domain :
https://quttera.com/detailed_report/sceper.ws

Blacklisted domains on that ASN :
http://urlquery.net/report.php?id=1466917118548
http://urlquery.net/report.php?id=1466917140794

Really bad (IP) history :
https://www.virustotal.com/en/ip-address/91.235.143.212/information/

Lot's of malicious activity :
http://zulu.zscaler.com/submission/show/1dc4f3b839b6fc9da03af421e947a4cb-1466917825

Vulnerable code used :
http://retire.insecurity.today/#!/scan/9b01a8178cdc5e65418b70838e30ab912ed5dfad76313cc18f92618482fdc0df

If I remember correctly, we have shown that the site isn't safe in the past already.
Since they haven't changed their practices, it is very unlikely the block on it will be lifted.

We do not help people to get on malicious websites.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: essexboy on June 26, 2016, 12:42:59 PM
Also reported by FF in addition to Avast

Quote
What are Deceptive/Phishing, Attack Sites, Unwanted Software and Malware?
Deceptive Site (also known as “Phishing”)

This is a form of identity theft that occurs when a malicious website impersonates a legitimate one in order to trick you into giving up sensitive information such as passwords, account details, or credit card numbers. Phishing attacks usually come from email messages that attempt to lure the recipient into updating their personal information on fake but very real-looking websites. More information on phishing can be found at the Anti-Phishing Working Group, and there are a number of examples and resources available at the Wikipedia Phishing page.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 26, 2016, 04:14:57 PM
Thanks for the info. I am well aware of what phishing is. I do not give out personal information even on most legitimate sites. If at all. It is still not clear to me how come when I disable URL blocking, I'm still getting the website blocked. Disable blocking should mean exactly that imo. What part of Web Shield is responsible for the Mal2 alert, and can I add an exclusion to it?
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Eddy on June 26, 2016, 05:35:13 PM
See reply #4
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: DavidR on June 26, 2016, 05:54:17 PM
@ Rundvleeskroket
Your are misinterpreting the Site Blocking function (there is no 'Block malware URLs' option that you mention) - it doesn't stop avast scanning and blocking sites - its purpose is to allow 'you' to add sites to block irrespective of avast scanning finding it clean.

The actions you are experiencing with sites detected as malicious, etc. are correct.

In the Site Blocking window you will see that it allows for URLs (to block) to be entered. Uncheck the 'Enable site blocking. and the screen changes so you can no longer enter URLs.

EDIT: attached image.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 26, 2016, 06:37:23 PM
@ DavidR: Avast --> Options --> Active Protection --> Web Shield --> Customize --> Block malware URLs.

In the Help via the question mark in top right it says about this function:

Quote
Block malware URLs - Block websites based on a database of known malware URLs.

I'm not even looking at site blocking. That's not what I'm talking about.

So, even though I uncheck 'block malware URLs', Avast still blocks malware URLs. Rendering that checkbox moot.

@ Eddy: Setting to 'Ask' will have the whole of Web Shield ask what to do if it detects something, not specifically only the malware URL blocking part of Web Shield, right? If so, again, that is not what I'm after. I want to have Web Shield enabled, actively scanning my browsing, but allow me to proceed to access a site known to be of higher risk, at my own discretion. I don't consider phishing much of a risk to myself. I would however like the benefit of the script checking and such. So adding an exclusion for the whole of Web Shield is not preferred. And this isn't even adding an exclusion, but instead changing global behaviour. I don't want Web Shield asking me what to do for all browsing all the time.

Also, if the warning that pops up would tell me what malware is found, this would be helpful. I'm less inclined to proceed if a malicious script is trying to run, but less concerned if the site just contains a fake login or something of that ilk. The generic 'URL:Mal' warning doesn't give me enough detailed information about what exactly is wrong with the site I'm trying to visit.

Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Eddy on June 26, 2016, 07:03:11 PM
Why do you even have avast installed if all you want to do is bypassing the protection it is offering you ?
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 26, 2016, 07:25:52 PM
I'm not bypassing 99% of the protection. I just want to customize that one remaining percent of protection to allow me to access a site. By all means, keep the blacklist, but give me a way to override the blockade with minimal deactivation of other components of Avast.

I'll ask yet again: why does disabling 'block malware URLs' not actually disable said blocking? Yes, the alert changes from Mal to Mal2. So another part of Avast is now blocking. Which part is that specifically, and can I change it settings to my liking?

Malware comes in different guises. Not all are equally dangerous. I understand the default one size fits all approach, but that leaves advanced users out of options.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: DavidR on June 26, 2016, 07:26:20 PM
@ DavidR: Avast --> Options --> Active Protection --> Web Shield --> Customize --> Block malware URLs.

In the Help via the question mark in top right it says about this function:

Quote
Block malware URLs - Block websites based on a database of known malware URLs.

I'm not even looking at site blocking. That's not what I'm talking about.

So, even though I uncheck 'block malware URLs', Avast still blocks malware URLs. Rendering that checkbox moot.
<snip>

Apologies, I did think you were looking in the Site Blocking, since you were still getting alerts.

I visited the link that you gave using Firefox 47.0 and that gave a FF alert, blocking it before even avast got there. See attached FF notice image and the 'Why was this page blocked ?' https://support.mozilla.org/en-US/kb/how-does-phishing-and-malware-protection-work?as=u&utm_source=inproduct (https://support.mozilla.org/en-US/kb/how-does-phishing-and-malware-protection-work?as=u&utm_source=inproduct)

Can you attach an image of the alert you are getting.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 26, 2016, 07:38:38 PM
@ DavidR: I have unchecked 'Block reported attack sites' and 'block reported web forgeries' in Firefox :)

With the checkbox in Web Shield enabled:

(https://s31.postimg.org/f7lqzssqj/mal.png)

All six entries are identical.

With the checkbox disabled:

(https://s31.postimg.org/j5dm2y4xn/mal2.png)
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Eddy on June 26, 2016, 07:40:06 PM
That site is bad and not only because of phishing activities.
Do the smart thing and stay away from it.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 26, 2016, 07:49:53 PM
Disabling URL blocking should let me see the site, or the popup should give me another kind of alert notification for whatever reason Avast then has determined is cause to block.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 26, 2016, 08:06:39 PM
That site is bad and not only because of phishing activities.

I'm sure it is. So let Avast scan and check all of it, but let me access the site at my own peril if I so choose.

FYI: I run Avast on several machines. Some of those are virtual machines with more vanilla Firefox. No mixed content blocking (although 47 may do it by default now). No Privacy Badger. Just Adblock Plus. I've disabled Web Shield in one of them and visited the site in question.  Clicked all over the place. Then re-enabled Web Shield, ran a full system scan, ran antispyware tools, the works. No problems found. Seems the phishing is really the offending part (currently). And phishing is hardly a problem if one uses common sense.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Eddy on June 26, 2016, 08:51:13 PM
Quote
And phishing is hardly a problem if one uses common sense.
It is very obviously that you don't use common sense.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 26, 2016, 08:57:06 PM
Luckily that is not for you to decide. You have stated your concerns. They are duly noted. If you aren't going to contribute to this thread in a helpful fashion, please consider yourself excused. Thank you for your input.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: DavidR on June 26, 2016, 09:32:07 PM
@ DavidR: I have unchecked 'Block reported attack sites' and 'block reported web forgeries' in Firefox :)
<snip>
With the checkbox disabled:

(https://s31.postimg.org/j5dm2y4xn/mal2.png)

Whilst I might not agree with going to these lengths to visit a highly suspect site (it is your system), but the avast settings/options when set should work and this didn't. First time I disabled only the web shield Enable Web scanning, thinking that should work for all scanning, it didn't and I got the URL:Mal alert.

I can in a way confirm (but not exactly) what you reported, I didn't switch off the settings in firefox - allowing that warning to display and opt to ignore. I disabled not only the Block malware URLs but also Enable Web scanning (Note to readers don't do this if you haven't got a robust backup and recovery strategy in place).

Here is when it starts to get strange, first I didn't get the URL:Mal and URL:Mal2 alert that you posted, but two other popups.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: bob3160 on June 26, 2016, 09:32:39 PM
Why bother using an AV if you don't intend to listen to it's advice ???
If you think the detection  is incorrect, report it and wait for a change in the detection if you're correct.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: DavidR on June 26, 2016, 09:36:14 PM
Why bother using an AV if you don't intend to listen to it's advice ???
If you think the detection  is incorrect, report it and wait for a change in the detection if you're correct.

This really is a side issue to the fact that the avast web shield settings don't work as expected, regardless of wanting to commit on-line suicide.

We might not agree with it, but the settings should work.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 26, 2016, 10:00:39 PM
Exactly. The behaviour of this checkbox isn't as expected.

If my AV tells me: "This site might try to scam you", fine, noted. I'll be careful. If it just flat out blocks the site because some people might fall for said scams, at least give me an option to say: "I get it. I understand. Now let me proceed." Make it an advanced option somewhere. I'm perfectly willing to take my chances. I have backups. I have virtual machines. Blocking for viruses is one thing, blocking for phishing is another. I'm fine that Avast does this by default, and on most sites I would have it enabled. But ultimately I decide :)

Ideally I would have URL blocking enabled, except for explicitly added exclusions. So, that option doesn't exist, and the option that does exist doesn't do what it is supposed to do.

It would seem I have stumbled upon a bug then :)
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: HonzaZ on June 27, 2016, 10:49:26 AM
Hi all,
You can turn webshield, but the exclusions only work on (JS, HTML, ...) detections on the domain, not the domain itself. To override URL block, you have to disable shields.
DavidR: The 2 popups that you were seeing are from AOS (Avast Online Security, the browser plugin).
"This site could have harmed your computer" is an equivalent of URL:Mal detection, AOS uses the same database for that
"This site has been marked as a phishing site" is a result of our internal phishing database, that is created (loosely) based on Mailshell, Phishtank, APWG and Cyren databases.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: bob3160 on June 27, 2016, 03:01:03 PM
Hi all,
You can turn webshield, but the exclusions only work on (JS, HTML, ...) detections on the domain, not the domain itself. To override URL block, you have to disable shields.
DavidR: The 2 popups that you were seeing are from AOS (Avast Online Security, the browser plugin).
"This site could have harmed your computer" is an equivalent of URL:Mal detection, AOS uses the same database for that
"This site has been marked as a phishing site" is a result of our internal phishing database, that is created (loosely) based on Mailshell, Phishtank, APWG and Cyren databases.
If I understand this correctly, the only way to bypass a dangerous site is to turn off the Shield.

Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 27, 2016, 11:55:01 PM
I don't want to turn off the Web Shield. But I do expect that if I disable URL blocking, Avast actually stops blocking URLs (and giving me an alert when I visit a site that it blocked it based on URL --> URL:Mal(2)).

As it stands, disabling that feature doesn't in fact disable it.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: polonus on June 28, 2016, 12:09:04 AM
Ha die Rundvleeskroket,

Why you wanna turn of the obvious and reasonable Mal:URL or Mal:URL2 blocking alerts for Pete's sake?
I would keep all protection up and running, not make an exclusion for that website, but visit it using a webproxy of sorts,
like for instance https://whoer.net/webproxy
Whenever you are blocked against something in the code while running a site on a webproxy server,
then you should report back here, because then the site you wanna visit perse is laden with malcode or working something uncanny out into the browser executable.
What does the Google Safebrowsing report on that particular site say, and what do you get scanning it at: isithacked.com

When not enough of the vulnerable or suspicious or even malware code is showing a webproxy is a safe way of going somewhere
to keep a particular threat at bay and not given up the exisiting line of defense and protection.

Ik zou zeggen probeer dit eerst eens uit en dan zien we wel weer, gegroet,

polonus
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 28, 2016, 12:11:43 AM
I can visit the site in a dedicated VM if I must. But I'd rather avoid the hassle.

Tell me: what is the point of having a checkbox, if it doesn't matter whether it is checked or not? At the very least the feature should behave as expected. It currently does not.

Het komt er op neer dat ik zelf wel bepaal welke functionaliteit ik wel en niet gebruik, en welke sites ik wel of niet bezoek. Mijn virusscanner is een gereedschap om mijn risico's te mitigeren, en daar maak ik dankbaar gebruik van. Maar dat wil niet zeggen dat ik moet kiezen voor het keurslijf van Avast gericht op klikgrage lui. Ik ben me bewust van de risico's. Blijf vooral waarschuwen dat het gevaarlijk is. Ik weet het. Ik kies er voor om door te gaan. Serieus .. wat moet er gebeuren dat zo spannend is? In het ergste geval zet ik gewoon een image van vorige week terug. Of die van vorige maand. Of die van twee maanden terug. Etc. 10 minuten werk. Desnoods op een andere schijf die ik een stapel heb liggen. Ik vind het vreemd dat ik herhaaldelijk gevraagd word waarom ik het wil doen, want oei oei oei, terwijl ik dat al heb uitgelegd. Ik vind het alleen ook onzin om hele stukken van Avast uit te schakelen terwijl een klein deel een uitzondering geven zou kunnen volstaan. Dan geniet ik het voordeel van de AV, en kan die alsnog als ik daadwerkelijk de site bezoek over de zeik gaan en me waarschuwen.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: polonus on June 28, 2016, 12:21:11 AM
Well go watching it in that webproxy shows the site, but also the alert which is for JS:ScrptP-inf[Trj]
The site is redirecting to -http://www.adnetworkperformance.com/a/display.php?r=121020
which should be blocked.

It is malware, read http://www.freefixer.com/b/remove-adnetworkperformance-com-pop-up-ads/
You want that infection because you just like to go to that page?
It is a free world, do as you please. I would use a script blocker then to get away from,
and keep all the fake malware protection off of your machine, these phising guys seems in bed with.

polonus
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 28, 2016, 12:31:02 AM
Except it isn't!

Yes, if I use the proxy I get the same alert about a trojan. But I just visited the site in a VM. With Firefox and AdBlock Plus. And Avast.

After disabling Web Scanning I can visit the site. I am not redirected anywhere. No popups or popunders. Just the site. Works fine for what I need it for.

I'm not dismissing the risk. Just that it seems acceptable to me. ABP is probably already filtering out the threat.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: bob3160 on June 28, 2016, 12:35:47 AM
Since about 80 - 85% of all infections come through the internet, shutting down the WebShield
isn't really a safe remedy. Your computer, your risk and your choice. Definitely not mine.

Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: DavidR on June 28, 2016, 12:40:25 AM
Since about 80 - 85% of all infections come through the internet, shutting down the WebShield
isn't really a safe remedy. Your computer, your risk and your choice. Definitely not mine.



I think that we, as avast users have made our point clear, but it doesn't excuse why the avast settings don't work as expected.

Its broken and avast needs to either fix it or remove the options.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 28, 2016, 12:46:14 AM
I think I have made it perfectly clear that I prefer to have Web Shield enabled. And URL blocking too, for that matter. However, I'd like the option to add an exclusion. Or at least the option to proceed after Avast blocks a site.

All this is secondary though, to the fact that the option to disable the URL blocking feature all together is broken. What I need it disabled for isn't all that relevant. The option exists. It should work.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: bob3160 on June 28, 2016, 12:47:16 AM
Since about 80 - 85% of all infections come through the internet, shutting down the WebShield
isn't really a safe remedy. Your computer, your risk and your choice. Definitely not mine.



I think that we, as avast users have made our point clear, but it doesn't excuse why the avast settings don't work as expected.

Its broken and avast needs to either fix it or remove the options.
Totally agree. Either it works or it shouldn't be there.
(Reported to Avast let's see if that helps.)
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 28, 2016, 01:07:51 AM
Thanks. Let's hope a fix comes soon.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Eddy on June 28, 2016, 07:16:13 AM
There is no fix needed as everything is working as it should.
See HonzaZ's reply.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: DavidR on June 28, 2016, 02:51:18 PM
There is no fix needed as everything is working as it should.
See HonzaZ's reply.

It is a dogs hind leg completely bent out of shape. There is no delimiting factor in the UI that explains it is only for JS and HTML. We shouldn't expect users to be mind readers to know what is the unwritten rule. Sorry but users expect if they enter a URL exclusion that it bloody works and is excluded.

It needs fixed period, one way or another:
1. full explanation of exactly what can be excluded.
2. all URLs regardless of categorisation should be allowed.
3. remove the exclusions completely.

Why the hell it can't be excluded is beyond me (even after HonzaZ's post), when the only other option is to disable the web shield completely, so they might as well have the exclusions work for all instances.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on June 28, 2016, 08:16:22 PM
^

This guy gets it!  :)
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 12:53:38 PM
Bump. Can we expect the URL blocking/unblocking to work in the near future? Any other news regarding this bug?
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: HonzaZ on July 05, 2016, 01:31:39 PM
Uhm... I just tested a couple of URLs which we block. I added them to exclusions (I tried both global and webshield exclusions) and I can access the blocked sites normally without any popup. Seems like I have been mistaken about the fact that URL:Mal detection cannot be excluded; it seems like everything is working like a charm.

If you guys have any trouble setting up exclusions, please do let me know ;)
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 01:45:25 PM
I just tried http://sceper.ws/page/2 and it is blocked whether I enable or disable URL blocking in Web Shield.

I don't want to exclude a URL from the entire Web Shield scanning, just to unblock a URL and keep Web Shield scanning enabled.

But, If I do uncheck Web Shield URL blocking all together:

The URL blocking option in Web Shield does not disable URL blocking when disabled. It is not fixed.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: HonzaZ on July 05, 2016, 01:48:23 PM
It did work for me with adding "http://www.sceper.ws/page/2" to exclusions.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 01:49:36 PM
AGAIN .. that excludes it from all of Web Shield. Not just the URL blocking part of Web Shield.

And it does NOT fix the bug in that checkbox.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 01:53:20 PM
Or are you saying that Avast uses the term WEB for all of the internet? In that case Avast is just plain wrong. www or no www in the URL really shouldn't matter.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: HonzaZ on July 05, 2016, 02:09:01 PM
AGAIN .. that excludes it from all of Web Shield. Not just the URL blocking part of Web Shield.
Yes, that excludes it from all of webshield, and I think this is intended.

And it does NOT fix the bug in that checkbox.
If you mean the "Block malware URLs" checkbox, I filled in a bug report that is doesn't include URL:Mal2 detection. ;)

Or are you saying that Avast uses the term WEB for all of the internet?
Webshield scans everything that comes from the net.

www or no www in the URL really shouldn't matter.
Www or no www are two different subdomains. The fact that most webservers are configured to prepend "www." if no subdomain is specified does not mean they are the same.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 02:22:37 PM
Quote
If you mean the "Block malware URLs" checkbox, I will report that is doesn't include URL:Mal2 detection

Thank you.

Quote
Webshield scans everything that comes from the net.

That makes the name Webshield factually incorrect ;)

For the sceper url www or no makes no difference. Both work if typed in the browser. However, the page selector on the bottom of the site always leads to pages without www. Excluding the sceper domain, with or without www will indeed exclude it, but that is not what I want. It now also excludes the domain from the script scanning and other parts of Web Shield. That is an unnecessarily broad method of exclusion, imo. I really think it would be nice to have a way to exclude a specific URL from URL blocking, and keep the URL blocking feature itself enabled for all other URLs :)
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: HonzaZ on July 05, 2016, 02:56:35 PM
Quote
Webshield scans everything that comes from the net.
That makes the name Webshield factually incorrect ;)
Perhaps NetShield was not as catchy :)

Excluding the sceper domain, with or without www will indeed exclude it, but that is not what I want. It now also excludes the domain from the script scanning and other parts of Web Shield. That is an unnecessarily broad method of exclusion, imo. I really think it would be nice to have a way to exclude a specific URL from URL blocking, and keep the URL blocking feature itself enabled for all other URLs :)
But then we would have people who excluded their favourite site and are confused why there is a popup telling them an infection was blocked. Didn't they just add it to exclusions?

And then, we have the following case: Assume we have a domain (blockeddomain.com). On that site there is an iframe with advertisement/game/stream: <iframe src="blockeddomain.com/iframe.html">

When you load this in your browser, this is what happens:
* blockeddomain.com is checked against the blacklist
* blockeddomain.com is resolved and the IP is checked against the blacklist
* when a 201 reply arrives, the response (source code) is scanned

These three steps are repeated for each resource, in this case we have 2 resources: blockeddomain.com and blockeddomain.com/iframe.html. You have put blockeddomain.com to exclusions, so a popup will not be triggered in the first step. But the source code contains an iframe to a blocked URL, therefore a HTML:Iframe-inf detection would pop up while scanning the source code.

You might say: "Iframes are not that common, who cares!", but this works the same way with JS as well. How do we solve this issue? Without "disabling webshield" on this whole domain, Avast would still keep popping up.

To sum it up: Domain blocking and content scanning is more closely related than one might think, and for me it makes sense for the users to be able to turn off both, but not to be able to turn off only one or the other. If it depended on me, I would get rid of the "Block malware URLs" checkbox altogether. :)

I hope I explained it a little, of course feel free to keep the discussion going, we might think of an improvement to the next version ;)!
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 03:46:29 PM
I don't mind the popups. And the blocking of the actually malicious parts of a site. Either on the domain or loaded from 3rd parties. But that still does not mean a URL should be blocked entirely if the URL blocking checkbox is left unchecked. This needs to be cleared up. I might not agree on how it will implement a blockade, but at least the description should match the behaviour.

I also think that when an iframe or script wants to load a malicious external site, that is the specific part that should be blocked (and generate a popup with the correct warning). Not the site that hosts the iframe or similar. So the result would be that the site is loaded but without the harmful elements. It might not function properly, but that is fine. The popup would clue you in that something was amiss, and what you get to see is the portion of the site that could be displayed safely.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Eddy on July 05, 2016, 03:56:11 PM
Blocking only the malicious parts on a site is like censoring a book.
I say show everything or nothing at all, like it is now.

Especially when a warning is shown while parts of the website can be viewed without a problem will confuse the users.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 03:59:44 PM
That analogy makes no sense. Do you hold the same view with regard to ad blockers or script/cookie/tracking blockers? Otherwise, you are being quite the hypocrite. Ad networks are a prime source of malicious 3rd party code.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Eddy on July 05, 2016, 04:04:47 PM
Block the entire site that is using a malicious ad(network)
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: HonzaZ on July 05, 2016, 04:07:18 PM
But that still does not mean a URL should be blocked entirely if the URL blocking checkbox is left unchecked. This needs to be cleared up.
I agree, and that is why I filled in a bug report ;)

I also think that when an iframe or script wants to load a malicious external site, that is the specific part that should be blocked (and generate a popup with the correct warning). Not the site that hosts the iframe or similar. So the result would be that the site is loaded but without the harmful elements. It might not function properly, but that is fine. The popup would clue you in that something was amiss, and what you get to see is the portion of the site that could be displayed safely.

This is not implementable (is that a word :)?)
When we see a malicious code, we must block the whole file that this is in - this is how AV works. We might detect a couple of bytes in an .exe file, and block it all, the very same way we might detect a couple of characters in a HTML page to block it all. Because what if there is another piece of malicious code that we do not detect? The safest option is to block the whole file. The detections we create would have to be MUCH more complicated if we wanted the rest of the site load safely. That would just not be viable.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 04:18:59 PM
The safest option is to not go online at all. But that is also not viable in today's world. So we need granularity in our online access. A way to properly choose what elements are shown and which aren't. And for the sake of practicality it needs to be automated. That is what a Web Shield is for, imho.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Pondus on July 05, 2016, 04:22:41 PM
Blocking only the malicious parts on a site is like censoring a book.
I say show everything or nothing at all, like it is now.

Especially when a warning is shown while parts of the website can be viewed without a problem will confuse the users.
This is what Malwarebytes does  >>  https://blog.malwarebytes.com/malwarebytes-news/2013/05/oh-the-sites-you-will-never-see/

Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 04:24:05 PM
Especially when a warning is shown while parts of the website can be viewed without a problem will confuse the users.

Not if this is an advanced option for advanced users. Keep the default wholesale blocking, but give more experienced users more control I say.

This is what Malwarebytes does  >>  https://blog.malwarebytes.com/malwarebytes-news/2013/05/oh-the-sites-you-will-never-see/

This approach is much more sensible imo :)
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: HonzaZ on July 05, 2016, 04:31:45 PM
But that is not possible, because even advanced users do not fully understand what which detection means and what danger it poses.
I say, block everything or nothing. If the detection is correct, it will be brought to the owner much faster (than if everyone just excludes it) and therefore will be healed much faster. If it is a false positive, it will be fixed much faster (because the users actually want us to fix it).
Creating advanced options would be very time consuming and prone to bugs (and bug here could mean an infected user!)
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: bob3160 on July 05, 2016, 04:41:47 PM
Especially when a warning is shown while parts of the website can be viewed without a problem will confuse the users.

Not if this is an advanced option for advanced users. Keep the default wholesale blocking, but give more experienced users more control I say.

This is what Malwarebytes does  >>  https://blog.malwarebytes.com/malwarebytes-news/2013/05/oh-the-sites-you-will-never-see/ (https://blog.malwarebytes.com/malwarebytes-news/2013/05/oh-the-sites-you-will-never-see/)

This approach is much more sensible imo :)
Totally disagree. Block it. As far as I'm concerned, no site is that important that you can't report what you think is a false positive and wait
for a correction if you happen to be right.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 04:42:07 PM
I don't think you should force everybody to the same lowest common denominator of protection granularity. The all or nothing approach. A a default, yes fine, but not as the only option. By forcing this, you also force people to disable more of their AV than necessary, to enable the access they feel they must have. So as an AV supplier you then choose to expose them to more harm (by their own doing) because they will eventually opt for an all or nothing approach themselves, and disable their AV altogether. Because they were not provided the option to selectively waive certain precautions.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 04:49:31 PM
Totally disagree. Block it. As far as I'm concerned, no site is that important that you can't report what you think is a false positive and wait
for a correction if you happen to be right.

It is NOT a false positive. In the case of for example an ad network serving malicious ads, that ad network can be blocked without blocking every site that would otherwise show ads from that network. This is what ad blockers do. You would not be OK with ad blocking software that completely blocks every site with an ad. i.e. pretty much everything, these days. An ad blocker is more selective. The code is stripped, and the rest of the site is shown. Script blockers do the same but with other code. And yes, sometimes that breaks part of the site. That is why it should be an advanced option only. But an option nonetheless. Imo.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: bob3160 on July 05, 2016, 04:55:01 PM
You're reading something into my reply that wasn't stated.
Bock it. If the user doesn't agree, report it and wait for a correction from Avast if the users assumption was correct.

Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Eddy on July 05, 2016, 04:58:56 PM
I fully agree with HonzaZ
Quote
I say, block everything or nothing.

To most "common users" it is already confusing enough.
How things work, what all options/settings mean/do and such.
Let's not make it more complicated for them.

Quote
Because they were not provided the option to selectively waive certain precautions.
You are forgetting a major thing.
Common users have no clue about security and how it is working.
The more options, the more confusing it is for them.
Let's not make it more complicated for them than it is already.

Billions of people are riding a bike, but not even 1 out of 10 million know how to setup/maintain/ride it in the best way.

You are making the big mistake to only see things from your point of view.
Not from the developers side, not from the common user site and not from the technical site.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 05:01:36 PM
One can't very well report a false positive if the malicious code is present. The hit is correct. It just should not result in a blocked site if blocking / filtering a small part of it would suffice.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 05:09:51 PM
You are forgetting a major thing.

I am most certainly not. Did you not read the part where I state it should be an advanced option only? Or did you choose to ignore it?

Even a browser such as Firefox lets you tinker with more advanced settings. You have to agree to a disclaimer, and you can continue. That disclaimer discourages novices. You can now stop this talk about 'common people' and what they can or cannot comprehend. I am not now, nor have I ever, advocating a change in default AV behaviour. The operative being: default. This implies more control is available for those who seek it.

Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Eddy on July 05, 2016, 05:16:35 PM
You are completely wrong.
If there is something malicious on a site and you only block that, who says that there is not a unknown malicious thing on that site ?
With your philosophy you are putting people/systems at risk.

It really does not make any sense at all to remove just certain words/sentences from a novel.
Websites are not like classified documents that you are allowed to view, but things are made black because you are not allowed to see parts of it.

Yes you do forget a major thing.
Did you not read the part where I said "The more options, the more confusing it is for them." ?
Or did you choose to ignore it ?
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 06:10:21 PM
It is up to the user to choose the level of risk they are willing to accept. There is no one size fits all solution that works for everybody. Risk is fine, as long as it is managed well. There is no confusion if advanced options are behind a disclaimer warning. Apparently this is too much for you to grasp. That's fine, just stay out of this thread then. As I've asked you before.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Eddy on July 05, 2016, 06:15:31 PM
Quote
Risk is fine, as long as it is managed well.
And that is exactly my point.
The common user is far from capable of doing so.
Quote
Apparently this is too much for you to grasp
Apparently you have
de bel horen rinkelen, maar weet niet waar de klepel hangt.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 06:22:01 PM
You are going out of your way to fail to make the destinction between novices and advanced users. Either your attitude is deliberate, or you are not capable of normal comprehension. Both are not helpful. So, please, just go away. Your musings are not wanted.

I am not concerned with the policy regarding common users. Keep things as the are for them.  Power users need more control.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: bob3160 on July 05, 2016, 06:42:50 PM
You are going out of your way to fail to make the destinction between novices and advanced users. Either your attitude is deliberate, or you are not capable of normal comprehension. Both are not helpful. So, please, just go away. Your musings are not wanted.

I am not concerned with the policy regarding common users. Keep things as the are for them.  Power users need more control.
If you make the option available, every one becomes an advanced user if what they want to reach is blocked. :)



Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 06:55:12 PM
Well, then you can point to the disclaimer and remind them they were cautioned sternly and they explicitly accepted.

As it is, that is still better than the current situation which has people disabling their AV entirely (or far more than necessary, like all of Web Shield) and proceed.

The option wouldn't have to be in your face. Hide it from direct view. Or make it a command line parameter for the executable. Whatever. Live dangerously mode. Sounds nice :D

Besides .. in its current form Avast has all kinds of options available to disable components, and thus putting users at risk. If this is so problematic for most users, one wonders why they are given those choices. I honestly don't think that argument is valid.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: bob3160 on July 05, 2016, 07:00:08 PM
Well, then you can point to the disclaimer and remind them they were cautioned and they explicitly accepted.

As it is, that is still beter than the current situation which has people disabling their AV entirely (or far more than necessary, like all of Web Shield) and proceed.

The option wouldn't have to be in your face. Hide it from direct view. Or make it a command line parameter for the executable. Whatever. Expert mode. Sounds nice.
It is only better in your eyes. I'd personally like to see it made even more restrictive.
Better safe than sorry even if it may be a little inconvenient.


Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Rundvleeskroket on July 05, 2016, 07:12:06 PM
I agree the default should be very conservative. I don't however agree that it should be the only option.

But to get back to the heart of the matter: the least broad exclusion in URL blocking is made at Web Shield level, correct? There is no way to exclude a URL from just the URL blocklist and have Web Shield evaluate said URL when visited?

Which would possibly result in it still being blocked (partially?) if something nefarious is found, but not just because the URL is on a list.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: lukor on July 06, 2016, 02:06:08 PM
Hi, you are right. The option should be removed and we will most probaby do it for the next release.

This option is in the settings dialog since the time where Network Shield (reposible for URL blocking) and WebShield (responsible for the content checks) were merged. Back then it made sense to distinguish between these two detection methods, basically because they were also implemented differently using different system APIs (WebShield:proxy, Network Shield:network filter).

Today, the detections already include many more checks, they watch for HTTP referrers, URLs extracted from previously seen malware samples, they check the content-type and match it against the actual content, and finaly also check for the content, in some cases for parts of the content, such as begining or end, etc. These things are included in several calls from the networking stacks into the detection engine.

It is no longer easy to separate one of the methods (URL blocking) from the whole chain - other that let it happen and then specifically ignore virus results containing Url:Mal2.
While this could be implemented, I wouldn't vote for it -- there are many checks in the process of the detection that make the whole result -- either turn it off completely by using the URL Exclusions, or let them all happen.

Thanks again for pointing to us that the checkbox is now obsolete.

Lukas.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: REDACTED on January 04, 2017, 09:57:07 AM
I know this is an old thread, but I wanted to add that this issue is still happening. The only way for me to access blocked sites is to completely disable avast and kill the process with the task manager so I won't be renewing my license, but will check out this thread in the future to see if it is ever fixed.

For me it's a much larger issue because sites that I am required to access for school are blocked, as well as some shopping sites. (I can't submit homework if the university subdomain is blocked, and there's no way that blocking such sites should even happen!) Multiple people commented that we should all blindly follow the advice of the program to tell us what is and is not safe, but more often than not I am blocked from accessing sites I absolutely know to be safe. People would be livid if their ISP decided what they should and should not do on the internet, even if the ISP said it was just blocking content they know to be dangerous. In one case the FBI pulled bad websites, but made a mistake and pulled entire swath of safe family friendly websites and it was a huge issue. Ultimately, the end user - or at least the administrator account - should have the right to white list sites.

Besides that, it doesn't do a very good job of blocking sites. Avast says:  "this site could have harmed your computer" , but does not actually block the site from appearing or it's code from loading in the background. If it was pushing malicious code it's unlikely that avast would do anything about that. All it does is block the user from interacting with the site which is stupid. At least when google does it they fully redirect away which prevents malicious code from running at all and then gives the user the choice to go through.

The system would be more robust if I could both white list and black list sites. I have seen numerous unsafe websites I wish could not come up via popup, but at not at all blocked. If I could specify the url with a wildcard * to indicate everything in a range should be blocked that would stop that issue and make my computer safer without depending on avast to decide it's dangerous before blocking. Likewise, if I need to access a website for work or school I should not have to completely disable an antivirus because avast has decided on it's own that it wants to control my browsing habits. I should be able to tell it to let me access a website regardless of what it is programmed to think about that.

EDIT: Killing the avast process didn't help either, so I was forced to completely uninstall. It apparently injected code into the browser which I consider to be malicious behavior. I won't be re-installing. After uninstalling, and without having to restart the computer or even restart the browser, I regained control over my computer and was able to access blocked sites again.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: HonzaZ on January 04, 2017, 10:19:29 AM
...because sites that I am required to access for school are blocked, as well as some shopping sites. (I can't submit homework if the university subdomain is blocked, and there's no way that blocking such sites should even happen!)

If some university/shopping domains are blocked and you think this is incorrect, you can submit a false positive. User-based whitelisting is very dangerous and I do not recommend it. If it is a false positive, it should be unblocked by us, and if it is a true positive, it shouldn't be visited at all.

Avast says:  "this site could have harmed your computer" , but does not actually block the site from appearing or it's code from loading in the background.

Code from blocked URLs is NOT loaded if you have Avast on. Are you perhaps talking about our browser plugin (Avast Online Security)? AOS does ask asynchronously, because if it asked about each URL before loading its code (and there can easily be hundreds of resources when opening one URL), browsing experience would be very slow.

I have seen numerous unsafe websites I wish could not come up via popup, but at not at all blocked.

If you think some URLs are unsafe/malicious and not detected by Avast, you should report them.

We of course do not block URLs willy-nilly; we only block a URL if there is a strong evidence of something malicious happening. Of course false positives may happen, but we do our best to keep the number (and impact) of them as low as possible.
Title: Re: URLs are blocked even with 'Block malware URLs' disabled?
Post by: Eddy on January 04, 2017, 11:30:03 AM
Quote
I can't submit homework if the university subdomain is blocked, and there's no way that blocking such sites should even happen!
So, you rather visit a malicious domain/server and loose days/weeks or even months of work because the malware destroys your work ?
Quote
Multiple people commented that we should all blindly follow the advice of the program to tell us what is and is not safe, but more often than not I am blocked from accessing sites I absolutely know to be safe.
And how do you know they are safe ?
Are you a programmer ?
Are you highly skilled in computer networks, traffic analyzing and such ?

Giving users the option to white list domains/IP's is a really bad idea.
A domain/IP that is safe right now can be infected the next second.
Do you really think people will check a domain/IP before they visit it if it is still safe and still should be white listed or not ?
No, they just visit it because it was safe in the past.
Giving users the option to white list domains/IP's (and other things), is like telling them not to use any security at all.

Sure, it can be annoying if you want to visit a domain/IP and it is wrongly blacklisted.
But as they say "Better safe than sorry".