False positive URL:Mal URL:Blaclist

[IvanMiha]

Hello Avast Team,

I am working for a streaming service provider and from several days ago we are receiving customer complaints about Avast blocking access to our CDN, effectively blocking our users to use our service on Windows.
So far, from customer reports these two URL were detected as URL:Mal and/or URL:Blaclist:
sbb-bg-ne-s1-3.ug-be.cdn.united.cloud
sbb-bg-ku-r1-2.ug-be.cdn.united.cloud
I have checked:
https://www.virustotal.com/pt/url/223a493c2068170719f5f4589812a63e0cec80ebbf3c81fad44b99ffb4e40906/analysis/1555602057/
https://sitecheck.sucuri.net/results/sbb-bg-ne-s1-3.ug-be.cdn.united.cloud
https://quttera.com/detailed_report/sbb-bg-ne-s1-3.ug-be.cdn.united.cloud
http://urlquery.net/report/579d0894-8eae-4451-853e-644f7daaae06

Please let me know why these URL's got blacklisted by Avast, since we are not seeing anything suspicious from our side.

Thanks,
Ivan

[Asyn]

You can report a suspected FP (File/Website) here: https://www.avast.com/false-positive-file-form.php

[polonus]

Witam IvanMiha

Re:  The site returned an HTTP status code other than 200 (OK),
consider: https://observatory.mozilla.org/analyze/sbb-bg-ne-s1-3.ug-be.cdn.united.cloud
5 security related reconmmendations: https://webhint.io/scanner/03337b69-d7f7-4422-bf38-6e29ee8faab2#category-Security

At the crux of the problem are the various 404 not found errors we see here: -https://www.shodan.io/host/5.22.186.140
HTTP Connect    The remote server returned an error: (404) Not Found. (-http://sbb-bg-ne-s1-3.ug-be.cdn.united.cloud)
also a 404 Not Found on -http://5.22.186.140
So the issue could be completely unrelated to avast flagging.

Wait for an avast team member to give a final verdict, as we here are just volunteers with relevant knowledge,
but only avast team members can come and unblock. But this could be a DNS error returned by -dns3.sbb.rs,
as we get "Unable to connect to the remote server (-http://dns3.sbb.rs)".

Consider also: https://toolbar.netcraft.com/site_report?url=http%3A%2F%2Fsbb-bg-ne-s1-3.ug-be.cdn.united.cloud

pozdravi,

polonus (volunteer 3rd party cold reconnaissance website security analyst and website error-hunter)

[sibin.arsenijevic]

Hello polonus and Asyn,

thank you for your prompt answers and your help!

I just want to clarify few more things:

- That domain is not serving a website, it is a IPTV streaming server that is part of a larger CDN, it is not supposed to serve anything on it's bare (sub)domain (hence 404). There are other endpoints on that domain that will serve the video content (and use HTTPS) for user and apps should never request bare domain name. 404 on bare domain is "by design".
- I am not comfortable discussing security on an public forum but if you need further clarification as to why some things are done the way they are you are free to DM or (better yet) email me, i'll try to give you as much information as possible.

We've already reported some subdomains on url that you provided, but since we have hundreds of possible subdomains we can't manually whitelist each and every of them and we don't want to send you a lot of false reports from our side through some automated reporter. Is it possible to report *.domain.name there?

Our CDN nodes all have these domain names in common *.ug.cdn.united.cloud, *.ug-be.cdn.united.cloud and *.ug-af31.cdn.united.cloud, maybe that would be the way to "whitelist" them all?

Whitelisting a node at a time is hardly maintainable as we are adding new nodes all the time and we don't want to send you a whitelist request before the domain is blacklisted and by that time our users are already affected and we are getting user complaints. And requesting that users create rules to whitelist these domains on their own machines is not exactly good customer experience

We are willing to work with you in order to resolve these issues in a maintainable way.

Some more context - we are IPTV provider with content delivery network consisting of hundreds of nodes and hundreds of thousands of users on multitude of devices.

Thank you again for your support!

Best regards,
Sibin

[polonus]

Hi Sibin,

Here you can see that the blocking is not caused by avast, but caused by DNSBL blocking:
https://www.robtex.com/dns-lookup/sbb-bg-ne-s1-3.ug-be.cdn.united.cloud

So you should take this up with clueless dot org in the first place,
and in case of an FP they should unblock 

Still get the non-founds here: https://www.shodan.io/host/5.22.186.140
probably while http does not automattically redirects to https.
Your stratum 2.3 service 123 however seems OK. (on a zero day read -http://mvfjfugdwgc5uwho.onion )

Also read at serverfault: https://serverfault.com/questions/493121/ubuntu-open-udp-port-123

polonus

[sibin.arsenijevic]

Thank you again for your help!

After closer inspection of clueless urls we have found that our domains are not blacklisted but rather ".cloud" TLD.

When we request blacklist information for domain "sbb-bg-ne-s1-3.ug-be.cdn.united.cloud" on rfc-clueless.org (http://rfc-clueless.org/lookup/sbb-bg-ne-s1-3.ug-be.cdn.united.cloud) we get that

sbb-bg-ne-s1-3.ug-be.cdn.united.cloud is INDIRECTLY listed in an RFC2 list. An ancestor of sbb-bg-ne-s1-3.ug-be.cdn.united.cloud is causing the domain to be listed. You cannot directly remove sbb-bg-ne-s1-3.ug-be.cdn.united.cloud as it is not directly listed.". After following the ancestor url we get that "cloud IS listed in an RFC2 list.

When we follow ancestor URL we get that

cloud IS listed in an RFC2 list.

Also, their documentation states that:

Some entire TLDs lack a public whois server or have a non-standard whois server (including servers which do not return contact information). These domains are listed in the whois2 zone. Individual hosts cannot be removed from whois2...

(http://rfc-clueless.org/pages/listing_policy-whois)

However, on the same page they are discouraging blacklisting of subdomains if TLD is blacklisted as subdomains don't have control over registrar:

We discourage users from blocking eMail solely based on inclusion in the RFC2 lists, and would like to strongly reiterate that in the case of hosts listed only in WHOIS2, as registrar policy is outside their control.

Anyway, we have already sent an email notifying our registrar about this issue even though it is not necessarily their fault.

Regarding the HTTPS redirection and 404 errors, we can make an empty index.html page that will be served after being redirected from http to https, but IMHO this is just a waste of resources as we will be wasting cpu cycles for https termination on an empty webpage. To me, it is very strange that a page should be blacklisted because root (sub)domain is not serving https webpage and clients are not requesting that root subdomain but rather some other path like <subdomain.domain.tld>/stream for example.

Is it possible that we get a response from Avast with root cause of blacklisting so we can target specific reason?

Thank you again and again, you've been very helpful with your assistance!

[polonus]

Well, you have to wait until to-morrow, before an avast team member will react.

I am still of the opinion that the errors found stem from a wrong configuration at the cloud server,
so you have to start from there.

Here it has been given as "above board", proxy/green, no bad web bot/green, no spam/green, no attack source/green.
Re: https://www.proxydocker.com/en/iplookup/5.22.186.1  response: ERR_CONNECTION_REFUSED
& also here: https://www.proxydocker.com/en/iplookup/sbb.rs 
where I get not secure connection: Http/1.1 Service Unavailable...
Also consider these results: https://dnscheck.pingdom.com/#5a9186d122400000

polonus

[HonzaZ]

The two URLs were unblocked on the 15th and 18th.