AVG produces ghost traffic with linkscanner - Webmasters are not amused!

[polonus]

Hi malware fighters,

Ghost traffic produced by AVG's incorporated Linkscanner can be a pain in the neck for Webmasters, because they can no longer rely on their traffic stats. Read here: http://www.theregister.co.uk/2008/06/13/avg_scanner_skews_web_traffic_numbers/

pol

[DavidR]

This is the proverbial sledge hammer to crack a nut, and not only is it a waste of bandwidth for a number of web sites, but it would also slow search returns if it is going to scan all links on the first page of hits.

I have mine set to return 50 hits per page and out of that 50 hits I would say I would probably only visit 3 or 4 so the scans on the other 46 are wasted.

Boy am I glad that avast doesn't do this but takes the better step of only scanning pages that you actually visit in the web shield proxy before you actually display the page contents.

[igor]

Yeah, I always thought this LinkScanner "technology" should be forbidden/blocked/banned/punished, cause it's just flooding the Internet.
What the hell were they thinking?

[polonus]

Hi Igor,

What do you think of the third parties they sell their linkscanner results to? Ever considered this aspect of the matter? The user thinks of security, and the seller of security thinks of....,

pol

[Lisandro]

And this is more than just an inconvenience. After all, sites live and die by their traffic numbers. And net resources aren't free.
That could destroy web analytics as we know it.

For the ones that thought that LinkScanner will be the best into AVG antivirus.

Roger Thompson earlier this week, he was unaware of these issues.

Is it a way to conduct a security application? I don't know, I'm not aware... Shame on Grisoft.

Polonus, thanks for pointing it up.

[polonus]

Hi Tech,

Just another important aspect:
--

This user-agent is used by Exploit Prevention Labs LinkScanner:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)

It pre-fetches HTML and JavaScript from searches on Google/Yahoo/MSN done by humans - the IP in your logs will be that of the user who has it installed and is searching on your keywords.

The software was recently acquired by Grisoft AVG but is still available for download on CNET, and if you block it you will be discouraging visitors and filling your logs with 403s for no good reason.

--

This user-agent is used by Grisoft AVG 8.0 LinkScanner:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813)

As above, a bandwidth-wasting (and easily fooled) pre-fetcher best dealt with by cloaking minimal content (example given by jdMorgan in the AVG thread). When I had it blocked I lost a lot of traffic and on one of my tests it produced an impressive 120 (one hundred and twenty) 403s in 12 seconds - without me even visiting the site.

--

This user-agent is used by Trend Micro Internet Security and TrendProtect:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

Unlike the others it does not pre-fetch SERPs in real time, but can be triggered from the "website authentication" feature of the Internet Security Pro package on demand, and also appears to be doing some general spidering for the Trend Micro "rating server".

Sometimes it comes from the Trend Micro IP range (66.180.80.0 - 66.180.95.255) but more often it comes from the Japan Network Information Center with various IPs in the 150.70.84.xx range - so you can probably expect your site to be classed as "Suspicious" (the outrageous term they use for unknown sites) if you have APNIC blocked.

--

This user-agent is used by the DrWeb plugin for Explorer and FireFox:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Maxthon)

This one pre-fetches HTML and JavaScript, but only on a specific request from the user, and in my tests it always came from 81.176.67.173 (the DrWeb server) as advertised - more reasonable than the others, but like them working for real oxygen-breathing humans.

--

I have not been able to identify where McAfee SiteAdvisor gets its information, but I do have an amusing screenshot of the related Yahoo SearchScan flagging google.com as a purveyor of "Dangerous Downloads".

On all my sites it says "We've tested millions of sites but haven't tested this one yet" - and unless McAfee scans the entire web as frequently as GoogleBot it is presumably worthless.

--

None of the other 18 anti-virus packages I tested currently interfere with search results, but it may only be a matter of time, and if you don't appease them and get flagged as "clean" you may find that your ranking is considered irrelevant.

"Paranoia strikes deep - into your SERPS it will creep" 

Conclusion: So DrWeb's AV linkscanner plug-in etc. can later suffer from the measures taken because of Grisoft's irresponsible behavior,

polonus

 

[Lisandro]

No Grisoft products on my machines...

[sopadeajo]

Iobit is not AVG?

Cause AdvancedWindowsCare V2, that you recommended, Tech, is IOBIT and i am worried with what they really do with treir Segurity Defense (preventing Spyware from being installed in your PC).

Is it really Safe for your PC?.
How can they prevent spyware from installing? What do they have to modify in your PC to do so?

[MikeBCda]

Between their historically dismal customer support, totally abandoning all their free products, and now this latest b.s., I think it's a fair guess that Grisoft is committing corporate suicide. 

[bunk]

I am also an ex-AVG user, after their creation of AVG 8 is when I moved up to avast!

[Lisandro]

I am also an ex-AVG user, after their creation of AVG 8 is when I moved up to avast!

Welcome... you're not alone... tons of users made this movement.