Author Topic: Avast and Jumpshot  (Read 20393 times)

0 Members and 1 Guest are viewing this topic.

Offline Vlk

  • Global Moderator
  • Serious Graphoman
  • **
  • Posts: 11663
  • Please don't send me IM's. Email only. Thx.
    • ALWIL Software
Avast and Jumpshot
« on: May 30, 2015, 05:50:28 AM »
Hi all,
We recently announced our investment in a marketing analytics platform called Jumpshot.

https://blog.avast.com/2015/05/29/avast-data-drives-new-analytics-engine/

It’s a very exciting investment for us for two reasons. First, because they create really interesting and unique market insights. Go to jumpshot.com if you want to see some of it – there’s a free trial that anyone can sign up for.  And second, because they do it using a proprietary algorithm that strips all the PII out of the data they use. It’s the only data stripping tool we’ve seen that does that successfully.

Here’s how Jumpshot works:

Data is collected on computers and Android devices through the browser.  Each record contains a set of fields that help Jumpshot algorithms assign the clickstream data appropriately. These fields include:
-   Installation identifiers (proprietary identifiers that do not contain any PII)
-   URL being visited
-   Referral URL (if this exists)
-   Window identifier
-   Tab identifier
-   Additional fields for processing purposes

In reality, the information Avast passes on to Jumpshot looks like this:
-   Identifier: 00002437-705b-4bc6-b062-54b7ea511c93
-   URL being visited: http://www.cnn.com/US/?hpt=sitenav
-   Referral URL: http://edition.cnn.com/
-   Window identifier: 3
-   Tab identifier: 42

Prior to processing, all records are automatically scanned for PII, and all PII parameter values are removed from the raw data. To strip PII, Jumpshot uses a proprietary algorithm that calculates multiple statistical features for parameters on all known websites. Based on these statistical values, only parameters that are proven not to be PII are whitelisted and their values are kept. All parameter values that are not whitelisted are stripped in the process, which leaves those parameter values overwritten by the word “REMOVED”. The stripping of PII is done on the Avast premises in Prague, to ensure that the PII never leaves our hands.

Let’s have a look at an example. With a shopping site like Amazon, the URL before stripping contains some PII:

Code: [Select]
https://www.amazon.com/gp/buy/addressselect/handlers/edit-address.html?ie=UTF8&addressID=jirptvmsnlp&addressIdToBeDeleted=&enableDeliveryPreferences=1&from=&isBillingAddress=&numberOfDistinctItems=1&showBackBar=0&skipFooter=0&skipHeader=0&hasWorkingJavascript=1
The algorithm automatically replaces the PII with the word REMOVED in order to protect our users’ privacy, like this:

Code: [Select]
https://www.amazon.com/gp/buy/addressselect/handlers/edit-address.html?ie=UTF8&addressID=REMOVED&addressIdToBeDeleted=&enableDeliveryPreferences=1&from=&isBillingAddress=&numberOfDistinctItems=1&showBackBar=0&skipFooter=0&skipHeader=0&hasWorkingJavascript=1
The stripping processes doesn’t end here, though. Next is aggregation. Data processing is performed once a day in a cascade of data transforming and aggregating map-reduce jobs. Aggregations are typically applied on a per-domain (website) and per-URL (web page) basis.  To further protect our users‘ privacy, we only accept websites where we can observe at least 20 users. This ensures that no reverse engineering is possible on the aggregated data – there’s nothing that can lead back to a specific user. All aggregated data is then stored in an RDBMS (currently PostreSQL) database on a per-domain and keyword basis.

These aggregated results are the only thing that Avast makes available to Jumpshot customers and end users.

Now, the key is not only what information is actually collected and how it is processed, but also how transparent we are about the whole process. Avast is committed to protecting its customers on all fronts, which is why we inform our users, even beyond our EULA and Privacy policy, that their browsing information will be collected but stripped of personally identifiable information and will be used to help us better understand new and interesting trends. We actually tried to make this very, very explicit, and that’s why we have an extra step in the Avast installer which informs our users in a very straightforward way about what we’re doing.

Users can remove themselves from the system in two ways – by unchecking the “Statistics“ box in the Avast browser add-on settings (see attached picture), or by sending an email to customer support requesting to have their information deleted. If a user wants to be deleted, the system automatically blacklists their user ID from all data transforming activities.

By focusing on protecting our users, we ensure that the data Jumpshot customers get is accurate because the larger the data pool, the more statistically valid the data customers get to work with.  So Jumpshot has a vested interest in protecting our users' privacy.


If you have any questions, please don't hesitate to ask.


Thanks,
Vlk
If at first you don't succeed, then skydiving's not for you.

Offline essexboy

  • Malware removal instructor
  • Avast Überevangelist
  • Probably Bot
  • *****
  • Posts: 40641
  • Dragons by Sasha
    • Malware fixes
Re: Avast and Jumpshot
« Reply #1 on: May 30, 2015, 12:29:42 PM »
Quote
Currently we do not make any money from this relationship but it is an experiment as to whether we can fund our security products indirectly instead of nagging our users to upgrade.
As most people are aware, most all products we use every day—Chrome, Facebook, Firefox, WhatsApp, Gmail, etc.—are indirectly funded by advertisements.
In most cases though, the products directly examine what users are doing and provide them targeted advertisements.
Although we suspect some security companies are doing this, we do not believe it is the proper approach.
Instead, we think that this anonymized, aggregated approach is much better to maintain the trust relationship that we think is so important between us and you, our loyal users.
So does this imply that if it works the ad popups will become history ?

Online bob3160

  • Avast Überevangelist
  • Probably Bot
  • *****
  • Posts: 36291
  • 57 Years of Happiness
    • bob3160 Protecting Yourself, Your Computer and, Your Identity
Re: Avast and Jumpshot
« Reply #2 on: May 30, 2015, 03:16:42 PM »
Quote
Currently we do not make any money from this relationship but it is an experiment as to whether we can fund our security products indirectly instead of nagging our users to upgrade.
As most people are aware, most all products we use every day—Chrome, Facebook, Firefox, WhatsApp, Gmail, etc.—are indirectly funded by advertisements.
In most cases though, the products directly examine what users are doing and provide them targeted advertisements.
Although we suspect some security companies are doing this, we do not believe it is the proper approach.
Instead, we think that this anonymized, aggregated approach is much better to maintain the trust relationship that we think is so important between us and you, our loyal users.
So does this imply that if it works the ad popups will become history ?
Thanks you, it was the exact question I was going to ask.
Now that everyone knows the information is being collected, will that impact the amount of data you receive ???
I'm sure more people will opt out now that they are aware that Avast is harvesting such information and will eventually use it as a money making endeavor.
Is Avast the only Security company doing this ???
Free avast! Security Seminar: http://www.authorstream.com/Presentation/bob3160-1425909-protecting-yourself/    -  Important: http://www.organdonor.gov/ -- My Blog: http://bob3160.blogspot.com/ - Win 10 Pro v1703 64bit, 8 Gig Ram, AvastFree 17.6.2307, WinPatrol, Unchecky How to Successfully Install Avast http://goo.gl/VLXde

Offline essexboy

  • Malware removal instructor
  • Avast Überevangelist
  • Probably Bot
  • *****
  • Posts: 40641
  • Dragons by Sasha
    • Malware fixes
Re: Avast and Jumpshot
« Reply #3 on: May 30, 2015, 03:55:29 PM »
Quote
Now that everyone knows the information is being collected, will that impact the amount of data you receive ???
I'm sure more people will opt out now that they are aware that Avast is harvesting such information and will eventually use it as a money making endeavor.
Is Avast the only Security company doing this ???
If I am reading this correctly then then it is just the same as counting the number of people going in and out of a train system, where you see a lot of people going to and from and you can see where they are going but you do not have the foggiest as to who they are.  Total invisibility as far as you are concerned.  If this then reduces or gets rid of the ad nags then everyone comes out on top   

Online DavidR

  • Avast Überevangelist
  • Certainly Bot
  • *****
  • Posts: 79176
  • No support PMs thanks
Re: Avast and Jumpshot
« Reply #4 on: May 30, 2015, 03:58:43 PM »
Not sure how that would differ from the old Save Surf that was in the AOS browser add-on. Many people disliked that feature thinking that a security base product has no right to be offering this type of thing.

Personally I would probably opt-out of this as I actively block ads (targeted or otherwise) in my browser.

Even with the assurances in the information that personal data will be removed, I still see this as an area that the Privacy naysayers will jump on (excuse the cheap jump shot pun).
Core2Duo E8300/ 4GB Ram/ WinXP ProSP3/ avast! free 17.8.2318/ Outpost Firewall Pro9.3/ Firefox 52.4.0 ESR, uBlock Origin, RequestPolicy/ MailWasher Pro7.8.0/ DropMyRights/ WinPatrol+/ Drive Image 7.1/ SnagIt 10.0/ avast! mobile security

Online bob3160

  • Avast Überevangelist
  • Probably Bot
  • *****
  • Posts: 36291
  • 57 Years of Happiness
    • bob3160 Protecting Yourself, Your Computer and, Your Identity
Re: Avast and Jumpshot
« Reply #5 on: May 30, 2015, 03:59:03 PM »
Quote
Now that everyone knows the information is being collected, will that impact the amount of data you receive ???
I'm sure more people will opt out now that they are aware that Avast is harvesting such information and will eventually use it as a money making endeavor.
Is Avast the only Security company doing this ???
If I am reading this correctly then then it is just the same as counting the number of people going in and out of a train system, where you see a lot of people going to and from and you can see where they are going but you do not have the foggiest as to who they are.  Total invisibility as far as you are concerned.  If this then reduces or gets rid of the ad nags then everyone comes out on top
If that's the end result, then Avast can get back to growing and protecting since it will eliminate one of the biggest complaints on this forum.
Free avast! Security Seminar: http://www.authorstream.com/Presentation/bob3160-1425909-protecting-yourself/    -  Important: http://www.organdonor.gov/ -- My Blog: http://bob3160.blogspot.com/ - Win 10 Pro v1703 64bit, 8 Gig Ram, AvastFree 17.6.2307, WinPatrol, Unchecky How to Successfully Install Avast http://goo.gl/VLXde

Offline essexboy

  • Malware removal instructor
  • Avast Überevangelist
  • Probably Bot
  • *****
  • Posts: 40641
  • Dragons by Sasha
    • Malware fixes
Re: Avast and Jumpshot
« Reply #6 on: May 30, 2015, 04:07:18 PM »
Quote
Personally I would probably opt-out of this as I actively block ads (targeted or otherwise) in my browser.

Even with the assurances in the information that personal data will be removed, I still see this as an area that the Privacy naysayers will jump on (excuse the cheap jump shot pun).


My reading of this is that it is similar to google analytics and has no relationship to ads but just seeing who goes where from where

Online bob3160

  • Avast Überevangelist
  • Probably Bot
  • *****
  • Posts: 36291
  • 57 Years of Happiness
    • bob3160 Protecting Yourself, Your Computer and, Your Identity
Re: Avast and Jumpshot
« Reply #7 on: May 30, 2015, 04:30:47 PM »
@ David,
I've long since realized that it's become too small a world and that hiding in it has become impossible.
I've also realized quite a while ago that Privacy as we knew it many years ago no longer exists.
I don't block Avast or Google or Bing or Yahoo. I do block ads with uBlock.
I do realize that nothing in life is free and one way or another all things need to be paid for.
If this technology is a means of allowing Avast to earn the income they need to protect us and grow
their technology without the Popups we now complain about, then by all means, go for it.
Free avast! Security Seminar: http://www.authorstream.com/Presentation/bob3160-1425909-protecting-yourself/    -  Important: http://www.organdonor.gov/ -- My Blog: http://bob3160.blogspot.com/ - Win 10 Pro v1703 64bit, 8 Gig Ram, AvastFree 17.6.2307, WinPatrol, Unchecky How to Successfully Install Avast http://goo.gl/VLXde

Online DavidR

  • Avast Überevangelist
  • Certainly Bot
  • *****
  • Posts: 79176
  • No support PMs thanks
Re: Avast and Jumpshot
« Reply #8 on: May 30, 2015, 04:32:16 PM »
Quote
Personally I would probably opt-out of this as I actively block ads (targeted or otherwise) in my browser.

Even with the assurances in the information that personal data will be removed, I still see this as an area that the Privacy naysayers will jump on (excuse the cheap jump shot pun).


My reading of this is that it is similar to google analytics and has no relationship to ads but just seeing who goes where from where

Yes, but I also block google-analytics or rather don't allow it in NoScript and RequestPolicy add-ons.
Core2Duo E8300/ 4GB Ram/ WinXP ProSP3/ avast! free 17.8.2318/ Outpost Firewall Pro9.3/ Firefox 52.4.0 ESR, uBlock Origin, RequestPolicy/ MailWasher Pro7.8.0/ DropMyRights/ WinPatrol+/ Drive Image 7.1/ SnagIt 10.0/ avast! mobile security

Offline CraigB

  • Avast Überevangelist
  • Serious Graphoman
  • *****
  • Posts: 10585
Re: Avast and Jumpshot
« Reply #9 on: May 30, 2015, 05:38:28 PM »
Be interesting to know if this will be the end of the advertising popups but I don't think it will TBH, analytics will provide Avast with browsing habit information ( data mining ) while the popup Ad's are for the other side of the marketing division which is to bring in $

« Last Edit: May 30, 2015, 05:45:08 PM by CraigB »

Offline Secondmineboy

  • Avast Evangelist
  • Massive Poster
  • ***
  • Posts: 3593
Re: Avast and Jumpshot
« Reply #10 on: May 31, 2015, 03:22:12 PM »
This may answer your question about the end of Popups:

Quote
We have always strived to have an honest relationship with our users, and we will continue to do so. Currently we do not make any money from this relationship but it is an experiment as to whether we can fund our security products indirectly instead of nagging our users to upgrade. As most people are aware, most all products we use every day—Chrome, Facebook, Firefox, WhatsApp, Gmail, etc.—are indirectly funded by advertisements. In most cases though, the products directly examine what users are doing and provide them targeted advertisements. Although we suspect some security companies are doing this, we do not believe it is the proper approach. Instead, we think that this anonymized, aggregated approach is much better to maintain the trust relationship that we think is so important between us and you, our loyal users.

Vincent Steckler
Emsisoft Internet Security, MCShield
SAMSUNG Galaxy S7 Edge, Android 7.0, Avast Mobile Security

https://forum.thepcsecuritychannel.com/

Offline Lisandro

  • Avast team
  • Certainly Bot
  • *
  • Posts: 67269
Re: Avast and Jumpshot
« Reply #11 on: June 04, 2015, 07:55:09 PM »
@Vlk:

Vince said that around 100 million users have joined (kept) in the program. We're 230 million users out there.
I suppose (just my feeling) that the majority of the ones who dropped did because it requires a browser addon. Others should have opt out.
Is it technology possible to do the same job using Web Shield instead of the browser addon?
The best things in life are free.

Offline lucD

  • Full Member
  • ***
  • Posts: 106
Re: Avast and Jumpshot
« Reply #12 on: June 07, 2015, 09:48:15 AM »
Sorry to be contrarian but I think I'd prefer to receive the popups. They are a bit annoying but they're surely less intrusive that my entire browsing history being sucked up on an ongoing basis. I'm not entirely comfortable even with anonymised data transfers and I'm certainly not convinced that any algorithm is fool-proof ... and if that algorithm is proprietary then we'll never really know for sure. That said ... I do appreciate that opt-out is possible (I shall do so by continuing not to install AOS) and I hope that it stays that way.

Offline Lisandro

  • Avast team
  • Certainly Bot
  • *
  • Posts: 67269
Re: Avast and Jumpshot
« Reply #13 on: June 07, 2015, 09:21:39 PM »
I'm not entirely comfortable even with anonymised data transfers and I'm certainly not convinced that any algorithm is fool-proof ... and if that algorithm is proprietary then we'll never really know for sure.
All the data is processed in Avast servers. So, we DO certainly know for sure because any data delivered could be analysed by Avast.
The best things in life are free.

Offline lucD

  • Full Member
  • ***
  • Posts: 106
Re: Avast and Jumpshot
« Reply #14 on: June 09, 2015, 02:14:12 PM »
I have a lot of trust in Avast, the anti-virus developer and its core program (I don't install the bloat), its has protected my pc for years and I pay Avast all respect and gratitude. But Avast has another side, it's the side of the company that introduced Safe Price (default on), a browser clean up product which changes users' default search provider to its own sponsor and a software updater that directs users to a site that installs adware with the updates. Avast has every right (and need!) to try to earn money but if it follows paths that undermine trust then people may be less inclined to feel confidence in its screening of browsing data when Avast proposes to suck it up in its entirety.