Author Topic: The consequence of poor testing  (Read 15054 times)

0 Members and 1 Guest are viewing this topic.

ady4um

  • Guest
Re: The consequence of poor testing
« Reply #15 on: February 26, 2012, 09:13:12 AM »
This is not a testing fail, but a procedure fail. In version updates should be done automatically, but new versions should always be clean installed.

I'd like to point out that many users didn't have problems while updating, and others didn't have problems while installing anew. But, the opposite is also true; many users had problems updating, and many users had problems while installing clean. So saying this is only a procedure fail... Well, it may, but avast announcement explicitly said that a program update is possible, and there was not explicit recommendation to install clean. Many users received a tray notification "click to update" and that was it.

Some problems are unavoidable beforehand. But, if the clean install procedure would be enough to avoid problems, then many users shouldn't be having problems like a repetitive final screen as part of the installation procedure on "every" reboot (or at least, after several reboots). There are more examples; problems after a clean install.

This example of the repetitive screen after reboot may be a "little bug", and I'm not sure how much additional test would had been necessary so to avoid it in the final release. Yet, several "little bugs" (not only in the program itself) caused some amount of "veteran" users to recognized that this was not the best program update in the long history of Avast good quality.

Offline AntiVirusASeT

  • Poster
  • *
  • Posts: 462
Re: The consequence of poor testing
« Reply #16 on: February 26, 2012, 09:30:05 AM »
personally i think the RC version was too rushed...too little exposure time to the beta testers.
this is because i do not remember any of the beta testers having these install problems while observing the betas-->release candidate

 :-\

LorenzoC

  • Guest
Re: The consequence of poor testing
« Reply #17 on: February 26, 2012, 10:00:19 AM »
I have my little experience in the IT business.

Follow the simple reasoning:
1. If auto-updates from v.X to v.Y are not safe, Avast should have three levels for updates:
a. green, signature update
b. blue, v.X.x update, inside the same X "branch".
Here I am giving from granted that you don't need to unistall and reinstall Avast any time there is a program update because this would be like saying "do not use Avast".
c. yellow/black stripes, Avast notifies the user that a new version (from v.X to v.Y) is available but that's all, no auto-update possible.
In case "c" the user can click on the message to get information on the proper "unistall v.X-inistall v.Y" procedure. It would be nice to add the unistaller tool to Avast package, that could be handy if the Internet connection is not operable.

What happened instead was that Avast alerted of a program update and the user had the option to refuse or to proceed.

It is NOT true that ANY software that updates necessarily breaks. It is just a matter of quality control.
Recently I had issues with both Firefox and Thunderbird that did not work properly on XP after an update.
The reason was simply that the installer HAD NOT BEEN TESTED over an XP clean install but only on some systems that had lots of other MS software installed (.net, runtimes, etc).
Fortunately if your browser does not work that doesn't affect the system and you just use IE or Opera as backup.
It is MUCH worse when it is your antivirus.

Speaking of beta testers, you must have a pool of testers who cover the whole "spectrum" of systems you plan to support.
What is happening nowadays is that "developers", "power users", "testers", etc are all on modern systems while there are still MANY common users who are on vintage systems. This creates a situation where the development and the testing diverge from the "real" user base. It is obviously made worse by the "release often" policy and the race among firms to add the "newest feature".

For example I can't install any of the current Linux distros on an old PC I have in the garage because, despite the hardware is supported IN THEORY and it worked with previous versions, recent modifications on the kernel and Xorg make the installation to fail and/or Xorg inoperable. I guess it is some regression introduced on old hardware support that went unnoticed simply because NOBODY has tested it (and probably nobody ever will).

Last thing: there are very few things on earth that are more annoying than the FALSE sense of superiority coming from IT guys. And nothing more lame than blaming "the user". It is actually the biggest issue with software development and developers.
« Last Edit: February 26, 2012, 10:04:09 AM by LorenzoC »

Offline AntiVirusASeT

  • Poster
  • *
  • Posts: 462
Re: The consequence of poor testing
« Reply #18 on: February 26, 2012, 10:16:34 AM »
i am sry but a whole spectrum of systems is theoretically ideal but practically impossible. to get 'whole spectrum of systems' = 'i will never be able to release my software' (imagine that u have to test with each and every other software in the world to ensure compatibility- there is almost infinitely different permutations of software u got there  :)
show me a software where there is zero bugs after release to public--none
software are pieces of code not physical goods.

notifying the end user of new version without allowing update is redundant and creates confusion (they will be thinking theres a new version but why i cannot update?)

well Avast! does not release new major versions that often. the only problem is that this version is not tested long enough
« Last Edit: February 26, 2012, 10:21:41 AM by AntiVirusASeT »

AdrianH

  • Guest
Re: The consequence of poor testing
« Reply #19 on: February 26, 2012, 10:19:25 AM »
personally i think the RC version was too rushed...too little exposure time to the beta testers.
this is because i do not remember any of the beta testers having these install problems while observing the betas-->release candidate

 :-\

Rushed ? Possibly.  beta testers did have upgrade issues which were worked on.





What happened instead was that Avast alerted of a program update and the user had the option to refuse or to proceed.

Not true at all, you could ( and should ) have looked at avast.com to read what new features were involved in this updated program and there you could download a full copy for yourself so as you can run a custom install and select what features you want to use.



Quote
Speaking of beta testers, you must have a pool of testers who cover the whole "spectrum" of systems you plan to support.
What is happening nowadays is that "developers", "power users", "testers", etc are all on modern systems while there are still MANY common users who are on vintage systems. This creates a situation where the development and the testing diverge from the "real" user base. It is obviously made worse by the "release often" policy and the race among firms to add the "newest feature".
Not the case really, avast 7 beta was open to a wide range of users and if you look here on the forum many of them ,Evangelists included are on older systems running XP.

Quote
Last thing: there are very few things on earth that are more annoying than the FALSE sense of superiority coming from IT guys. And nothing more lame than blaming "the user". It is actually the biggest issue with software development and developers.

Hmm, almost as bad as those that do not check first what they are adding to their system and then scream blue murder afterwards?

Offline Asyn

  • Avast √úberevangelist
  • Certainly Bot
  • *****
  • Posts: 76118
  • Urlaub/Vacation
    • >>>  Avast Forum - Deutschsprachiger Bereich  <<<
Re: The consequence of poor testing
« Reply #20 on: February 26, 2012, 10:34:07 AM »
Not the case really, avast 7 beta was open to a wide range of users and if you look here on the forum many of them ,Evangelists included are on older systems running XP.

Confirmed..! ;)
W8.1 [x64] - Avast PremSec 22.7.7366.BC [UI.713] - Firefox ESR 91.11 [NS/uBO/PB] - Thunderbird 91.11
Avast-Tools: Secure Browser 103.0 - Cleanup 22.2 - SecureLine 5.18 - DriverUpdater 22.2 - CCleaner 6.01
Avast Wissenswertes (Downloads, Anleitungen & Infos): https://forum.avast.com/index.php?topic=60523.0

LorenzoC

  • Guest
Re: The consequence of poor testing
« Reply #21 on: February 26, 2012, 10:39:52 AM »
@AdrianH
It seems you live in some alternate reality where if Avast fails it would GAIN reputation and customers instead of LOSING both.

If I was the CEO at Avast headquarters it would be lots of torpedoes flying at mid air looking for somebody's #@@.
This "it is the user's fault" attitude is typical of developers who don't consider that if the software fails, WHATEVER THE REASON IS, they don't get the money. Because it is the dumb user who pays everybody's work. And kicking your customers and potential customers in the groin is not a wise move.

This discussion is rather pointless because it is like the three monkeys who don't listen, don't see and don't speak.

AdrianH

  • Guest
Re: The consequence of poor testing
« Reply #22 on: February 26, 2012, 10:57:29 AM »
@AdrianH
It seems you live in some alternate reality where if Avast fails it would GAIN reputation and customers instead of LOSING both.

If I was the CEO at Avast headquarters it would be lots of torpedoes flying at mid air looking for somebody's #@@.
This "it is the user's fault" attitude is typical of developers who don't consider that if the software fails, WHATEVER THE REASON IS, they don't get the money. Because it is the dumb user who pays everybody's work. And kicking your customers and potential customers in the groin is not a wise move.

This discussion is rather pointless because it is like the three monkeys who don't listen, don't see and don't speak.

Hmm, just who is it that is not listening? Like I said yes, things were possibly "rushed", maybe it was the CEO that pushed the "we must release" button early but I would ask you this, who is it that has the problem here?

I beta rested v7, I found and reported issues, I have installed avast7 Internet Security on 3 machines in this household and it is running flawlessly. I help out in a local PC store and since the v7 Final have installed the free version on @ 20 machines now, they are not new super computers, several were at least 8 years old and running XP, avast7 is working fine on them. .............

n01clueless

  • Guest
Re: The consequence of poor testing
« Reply #23 on: February 26, 2012, 11:09:57 AM »
I completely agree with Lorenzo. Most people here ARE missing the point. Avast is aimed at a mass market, not just power or advanced users. Thus, one should not reasonably expect the problems encountered with Avast 7 (compounded in my case because I use Comodo firewall).

When all is working well Avast is excellent. Unfortunately, for many people, this has not been the case with version 7. I would suggest the move from beta was premature.

clubhouse

  • Guest
Re: The consequence of poor testing
« Reply #24 on: February 26, 2012, 11:21:08 AM »
I completely agree with Lorenzo. Most people here ARE missing the point. Avast is aimed at a mass market, not just power or advanced users. Thus, one should not reasonably expect the problems encountered with Avast 7 (compounded in my case because I use Comodo firewall).

When all is working well Avast is excellent. Unfortunately, for many people, this has not been the case with version 7. I would suggest the move from beta was premature.


I agree with both you and Lorenzo, why should the average end user have to check on the Avast forum to see if they need to do a 'clean install'.....The program offers to update to the latest version....an awful lot of people understand they need an AV...they install and wish to go about their activities on the web...not become some kind of involuntary computer boffin.....some of the 'expert' members of this forum should step off their mantles and stop being so lofty.



I should add that this is a superb AV, I've been using AV pro for 3 years now and its free counterpart is streets ahead of the rivals now!
« Last Edit: February 26, 2012, 11:30:39 AM by clubhouse »

Jack 1000

  • Guest
Re: The consequence of poor testing
« Reply #25 on: February 26, 2012, 11:26:05 AM »
I was really shocked that an update pop-up for version 7 was out 1-2 days or so after general release.  This in sharp contrast to the several MONTHS that I can recall when Avast 5 and 6 came out.  The update was in the GUI, but we were told it would be awhile before the pop-up notifications would be released.

I think that the community feels that when the pop-up notification is ready and has been seen, the update should be simple and painless for at least 90% of the population.  It used to be that way with the pop-up notices for Avast 4-6.  I think there should be as many as three public release builds before you push out an auto-update. 

Having to uninstall, go into Safe Mode, get the Uninstall Program.  I actually did my safe mode restarts all at once and everything seemed OK after two reboots.  But Avast, you can't expect Joe and Jane six-pack who barely know how to get to their e-mail to know how to get to the all those processes because the final product, while probably ready for some people with intermediate to advanced knowledge, was not ready for general populations.

You need to go back to a slower roll-out for the pop-up notifications on program updates.  I suppose if you want to put them in the GUI that is OK after about 3 weeks to a month of being out of Beta.  However, I would say don't push stuff out through Automatic update, till you have gone through at least two general releases with a 90% success rate.  You have Avast 7 at maybe a 60% success rate, because you pushed out the pop-up program notification too fast.

And it's sad for people because Jane and Joe six pack aren't going to spend the time or have the knowledge to know how to fix things that should have been caught before final release.  The constant splash screen on reboot unless you do a clean install before hand.  That issue should have been squashed in Beta testing.  You have to look at the hundreds of millions of users who don't even know about this forum.  And what's going to stop them from turning to a competitive product like MSE if they have frustrating experiences with a program that was not ready for pop-up update notification?

I still love your services and support.  However, I have to agree with the majority of the community.  You released Avast 7 too fast.

Jack
« Last Edit: February 26, 2012, 08:07:24 PM by Jack 1000 »

LorenzoC

  • Guest
Re: The consequence of poor testing
« Reply #26 on: February 26, 2012, 11:28:21 AM »
It doesn't matter who took the decision, if it was deliberate or a misunderstanding.

Forcing "users" to do the actual beta testing by releasing an unfinished product and then even say it is their own fault if something breaks is plain WRONG and disruptive of TRUST.

The disaster of Windows Vista should be rather explanatory.

MAG

  • Guest
Re: The consequence of poor testing
« Reply #27 on: February 26, 2012, 11:36:46 AM »

For example I can't install any of the current Linux distros on an old PC I have in the garage because, despite the hardware is supported IN THEORY and it worked with previous versions, recent modifications on the kernel and Xorg make the installation to fail and/or Xorg inoperable.


Off topic I know - but just to try to lighten the mood a bit-

Ubuntu/debian won't work any more I agree.

Try the micro distros slitaz or puppylinux. They seem still to work fine on old machines.

The real joy for me is that PCLinuxOS, a mainstream distro, works fine on my ~ 7 year old machine - even with the 2.6.38 kernel. 8) As LXDE or Openbox it's even fast!
« Last Edit: February 26, 2012, 11:38:19 AM by mag »

Offline Asyn

  • Avast √úberevangelist
  • Certainly Bot
  • *****
  • Posts: 76118
  • Urlaub/Vacation
    • >>>  Avast Forum - Deutschsprachiger Bereich  <<<
Re: The consequence of poor testing
« Reply #28 on: February 26, 2012, 11:53:27 AM »
...and then even say it is their own fault if something breaks is plain WRONG and disruptive of TRUST.

Well, nobody said so, afaik.
W8.1 [x64] - Avast PremSec 22.7.7366.BC [UI.713] - Firefox ESR 91.11 [NS/uBO/PB] - Thunderbird 91.11
Avast-Tools: Secure Browser 103.0 - Cleanup 22.2 - SecureLine 5.18 - DriverUpdater 22.2 - CCleaner 6.01
Avast Wissenswertes (Downloads, Anleitungen & Infos): https://forum.avast.com/index.php?topic=60523.0

Alievitan

  • Guest
Re: The consequence of poor testing
« Reply #29 on: February 26, 2012, 12:09:47 PM »
I also agree with Lorenzo about the implicit trust concerning the autoupdate mechanism.  Microsoft is notorious for dragging its feet in regards to even security updates which they justify by saying they have stringent quality controls, but as far back as Windows XP, I do not remember an occasion where auto updates crippled my machine or cause me any significant problems.  In fact the only time I deal with autoupdates, is when I have a fresh windows install, which is the whole point autoupdates, you set and forget.   

However, updating from Avast 6 to 7 has been a nightmare,  realtime modules refusing to start, and registration problems, disruptions of popular programs like Chrome all of which in the end required me to use the specialized Avast removal tool and then clean install.  Any one of these problems are a big inconvenience for a Avast enthusiast and technical users like me who have the know how and patience to work it through, for everybody else meaning the average users, they can be with some, not even aware and vulnerable if for example realtime modules refuse to start.  extremely disruptive.   Even if you lowball the estimate of 10 percent of users will inherit problems, that still means potentially millions and millions of users who will experience problem

Avast must understand that as a security vendor, it is held to a much higher standard that any other with exception of the OS itself; it isn't like itunes, where bonked update only means users can't listen to music.    Users implicitly trust by their security software, to work properly, protect them and not cause disruptions which Avast 7 failed in all three.  That's the whole point of security software.  If you cannot trust your security software to update itself especially when it prompts you, then that reflects very poorly on the vendor to put it mildly.