Well I don't have a powerful system as you can see from my signature and I see zero impact from running avast and the resident scanners. With one exception, they are on the Normal 'default' sensitivity setting and that provides a good compromise between performance and protection. I have Standard Shield, Internet Mail (High), Web Shield and Network Shield providers running, I don't use P2P and Instant Messaging applications.
I also use an imaging application (Drive Image 7.1) but I feel that there really is no performance benefit from crippling avast, certainly not on my system. Over the past program updates the scanning engine has been improved and I'm a believer in the prevention rather than cure is best.
For that very unlikely event of being hit by a trojan, keylogger, etc. that could seriously compromise your security without your knowledge. So any malware that manages to get past your defences (non existent if you have no resident providers enabled) could have free reign to connect to the internet to either download more of the same, pass your personal data (sensitive or otherwise, user names, passwords, keylogger retrieved data, etc.) or open a backdoor to your computer, so outbound protection is essential.
Your system, your choice.