Hi vishu9, welcome to the forum.
It would have been better to post your question in a new topic, rather than adding it to an old and not-very related thread. But I can answer your question in general terms.
You can not install or run another AV with Avast. Nor with most other resident AV's.
They are likely to conflict, fight over file access, and in the end you end up with less protection than if just one were installed.
That is probably why you are getting the rootkit warning, if the AVG installer you have is genuine. (If it is not genuine, ie: a crack/warez version, it likely contains a rootkit or trojan.)
I know of no combination of resident AV that is compatible. I have read reports that MSE is compatible, but wouldn not try that myself. I do know that Threatfire, a behaviour blocker, works well with Avast 4.8
You can have demand scanners, such as MBAM or Superantispyware, installed alongside Avast, which I recommend. Just update and run a scan with one of them every week or two, or any time malware is suspected. One of these can be a useful adjunct to Avast, which does a very good job but can't always detect and stop all malware. Having a demand scanner improves the chance that you can detect and remove malware that Avast has not detected.
A new version of Avast is available; version 5.0.462, reported
here (with download links) and apparently this is quite stable for most.