I use remote desktop regularly, have avast! both on the computers I'm connecting from and the computers I'm connecting to, and never had any problems with that.
What edition of avast! do you have? If it's AIS, what network profile do you have set in the firewall?