Of course it would load faster, have you any idea just how many files firefox loads when it starts a lot, more so if you have many, many add-ons (how many add-ons do you have ?).
For me I can't remember how long firefox took to load as I have had avast for almost 6 years and it is always on. My firefox loads in about 8 seconds, so what are your system specifications, CPU, RAM, OS, etc. ?
I turn on firefox once at the start of the day and leave it on until I shutdown my system at night. So that may be an option if you are turning it of after each browsing session.
Have (or did) you another Anti-Virus installed in this system, if so what was it and how did you get rid of it ?
What other security based software do you have installed ?