IE is an integral part of the OS and is used for many things (besides the IE browser), like windows explorer for displaying the folder views and for displaying help files, etc. So there should be no conflict or doubtless we would have bumped into it already and wouldn't offer it as a suggestion.
I would ignore this supposed illegal operation, especially as a) it is a one off unless it is a regular occurrence and b) since you appear to have gone on a tweaking fest to see what it can do. Just restart avant or reboot and monitor it in the future.
I already tried it again this morning ... 3 times!
And Nope. It is obvious this is going to be more critical than just ignoring and sidestepping the Illegal Operation message. I flat out cannot get past the point where I acknowledge that it is OK to use Large Icons on my Toolbars. So, No Go on the Avant Restarts. Mission gets aborted every time. I think it quickly got to the point that there's only one possible alternative. Uninstalling and Reinstalling Avant. If that doesn't get it in gear, then it's NOT going to work period.
Since you haven't had AOL, you couldn't have known that there'd conceivably be sibling rivalry between the AOL & Avant IE siblings.
It's possible a Reinstall of Avant might do the trick. After all, my initial Install of Opera 9.51 left me without the ability to open ANY website. I uninstalled it and reinstalled it and it then worked OK. So, maybe this afternoon I'll reinstall Avant. If it is ascertained that Avant is just not going to be welcomed by AOL, most likely neither will Maxthon or any other IE-based Browser.