The Avast Secure Browser (ASB) doesn't come packaged with the Avast Antivirus, there may well be an option when you are installing Avast for the ASB. So you could have missed that.
You should be able to stop avast using ASB by ensuring your Browser of choice is selected as the default browser.
To uninstall ASB, if you have Windows 10 use the windows button and find Avast Secure Browser in the list, right click on it and select Uninstall from the drop-down list.
Or you can uninstall it from the control panel in the same way as other programs.