Hi bajjerfan,
I've tried this problem several times and it seems that multiple tabs in Bank Mode happens when you also have a multiple chase.com tabs opened in a desktop (e.g. reopening browser from last session). Can this be the cause for you?
If you want to change the websites which are opened in Bank Mode directly you can open in the new tab of the browser: chrome-extension://lhnnoklckomcfdlknmjaenoodlpfdclc/html/options.html and there is a option Automatically open Bank Mode on the following websites.
Thank you and let me know if it worked for you.