First, if your Avast is acting like mine does, youll find that it isnt scanning the time machine files on your external drive, its scanning a hidden volume on your macs internal drive. Apparently Mac 12.3 Monterey keeps a cop of the timemachine back ups on its own hard drive in a hidden volume. I found this out when Avast found infected files because it gives the path of the infection. That path is /volumes/.timemachine/ (long obnoxious number) /2022-3-3-101022backup/2022-3-3-101022backup/data/user/mac/library/ etc, etc, etc.
So /volumes/.timemachine/ is on the main disk. .timemachine is a hidden file because of the dot prefix.
Now, why unchecking the scan time machine setting doesnt exclude this hidden time machine partition, I have no idea, but it doesnt exclude it at all and since its scanning these hidden files, deep system scans for me, take hours.