Well, I can think in some scenarios:
1. Other things are being scanned at the same time (other process, other files) than your excluded ones.
2. The scanner is just checking the file is on exclusion list and the icon swirl, although the file is not scanned, i.e., the icon is showing avast! activity (checking files) and not exactly scanning.