Hallo,
that's quite normal that after scanning many files the address space might get "polluted" a bit and the memory footprint might become larger - because all scanned objects are either mmap()-ed, or their (extracted) parts are malloc()-ed.
The 500MB size (not 50-60MB as stated - that's normal) is, indeed, too much - normally there might be some compressed biig file (and for decompression purposes, the heap-region boundary would be extended - to allow processing of this file).
BUT, that Apple's Activity monitor is probably ... buggy a bit (sigh... hell, what's NOT buggy in MacOS?), because it apparently lies:

Compare it, please, with the (correct) vmmap output:
Virtual Memory Map of process 197 (com.avast.MacAvast.MAD)
Output report format: 2.2 -- 32-bit process
==== Non-writable regions for process 197
__PAGEZERO 00000000-00001000 [ 4K] ---/--- SM=NUL /Applications/avast!.app/Contents/Resour
ces/com.avast.MacAvast.MAD
__TEXT 00001000-00234000 [ 2252K] r-x/rwx SM=COW /Applications/avast!.app/Contents/Resour
ces/com.avast.MacAvast.MAD
__LINKEDIT 00f15000-00f18000 [ 12K] r--/rwx SM=COW /Applications/avast!.app/Contents/Resour
ces/com.avast.MacAvast.MAD
STACK GUARD 00f18000-00f19000 [ 4K] ---/rwx SM=NUL
STACK GUARD 00f1a000-00f1b000 [ 4K] ---/rwx SM=NUL
shared memory 00f1b000-00f1c000 [ 4K] r--/rw- SM=SHM
VM_ALLOCATE ? 00f29000-00f30000 [ 28K] r-x/rwx SM=PRV
VM_ALLOCATE ? 00f38000-00f40000 [ 32K] r-x/rwx SM=PRV
VM_ALLOCATE ? 049be000-04ee8000 [ 5288K] r-x/rwx SM=PRV
__TEXT 8fe00000-8fe31000 [ 196K] r-x/rwx SM=COW /usr/lib/dyld
__LINKEDIT 8fe68000-8fe76000 [ 56K] r--/rwx SM=COW /usr/lib/dyld
__TEXT 90318000-904b2000 [ 1640K] r-x/r-x SM=COW /usr/lib/libSystem.B.dylib
__TEXT 9582d000-95833000 [ 24K] r-x/r-x SM=COW /usr/lib/system/libmathCommon.A.dylib
__LINKEDIT 96dbf000-9722c000 [ 4532K] r--/r-- SM=COW /usr/lib/system/libmathCommon.A.dylib
STACK GUARD bc000000-bf800000 [ 56.0M] ---/rwx SM=NUL
STACK GUARD f0000000-f0001000 [ 4K] ---/rwx SM=NUL
STACK GUARD f0082000-f0083000 [ 4K] ---/rwx SM=NUL
STACK GUARD fffec000-ffff0000 [ 16K] ---/rwx SM=NUL
shared memory ffff8000-ffffa000 [ 8K] r-x/r-x SM=SHM
==== Writable regions for process 197
__DATA 00234000-00256000 [ 136K] rw-/rwx SM=COW /Applications/avast!.app/Contents/Resour
ces/com.avast.MacAvast.MAD
__DATA 00256000-00f15000 [ 12.7M] rw-/rwx SM=PRV /Applications/avast!.app/Contents/Resour
ces/com.avast.MacAvast.MAD
MALLOC (freed?) 00f19000-00f1a000 [ 4K] rw-/rwx SM=PRV
VM_ALLOCATE ? 00f1c000-00f1d000 [ 4K] rwx/rwx SM=PRV
MALLOC_LARGE 00f1d000-00f29000 [ 48K] rw-/rwx SM=PRV DefaultMallocZone_0x1000000
MALLOC_LARGE 00f30000-00f38000 [ 32K] rw-/rwx SM=PRV DefaultMallocZone_0x1000000
VM_ALLOCATE ? 00f40000-00f42000 [ 8K] rwx/rwx SM=PRV
MALLOC_LARGE 00f42000-00f4c000 [ 40K] rw-/rwx SM=PRV DefaultMallocZone_0x1000000
MALLOC_LARGE 00f50000-00f55000 [ 20K] rw-/rwx SM=PRV DefaultMallocZone_0x1000000
MALLOC_LARGE 00f57000-00f6b000 [ 80K] rw-/rwx SM=PRV DefaultMallocZone_0x1000000
MALLOC_LARGE 00f75000-00f79000 [ 16K] rw-/rwx SM=PRV DefaultMallocZone_0x1000000
VM_ALLOCATE ? 00f79000-00fab000 [ 200K] rwx/rwx SM=PRV
MALLOC_TINY 01000000-01100000 [ 1024K] rw-/rwx SM=PRV DefaultMallocZone_0x1000000
VM_ALLOCATE ? 01100000-01300000 [ 2048K] rw-/rwx SM=PRV
MALLOC_TINY (freed) 01300000-01400000 [ 1024K] rw-/rwx SM=COW DefaultMallocZone_0x1000000
MALLOC_TINY 01400000-01500000 [ 1024K] rw-/rwx SM=PRV DefaultMallocZone_0x1000000
MALLOC_TINY (freed) 01500000-01600000 [ 1024K] rw-/rwx SM=COW DefaultMallocZone_0x1000000
MALLOC_TINY 01600000-01700000 [ 1024K] rw-/rwx SM=COW DefaultMallocZone_0x1000000
VM_ALLOCATE ? 01700000-01800000 [ 1024K] rw-/rwx SM=PRV
MALLOC_SMALL 01800000-02000000 [ 8192K] rw-/rwx SM=COW DefaultMallocZone_0x1000000
MALLOC_TINY (freed) 02000000-02200000 [ 2048K] rw-/rwx SM=COW DefaultMallocZone_0x1000000
MALLOC_TINY 02200000-02400000 [ 2048K] rw-/rwx SM=COW DefaultMallocZone_0x1000000
VM_ALLOCATE ? 02400000-02b01000 [ 7172K] rwx/rwx SM=PRV
MALLOC_LARGE 02b01000-02d02000 [ 2052K] rw-/rwx SM=PRV DefaultMallocZone_0x1000000
VM_ALLOCATE ? 03c0a000-042e5000 [ 7020K] rwx/rwx SM=PRV
VM_ALLOCATE ? 042e5000-049be000 [ 7012K] rw-/rwx SM=PRV
__DATA 8fe31000-8fe34000 [ 12K] rw-/rwx SM=COW /usr/lib/dyld
__DATA 8fe34000-8fe68000 [ 208K] rw-/rwx SM=ZER /usr/lib/dyld
__DATA a0072000-a00f0000 [ 504K] rw-/rwx SM=COW /usr/lib/libSystem.B.dylib
__DATA a0b83000-a0b84000 [ 4K] rw-/rwx SM=COW /usr/lib/system/libmathCommon.A.dylib
Stack bf800000-bffff000 [ 8188K] rw-/rwx SM=PRV thread 0
Stack bffff000-c0000000 [ 4K] rw-/rwx SM=COW
Stack f0001000-f0082000 [ 516K] rw-/rwx SM=PRV thread 2
Stack f0083000-f0104000 [ 516K] rw-/rwx SM=PRV thread 1
==== Legend
SM=sharing mode:
COW=copy_on_write PRV=private NUL=empty ALI=aliased
SHM=shared ZER=zero_filled S/A=shared_alias
==== Summary for process 197
ReadOnly portion of Libraries: Total=8712K resident=5300K(61%) swapped_out_or_unallocated=3412K(39%)
Writable regions: Total=65.1M written=26.2M(40%) resident=18.4M(28%) swapped_out=14.3M(22%) unallocated=46
.7M(72%)
REGION TYPE [ VIRTUAL]
=========== [ =======]
MALLOC [ 19.2M]
STACK GUARD [ 56.0M]
Stack [ 9224K]
VM_ALLOCATE ? [ 29.1M]
__DATA [ 13.6M]
__LINKEDIT [ 4600K]
__PAGEZERO [ 4K]
__TEXT [ 4112K]
shared memory [ 12K]
Try to run something trivial (sleep 60; from a terminal), and the consumption will be also 600MB

.
BTW. anyway, what matters here is the RSS size (you can do mmap(500000000), the Vm size will go to 500MB, but the RSS stays low until the area is accessed, page-per-page).
Regards,
PC