Many people are not aware that they already may have a hardware firewall which is much better than a software one.
Check your modem/router manual to see if it has a firewall or not.
If it has, login to the modem/router and make adjustments to the settings (if needed/wanted)
If you have a hardware firewall, there is no need to have a software one as well.
As I have said several times before on this webboard:
See the firewall as a doorman.
Put him outside and in front of the door and he will not let unwanted people in. (hardware firewall)
Put him inside and the unwated people are already in before he sees them. (software firewall)
Most hardware firewalls only provide inbound protection not outbound. Connections originating from your system will be able to get back in to your system without a second glance from your firewall as the request originated from your system.
You only have to browse the viruses and worms forum to see that many are already in your house, they have already got past your firewall and AV. The first people know is when they try to get outbound connections to malicious sites and avast alerts.
So outbound protection is none the less important; to use your analogy, OK the thief has got in, but you want to catch him and not let him out to continue his criminal activity.