Odd, I've never noticed this before. Now that I have it's obvious.
I got some food out of the freezer before to cook for dinner, I closed the freezer, and it started hissing at me. "ssssssssssssss" it was saying. I thought that was a bit strange, and opened it to see what was creating the noise, and the noise stopped. I closed it again, and it started to make the noise again. About 1/4 of a second later I concluded that this was the sound of air getting inside it. This is the obvious bit, but had I not heard it, I would not have thought about this ever. When I open the freezer, warmer air from the kitchen gets inside, raising the temperature of the freezer. When I shut the door, the warmer air cools down, and the molecules vibrate less, and the air "contracts". This leaves a vacuum, which air from outside the fridge comes and fills - I'm assuming because of possibly dodgy seals on the fridge.
Well, that's my new learning for the day.