Photo from wikipedia.
I used to do this too, so I know how tempting it is. We realized the error in our way while watching our food guru, Alton Brown (hey, we're scientists, leave us alone!).
I know a lot of people who hate tomatoes, I suspect if they would have just eaten non-refrigerated tomatoes they would've changed their tune.
According to about.com:
"Refrigeration is the enemy of the tomato as it nullifies flavor and turns the flesh mealy. The culprit is a compound called Z-3 hexenel, which accounts for the tomato's scent and taste. The development process which turns tomato's linolenic acid to the Z-3 that makes our mouth and nose sing is hindered by cold."
So, room temperature tomatoes = flavorful and smooth. Fridge tomatoes? mealy and bland.
Please, do yourself (and your tomatoes) a favor and leave them out! You will be so much happier, promise.