I was told that a hat was a sign that someone was in transit. Wearing one inside shows that you don't plan on staying or don't really want to be there just like wearing a coat somewhere that you don't really want to be so you can get out of there faster.

In the south there were two major reasons. I was raised that wearing a hat in someone's house meant you thought their roof was exposed or broken and you need a hat to protect you from rain or heat. The second, wearing a hat is like your jacket. Men wore hats to protect their heads from things like dust, dirt, rain, and snow. If you walked into a building, you took off your hat and hung it up so you weren't tracking that inside. It was considered a sign of respect after that. Y'know, don't wanna get gunk everywhere.