Why does my husband not want to be around my family?

We have been married for 4 mo & lately I have been noticing my husband's rudeness when it comes to visiting with my family.This weekend my brother & his wife came to stay with us. Whenever my brother is in town we all go over to my Dad's to spend time together. The last few times I have wanted to go over to his house my husband gets very grumpy & pouts. I have asked him why he acts this way and he says it is because I tell him we are going instead of asking. This time when my bro was here, I umed that my husbnd knew we would be spending the wkend w/ family. When we went over he pouted & slept on the sofa while I visited. I was so embared. Next day, he slept in the car while we were out sightseeing, and then slept again at my dad's neighbors who invited them over to watch football. my entire family was uncomfortable.My brother even said said that he wouldn't stay with us again if we didn't want him to! I don't understand, my fam loves my hsbnd and he just pushes them away