We become insanely insular with marriage, family, career etc. for whatever reason that we don't realize the importance of outside support, building with your day ones etc. Without strong ties, who are we? He with the most value based relationships wins. If you really value this brother, there's ways for you to both get what you want. Is it really an all or nothing proposition? Just say "I know you wanted to do X,Y,Z and I want to kick it with you but this weekend, I'm only able to put A,B, and C on the agenda." There's room for compromise.

The one thing you shouldn't do is try to grandstand on some "You need to grow up." about how *you* want the weekend to play out. Yes, you have your life and he has his, and it seems like you're both living positively - but no need to try to establish a moral highground to simultaneously shame him and substantiate your decision. If he's a friend, he'll receive where you're coming form.