More like a lot of men don't care, and homeless men can't exactly unite and lobby.

If enough men cared about the plight of the homeless in general, they would be better off. However, the ranks of influential men are packed with those who lack empathy. Thus the weight of their influence makes it seem like nobody cares, whereas it's just that not enough of the right people care.