"these films have generally little substance and are just shallow excuses to tap into the audience's vicarious and escapist fantasies"

Butch Cassidey (which came at the start of the 70s "american new wave"), and The Jungle Book. Are the only really good top grossing films of the 60's and not even they are very intellectually stimulating. Saving Private Ryan has more substance than any of those films. So action is big now instead of singing and dancing, not much of a difference. It was all escapism back then as well.
And art films aren't black balled, they wouldnt make them period if they didn't want to, the studios just know that their audience is more limited. You cant blame Hollywood for a public interested in spectacle over substance can you?