My approach is to have a few cards with no annual fee, unlimited cash back as the perk, and cards that give higher percentages for specific types of purchases I frequently make. I've got 3 cards, and each one gives 3-4% cash back on either gas, groceries, restaurants, or travel expenses. That covers the majority of what I spend.

The alternative to cash back is cards that give you air miles, hotel points, etc. But you might end up spending more money to take advantage of those perks (e.g. taking an extra vacation you wouldn't otherwise take). I prefer cash back because it's always a net reduction to my bottom line.