Rick and the crew explicitly stated there were multiple routes to Hilltop and they were at every one of them. Some routes were taken (like at night) off the main trail.

Ok - maybe as a viewer I am underestimating the Saviors numbers or it's just a somewhat lazy and convenient way to tell the story.