They knew what Agents looked like. It wasn't like Smith was in disguise...
-----------------------------HEY KIDS, (BUY MY) COMICS!! https://www.mythworldemedia.com/store