I would guess that you're getting at the assumed 'ownership' or something like that? Or I guess the 'hey baby, smile' part?
If you feel like leaving you know you can go But why don't you stay until tomorrow? And if you want to be free You know all you got to do is say so
Seems like a reach but I suppose you could say that there's a presumption that she's only free cause he allows her to be free? Kinda like 'I've decided that I will leave you alone if you want me to' which suggests that NOT leaving her alone is one of the protaganist's options. lol. I'm TRYING to be critical but feel kinda silly. Is B Marley's 'no woman no cry' also questionable?
As for the 'hey baby, smile' thing. Obviously, there's alot of contexts that this is offensive. Like bugging a woman on the subway saying 'smile' while she is clearly annoyed. That's not what's happening in the song though. My SO and I might tell each other to smile while we're hanging out....trying to cheer each other up or something like that.
Maybe I'm missing something. I seem to recall someone saying this song was sexist in the past though.