Use what you have starting out, then if you get 3-5 episodes recorded and edited and you still want to keep going, buy stuff.
I've met too many people who buy stuff and then don't stick it out because they don't want to do the work, or they just don't have enough interesting stuff to say, or they're not actually interested enough in anything to keep the ideas fresh, etc.
For real, have the guests record into their voice memo app using their earbuds, transfer those files into the computer and then Audacity and then sync them up, and then learn the basic editing decisions and techniques.
If you do that 3-5 times and you still wanna buy stuff, buy stuff.