You can step your game up without straying from the basics if your fit is on point.

Judging by your description, you're a tough fir off the rack, but alterations aren't prohibitively expensive. The first thing I did when I started to dress better was take a couple of my favorite shirts to a tailor to have them taken in. Cost me about $15 a pop.

Same applies to pants. You can have the seat taken in and pants legs tapered. Once you have clothes that fit the way you like, measure them. Knowing the measurement of a well-fitting shirt or jacket is actually more valuable than knowing your body's measurements. A lot of online retailers can give you measurements of their pieces if you ask via email or the live-chat function on their websites