Right now can pick up several of the classic Mingus albums on disc for relatively cheap:

Original album series overlapping with the Atlantic box set (the clown, pithecanthropus erectus, etc) as low as $18:
http://amzn.to/IzEjfH

Similar release of three albums, favoring Columbia releases (incl. Mingus Ah Um) as low as $11:
http://amzn.to/HP8Owg

One of the recent two-on-one Impulse releases that includes Black Saint and Mingus x 5 for as low as $7:
http://amzn.to/IMZHjm

Theoretically, for less than $50 can have a lot of the most significant music, packaging obviously isn't going to be ideal but a bargain quantity-wise; if he catches you it's not unlikely you'd want to keep hearing more. I find him among the most accessible due to being so arranged (the Impulse albums appear to extend this into the recording process to a greater extent).