I have a 3.06ghz iMac, and there's 1 video that it can't handle because of the insane bitrate spike in one of the scenes. Other than that, it flawlessly plays everything well. Cost of an iMac isn't reasonable for a HTPC, however.

Plex is great, but Quicktime (I prefer 7 over X) actually seems to work better for 1080p stuff and Quicktime is the only one that could have used hardware acceleration (your nVidia 9400m) to offload onto the graphics card.

If you're going the Windows route, I'd recommend using MediaPlayerClassic Home Cinema and CCCP (http://www.cccp-project.net/). CCCP will have MPCHC. Also, I'd recommend an nVidia card where you can activate CUDA, which will allow you to offload a lot of the work onto the graphics card, since most video playback normally just uses the CPU to do everything. This is a guide for anime stuff but gives you the general idea for playback: http://nunnally.ahmygoddess.net/watching-h264-videos-using-compute-unified-device-architecture-cuda/