I wouldn't take on directing. I just would pull out of doing it altogether...because you're financing it right? Either pull out, or don't let this dude have any producer-like responsibility (setting meetings, etc.) Make sure he knows what he wants, make sure he's rehearsing actors, make sure your camera/special effects departments are on point, etc.

Otherwise, you'll have a mess on your hands.