Mplayer users can get a list of deinterlacing options by typing mplayer -pphelp at the command line.
Find the option that you want to use, and then use the -vf pp=<option> syntax to activate the option.
For example, I usually use the lb option, which is done with the command: mplayer -vf pp=lb, followed by whatever other options you would normally use to play the DVD.
Try md, fd.. and see if you get better results.
Use the -vf pp=<option> syntax to activate the option
Note that adding it to mplayer.conf does not require you to add that '-' just "vf=filmdint"
(Copy and pasted from a mIRC log from last night with some of the team members of XBMC)
I hope this helps.