There are 2 ways you can do this :
Create an icon file (.jpg) of what you would like to disply e.g the DVD cover for the movie, then copy it to the same directory your movie is stored.
Name the icon file the same as the movie you would like it to display it for with a .tbn extension
ie: Transformers.avi, Transformers.tbn
This can be done for any format that is recognized by XBMC. This can be anything, audio, data,.. etc
Folder Thumbs: *.jpg (or *.png).
create the icon file as before, Put it in the same folder and rename it to folder.jpg
however with this method you can only have 1 icon per folder (this is useful though if you have a t.v series for example with many episodes in one folder)
This works for all folders!