You don't say what type of mouse you have, wireless, optical, manufacturer, OS drivers or makers driver, etc.?
I very occasionally get the pointer skip about (Genius, optical, wireless 5 button mouse), usually after I have been using the wheel. When does it happen with your mouse pointer?
Clicking one of the buttons or the wheel button normally cures it.