Issue with resolution

When I try to play in 800x600 (My desktop res is set to 1024x768, but it happens on any resolution) it comes up with a small window in the top left of my screen. I can't get it to go fullscreen...