My own website seems to handle this just fine, maybe it's because I actually explore the options inside WordPress and it defaults to what I like.
However, when running a website for a large group of other people to mess around with, all their images link to an attachment page rather than the image itself (which causes havoc for shadowbox plugins etc.)
So, you can force the default link to be the file (or the attachment page... or even blank if you would like).
Goto the "secret WordPress options page" at http://www.example.com/wp-admin/options.php (obviously, replace example.com with your domain)
Find the option "image_default_link_type" and change as required:
- leave - it blank no URL at all
- file - Default to File URL eg. /wp-content/image.png
- post - Default to Attachment Page URL eg. /?attachment=111