You'd be surprised by how much these 16 by 16 pixel images can improve the image of your site in the eyes of users: Almost all large sites have favicons, hardly any amateur sites do. User's notice this, as they notice the padlock when purchasing, and therefore trust a site more.
An example of a favicon I made in less than five minutes on a site: http://www.historyempire.com
Another one is this site!
Open paint, make a basic 16x16 image, save as filetype BMP but as filename favicon.ico. Upload this to your domain root (so the location will be http://www.yourdomain.com/favicon.ico). Then add to all pages
<link rel="icon" href="http://www.yourdomain.com/favicon.ico" type="image/x-icon">
Chris take note - I don't think website publisher does this, and hence when you try to add a bookmarks toolbar link to it in firefox, the favicon does not show.
Bookmarks