The cover image stored in iTunes is the one shown by the iPhone when you browse albums with Cover Flow. If you don't have a cover image for all your albums, you can download them from iTunes directly, but this requires an iTunes Store account, which in turns require you to leave them your credit card (even if you don't intend to make a purchase).

iArtWork is a great alternative that integrates perfectly with iTunes, is 100% freeware and is super simple to use. Just run it, and it will automatically download the covers for all albums in iTunes that don't have one already, and update the information directly into iTunes.