iPhone Atlas has a run-down of the main differences between the iPhone and iPod Touch. The main ones are that the iPod Touch has no built-in camera, microphone, loudspeaker, mail application or Google Maps, so it isn't quite the PDA it could have been. The lack of a mic means that, unlike many cheaper rival MP3 players, you can't use it as a voice recorder.
Lacking the phone software, the iPod Touch has separate application for handling contacts.