I have to admit that with 5466 songs in my main iTunes library I nonetheless generally use the main shuffle mode and just envision it as my own personal radio station with no ads. Nonetheless since I have a couple of movies and a bunch of audio books in my iTunes library I have created one smart playlist, called "All Music":

As you can see, it matches all music in my library that isn't either genre "Audiobooks" or genre "Movies". And, yes, I'm very methodical in assigning genres, such that my 5466 songs are completely represented by only 12 genres. It's very helpful and easy to do if you apply the correct genre as you import music. But I digress... :-)

To create a smart playlist, choose File --> New Smart Playlist... and you'll be shown a window that looks a ton like the above, but simpler. You don't realize the level of sophistication you can use until you actually click on the first field, at which point you suddenly realize that there's a whole world hidden in iTunes smart playlists:

(Yes, I've hacked this incredibly long menu up for this posting)

For example, you might want to have a smart playlist that's just songs with a zero play count, that is, songs that you've never heard in iTunes from your library. Presumably, over time you would find that it would shrink to zero, but it's surprising just how much of your library isn't played even if you leave your computer in shuffle mode 24/7. To do that, you'd simply create this criterion:

Another interesting smart playlist could be based on your own rating of individual tracks, but I have to admit that I have never rated any of my own tracks so it wouldn't work for me. But how about a playlist that's just TV shows you've downloaded / bought from the iTunes store?

Or perhaps your musical tastes run to disco and you'd like a smart playlist that is constrained to disco music released between 1970 and 1975. No problem for iTunes:

Finally, you might just be more interested in different constraints, in which case you might well find that the Limit To field is more your speed. For example, here's how you can have a playlist that contains one hour of your most recently played music:

Anyway, suffice to say that there are a lot of fun things you can do with a smart playlist, and as long as you leave "Live Updating" selected, any new music you download or add manually will automatically be matched by your playlists too, as appropriate.

Also, I recommend a fun article on The Apple Blog about smart playlists. Make sure you read through the comments too. There are some great additional ideas there!