I've encountered the same problem and there are two ways I have found to "fix" shuffle mode so that you don't bump into tracks or genres that you don't want. The first works for most versions of iTunes and the second has been added in iTunes 7.0 as far as I can tell.

One way to tackle this problem is to create a Smart Playlist in iTunes that simply excludes the tracks you don't want, then choose that before you go into shuffle mode and press "play". My Smart Playlist looks like this:

To create a Smart Playlist, simply choose File --> New Smart Playlist and it's pretty straightforward to duplicate what I have above. You might have "Spoken" instead of "Audiobooks" as your genre, too, so I'd check that or tweak your Smart Playlist to match both.

Now it turns out that audiobooks downloaded from services like Audible.com (of which I'm a big fan, btw, and highly recommend if you're into audio books too) are already in a special category "Audiobooks" and aren't going to show up anyway. What you're talking about, however, are regular Mp3 files you import or audio book CDs that you rip and add to your library.

To try the fancy new iTunes 7.0 solution, you need to select more than one audiobook track, then choose File --> Get Info, which brings up this sort of dialog box:

Click for full-size image

Let me zoom in on the relevant field on the bottom row so you can see the key element:

That's it. Select all your audiobook tracks, bring up the Multiple Item Info, then simply select Yes and check the box adjacent to "Skip When Shuffling".

Either way, I hope this'll help you get more enjoyment out of your iTunes library.