You're absolutely right: I was just trying to get this to work with my brand new MacBook Pro running 10.4.9, and quickly realized that vi wasn't able to handle the new binary plist file format.

Digging around (with the super-useful "man -k plist" in the Terminal) yielded the solution: you need to use the plutil command to convert the specific "plist" file back to the older xml format, edit it as needed with "vi", then convert it back to the newer binary format.

Here's how I did that...

First, I logged in as the administrator (you can also just type su) and moved to the correct directory:

This gave me an emergency backup of the file in case I did something wrong.

To convert the plist flie into the older, editable format, I typed in:

Now it was ready for editing and was in a format that "vi" could handle, so it was easy to search for "guest" then change the tag immediately following from "<true/>"> to "<false/>".

Save the changes and it's time to convert the file's format back to the newer plist form:

That's all there is to it. You should now have the Guest login option greyed out and unavailable for people connecting to your computer via Apple's Mac file server capabilities.