Dave Taylor answers free tech support questions about a wide variety of business and technical topics, including blogging, iphone help, ipod help, AdSense, MySpace, Sony PSP help, Mp3 players, Windows XP, Windows Vista, Linux, SEO, Mac OS X, Facebook, Twitter and LinkedIn.

When was the last time Google visited my Web site?

Is there any way to figure out the last time that Google visited my Web site? This would also be helpful when I analyze competitor's sites too...


Dave's Answer:

First, let's name the beast: Google's spider that visits your site, analyzes the content and then categories your material for the search engine is called "Googlebot" (Google + robot). This isn't anything hidden, it's just useful to be able to differentiate between Google the search engine and the Googlebot that, behind the scenes, actually visits your site so that you can be found in the search engine.

If you've been doing searches online -- and I can't imagine anyone who hasn't! -- then you're probably used to just skimming quickly through the search results, clicking on likely matches and basically zooming off of the Google page as fast as you can. Next time, though, slow down a bit. It turns out that there's a lot of interesting information you can glean from Google search results, information that's there, but not necessarily obvious. Indeed, it's often the case that search engines overall, not just Google, make extra information available, we just gloss past it in our zeal to find the answer to our specific search.

To understand how to ascertain Googlebot visit frequency, it's important to know that Google keeps its own copy of every page on your Web site that it indexes in its own local storage, called its "cache". Your computer caches information too, which is why the second time you load a Web page is always faster than the first time you visit it. On Google, it turns out that if we examine the cache information, it will tell us what we want to know. But let me show you, rather than just talk about it.

For example, let's do a quick Google search on iphone help and see what comes up... ah, what a coincidence! It's my own Web site. :-)

If you do the search, though, you'll see that not only do you get the title of the matching page and a twenty or so word description, but there's other information displayed, in smaller type and a more subtle color.

The last line shows: "www.askdavetaylor.com/about_iphone.html - 30k - Cached - Similar pages - Note this", and it's this the last line I want you to focus on. In order, the information displayed is the actual URL of the matching page followed by the approximate page size (in this case, 30k), a link to the cached copy of the page (which is critically important for our detective effort!), a way to search for similar pages in the Google engine and, finally, if you're logged in to your Google account, the "Note this" link is a simple way to add this link to your Google Notebook, if you use it.

Click on the "Cached" link and you'll be shown an archived copy of the Web page from Google's internal storage system, with a long, complicated header that's oh-so-important to our quest. The first line says: "This is G o o g l e's cache of the URL as retrieved on May 16, 2008 07:00:18 GMT."

So there's your answer. This particular page was last crawled by Google on 16 May, 2008. Since I'm writing this on 23 May, this means that Google's snapshot of this page is only a week old.

You can use the trick of clicking on the "Cached" link for any search result you find and then easily identify how old that copy is in the Google database. If it's weeks or months old, it means that nothing's changed and that the Googlebot isn't visiting you very often. This can only be changed by updating your content with frequency.



Help others find this article at Del.icio.us, Digg, Netscape, Reddit, and Simpy.


Subscribe!

Never miss another useful Q&A article again! Subscribe to AskDaveTaylor with Google Reader.

Comments

I am trying to figure out how to create a mailing list for my myspace. I am an musician and I want to have a text box for people to submit their email.
Right now I am using a comment box that I created from some website online but it looks really bad.
So I was wondering if you had any ideas...?

thanks so much,
aaron.

myspace.com/aaronpurdy

Posted by: aaron purdy at June 2, 2008 4:58 PM

Dave,
Deja vu?
You addressed this same issue on April 7, 2008
Just goes to show that you can't trick us.
Still love your advice though.
regards,
Darryl

Posted by: Darryl Chrisp at June 2, 2008 5:17 PM

I have a lot to say, but ...
Starbucks coffee cup I have a lot to say, and questions of my own for that matter, but most of all I'd like to say thank you for all your efforts on this Web site by buying you a chai!

I do have a comment, now that you mention it!









Remember personal info?


Please note that I will never send you any unsolicited commercial email. Ever.

While I'm at it, please note that by submitting a question or comment you're agreeing to my terms of service, which are: you relinquish any subsequent rights of ownership to your material by submitting it on this site.









Uniblue: Free Virus Scan


Join Me At:
Aloha Social Media Summit, Oct 2008


Search
Find just the answers you seek from among our 1700+ free tech support articles by using our Lijit search engine.


Help!





Subscribe to
Ask Dave Taylor!

Add to Google Reader
Add to My Yahoo!
Subscribe in NewsGator Online

RDF   XML

Free Updates!
Sign up and get free weekly updates and special offers on books, seminars, workshops and more.


Recent Entries
Join the List!
Join my author info mailing list, where you'll learn about my upcoming books, speaking gigs, and more!


Book Links
© 2002 - 2008 by Dave Taylor. All Rights Reserved.

Note: This web site is for the purpose of disseminating information for educational purposes, free of charge, for the benefit of all visitors. We take great care to provide quality information. However, we do not guarantee, and accept no legal liability whatsoever arising from or connected to, the accuracy, reliability, currency or completeness of any material contained on this web site or on any linked site.

[whiteboard marker tray]