Dave Taylor answers free tech support questions about a wide variety of business and technical topics, including blogging, iphone help, ipod help, AdSense, MySpace, Sony PSP help, Mp3 players, Windows XP, Windows Vista, Linux, SEO, Mac OS X, Facebook, Twitter and LinkedIn.

How can I unwrap or decode Web URL redirects?

I'm frustrated as heck! I clicked on a URL and next thing I knew I was on a completly different site that was quite, um, inappropriate for work. I know how to write a redirect in my own script, but I am wondering: how can I unwrap, decode, slow down or trace a sequence of redirects so I can see what's going on? If it helps, I have a Linux system I can use for the task.


Dave's Answer:

That does help, actually. In fact, the easiest way I know to understand a sequence of redirects is from the Linux (or Mac OS X) command line, using the splendid, powerful curl utility.

If you dig into it -- type "man curl" on the command line -- you'll find that it has about a zillion different things it can do that encompass not only Web site interaction and form hacking, but also FTP and more. Here's the description from the author:

"curl is a tool to transfer data from or to a server, using one of the supported protocols (HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, DICT, TELNET, LDAP or FILE). The command is designed to work without user interaction.

curl offers a busload of useful tricks like proxy support, user authentication, ftp upload, HTTP post, SSL connections, cookies, file transfer resume and more. As you will see below, the amount of features will make your head spin!"
How can you not love that? :-)

To trace a series of redirects what you want to see are the http header sections. You can get that with the -I or --head flag. Add -L or --location for location related information (redirects) and you're good to go.

Here, again, is a quote from the man page: "(HTTP/HTTPS) If the server reports that the requested page has moved to a different location (indicated with a Location: header and a 3XX response code) this option will make curl redo the request on the new place. If used together with -i/--include or -I/--head, headers from all requested pages will be shown."

So that's what we'll do. To test it, I'll use a redirect of my own, which you can first experience by clicking on http://dtlikes.com/stomp. Now, here's what happens when I ask "curl" to unwrap it:

$ curl -I -L http://dtlikes.com/stomp
HTTP/1.1 301 Moved Permanently
Date: Wed, 03 Sep 2008 18:45:46 GMT
Server: Apache
Location: http://dtlikes.com/stomp/
Connection: close
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 302 Found
Date: Wed, 03 Sep 2008 18:45:47 GMT
Server: Apache
Location: https://member.stompernet.net/?r=26&i=68
Connection: close
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 302 Found
Date: Wed, 03 Sep 2008 18:45:56 GMT
Server: Apache/2.2.3 (Red Hat)
X-Powered-By: PHP/5.2.6
Set-Cookie: PHPSESSID=hrjve2g3si1279hc5am49154r7; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: aff_id=114; expires=Sat, 03-Sep-2011 18:45:56 GMT; path=/; domain=.stompernet.net
Set-Cookie: tron_hash=7b2934d24a; expires=Sat, 03-Sep-2011 18:45:56 GMT; path=/; domain=.stompernet.net
Set-Cookie: aff_id=26; expires=Sat, 03-Sep-2011 18:45:56 GMT; path=/; domain=.stompernet.net
location: http://stse2.stompernet.net
Content-Type: text/html; charset=UTF-8

HTTP/1.1 200 OK
Date: Wed, 03 Sep 2008 18:46:04 GMT
Server: Apache/2.2.3 (Red Hat)
Last-Modified: Wed, 03 Sep 2008 16:55:47 GMT
ETag: "f8851a-3cee-b5354ac0"
Accept-Ranges: bytes
Content-Length: 15598
Content-Type: text/html; charset=UTF-8

Phew! That's a lot of data, but when you look through it you can see that the original URL bounces you to an affiliate link, https://member.stompernet.net/?r=26&i=68, which then bounces you one more time, to "http://stse2.stompernet.net".

I hope that'll help you unwind what's going on with the other site you encountered!



Help others find this article at Del.icio.us, Digg, Netscape, Reddit, and Stumble Upon    

Subscribe!

Never miss another useful Q&A article again! Subscribe to AskDaveTaylor with Google Reader.

Comments
Rather amazingly, there are no comments on this article yet.


I have a lot to say, but ...
Starbucks coffee cup I have a lot to say, and questions of my own for that matter, but most of all I'd like to say thank you for all your efforts on this Web site by buying you a chai!

I do have a comment, now that you mention it!









Remember personal info?


Please note that I will never send you any unsolicited commercial email. Ever.

While I'm at it, please note that by submitting a question or comment you're agreeing to my terms of service, which are: you relinquish any subsequent rights of ownership to your material by submitting it on this site.









Uniblue: Free Virus Scan

Search
Find just the answers you seek from among our 1700+ free tech support articles by using our Lijit search engine.


Member of the B5Media Network

Help!





Subscribe to
Ask Dave Taylor!

Add to Google Reader
Add to My Yahoo!
Subscribe in NewsGator Online

RDF   XML

Free Updates!
Sign up and get free weekly updates and special offers on books, seminars, workshops and more.


Recent Entries
Join the List!
Join my author info mailing list, where you'll learn about my upcoming books, speaking gigs, and more!


Book Links
© 2002 - 2008 by Dave Taylor. All Rights Reserved.

Note: This web site is for the purpose of disseminating information for educational purposes, free of charge, for the benefit of all visitors. We take great care to provide quality information. However, we do not guarantee, and accept no legal liability whatsoever arising from or connected to, the accuracy, reliability, currency or completeness of any material contained on this web site or on any linked site.

[whiteboard marker tray]