|
|
How can I unwrap or decode Web URL redirects?I'm frustrated as heck! I clicked on a URL and next thing I knew I was on a completly different site that was quite, um, inappropriate for work. I know how to write a redirect in my own script, but I am wondering: how can I unwrap, decode, slow down or trace a sequence of redirects so I can see what's going on? If it helps, I have a Linux system I can use for the task. That does help, actually. In fact, the easiest way I know to understand a sequence of redirects is from the Linux (or Mac OS X) command line, using the splendid, powerful curl utility. If you dig into it -- type "man curl" on the command line -- you'll find that it has about a zillion different things it can do that encompass not only Web site interaction and form hacking, but also FTP and more. Here's the description from the author: "curl is a tool to transfer data from or to a server, using one of the supported protocols (HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, DICT, TELNET, LDAP or FILE). The command is designed to work without user interaction.How can you not love that? :-) To trace a series of redirects what you want to see are the http header sections. You can get that with the -I or --head flag. Add -L or --location for location related information (redirects) and you're good to go. Here, again, is a quote from the man page: "(HTTP/HTTPS) If the server reports that the requested page has moved to a different location (indicated with a Location: header and a 3XX response code) this option will make curl redo the request on the new place. If used together with -i/--include or -I/--head, headers from all requested pages will be shown." So that's what we'll do. To test it, I'll use a redirect of my own, which you can first experience by clicking on http://dtlikes.com/stomp. Now, here's what happens when I ask "curl" to unwrap it: $ curl -I -L http://dtlikes.com/stomp Phew! That's a lot of data, but when you look through it you can see that the original URL bounces you to an affiliate link, https://member.stompernet.net/?r=26&i=68, which then bounces you one more time, to "http://stse2.stompernet.net".
HTTP/1.1 301 Moved Permanently Date: Wed, 03 Sep 2008 18:45:46 GMT Server: Apache Location: http://dtlikes.com/stomp/ Connection: close Content-Type: text/html; charset=iso-8859-1 HTTP/1.1 302 Found HTTP/1.1 302 Found HTTP/1.1 200 OK I hope that'll help you unwind what's going on with the other site you encountered!
More Useful HTML, JavaScript and Web Site Programming Articles:
✔ How to Create Predefined Google Image Search Links?
Thanks for the Amazon URL [see Creating Amazon Search Links]. That worked beautifully. In fact, I sent you $5.00 for coffee in thanks....
✔ Can I embed a Facebook search box on my blog site?I've seen your articles about how to add a Twitter or Google search box on a Web page, but I have a tougher...
✔ Can I use CSS for drop shadows on my blog?I want to give my site a bit of a facelift and add some neat graphical elements. One of which is drop shadows....
✔ How can I embed interactive photo panoramas on my site/blog?I read through your blog entry about how to take panoramic photos with iOS 6 and an iPhone 5 and got enthused. I've...
✔ How can I create a Twitter search URL shortcut?I'd like to add a few Twitter search links to my Web site. Is that possible, or does Twitter prohibit this sort of...
Let's stay in touch!
Sign up for my weekly AskDaveTaylor Newsletter and you'll receive even more tech and gadget help
right to your inbox, along with exclusive news and industry updates. It's good stuff. I promise!
Categorized:
HTML, JavaScript and Web Site Programming
,
Unix and Linux Help
(Article 8487,
Written by Dave Taylor)
Tagged: 301 redirect, 302 redirect, Linux, stompernet, url redirection, web server hacking Previous: How can I learn how to use LinkedIn without leaving my house? Next: Report from the floor of the CEDIA trade show Reader Comments To Date: 2Peter Sundstrom said, on September 24, 2012 9:29 PM:
Using curl is the best command line tool for the job, but I did come across http://redirectdetective.com/ that gives you a graphical representation of all the redirect hops.
I do have a comment, now that you mention it!Check This Out Too... |
Recent Entries
Look for Answers
Recommended
All Our Categories
Apple iPad Help
Articles and Reviews Auctions and Online Shopping Blogs and Blogging Building Web Site Traffic Business and Management Computer and Internet Basics d) None of the Above Facebook Help Google Gmail Help Google Plus Help HTML, JavaScript and Web Site Programming Industry News and Trade Shows iPhone and Cell Phone Help iPod, Sony PSP and MP3 Player Help Kindle Fire Help Mac OS X Help Pay Per Click (PPC) Advertising Pinterest Help Search Engine Optimization (SEO) Shell Script Programming Tech Support Video Help The Writing Business Twitter, LinkedIn and Social Network Help Unix and Linux Help Video Game Tips and Help Windows PC Help Find Me on Google+ ADT on G+ |
Sweet tip Dave, thanks,)))
I just used curl to de-obfuscate an http://relay.radioreference.com/321003334 uri back to the source provider at liveatc.com
Had to read through the man page and add a few params...
-A to spoof a qualifying user agent string
-H to include the Accept header
--trace-ascii to capture the header info and content
but all together, this solution worked perfectly; I'm going to be all over curl from now on!!11
Thanks again,
Buffalo