Industry guru Dave Taylor offers tech support on technical and business topics, including iPhone, iPod, Microsoft Windows, Sony PSP, cellphones, online advertising, CSS, Web design, business, Unix, Linux, SEO, Mac OS X, and shell script programming.     


Chopping off the last field of each line?

I'm working on the Web site for Creating Cool Web Sites and just realized that the little Unix trick I used while editing a file is actually a beautiful example of why so many people love Unix so much, so I thought I'd share it. The problem: a file where lines have 1-10 words + a last field value (in this case, a page number) that I don't want. The challenge is to figure out how to easily remove that last field, when there are a variable number of fields on the line.

Dave's Answer:
My first instinct was to turn to awk or perl, but in fact there's a much easier way using basic Unix utilities: rev and cut:
$ rev inputfile | cut -f2- | rev > outputfile
How does this work? The rev command reverses each line of input, so that means that the first field of each now-reversed line is the field that we want to remove. That's easily done with cut with the -f2- flag (that means output field two through the end of the line). Then, finally, we re-reverse (that is, fix) each line and save the output in a new file.

Quickly and easily done.


Related Unix and Linux Help articles:
✔   Copy and Paste from the Mac OS X Command Line?
I am constantly running commands in Terminal.app on my MacBook and then copying and pasting the results into email messages or documents. Yes,...
✔   Shell script to convert lowercase to title case?
As part of a project I'm working on, I find myself deep in a Linux shell script, needing to have a subroutine that...
✔   Can I script renaming files based on an XML data map?
I have a folder full of files which are named with four digits and a file extension e.g. 0312.file and an XML-file describing...
✔   Test for valid numbers in a Bash shell script?
In a different discussion on this site [see Redirecting input in a shell script] a visitor commented that "I was too busy trying...
✔   Review: iSSH for the iPad/iPhone
If you're running an online business like I am, there are times when you need to connect and log in to the server...

Let's stay in touch!
Sign up for my weekly AskDaveTaylor Newsletter and you'll receive even more tech and gadget help right to your inbox, along with exclusive news and industry updates. It's good stuff. I promise!
    Enter your name: and your email addr:  





Categorized: Unix and Linux Help   (Article 3690, Written by )
Tagged:
Previous: How do I tweak my PATH?
Next: Another nice review of Wicked Cool Shell Scripts




Reader Comments To Date: 9

pedestrian said, on August 19, 2007 9:20 AM:

well done

Amit said, on July 7, 2008 12:28 AM:

try this

awk '{$NF=""; print $0}'

Cheers
Amit

porges said, on March 5, 2009 7:50 AM:

I prefer:

awk '{NF--;print}'

With NF="" you will have additional spaces at the end of each line.

Cannon said, on February 13, 2010 2:07 PM:

Nice, this just solved a problem I've been working on. So thanks.

I figured I'd try to make it work without the rev. Reversing the argument of the -f parameter should returned the same results.

Rywin said, on March 25, 2010 9:22 AM:

awk '{NF--;print}' Prints the whole name for me. What am I missing?

awk '{NF="";print}' works, but I need to include the periods.

Sean Meacher said, on January 4, 2011 7:39 AM:

I had to strip the last field from some 2Million line apache logs. First I tried the reverse-cut-reverse approach and that was likely to take 'forever'.
So I found a solution with sed that worked a lot better:
date; time (gzcat infile.gz | sed 's/\(.*\)\ \(.*\)/\1/' | gzip -c - > outfile.gz ); date
What the sed command does is strip 1 parameter to the preceeding space from the end of the line.

Nick said, on January 17, 2011 9:15 AM:

I managed to do this with awk:

cat filename |awk -F/ '{gsub($NF,"");print}'

The gsub replaces the last field with a null string. My delimiter is /

Nick

smilyface said, on July 30, 2011 7:09 AM:

Thanks Nick..


awk -F/ '{gsub($NF,"");print}'

helped me too much.. was searching for this..

I am adding something else

awk -F/ '{gsub($NF,"");sub(".$", "");print}'

to cut the last part with delimiter as /

input
=====
/home/smilyface/aaa/bbb
output
======
/home/smilyface/aaa

Ice said, on November 15, 2012 5:07 PM:

Thanks a lot!

Starbucks coffee cup I do have a lot to say, and questions of my own for that matter, but first I'd like to say thank you, Dave, for all your helpful information by buying you a cup of coffee!

I do have a comment, now that you mention it!











I will never send you any unsolicited email. Ever.






Check This Out Too...

 
Look for Answers
Need Help? Ask Dave Taylor!
Powered By
Linux Journal: Free Issue!


Follow Me on Pinterest

Find Me on Google+
ADT on G+
© 2002 - 2013 by Dave Taylor. All Rights Reserved.

Note: This web site is for the purpose of disseminating information for educational purposes, free of charge, for the benefit of all visitors. We take great care to provide quality information. However, we do not guarantee, and accept no legal liability whatsoever arising from or connected to, the accuracy, reliability, currency or completeness of any material contained on this web site or on any linked site. Further, please note that by submitting a question or comment you're agreeing to my terms of service, which are: you relinquish any subsequent rights of ownership to your material by submitting it on this site. My lawyer says "Thanks".
"Ask Dave Taylor®" is a registered trademark of Intuitive Systems, LLC.