|
|
How do I strip leading zeroes for math in a shell script?I have a file containing lines of data that are amounts padded with leading zeros, similar to the snip below. 0000000004 Using a shell script, how can I add up a column of numbers contained in lines when BASH interprets the numbers as octal when I do in-line math? What an interesting puzzle you present to me! My first thought was that there's a cool way I can solve this using a shell function, one that stripped a single leading zero then compared its results to the pre-truncated version of the value, until they matched (e.g., all the leading zeroes were deleted). It'd look something like this: function stripzeroes This would then be called as stripzeroes "0000000003434" (or whatever value you'd read in from the data file) and the result would be returned as the value newvalue without the leading zeroes. Nice solution, classic little shell script function, but there's one problem. With the right regular expression, you can strip all the leading zeroes from your data fields with just a few characters: valueWithoutZeroes="$(echo $valueWithZeroes | sed 's/0*//')" That's right, but using the proper regular expression to the sed command, you can very easily strip off all the leading zeroes, making your math far, far simpler and all without complicating your shell script with useless functions! Hope that helps you out.
More Useful Shell Script Programming Articles:
✔ Secretly capture screenshots on my Mac?
When I used to work on a Linux system, there was a utility we had that would let me take screen captures every...
✔ Parsing "id" strings in a Shell Script?Hello Dave. I need a Bash shell script that creates a directories with the group names automatically when user logs in to the...
✔ Copy and Paste from the Mac OS X Command Line?I am constantly running commands in Terminal.app on my MacBook and then copying and pasting the results into email messages or documents. Yes,...
✔ Script to test line lengths for Twitter compatibility?I've been tasked with writing a series of tweets for a Black Friday marketing campaign and am finding it a bit tricky because...
✔ Shell script to convert lowercase to title case?As part of a project I'm working on, I find myself deep in a Linux shell script, needing to have a subroutine that...
Let's stay in touch!
Sign up for my weekly AskDaveTaylor Newsletter and you'll receive even more tech and gadget help
right to your inbox, along with exclusive news and industry updates. It's good stuff. I promise!
Categorized:
Shell Script Programming
(Article 6487,
Written by Dave Taylor)
Tagged: shell script programming, unix programming, wicked cool shell scripts Previous: How do I get a cool MySpace URL? Next: How do I configure a DMZ on my local LAN? Reader Comments To Date: 20Dave Taylor said, on April 28, 2006 12:44 AM:
Darn, you're right, Dave! Using the 'sed' solution, simply add a test after the conditional: if [ -z "$valueWithoutZeroes" ] ; then that should do the trick! Alejandro Biasin said, on November 2, 2006 12:04 PM:
I figure out a much simpler way of doing it: newValue=`expr $oldValue + 0` Note that this only works with "data that are amounts", as the question says. Garrett Nievin said, on January 7, 2007 12:59 PM:
Here's my usual solution, pretty much equivalent to Alejandro's use of expr:
Ben said, on April 26, 2007 8:58 AM:
I'm fairly new to this and I do follow the logic. However, I have problems trying to tie it all up together. For instance I have a file 'benshrs' with figures I need to add up in columns 21 thru 27. Some are all zeroes. What's the best way (complete script)that combines a. the cut -c 21-27 b. strip the leading zeroes except when it is all zeroes, c. add up the figures in do ... done loop, and lastly print the final sum. My crude attempt (below) has failed miserably! Dave Taylor said, on May 7, 2007 7:55 PM:
I'm not surprised this isn't working, Ben: you can't use floating point / real numbers in a shell script, so even your initial assignment of hrs_total=0.00 is going to fail before you even get into the for loop. It's possible that you need to do this in awk or, better, Perl, to get it working. curleb said, on May 25, 2007 1:55 PM:
Not that your option wouldn't work, Dave, but I'm curious if it isn't a bit of an overkill solution. Why not simply employ something like: $ typeset -i example1="00001" This approach is available to both ksh and bash, and would immediately strip out the zeroes or otherwise negate the value to "0" if there were some alpha values. Same applies to a fully zero-filled value. This eliminates the need for any additional function definitions at all...but is this a matter for portability? Sure it restricts the math to integers, but sed would totally skew the numbers if it was used to massage the value. [Your page doesn't seem to restrict the discussion to a particular shell that I can see.] Dave Taylor said, on May 25, 2007 8:13 PM:
Cool. Never knew about the "typeset" built-in to the shell. Very interesting solution, thanks! Alexandre H said, on October 3, 2007 9:02 AM:
The typeset -i (or declare -i in more recent versions of bash) doesn't work if the value is greater than 7 -- it's interpreted as an octal number. $ declare -i x=009 The most reliable solution so far is using expr + 0 I think. Cephi said, on September 8, 2008 4:51 PM:
if the variables are all of the same length, then: (example: variables with eight characters, var1=00005555) var1=$((1$var1-100000000)) Now var1 is 5555. Easy peasy gumdrops. jesse said, on December 5, 2008 2:20 PM:
valueWithoutZeroes=`echo ${valueWithZeroes#0}` Erik De Sonville said, on February 15, 2009 4:14 AM:
To remove leading zeroes, while keeping the last one for an all-zero input string: sed 's/^0*//;s/^$/0/' Use it e.g. as As a side-effect (or rather as interesting advantage), an empty (null) input string is also converted into a single zero. 123456 -> 123456 Ryan B. Lynch said, on June 25, 2009 10:58 AM:
Seems like your (Dave) solutions and the comments fall into one of two categories: Heavyweight solutions that rely on external tools (bc, awk, sed), or incomplete solutions (try Jesse's with multiple leading zeros). In pure Bourne (/bin/sh), I don't actually have an answer. But BASH itself has enough functionality to handle this without *any* external programs: `shopt -s extglob There are two advanced BASH tricks, here. First, `shopt -s extglob` turns on BASH's "extended globbing" functionality ('-u' turns it off). In addition to the usual BASH globbing of wildcards, you can specify '+(0)' to indicate "a sequence one or more '0' characters". (More info here: http://aplawrence.com/Words2005/2005_05_25.html) Second, the '${my_var/#+(0)/}' construction, which applies BASH's string manipulation operators. '${var/seq/rep}' takes the value of the variable '$var' and replaces any instances of the literal string 'seq' with the literal string 'rep'. I added that '#' character, which restricts the match-replacement to the beginning of the string, only. I get the impression that almost nobody knows about and uses BASH's string manipulation and extended globbing functionality. They're not intuitive to me, so I generally check the docs when using them. Still, I'd prefer this to 'echo'ing the original string to 'sed', and using a regex substition. (Coincidentally, that's how I'd handle this, if I couldn't use the BASH techniques I've outlined, here.) Peter said, on January 11, 2010 6:52 AM:
I've just written my own SED command for this, which is slightly purer than Erik's one in that it will leave an empty string as is: sed 's/^0\+\([0-9]\)/\1/' The trick here is to check that there is at least one digit after the zeros, and put that back into the output. (The bracketed expression becomes \1 in the replace pattern.) Antriksh Pany said, on March 9, 2010 10:24 AM:
Interesting set of comments.. fairly wide array of solutions. This is what I tried and got to work: ['a' is the file with the numbers.] However, I don't know why I had to do the echo of the entire for loop output. I would have thought that the following would also work: Any suggestions? PowerPaul said, on August 25, 2010 4:57 AM:
Instead of manipulating the actual number by stripping the leading zeros you can force bash to use decimal representation by using: $ echo $[000000000200-1] jice said, on November 14, 2010 12:08 PM:
indeed I'm using Raj Singh said, on December 20, 2011 4:03 AM:
The answer is rather a single line of code. For e.g. echo "1000000001111111" | tr -s [0-9] would squeez all the repeating numbers in the string. Fred said, on January 3, 2012 1:47 PM:
(( var1 = $var + 0 )) just another way to do it Zoran said, on March 2, 2012 1:24 AM:
I have used this way: Fred's and RAj Singh's way is more elegant although :)
I do have a comment, now that you mention it!Check This Out Too... |
Recent Entries
Look for Answers
Recommended
All Our Categories
Apple iPad Help
Articles and Reviews Auctions and Online Shopping Blogs and Blogging Building Web Site Traffic Business and Management Computer and Internet Basics d) None of the Above Facebook Help Google Gmail Help Google Plus Help HTML, JavaScript and Web Site Programming Industry News and Trade Shows iPhone and Cell Phone Help iPod, Sony PSP and MP3 Player Help Kindle Fire Help Mac OS X Help Pay Per Click (PPC) Advertising Pinterest Help Search Engine Optimization (SEO) Shell Script Programming Tech Support Video Help The Writing Business Twitter, LinkedIn and Social Network Help Unix and Linux Help Video Game Tips and Help Windows PC Help Find Me on Google+ ADT on G+ |
Not to be confused with this Dave Taylor, I'm the Dave Taylor from the game industry, and I'm visiting because my uncle Paul Taylor thought I might have been doing this as I share your penchant for answering things.
It's a neat site, and I think you're providing a darned handy service, btw.
I think there's a small bug with both solutions. You've got an issue if one of the lines reads thusly:
0000000000
You'd want to strip that down to just "0", but both algorithms you use basically delete the line, which could cause some pretty serious issues, depending on what's eating the output.