How do I strip leading zeroes for math in a shell script?

Not to be confused with this Dave Taylor, I'm the Dave Taylor from the game industry, and I'm visiting because my uncle Paul Taylor thought I might have been doing this as I share your penchant for answering things.

It's a neat site, and I think you're providing a darned handy service, btw.

I think there's a small bug with both solutions. You've got an issue if one of the lines reads thusly:

0000000000

You'd want to strip that down to just "0", but both algorithms you use basically delete the line, which could cause some pretty serious issues, depending on what's eating the output.

Darn, you're right, Dave! Using the 'sed' solution, simply add a test after the conditional:

if [ -z "$valueWithoutZeroes" ] ; then
valueWithoutZeroes="0"
fi

that should do the trick!

I figure out a much simpler way of doing it:

newValue=`expr $oldValue + 0`

Note that this only works with "data that are amounts", as the question says.

Here's my usual solution, pretty much equivalent to Alejandro's use of expr:
newValue=`echo $oldValue | awk '{print $1 + 0}'`
or
newValue=`echo $oldValue | awk '{printf "%d\n", $1}'

I'm fairly new to this and I do follow the logic. However, I have problems trying to tie it all up together. For instance I have a file 'benshrs' with figures I need to add up in columns 21 thru 27. Some are all zeroes. What's the best way (complete script)that combines a. the cut -c 21-27 b. strip the leading zeroes except when it is all zeroes, c. add up the figures in do ... done loop, and lastly print the final sum.

My crude attempt (below) has failed miserably!
hrs_total=0.00
for line_num in `cut -c 21-27 $0`
do
line_hrs= `expr `sed 's/0*//' $line_num` + 0`
(hrs_total=$hrs_total + $line_hrs)
done
hrs_total=`echo "scale=2;$hrs_total/100" | bc`
echo "Hours Total = $hrs_total"

I'm not surprised this isn't working, Ben: you can't use floating point / real numbers in a shell script, so even your initial assignment of hrs_total=0.00 is going to fail before you even get into the for loop. It's possible that you need to do this in awk or, better, Perl, to get it working.

Not that your option wouldn't work, Dave, but I'm curious if it isn't a bit of an overkill solution. Why not simply employ something like:

$ typeset -i example1="00001"
$ typeset -i example2="a0001"
$ typeset -i example3="00000"
$ echo $example1 + 10 |bc
11
$ echo $example2 + 10 |bc
10
$ echo $example3 + 10 |bc
10
$ print $example1
1
$ print $example2
0
$ print $example3
0
===

This approach is available to both ksh and bash, and would immediately strip out the zeroes or otherwise negate the value to "0" if there were some alpha values. Same applies to a fully zero-filled value. This eliminates the need for any additional function definitions at all...but is this a matter for portability? Sure it restricts the math to integers, but sed would totally skew the numbers if it was used to massage the value.

[Your page doesn't seem to restrict the discussion to a particular shell that I can see.]

Cool. Never knew about the "typeset" built-in to the shell. Very interesting solution, thanks!

The typeset -i (or declare -i in more recent versions of bash) doesn't work if the value is greater than 7 -- it's interpreted as an octal number.

$ declare -i x=009
bash: declare: 009: value too great for base (error token is 009)
$ declare -i x=010
$ echo $x
8

The most reliable solution so far is using expr + 0 I think.

if the variables are all of the same length, then:

(example: variables with eight characters, var1=00005555)

var1=$((1$var1-100000000))

Now var1 is 5555. Easy peasy gumdrops.

valueWithoutZeroes=`echo ${valueWithZeroes#0}`

To remove leading zeroes, while keeping the last one for an all-zero input string:

sed 's/^0*//;s/^$/0/'

Use it e.g. as
NEW=$(echo $INPUT | sed 's/^0*//;s/^$/0/')

As a side-effect (or rather as interesting advantage), an empty (null) input string is also converted into a single zero.

123456 -> 123456
000123 -> 123
000000 -> 0
(empty) -> 0

Seems like your (Dave) solutions and the comments fall into one of two categories: Heavyweight solutions that rely on external tools (bc, awk, sed), or incomplete solutions (try Jesse's with multiple leading zeros).

In pure Bourne (/bin/sh), I don't actually have an answer. But BASH itself has enough functionality to handle this without *any* external programs:

`shopt -s extglob
echo -e "${my_var/#+(0)/}"
shopt -u extglob`

There are two advanced BASH tricks, here.

First, `shopt -s extglob` turns on BASH's "extended globbing" functionality ('-u' turns it off). In addition to the usual BASH globbing of wildcards, you can specify '+(0)' to indicate "a sequence one or more '0' characters". (More info here: http://aplawrence.com/Words2005/2005_05_25.html)

Second, the '${my_var/#+(0)/}' construction, which applies BASH's string manipulation operators. '${var/seq/rep}' takes the value of the variable '$var' and replaces any instances of the literal string 'seq' with the literal string 'rep'. I added that '#' character, which restricts the match-replacement to the beginning of the string, only.

I get the impression that almost nobody knows about and uses BASH's string manipulation and extended globbing functionality. They're not intuitive to me, so I generally check the docs when using them.

Still, I'd prefer this to 'echo'ing the original string to 'sed', and using a regex substition. (Coincidentally, that's how I'd handle this, if I couldn't use the BASH techniques I've outlined, here.)

I've just written my own SED command for this, which is slightly purer than Erik's one in that it will leave an empty string as is:

sed 's/^0\+\([0-9]\)/\1/'

The trick here is to check that there is at least one digit after the zeros, and put that back into the output. (The bracketed expression becomes \1 in the replace pattern.)

Interesting set of comments.. fairly wide array of solutions.

This is what I tried and got to work:
echo $(for i in `cat a`; do echo -n $i | sed 's/^0*\([0-9]\)/\1/g'; echo -n " + "; done; echo "0") | bc

['a' is the file with the numbers.]

However, I don't know why I had to do the echo of the entire for loop output. I would have thought that the following would also work:
for i in `cat a`; do echo -n $i | sed 's/^0*\([0-9]\)/\1/g'; echo -n " + "; done; echo "0" | bc

Any suggestions?

Instead of manipulating the actual number by stripping the leading zeros you can force bash to use decimal representation by using:

$ echo $[000000000200-1]
127
$ echo $[10#000000000200-1]
199
echo $[10#000000000200]
200

indeed I'm using
myvalue=$((10#$myvalue))
(see previous comment for explanations)
must be much faster than calling an external program (e.g. sed)

The answer is rather a single line of code.

For e.g. echo "1000000001111111" | tr -s [0-9]

would squeez all the repeating numbers in the string.

(( var1 = $var + 0 ))

just another way to do it