I’m trying to write an awk script that will take a text file phone report from a windows box and break it up into smaller text files. I know it’ll only take a few minutes, but I’m not all that familiar with AWK and slogging through this O’Reilly book is really starting to tick me off. Any chance you can help?
Though I realize that Perl is far more popular nowadays, I’m an old-time Unix guy, so I really still have a soft spot for awk, and am always glad to get a question about how to accomplish something in this simple, lightweight, and powerful scripting language. In a lot of ways, awk is really the perfect sidekick to shell script programming too, and in my book Wicked Cool Shell Scripts I delve into the world of awk more than once.
First off, to clear up a common misconception, it’s not called awk as a shorthand for “awkward”! In fact, the program gets its name from the three programmers who wrote it: Alfred Aho, Brian Kernighan, and Peter Weinberger, all from AT&T Bell Telephone Labs at the time.
You were kindly enough to send along a snippet of your data file, so let’s start by having a look:
Department: Undefined Ext. 5648 10/10 3:18p 014153 INCOMING 312-555-2922 IL 0:00:24 0.00 10/30 12:10p 014148 INCOMING 773-555-7473 IL 0:03:43 0.00 Totals for ext. 5648: 2 call(s). 0 hours, 4 min. $0.00 Ext. 5724 Totals for ext. 5724: 0 call(s). 0 hours, 0 min. $0.00 Dept. Undefined had 2 call(s). Totals: 0 hours, 4 min. $0.00 Averages: 2 min., 3 sec. $0.00 Department: ACCOUNTING Ext. 5590 ORTIZ, RON 10/05 9:38a 012144 LOCAL 555-7800 IL 0:01:13 0.00 10/05 10:38a 014157 INCOMING 618-555-6768 IL 0:04:57 0.00 10/05 2:44p 010108 LOCAL 630-555-8200 IL 0:01:00 0.00 .... 10/26 4:30p 010112 LOCAL 708-555-6436 IL 0:00:00 0.00 Totals for ext. 5592: 25 call(s). 0 hours, 39 min. $0.00 Dept. ACCOUNTING had 35 call(s). Totals: 0 hours, 59 min. $0.00 Averages: 1 min., 41 sec. $0.00 Department: ACCOUNTS RECEIVABLE Ext. 5214 BIGA, PAT 10/03 11:30a 010113 LONGDIST 805-555-5100 CA 0:00:30 0.00 10/03 3:38p 010101 LONGDIST 815-555-7091 IL 0:00:10 0.00 10/03 4:09p 010120 LONGDIST 617-555-7449 MA 0:00:55 0.00 10/04 9:43a 014151 INCOMING 815-555-7769 IL 0:02:09 0.00 10/04 11:14a 010114 LONGDIST 617-555-7449 MA 0:01:12 0.00
What you need to do is split this one data file into a set of separate information files, one per department, where the departments are delineated by the line Department: xxx, where ‘xxx’ is the name of the department.
This is surprisingly simple to accomplish in awk once you have a familiarity with the language. The key is to realize that you can redirect the output of a script by simply using the shell-like > symbol. Here’s my entire script, saved as “test.awk”:
{ if ( $1 == "Department:" ) {
if (NF > 3) {
department = $2"-"$3
} else {
department = $2
}
print "Starting new file for Department " department
outputfile=department ".dat"
print $0 > outputfile
}
else {
print $0 > outputfile
}
}
My code tends to be very open, with lots of white space for readability, but you really could trim this down to four or five lines if you preferred.
The trick here is that we check the first word of every line for the word “Department:” and when that’s matched, we save the new department name in variable department and then tack on a “.dat” suffix to create the output file name.
Run this on the sample data file and in less than a second the task is accomplished:
$ awk -f test.awk < SampleInput Starting new file for Department Undefined Starting new file for Department ACCOUNTING Starting new file for Department ACCOUNTS-RECEIVABLE
That should help you move along in the right direction.
In terms of your confusion with the O’Reilly book, I strongly encourage you to write to the author and publisher detailing why you find the book confusing, so they can fix it in the next edition.
How could I generate a list of 10 files with just modification at one single place.