How can I generate reports in Linux with an Awk script?

Though I realize that Perl is far more popular nowadays, I'm an old-time Unix guy, so I really still have a soft spot for awk, and am always glad to get a question about how to accomplish something in this simple, lightweight, and powerful scripting language. In a lot of ways, awk is really the perfect sidekick to shell script programming too, and in my book Wicked Cool Shell Scripts I delve into the world of awk more than once.

First off, to clear up a common misconception, it's not called awk as a shorthand for "awkward"! In fact, the program gets its name from the three programmers who wrote it: Alfred Aho, Brian Kernighan, and Peter Weinberger, all from AT&T Bell Telephone Labs at the time.

You were kindly enough to send along a snippet of your data file, so let's start by having a look:

Department: Undefined                
 Ext. 5648                          
 10/10  3:18p 014153               INCOMING      312-555-2922 IL 0:00:24   0.00
 10/30 12:10p 014148               INCOMING      773-555-7473 IL 0:03:43   0.00
 Totals for ext. 5648:  2 call(s).  0 hours, 4 min.   $0.00
 Ext. 5724                          
 Totals for ext. 5724:  0 call(s).  0 hours, 0 min.   $0.00
 Dept. Undefined had 2 call(s).
 Totals: 0 hours, 4 min.  $0.00   Averages: 2 min., 3 sec.  $0.00
Department: ACCOUNTING               
 Ext. 5590 ORTIZ, RON               
 10/05  9:38a 012144               LOCAL             555-7800 IL 0:01:13   0.00
 10/05 10:38a 014157               INCOMING      618-555-6768 IL 0:04:57   0.00
 10/05  2:44p 010108               LOCAL         630-555-8200 IL 0:01:00   0.00
....
 10/26  4:30p 010112               LOCAL         708-555-6436 IL 0:00:00   0.00
 Totals for ext. 5592:  25 call(s).  0 hours, 39 min.   $0.00
 Dept. ACCOUNTING had 35 call(s).
 Totals: 0 hours, 59 min.  $0.00   Averages: 1 min., 41 sec.  $0.00
Department: ACCOUNTS RECEIVABLE      
 Ext. 5214 BIGA, PAT                
 10/03 11:30a 010113               LONGDIST      805-555-5100 CA 0:00:30   0.00
 10/03  3:38p 010101               LONGDIST      815-555-7091 IL 0:00:10   0.00
 10/03  4:09p 010120               LONGDIST      617-555-7449 MA 0:00:55   0.00
 10/04  9:43a 014151               INCOMING      815-555-7769 IL 0:02:09   0.00
 10/04 11:14a 010114               LONGDIST      617-555-7449 MA 0:01:12   0.00

What you need to do is split this one data file into a set of separate information files, one per department, where the departments are delineated by the line Department: xxx, where 'xxx' is the name of the department.

This is surprisingly simple to accomplish in awk once you have a familiarity with the language. The key is to realize that you can redirect the output of a script by simply using the shell-like > symbol. Here's my entire script, saved as "test.awk":

{  if ( $1 == "Department:" ) {

     if (NF > 3) {
       department = $2"-"$3
     } else {
       department = $2
     }

     print "Starting new file for Department " department
     outputfile=department ".dat"
     print $0 > outputfile

   }
   else { 
     print $0 > outputfile
   }

}

My code tends to be very open, with lots of white space for readability, but you really could trim this down to four or five lines if you preferred.

The trick here is that we check the first word of every line for the word "Department:" and when that's matched, we save the new department name in variable department and then tack on a ".dat" suffix to create the output file name.

Run this on the sample data file and in less than a second the task is accomplished:

$ awk -f test.awk < SampleInput
Starting new file for Department Undefined
Starting new file for Department ACCOUNTING
Starting new file for Department ACCOUNTS-RECEIVABLE

That should help you move along in the right direction.

In terms of your confusion with the O'Reilly book, I strongly encourage you to write to the author and publisher detailing why you find the book confusing, so they can fix it in the next edition.