Go to the first, previous, next, last section, table of contents.
Write documentation as if whoever reads it is a violent psychopath who knows where you live.
Steve English, as quoted by Peter Langston
This major node discusses advanced features in @command{gawk}. It's a bit of a "grab bag" of items that are otherwise unrelated to each other. First, a command-line option allows @command{gawk} to recognize non-decimal numbers in input data, not just in @command{awk} programs. Next, two-way I/O, discussed briefly in earlier parts of this Info file, is described in full detail, along with the basics of TCP/IP networking and BSD portal files. Finally, @command{gawk} can profile an @command{awk} program, making it possible to tune it for performance.
@ref{Dynamic Extensions, ,Adding New Built-in Functions to @command{gawk}}, discusses the ability to dynamically add new built-in functions to @command{gawk}. As this feature is still immature and likely to change, its description is relegated to an appendix.
If you run @command{gawk} with the @option{--non-decimal-data} option, you can have non-decimal constants in your input data:
$ echo 0123 123 0x123 |
> gawk --non-decimal-data '{ printf "%d, %d, %d\n",
> $1, $2, $3 }'
-| 83, 123, 291
For this feature to work, write your program so that @command{gawk} treats your data as numeric:
$ echo 0123 123 0x123 | gawk '{ print $1, $2, $3 }'
-| 0123 123 0x123
The print statement treats its expressions as strings.
Although the fields can act as numbers when necessary,
they are still strings, so print does not try to treat them
numerically. You may need to add zero to a field to force it to
be treated as a number. For example:
$ echo 0123 123 0x123 | gawk --non-decimal-data '
> { print $1, $2, $3
> print $1 + 0, $2 + 0, $3 + 0 }'
-| 0123 123 0x123
-| 83 123 291
Because it is common to have decimal data with leading zeros, and because using it could lead to surprising results, the default is to leave this facility disabled. If you want it, you must explicitly request it.
Caution:
Use of this option is not recommended.
It can break old programs very badly.
Instead, use the strtonum function to convert your data
(see section Octal and Hexadecimal Numbers).
This makes your programs easier to write and easier to read, and
leads to less surprising results.
From: brennan@whidbey.com (Mike Brennan) Newsgroups: comp.lang.awk Subject: Re: Learn the SECRET to Attract Women Easily Date: 4 Aug 1997 17:34:46 GMT Message-ID: <5s53rm$eca@news.whidbey.com> On 3 Aug 1997 13:17:43 GMT, Want More Dates??? <tracy78@kilgrona.com> wrote: >Learn the SECRET to Attract Women Easily > >The SCENT(tm) Pheromone Sex Attractant For Men to Attract Women The scent of awk programmers is a lot more attractive to women than the scent of perl programmers. -- Mike Brennan
It is often useful to be able to send data to a separate program for processing and then read the result. This can always be done with temporary files:
# write the data for processing
tempfile = ("/tmp/mydata." PROCINFO["pid"])
while (not done with data)
print data | ("subprogram > " tempfile)
close("subprogram > " tempfile)
# read the results, remove tempfile when done
while ((getline newdata < tempfile) > 0)
process newdata appropriately
close(tempfile)
system("rm " tempfile)
This works, but not elegantly.
Starting with version 3.1 of @command{gawk}, it is possible to open a two-way pipe to another process. The second process is termed a coprocess, since it runs in parallel with @command{gawk}. The two-way connection is created using the new `|&' operator (borrowed from the Korn Shell, @command{ksh}):(47).}
do {
print data |& "subprogram"
"subprogram" |& getline results
} while (data left to process)
close("subprogram")
The first time an I/O operation is executed using the `|&'
operator, @command{gawk} creates a two-way pipeline to a child process
that runs the other program. Output created with print
or printf is written to the program's standard input, and
output from the program's standard output can be read by the @command{gawk}
program using getline.
As is the case with processes started by `|', the subprogram
can be any program, or pipeline of programs, that can be started by
the shell.
There are some cautionary items to be aware of:
getline in order to read
the coprocess's results. This could lead to a situation
known as deadlock, where each process is waiting for the
other one to do something.
It is possible to close just one end of the two-way pipe to
a coprocess, by supplying a second argument to the close
function of either "to" or "from"
(see section Closing Input and Output Redirections).
These strings tell @command{gawk} to close the end of the pipe
that sends data to the process or the end that reads from it,
respectively.
This is particularly necessary in order to use the system @command{sort} utility as part of a coprocess; @command{sort} must read all of its input data before it can produce any output. The @command{sort} program does not receive an end-of-file indication until @command{gawk} closes the write end of the pipe.
When you have finished writing data to the @command{sort}
utility, you can close the "to" end of the pipe, and
then start reading sorted data via getline.
For example:
BEGIN {
command = "LC_ALL=C sort"
n = split("abcdefghijklmnopqrstuvwxyz", a, "")
for (i = n; i > 0; i--)
print a[i] |& command
close(command, "to")
while ((command |& getline line) > 0)
print "got", line
close(command)
}
This program writes the letters of the alphabet in reverse order, one per line, down the two-way pipe to @command{sort}. It then closes the write end of the pipe, so that @command{sort} receives an end-of-file indication. This causes @command{sort} to sort the data and write the sorted data back to the @command{gawk} program. Once all of the data has been read, @command{gawk} terminates the coprocess and exits.
As a side note, the assignment `LC_ALL=C' in the @command{sort} command ensures traditional Unix (ASCII) sorting from @command{sort}.
EMISTERED: A host is a host from coast to coast,
and no-one can talk to host that's close,
unless the host that isn't close
is busy hung or dead.
In addition to being able to open a two-way pipeline to a coprocess on the same system (see section Two-Way Communications with Another Process), it is possible to make a two-way connection to another process on another system across an IP networking connection.
You can think of this as just a very long two-way pipeline to a coprocess. The way @command{gawk} decides that you want to use TCP/IP networking is by recognizing special file names that begin with `/inet/'.
The full syntax of the special file name is `/inet/protocol/local-port/remote-host/remote-port'. The meaning of the components are:
getservbyname function.
Consider the following very simple example:
BEGIN {
Service = "/inet/tcp/0/localhost/daytime"
Service |& getline
print $0
close(Service)
}
This program reads the current date and time from the local system's TCP `daytime' server. It then prints the results and closes the connection.
Because this topic is extensive, the use of @command{gawk} for TCP/IP programming is documented separately. @ifnotinfo See TCP/IP Internetworking with @command{gawk}, which comes as part of the @command{gawk} distribution, for a much more complete introduction and discussion, as well as extensive examples.
Similar to the `/inet' special files, if @command{gawk}
is configured with the @option{--enable-portals} option
(@pxref{Quick Installation, , Compiling @command{gawk} for Unix}),
then @command{gawk} treats
files whose pathnames begin with /p as 4.4 BSD-style portals.
When used with the `|&' operator, @command{gawk} opens the file for two-way communications. The operating system's portal mechanism then manages creating the process associated with the portal and the corresponding communications with the portal's process.
Beginning with version 3.1 of @command{gawk}, you may produce execution traces of your @command{awk} programs. This is done with a specially compiled version of @command{gawk}, called @command{pgawk} ("profiling @command{gawk}").
@command{pgawk} is identical in every way to @command{gawk}, except that when it has finished running, it creates a profile of your program in a file named `awkprof.out'. Because it is profiling, it also executes up to 45 percent slower than @command{gawk} normally does.
As shown in the following example, the @option{--profile} option can be used to change the name of the file where @command{pgawk} will write the profile:
$ pgawk --profile=myprog.prof -f myprog.awk data1 data2
In the above example, @command{pgawk} places the profile in `myprog.prof' instead of in `awkprof.out'.
Regular @command{gawk} also accepts this option. When called with just @option{--profile}, @command{gawk} "pretty prints" the program into `awkprof.out', without any execution counts. You may supply an option to @option{--profile} to change the file name. Here is a sample session showing a simple @command{awk} program, its input data, and the results from running @command{pgawk}. First, the @command{awk} program:
BEGIN { print "First BEGIN rule" }
END { print "First END rule" }
/foo/ {
print "matched /foo/, gosh"
for (i = 1; i <= 3; i++)
sing()
}
{
if (/foo/)
print "if is true"
else
print "else is true"
}
BEGIN { print "Second BEGIN rule" }
END { print "Second END rule" }
function sing( dummy)
{
print "I gotta be me!"
}
Following is the input data:
foo bar baz foo junk
Here is the `awkprof.out' that results from running @command{pgawk} on this program and data. (This example also illustrates that @command{awk} programmers sometimes have to work late.):
# gawk profile, created Sun Aug 13 00:00:15 2000
# BEGIN block(s)
BEGIN {
1 print "First BEGIN rule"
1 print "Second BEGIN rule"
}
# Rule(s)
5 /foo/ { # 2
2 print "matched /foo/, gosh"
6 for (i = 1; i <= 3; i++) {
6 sing()
}
}
5 {
5 if (/foo/) { # 2
2 print "if is true"
3 } else {
3 print "else is true"
}
}
# END block(s)
END {
1 print "First END rule"
1 print "Second END rule"
}
# Functions, listed alphabetically
6 function sing(dummy)
{
6 print "I gotta be me!"
}
The previous example illustrates many of the basic rules for profiling output. The rules are as follows:
BEGIN rule,
pattern/action rules, END rule and functions, listed
alphabetically.
Multiple BEGIN and END rules are merged together.
if-else statement shows how many times
the condition was tested.
To the right of the opening left brace for the if's body
is a count showing how many times the condition was true.
The count for the else
indicates how many times the test failed.
for
or while) shows how many times the loop test was executed.
(Because of this, you can't just look at the count on the first
statement in a rule to determine how many times the rule was executed.
If the first statement is a loop, the count is misleading.)
function
keyword indicates how many times the function was called.
The counts next to the statements in the body show how many times
those statements were executed.
if, else, or loop is only a single statement.
print
and printf only when
the print or printf statement is followed by a redirection.
Similarly, if
the target of a redirection isn't a scalar, it gets parenthesized.
BEGIN and END rules,
the pattern/action rules, and the functions.
The profiled version of your program may not look exactly like what you
typed when you wrote it. This is because @command{pgawk} creates the
profiled version by "pretty printing" its internal representation of
the program. The advantage to this is that @command{pgawk} can produce
a standard representation. The disadvantage is that all source code
comments are lost, as are the distinctions among multiple BEGIN
and END rules. Also, things such as:
/foo/
come out as:
/foo/ {
print $0
}
which is correct, but possibly surprising.
Besides creating profiles when a program has completed, @command{pgawk} can produce a profile while it is running. This is useful if your @command{awk} program goes into an infinite loop and you want to see what has been executed. To use this feature, run @command{pgawk} in the background:
$ pgawk -f myprog & [1] 13992
The shell prints a job number and process ID number, in this case, 13992.
Use the @command{kill} command to send the USR1 signal
to @command{pgawk}:
$ kill -USR1 13992
As usual, the profiled version of the program is written to `awkprof.out', or to a different file if you use the @option{--profile} option.
Along with the regular profile, as shown earlier, the profile includes a trace of any active functions:
# Function Call Stack: # 3. baz # 2. bar # 1. foo # -- main --
You may send @command{pgawk} the USR1 signal as many times as you like.
Each time, the profile and function call trace are appended to the output
profile file.
If you use the HUP signal instead of the USR1 signal,
@command{pgawk} produces the profile and the function call trace, and then exits.
Go to the first, previous, next, last section, table of contents.