GAWK: Effective AWK Programming

A User's Guide for GNU Awk

Edition 3

March, 2001

This file documents @command{awk}, a program that you can use to select particular records in a file and perform operations upon them.

This is Edition 3 of GAWK: Effective AWK Programming: A User's Guide for GNU Awk, for the 3.1.0 version of the GNU implementation of AWK.

• Foreword
• Preface
  • History of @command{awk} and @command{gawk}
  • A Rose by Any Other Name
  • Using This Book
  • Typographical Conventions
  • The GNU Project and This Book
  • How to Contribute
  • Acknowledgments
• Getting Started with @command{awk}
  • How to Run @command{awk} Programs
    • One-Shot Throw-Away @command{awk} Programs
    • Running @command{awk} Without Input Files
    • Running Long Programs
    • Executable @command{awk} Programs
    • Comments in @command{awk} Programs
    • Shell Quoting Issues
  • Data Files for the Examples
  • Some Simple Examples
  • An Example with Two Rules
  • A More Complex Example
  • @command{awk} Statements Versus Lines
  • Other Features of @command{awk}
  • When to Use @command{awk}
• Regular Expressions
  • How to Use Regular Expressions
  • Escape Sequences
  • Regular Expression Operators
  • Using Character Lists
  • @command{gawk}-Specific Regexp Operators
  • Case Sensitivity in Matching
  • How Much Text Matches?
  • Using Dynamic Regexps
• Reading Input Files
  • How Input Is Split into Records
  • Examining Fields
  • Non-Constant Field Numbers
  • Changing the Contents of a Field
  • Specifying How Fields Are Separated
    • Using Regular Expressions to Separate Fields
    • Making Each Character a Separate Field
    • Setting FS from the Command Line
    • Field Splitting Summary
  • Reading Fixed-Width Data
  • Multiple-Line Records
  • Explicit Input with getline
    • Using getline with No Arguments
    • Using getline into a Variable
    • Using getline from a File
    • Using getline into a Variable from a File
    • Using getline from a Pipe
    • Using getline into a Variable from a Pipe
    • Using getline from a Coprocess
    • Using getline into a Variable from a Coprocess
    • Points About getline to Remember
    • Summary of getline Variants
• Printing Output
  • The print Statement
  • Examples of print Statements
  • Output Separators
  • Controlling Numeric Output with print
  • Using printf Statements for Fancier Printing
    • Introduction to the printf Statement
    • Format-Control Letters
    • Modifiers for printf Formats
    • Examples Using printf
  • Redirecting Output of print and printf
  • Special File Names in @command{gawk}
    • Special Files for Standard Descriptors
    • Special Files for Process-Related Information
    • Special Files for Network Communications
    • Special File Name Caveats
  • Closing Input and Output Redirections
• Expressions
  • Constant Expressions
    • Numeric and String Constants
    • Octal and Hexadecimal Numbers
    • Regular Expression Constants
  • Using Regular Expression Constants
  • Variables
    • Using Variables in a Program
    • Assigning Variables on the Command Line
  • Conversion of Strings and Numbers
  • Arithmetic Operators
  • String Concatenation
  • Assignment Expressions
  • Increment and Decrement Operators
  • True and False in @command{awk}
  • Variable Typing and Comparison Expressions
  • Boolean Expressions
  • Conditional Expressions
  • Function Calls
  • Operator Precedence (How Operators Nest)
• Patterns, Actions, and Variables
  • Pattern Elements
    • Regular Expressions as Patterns
    • Expressions as Patterns
    • Specifying Record Ranges with Patterns
    • The BEGIN and END Special Patterns
      • Startup and Cleanup Actions
      • Input/Output from BEGIN and END Rules
    • The Empty Pattern
  • Using Shell Variables in Programs
  • Actions
  • Control Statements in Actions
    • The if-else Statement
    • The while Statement
    • The do-while Statement
    • The for Statement
    • The break Statement
    • The continue Statement
    • The next Statement
    • Using @command{gawk}'s nextfile Statement
    • The exit Statement
  • Built-in Variables
    • Built-in Variables That Control @command{awk}
    • Built-in Variables That Convey Information
    • Using ARGC and ARGV
• Arrays in @command{awk}
  • Introduction to Arrays
  • Referring to an Array Element
  • Assigning Array Elements
  • Basic Array Example
  • Scanning All Elements of an Array
  • The delete Statement
  • Using Numbers to Subscript Arrays
  • Using Uninitialized Variables as Subscripts
  • Multidimensional Arrays
  • Scanning Multidimensional Arrays
  • Sorting Array Values and Indices with @command{gawk}
• Functions
  • Built-in Functions
    • Calling Built-in Functions
    • Numeric Functions
    • String Manipulation Functions
      • More About `\' and `&' with sub, gsub, and gensub
    • Input/Output Functions
    • Using @command{gawk}'s Timestamp Functions
    • Using @command{gawk}'s Bit Manipulation Functions
    • Using @command{gawk}'s String Translation Functions
  • User-Defined Functions
    • Function Definition Syntax
    • Function Definition Examples
    • Calling User-Defined Functions
    • The return Statement
    • Functions and Their Effect on Variable Typing
• Internationalization with @command{gawk}
  • Internationalization and Localization
  • GNU gettext
  • Internationalizing @command{awk} Programs
  • Translating @command{awk} Programs
    • Extracting Marked Strings
    • Rearranging printf Arguments
    • @command{awk} Portability Issues
  • A Simple Internationalization Example
  • @command{gawk} Can Speak Your Language
• Advanced Features of @command{gawk}
  • Allowing Non-Decimal Input Data
  • Two-Way Communications with Another Process
  • Using @command{gawk} for Network Programming
  • Using @command{gawk} with BSD Portals
  • Profiling Your @command{awk} Programs
• Running @command{awk} and @command{gawk}
  • Invoking @command{awk}
  • Command-Line Options
  • Other Command-Line Arguments
  • The @env{AWKPATH} Environment Variable
  • Obsolete Options and/or Features
  • Undocumented Options and Features
  • Known Bugs in @command{gawk}
• A Library of @command{awk} Functions
  • Naming Library Function Global Variables
  • General Programming
    • Implementing nextfile as a Function
    • Assertions
    • Rounding Numbers
    • The Cliff Random Number Generator
    • Translating Between Characters and Numbers
    • Merging an Array into a String
    • Managing the Time of Day
  • Data File Management
    • Noting Data File Boundaries
    • Rereading the Current File
    • Checking for Readable Data Files
    • Treating Assignments as File Names
  • Processing Command-Line Options
  • Reading the User Database
  • Reading the Group Database
• Practical @command{awk} Programs
  • Running the Example Programs
  • Reinventing Wheels for Fun and Profit
    • Cutting out Fields and Columns
    • Searching for Regular Expressions in Files
    • Printing out User Information
    • Splitting a Large File into Pieces
    • Duplicating Output into Multiple Files
    • Printing Non-Duplicated Lines of Text
    • Counting Things
  • A Grab Bag of @command{awk} Programs
    • Finding Duplicated Words in a Document
    • An Alarm Clock Program
    • Transliterating Characters
    • Printing Mailing Labels
    • Generating Word Usage Counts
    • Removing Duplicates from Unsorted Text
    • Extracting Programs from Texinfo Source Files
    • A Simple Stream Editor
    • An Easy Way to Use Library Functions
• The Evolution of the @command{awk} Language
  • Major Changes Between V7 and SVR3.1
  • Changes Between SVR3.1 and SVR4
  • Changes Between SVR4 and POSIX @command{awk}
  • Extensions in the Bell Laboratories @command{awk}
  • Extensions in @command{gawk} Not in POSIX @command{awk}
  • Major Contributors to @command{gawk}
• Installing @command{gawk}
  • The @command{gawk} Distribution
    • Getting the @command{gawk} Distribution
    • Extracting the Distribution
    • Contents of the @command{gawk} Distribution
  • Compiling and Installing @command{gawk} on Unix
    • Compiling @command{gawk} for Unix
    • Additional Configuration Options
    • The Configuration Process
  • Installation on Other Operating Systems
    • Installing @command{gawk} on an Amiga
    • Installing @command{gawk} on BeOS
    • Installation on PC Operating Systems
      • Installing a Prepared Distribution for PC Systems
      • Compiling @command{gawk} for PC Operating Systems
      • Using @command{gawk} on PC Operating Systems
    • How to Compile and Install @command{gawk} on VMS
      • Compiling @command{gawk} on VMS
      • Installing @command{gawk} on VMS
      • Running @command{gawk} on VMS
      • Building and Using @command{gawk} on VMS POSIX
  • Unsupported Operating System Ports
    • Installing @command{gawk} on the Atari ST
      • Compiling @command{gawk} on the Atari ST
      • Running @command{gawk} on the Atari ST
    • Installing @command{gawk} on a Tandem
  • Reporting Problems and Bugs
  • Other Freely Available @command{awk} Implementations
• Implementation Notes
  • Downward Compatibility and Debugging
  • Making Additions to @command{gawk}
    • Adding New Features
    • Porting @command{gawk} to a New Operating System
  • Adding New Built-in Functions to @command{gawk}
    • A Minimal Introduction to @command{gawk} Internals
    • Directory and File Operation Built-ins
      • Using chdir and stat
      • C Code for chdir and stat
      • Integrating the Extensions
  • Probable Future Extensions
• Basic Programming Concepts
  • What a Program Does
  • Data Values in a Computer
  • Floating-Point Number Caveats
• Glossary
• GNU General Public License
  • Preamble
  • Terms and Conditions for Copying, Distribution and Modification
  • How to Apply These Terms to Your New Programs
• GNU Free Documentation License
  • ADDENDUM: How to use this License for your documents
• Index

This document was generated on 23 October 2001 using the texi2html translator version 1.54.