8. Add-on packages for AutoGen

This chapter includes several programs that either work closely with AutoGen (extracting definitions or providing special formatting functions), or leverage off of AutoGen technology. There is also a formatting library that helps make AutoGen possible.

AutoOpts ought to appear in this list as well, but since it is the primary reason why many people would even look into AutoGen at all, I decided to leave it in the list of chapters.

8.1 Automated Finite State Machine

The templates to generate a finite state machine in C or C++ is included with AutoGen. The documentation is not. The documentation is in HTML format for viewing, or you can download FSM.

8.2 Combined RPC Marshalling

The templates and NFSv4 definitions are not included with AutoGen in any way. The folks that designed NFSv4 noticed that much time and bandwidth was wasted sending queries and responses when many of them could be bundled. The protocol bundles the data, but there is no support for it in rpcgen. That means you have to write your own code to do that. Until now. Download this and you will have a large, complex example of how to use AutoXDR for generating the marshalling and unmarshalling of combined RPC calls. There is a brief example on the web, but you should download AutoXDR.

8.3 Automated Event Management

Large software development projects invariably have a need to manage the distribution and display of state information and state changes. In other words, they need to manage their software events. Generally, each such project invents its own way of accomplishing this and then struggles to get all of its components to play the same way. It is a difficult process and not always completely successful. This project helps with that.

AutoEvents completely separates the tasks of supplying the data needed for a particular event from the methods used to manage the distribution and display of that event. Consequently, the programmer writing the code no longer has to worry about that part of the problem. Likewise the persons responsible for designing the event management and distribution no longer have to worry about getting programmers to write conforming code.

This is a work in progress. See my web page on the subject, if you are interested. I have some useful things put together, but it is not ready to call a product.

8.4 Invoking columns

This program was designed for the purpose of generating compact, columnized tables. It will read a list of text items from standard in or a specified input file and produce a columnized listing of all the non-blank lines. Leading white space on each line is preserved, but trailing white space is stripped. Methods of applying per-entry and per-line embellishments are provided. See the formatting and separation arguments below.

This program is used by AutoGen to help clean up and organize its output.

See `autogen/agen5/fsm.tpl' and the generated output `pseudo-fsm.h'.

This function was not implemented as an expression function because either it would have to be many expression functions, or a provision would have to be added to provide options to expression functions. Maybe not a bad idea, but it is not being implemented at the moment.

A side benefit is that you can use it outside of AutoGen to columnize input, a la the ls command.

This section was generated by AutoGen, the aginfo template and the option descriptions for the columns program. It documents the columns usage text and option meanings.

This software is released under the GNU General Public License.

8.4.1 columns usage help (-?)

This is the automatically generated usage text for columns:

@exampleindent 0

columns - Columnize Input Text - Ver. 1.1
USAGE:  columns [ -<flag> [<val>] | --<name>[{=| }<val>] ]...
  Flg Arg Option-Name    Description
   -W Num width          Maximum Line Width
   -c Num columns        Desired number of columns
   -w Num col-width      Set width of each column
      Num spread         maximum spread added to column width
   -I Str indent         Line prefix or indentation
      Str first-indent   First line prefix
                                - requires these options:
                                indent
      Num tab-width      tab width
   -s opt sort           Sort input text
   -f Str format         Formatting string for each input
   -S Str separation     Separation string - follows all but last
      Str line-separation string at end of all lines but last
      no  by-columns     Print entries in column order
   -i Str input          Input file (if not stdin)
   -v opt version        Output version information and exit
   -? no  help           Display usage information and exit
   -! no  more-help      Extended usage information passed thru pager

Options are specified by doubled hyphens and their name
or by a single hyphen and the flag character.

This program was designed for the purpose of generating compact,
columnized tables.  It will read a list of text items from standard
in or a specified input file and produce a columnized listing of
all the non-blank lines.  Leading white space on each line is
preserved, but trailing white space is stripped.  Methods of
applying per-entry and per-line embellishments are provided.
See the formatting and separation arguments below.

This program is used by AutoGen to help clean up and organize
its output.

please send bug reports to:  autogen-bugs@lists.sf.net

8.4.2 width option (-W)

This is the "maximum line width" option. This option specifies the full width of the output line, including any start-of-line indentation. The output will fill each line as completely as possible, unless the column width has been explicitly specified. If the maximum width is less than the length of the widest input, you will get a single column of output.

8.4.3 columns option (-c)

This is the "desired number of columns" option. Use this option to specify exactly how many columns to produce. If that many columns will not fit within line_width, then the count will be reduced to the number that fit.

8.4.4 col-width option (-w)

This is the "set width of each column" option. Use this option to specify exactly how many characters are to be allocated for each column. If it is narrower than the widest entry, it will be over-ridden with the required width.

8.4.5 spread option

This is the "maximum spread added to column width" option. Use this option to specify exactly how many characters may be added to each column. It allows you to prevent columns from becoming too far apart.

8.4.6 indent option (-I)

This is the "line prefix or indentation" option. If a number, then this many spaces will be inserted at the start of every line. Otherwise, it is a line prefix that will be inserted at the start of every line.

8.4.7 first-indent option

This is the "first line prefix" option.

This option has some usage constraints. It:

• must appear in combination with the following options: indent.

If a number, then this many spaces will be inserted at the start of the first line. Otherwise, it is a line prefix that will be inserted at the start of that line.

8.4.8 tab-width option

This is the "tab width" option. If an indentation string contains tabs, then this value is used to compute the ending column of the prefix string.

8.4.9 sort option (-s)

This is the "sort input text" option. Causes the input text to be sorted. If an argument is supplied, it is presumed to be a pattern and the sort is based upon the matched text. If the pattern starts with or consists of an asterisk (*), then the sort is case insensitive.

8.4.10 format option (-f)

This is the "formatting string for each input" option. If you need to reformat each input text, the argument to this option is interpreted as an sprintf(3) format that is used to produce each output entry.

8.4.11 separation option (-S)

This is the "separation string - follows all but last" option. Use this option if, for example, you wish a comma to appear after each entry except the last.

8.4.12 line-separation option

This is the "string at end of all lines but last" option. Use this option if, for example, you wish a backslash to appear at the end of every line, except the last.

8.4.13 by-columns option

This is the "print entries in column order" option. Normally, the entries are printed out in order by rows and then columns. This option will cause the entries to be ordered within columns. The final column, instead of the final row, may be shorter than the others.

8.4.14 input option (-i)

This is the "input file (if not stdin)" option. This program normally runs as a filter, reading from standard input, columnizing and writing to standard out. This option redirects input to a file.

8.5 Invoking getdefs

    #line nnn "source-file-name"
    entry_type = {
        name = entry_name;
        ...
    };

The ellipsis "..." is filled in by text found between the two delimiters, using the following rules:

1. Each entry is located by the pattern "\n[^*\n]*\\*[ \t]*([a-z][a-z0-9_]*):". Fundamentally, it finds a line that, after the first asterisk on the line, contains whitespace then a name and is immediately followed by a colon. The name becomes the name of the attribute and what follows, up to the next attribute, is its value.
2. If the first character of the value is either a single or double quote, then you are responsible for quoting the text as it gets inserted into the output definitions.
3. All the leading text on a line is stripped from the value. The leading text is everything before the first asterisk, the asterisk and all the whitespace characters that immediately follow it. If you want whitespace at the beginnings of the lines of text, you must do something like this:

   * mumble: * " this is some\n" * " indented text."

4. If the `<entry-name>' is followed by a comma, the word `ifdef' (or 'ifndef') and a name `if_name', then the above entry will appear as:

   #ifdef if_name #line nnn "source-file-name" entry_type = { name = entry_name; ... }; #endif

5. If you use of the subblock option, you can specify a nested value, See section 8.5.6 subblock option. That is, this text:

   * arg: int, this, what-it-is

   with the `--subblock=arg=type,name,doc' option would yield:

   arg = { type = int; name = this; doc = what-it-is; };

This section was generated by AutoGen, the aginfo template and the option descriptions for the getdefs program. It documents the getdefs usage text and option meanings.

This software is released under the GNU General Public License.

8.5.1 getdefs usage help

This is the automatically generated usage text for getdefs:

@exampleindent 0

getdefs - AutoGen Definition Extraction Tool - Ver. 1.4
USAGE:  getdefs [ <option-name>[{=| }<val>] ]...
   Arg Option-Name    Description
   Str defs-to-get    Regexp to look for after the "/*="
   opt ordering       Alphabetize or use named file
                                - disabled as --no-ordering
                                - enabled by default
   Num first-index    The first index to apply to groups
   Str input          Input file to search for defs
                                - may appear multiple times
                                - default option for unnamed options
   Str subblock       subblock definition names
                                - may appear multiple times
   Str listattr       attribute with list of values
                                - may appear multiple times
   opt filelist       Insert source file names into defs

Definition insertion options

   Arg Option-Name    Description
   Str assign         Global assignments
                                - may appear multiple times
   Str common-assign  Assignments common to all blocks
                                - may appear multiple times
   Str copy           File(s) to copy into definitions
                                - may appear multiple times
   opt srcfile        Insert source file name into each def
   opt linenum        Insert source line number into each def

Definition output disposition options:

   Arg Option-Name    Description
   Str output         Output file to open
                                - an alternate for autogen
   opt autogen        Invoke AutoGen with defs
                                - disabled as --no-autogen
                                - enabled by default
   Str template       Template Name
   Str agarg          AutoGen Argument
                                - prohibits these options:
                                output
                                - may appear multiple times
   Str base-name      Base name for output file(s)
                                - prohibits these options:
                                output

Auto-supported Options:

   Arg Option-Name    Description
   opt version        Output version information and exit
   no  help           Display usage information and exit
   no  more-help      Extended usage information passed thru pager
   opt save-opts      Save the option state to an rc file
   Str load-opts      Load options from an rc file
                                - disabled as --no-load-opts
                                - may appear multiple times

All arguments are named options.

If no ``input'' argument is provided or is set to simply "-", and if
``stdin'' is not a ``tty'', then the list of input files will be
read from ``stdin''.

The following option preset mechanisms are supported:
 - reading file /dev/null

This program extracts AutoGen definitions from a list of source files.
Definitions are delimited by `/*=<entry-type> <entry-name>\n' and
`=*/\n'.  From that, this program creates a definition of the
following form:

    #line nnn "source-file-name"
    entry_type = {
        name = entry_name;
        ...
    };

The ellipsis '...' is filled in by text found between the two
delimiters, with everything up through the first sequence of
asterisks deleted on every line.

There are two special ``entry types'':

*  The entry_type enclosure and the name entry will be omitted
   and the ellipsis will become top-level definitions.

-- The contents of the comment must be a single getdefs option.
   The option name must follow the double hyphen and its argument
   will be everything following the name.  This is intended for use
   with the ``subblock'' and ``listattr'' options.

please send bug reports to:  autogen-bugs@lists.sf.net

8.5.2 defs-to-get option

This is the "regexp to look for after the "/*="" option. If you want definitions only from a particular category, or even with names matching particular patterns, then specify this regular expression for the text that must follow the /*=.

8.5.3 ordering option

This is the "alphabetize or use named file" option.

This option has some usage constraints. It:

• is enabled by default.

By default, ordering is alphabetical by the entry name. Use, no-ordering if order is unimportant. Use ordering with no argument to order without case sensitivity. Use ordering=<file-name> if chronological order is important. getdefs will maintain the text content of file-name. file-name need not exist.

8.5.4 first-index option

This is the "the first index to apply to groups" option. By default, the first occurrance of a named definition will have an index of zero. Sometimes, that needs to be a reserved value. Provide this option to specify a different starting point.

8.5.5 input option

This is the "input file to search for defs" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

All files that are to be searched for definitions must be named on the command line or read from stdin. If there is only one input option and it is the string, "-", then the input file list is read from stdin. If a command line argument is not an option name and does not contain an assignment operator (=), then it defaults to being an input file name. At least one input file must be specified.

8.5.6 subblock option

This is the "subblock definition names" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

This option is used to create shorthand entries for nested definitions. For example, with:

using subblock thus

--subblock=arg=argname,type,null

and defining an arg thus

arg: this, char *

will then expand to:

arg = { argname = this; type = "char *"; };

8.5.7 listattr option

This is the "attribute with list of values" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

This option is used to create shorthand entries for definitions that generally appear several times. That is, they tend to be a list of values. For example, with:
listattr=foo defined, the text:
foo: this, is, a, multi-list will then expand to:
foo = 'this', 'is', 'a', 'multi-list';
The texts are separated by the commas, with the white space removed. You may use characters other than commas by starting the value string with a punctuation character other than a single or double quote character.

8.5.8 filelist option

This is the "insert source file names into defs" option. Inserts the name of each input file into the output definitions. If no argument is supplied, the format will be:

infile = '%s';

8.5.9 assign option

This is the "global assignments" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

The argument to each copy of this option will be inserted into the output definitions, with only a semicolon attached.

8.5.10 common-assign option

This is the "assignments common to all blocks" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

The argument to each copy of this option will be inserted into each output definition, with only a semicolon attached.

8.5.11 copy option

This is the "file(s) to copy into definitions" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

The content of each file named by these options will be inserted into the output definitions.

8.5.12 srcfile option

This is the "insert source file name into each def" option. Inserts the name of the input file where a definition was found into the output definition. If no argument is supplied, the format will be:

srcfile = '%s';

8.5.13 linenum option

This is the "insert source line number into each def" option. Inserts the line number in the input file where a definition was found into the output definition. If no argument is supplied, the format will be:

linenum = '%s';

8.5.14 output option

This is the "output file to open" option.

This option has some usage constraints. It:

• is a member of the autogen class of options.

If you are not sending the output to an AutoGen process, you may name an output file instead.

8.5.15 autogen option

This is the "invoke autogen with defs" option.

This option has some usage constraints. It:

• is enabled by default.
• is a member of the autogen class of options.

This is the default output mode. Specifying no-autogen is equivalent to output=-. If you supply an argument to this option, that program will be started as if it were AutoGen and its standard in will be set to the output definitions of this program.

8.5.16 template option

This is the "template name" option. Specifies the template name to be used for generating the final output.

8.5.17 agarg option

This is the "autogen argument" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.
• must not appear in combination with any of the following options: output.

This is a pass-through argument. It allows you to specify any arbitrary argument to be passed to AutoGen.

8.5.18 base-name option

This is the "base name for output file(s)" option.

This option has some usage constraints. It:

• must not appear in combination with any of the following options: output.

When output is going to AutoGen, a base name must either be supplied or derived. If this option is not supplied, then it is taken from the template option. If that is not provided either, then it is set to the base name of the current directory.

8.6 Invoking xml2ag

• The --override-tpl command line option
• A top level XML attribute named, "template"

The base-name for the output will similarly be either:

• The --base-name command line option.
• The base name of the `.xml' file.

The definitions derived from XML generally have an extra layer of definition. Specifically, this XML input:

<mumble attr="foo">
  mumble-1
  <grumble>
  grumble, grumble, grumble.
</grumble>mumble, mumble
</mumble>

mumble = {
  grumble = {
    text = 'grumble, grumble, grumble';
  };
  text = 'mumble-1';
  text = 'mumble, mumble';
};

grumble = 'grumble, grumble, grumble.';

This section was generated by AutoGen, the aginfo template and the option descriptions for the xml2ag program. It documents the xml2ag usage text and option meanings.

This software is released under the GNU General Public License.

8.6.1 xml2ag usage help (-?)

This is the automatically generated usage text for xml2ag:

@exampleindent 0

xml2ag - XML to AutoGen Definiton Converter - Ver. 5.5.5
USAGE:  xml2ag [ -<flag> [<val>] | --<name>[{=| }<val>] ]... [ <def-file> ]
  Flg Arg Option-Name    Description
   -O Str output         Output file in lieu of AutoGen processing
   -L Str templ-dirs     Template search directory list
                                - may appear multiple times
   -T Str override-tpl   Override template file
   -l Str lib-template   Library template file
                                - may appear multiple times
   -b Str base-name      Base name for output file(s)
      Str definitions    Definitions input file
   -S Str load-scheme    Scheme code file to load
   -F Str load-functions Load scheme callout library
   -s Str skip-suffix    Omit the file with this suffix
                                - may appear multiple times
   -o opt select-suffix  specify this output suffix
                                - may appear multiple times
      no  source-time    set mod times to latest source
      Str equate         characters considered equivalent
      no  writable       Allow output files to be writable
                                - disabled as --not-writable
      Num loop-limit     Limit on increment loops
                                  it must lie in one of the ranges:
                                  -1 exactly, or
                                  1 to 16777216
   -t Num timeout        Time limit for servers
                                  it must lie in the range: 0 to 3600
      KWd trace          tracing level of detail
      Str trace-out      tracing output file or filter
      no  show-defs      Show the definition tree
      no  show-shell     Show shell commands
   -D Str define         name to add to definition list
                                - may appear multiple times
   -U Str undefine       definition list removal pattern
                                - an alternate for define
   -v opt version        Output version information and exit
   -? no  help           Display usage information and exit
   -! no  more-help      Extended usage information passed thru pager

Options are specified by doubled hyphens and their name
or by a single hyphen and the flag character.

This program will convert any arbitrary XML file into equivalent
AutoGen definitions, and invoke AutoGen.

The valid trace option keywords are:
        nothing
        templates
        block-macros
        expressions
        everything

The template will be derived from either:
*  the ``--override-tpl'' command line option
*  a top level XML attribute named, "template"

The ``base-name'' for the output will similarly be either:
*  the ``--base-name'' command line option
*  the base name of the .xml file

please send bug reports to:  autogen-bugs@lists.sf.net

8.6.2 output option (-O)

This is the "output file in lieu of autogen processing" option. By default, the output is handed to an AutoGen for processing. However, you may save the definitions to a file instead.

8.6.3 templ-dirs option (-L)

This is the "template search directory list" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

Pass-through AutoGen argument

8.6.4 override-tpl option (-T)

This is the "override template file" option. Pass-through AutoGen argument

8.6.5 lib-template option (-l)

This is the "library template file" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

Pass-through AutoGen argument

8.6.6 base-name option (-b)

This is the "base name for output file(s)" option. Pass-through AutoGen argument

8.6.7 definitions option

This is the "definitions input file" option. Pass-through AutoGen argument

8.6.8 load-scheme option (-S)

This is the "scheme code file to load" option. Pass-through AutoGen argument

8.6.9 load-functions option (-F)

This is the "load scheme callout library" option. Pass-through AutoGen argument

8.6.10 skip-suffix option (-s)

This is the "omit the file with this suffix" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

Pass-through AutoGen argument

8.6.11 select-suffix option (-o)

This is the "specify this output suffix" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

Pass-through AutoGen argument

8.6.12 source-time option

This is the "set mod times to latest source" option. Pass-through AutoGen argument

8.6.13 equate option

This is the "characters considered equivalent" option. Pass-through AutoGen argument

8.6.14 writable option

This is the "allow output files to be writable" option. Pass-through AutoGen argument

8.6.15 loop-limit option

This is the "limit on increment loops" option. Pass-through AutoGen argument

8.6.16 timeout option (-t)

This is the "time limit for servers" option. Pass-through AutoGen argument

8.6.17 trace option

This is the "tracing level of detail" option. Pass-through AutoGen argument

8.6.18 trace-out option

This is the "tracing output file or filter" option. Pass-through AutoGen argument

8.6.19 show-defs option

This is the "show the definition tree" option. Pass-through AutoGen argument

8.6.20 show-shell option

This is the "show shell commands" option. Pass-through AutoGen argument

8.6.21 define option (-D)

This is the "name to add to definition list" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.
• is a member of the define class of options.

Pass-through AutoGen argument

8.6.22 undefine option (-U)

This is the "definition list removal pattern" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.
• is a member of the define class of options.

Pass-through AutoGen argument

8.7 Replacement for Stdio Formatting Library

Using the `printf' formatting routines in a portable fashion has always been a pain, and this package has been way more pain than anyone ever imagined. Hopefully, with this release of snprintfv, the pain is now over for all time.

The issues with portable usage are these:

1. Argument number specifiers are often either not implemented or are buggy. Even GNU libc, version 1 got it wrong.

2. ANSI/ISO "forgot" to provide a mechanism for computing argument lists for vararg procedures.

3. The argument array version of printf (`printfv()') is not generally available, does not work with the native printf, and does not have a working argument number specifier in the format specification. (Last I knew, anyway.)

4. You cannot fake varargs by calling `vprintf()' with an array of arguments, because ANSI does not require such an implementation and some vendors play funny tricks because they are allowed to.

These four issues made it impossible for AutoGen to ship without its own implementation of the `printf' formatting routines. Since we were forced to do this, we decided to make the formatting routines both better and more complete :-). We addressed these issues and added the following features to the common printf API:

1. The formatted output can be written to
   • a string allocated by the formatting function (`asprintf()').
   • a file descriptor instead of a file stream (`dprintf()').
   • a user specified stream (`stream_printf()').

2. The formatting functions can be augmented with your own functions. These functions are allowed to consume more than one character from the format, but must commence with a unique character. For example,

   "%{struct stat}\n"

   might be used with '{' registered to a procedure that would look up "struct stat" in a symbol table and do appropriate things, consuming the format string through the '}' character.

Gary V. Vaughan was generous enough to supply this implementation. Many thanks!!

For further details, the reader is referred to the snprintfv documentation. These functions are also available in the template processing as `sprintf' (see section 3.5.25 `sprintf' - format a string), `printf' (see section 3.5.20 `printf' - format to stdout), `fprintf' (see section 3.5.6 `fprintf' - format to a file), and `shellf' (see section 3.5.24 `shellf' - format a string, run shell).