I'm pleased to announce the release of Declt 1.0, the first officially
stable version[1].
Declt (pronounce "dec'let") is a reference manual generator for Common
Lisp libraries. It works by loading an ASDF system and introspecting
its contents. The generated documentation contains the description for
the system itself and its components (modules and files), the packages
defined in that system and the definitions found in those packages.
Exported and internal definitions are listed separately. This allows the
reader to have a quick view on the library's public API. Within each
section, definitions are sorted lexicographically.
In addition to ASDF system components and packages, Declt documents the
following definitions: constants, special variables, symbol macros,
macros, setf expanders, compiler macros, functions (including setf
ones), generic functions and methods (including setf ones), method
combinations, conditions, structures, classes and types.
The generated documentation includes every possible bit of information
that introspecting can provide: documentation strings, lambda lists
(including qualifiers and specializers where appropriate), slots
(including type, allocation and initialization arguments), definition
source file etc.
Every documented item provides a full set of cross-references to related
items: ASDF component dependencies, parents and children, classes direct
methods, super and subclasses, slot readers and writers, setf expanders
access and update functions etc.
Finally, Declt produces exhaustive and multiple-entry indexes for every
documented item.
Reference manuals are generated in Texinfo format (compatible, but not
requiring Texinfo 5). From there it is possible to produce readable /
printable output in info, HTML, PDF, DVI and PostScript with tools such
as makeinfo, texi2dvi or texi2pdf.
The Declt reference manual is the primary example of documentation
generated by Declt itself.
Footnotes:
[1] http://www.lrde.epita.fr/~didier/software/lisp/misc.php#declt
--
Resistance is futile. You will be jazzimilated.
Lisp, Jazz, Aïkido: http://www.didierverna.info
Hello,
I am pleased to announce that I will give 2 talks at the next Tex Users
Group conference, to be held in Tokyo, Japan, next October. Please find
the abstracts below.
TiCL: The prototype
Last year, I presented some ideas about using one of the oldest
programming language (Lisp), in order to modernize one of the oldest
typesetting systems (TeX). This talk was mostly focused on justifying
the technical fitness of Lisp for this task. This time, I would like to
take the opposite view and demonstrate a prototype, from the user's
perspective. This will involve showing what a TiCL document could look
like, the implications in terms of typesetting vs. programmatic
features, and also in terms of extensibility (relating this to package
authoring).
The incredible tale of the author who didn't want to do the publisher's job
In this talk, I will relate a recent experience of mine: writing a book
chapter for a publisher who doesn't have a clue about typesetting. I
will confess my futile attempt at using TeX for writing the chapter in
question. I will then describe the hell that descended upon me for
daring to do that. I will however admit that the hell in question would
have been even greater, had I not done so. I will give this talk both
crying and laughing, and I will seek your comfort.
--
Resistance is futile. You will be jazzimilated.
Lisp, Jazz, Aïkido: http://www.didierverna.info
L'équipe Bison est heureuse d'annoncer la publication de GNU Bison 3.0,
qui introduit de nombreuses nouvelles fonctionnalités. GNU Bison est
un générateur d'analyseurs syntaxiques dans la lignée de Yacc.
Les nouveautés comprennent entre autres : (i) profonde rénovation du
système de diagnostiques (erreurs et avertissements), (ii) des moyens
plus souples de définition des types des valeurs sémantiques (avec en
particulier la possibilité d'utiliser de véritables objets en C++),
(iii) une interface "push" pour les parleurs Java, et (iv) des prédicats
de filtrage en mode GLR.
Certaines de ces nouveautés sont des contributions d'épitéens.
http://www.gnu.org/software/bison/
Voici les changements, plus en détails, et plus en anglais.
* Noteworthy changes in release 3.0 (2013-07-25) [stable]
** WARNING: Future backward-incompatibilities!
Like other GNU packages, Bison will start using some of the C99 features
for its own code, especially the definition of variables after statements.
The generated C parsers still aim at C90.
** Backward incompatible changes
*** Obsolete features
Support for YYFAIL is removed (deprecated in Bison 2.4.2): use YYERROR.
Support for yystype and yyltype is removed (deprecated in Bison 1.875):
use YYSTYPE and YYLTYPE.
Support for YYLEX_PARAM and YYPARSE_PARAM is removed (deprecated in Bison
1.875): use %lex-param, %parse-param, or %param.
Missing semicolons at the end of actions are no longer added (as announced
in the release 2.5).
*** Use of YACC='bison -y'
TL;DR: With Autoconf <= 2.69, pass -Wno-yacc to (AM_)YFLAGS if you use
Bison extensions.
Traditional Yacc generates 'y.tab.c' whatever the name of the input file.
Therefore Makefiles written for Yacc expect 'y.tab.c' (and possibly
'y.tab.h' and 'y.outout') to be generated from 'foo.y'.
To this end, for ages, AC_PROG_YACC, Autoconf's macro to look for an
implementation of Yacc, was using Bison as 'bison -y'. While it does
ensure compatible output file names, it also enables warnings for
incompatibilities with POSIX Yacc. In other words, 'bison -y' triggers
warnings for Bison extensions.
Autoconf 2.70+ fixes this incompatibility by using YACC='bison -o y.tab.c'
(which also generates 'y.tab.h' and 'y.output' when needed).
Alternatively, disable Yacc warnings by passing '-Wno-yacc' to your Yacc
flags (YFLAGS, or AM_YFLAGS with Automake).
** Bug fixes
*** The epilogue is no longer affected by internal #defines (glr.c)
The glr.c skeleton uses defines such as #define yylval (yystackp->yyval) in
generated code. These weren't properly undefined before the inclusion of
the user epilogue, so functions such as the following were butchered by the
preprocessor expansion:
int yylex (YYSTYPE *yylval);
This is fixed: yylval, yynerrs, yychar, and yylloc are now valid
identifiers for user-provided variables.
*** stdio.h is no longer needed when locations are enabled (yacc.c)
Changes in Bison 2.7 introduced a dependency on FILE and fprintf when
locations are enabled. This is fixed.
*** Warnings about useless %pure-parser/%define api.pure are restored
** Diagnostics reported by Bison
Most of these features were contributed by Théophile Ranquet and Victor
Santet.
*** Carets
Version 2.7 introduced caret errors, for a prettier output. These are now
activated by default. The old format can still be used by invoking Bison
with -fno-caret (or -fnone).
Some error messages that reproduced excerpts of the grammar are now using
the caret information only. For instance on:
%%
exp: 'a' | 'a';
Bison 2.7 reports:
in.y: warning: 1 reduce/reduce conflict [-Wconflicts-rr]
in.y:2.12-14: warning: rule useless in parser due to conflicts: exp: 'a' [-Wother]
Now bison reports:
in.y: warning: 1 reduce/reduce conflict [-Wconflicts-rr]
in.y:2.12-14: warning: rule useless in parser due to conflicts [-Wother]
exp: 'a' | 'a';
^^^
and "bison -fno-caret" reports:
in.y: warning: 1 reduce/reduce conflict [-Wconflicts-rr]
in.y:2.12-14: warning: rule useless in parser due to conflicts [-Wother]
*** Enhancements of the -Werror option
The -Werror=CATEGORY option is now recognized, and will treat specified
warnings as errors. The warnings need not have been explicitly activated
using the -W option, this is similar to what GCC 4.7 does.
For example, given the following command line, Bison will treat both
warnings related to POSIX Yacc incompatibilities and S/R conflicts as
errors (and only those):
$ bison -Werror=yacc,error=conflicts-sr input.y
If no categories are specified, -Werror will make all active warnings into
errors. For example, the following line does the same the previous example:
$ bison -Werror -Wnone -Wyacc -Wconflicts-sr input.y
(By default -Wconflicts-sr,conflicts-rr,deprecated,other is enabled.)
Note that the categories in this -Werror option may not be prefixed with
"no-". However, -Wno-error[=CATEGORY] is valid.
Note that -y enables -Werror=yacc. Therefore it is now possible to require
Yacc-like behavior (e.g., always generate y.tab.c), but to report
incompatibilities as warnings: "-y -Wno-error=yacc".
*** The display of warnings is now richer
The option that controls a given warning is now displayed:
foo.y:4.6: warning: type clash on default action: <foo> != <bar> [-Wother]
In the case of warnings treated as errors, the prefix is changed from
"warning: " to "error: ", and the suffix is displayed, in a manner similar
to GCC, as [-Werror=CATEGORY].
For instance, where the previous version of Bison would report (and exit
with failure):
bison: warnings being treated as errors
input.y:1.1: warning: stray ',' treated as white space
it now reports:
input.y:1.1: error: stray ',' treated as white space [-Werror=other]
*** Deprecated constructs
The new 'deprecated' warning category flags obsolete constructs whose
support will be discontinued. It is enabled by default. These warnings
used to be reported as 'other' warnings.
*** Useless semantic types
Bison now warns about useless (uninhabited) semantic types. Since
semantic types are not declared to Bison (they are defined in the opaque
%union structure), it is %printer/%destructor directives about useless
types that trigger the warning:
%token <type1> term
%type <type2> nterm
%printer {} <type1> <type3>
%destructor {} <type2> <type4>
%%
nterm: term { $$ = $1; };
3.28-34: warning: type <type3> is used, but is not associated to any symbol
4.28-34: warning: type <type4> is used, but is not associated to any symbol
*** Undefined but unused symbols
Bison used to raise an error for undefined symbols that are not used in
the grammar. This is now only a warning.
%printer {} symbol1
%destructor {} symbol2
%type <type> symbol3
%%
exp: "a";
*** Useless destructors or printers
Bison now warns about useless destructors or printers. In the following
example, the printer for <type1>, and the destructor for <type2> are
useless: all symbols of <type1> (token1) already have a printer, and all
symbols of type <type2> (token2) already have a destructor.
%token <type1> token1
<type2> token2
<type3> token3
<type4> token4
%printer {} token1 <type1> <type3>
%destructor {} token2 <type2> <type4>
*** Conflicts
The warnings and error messages about shift/reduce and reduce/reduce
conflicts have been normalized. For instance on the following foo.y file:
%glr-parser
%%
exp: exp '+' exp | '0' | '0';
compare the previous version of bison:
$ bison foo.y
foo.y: conflicts: 1 shift/reduce, 2 reduce/reduce
$ bison -Werror foo.y
bison: warnings being treated as errors
foo.y: conflicts: 1 shift/reduce, 2 reduce/reduce
with the new behavior:
$ bison foo.y
foo.y: warning: 1 shift/reduce conflict [-Wconflicts-sr]
foo.y: warning: 2 reduce/reduce conflicts [-Wconflicts-rr]
$ bison -Werror foo.y
foo.y: error: 1 shift/reduce conflict [-Werror=conflicts-sr]
foo.y: error: 2 reduce/reduce conflicts [-Werror=conflicts-rr]
When %expect or %expect-rr is used, such as with bar.y:
%expect 0
%glr-parser
%%
exp: exp '+' exp | '0' | '0';
Former behavior:
$ bison bar.y
bar.y: conflicts: 1 shift/reduce, 2 reduce/reduce
bar.y: expected 0 shift/reduce conflicts
bar.y: expected 0 reduce/reduce conflicts
New one:
$ bison bar.y
bar.y: error: shift/reduce conflicts: 1 found, 0 expected
bar.y: error: reduce/reduce conflicts: 2 found, 0 expected
** Incompatibilities with POSIX Yacc
The 'yacc' category is no longer part of '-Wall', enable it explicitly
with '-Wyacc'.
** Additional yylex/yyparse arguments
The new directive %param declares additional arguments to both yylex and
yyparse. The %lex-param, %parse-param, and %param directives support one
or more arguments. Instead of
%lex-param {arg1_type *arg1}
%lex-param {arg2_type *arg2}
%parse-param {arg1_type *arg1}
%parse-param {arg2_type *arg2}
one may now declare
%param {arg1_type *arg1} {arg2_type *arg2}
** Types of values for %define variables
Bison used to make no difference between '%define foo bar' and '%define
foo "bar"'. The former is now called a 'keyword value', and the latter a
'string value'. A third kind was added: 'code values', such as '%define
foo {bar}'.
Keyword variables are used for fixed value sets, e.g.,
%define lr.type lalr
Code variables are used for value in the target language, e.g.,
%define api.value.type {struct semantic_type}
String variables are used remaining cases, e.g. file names.
** Variable api.token.prefix
The variable api.token.prefix changes the way tokens are identified in
the generated files. This is especially useful to avoid collisions
with identifiers in the target language. For instance
%token FILE for ERROR
%define api.token.prefix {TOK_}
%%
start: FILE for ERROR;
will generate the definition of the symbols TOK_FILE, TOK_for, and
TOK_ERROR in the generated sources. In particular, the scanner must
use these prefixed token names, although the grammar itself still
uses the short names (as in the sample rule given above).
** Variable api.value.type
This new %define variable supersedes the #define macro YYSTYPE. The use
of YYSTYPE is discouraged. In particular, #defining YYSTYPE *and* either
using %union or %defining api.value.type results in undefined behavior.
Either define api.value.type, or use "%union":
%union
{
int ival;
char *sval;
}
%token <ival> INT "integer"
%token <sval> STRING "string"
%printer { fprintf (yyo, "%d", $$); } <ival>
%destructor { free ($$); } <sval>
/* In yylex(). */
yylval.ival = 42; return INT;
yylval.sval = "42"; return STRING;
The %define variable api.value.type supports both keyword and code values.
The keyword value 'union' means that the user provides genuine types, not
union member names such as "ival" and "sval" above (WARNING: will fail if
-y/--yacc/%yacc is enabled).
%define api.value.type union
%token <int> INT "integer"
%token <char *> STRING "string"
%printer { fprintf (yyo, "%d", $$); } <int>
%destructor { free ($$); } <char *>
/* In yylex(). */
yylval.INT = 42; return INT;
yylval.STRING = "42"; return STRING;
The keyword value variant is somewhat equivalent, but for C++ special
provision is made to allow classes to be used (more about this below).
%define api.value.type variant
%token <int> INT "integer"
%token <std::string> STRING "string"
Code values (in braces) denote user defined types. This is where YYSTYPE
used to be used.
%code requires
{
struct my_value
{
enum
{
is_int, is_string
} kind;
union
{
int ival;
char *sval;
} u;
};
}
%define api.value.type {struct my_value}
%token <u.ival> INT "integer"
%token <u.sval> STRING "string"
%printer { fprintf (yyo, "%d", $$); } <u.ival>
%destructor { free ($$); } <u.sval>
/* In yylex(). */
yylval.u.ival = 42; return INT;
yylval.u.sval = "42"; return STRING;
** Variable parse.error
This variable controls the verbosity of error messages. The use of the
%error-verbose directive is deprecated in favor of "%define parse.error
verbose".
** Renamed %define variables
The following variables have been renamed for consistency. Backward
compatibility is ensured, but upgrading is recommended.
lr.default-reductions -> lr.default-reduction
lr.keep-unreachable-states -> lr.keep-unreachable-state
namespace -> api.namespace
stype -> api.value.type
** Semantic predicates
Contributed by Paul Hilfinger.
The new, experimental, semantic-predicate feature allows actions of the
form "%?{ BOOLEAN-EXPRESSION }", which cause syntax errors (as for
YYERROR) if the expression evaluates to 0, and are evaluated immediately
in GLR parsers, rather than being deferred. The result is that they allow
the programmer to prune possible parses based on the values of run-time
expressions.
** The directive %expect-rr is now an error in non GLR mode
It used to be an error only if used in non GLR mode, _and_ if there are
reduce/reduce conflicts.
** Tokens are numbered in their order of appearance
Contributed by Valentin Tolmer.
With '%token A B', A had a number less than the one of B. However,
precedence declarations used to generate a reversed order. This is now
fixed, and introducing tokens with any of %token, %left, %right,
%precedence, or %nonassoc yields the same result.
When mixing declarations of tokens with a litteral character (e.g., 'a')
or with an identifier (e.g., B) in a precedence declaration, Bison
numbered the litteral characters first. For example
%right A B 'c' 'd'
would lead to the tokens declared in this order: 'c' 'd' A B. Again, the
input order is now preserved.
These changes were made so that one can remove useless precedence and
associativity declarations (i.e., map %nonassoc, %left or %right to
%precedence, or to %token) and get exactly the same output.
** Useless precedence and associativity
Contributed by Valentin Tolmer.
When developing and maintaining a grammar, useless associativity and
precedence directives are common. They can be a nuisance: new ambiguities
arising are sometimes masked because their conflicts are resolved due to
the extra precedence or associativity information. Furthermore, it can
hinder the comprehension of a new grammar: one will wonder about the role
of a precedence, where in fact it is useless. The following changes aim
at detecting and reporting these extra directives.
*** Precedence warning category
A new category of warning, -Wprecedence, was introduced. It flags the
useless precedence and associativity directives.
*** Useless associativity
Bison now warns about symbols with a declared associativity that is never
used to resolve conflicts. In that case, using %precedence is sufficient;
the parsing tables will remain unchanged. Solving these warnings may raise
useless precedence warnings, as the symbols no longer have associativity.
For example:
%left '+'
%left '*'
%%
exp:
"number"
| exp '+' "number"
| exp '*' exp
;
will produce a
warning: useless associativity for '+', use %precedence [-Wprecedence]
%left '+'
^^^
*** Useless precedence
Bison now warns about symbols with a declared precedence and no declared
associativity (i.e., declared with %precedence), and whose precedence is
never used. In that case, the symbol can be safely declared with %token
instead, without modifying the parsing tables. For example:
%precedence '='
%%
exp: "var" '=' "number";
will produce a
warning: useless precedence for '=' [-Wprecedence]
%precedence '='
^^^
*** Useless precedence and associativity
In case of both useless precedence and associativity, the issue is flagged
as follows:
%nonassoc '='
%%
exp: "var" '=' "number";
The warning is:
warning: useless precedence and associativity for '=' [-Wprecedence]
%nonassoc '='
^^^
** Empty rules
With help from Joel E. Denny and Gabriel Rassoul.
Empty rules (i.e., with an empty right-hand side) can now be explicitly
marked by the new %empty directive. Using %empty on a non-empty rule is
an error. The new -Wempty-rule warning reports empty rules without
%empty. On the following grammar:
%%
s: a b c;
a: ;
b: %empty;
c: 'a' %empty;
bison reports:
3.4-5: warning: empty rule without %empty [-Wempty-rule]
a: {}
^^
5.8-13: error: %empty on non-empty rule
c: 'a' %empty {};
^^^^^^
** Java skeleton improvements
The constants for token names were moved to the Lexer interface. Also, it
is possible to add code to the parser's constructors using "%code init"
and "%define init_throws".
Contributed by Paolo Bonzini.
The Java skeleton now supports push parsing.
Contributed by Dennis Heimbigner.
** C++ skeletons improvements
*** The parser header is no longer mandatory (lalr1.cc, glr.cc)
Using %defines is now optional. Without it, the needed support classes
are defined in the generated parser, instead of additional files (such as
location.hh, position.hh and stack.hh).
*** Locations are no longer mandatory (lalr1.cc, glr.cc)
Both lalr1.cc and glr.cc no longer require %location.
*** syntax_error exception (lalr1.cc)
The C++ parser features a syntax_error exception, which can be
thrown from the scanner or from user rules to raise syntax errors.
This facilitates reporting errors caught in sub-functions (e.g.,
rejecting too large integral literals from a conversion function
used by the scanner, or rejecting invalid combinations from a
factory invoked by the user actions).
*** %define api.value.type variant
This is based on a submission from Michiel De Wilde. With help
from Théophile Ranquet.
In this mode, complex C++ objects can be used as semantic values. For
instance:
%token <::std::string> TEXT;
%token <int> NUMBER;
%token SEMICOLON ";"
%type <::std::string> item;
%type <::std::list<std::string>> list;
%%
result:
list { std::cout << $1 << std::endl; }
;
list:
%empty { /* Generates an empty string list. */ }
| list item ";" { std::swap ($$, $1); $$.push_back ($2); }
;
item:
TEXT { std::swap ($$, $1); }
| NUMBER { $$ = string_cast ($1); }
;
*** %define api.token.constructor
When variants are enabled, Bison can generate functions to build the
tokens. This guarantees that the token type (e.g., NUMBER) is consistent
with the semantic value (e.g., int):
parser::symbol_type yylex ()
{
parser::location_type loc = ...;
...
return parser::make_TEXT ("Hello, world!", loc);
...
return parser::make_NUMBER (42, loc);
...
return parser::make_SEMICOLON (loc);
...
}
*** C++ locations
There are operator- and operator-= for 'location'. Negative line/column
increments can no longer underflow the resulting value.
