Transformers

Download

transformers@lrde.epita.fr

May 2006

1 participants
1 discussions

The ASF+SDF Meta-Environment 2.0-RC1 (Release Candidate)
by Akim Demaille 16 May '06

16 May '06

Début du message réexpédié : > De : "Jurgen J. Vinju" <Jurgen.Vinju(a)cwi.nl> > Date : 16 mai 2006 10:31:14 HAEC > À : meta-users-list(a)cwi.nl > Objet : [meta-users-list] The ASF+SDF Meta-Environment 2.0-RC1 > (Release Candidate) > Répondre à : meta-users-list(a)cwi.nl > > Dear ASF+SDF Meta-Environment people, > > We have decided to put a Release Candidate of the full ASF+SDF > Meta-Environment online: version 2.0-RC1 > > It has been a while since the previous release, and a lot of work > has been done: > - many requests for enhancement and bugfixes have been > implemented. - a number of experimental features have been > implemented > - some features have been optimized for speed or memory consumption > - and some legacy implementations have been replaced > > This is a Release Candidate, because we think a lot more has to be > done to > finish version 2.0 as a product: - some features are only half > implemented (e.g. only by the ASF interpreter, not by the ASF > compiler) > - some features are not optimized (too slow or too big) > - the documentation is out of date > > Below you will find the only available documentation on what > changed and > what you need to do to get your specifications working. We ARE > catching > up on the documentation front. > ================= > BASIC INFORMATION > ================= > > Website > ------- > * http://www.meta-environment.org > > Help > ---- > * Problems, bugs and enhancements: * mailto:meta-support- > list(a)cwi.nl > * General help (also by other users): * mailto:meta-users- > list(a)cwi.nl > * Submitting issues directly: * bugzilla.sen.cwi.nl:8080/ > index.cgi > > Required software > ----------------- > * The following software is necessary to compile and run 2.0RC1: > * GNU gcc 3.4.x (the 4.x series contains bugs that we can not > work around) > * GNU make > * Graphviz dot >= 1.12 > * Java JRE 1.5 (will not work with 1.4.x) > > ============== CHANGES & TIPS > ============== > Commandline > ----------- > * The ASF+SDF Meta-Environment starts up with the command > "asfsdf-meta", not with "meta" anymore. > * The SDF Meta-Environment (without ASF) can be started with: > "sdf-meta". > * The dump scripts, pt-dump, eqs-dump, def-dump, etc. do not > have options to save the "term-store" anymore. This feature > has dissappeared and we are thinking about better ways to cache > intermediate results. > > User interface > -------------- > * The third-party editor interface is gone. So no more emacs or vi. > * The editors are now implemented using a standard Swing > implementation. > * All Meta-Environment GUI tools are implemented in Java/Swing and > hosted in a dockable window environment, implemented by > InfoNode.com (GPL) * Note that many menu options have > keyboard shortcuts, and all menus > have changed. > * Actions->Parse does not exist anymore, after saving an editor > (CTRL+S), all necessary updates are done to the state of the > system > automatically. Any errors that may be the result of your changes > are listed in the tab of the error viewer. (This is one of > the experimental features that are not fast enough > yet, sometimes after a save it takes a long time for the system > to become idle again) > * The full state of the system is viewable from the 'Progress' > viewer > tab. > * All editors apply a generic syntax highlighting scheme, which you > can influence in a number of ways. Contact meta-support- > list(a)cwi.nl > for questions. > * We use "prefuse", an open source graph visualization and animation > package: www.prefuse.org. The graphviz layout algorithm is used, > but the actual displaying is done by prefuse: > * left click and drag: move a node > * right click and drag up/down: zoom out/in > * right click on canvas: zoom to fit > * left click on node: select node > * right click on node: open context popup menu > * the "Graph" menu contains some other options > * Switch workspace: The "Workspace" is the directory where new > modules > will appear and existing modules are found. The File menu contains > a switch workspace button that can be used at anytime. > * Open module: the file browser that is opened gives access to: > * the Workspace > * the SDF library > * the ASF+SDF library > You should not open files outside of these domains, the interface > tries to limit this, but it is not hacker proof. > * Note that syntax definitions are now in the SDF library, while > ASF+SDF data-structures and utilities are still in the ASF+SDF > library. > * Library modules are read-only, and coloured grey in the import > graph. > * The save termstore feature has dissappeared. We are thinking about > a replacement for caching intermediate results between runs of > the Meta-Environment. > > Syntax Definition Formalism and SGLR > ------------------------------------ > > * The SDF library now comes with some new grammars: > * C grammar > * Parameterized C preprocessor grammar > * The syntax of SDF has been redesigned to use the same syntax > for string literals everywhere. This also implies that unquoted > string literal symbols are not literals anymore. You will get > parse errors in your SDF modules. Before, the SDF checker > would warn you about deprecated use of unquoted literals. > * The AsFix2ME format has changed. It does not flatten lexical > syntax > anymore. Fully structured lexical trees are generated from now on. > * AsFix2ME also does some subtle rearrangements of LAYOUT nodes. > For details read the source code or contact meta-support- > list(a)cwi.nl > * The SDF checker generates less false warnings on old issues, and > some more false warnings for new issues. * CASE-INSENSITIVE > LITERALS are added as a feature to SDF2. The > syntax is as follows: > 'begin' STAT* 'end' -> PROGRAM > Defines a program with two case-insensitive literals begin and > end. > (use of single quotes instead of double quotes). The SDF > normalizer > generates for you: > [bB][eE][gG][iI][nN] -> 'begin' [eE][nN][dD] -> 'end' > remember the case-sensitive notation is still: > "begin" STAT* "end" -> PROGRAM > which generates: > [b][e][g][i][n] -> "begin" [e][n][d] -> "end" > * ci-lit(<str>) was added as a symbol constructor to AsFix2[ME] > * CYCLIC GRAMMARS: SGLR now accepts cyclic grammars. A parse forest > with a representation of a cycle will be generated instead of > a parse error. The representation is NOT minimal. Contact > meta-support-list(a)cwi.nl for details. > * AsFix2[ME] is extended with a new Tree constructor, next to > appl, amb and char we now have: cycle. > * The backend of SGLR was optimized for highly ambiguous languages. > The use of memoization in several algorithms has improved the > worst-case complexity from exponential to polynominal. > * SGLR does not provide a detailed ambiguity report anymore, use the > renewed implementation of the tool "ambtracker". > * NOTE: we are currently reimplementing the API's around SGLR and > remodularizing its implementation. * Syntax productions are > generated from priority sections like was > done several versions back. BEWARE: we removed this then because > the attributes of productions are merged if the rest of the > production is the same. It might lead to unexpected shapes of > productions in an AsFix2 parse forest. The benefit in > grammars for C, Java and C++ is so big, that we > decided to put it back in (several dozens of lines less in SDF > modules with large expression grammars) > * PRIORITY FOR SELECTED ARGUMENTS. A priority can now be limited > to be applicable in a certain argument only. This works for > single productions as well as groups, and priorities are still > transitively closed. Example: > > context-free priorities > { E "[" E "]" -> E > E "bla" -> E > } <0> > > E "*" E -> E {left} > > The first group has a higher priority than the second group, but > only in the first (0) argument. Between the <..> brackets comma > separated lists of argument numbers are accepted. We start > counting > at zero, and count EVERY MEMBER of the left hand side including > literals, counting nested symbols as 1 member, and ignoring the > implicit LAYOUT? non-terminal. > > Algebraic Specification Formalism > --------------------------------- > > * STRUCTURED LEXICALS: this is the new feature that breaks old > ASF+SDF programs that made use of lexical constructor functions. > This is a long story, here is the summary: > * The CHAR sort is not to be used by you anymore. From now on > you use 'lexical variables' as such defined in SDF2: > lexical syntax > [0-9]+ -> Number > Number "." Number -> Real > lexical variables > "digit"[0-9]* -> [0-9] > "digits"[0-9]* -> [0-9]+ > "num"[0-9]* -> Number > "real"[0-9]* -> Real > * In other words, for lexical non-terminals and character classes > do not define normal variables, but lexical variables. > * Variables that range over SUBCLASSES may be used. If a variable > does not range over a subclass of the expected character class, > an error will be generated. Example: > lexical variables > "half" -> [0-4] > "more" -> [0-9A-F] > The variable 'half' is acceptable everywhere [0-9] is also > acceptible. The variable 'more' is not accepted at [0-9] > locations. > * Lexical constructors are now fully NESTED, because lexical > syntax is not flattened to a list representation anymore, > we now also need STRUCTURED LEXICAL CONSTRUCTORS. * For > each production in lexical syntax a lexical constructor > function is generated: > - the prefix function name is the sort name in lowercase > - each function has as many arguments as the production > had members, including literals. > - literals are to be typed literally > - characters are escaped in the same manner as characters > are escaped in the definitions of SDF2 character classes. > - lexical variables are only accepted in the context of > these generated constructor functions > - example (see syntax and variables above): > equations > [zeros] number(0 digits) = number > (digits) [trunk] real(num . number(digits2)) = real > (num . 0) > - another example: > lexical syntax > [\"] ~[\"]* [\"] -> String > lexical variables > "char"[0-9]* -> ~[\"] > "chars"[0-9]* -> ~[\"]* equations > [remove-nl] string(\" chars1 \n chars2 \") > = string(\" chars1 chars2 \") > - You can not construct any lexical that does not conform to > the SDF definition of the lexical syntax anymore. Not even > as an intermediate step. This sometimes requires some extra > thinking. See the library modules basic/Identifiers and > basic/Strings and basic/Bytes for examples on how > to analyse or transform lexical syntax. > * (Traversal) Functions that returns lists, there used to be many > problems with functions that returns lists, including traversal > functions that visit list nodes: e.g. f(A) -> {B ","}* > SDF, AsFix2 and ASF have been changed to fix these issues. * > REWRITING LAYOUT. Layout nodes are now first class citizens in > ASF specifications. Each layout node in a syntactic pattern (i.e. > left-hand sides or right-hand sides of conditions and equations) > can now be in either one of these two modes: > * IGNORED: the layout is ignored by a match, and put in literally > by a constructing pattern. The syntax for this is just as it > used to be: simply type the layout as it is defined and it will > be ignored. > * NOT IGNORED: the layout is matched and constructed according to > your specification. The mode is activated when: > * You use a lexical constructor function for layout > * You use 'layout brackets' > * Examples: > lexical syntax > [\ \t\n] -> LAYOUT > "%%" ~[\n]* [\n] -> LAYOUT > context-free syntax > "_" "_" -> DUMMY {layout-bracket} > lexical variables > "L"[0-9]* -> LAYOUT > equations > [] if !E then S*1 else S*2 fi = if E then S*2 else S*1 fi > [] if layout(\n) E then S* fi = if layout(\ ) E then S* fi > [] if L1 layout(\n) L2 E then S* fi = if L1 layout(\ ) L2 E > then S* fi > [] _ layout(\t) _ = _ layout(\ ) layout(\ ) _ > * The first equation ignores layout completely, like usual. > * The second equation only matches an if statement if there is > EXACTLY one newline between the if and the E. > * The third equation matches an if statement with a newline > SOMEWHERE between the if and the E. > * The fourth equation matches ALL single tabs and translates > them to two spaces. > * Condition syntax > * The deprecated '=' condition operator has dissappeared > * Integrated unit tests > * There are still unit tests in ASF specifications, example: > equations > [] myfunction(...) = myanswer(...) > tests > [] myfunction(xyz) == myanswer(abc) > * Rewriting with pos-info annotations and others > * You can store and retrieve parse tree annotations using the > modules utilities/Annotations, utilities/ATermAnnotations and > utilities/PosInfo. * The ASF interpreter supports this > by default > * The ASF compiler has an extra commandline option to > activate support for annotations in the generated code. > * New builtin functions: > * See utilities/Parsing for calling the parser on files and > strings > * See utilities/Files for reading and writing files from ASF > * The commandline versions of the interpreter and the compiler > have > a new option to pass a parse table in, for use with the > parsing builtin > * Interface of compiled specifications * A compiled > specification can now read source code directly, without > using sglr first. > * If you compiled a specification from the Meta-Environment > directly, > this is the default behavior: the binary will rewrite source- > to-source. > * Wild and strict variables (not experimental anymore) > * The experimental feature of wild and strict variables is > consolidated. > * example: > variables > "_x"[0-9]* -> Integer {wild} > "x"[0-9]* -> Integer {strict} > * wild variables may only be used as 'wildcards'. No further use > of a variable is allowed after using it in a match. > * strict variables are used for information propagation. A strict > variable MUST be used after it's introduction. * NOTE: > the use of strict variables leads to less errors, it > prohibits unintended loss of information. > * NOTE: the use of wild variables also leads to less errors, it > prohibits unintended failure of matches by use of the same > variable > names twice. * Complete functions > * An experimental feature: complete functions. You can annotate a > function to be complete, which means that it may not appear in > a normal form. Use this if you intend to implement a > function that > deals with all input correctly. > * You will get a run-time error (interpreter only) if a > complete function > does not reduce. > * Example: > context-free syntax > myfunction(Statement) -> Booleans {complete} > * Rewriting ambiguities > * The rewriting of ambiguities implementation has changed > recently, > but not tested thoroughly yet. Use with caution. > > >

1 0