gmane.comp.java.sablecc.general

Re: Grammar defenition issue. expr = plus plus;

Necto — 2012-03-24T16:00:03

Thanks a lot!

2012/3/24 Etienne M. Gagnon <egagnon< at >j-meg.com>

Re: Grammar defenition issue. expr = plus plus;

Etienne M. Gagnon — 2012-03-24T14:59:47

Hi Necto,

Yes, it is possible to use the same token twice.

SableCC does not only build a parser for your language, it also builds a 
syntax tree (AST) that you can use for implementing your 
interpreter/compiler/etc. In the tree, each element of an alternative 
needs a name. The problem, with your grammar, is that two elements share 
the same name.

Here's a solution:

...
Productions
   expr = [left_plus]:plus [right_plus]:plus;

The "[name]:" syntax allows you to specify an explicit name for an 
element. By default, the name of an element is itself. In other words, 
your grammar was viewed by SableCC as:

   expr = [plus]:plus [plus]:plus;

In the AST, that would make the AExpr.getPlus() method ambiguous, as it 
could refer to either the first or the second plus element. In my 
proposed solution, the AST will contain two distinct methods: 
AExpr.getLeftPlus() and AExpr.getRightPlus().

I hope this helps.

Have fun!

Etienne

Le 12-03-24 06:26, Necto a écrit :

Grammar defenition issue. expr = plus plus;

Necto — 2012-03-24T10:26:13

Hello, everyone!

I've just started using SableCC, and tried to define my grammar:

Package postfix;

Tokens
 plus = '+';

Productions
 expr = plus plus;

When I was compiling it, SableCC failed:


Why does it impossible, to use the same tokens in a one production? Is it
CC's bug, or a principal feature?

Re: Macro preprocessor

Etienne M. Gagnon — 2012-02-09T19:22:55

Hi Jonathan,

You are right; the two parsers can't share the same tokens.

Here's some additional trick, in case it might help you... The main 
grammar parser does not need a real lexer, only your custom one. I 
recommend that you define the tokens without providing a regular 
expression :

Tokens
   l_par = ;
   r_par = ;
   identifier = ;
   ...

This will cause SableCC to generate the token classes, but will not 
waste time generating an unneeded lexical automaton.

I'm glad my recommendation helped.

Have fun and thanks for using SableCC!

Etienne

On 2012-02-09 12:46, Jonathan Lee write:

Re: Macro preprocessor

Jonathan Lee — 2012-02-09T17:46:08

Hi Etienne,

Thank you for the quick response! I followed your recommendation of
creating a preprocessor grammar, parsing the input, then traversing
the AST saving encountered tokens into a separate list. When a macro
is found, the macro's tokens are added to the list instead.  A custom
lexer returns these tokens to the main parser. This works for me so
far, and I'm able to do it entirely within SableCC!

One caveat is that the two parsers don't share the same Token classes,
so the preprocessor ones need to be translated to ones used main
grammar. I assume that there's no way for the two generated parsers to
share Tokens, but please let me know if that's not the case.

Thanks for your help!

Jonathan

On Wed, Feb 8, 2012 at 9:17 PM, Etienne M. Gagnon <egagnon< at >j-meg.com> wrote:

Re: Macro preprocessor

Etienne M. Gagnon — 2012-02-09T02:17:04

Hi Jonathan,

I did some reading of the Erlang specification ( http:// 
www.erlang.org/download/erl_spec47.ps.gz ). On page 107, the steps, for 
compiling Erlang, are enumerated.

The Erlang preprocessor does not operate on a character stream (i.e. 
strings). Instead, it operates on a token stream and it delivers a new 
token stream (not a string).

To achieve this in SableCC, I would personally implement the system as 
follows:

 1. develop a Lexer using SableCC, to scan the original text file.
 2. develop a Parser/AST for the preprocessor grammar using SableCC.
 3. implement the preprocessor using tree walkers (DepthFirstAdaptor). I
    would create/forward generated tokens into a token list.
 4. develop a custom Lexer that will return the tokens stored in the
    token list.
 5. develop a Parser/AST for the main Erlang grammar. The parser would
    be instantiated with the custom lexer as parameter.
 6. implement the rest of the compiler.

There is no "standard way" of doing this. The approach has to be a

Macro preprocessor

Jonathan Lee — 2012-02-08T18:42:37

Hello everyone,

I am very new to SableCC, so please bear with me.  I would like to
implement an Erlang parser, and I've written the main grammar.
However, Erlang supports preprocessing.  For example, macros can be
defined and applied as:

  -define(MACRO, "abc").
  f() -> ?MACRO.

I am trying to understand how to create a preprocessor that integrates
with SableCC.  I've looked at the Java 1.5 unicode preprocessor, but
that is much simpler than what I need -- it simply replaces escaped
tokens as they occur.

I've thought of writing a preprocessor grammar, parsing the text once,
modifying the AST, outputting back to a string, and finally reparsing
with the main grammar.  But my gut feel is that that's a poor
approach, especially with the need to output an intermediate string.

Is there a standard way of doing this, or an example of this anywhere?
Is there a simple way to chain a preprocessor and parser together in
SableCC?

Thanks for your help!

Jonathan

Re: Thank you Etienne!

Etienne M. Gagnon — 2011-12-19T16:48:29

Hi Emanuele,

Congratulations, Master Ianni!

I have looked at the demo: what you have done is really awesome! I 
really like your concept of syntax-directed coloring, instead of the 
usual lexical-directed coloring.

I wish you wonderful holidays and a lot of success!

Etienne

On 2011-12-17 11:04, Emanuele Ianni wrote:

Thank you Etienne!

Emanuele Ianni — 2011-12-17T16:04:26

Hi Etienne and everybody else! I would like to thank you, Etienne, for your
help in this mailing list during this year in which I wrote my thesis. Two
days ago I got my master of science in computer engineering with
first-class honours. I formalized a grammar of a DSL language used by my
department. Then I also implemented an editor with syntax coloring and some
other feature you usually find in tools like Eclipse. I enclose a video of
the demo, everything you see there works on top of SableCC3.2. I could
write only one grammar and define more types of Lexer, Parser (I used the
filter method -A LOT-) and Translation Rules (different extensions of the
DepthFirstAdapter).

http://dl.dropbox.com/u/14701979/tesi_manu3.mpg

Now I'm applying for a Ph.D at Inria.fr, finger crossed!

How do you traverse the AST?

Emanuele Ianni — 2011-12-03T20:08:36

Hi! I was curious about you implement the depth first used in the
tree-walker. I'm asking because I've noticed that it's pretty cool because,
"obviously", the tree-walker goes first on the nodes which are deepest. In
my case I have an instruction like: showattr("degree") NOT (v.label == 10)
and I noticed that it goes first in "degree" and then in 10 which are the
deepest grammar rules. Do you order the children in some way? Do you apply
some euristic?

Re: Java 1.5 grammar with CST -> AST;issue with default case in switch statements

Janus Dam Nielsen — 2011-12-02T11:45:20

Hi Thomas,

Did you integrate the patch in the grammar?

If so did you create and include a testcase for it as well? 

Den 02/12/2011 kl. 12.00 skrev Thomas Schwinge:


____________________________________________________

Janus Dam Nielsen

Research and Innovationspecialist, CSM, PhD.
CENTRE FOR IT-SECURITY

THE ALEXANDRA INSTITUTE LTD. 

T +45 40 83 09 10
E janus.nielsen< at >alexandra.dk
W alexandra.dk

See our blog about security at blog.sikkerhed.alexandra.dk
____________________________________________________

_______________________________________________
SableCC-Discussion mailing list
SableCC-Discussion< at >lists.sablecc.org
http://lists.sablecc.org/listinfo/sablecc-discussion

Re: Java 1.5 grammar with CST -> AST;issue with default case in switch statements

Thomas Schwinge — 2011-12-02T11:00:53

Hi!

On Mon, 21 Nov 2011 17:00:26 +0100, I wrote:

Here is a patch; Janus has confirmed in private email that this matches
the method he'd use to address the issue:

--- java5.sablecc
+++ java5.sablecc
< at >< at > -1075,9 +1075,9 < at >< at > switch_block {-> statement*} =
 switch_block_statement_group {-> statement} =
     switch_label+ block_statement+ {-> New statement.switch_block([switch_label.expression], [block_statement.statement])};
 
-switch_label {-> expression?} =
+switch_label {-> expression} =
     {expression} case constant_exp colon {-> constant_exp.expression} |
-    {default} default colon {-> Null};
+    {default} default colon {-> New expression.default()};
 
 while_statement {-> statement} =
     while condition statement {-> New statement.while(condition.expression, statement)};
< at >< at > -1677,6 +1677,7 < at >< at > finally_clause =
 statement*;
 
 expression =
+{default} |
 {annotation} annotation |
 {values} [values]:expression* |
 {value_pair} identifier [value]:expression |


Grüße,
 Thomas
__________________

Java 1.5 grammar with CST -> AST;issue with default case in switch statements

Thomas Schwinge — 2011-11-21T16:00:26

Hi!

In my diploma thesis/project I'm using the Java 1.5 grammar as originally
published by Stefan Mandel in
<http://lists.sablecc.org/pipermail/sablecc-user/binrUhxOff8TC.bin> in
message <http://lists.sablecc.org/pipermail/sablecc-user/msg00435.html>,
then picked up by Janus Nielsen in
<http://janusnielsen.wordpress.com/2007/04/18/java-5-grammar-for-sablecc-32/>,
<http://janusnielsen.wordpress.com/2008/06/08/java-5-grammar-for-sablecc-32-now-with-copyright/>,
and, after me noticing that the links are 404 again, published in a
public repository, <http://bitbucket.org/jdn/java5grammar>.

That advantage over the grammar on
<http://sablecc3.sablecc.org/wiki/Java-1.5> is that it already contains
CST -> AST transformation rules.

To make it work with stock SableCC 3.2, I'm using the following tiny
patch on top of this version.  (The change log comment on top of the file
would also need updating.)

--- /home/thomas/tmp/source/sablecc/java5grammar/master/grammar/java5.sablecc2011-03-23 17:59:45.000000000 +0100
+++

Re: Precedence keyword

Etienne M. Gagnon — 2011-10-11T19:53:16

Hi Stéphane,

Yes, it is possible. Here's an example :

Grammar example:

  Lexer

   number = digit+;
   id = letter (letter | digit)*;
   zero = '0' | 'zero';
   digit = '0'..'9';
   letter = 'a'..'z' | 'A'..'Z';

   Token
     number, id, zero, 'if';

   Precedence
     zero>  id;
     zero>  number;

Note that I had to create an unusual token, zero, to illustrate this. In 
the normal cases that we see in real grammars, the /natural precedence/ 
that SableCC applies between tokens eliminates the need to specify 
precedence. We say that a token /A/ has natural precedence over a token 
/B/ if the set of strings that /A/ matches is a subset of the set of 
strings matched by /B/. For example, the 'if' token has natural 
precedence over the id token.

Have fun!

Etienne

On 2011-10-11 13:58, "Stéphane HENRY" wrote:

Re: Precedence keyword

Stéphane HENRY — 2011-10-11T17:58:55

Thanks,
is it possible to have multiple Precedence rules in one line using one
"Precedence" keyword ?

Stéphane

Re: Precedence keyword

Etienne M. Gagnon — 2011-10-11T12:28:53

Hi Stéphane,

In the Lexer section, you can only put a single Precedence subsection.

The Precedence subsection must be located after the Token/Ignored 
subsections.

Have fun!

Etienne

On 2011-10-10 22:20, "Stéphane HENRY" wrote:

Precedence keyword

Stéphane HENRY — 2011-10-11T02:20:53

Hi,

is it possible to have the "Precedence" keyword several times ?

I get this message :
Syntax error on unexpected PrecedenceKeyword token "Precedence":
 expecting: EOF.

I use SableCC version 4-beta.4

Thanks

Stéphane

Re: Extending Depth First Adapter

César — 2011-09-23T03:43:22

Hi Ettiene, many thanks for the answer.

I ended solving the problem using a stack to store the "result" values.

César.

2011/9/18 Etienne M. Gagnon <egagnon< at >j-meg.com>:

Re: Extending Depth First Adapter

Etienne M. Gagnon — 2011-09-18T17:28:43

Hi Cesar,

Have a look at this archived message: 
http://lists.sablecc.org/pipermail/sablecc-discussion/msg00801.html .

It contains, as attachment ( 
http://lists.sablecc.org/pipermail/sablecc-discussion/zipIc4GGuoL76.zip 
), the source code of a little interpreter that provides a template for 
evaluating expressions. The key highlight is that it allows you to 
evaluate an expression as follows:

Value result = eval(node.getExp());

The eval() function is probably what you are looking for.

Let us know if this helps.

Have fun!

Etienne

On 2011-09-17 17:58, César wrote:

Re: A Wrapper for PushbackReader

Phuc Luoi — 2011-09-18T10:09:03

Many thank  for your answer.
I think the idea of  parsing partial files is great.

Hong Phuc

Extending Depth First Adapter

César — 2011-09-17T21:58:24

Hello.

I'm coding the AST -> IR (tree) phase of my compiler and I'm a little
lost in how to represent the tree and even extends the adapter.

Does any of you have some code snippet that I could use as a base for my own?

The way I thought implementing it uses the return of methods "caseXXX"
but these methods must be void, so I thought to use global vars but I
think this will be a little messy. Any idea?


thanks.
César.