question about tokens

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

question about tokens

Wieger Wesselink
Hello,

I have the following grammar that parses the string "x2" successfully
as two different tokens:

----
%tokens%
OPERATOR         = "x"
NUMBER           = <<[0-9]*>>
WHITESPACE       = <<[ \t\n\r]+>> %ignore%

%productions%
Value            = OPERATOR NUMBER ;
---

When I add the token definition IDENTIFIER = <<[a-zA-Z][a-zA-Z0-9-_]*>>
,
the string "x2" results in a parse error, since it is parsed as a single
IDENTIFIER token. Is there something that I can do to prevent this?

Best regards,

Wieger

--
http://www.fastmail.fm - mmm... Fastmail...


_______________________________________________
Grammatica-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/grammatica-users
Reply | Threaded
Open this post in threaded view
|

Re: question about tokens

Oliver Gramberg-2
Hi,

> Is there something that I can do to prevent this?

this is a classical problem in languages that have operators that look like identifiers. Not much you can do from the grammar side. Have the user insert WHITESPACE between OPERATOR and NUMBER, though...


Regards,
Oliver
--
Diese Signatur besteht zu 100% aus wiederverwendeten Pixeln.


Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de

_______________________________________________
Grammatica-users mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/grammatica-users