Lexing newlines in scala StdLexical?

Posted by Nick Fortescue on Stack Overflow See other posts from Stack Overflow or by Nick Fortescue
Published on 2010-04-14T08:26:37Z Indexed on 2010/04/14 12:43 UTC
Read the original article Hit count: 245

Filed under:

scala

|

parser-combinators

|

lexer

|

newline

I'm trying to lex (then parse) a C like language. In C there are preprocessor directives where line breaks are significant, then the actual code where they are just whitespace.

One way of doing this would be do a two pass process like early C compilers - have a separate preprocessor for the # directives, then lex the output of that.

However, I wondered if it was possible to do it in a single lexer. I'm pretty happy with writing the scala parser-combinator code, but I'm not so sure of how StdLexical handles whitespace.

Could someone write some simple sample code which say could lex a #include line (using the newline) and some trivial code (ignoring the newline)? Or is this not possible, and it is better to go with the 2-pass appproach?

© Stack Overflow or respective owner

Related posts about scala

ensime scala errors (class scala.Array not found, object scala not found)

as seen on Stack Overflow - Search for 'Stack Overflow'
I've installed ensime according to the README.md file, however, I get errors in the inferior-ensime-server buffer with the following: INFO: Fatal Error: scala.tools.nsc.MissingRequirementError: object scala not found. scala.tools.nsc.MissingRequirementError: object scala not found. at scala… >>> More
infinite loop shutting down ensime

as seen on Stack Overflow - Search for 'Stack Overflow'
When I run M-X ensime-disconnect I get the following forever: string matching regex `\"((?:[^\"\\]|\\.)*)\"' expected but `^@' found and I see this exception when I use C-c C-c Uncaught exception in com.ensime.server.SocketHandler@769aba32 java.net.SocketException: Broken pipe at java… >>> More
How to resolve java.nio.charset.UnmappableCharacterException in Scala 2.8.0?

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm using Scala 2.8.0 and trying to read pipe delimited file like in code snipped below: object Main { def main(args: Array[String]) :Unit = { if (args.length > 0) { val lines = scala.io.Source.fromPath("QUICK!LRU-2009-11-15.psv") for (line <-lines) print(line) } … >>> More
Question on Scala Closure (From "Programming in Scala")

as seen on Stack Overflow - Search for 'Stack Overflow'
I don't understand why authors said that Code Listing 9.1 from "Programming in Scala" use closure. In chapter 9, they show how to refactor code into more less duplicated form, from this original code: object FileMatcher { private def filesHere = (new java.io.File(".")).listFiles def filesEnding(query:… >>> More
Common idiom in Java to Scala, traverse/Iterate Java list into Scala list

as seen on Stack Overflow - Search for 'Stack Overflow'
I am processing a XML document and iterating through nodes. I want to iterate through the nodes and build a new List of some type. How would I do this with Scala: Here is my XML traverse code: def findClassRef(xmlNode: Elem) = { xmlNode\"classDef" foreach { (entry) => val name… >>> More

Related posts about parser-combinators

Can parser combinators be made efficient?

as seen on Stack Overflow - Search for 'Stack Overflow'
Around 6 years ago, I benchmarked my own parser combinators in OCaml and found that they were ~5× slower than the parser generators on offer at the time. I recently revisited this subject and benchmarked Haskell's Parsec vs a simple hand-rolled precedence climbing parser written in F# and was… >>> More
Scala: How to combine parser combinators from different objects

as seen on Stack Overflow - Search for 'Stack Overflow'
Given a family of objects that implement parser combinators, how do I combine the parsers? Since Parsers.Parser is an inner class, and in Scala inner classes are bound to the outer object, the story becomes slightly complicated. Here's an example that attempts to combine two parsers from different… >>> More
Parser combinators info

as seen on Stack Overflow - Search for 'Stack Overflow'
I am using parsing combinators in scala If I have recursive parser: val parser = "(?ui)(regexvalue)".r | (parser2~value) How can I check how many characters of input my parser consumed? >>> More
Scala parser combinators: how to parse "if(x)" if x can contain a ")"

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm trying to get this to work: def emptyCond: Parser[Cond] = ("if" ~ "(") ~> regularStr <~ ")" ^^ { case s => Cond("",Nil,Nil) } where regularStr is defined to accept a number of things, including ")". Of course, I want this to be an acceptable input: if(foo()). But for any if(x) it is… >>> More
Generalized Bottom up Parser Combinators in Haskell

as seen on Stack Overflow - Search for 'Stack Overflow'
I am wondered why there is no generalized parser combinators for Bottom-up parsing in Haskell like a Parsec combinators for top down parsing. ( I could find some research work went during 2004 but nothing after https://haskell-functional-parsing.googlecode.com/files/Ljunglof-2002a.pdf http://www… >>> More