Lexers and parsers are typically defined separately and connected by a token stream. This separate definition is important for modularity and reduces the potential for parsing ambiguity. However, materializing tokens as data structures and case-switching on tokens comes with a cost.
We show how to fuse separately-defined lexers and parsers, drastically improving performance without compromising modularity or increasing ambiguity. We propose a deterministic variant of Greibach Normal Form that ensures deterministic parsing with a single token of lookahead and makes fusion strikingly simple, and prove that normalizing context free expressions into the deterministic normal form is semantics-preserving. Our staged parser combinator library, flap
, provides a standard interface, but generates specialized token-free code that runs two to six times faster than ocamlyacc
on a range of benchmarks.
Wed 21 JunDisplayed time zone: Eastern Time (US & Canada) change
16:00 - 18:00 | PLDI: Parsing & Formal LanguagesPLDI Research Papers at Cypress 2 Chair(s): Eric Eide University of Utah | ||
16:00 20mTalk | Search-Based Regular Expression Inference on a GPU PLDI Research Papers DOI Pre-print | ||
16:20 20mTalk | Derivative Based Nonbacktracking Real-World Regex Matching with Backtracking Semantics PLDI Research Papers Dan Moseley Microsoft DevDiv, Mario Nishio Microsoft Azure, Jose Perez Rodriguez Microsoft DevDiv, Olli Saarikivi Microsoft Research, Redmond, Stephen Toub Microsoft DevDiv, Margus Veanes Microsoft, Tiki Wan Microsoft Azure, Eric Xu Microsoft, USA DOI | ||
16:40 20mTalk | Repairing Regular Expressions for Extraction PLDI Research Papers DOI | ||
17:00 20mTalk | Recursive State Machine Guided Graph Folding for Context-Free Language Reachability PLDI Research Papers Yuxiang Lei University of New South Wales, Yulei Sui University of New South Wales, Sydney, Shin Hwei Tan Concordia University, Qirun Zhang Georgia Institute of Technology DOI | ||
17:20 20mTalk | Interval Parsing Grammars for File Format Parsing PLDI Research Papers Jialun Zhang Pennsylvania State University, Greg Morrisett Cornell University, Gang (Gary) Tan Pennsylvania State University DOI | ||
17:40 20mTalk | flap: A Deterministic Parser with Fused Lexing PLDI Research Papers Jeremy Yallop University of Cambridge, Ningning Xie University of Toronto, Neel Krishnaswami University of Cambridge DOI Pre-print |