While synthesizing and repairing regular expressions (regexes) based on Programming-by-Examples (PBE) methods have seen rapid progress in recent years, all existing works only support synthesizing or repairing regexes for membership testing, and the support for extraction is still an open problem. This paper fills the void by proposing the first PBE-based method for synthesizing and repairing regexes for extraction. Our work supports regexes that have real-world extensions such as backreferences and lookarounds. The extensions significantly affect the PBE-based synthesis and repair problem. In fact, we show that there are unsolvable instances of the problem if the synthesized regexes are not allowed to use the extensions, i.e., there is no regex without the extensions that correctly classify the given set of examples, whereas every problem instance is solvable if the extensions are allowed. This is in stark contrast to the case for the membership where every instance is guaranteed to have a solution expressible by a pure regex without the extensions. The main contribution of the paper is an algorithm to solve the PBE-based synthesis and repair problem for extraction. Our algorithm builds on existing methods for synthesizing and repairing regexes for membership testing, i.e., the enumerative search algorithms with SMT constraint solving. However, significant extensions are needed because the SMT constraints in the previous works are based on a non-deterministic semantics of regexes. Non-deterministic semantics is sound for membership but not for extraction, because which substrings are extracted depends on the deterministic behavior of actual regex engines. To address the issue, we propose a new SMT constraint generation method that respects the deterministic behavior of regex engines. For this, we first define a novel formal semantics of an actual regex engine as a deterministic big-step operational semantics, and use it as a basis to design the new SMT constraint generation method. The key idea to simulate the determinism in the formal semantics and the constraints is to consider continuations of regex matching and use them for disambiguation. We also propose two new search space pruning techniques called approximation-by-pure-regex and approximation-by-backreferences that make use of the extraction information in the examples. We have implemented the synthesis and repair algorithm in a tool called R3 (Repairing Regex for extRaction) and evaluated it on 50 regexes that contain real-world extensions. Our evaluation shows the effectiveness of the algorithm and that our new pruning techniques substantially prune the search space.
Wed 21 JunDisplayed time zone: Eastern Time (US & Canada) change
16:00 - 18:00 | PLDI: Parsing & Formal LanguagesPLDI Research Papers at Cypress 2 Chair(s): Eric Eide University of Utah | ||
16:00 20mTalk | Search-Based Regular Expression Inference on a GPU PLDI Research Papers DOI Pre-print | ||
16:20 20mTalk | Derivative Based Nonbacktracking Real-World Regex Matching with Backtracking Semantics PLDI Research Papers Dan Moseley Microsoft DevDiv, Mario Nishio Microsoft Azure, Jose Perez Rodriguez Microsoft DevDiv, Olli Saarikivi Microsoft Research, Redmond, Stephen Toub Microsoft DevDiv, Margus Veanes Microsoft, Tiki Wan Microsoft Azure, Eric Xu Microsoft, USA DOI | ||
16:40 20mTalk | Repairing Regular Expressions for Extraction PLDI Research Papers DOI | ||
17:00 20mTalk | Recursive State Machine Guided Graph Folding for Context-Free Language Reachability PLDI Research Papers Yuxiang Lei University of New South Wales, Yulei Sui University of New South Wales, Sydney, Shin Hwei Tan Concordia University, Qirun Zhang Georgia Institute of Technology DOI | ||
17:20 20mTalk | Interval Parsing Grammars for File Format Parsing PLDI Research Papers Jialun Zhang Pennsylvania State University, Greg Morrisett Cornell University, Gang (Gary) Tan Pennsylvania State University DOI | ||
17:40 20mTalk | flap: A Deterministic Parser with Fused Lexing PLDI Research Papers Jeremy Yallop University of Cambridge, Ningning Xie University of Toronto, Neel Krishnaswami University of Cambridge DOI Pre-print |