Scope Atoms

RustRubyconceptualintermediate

Create isolated capture contexts with scope atoms. Captures made inside a scope are discarded when the scope exits, preventing capture pollution.

Key Concepts

capture isolation
scope nesting
recursive parsing
automatic capture cleanup

Why Parse This Format?

Scopes prevent inner captures from polluting outer contexts. Essential for parsing nested structures where each level has its own context.

Use Cases

Parsing nested configuration with per-section captures
Recursive descent parsing without capture accumulation
Processing repeated structures with isolated state
Preventing capture name collisions

Input Format

Nested structures where inner contexts should be isolated.

[section1]
key=value
[section2]
key=value

Examples

[database]
host=localhost
port=5432
[server]
port=8080

VALID

INI-style config with sections

Output Format

Only captures from outer scope are preserved.

Structure

section

Section name persists to outer scope

key

Keys are isolated within each section scope

value

Values are isolated within each section scope

How the Code Works

This section explains how the parsing rules map to the format semantics.

Why Scopes?

let grammar = GrammarBuilder::new()
    .rule("items", sequence([
        capture("temp", str("a")),
        capture("temp", str("b")),
        capture("temp", str("c")),
    ]))
    .build();
// Result: captures = ["temp" = "c"]  (last wins)

Without scopes, captures accumulate across the entire parse: With scopes, inner captures are automatically discarded: let grammar = GrammarBuilder::new() .rule("outer", sequence([ capture("outer_name", str("prefix")), scope(sequence([ capture("inner_name", re(r"[a-z]+")), // Discarded ])), ])) .build(); // Result: captures = ["outer_name" = "prefix"]

Scope Isolation

scope(sequence([
    capture("key", re(r"[a-z]+")),
    str("="),
    capture("value", re(r"[^\n]+")),
]))

Captures inside `scope()` are: - Available during parsing for inner rules - Automatically discarded when scope exits - Never visible to outer capture inspection

Nested Scopes

let grammar = GrammarBuilder::new()
    .rule("nested", sequence([
        capture("level", str("L1")),      // Persisted
        scope(sequence([
            capture("level", str("L2")),  // Discarded
            scope(sequence([
                capture("level", str("L3")), // Discarded
            ])),
        ])),
    ]))
    .build();
// Result: captures = ["level" = "L1"]

Scopes can be nested. Only the outermost captures persist:

INI Configuration Example

let grammar = GrammarBuilder::new()
    .rule("section", sequence([
        str("["),
        capture("section", re(r"[a-zA-Z_]+")),  // Persists
        str("]\n"),
        scope(repeat("kv_pair", 0, None)),      // Key-values isolated
    ]))
    .build();

Parse sections with isolated key-value pairs: Each section's `key` and `value` captures are isolated.

Recursive Parsing

let grammar = GrammarBuilder::new()
    .rule("paren", sequence([
        str("("),
        scope(choice([
            capture("content", re(r"[^()]+")),
            recursive("paren"),  // Each level has own scope
        ])),
        str(")"),
    ]))
    .build();

Scopes are essential for recursive structures: Each nesting level has isolated captures.

Recursive Parsing

let grammar = GrammarBuilder::new()
    .rule("paren", sequence([
        str("("),
        scope(choice([
            capture("content", re(r"[^()]+")),
            recursive("paren"),  // Each level has own scope
        ])),
        str(")"),
    ]))
    .build();

Scopes are essential for recursive structures: Each nesting level has isolated captures.

Source Code

Rust

//! Scope Atoms Example
//!
//! Demonstrates how to create isolated capture contexts with scope atoms.
//! Captures made inside a scope are discarded when the scope exits.
//!
//! Run with: cargo run --example scopes --no-default-features

#![allow(clippy::print_literal)]

use parsanol::portable::{
    parser_dsl::{capture, dynamic, re, scope, seq, str, GrammarBuilder},
    AstArena, Atom, Grammar, PortableParser,
};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    println!("Scope Atoms Example");
    println!("===================\n");

    // =========================================================================
    // Example 1: Basic Scope Isolation
    // =========================================================================
    println!("--- Example 1: Basic Scope Isolation ---\n");

    // Without scope: captures accumulate, last value wins
    let grammar = GrammarBuilder::new()
        .rule(
            "items",
            seq(vec![
                dynamic(capture("temp", dynamic(str("a")))),
                dynamic(str("b")),
                dynamic(capture("temp", dynamic(str("c")))),
            ]),
        )
        .build();

    let input = "abc";
    let mut arena = AstArena::for_input(input.len());
    let mut parser = PortableParser::new(&grammar, input, &mut arena);
    let result = parser.parse_from_pos(0)?;

    println!("  Without scope:");
    println!("    Capture names: {:?}", result.capture_names());
    println!("    'temp' value:  {:?}", result.get_capture("temp", input)); // "c" (last wins)

    // With scope: inner captures are discarded
    let grammar = GrammarBuilder::new()
        .rule(
            "outer",
            seq(vec![
                dynamic(capture("outer_name", dynamic(str("prefix")))),
                dynamic(str(" ")),
                dynamic(scope(seq(vec![
                    dynamic(capture("inner_name", dynamic(re(r"[a-z]+")))),
                    dynamic(str(" ")),
                    dynamic(capture("inner_value", dynamic(re(r"\d+")))),
                ]))),
                dynamic(str(" ")),
                dynamic(capture("outer_value", dynamic(str("suffix")))),
            ]),
        )
        .build();

    let input = "prefix hello 123 suffix";
    let mut arena = AstArena::for_input(input.len());
    let mut parser = PortableParser::new(&grammar, input, &mut arena);
    let result = parser.parse_from_pos(0)?;

    println!("\n  With scope:");
    println!("    Capture names: {:?}", result.capture_names());
    println!(
        "    'outer_name':  {:?}",
        result.get_capture("outer_name", input)
    );
    println!(
        "    'outer_value': {:?}",
        result.get_capture("outer_value", input)
    );
    // Note: inner_name and inner_value are NOT in the result
    println!("    (inner_name and inner_value are discarded)");

    // =========================================================================
    // Example 2: Nested Scopes
    // =========================================================================
    println!("\n--- Example 2: Nested Scopes ---\n");

    let grammar = GrammarBuilder::new()
        .rule(
            "outer",
            seq(vec![
                dynamic(capture("level", dynamic(str("L1")))),
                dynamic(str(" ")),
                dynamic(scope(seq(vec![
                    dynamic(capture("level", dynamic(str("L2")))),
                    dynamic(str(" ")),
                    dynamic(scope(seq(vec![dynamic(capture(
                        "level",
                        dynamic(str("L3")),
                    ))]))),
                ]))),
            ]),
        )
        .build();

    let input = "L1 L2 L3";
    let mut arena = AstArena::for_input(input.len());
    let mut parser = PortableParser::new(&grammar, input, &mut arena);
    let result = parser.parse_from_pos(0)?;

    println!("  Nested scopes - only L1 persists:");
    println!("    Capture names: {:?}", result.capture_names());
    println!(
        "    'level' value:  {:?}",
        result.get_capture("level", input)
    );

    // =========================================================================
    // Example 3: Recursive Structure with Scopes (Raw Grammar API)
    // =========================================================================
    println!("\n--- Example 3: Recursive Structure with Scopes ---\n");

    // Parse balanced parentheses with per-level captures
    let mut grammar = Grammar::new();

    // Inner content (can be text or nested parens)
    let inner_text = grammar.add_atom(Atom::Re {
        pattern: r"[^()]+".into(),
    });

    // Capture text at current level (scoped)
    let text_capture = grammar.add_atom(Atom::Capture {
        name: "text".into(),
        atom: inner_text,
    });

    // Scope wraps the content to isolate captures
    let content = grammar.add_atom(Atom::Scope { atom: text_capture });

    // Full structure - create atoms first to avoid borrow issues
    let open_paren = grammar.add_atom(Atom::Str {
        pattern: "(".into(),
    });
    let close_paren = grammar.add_atom(Atom::Str {
        pattern: ")".into(),
    });

    let paren_content = grammar.add_atom(Atom::Sequence {
        atoms: vec![open_paren, content, close_paren],
    });

    grammar.root = paren_content;

    let input = "(hello)";
    let mut arena = AstArena::for_input(input.len());
    let mut parser = PortableParser::new(&grammar, input, &mut arena);
    let result = parser.parse_from_pos(0)?;

    println!("  Parsed: {}", input);
    println!("  Capture names: {:?}", result.capture_names());
    // Note: "text" capture is discarded because it's inside a scope
    println!("  ('text' capture is discarded because it's inside a scope)");

    // =========================================================================
    // Example 4: Scope for Memory Cleanup
    // =========================================================================
    println!("\n--- Example 4: Scope for Memory Cleanup ---\n");

    // Processing repeated structures - each gets its own scope
    let grammar = GrammarBuilder::new()
        .rule(
            "item",
            scope(seq(vec![
                dynamic(capture("id", dynamic(re(r"\d+")))),
                dynamic(str(":")),
                dynamic(capture("name", dynamic(re(r"[a-zA-Z]+")))),
            ])),
        )
        .rule("items", seq(vec![dynamic(str("item"))]))
        .build();

    let input = "item";
    println!("  Processing repeated items with scoped captures");
    println!("  Input: {}", input);

    let mut arena = AstArena::for_input(input.len());
    let mut parser = PortableParser::new(&grammar, input, &mut arena);
    let result = parser.parse_from_pos(0)?;

    println!("  Final captures: {:?}", result.capture_names());
    println!("  (id and name captures are discarded after each item)");

    // =========================================================================
    // Example 5: Accessing Capture State via Parser
    // =========================================================================
    println!("\n--- Example 5: Accessing Capture State ---\n");

    let grammar = GrammarBuilder::new()
        .rule(
            "outer",
            seq(vec![
                dynamic(capture("name", dynamic(str("outer")))),
                dynamic(scope(dynamic(capture("inner", dynamic(str("inner")))))),
            ]),
        )
        .build();

    let input = "outerinner";
    let mut arena = AstArena::for_input(input.len());
    let mut parser = PortableParser::new(&grammar, input, &mut arena);
    parser.parse()?;

    // Access capture state directly from parser
    let captures = parser.capture_state();
    let names: Vec<_> = captures.names().collect();
    println!("  Capture names from parser: {:?}", names);
    if let Some(value) = captures.get("name") {
        println!("  'name' capture: {:?}", value.get_text(input));
    }
    // Note: "inner" is not available because it was in a scope
    println!("  ('inner' capture is not available - it was in a scope)");

    // =========================================================================
    // Summary
    // =========================================================================
    println!("\n--- Benefits of Scope Atoms ---");
    println!("* Prevent capture pollution from nested parsing");
    println!("* Each recursion level has its own capture state");
    println!("* Automatic cleanup when scope exits");
    println!("* Memory bounded during parse");
    println!("* Essential for parsing nested structures");

    println!("\n--- Performance Notes ---");
    println!("* Scope push/pop is O(c_scope) where c_scope = captures in scope");
    println!("* Each nesting level adds ~2%% overhead");
    println!("* Use scopes liberally - they're cheap");

    println!("\n--- DSL Helper ---");
    println!("  scope(parslet)  // Wraps parslet in isolated capture context");

    println!("\n--- API Summary ---");
    println!("  scope(inner)              -> isolates captures");
    println!("  result.get_capture(name)  -> access captures (inner ones excluded)");

    Ok(())
}

Ruby

# frozen_string_literal: true

# Scope Atoms Example
#
# Demonstrates how to create isolated capture contexts with scope atoms.
# Captures made inside a scope are discarded when the scope exits.

$LOAD_PATH.unshift "#{File.dirname(__FILE__)}/../lib"
require "parsanol/parslet"

include Parsanol::Parslet

puts "Scope Atoms Example"
puts "===================\n"

# ===========================================================================
# Example 1: Basic Scope Isolation
# ===========================================================================
puts "--- Example 1: Basic Scope Isolation ---\n"

# Without scope: captures accumulate, last value wins
parser = str("a").capture(:temp) >> str("b") >> str("c").capture(:temp)

input = "abc"
result = parser.parse(input)

puts "  Without scope:"
puts "    'temp' value: #{result[:temp].inspect}" # "c" (last wins)

# With scope: inner captures are discarded
parser = str("prefix").capture(:outer) >> str(" ") >>
  scope { str("inner").capture(:inner) } >>
  str(" ") >> str("suffix").capture(:outer_end)

input = "prefix inner suffix"
result = parser.parse(input)

puts "\n  With scope:"
puts "    'outer': #{result[:outer].inspect}"
puts "    'outer_end': #{result[:outer_end].inspect}"
puts "    'inner': #{begin
  result[:inner].inspect
rescue StandardError
  'nil'
end}" # Not present

# ===========================================================================
# Example 2: Nested Scopes
# ===========================================================================
puts "\n--- Example 2: Nested Scopes ---\n"

parser = str("L1").capture(:level) >> str(" ") >>
  scope do
    str("L2").capture(:level) >> str(" ") >>
      scope { str("L3").capture(:level) }
  end

input = "L1 L2 L3"
result = parser.parse(input)

puts "  Nested scopes - only L1 persists:"
puts "    'level' value: #{result[:level].inspect}" # "L1"

# ===========================================================================
# Example 3: INI Configuration Parsing
# ===========================================================================
puts "\n--- Example 3: INI Configuration Parsing ---\n"

class IniParser < Parsanol::Parser
  include Parsanol::Parslet

  rule(:newline) { str("\n") | str("\r\n") }
  rule(:whitespace) { match('[ \t]*') }
  rule(:section_header) do
    str("[") >> match("[a-zA-Z_]+").capture(:section) >> str("]") >> whitespace >> newline
  end
  rule(:kv_pair) do
    match("[a-zA-Z_]+").capture(:key) >> str("=") >> match('[^\r\n]+').capture(:value) >> newline
  end
  rule(:section) { section_header >> scope { kv_pair.repeat(1) } }
  rule(:config) { section.repeat(1) }
  root :config
end

input = "[database]\nhost=localhost\nport=5432\n\n[server]\nport=8080\ndebug=true\n"

puts "  Input:\n#{input}"
parser = IniParser.new
result = parser.parse(input)

puts "  Outer captures: #{result.keys}"
puts "  (key/value captures are discarded after each section)"

# ===========================================================================
# Example 4: Scope for Memory Cleanup
# ===========================================================================
puts "\n--- Example 4: Scope for Memory Cleanup ---\n"

# Processing repeated structures - each gets its own scope
class ItemParser < Parsanol::Parser
  include Parsanol::Parslet

  rule(:item) do
    scope do
      match('\d+').capture(:id) >> str(":") >> match("[a-zA-Z]+").capture(:name)
    end
  end
  rule(:items) { str("item") >> item.repeat(1) }
  root :items
end

input = "item123:apple456:banana789:cherry"
puts "  Processing repeated items with scoped captures"
puts "  Input: #{input}"

parser = ItemParser.new
result = parser.parse(input)

puts "  Final captures: #{result.keys}"
puts "  (id and name captures are discarded after each item)"

# ===========================================================================
# Example 5: Scope with Dynamic
# ===========================================================================
puts "\n--- Example 5: Scope with Dynamic ---\n"

# Scope preserves outer captures for dynamic blocks
parser = str("a").capture(:a) >>
  scope { str("b").capture(:a) } >>
  dynamic { |_s, c| str(c.captures[:a]) }

begin
  parser.parse("aba")
  puts "  Parses 'aba' - scope preserved outer capture 'a'"
rescue StandardError
  puts "  Exception - scope isolation working"
end

# ===========================================================================
# Summary
# ===========================================================================
puts "\n--- Benefits of Scope Atoms ---"
puts "* Prevent capture pollution from nested parsing"
puts "* Each recursion level has its own capture state"
puts "* Automatic cleanup when scope exits"
puts "* Memory bounded during parse"
puts "* Essential for parsing nested structures"

puts "\n--- Performance Notes ---"
puts "* Scope push/pop is O(c_scope) where c_scope = captures in scope"
puts "* Each nesting level adds ~2% overhead"
puts "* Use scopes liberally - they're cheap"

puts "\n--- DSL Helper ---"
puts "  scope { parslet }  // Wraps parslet in isolated capture context"

puts "\n--- API Summary ---"
puts "  scope { inner }          -> isolates captures"
puts "  result[:name]            -> access captures (inner ones excluded)"