Skip to content
← Back to Examples

Incremental Parsing

Rustconceptualadvanced

Demonstrates how to reparse only affected regions after edits for editor integration.

Key Concepts

  • incremental parsing
  • region tracking
  • edit application
  • partial reparse

Why Parse This Format?

Incremental parsing enables fast response in editors by only reparsing changed portions of the input.

Use Cases

  • IDE integration with real-time parsing
  • Code editors with syntax highlighting
  • Collaborative editing tools
  • Live preview systems

Input Format

Any text that can be edited incrementally.

text with edit operations

Examples

Hello\nWorld\nTestVALID

Initial input with three lines

Hello\nBeautiful World\nTestVALID

After inserting 'Beautiful ' at position 6

Output Format

Parse results with region tracking for incremental updates.

Structure

regions
array

List of parsed regions with validity status

invalid_count
integer

Number of regions needing reparse

How the Code Works

This section explains how the parsing rules map to the format semantics.

Region Structure

struct Region {
    start: usize,
    end: usize,
    rule: String,
    valid: bool,
}

Track parsed content with positions: Each region knows its position and validity status.

Initial Parse

fn parse_full(&mut self, input: &str) {
    // Build regions based on structure
    for line in input.lines() {
        self.regions.push(Region {
            start: line_start,
            end: line_end,
            rule: "line",
            valid: true,
        });
    }
}

Parse and record all regions: Store regions for later incremental updates.

Edit Application

fn apply_edit(&mut self, pos: usize, delete_len: usize, insert: &str) {
    // Find affected region range
    let affected_start = self.find_region_start(pos);
    let affected_end = self.find_region_end(pos + delete_len);

    // Mark affected regions as invalid
    for region in &mut self.regions {
        if region.start >= affected_start && region.end <= affected_end {
            region.valid = false;
        }
    }
}

Apply edit and find affected regions: Only mark affected regions for reparse.

Incremental Reparse

fn reparse_invalid(&mut self) {
    for region in &mut self.regions {
        if !region.valid {
            // Reparse only this region
            region.valid = true;
        }
    }
}

Reparse only invalid regions: O(affected) instead of O(entire input).

Position Adjustment

let delta = insert.len() as isize - delete_len as isize;
for region in &mut self.regions {
    if region.start > pos + delete_len {
        region.start = (region.start as isize + delta) as usize;
        region.end = (region.end as isize + delta) as usize;
    }
}

Update positions after edits: Maintain correct positions after insertions/deletions.

Position Adjustment

let delta = insert.len() as isize - delete_len as isize;
for region in &mut self.regions {
    if region.start > pos + delete_len {
        region.start = (region.start as isize + delta) as usize;
        region.end = (region.end as isize + delta) as usize;
    }
}

Update positions after edits: Maintain correct positions after insertions/deletions.

Source Code

Rust
//! Incremental Parser Example
//!
//! Demonstrates how to reparse only affected regions after edits.
//! Useful for editor integration and IDE features.
//!
//! Run with: cargo run --example incremental --no-default-features

#![allow(clippy::print_literal)]

use parsanol::portable::{
    parser_dsl::{re, GrammarBuilder},
    Grammar,
};

/// Represents a region of parsed content
#[derive(Debug, Clone)]
#[allow(dead_code)]
struct Region {
    start: usize,
    end: usize,
    rule: String,
    valid: bool,
}

/// Incremental parser that tracks regions and can reparse affected areas
#[allow(dead_code)]
struct IncrementalParser {
    grammar: Grammar,
    regions: Vec<Region>,
    last_input: String,
}

impl IncrementalParser {
    fn new(grammar: Grammar) -> Self {
        Self {
            grammar,
            regions: Vec::new(),
            last_input: String::new(),
        }
    }

    /// Parse the entire input and record regions
    fn parse_full(&mut self, input: &str) {
        self.last_input = input.to_string();
        self.regions.clear();

        // Build simple regions based on lines
        let mut pos = 0;
        for line in input.lines() {
            let line_start = pos;
            let line_end = pos + line.len();

            self.regions.push(Region {
                start: line_start,
                end: line_end,
                rule: "line".to_string(),
                valid: true,
            });

            pos = line_end + 1; // +1 for newline
        }

        println!("Parsed {} regions", self.regions.len());
    }

    /// Apply an edit and return affected region range
    fn apply_edit(&mut self, pos: usize, delete_len: usize, insert: &str) -> String {
        let old_input = self.last_input.clone();

        // Apply the edit
        let new_input = format!(
            "{}{}{}",
            &old_input[..pos],
            insert,
            &old_input[pos + delete_len..]
        );

        // Find affected regions
        let affected_start = self.find_region_start(pos);
        let affected_end = self.find_region_end(pos + delete_len);

        // Mark affected regions as invalid
        for region in &mut self.regions {
            if region.start >= affected_start && region.end <= affected_end {
                region.valid = false;
            }
        }

        // Update region positions after the edit
        let delta = insert.len() as isize - delete_len as isize;
        for region in &mut self.regions {
            if region.start > pos + delete_len {
                region.start = (region.start as isize + delta) as usize;
                region.end = (region.end as isize + delta) as usize;
            }
        }

        self.last_input = new_input.clone();
        new_input
    }

    /// Reparse only invalid regions
    fn reparse_invalid(&mut self) {
        let invalid_count = self.regions.iter().filter(|r| !r.valid).count();
        println!("Reparsing {} invalid regions...", invalid_count);

        // In a real implementation, we would only reparse these regions
        for region in &mut self.regions {
            if !region.valid {
                // Reparse this region
                region.valid = true;
            }
        }

        println!("Done! Reparsed {} regions", invalid_count);
    }

    fn find_region_start(&self, pos: usize) -> usize {
        for region in &self.regions {
            if region.start <= pos && region.end >= pos {
                return region.start;
            }
        }
        pos
    }

    fn find_region_end(&self, pos: usize) -> usize {
        for region in self.regions.iter().rev() {
            if region.start <= pos && region.end >= pos {
                return region.end;
            }
        }
        pos
    }
}

fn main() {
    println!("Incremental Parser Example");
    println!("==========================");
    println!();

    println!("This example demonstrates how to update parse results after edits.");
    println!("Useful for IDE/editor integration.\n");

    let grammar = GrammarBuilder::new().rule("line", re(r"[^\n]*")).build();

    let mut parser = IncrementalParser::new(grammar);

    // Initial parse
    println!("--- Initial Parse ---");
    let input = "Hello\nWorld\nTest";
    parser.parse_full(input);

    // Apply edit
    println!("\n--- Apply Edit ---");
    println!("Edit: Insert 'Beautiful ' at position 6");
    let new_input = parser.apply_edit(6, 0, "Beautiful ");
    println!("New input: {:?}", new_input);

    // Reparse only affected regions
    println!("\n--- Incremental Reparse ---");
    parser.reparse_invalid();

    println!("\n--- Benefits ---");
    println!("* O(affected regions) instead of O(entire input)");
    println!("* Constant memory usage");
    println!("* Fast response for user edits");
}