Architecture — Parsanol

The 5 Approaches for Ruby Parsing

Parsanol offers 5 different approaches for parsing in Ruby, each moving more work from Ruby to Rust:

┌──────────────────────────────────────────────────────────────────────────────┐
│                    5 APPROACHES FOR RUBY PARSING                             │
│                                                                              │
│   Each approach moves more work from Ruby to Rust, increasing performance.   │
│   Measured with Expressir parsing EXPRESS schemas (22KB file).               │
└──────────────────────────────────────────────────────────────────────────────┘


APPROACH 1: parslet-ruby (BASELINE)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   INPUT     │     │  PARSLET    │     │   OUTPUT    │
│   String    │────▶│  (Ruby)     │────▶│  Ruby Hash  │
└─────────────┘     └─────────────┘     └─────────────┘
                          │
                    SLOW parsing
                    Pure Ruby

SPEED: 1x (baseline) - 425s (SRL benchmark: 179K lines)


APPROACH 2: parsanol-ruby
━━━━━━━━━━━━━━━━━━━━━━━━━
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   INPUT     │     │  PARSANOL   │     │   OUTPUT    │
│   String    │────▶│  (Ruby)     │────▶│  Ruby Hash  │
└─────────────┘     └─────────────┘     └─────────────┘

SPEED: ~1x (equivalent to Parslet)


APPROACH 3: parsanol-native (Batch)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   INPUT     │     │  PARSANOL   │     │   OUTPUT    │
│   String    │────▶│  (Rust)     │────▶│  Ruby Hash  │
└─────────────┘     └─────────────┘     └─────────────┘
                          │
                    FAST parsing
                    AST via u64 array

SPEED: ~17x faster - 54s (SRL benchmark)


APPROACH 4: parsanol-native (ZeroCopy)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
┌─────────────┐     ┌─────────────────────────┐
│   INPUT     │     │        PARSANOL         │
│   String    │────▶│        (Rust FFI)       │────▶ Ruby Objects
└─────────────┘     │  Direct construction    │
                    └─────────────────────────┘

SPEED: ~17x faster


APPROACH 5: parsanol-native (ZeroCopy + Slice) ← FASTEST
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
┌─────────────┐     ┌────────────────────────────────────┐
│   INPUT     │     │            PARSANOL                │
│   String    │────▶│            (Rust)                  │────▶ Slice Objects
└─────────────┘     │  Zero-copy + Source positions      │
                    └────────────────────────────────────┘
                          │
                    FASTEST parsing
                    Source position tracking

SPEED: ~17x faster - 54s (17x vs baseline)
FEATURES: Preserves source positions for linters, IDEs

New: Slice Support - ZeroCopy + Slice mode preserves source positions (offset, length) for each parsed element. This is essential for tools like linters, IDEs, and Expressir that need to track where in the source code each element came from.

Evidence-Based: These results are measured with Expressir parsing real EXPRESS schemas. Run the benchmarks yourself: bundle exec ruby benchmark/run_all.rb --quick

Evidence-Based Benchmarks

These are actual benchmark results from Expressir parsing the SRL benchmark (135 schemas, 179K lines). This represents a real-world production workload.

Mode	Time	Speedup	Notes
Ruby (Parslet)	425,000 ms	1x (baseline)	Pure Ruby parsing (SRL benchmark)
Native Batch (u64)	54,400 ms	17x faster	AST via u64 array transfer
Native ZeroCopy (Slice)	54,400 ms	17x faster	Zero-copy with source positions

JSON Parsing Benchmarks

Input Size	parslet-ruby	parsanol-native	parsanol-zerocopy
tiny (76 bytes)	1,156 iter/s	4,745 iter/s	33,000+ iter/s
small (791 bytes)	111 iter/s	469 iter/s	3,200+ iter/s
medium (8KB)	9.3 iter/s	47.2 iter/s	270+ iter/s

Note: Results vary based on hardware, Ruby version, and input complexity. Always run benchmarks on your own system for accurate comparisons.

Slice Support for Source Positions

The ZeroCopy + Slice mode preserves source position information, which is essential for tools that need to track where in the source code each element came from.

# Before (plain strings):
[{"word"=>"hello"}, " ", {"name"=>"world"}]

# After (Slice objects with position info):
[{"word"=>"hello"@0}, " "@5, {"name"=>"world"@6}]

# The @N notation shows the byte offset in the original input

Parslet::Slice Compatible

Parsanol::Slice objects are compatible with Parslet::Slice, making migration seamless for existing code that depends on source position tracking.

Offset + Length

Each slice contains the byte offset and length, allowing tools to precisely locate and highlight source code elements.

Use Cases

Linters

Show precise error locations with line/column information derived from byte offsets.

IDE Integration

Enable go-to-definition, find-references, and syntax highlighting based on parsed structure.

Code Generation

Generate output that preserves mappings back to original source locations.

When to Use Which Approach

Approach	Best For	Trade-offs
parslet-ruby	Existing Parslet code, maximum compatibility	Slowest, but most reliable
parsanol-native (Batch)	Need Ruby objects, good performance	AST transfer overhead
ZeroCopy + Slice	Need source positions (linters, IDEs)	Best overall - fastest + position tracking

Rust Core Engine

The Rust core provides the parsing engine with maximum performance:

Packrat Parser

Memoization-based parsing guarantees O(n) time complexity for PEG grammars. No exponential backtracking, predictable performance.

Arena Allocation

All AST nodes are allocated in a single arena. Zero individual allocations, perfect cache locality, instant cleanup.

SIMD Acceleration

Uses memchr for fast substring search, AHash for fast hashing, SmallVec for stack-allocated small collections.

Zero-Copy Slices

InputRef nodes return slice references with offset/length instead of copying strings, preserving source position information.

Parslet Compatibility Layer

For existing Parslet users, we provide a drop-in compatibility layer:

┌─────────────────────────────────────────────────────────────────────┐
│                    PARSLET COMPATIBILITY LAYER                      │
│                                                                     │
│  require 'parslet'           →        require 'parsanol/parslet'    │
│                                                                     │
│  class MyParser < Parslet::Parser    class MyParser < Parsanol::    │
│    rule(:expr) { ... }                 Parslet::Parser              │
│  end                                     rule(:expr) { ... }        │
│                                        end                          │
│                                                                     │
│  NO CODE CHANGES REQUIRED!                                          │
│                                                                     │
│  Benefits:                                                          │
│  • Same API as Parslet                                              │
│  • Same output format (including Slice objects)                     │
│  • Rust backend by default - ~17x faster                             │
└─────────────────────────────────────────────────────────────────────┘

Running Benchmarks Yourself

All performance claims on this page can be verified by running the benchmarks on your own machine:

# Clone and setup
git clone https://github.com/parsanol/parsanol-ruby
cd parsanol-ruby
bundle install

# Build native extension (requires Rust)
bundle exec rake compile

# Run quick benchmarks (skips large inputs)
bundle exec ruby benchmark/run_all.rb --quick

# Run all benchmarks
bundle exec ruby benchmark/run_all.rb

# View the approaches diagram
cat benchmark/APPROACHES.md

Benchmark results are saved to benchmark/reports/ as JSON files for comparison over time.