@lanexio/parser-core

This page documents @lanexio/parser-core, the foundational package that defines the shared buffer protocol, tree data structures, and parser host for all Lanexio Parser packages.

• Version: Stable
• Module name: parser-core
• Package: @lanexio/parser-core
• Import path: @lanexio/parser-core
• Layer: 1 (Core)
• Runtime: Universal (browser, server, edge worker, test runner)
• Module format: ESM
• Stability: Stable
• Primary use case: Build and traverse zero-copy flat ASTs, register grammars, use core primitives.

Layer contract

Layer 1 dependency rules

This package is pure TypeScript. It has zero DOM dependencies, zero framework imports, and zero Node.js-specific APIs. It runs in browser, server, edge worker, and test runner environments.

When to use this module

• You need to create, traverse, or modify a LexTree directly.
• You want to use the grammar registry to manage multiple grammars.
• You need to build a custom grammar or parser integration.
• You need the incremental editing API (applyEdit, reparse) or streaming parse (createParseStream).
• You want the shared buffer protocol constants for value-level interop.

Module boundary

Boundary	Description
Inputs	Uint8Array (source bytes for parse), or preconstructed buffers for tree manipulation
Outputs	LexTree, LexNode, LexCursor, LexEdit results
Side effects	None
Determinism	Yes (same bytes and same protocol version produce same tree)
External dependencies	None
Never-throw guarantee	Yes for parse()
Security surface	None (no HTML output, no I/O)

Installation

1. Install the package.

   pnpm

   pnpm add @lanexio/parser-core

   npm

   npm install @lanexio/parser-core

   yarn

   yarn add @lanexio/parser-core

2. Import the named export.

   import { parse, LexTree, LexNode, LexCursor } from '@lanexio/parser-core';

Peer dependencies

This package has no peer dependencies. It is the foundation that all other packages depend on.

Basic Usage

Headless usage (pure TypeScript)

import { parse, LexNode, LexCursor, LexTree } from '@lanexio/parser-core';

const encoder = new TextEncoder();
const bytes = encoder.encode('toy input');

const tree: LexTree = parse(bytes);
console.log(tree.nodeCount);
console.log(tree.root);

Traverse a tree with cursor

const tree = parse(encoder.encode('example'));
const cursor = tree.cursor();

do {
  const node = cursor.current;
  console.log('kind:', node.kind, 'range:', node.range);
} while (cursor.gotoFirstChild() || cursor.gotoNextSibling() || cursor.gotoParent());

Exports

Export	Type	Description
parse	(source: Uint8Array) => LexTree	Parse toy-grammar bytes into a flat AST. Never throws.
LexTree	class	Root handle for a zero-copy flat AST backed by a single ArrayBuffer.
LexNode	class	Reference to a single 16-byte node within a LexTree.
LexCursor	class	Preorder DFS cursor over a LexTree, backed by a node index stack.
PROTOCOL_VERSION	number	Current shared buffer protocol version (value: 3).
LEX_TREE_MAGIC	number	Magic number identifying a valid tree header.
LEX_TREE_VERSION	number	Current tree layout version.
createLexErrorTree	(source: Uint8Array) => LexTree	Create a tree with a single root LexError node.
applyEdit	(tree: LexTree, edit: LexEdit) => LexTree	Apply a structural edit to a tree.
reparse	(tree: LexTree, options?: ReparseOptions) => LexTree	Incrementally reparse after edits.
reparseWithStats	(tree: LexTree, options?: ReparseOptions) => { tree: LexTree; stats: ReparseStats }	Reparse and return performance statistics.
computeDirtyRegion	(tree: LexTree, edit: LexEdit) => DirtyRegion	Compute the byte range affected by an edit.
createParseStream	() => LexParseStream	Create a streaming push parser for chunked input.
grammarRegistry	object	Global registry for grammar registrations.
embedGuests	(tree: LexTree, options: EmbedGuestsOptions) => LexTree	Embed guest-language subtrees into host tree.
embedRegistry	object	Registry for guest-language embeddings.
graftSubtree	(target: LexTree, source: LexTree, targetIndex: number) => LexTree	Graft a source tree onto a target tree.
findParentIndex	(tree: LexTree, childIndex: number) => number	Find the parent node index of a child.
fixSubtreeSizes	(tree: LexTree) => LexTree	Recompute subtree sizes after manual edits.
rebaseRecords	(records: Uint32Array, delta: number) => Uint32Array	Offset all byte ranges in a node record array.
assertLossless	(tree: LexTree) => void	Assert that the tree preserves all source bytes.
emitSource	(tree: LexTree) => Uint8Array	Reconstruct the source bytes from the tree.
extractLeafRanges	(tree: LexTree) => LeafRange[]	Extract leaf-level source byte ranges.
countFalseHasError	(tree: LexTree) => number	Count nodes with false-positive error flags.
countInvertedRanges	(tree: LexTree) => number	Count nodes with start > end ranges.
assertWellFormed	(tree: LexTree) => void	Assert all structural invariants hold.
LexEditError	class	Error thrown on invalid edits.
LexEditErrorCode	const object	Error code constants for edit failures.
LosslessError	class	Error thrown when lossless integrity check fails.
NODE_STRIDE	number	Stride (in Uint32 slots) per node record.
SLOT_KIND	number	Slot index for node kind.
SLOT_FLAGS	number	Slot index for node flags.
SLOT_FIELD	number	Slot index for node field id.
SLOT_START	number	Slot index for byte range start.
SLOT_END	number	Slot index for byte range end.
SLOT_SIZE	number	Slot index for subtree size.
createTree	(records: Uint32Array, source: Uint8Array) => LexTree	Create a tree from raw node records.
createTreeFromUint32Array	(records: Uint32Array, source: Uint8Array) => LexTree	Create a tree from a Uint32Array buffer.
growNodeData	(nd: Uint32Array, needed: number) => Uint32Array	Grow the node data buffer.
writeTreeHeader	(records: Uint32Array, nodeCount: number, sourceLength: number) => Uint32Array	Write the tree header into a record buffer.
createPendingNode	() => PendingNode	Create a pending node for tree construction.
LANEXIO_PARSER_CORE_PACKAGE_NAME	string	Stable npm package name constant.
LexToyKind	const object	Kind constants for the built-in toy grammar.
LexToyField	const object	Field constants for the built-in toy grammar.

Options

The core module accepts options through LexTreeOptions when constructing trees programmatically.

Field	Type	Required	Default	Description
nodeCount	number	Yes	-	Number of nodes in the tree.
sourceLength	number	Yes	-	Length of the source buffer in bytes.

Return shape

The primary return value from parse() is LexTree:

Property	Type	Description
root	LexNode	Root node of the flat AST.
nodeCount	number	Total nodes in the tree.
source	Uint8Array	Original parsed bytes.
cursor()	() => LexCursor	Create a new preorder DFS cursor starting at root.

Exported types

Type	Purpose	Notes
LexTreeOptions	Options for programmatic tree construction	Used by createTree
LexTreeMetadata	Metadata stored in the tree header	Internal use
LexRange	{ start: number; end: number } byte offset range	Used across all grammar APIs
LexEdit	Structural edit descriptor	Passed to applyEdit
DirtyRegion	Byte range affected by an edit	Returned by computeDirtyRegion
ReparseOptions	Options for incremental reparse	Passed to reparse
ReparseStats	Performance statistics from reparse	Returned by reparseWithStats
ReuseOracle	Oracle for incremental node reuse	Used by grammar incremental modules
LexParseStream	Streaming push parser interface	Created by createParseStream
LexTreeValidationCode	Validation result code	Produced by integrity checks
LexTreeValidationError	Error from tree validation	Thrown on invalid tree structure
LeafRange	Leaf-level source range	Used by extractLeafRanges
LosslessDiagnostics	Diagnostic info from lossless check	Returned by lossless operations
PendingNode	Node pending insertion	Used during tree construction
LanexioParserPureGrammar	Pure-TS grammar descriptor interface	Used by grammar packages to register themselves
GrammarRegistration	Full grammar registration descriptor	Used with grammarRegistry
EmbedGuestsOptions	Options for guest embedding	Passed to embedGuests
EmbedRule	Guest embedding rule	Used in embed registry

Configuration and Extension

Grammar registry

The grammarRegistry manages grammar registrations globally. Register a grammar to make it available for auto-detection and unified parsing.

import { grammarRegistry } from '@lanexio/parser-core';
import { htmlRegistration } from '@lanexio/parser-grammar-html';

grammarRegistry.register(htmlRegistration);

Embedding guests

embedGuests inserts guest-language subtrees (for example, a CSS block inside an HTML style element) into a host tree. The embedRegistry stores the embedding rules.

Streaming parse

createParseStream creates a LexParseStream that accepts chunked input via push() and finalizes with end():

import { createParseStream } from '@lanexio/parser-core';

const stream = createParseStream();
stream.push(encoder.encode('chunk1 '));
stream.push(encoder.encode('chunk2'));
const tree = stream.end();

Accessibility

Accessibility requirements

• No direct accessibility surface. The core module produces flat AST data structures. Consuming code is responsible for rendering output with appropriate semantics.

Accessibility checklist

Concern	Status
Generated output semantics	Not applicable (core data structures only)
ARIA attributes in serialized output	Not applicable
Semantic element round-trip	Not applicable

Security

Security considerations

• No direct security surface. The core module provides data structures and parsing infrastructure. All parse functions carry the never-throw guarantee.
• The toy grammar (parse()) is not intended for untrusted input validation — it is a minimal reference implementation.

Threat	Mitigation	Status
Malformed input byte sequence	Panic-free guarantee: all inputs accepted, errors produce LexError AST nodes	Implemented

Companion packages

Package	Relationship	Layer	Notes
@lanexio/parser-grammar-html	Consumes	2	HTML grammar built on core primitives
@lanexio/parser-grammar-markdown	Consumes	2	Markdown grammar built on core primitives
@lanexio/parser-grammar-json	Consumes	2	JSON grammar built on core primitives
@lanexio/parser-query	Consumes	3	Query engine operates on core LexTree
@lanexio/parser	Re-exports	6	Entry point re-exporting core API

Changelog

Version	Date	Status	Notable changes
1.0.0	2026-05-29	Current	Initial stable release. Apache-2.0.

Migration notes

• None. This is the initial stable release.

Related Content

Flat AST How the 16-byte node layout works and how to traverse it efficiently.

Bimodal AST Flat and hierarchical AST modes and when to use each.

Stability Guarantees Never-throw, panic-free, and 3-token recovery.

Parsing HTML Parse HTML documents with the HTML grammar.