Repository navigation
Conversation
loadGrammarsForLanguages loads grammars one at a time, because web-tree-sitter corrupts a grammar loaded while another one is loading (tree-sitter#2338). That held only within one call. Two calls started together, like a daemon's sync beside a request that loads a grammar for branch guards or syntax tokens, each ran Parser.init and loaded side by side. A grammar loaded that way stayed cached, and every later parse of its language threw "Incompatible language version" until the process exited. initGrammars now initializes the runtime once for callers that arrive together, and every load joins one queue that skips a grammar an earlier call has loaded. The new test starts six loads together: on main the runtime initializes six times, and parses fail. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 15, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Grammars that load at the same moment break each other on main, and stay broken for the life of the process.
The problem
loadGrammarsForLanguagesloads grammars one at a time, because web-tree-sitter corrupts a grammar loaded while another is loading (tree-sitter#2338). That only holds within one call. Two calls started together each runParser.initand load side by side. A grammar loaded that way stays cached, and every later parse of its language throws "Incompatible language version" until the process exits.Calls can overlap wherever grammars load lazily. A daemon's sync loads the grammars its files need, while a request may load another for branch guards (
graph/branch-guards.ts) or syntax tokens (extraction/syntax-tokens.ts).loadGrammarsForLanguagesis also exported.To reproduce, start six loads together in a fresh process (
loadGrammarsForLanguages(['lua']),(['rust']),(['python']),(['lua', 'rust']),(['go']),(['typescript', 'python'])), then parse a line of each language. On main, three runs in a row gave:Parser.initran 6 times.Incompatible language version 3538997. Compatibility range 13 through 15.on its first parse.What changes
initGrammarsinitializes the runtime once for callers that arrive together.What I ran
On Linux x64 (WSL2), Node 24.19.0, against main at
f72c9d2:__tests__/grammar-load-concurrency.test.ts(new). It starts the six loads together, and requires oneParser.init, one load per grammar, never two loads at once, and a clean parse of every language. On main it fails at the first assertion:Parser.initis called 6 times.CODEGRAPH_KERNEL_EXPECT=1: 11,070 passed, 2 expected failures, 58 skipped (platform-gated), 0 failed.Not run: macOS and Windows. The change has no platform-specific code.
This came out of the LuaJIT → Rust bridge work, which loads grammars during synthesis. It's split out so it can be reviewed and merged on its own.
🤖 Generated with Claude Code