nuget
A lenient (fault-tolerant) JSON parser designed to automatically repair and parse JSON output produced by Large Language Models (LLMs).
LenientJsonParser
00 Remarks
This parser implements a "Lenient Recursive Descent Parser" strategy inspired by the json-repair library. Instead of parsing first and then repairing, it tolerates errors during the parsing process itself by scanning the input character-by-character with a cursor.
The following 16 repair strategies are implemented:
- Skip comments (// line and / block / comments)
- Ignore extra content after the first JSON value
- Accept single-quoted strings ('...')
- Convert Infinity to null
- Skip unrecognized characters
- Auto-close unclosed objects (truncation repair)
- Skip leading/trailing commas
- Tolerate missing colons in objects
- Accept unquoted keys (JavaScript-style)
- Accept unescaped control characters in strings
- Auto-close unclosed strings (truncation repair)
- Accept leading + sign in numbers
- Convert NaN to null
- Partial keyword matching (tru -> true, etc.)
- Smart quote closure detection — when a quote is encountered inside a string, lookahead to check if the next non-whitespace character is a structural character (, } ] : or EOF) before treating it as the closing quote; otherwise keep it as an internal unescaped quote
- Missing closing quote recovery — repair a value string whose closing quote was dropped, causing it to swallow the
", "separator and the following key name
Strategy 16 in detail. A very common LLM failure is to omit the closing quote of a string value, for example emitting
{"module_name": "demo,goal": "the goal text"}instead of the correct
{"module_name": "demo", "goal": "the goal text"}Without special handling this is catastrophic rather than local: the value is read as demo,goal, the stray : is skipped, and the following text is consumed as the next key. Every subsequent key and value is shifted by one, so the whole document degenerates into garbage keys and no usable content survives.
The repair keys off a hard guarantee of the JSON grammar rather than a heuristic: a string in value position may only be followed by ,, }, ] or EOF — never by :. So a colon there is proof that the closing quote was dropped. The parser then splits the string at its last comma: the text before it is the true value, the text after it is the swallowed key. The last comma is used because real values often contain commas (prose, file paths, CJK text) whereas the swallowed key is an identifier that virtually never does. Because valid JSON can never trigger the condition, well-formed documents are unaffected. If the string contains no comma the parser degrades to the previous behaviour, so the strategy can only ever improve the result.
01 Syntax
Microsoft.VisualBasic.MIME.application.json.LenientJson.LenientJsonParser
02 Methods
| Name | Overloads | Summary |
|---|---|---|
| .ctor | 1 | Create a new lenient JSON parser for the given text. |
| Parse | 1 | Parse the JSON text and return the root JsonElement. |
| ParseJSON | 1 | Convenience shared method: parse a JSON string leniently. |
| Open | 1 | Open and parse a JSON file leniently. |
| OpenStream | 1 | Parse JSON text from a stream leniently. |
| at_end | 1 | Returns True if the cursor has reached the end of input. |
| peek | 2 | Returns the character at the current cursor position without advancing. |
| skip_whitespace_and_comments | 1 | Skip whitespace characters and comments (// line comments and /* block comments */). |
| skip_line_comment | 1 | Skip to the end of the current line (for // and # comments). |
| skip_block_comment | 1 | Skip a block comment /* ... |
| parse_value | 1 | Parse a JSON value (object, array, string, number, boolean, or null). |
| parse_object | 1 | Parse a JSON object { "key": value, ... |
| clean_recovered_key | 1 | Normalise a key name recovered by the Strategy 16 split so that it is safe to use as an object property name. |
| parse_array | 1 | Parse a JSON array [ value, value, ... |
| parse_string | 1 | Parse a JSON string. The opening quote (either " or ') should be at the current cursor position. Implements: - Strategy 3: Accept both single-quoted and double-quoted strings -… |
| parse_unicode_escape | 1 | Parse a \uXXXX unicode escape sequence and append the decoded character to the StringBuilder. |
| parse_number | 1 | Parse a JSON number (integer or floating-point). |
| parse_keyword | 1 | Parse a JSON keyword (true, false, null) or special value (NaN, Infinity). |
| parse_key | 1 | Parse an object key. The key may be: - A double-quoted string ("key") - A single-quoted string ('key') — Strategy 3 - An unquoted identifier (key) — Strategy 9 |
| check_infinity | 1 | Check if the current position (after an optional - or + sign) spells "Infinity". |
| is_hex_digit | 1 | Check if a character is a hexadecimal digit (0-9, a-f, A-F). |
| is_likely_closing_quote | 1 | Determine whether the quote character just consumed at m_index - 1 is likely the real closing quote of a string. |
03 Fields
| Name | Overloads | Summary |
|---|---|---|
| m_input | 1 | The input JSON text (possibly malformed) to be parsed. |
| m_index | 1 | Current cursor position in the input string. |
| m_length | 1 | Total length of the input string. |
| m_last_string_broken_at_colon | 1 | Set by LenientJsonParser.parse_string()) when a string parsed in the StringContext.Value position was terminated by a quote whose next non-whitespace character is a colon. |
04 Members
#ctor(
String)Create a new lenient JSON parser for the given text.
Parameters
| Name | Type | Description |
|---|---|---|
jsonText | String | The JSON text (possibly malformed) to parse. If Nothing or empty, the parser will return Nothing from LenientJsonParser.Parse(). |
Parse
Parse the JSON text and return the root JsonElement.
Returns
A JsonElement representing the parsed JSON value, or Nothing if the input is empty or contains no recognizable JSON value.
ParseJSON(
String)Convenience shared method: parse a JSON string leniently.
Parameters
| Name | Type | Description |
|---|---|---|
json | String | The JSON text to parse |
Returns
The parsed JsonElement, or Nothing if input is empty.
Open(
String)Open and parse a JSON file leniently.
Parameters
| Name | Type | Description |
|---|---|---|
file | String | Path to the JSON file |
Returns
The parsed JsonElement, or Nothing if file is empty.
OpenStream(
Stream)Parse JSON text from a stream leniently.
Parameters
| Name | Type | Description |
|---|---|---|
stream | Stream | The stream containing JSON text |
Returns
The parsed JsonElement, or Nothing if stream is empty.
at_end
Returns True if the cursor has reached the end of input.
peek
Returns the character at the current cursor position without advancing. Returns ChrW(0) if at end of input (caller should check at_end() first).
peek(
Int32)Returns the character at the given offset from the current cursor. Returns ChrW(0) if the offset is out of range.
skip_whitespace_and_comments
Skip whitespace characters and comments (// line comments and / block comments /). This implements Strategy 1.
skip_line_comment
Skip to the end of the current line (for // and # comments).
skip_block_comment
Skip a block comment / ... /. If the block comment is unclosed (truncated), skip to the end of input. This implements the "unclosed block comment" part of Strategy 1.
parse_value(
Boolean)Parse a JSON value (object, array, string, number, boolean, or null). This is the main dispatcher that examines the current character and delegates to the appropriate sub-parser.
Parameters
| Name | Type | Description |
|---|---|---|
stop_at_structural | Boolean | If True, stop and return null when encountering a structural character (comma, closing brace, closing bracket). This is used inside objects and arrays to handle missing values gracefully. |
Returns
The parsed JsonElement, or Nothing if no value found.
parse_object
Parse a JSON object { "key": value, ... }. Implements:
- Strategy 6: Auto-close unclosed objects at EOF (truncation repair)
- Strategy 7: Skip trailing commas
- Strategy 8: Tolerate missing colons
- Strategy 9: Accept unquoted keys
- Strategy 16: Missing closing quote recovery — split a value string that swallowed the separator and the following key back into its two parts
Remarks
The Strategy 16 recovery runs as a loop because the malformed pattern usually repeats across consecutive pairs of the same object: repairing one pair exposes the next. Each iteration writes the corrected value for the current key, adopts the recovered key, and parses its value, which may in turn raise the same signal.
clean_recovered_key(
String)Normalise a key name recovered by the Strategy 16 split so that it is safe to use as an object property name.
Remarks
The recovered fragment comes from inside a string literal, so it may carry surrounding whitespace and stray quote characters left over from the malformed input. Both are stripped here to avoid producing property names such as " goal.
Parameters
| Name | Type | Description |
|---|---|---|
raw | String | The raw text following the split comma. |
Returns
The cleaned key, or an empty string if nothing usable remains.
parse_array
Parse a JSON array [ value, value, ... ]. Implements:
- Strategy 6: Auto-close unclosed arrays at EOF (truncation repair)
- Strategy 7: Skip trailing commas
parse_string(StringContext)
Parse a JSON string. The opening quote (either " or ') should be at the current cursor position. Implements:
- Strategy 3: Accept both single-quoted and double-quoted strings
- Strategy 10: Accept unescaped control characters (raw newlines, etc.)
- Strategy 11: Auto-close unclosed strings at EOF (truncation repair)
- Strategy 15: Smart quote closure detection — when a quote is encountered inside the string, lookahead to check whether the next non-whitespace character is a structural character (, } ] : or EOF). If so, treat the quote as the real closing quote; otherwise keep it as an internal unescaped quote.
- Strategy 16: Missing closing quote detection — when parsing in the StringContext.Value position, a terminating quote followed by
:flags LenientJsonParser.m_last_string_broken_at_colon so that LenientJsonParser.parse_object() can split the swallowed key back out.
Parameters
| Name | Type | Description |
|---|---|---|
context | StringContext | The syntactic position of this string (key or value). This controls whether a |
Returns
The decoded string value.
parse_unicode_escape(
StringBuilder)Parse a \uXXXX unicode escape sequence and append the decoded character to the StringBuilder. Handles surrogate pairs (\uD800-\uDBFF followed by \uDC00-\uDFFF). The cursor should be positioned right after the "\u" prefix.
parse_number
Parse a JSON number (integer or floating-point). Implements:
- Strategy 12: Accept leading + sign
- Bonus: Accept hex numbers (0x...)
parse_keyword
Parse a JSON keyword (true, false, null) or special value (NaN, Infinity). Reads all consecutive letters and attempts to match. Implements:
- Strategy 4: Convert Infinity to null
- Strategy 13: Convert NaN to null
- Strategy 14: Partial keyword matching (tru -> true, etc.)
Returns
The parsed JsonElement, or Nothing if the text does not match any keyword (caller should skip and continue).
parse_key
Parse an object key. The key may be:
- A double-quoted string ("key")
- A single-quoted string ('key') — Strategy 3
- An unquoted identifier (key) — Strategy 9
Returns
The key string.
check_infinity
Check if the current position (after an optional - or + sign) spells "Infinity". If so, advance the cursor past it and return True. This implements Strategy 4: Infinity/-Infinity/+Infinity -> null.
is_hex_digit(
Char)Check if a character is a hexadecimal digit (0-9, a-f, A-F).
is_likely_closing_quote(StringContext,
Boolean)Determine whether the quote character just consumed at m_index - 1 is likely the real closing quote of a string.
Parameters
| Name | Type | Description |
|---|---|---|
context | StringContext | The syntactic position of the string being parsed. |
broken_at_colon | Boolean | Receives |
Returns
True if the quote is likely the real closing quote; False if it is likely an internal unescaped quote.
Example
"evidence": "Multiple papers: "HIF-1 induces GLUT1" and "PDK1" in cells."
The internal quotes around "HIF-1 induces GLUT1" and "PDK1" are followed by letters/spaces, so they are kept. Only the final quote (followed by } or ,) is treated as the closing quote.
m_input
The input JSON text (possibly malformed) to be parsed.
m_index
Current cursor position in the input string.
m_length
Total length of the input string.
m_last_string_broken_at_colon
Set by LenientJsonParser.parse_string() when a string parsed in the StringContext.Value position was terminated by a quote whose next non-whitespace character is a colon.
Remarks
In valid JSON a value string's closing quote can only be followed by ,, }, ] or EOF — never by :. Therefore a colon here is a deterministic signal that the closing quote of the value was omitted and the string has swallowed the ", " separator together with the following key name (Strategy 16).
This field carries that side-band information back to LenientJsonParser.parse_object(), because LenientJsonParser.parse_string() itself returns a plain String. It is reset at the start of every LenientJsonParser.parse_string() call, so it always reflects only the most recently parsed string.