Simon Willison has released version 1.0 of condense-json, a Python library that condenses JSON by replacing repeated substrings with a compact reference syntax. The library is about a year and a half old at the time of release. The new version includes "sensible and non-disruptive fixes," according to the author.
How the Library Works
condense-json scans for strings or substrings present in a replacements object and swaps them for a special syntax in the output. The core function is condense_json(input_json, replacements), which performs the condensation. A second function, uncondense_json(condensed, replacements), reverses the effect and restores the original JSON.
The special syntax uses a {"$r": ...} structure. This lets the library embed references inside strings and arrays without breaking the JSON format. The result is smaller data that still parses cleanly.
A Concrete Example
The README shows a simple case. Input JSON contains a string with " with foxes in it " repeated. A replacements object maps '1' to " with foxes in it ". After running condense_json, the substring is replaced with a reference. The condensed output looks like this: {"$r": [" This is a string ", {"$": " 1 "}]}. For nested arrays, the output becomes {"$r": [" another ", {"$": " 1 "}, " too "]}.
Willison's quote from the documentation walks through the process: "] } } } } Combine that with a replacements object: {" He then shows the function call: "} And condense_json(input_json, replacements) produces the following: {" Finally, he explains the mechanism: "]}] } } } } It scans for strings or substrings that are present in that replacements object and replaces those with a special {"
Why Condense JSON at All
The purpose is to make it easier to store JSON that includes duplicated data from other related structures. Large logs and caches often repeat the same long strings many times. Replacing those with short references cuts storage costs significantly.
Stay ahead of the AI curve
The most important updates, news, and content — delivered weekly.
No spam. Unsubscribe anytime.
Willison built the library for a specific use case. It saves space in SQLite logs generated by LLM, his command-line tool for working with large language models. The latest iteration of that usage appears in PR #1586. That pull request shows how condense-json integrates with LLM's logging pipeline.
Reaching 1.0 After a Year and a Half
The library has been in development for roughly 18 months. Willison notes that he is "trying to get braver at releasing 1.0 versions." The 1.0 release marks a stable point for the project, with fixes that do not break existing behavior.
The article is tagged with json, projects, python, and llm. It is a personal blog post by Willison, who also created the LLM tool. The release signals that the library is ready for broader use, especially for developers who handle large JSON payloads with repetitive content.
Practical Implications for Developers
For anyone storing JSON in databases or log files, condense-json offers a straightforward way to reduce size. The two-function API keeps the workflow simple: condense before writing, uncondense after reading. The replacements object must be shared between both steps, so it works best when the same repeated strings appear across many records.
The library is written in Python, which fits naturally into data pipelines and backend services. Version 1.0 gives it a stable foundation for future improvements. Willison's willingness to ship a 1.0 version reflects a broader shift in how he approaches project releases, prioritizing momentum over perfection.