How to read deeply nested JSON without losing your mind

You open a multi-MB file on step 9 of a failing agent run. Your editor thinks about it for a second, then folds the whole thing into a wall of brackets. You start scrolling. Somewhere in there is the one field that explains the failure, and you are going to find it by hand.

There is a better loop. This post walks through four ways to actually read deep JSON, when each one holds up, and where each one falls apart. The through-line is a reframe: most of the time the job is to see the file's shape and ask it a question, rather than parse it into code again. Everything below is about closing the gap between "I have the file" and "I have the answer."

Four ways to read it, ranked by how much you'll suffer

Scroll and collapse

This is the default, and it is fine for a 40-line config. You expand a node, collapse a node, and your eyes do the query. Past a few thousand lines it stops working. Depth is the killer: the value you want sits eight levels down, repeated across two hundred sibling objects, and no amount of folding surfaces it. Reading by scrolling does not scale with depth, and deep files are exactly the ones that hurt.

Write a throwaway script

So you write code. The common move is a recursive walk over the tree that pulls the property you want, which is the accepted pattern in the Stack Overflow thread "Getting data from a deeply nested json object" (2013). It works. The cost is that you write it again for the next file with a different shape, and the feedback loop is slow: edit, run, print, squint, repeat. That is a lot of effort for a lookup you will do once.

Query it with jq

jq is the right tool when you want an exact answer and you know roughly where it lives. Pull the last step of every errored run:

jq '.runs[] | select(.status == "error") | .steps[-1]' trace.json

Count them:

jq '[.runs[] | select(.status == "error")] | length' trace.json

Precise and composable, and it drops straight into a shell pipeline. The tax is syntax. If you reach for jq by reflex you already pay it happily; if you touch it twice a month you re-learn select and array slicing every time.

Load it into a database

For repeated analytical questions over the same data, put it in a database. DuckDB reads JSON files directly as a table, inferring column names and types automatically through read_json, per the DuckDB blog "Shredding Deeply Nested JSON, One Vector at a Time" (Laurens Kuiper, 2023, duckdb.org). Under the hood it parses with yyjson, a fast JSON parser written in C, and it can read newline-delimited JSON in parallel via read_ndjson. That is real power for slicing a dataset many ways. It is also overkill when the question is just "what is in this one file right now."

Look at what all four share. Scrolling makes you read the shape but never ask a question. A script and jq let you ask a precise question but show you no shape until you already know where to point. A database gives you both, after you commit to loading it. The fast loop, seeing the shape and asking a question in the same place, is the thing none of them hands you directly.

Pitfalls that make deep JSON worse

Before you reach for the obvious shortcut, know where it bites.

Do not paste the whole file into an LLM. Modern context windows are large now, and a nested trace usually fits, so size is rarely the blocker it once was. The real problem is what happens after it fits. Hand raw values to a model and its counts and sums are guesses, so any aggregation you ask for is unreliable. You want the math computed exactly by deterministic jq running on the full file. Let jq do the arithmetic and the hallucination risk on a large or deep file goes away.

The second trap is quieter. Uploading a sensitive payload to a hosted formatter or a model means your actual values leave your machine. If the data is customer records or auth tokens, treat that upload as a real decision with consequences. Prefer tools that execute locally.

This is the loop QDev builds. Open a JSON, YAML, or TOON file and you get a virtualized tree, an inferred schema, and a navigable graph, plus a single ⌘K omnibox. The inferred schema (field names and types) is the move that saves you: deep files repeat the same shape, so reading structure first tells you where to point jq without scrolling to find out. The omnibox then fuzzy-searches keys and values, runs raw jq, or takes a plain-English question ("last step of every errored run") and writes editable jq for you. Execution is deterministic jq on the full file, run on-device in WebAssembly, so your values never leave the tab. The hosted AI on Pro only ever sees a value-masked schema and your question, never the data itself.

Pick the tool for the question

Match the method to what you are actually doing. Scroll for a tiny config. Reach for jq when you want an exact, scriptable answer and know the shape. Load it into DuckDB when you will query the same dataset many ways. And when the file is unfamiliar or deep, you want the answer now, and the values should not leave your machine, use a viewer that lets you see the shape and ask in words in one place. Stop scrolling JSON. Start asking it.