How Lens works
Every Lens workflow revolves around one selection: a point or region on a rendered target, with an optional note. Lens links the target to its producing cells, builds bounded notebook context, and captures an annotated image. A code-mode agent uses that evidence to inspect, change, and verify the notebook, then shows its work back in the notebook. This page follows the selection through that lifecycle.
The collaboration loop
- Mark the resultThe person selects evidence and describes what should change.
- Ground the requestLens connects the mark to its producing cell and graph context.
- Revise and verifyThe agent changes notebook code and checks the affected result.
- Review the resultLens brings the result into view and confirms the resolved request with a receipt.
Try the loop
Press Select, mark one bar, and add a note such as "Make this blue." The panel reads the same Lens state available to a code-mode agent. Click the API cards in order to send activity and review feedback back to the notebook.
Each card calls the method printed on it. A real agent keeps the activity handle while code mode inspects, edits, runs, and verifies the producing cells.
Targets and selections
A target is the selectable unit. By default every rendered notebook output is a target, identified by its cell ID. A cell that shows its code but renders no output is a target through that code, so a person can select the cell itself and ask for a change "in this cell." Authors can also declare regions of their own HTML as targets, with or without notebook inputs. Custom targets shows how.
A point or region narrows attention inside the target. Lens stores the geometry normalized to the target, so the selection survives resizing and reconnects when the same output renders again in the same browser document.
Each selection has a stable S<n> label, an optional note, producing-cell references, and image status. Selections explains how labels are numbered. Open selections live in Open. The current selection is the one most recently created or activated. It is the likely referent when a person says "this" or "here." One request to an agent can refer to several selections.
What the agent receives
Lens.context() returns a detached LensContext. It describes one selection-state revision and does not change when the notebook or Lens state changes afterwards. Mutating calls take that revision as expected_revision, so an agent cannot act on stale attention by accident.
| Evidence | Question it answers | Where |
|---|---|---|
| Brief | What is the current request, and what should I read next? | brief |
| Selection reference | What did the person select and ask for? | references |
| Chat request | What did the person ask in chat that the agent recorded? | references |
| Request history | Which recorded requests changed these cells, and do they hold? | history() |
| Target description | What name and rendering reference did the author supply? | references |
| DOM hint | Which rendered element sat under the point or region? | references |
| Graph context | Which cells and control values produced the target? | text |
| Selection image | What did the target look like when the selection was captured? | images |
| Cell-output image | What does the producing cell's output look like now? | cell_image() |
Read brief first. It names the current request and the Lens calls that fetch more.
references is JSON-safe and builds immediately. text renders on first read and is cached. It lists producing cells and their nearest upstream dependencies in dependency order, keeps at most 64 cells inside a shared 24,000-character source budget, and reports what it omitted. Safely displayable native marimo control values appear inline. Passwords, file payloads, custom controls, and opaque state appear as [redacted] or [unavailable].
The graph decides computational relevance. The image preserves capture-time visual focus. The DOM hint distinguishes nearby labels, rows, and containers. The agent still interprets what a chart mark or application object means. LensContext defines every field and bound.
Selection images
Lens starts image capture after it stores a selection. The image is an annotated PNG of the whole target with the S<n> marker drawn on it. Large or scrolled targets add a detail view at readable scale, and small targets include nearby context. The selection stays usable while capture is pending or after it fails.
| Status | Meaning | Bytes in images |
|---|---|---|
pending | Lens stored the selection and is preparing its image. | No |
available | The stored image matches the current point or region. | Yes |
outdated | The marker moved after the image was captured. | Yes, prior position |
failed | Lens could not produce an image for the current selection. | No |
Moving or resizing a marker starts replacement capture and keeps the previous image as outdated until the new one succeeds. Deleting or resolving a selection releases its bytes.
A cell-output image is different: a fresh, unannotated PNG of one current notebook output that an agent requests through cell_image() after changing and running a cell. It is a one-use transfer for verification. Connect an agent shows the polling loop.
Activity, reveal, and resolve
Lens gives an agent three ways to show work in the notebook. Each serves a different stage and has a different lifetime.
| Stage | Operation | Visible result | State change |
|---|---|---|---|
| Work | start_activity() | Marks the selected target or cell the agent is working on. | None |
| Review | reveal() | Brings the target or cell into view with a label and message. | None |
| Resolution | resolve() | Shows an Addressed receipt. | Requests close. |
Activity is transient. start_activity() returns an ActivityHandle, and stop_activity(handle) clears it only while that handle still owns the presentation, so a delayed stop cannot erase newer work. Activity also ends when its duration expires, a later activity or reveal replaces it, or the Lens view tears down.
Reveal brings a result into view after verification. A single-step reveal holds for duration_ms or until dismissed. A sequence of 1–16 steps is a Trail: an ordered explanation attached to notebook cells that the person pages through at their own pace. Trails work without any selection, which makes them the tool for "walk me through this notebook." Reveals are transient. A reveal ends when a referenced cell or its upstream inputs rerun, or the cell is deleted.
Resolve is the revision-checked state change. It validates the whole batch of selection and chat request IDs, closes the requests, and returns the revision, which advances when the batch includes a selection. The receipt shows the agent's summary of what changed and how the agent verified it. A Lens that keeps a record first hands one line per request, with each selection image, to its record, which stores the summary beside the original request. When a reveal precedes resolution, Lens shows the receipt after the reveal hold ends, so the person sees the result first and the acknowledgement second.
History and reopen
A Lens built with Lens(record=True) keeps a request record and shows it in History: each resolved request with its note, the agent's summary, and whether each change it made still holds. Press Reopen to restore a resolved selection as current in Open with a fresh image. The reopened selection carries previousResolution so the agent can see the earlier outcome. Resolve or reopen a selection covers the History entries and reopen rules.
The Addressed receipt tells the person that the agent returned the request for review. It does not claim the answer is right.
Browser and Python
Lens uses anywidget to connect a browser view to a Python model in the kernel. Each side owns what it can see.
| Browser | Python |
|---|---|
| Finds rendered outputs and handles pointer and keyboard input | Stores Open selections and writes the record it keeps |
| Positions markers, the dock, and agent feedback | Reads the live marimo graph and builds LensContext |
| Captures selection and cell-output images | Validates revisions, activity, reveal, and resolution |
Compact selection records and explicit commands cross the widget connection. PNG bytes travel separately and never enter trait state, JSON references, or the standalone text.
Continue with Selections for the person's controls, or Connect an agent for the agent's side of the loop.