Skip to content

About

Manage Multimodal Agentic Context Lifecycle with Lance

Resources

Stars

89 stars

Watchers

3 watching

Forks

Repository files navigation

lance-context

A storage engine for AI data — agent memory and RL training data — built on Lance.

Modern AI systems produce two kinds of data that are awkward to store well:

  1. Agent memory — the running history of a chat or agent: text, images, tool calls, and their embeddings, which you later search over to recall context.
  2. RL training data — the trajectories, rewards, and logprobs produced when you train models with reinforcement learning (GRPO, RLVR, PPO, ...).

lance-context stores both in one place. It gives you a durable, columnar, versioned table you can append to, search, filter, and time-travel through — without standing up a database server (though a server is available if you want one).

Why use it

  • Two use cases, one engine. A Context store for agent memory and a RolloutStore (a purpose-built RolloutDB) for RL rollouts. Same storage format, same versioning, same cloud backends.
  • Multimodal. Store text, images, and binary blobs next to their embeddings and typed metadata — the raw bytes are kept, not just a pointer.
  • Search built in. Run vector search, full-text search, or hybrid retrieval directly on the table. No separate vector database.
  • Versioned. Every write creates a new immutable snapshot. Roll back, branch, or reproduce an exact state with checkout(version).
  • Runs anywhere. Local files, or S3 / GCS / Azure via a simple storage_options dict. Embedded in your process, or behind an HTTP server.

Documentation

Full documentation: https://lance-format.github.io/lance-context/

For deduplicated conversation turns, see Trace records: TraceRecord supplies session_id, ordered turn_id, role and content in one dataset, with typed local/remote access and ID-only batch lookup through the existing GenericStore API.

Install

pip install lance-context

Quick start

1. Agent memory

Store what your agent sees and says, then search it back.

from lance_context import Context

ctx = Context.create("memory.lance")

# Add a message with an embedding so it's searchable.
ctx.add(
    "user",
    "Where should I travel in spring?",
    embedding=[0.1, 0.2, 0.3],  # from your own embedding model
    metadata={"tenant": "acme", "tags": ["travel"]},
)

# Semantic search returns the closest records.
hits = ctx.search([0.1, 0.2, 0.3], limit=5)
print(hits[0]["text"])

# Time-travel: go back to an earlier version of the store.
v = ctx.version()
ctx.add("assistant", "How about Japan?")
ctx.checkout(v)  # the second message is no longer visible

Context also supports images and other binary payloads, metadata filtering, hybrid (text + vector) retrieval, batch ingestion, and object-storage backends. See examples/ for runnable projects.

2. RL rollouts (a "RolloutDB")

Store training trajectories — one row per step (an assistant turn, a tool call, a grade, or an artifact). Only id and rollout_id are required; everything else (tokens, logprobs, rewards, advantages) is optional and filled in as your pipeline computes it. We call this shape a RolloutDB: a versioned, columnar store purpose-built for RL rollout data, the same way a vector DB is built for embeddings.

from lance_context import RolloutStore

store = RolloutStore.open("rollouts.lance")

# One step of a trajectory. `id` is auto-generated if you omit it.
store.add({
    "rollout_id": "traj-1",   # the trajectory this row belongs to
    "problem_id": "prompt-7", # groups the N samples of one prompt (for GRPO)
    "role": "assistant",
    "content": "The answer is 42.",
    "reward": 1.0,
    "policy_version": "ckpt-100",
})

for row in store.list():
    print(row["rollout_id"], row["reward"])

# Exact-match filters are combined with AND and applied before pagination.
training_rows = store.list(filters={
    "policy_version": "ckpt-100",
    "include_in_training": True,
    "role": "assistant",
})

In a real training run, generation workers and the learner talk to a shared store over HTTP. Use AsyncRolloutStore from async code so writes don't block your event loop:

from lance_context import AsyncRolloutStore

store = await AsyncRolloutStore.connect_or_create("http://localhost:8080", "rl-run-1")
await store.add({"rollout_id": "traj-1", "role": "assistant", "reward": 1.0})

Automatic JSON shredding

DocumentStore discovers useful scalar fields automatically, stores extracted values in typed columns, and removes them from the residual JSON. Reads restore the complete logical document; callers do not declare shredding paths.

from lance_context import DocumentStore

store = DocumentStore.create("documents.lance")
ids = store.add([{"body": "otters swim upstream", "source": "wiki"}])
hits = store.search("otters")
document = store.get(ids[0])

This initial embedded API uses direct Lance commits with one writer per dataset. The first text search after layout growth prepares historical values and indexes. See automatic shredding for policy limits, logical field queries, versioned reads, and current array/indexing boundaries.

Rust

The engine is written in Rust and usable directly:

use lance_context::ContextStore;

let mut store = ContextStore::open("memory.lance").await?;
store.add(&[record]).await?;
println!("version {}", store.version());

See crates/lance-context-core for the full ContextRecord shape and the RolloutRecord schema.

Project layout

crates/lance-context-core    # Rust engine: Context + Rollout stores (no Python deps)
crates/lance-context-api     # Shared request/response types (DTOs)
crates/lance-context-server  # HTTP server for remote access
crates/lance-context-client  # HTTP client for the server
crates/lance-context-master  # Control plane, scheduler, and admin UI
crates/lance-context         # Re-export crate for downstream clients
python/                      # Python bindings (PyO3) + tests
deploy/kubernetes/           # Kubernetes master examples: RocksDB/PVC or etcd HA
examples/                    # Runnable example projects

Storage backends

Point any store at cloud storage with storage_options:

ctx = Context.create(
    "s3://my-bucket/memory.lance",
    storage_options={
        "aws_access_key_id": "...",
        "aws_secret_access_key": "...",
        "aws_region": "us-east-1",
    },
)

GCS (gs://) and Azure (az://) work the same way. When storage_options is omitted, standard environment variables (AWS_ACCESS_KEY_ID, ...) are used.

Examples

The examples/ directory has self-contained, runnable projects:

See the examples index for the full list.

Development

make venv      # create python/.venv using uv
make install   # editable install with test extras
make test      # run the Python test suite
cargo test --manifest-path crates/lance-context-core/Cargo.toml

Linting and type checks: python/.venv/bin/ruff check python/, python/.venv/bin/pyright, and cargo fmt -- --check.

Contributing

  1. Fork and clone the repository.
  2. Create a feature branch off main.
  3. Make your change, add tests, and run the checks above.
  4. Open a Pull Request with a clear summary.

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.

About

Manage Multimodal Agentic Context Lifecycle with Lance

Resources

Stars

89 stars

Watchers

3 watching

Forks

Releases

Packages

Contributors

Languages