# memtrails for Rust: async agent memory on tokio

> How the Rust port of memtrails runs on tokio end to end and still writes the same memory folder, vectors and hashes as the TypeScript library.

By Liviu Maftuleac · published 2026-11-06 · Open source, Rust, Memory
Source: https://tesseract.dev/blog/memtrails-for-rust/

memtrails for Rust is a native, async port of the memtrails agent memory library: `cargo add memtrails`, MIT-licensed, built on tokio from top to bottom. It is not a binding. It is a second implementation that reproduces the TypeScript library's keys, prompts and files, so a memory folder written by either one opens and searches the same in the other.

The Python version, fully async on asyncio, has its own post: [memtrails for Python](https://tesseract.dev/blog/memtrails-for-python/).

The first Rust release was memtrails 0.4.1, published to crates.io on 5 November 2026 alongside the Python version. This post is about how the port was built: what runs on the async runtime and what is kept off it, how it loads the exact ONNX Runtime build the original uses, and how much of JavaScript it had to imitate to produce the same bytes. What the library does, and how its search works, is in the earlier post, [memtrails: open-source memory for AI agents](https://tesseract.dev/blog/memtrails-open-source-agent-memory/). This one does not repeat it.

## What is memtrails for Rust?

It is the same library, written again in Rust. The [Rust README](https://github.com/cmaftuleac/memtrails/blob/main/rust/README.md) states the contract plainly: the TypeScript library is the reference, and the port follows it with "the same algorithms and constants, the same prompts byte for byte, the same files on disk and the same Qdrant index."

What you get is the full feature set of the original: add records and conversations, search, fit results into a small room, open an item whole, explore the concept graph, and with a language model, consolidate, sleep and gather. Six importers stream records from mail, Slack exports, Git, files and JSON Lines, and three agent tools come with their JSON Schemas.

Installing it takes one line:

```bash
cargo add memtrails
```

Or in `Cargo.toml`, with tokio beside it:

```toml
[dependencies]
memtrails = "0.5"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
```

It needs Rust 1.88 or newer. The first `Memory::open` downloads three things into `~/.cache/memtrails` and checks each against a pinned SHA-256: the embedding model (EmbeddingGemma-300m at 4 bits, 218 MB), ONNX Runtime 1.23.2 (about 10 MB, Microsoft's own release), and the Qdrant binary (about 30 MB). Why the Rust crate fetches its own ONNX Runtime is the subject of a later section.

## How do you use it from async Rust?

The quick start from the README:

```rust
use memtrails::{At, Memory, MemoryOptions, MemoryRecord, SearchQuery};

#[tokio::main]
async fn main() -> memtrails::Result<()> {
    let memory = Memory::open(MemoryOptions::new("./memory")).await?;

    memory.add(vec![
        MemoryRecord {
            kind: Some("email".into()), author: Some("Ana".into()), title: Some("Proxy certificate expired".into()),
            at: Some(At::from("2026-04-02")),
            ..MemoryRecord::new("msg-1", "The TLS certificate expired on the 1st. Renewal stopped because the cron job was removed in March.")
        },
        MemoryRecord {
            kind: Some("commit".into()), author: Some("Ion".into()), title: Some("Restore certificate renewal cron".into()),
            ..MemoryRecord::new("c9f1e2", "Files: deploy/cron/renew.sh")
        },
    ]).await?;

    let hits = memory.search(&SearchQuery::new("why did the certificate stop renewing?")).await?;
    println!("{} {}", hits[0].id, hits[0].title);

    memory.close().await
}
```

Two properties matter for how you structure an application around it. Every method that does I/O is `async`. And `Memory` is `Clone + Send + Sync`, where every clone is the same memory, so you can hand one to each task, each request handler or each agent loop without wrapping it in an `Arc<Mutex<_>>` of your own.

The method names follow Rust conventions rather than the TypeScript ones: `add_conversation`, `search_text`, `open_item`, `index_words`. The behaviour behind them is the reference's.

## How is the Rust version built on tokio?

![The Rust port on tokio: async tasks do disk, Qdrant HTTP and process I/O; importers are Streams; the language model is an async trait; the embedding model runs in spawn_blocking on the pinned ONNX Runtime; all of it writes the same memory folder as TypeScript and Python](https://app.tesseract.dev/api/core/tesseract/blog/media/f1b00d30e0c038bf9a6886d6)

The rule the port follows is simple to state: every call that touches disk, network or a process is async, and blocking work never runs on the runtime's own threads.

The async side covers three kinds of I/O:

- **Disk.** The append-only logs (`records.jsonl`, `knowledge.jsonl`) and the lock file. Where a disk step is heavy, such as loading or compacting the logs, the async call hands it to a blocking thread.
- **Network.** Qdrant is driven over HTTP, and so are any OpenAI-compatible endpoint for the language model and any embeddings or rerank server you configure.
- **Processes.** By default memtrails runs a Qdrant process of its own, starts it, checks it, and stops it on `close()`, when it is dropped, or when your process exits.

The blocking side is longer than people usually expect. The README lists what is moved off the runtime: model inference, tokenising, word-by-word scoring over many items, hashing a download, unpacking an archive, loading and compacting the logs, and the word-vector cache. Each of these is CPU work or a synchronous library call that can take long enough to stall other tasks on the same worker thread, which in an agent server means other users' requests.

The practical effect is that you can put memtrails inside an existing tokio service (an HTTP API, an MCP server, a bot) and its searches and imports share the runtime with everything else, rather than needing a thread pool or a separate process of their own.

## Why are the importers Streams?

In TypeScript, `memory.import` takes any async iterable. The Rust equivalent of an async iterable is a `Stream`, and the port uses it directly.

The importers in `memtrails::importers` are `from_mbox`, `from_eml`, `from_slack_export`, `from_git`, `from_files`, `from_jsonl` and `batches`. Each is an `async fn` that returns a `RecordStream`, and `memory.import(stream, batch, on_progress)` accepts any `Stream<Item = Result<MemoryRecord>>`. For data that is already in memory, `import_iter(iter, batch)` takes a plain iterator.

Streams give three things an importer needs:

1. **Bounded memory.** An mbox archive or a Git history is read record by record and added in batches of the size you pass.
2. **Back-pressure.** The importer only produces the next record when `import` asks for it, so a slow index does not let a fast reader pile up records.
3. **Errors per record.** The item type is a `Result`, so a source can report one bad record and keep going. The README's known-differences section shows where this differs from the reference: a JSON Lines field of the wrong type is skipped with a reason, where the TypeScript importer would keep it or fail.

Because `import` takes any stream of that item type, a database cursor or a paginated API client works the same way as the built-in importers. Records that have not changed since the last import cost nothing, as in the original.

## How does the language model plug in?

Adding and searching never call a language model. Consolidation, sleep and `gather` do, and in Rust that model is the async trait `memtrails::Ask`: one structured request in (a system text, a prompt and a JSON Schema the answer must follow), one answer out.

The crate ships one implementation, `openai_ask`, which speaks the OpenAI-compatible Responses API with strict JSON Schema output. Any provider or local server that serves that API works by changing its base URL. The README's example:

```rust
use memtrails::{openai_ask, Llm, MemoryOptions, OpenAiAskOptions};

let options = MemoryOptions {
    llm: Llm::Custom(openai_ask(OpenAiAskOptions { model: Some("gpt-6-luna".into()), ..Default::default() })),
    ..MemoryOptions::new("./memory")
};
```

`llm` has three settings: `Llm::Env` (the default, read from `MEMTRAILS_MODEL`, `MEMTRAILS_LLM_API_KEY` or `OPENAI_API_KEY`, and `MEMTRAILS_LLM_BASE_URL`), `Llm::OpenAi(options)`, or `Llm::Custom(Arc<dyn Ask>)` for anything you implement yourself. `OPENAI_API_KEY` is only ever sent to OpenAI's own address.

### How is an ask cancelled?

This is one of the places where Rust's model is simpler than the original's. The TypeScript library hands each ask an `AbortSignal` so it can be stopped when nobody will read the answer. In Rust, a future that is dropped stops running, so the port stops an ask by dropping its future. Every `Ask` implementation supports that without writing any cancellation code, and `LlmRequest` has no signal field.

The behaviour that depends on it is the same as the reference's: asks made side by side (consolidation's windows, gathering's parts) are dropped when one fails, and a consolidation that fails or is cancelled stops the readings in flight. `concurrency` (default 4) sets how many windows the model reads at once during consolidation.

## Why does the embedding model run in spawn_blocking?

The embedding model is EmbeddingGemma-300m, quantised to 4 bits, run on the CPU through ONNX Runtime. One inference is synchronous native code that holds a thread for as long as it takes. Run that inside an async task and the worker thread is stuck; with a few concurrent searches, the whole runtime stalls.

So the port does what [tokio's documentation](https://docs.rs/tokio/latest/tokio/task/fn.spawn_blocking.html) recommends for this kind of work. The model is loaded inside `tokio::task::spawn_blocking`, each inference is sent to a blocking thread the same way, and even dropping the model at close happens there. ONNX Runtime runs one inference at a time behind a lock, so the async side waits on a future while the blocking pool does the work. The `threads` option on `embedding` (default 2) sets how many threads ONNX Runtime itself uses.

One run of the model gives two outputs, both of which the library keeps: the averaged sentence vector that goes into the index, and the per-token vectors from the step before averaging, which drive the word-by-word reordering. The port reads both from the same call, as the original does.

If you would rather not run the model locally at all, `embedding.server` points at any OpenAI-compatible embeddings API instead.

## How do the vectors match the TypeScript library?

This was the hardest part to get exactly right, and the reason the Rust crate downloads a runtime of its own.

ONNX Runtime builds differ in their 4-bit matrix kernels. The module that loads the runtime says it directly: with another build, the vectors drift by up to about 0.007 a number (cosine about 0.999) from the reference's. That is close enough for search to work, and not close enough for "a memory folder written by either library is searched alike by the other," because the index stores those vectors.

The fix is to run the same build. The TypeScript library uses Microsoft's ONNX Runtime 1.23.2 through `onnxruntime-node`. The Rust port loads that same release: it fetches Microsoft's own [ONNX Runtime 1.23.2](https://github.com/microsoft/onnxruntime/releases/tag/v1.23.2) from its GitHub release on first use, checks it against a pinned size and SHA-256 like the model and Qdrant, and loads it at run time through the `ort` crate's dynamic loading. No other build is substituted unless you name one yourself.

The measured result, from the conformance suite:

| Processor family | Rust vectors against the reference's | Source |
|---|---|---|
| Apple silicon (macOS, arm64) | equal to 1e-6 | rust/README.md, conformance |
| x64 | differ by up to about 0.003 a number, cosine above 0.999 | rust/README.md |

The x64 gap is not a Rust problem: ONNX Runtime's own CPU kernels round differently on x64, so the fixture, made on arm64, is not reproduced to six places there. The interop test runs every language on the same machine for the same reason. The embedding conformance test is marked `--ignored` because it needs the real model.

If you already ship an ONNX Runtime, `MEMTRAILS_ONNXRUNTIME` names your own library and nothing is downloaded. You then own the question of whether its kernels round like 1.23.2's.

## What did it take to behave like JavaScript?

Matching vectors is not enough. The index also holds keyword term ids, record ids, point ids and fingerprints, and the logs hold JSON that the other languages read back. Every one of those is computed from text, and the reference computes them the way JavaScript handles text. The port has a module, `js.rs`, whose stated job is to do "what JavaScript does that the reference relies on, done the same way here." Every module that measures, cuts, hashes or dates text goes through it.

| What the reference relies on | What the Rust port does |
|---|---|
| String length (`s.length`) | Counted in UTF-16 code units, not bytes or `char`s |
| `s.slice(a, b)`, `cut(s, n)`, the 60-unit token cap, the 2-unit minimum word | Positions in UTF-16 units, mapped back to byte offsets |
| Keyword term id | 32-bit FNV-1a over UTF-16 code units (not bytes, not code points) |
| Word splitting for Chinese, Japanese, Thai, Lao, Khmer, Myanmar | ICU dictionary word segmentation, via ICU4X `icu_segmenter` |
| `Date.parse` | ISO 8601 first, then the legacy forms V8 accepts (RFC 2822, `asctime`, `April 2026`, `2024/04/02`); 54 forms are in the fixtures |
| `JSON.stringify` numbers | Integers without a fraction, exponents from 1e21 and below 1e-6 |
| Object key order | Integer-like keys first, ascending, then the rest as they came |
| `\s` and `trim` | JavaScript's whitespace set, which differs from Rust's `char::is_whitespace` on U+FEFF and U+0085 |

Each row is a way two correct-looking implementations would produce different bytes. Take the term id. [FNV-1a](https://www.isthe.com/chongo/tech/comp/fnv/) is a tiny hash, and hashing a Rust `&str` over its UTF-8 bytes is the natural thing to write. It would give a different id for every word outside ASCII, and keyword search over a folder written by TypeScript would silently miss. The [conformance rules](https://github.com/cmaftuleac/memtrails/blob/main/conformance/README.md) spell out "not bytes, not code points" for exactly that reason.

Dates have the same shape of problem. JavaScript reads an ISO date alone as UTC midnight and an ISO time without an offset as local time, and falls back to a long list of informal forms that mail programs write. The port reimplements that reader, including RFC 2822's US time-zone names, so a record dated by one language sorts and filters the same in another. Stored records always carry `Z` or an offset, so the local-time rule only applies to what you pass in.

## How is "the same" checked on every change?

Two mechanisms in the repository keep the claim honest, and both run in CI.

**Conformance fixtures.** The TypeScript library is run on hundreds of fixed, invented inputs (word splitting in many scripts, keyword vectors, ids and hashes, dates, the secret filter, every prompt), and its answers are written to `conformance/fixtures/*.json`. The Rust test suite reads the same fixtures and must reproduce them, floating-point values to 1e-6. Prompts are compared byte for byte along with their JSON Schemas. A separate CI job regenerates the Rust-only fixtures from the reference and fails if anything differs from what is committed, so a change in the reference is a change the port must follow.

**Interop.** On every change, a CI job has TypeScript, Python and Rust each write a memory folder from the same corpus, then has every language read every folder, and every language add a record to every folder. It makes 14 comparisons in all, and all 14 must agree: ids, order and text exactly, scores within 0.01. That tolerance exists because of the x64 rounding above; the interop script notes a score moving by up to 0.007 on GitHub's Linux runners, and agreement to 1e-6 on Apple silicon. Two hits whose scores fall within the tolerance may trade places; anything further apart must be in the same order.

Around those, the Rust crate had 190 tests at release, and CI runs `cargo clippy --all-targets -- -D warnings`, `cargo fmt --check`, and the test suite on Ubuntu and macOS against both stable Rust and 1.88, the minimum supported version. A further job runs the integration tests and the examples against the real model and Qdrant.

## Where does the Rust version differ?

The README has a "Known differences" section, and it is specific. The ones a user could notice:

- **Japanese and folded Thai words.** The port uses ICU4X for word segmentation; Node.js carries ICU4C. Their dictionaries split a few words differently. In Japanese, `がいっぱい` becomes `がい` + `っぱい`. In Thai, normalisation (NFKD) splits the vowel sara am and the result is cut differently. This affects keyword tokens only, not the embedding, and the conformance tests list the cases.
- **Half a character.** A Rust string cannot hold half of a surrogate pair. Where the reference would cut between the two halves of an emoji, the port cuts before it. In the secret filter, this only matters for a secret value longer than 512 UTF-16 units that contains emoji.
- **Text comparison.** The reference breaks ties between profile names with ICU collation (`localeCompare`). The port orders by letters, then accents, then lower case before upper case. Names that differ only in punctuation may order differently.
- **Importers.** A JSON Lines field of the wrong type is skipped with a reason; a Slack export without `channels.json` is read in folder-name order; a Slack message whose `ts` is a number is accepted.
- **Partial log lines.** A log line with a uid but missing a field this port needs (written by another version) is not read, where the reference would read it with the field missing. It still counts as that uid's latest line.

On platforms: 0.4.1 shipped for macOS and Linux (x64 and arm64), with Windows described as having the pieces it needs but untested. Windows x64 was added to CI in 0.5.4, the same day, and the changelog lists what testing found in the Rust port: it took every process for alive, never stopped its Qdrant at exit, and unpacked the Windows ONNX Runtime as gzip. All three are fixed. One limit remains: keep a memory's path under 90 characters on Windows, because Qdrant's files sit up to about 170 characters deeper and Windows stops a path at 260. memtrails says so when a path is too long.

## What do the benchmark numbers mean for the Rust version?

The benchmarks in the repository were measured with the TypeScript library. The Rust port was not benchmarked separately. It gives the same results by construction: the same prompts, keys and files, held to the reference by the conformance fixtures, and the same search results checked by interop.

So the numbers below are the TypeScript library's, and they apply to the Rust version to the extent the two agree, which on Apple silicon is to 1e-6 on vectors and on x64 within the tolerances above. From the [benchmark write-up](https://github.com/cmaftuleac/memtrails/blob/main/docs/BENCHMARKS.md), on 1,307 held-out LoCoMo questions with 6,000 characters of context and a strict judge:

| Setting | memtrails | Mem0 | Reading |
|---|---|---|---|
| `gather` with notes, LoCoMo, 1,307 questions | 80.9 | 76.5 | ahead beyond chance (+113/−55) |
| `search` alone, no model at search time, LoCoMo, 1,307 questions | 77.4 | 76.5 | a tie (+97/−85) |
| `gather` with notes, LongMemEval-S, 60-question sample | 90.0 | 88.3 | a tie (+4/−3) |

The win needs a language model reading what search found. Search on its own, with only the local model and the index, ties Mem0. On the small LongMemEval-S sample the difference is within noise. The method, the judges and the per-question comparisons are in the write-up; the earlier post covers them in more detail.

## Is it safe to run inside a service?

The port had an independent security and bug review before release. What it checks, from the README's safety section:

- Every download goes to a private temporary folder of its own and is checked on disk against its pinned size and SHA-256. One process fetches a given download at a time, under a lock file, and a stalled download is given up.
- The Qdrant process listens only on 127.0.0.1, behind a random key made for each run, and must prove it is memtrails' own process before it is given the key. It gets a minimal environment and is never reached through a configured proxy. One left behind by a crash is stopped by the next start.
- The memory folder is private (0700), and only one process at a time can open it (`memory.lock`).
- Secrets are scrubbed from everything sent to a language model and from everything consolidation keeps. The patterns are the reference's, written so that none takes more than linear time on hostile input.

## What does this make possible for a Rust developer?

Three things that were awkward before:

- **Memory inside a Rust agent or service, without Node.** No sidecar process, no JavaScript runtime in your container. The library runs on the tokio runtime you already have.
- **One memory folder, shared across languages.** A folder filled by a Python ingestion script or a TypeScript tool can be searched from a Rust service, and the other way round, with the same results.
- **A local memory for MCP apps from one binary.** Since 0.5.1 the crate also ships a `memtrails` command, an MCP server for Claude, Cursor and other clients, using the same files as the TypeScript and Python servers:

```bash
cargo install memtrails
claude mcp add memory -- memtrails mcp                             # Claude Code
memtrails add ~/notes ~/mail/archive.mbox ./slack-export          # notes and documents, mail, a Slack export
memtrails add --git ~/code/my-repo                                 # commit messages and changed files
memtrails search "why did the certificate stop renewing?"
```

The memory lives in `~/.memtrails` unless `--home` or `MEMTRAILS_HOME` says otherwise. The server opens it at the first tool call and closes it after five minutes without one.

memtrails comes out of the same research as the long-term memory in [Tesseract](https://tesseract.dev/); that design is described in [how AI agents remember across sessions](https://tesseract.dev/blog/how-ai-agents-remember-across-sessions/).

## Questions

**Is memtrails for Rust a wrapper around the TypeScript library?**
No. It is a native Rust implementation with no Node.js involved. It follows the TypeScript library as its reference, and conformance fixtures and interop tests check that it gives the same answers.

**Does it need a language model?**
No. Adding, searching, opening and exploring work with the local embedding model and index alone. A model, through the `Ask` trait, adds consolidation, sleep and `gather`.

**Can I bring my own model client instead of the OpenAI-compatible one?**
Yes. Implement the async trait `memtrails::Ask` and pass it as `Llm::Custom`. Cancellation comes from dropping the future, so your implementation needs no abort handling of its own.

**Will a memory written on a Linux x64 server search the same on a Mac?**
The files, ids, keyword vectors and prompts are the same everywhere. The embedding vectors differ by up to about 0.003 a number between x64 and Apple silicon, because ONNX Runtime's CPU kernels round differently, so scores can move slightly and near-ties can change places. The repository's interop test compares languages on one machine; it does not compare processor types.

**Does it work on Windows?**
Yes, on x64 since 0.5.4, tested in CI on every change. Keep the memory's path under 90 characters.

**Can I use an ONNX Runtime I already ship?**
Yes. Set `MEMTRAILS_ONNXRUNTIME` to your library and nothing is downloaded. Vectors will only match the other languages to 1e-6 if it is a build that rounds like Microsoft's 1.23.2.

## Get memtrails for Rust

Run `cargo add memtrails`, and read the code, the conformance rules and the known differences in the [memtrails repository on GitHub](https://github.com/cmaftuleac/memtrails), or the crate page on [crates.io](https://crates.io/crates/memtrails).