NeoCore, measured

NeoSift: a 32M model that picks the memories that help

For every prompt, recall finds about 48 candidate memories by words and by meaning. Most are related but not useful, and injecting them distracts the model. NeoSift reads each one with the prompt and asks one question: will this help? Only the best few get through.

The test: 103 real coding-assistant prompts and 8,397 candidate memories, locked away before training and labelled by hand-checked grades. AUC is the chance a useful memory is ranked above a non-useful one: 0.5 is random, 1.0 is perfect.

Same model size. Better ranking.

0.750.800.850.900.951.00Best open rerankerQwen3 4B · 0.835Ettin 32M · 0.796where we startedJev · 0.931hosted+0.118 from trainingNeoSift 32M · 0.914

We started from Ettin 32M, an open reranker, and trained it on GPT-6.1’s grades. Same 32 million parameters, and its score rose from 0.796 to 0.914: past every open reranker we tested.

Against the best open reranker

NeoSift 32MQwen3-Reranker-4B

Size

32M

4B · 125×

Ranking (AUC)

0.914

0.835

Time per prompt

0.099 s

8.50 s

Same batch, real time: ~81 candidates · RTX 3080

NeoSift

0.10 s ✓

Qwen3 4B

8.50 s ✓

1/125 of the parameters, about 86× faster in this test, and a higher ranking score. Open rerankers were run as published, without training for this task; NeoSift was trained for it.

Every scorer we tested AUC · 0.5 = random

Claude Opus 5.5hosted frontier model
0.976
GPT-6.1 · NeoSift’s teacherhosted frontier model
0.970
TypeSafe Jevhosted, ~$0.0007 per prompt
0.931
NeoSift 32M32M · local GPU, no API fees
0.914
Qwen3-Reranker-4B4B · local GPU
0.835
Qwen3-Reranker-0.6B600M · local GPU
0.821
Ettin 400M400M · local GPU
0.809
Ettin 150M150M · local GPU
0.801
Ettin 32M · NeoSift’s base32M · local GPU
0.796

+ 8 more open rerankers, all below 0.76: full table below.

Hosted scorers still lead. Jev scores 0.017 higher (the confidence intervals overlap), and the frontier judges about 0.06 higher. NeoSift runs on your own GPU, with no API fees.

When no memory helps lower is better

43 of the prompts had nothing useful in memory, so the right move is to inject nothing. How often did each scorer still let something through?

Claude Opus 5.5
7%
NeoSift 32M
40%
GPT-6.1
63%
TypeSafe Jev
74%

At its default cut, NeoSift still kept at least one memory on 40% of those prompts, against 74% for Jev. Opus is the strictest. These are results at the tested thresholds, not a full precision–recall comparison.

All 17 scorers, intervals and method

Scorer	Size	Runs	AUC	95% interval	Time per prompt
Claude Opus 5.5 (LLM judge)	not disclosed	hosted frontier model	0.976	0.963–0.986	–
GPT-6.1 (NeoSift’s teacher)	not disclosed	hosted frontier model	0.970	0.957–0.981	–
TypeSafe Jev 1.13	not disclosed	hosted, ~$0.0007 per prompt	0.931	0.909–0.949	–
NeoSift 32M round 10	32M	local GPU, no API fees	0.914	0.884–0.938	0.10 s
Qwen3-Reranker-4B	4B	local GPU	0.835	0.800–0.872	8.5 s
Qwen3-Reranker-0.6B	600M	local GPU	0.821	0.782–0.862	1.6 s
Ettin reranker 400M	400M	local GPU	0.809	0.764–0.853	0.90 s
Ettin reranker 150M	150M	local GPU	0.801	0.757–0.844	0.41 s
Ettin reranker 32M (NeoSift’s base)	32M	local GPU	0.796	0.752–0.841	0.10 s
gte-reranker-modernbert-base	149M	local GPU	0.753	0.706–0.798	0.42 s
bge-reranker-v2-m3	568M	local GPU	0.712	0.658–0.766	0.78 s
ms-marco-MiniLM-L6-v2	23M	local GPU	0.681	0.622–0.740	0.07 s
bge-reranker-large	560M	local GPU	0.635	–	0.72 s
bge-reranker-base	278M	local GPU	0.617	–	0.29 s
mxbai-rerank-large-v1	435M	local GPU	0.614	–	1.4 s
ms-marco-MiniLM-L12-v2	33M	local GPU	0.585	–	0.11 s
mxbai-rerank-base-v1	184M	local GPU	0.572	–	0.54 s

The 103 prompts are real Claude Code and Codex prompts from one developer, kept out of all training. Each candidate is labelled useful, related or not useful. Intervals are 95% bootstrap intervals.
Every scorer saw the same text: the prompt, the end of the previous turn and the memory. Open rerankers ran zero-shot, as published, in bf16 on an RTX 3080. The time is the median to score one prompt’s ~81 test candidates.
The frontier judges were given NeoSift’s grading question and graded every candidate. GPT-6.1 is also the teacher NeoSift learned from, which may favour both.
NeoSift is trained on coding prompts. On other kinds of work it agrees with its teacher less well, which is why opt-in sharing exists.

How NeoCore scores on memory benchmarks

Two public benchmarks ask questions about long conversation histories: the system recalls memory, a small answering model replies, and a judge marks it. These runs used NeoCore’s recall without NeoSift, with zero LLM calls during recall. We list only published results that used a gpt-4o-class or smaller answering model, because answering models and judge prompts move scores a lot.

LongMemEval-S: 89.6% 500 questions · official GPT-4o judge

Chronos (Low)GPT-4o reader + Cohere rerank + Gemini prompts
92.6%
NeoCore 0.2gpt-4.1-mini reader · 0 LLM calls during recall · ~4k tokens
89.6%
Mastra Observational Memorygpt-4o reader · ~30k tokens
84.2%
Supermemorygpt-4o reader
81.6%
Zepgpt-4o reader (Zep paper)
71.2%
Whole history in contextgpt-4o reader · ~115k tokens
60.6%

Second among the published small-model results we found. Chronos scores 92.6% with a GPT-4o reader, a hosted reranker and extra model calls. In its reported setup, giving the reader the whole history scored 60.6%.

LoCoMo: 87.5% 1,540 questions · gpt-4.1-mini answering model

Whole conversation + step-by-step answers~25k tokens per question
92.6%
EverMemOSaudit re-score (paper: 93.1)
92.3%
MemMachine v0.2agent mode
91.7%
NeoCore 0.20 LLM calls during recall · ~3.6k tokens
87.5%
Zepas run by the EverMemOS authors
85.2%
Whole conversation, short answers~25k tokens per question
82.0%
MemOSas run by the EverMemOS authors
80.8%
Mem0as run by the EverMemOS authors
64.2%

Here we don’t win. EverMemOS and MemMachine score 4–5 points higher, and so does the whole conversation in context with step-by-step answers, at about 25k tokens per question against NeoCore’s ~3.6k. NeoCore is ahead of Zep, MemOS and Mem0 as run by the EverMemOS authors. Answer prompts still differ between rows, so read small gaps with care.

How to read these numbers

Judges and answer prompts differ between published results. The LoCoMo audit shows how much the answer prompt alone can move a score, and that part of LoCoMo’s answer key is wrong.
Results of 92–95% from Zep, Mem0, Mastra, MemMachine and others use GPT-5-class answering models or judges, or do not say which. They are left out because they are not comparable, not because they are low.
NeoCore forms dated facts with one small-model call per stored session. Recall itself uses local search and embeddings, with no LLM calls. The benchmark runs did not use NeoSift.
Every external number links to its source. Ours come from the repository’s evals, run through the public package.