NeoCore gives Claude Code and Codex one shared memory. On every prompt it searches your past sessions and injects what it finds. That search is fast, local and makes no LLM calls. It finds the right memories well: 87.5% on LoCoMo, 89.6% on LongMemEval-S.
But finding is only half the job. Search works on words and meaning, so it also finds memories that just look related. Ask “is the worker deploy fixed?” and you get the fix, but also three old chats that mention the worker, a deploy from a different project and a lesson about something else. All of it lands in the context window.
Why not just add more memory?
Context windows are big now, so it’s tempting to inject everything that might matter and let the model sort it out. In practice that costs you twice:
- It crowds out the work. Every line of memory is a line of your code or your conversation the model doesn’t see as clearly.
- It distracts the model. A memory that shares words with the prompt but is about something else is not neutral. The model may act on it.
The other option is to ask a hosted model, on every prompt, which memories matter. That was 0.1’s optional filter, and it works. But it’s a network call per prompt, it costs a little each time, and your prompts and memories leave your machine.
The middle path is a small model that does just this one job. A cross-encoder reads the prompt and one candidate memory together and gives a single number: how likely is this to help? It doesn’t write text, so it can be very small.
NeoSift 32M
NeoSift reads the prompt, the end of the turn before it (a follow-up answers the last thing the assistant said) and each candidate memory. Recall offers about 48 candidates. NeoSift scores them all, and only the best are injected: a score of at least 0.4, at most 12.
How it was trained. We took real coding-assistant prompts and the memories recall offered for each. A frontier model (GPT-6.1) graded every candidate: would this help answer the prompt? Then a small model learned to reproduce those grades. It started from Ettin, an open 32M reranker (Apache-2.0). This is round 10. The big model does the expensive thinking once, offline; the small one carries it to every prompt, on your machine.
The numbers
We locked 103 real prompts away from training and measured how well each scorer ranks useful memories above useless ones:
| Scorer | Where it runs | Score |
|---|---|---|
| GPT-6.1 (the teacher) | hosted | 0.970 |
| TypeSafe Jev 1.13 | hosted, ~$0.0007 per prompt | 0.931 |
| NeoSift 32M | your graphics card, free | 0.914 |
The student doesn’t match the teacher. It gets close to the hosted filter, at no cost, with nothing leaving your machine. The full comparison, with 13 open rerankers and Claude Opus as a judge, is in Where NeoCore stands.
Two more results I find useful:
- Saying “nothing”. On prompts where no memory was useful, NeoSift still let something through 40% of the time, against 74% for Jev.
- Hindsight. On 885 real prompts graded after their replies, the default cut-off injected 5.7 memories per prompt and caught 66% of the memories the replies needed. So it misses about a third. Raising the cut-off trades needed memories away about as fast as it saves space.
The catch: it needs a graphics card
One prompt (48 candidates) takes 0.07 s on an RTX 3080. On a desktop processor it takes 7-8 s,
longer than a prompt can wait. So neocore setup installs NeoSift only when it finds an NVIDIA
card or Apple silicon, and only if a speed check passes. Without one, recall injects its top
matches, or uses the hosted filter if you set an OpenRouter key.
It also fails open. If NeoSift can’t answer in time, that prompt gets unfiltered recall, not nothing. A memory filter should never cost you your memory.
Short follow-ups
“Yes do it.” “And the other one?” These carry almost no words to search with, so NeoCore searches them with the opening of the assistant’s last reply. 0.2 also searches with the prompt alone and adds its best results. More of the right candidates reach the filter, which drops the rest.
Helping it learn (opt-in)
NeoSift has learned mostly from one developer’s coding prompts: mine. On two business assistants’ prompts it agreed with its teacher less well (0.68 against 0.80, in an earlier round). It needs more kinds of prompts.
So setup asks once whether you’d like to help. It is off unless you say yes. If you do:
- Sent: the prompt, the end of the turn before it, and the memories recall offered with their scores, after NeoCore’s usual secret redaction.
- Never sent: the reply to the prompt, tool output or whole conversations.
- On arrival each batch is encrypted with a key only our training machine holds.
- Used only to grade and train NeoSift, which outputs scores and can’t reproduce text.
neocore share deleteremoves everything your install sent, including copies already collected for training.
The exact rules are in the README, and the code is one short file (share.py).
Try it
pipx install --force "neocore-memory[embed] @ git+https://github.com/VincentPorquet777/[email protected]"
neocore setup
neocore sift statusYour existing memory is kept. 0.2 is beta: NeoSift needs a graphics card, and it’s still learning outside coding. If it keeps the wrong memory or drops the right one, please open an issue. Those are the cases it learns from.