DEV Community: Oleksander

Can You Build an Alternative to LLMs? 8 Months, ~200 Failed Experiments, One Wall. 2

Oleksander — Sun, 05 Jul 2026 07:22:18 +0000

This is part 2 of a research series documenting an attempt to build something adjacent to — and in some ways alternative to — large language models. Part 1: "My Synthetic Eval Said 30/30. LoCoMo Said 0.13." The code that survived lives in AuraSDK.

Not a chatbot wrapper.

Not another prompt stack.

Not a vector database with a nicer UI.

The question was narrower and more dangerous:

Can a non-neural, CPU-only system accumulate experience, mutate its own internal state, and improve future behavior without retraining an LLM?

After eight months, the honest answer is not "yes".

The honest answer is:

some mechanisms survived,
most carriers failed,
and the wall became precise.

The wall is not storage. Storage is easy.

The wall is transferable causal transition:

condition -> action -> consequence -> when NOT to apply it

That is the thing I kept failing to preserve.

1. What I Mean by "Alternative to LLMs"

The phrase is too broad, so I need to narrow it.

I was not trying to build a new frontier model. I was not training a transformer from scratch. I was not trying to beat GPT-class models at language.

The target was smaller:

Find a knowledge substrate that behaves like a tiny, mutable set of weights.

The required properties were:

it changes after experience;
it survives restart;
it changes future behavior without code changes;
it transfers to unseen but related cases;
it fails shuffled/null controls if the signal is fake;
it is compact in a behavioral sense, not merely compressed bytes;
it abstains when unsupported instead of confidently guessing.

In other words, I was looking for something between memory and weights.

Memory stores what happened.

Weights change what happens next.

The experiment was to find a symbolic or hybrid substrate that could cross that line.

2. Why Normal Memory Was Not Enough

The first tempting answer is always memory:

append the conversation;
summarize it;
store facts;
retrieve relevant chunks;
build a graph;
add embeddings;
add scores;
add typed edges.

All of those are useful engineering tools. None of them automatically become knowledge.

The distinction became brutal:

storage can preserve an event;
knowledge must change the next action.

A log can say:

candidate A failed in situation X
candidate B worked in situation X

But a useful system must do more:

in a new situation X',
recognize what is shared with X,
avoid A-like actions,
try B-like actions,
and know when the analogy no longer applies.

That last line is where most designs died.

3. The Carrier Bakeoff

I tested many candidate "carriers" of knowledge. A carrier is the internal form that is supposed to hold experience and make it reusable.

The early list looked reasonable:

Carrier	What it preserved	Where it failed
append-only memory	event history	did not transfer to unseen cases
surface compression	shorter text	compressed language, not behavior
graph / n-gram links	co-occurrence	became an index, not a causal model
route state	role and expectation	changed behavior, but stayed narrow
typed edges	relation labels	helped only when the relation grammar was already given
graded vectors	magnitude and pressure	carried direction, not an executable mechanism
living cells	local conflict	transferred some signal, but could transfer the wrong action
Jacobian-like footprint	local numeric effect	worked on linear repairs, failed on branching and bit-shift semantics
topology fields	learned attraction basins	signal on synthetic worlds; on real cargo snippets — 0/5 correct, 5/5 abstain

The pattern was consistent.

Each carrier preserved one aspect of consequence:

that something was supported;
that something was refuted;
that a choice had pressure;
that two roles were related;
that a numeric direction changed;
that a local conflict existed;
that a state transition had a shape.

But none of the early carriers preserved the full causal transition.

That is the difference between:

"A was good near B"

and:

"When input has property P and state has condition C,
action A changes state S in direction D,
unless boundary condition B is active."

The second form is much closer to knowledge.

It is also much harder to acquire without already having a model that can do it.

4. Signals That Were Real but Not Enough

Some results were not failures in the simple sense. They were real signals.

One route-state experiment proved that internal state can change behavior without changing code. The system saw support and refutation, mutated its state, and later selected differently. The shuffled-state control failed. That was important.

But it did not prove a replacement for weights. It proved that a narrow symbolic state can influence selection.

Another experiment used graded consequence vectors. The vector was updated by experience, not by gradient training. It showed useful properties:

same direction, more support       -> larger magnitude
50% shared experience              -> similarity ~0.41
0% shared experience               -> similarity ~0.05  (margin 0.35)
support followed by scar evidence  -> direction flipped
random nudges                      -> near-zero similarity

An important detail: 0.41 (not 1.0) on partially shared experience is exactly what real generalization looks like, as opposed to memorization. The similarity emerges from structure; it is not hard-coded. That too was a real signal.

But it was not enough. It carried pressure and similarity. It did not carry the executable rule of when a transition should fire.

The most honest label for these results:

SIGNAL, not solution.

That distinction saved months of self-deception.

5. The "Abstain" Lesson

One of the most useful discoveries was negative:

A weak knowledge carrier should abstain instead of guessing.

In several gates, the system got better when unsupported cases were answered with "I do not know" instead of being forced into the nearest known pattern.

This matters because many memory systems look good only because they always answer. But in a causal system, a wrong transfer is worse than silence.

If a stored pattern says:

this action repaired the previous case

and the new case looks similar but has a different boundary condition, reusing the action can be actively harmful.

The real requirement became:

transfer when supported;
abstain when the binding is not justified;
never convert weak similarity into certainty.

This is where many memory architectures quietly fail. They retrieve something. The model uses it. The answer looks grounded. But the binding between old evidence and the new situation may be invalid.

6. Executable Worlds Made the Failure Sharper

Synthetic gates are useful, but dangerous. If you design the world, you can accidentally design the success.

So part of the testing moved into small executable worlds: code snippets, controlled bugs, compiler feedback, repair attempts, and real pass/fail consequences.

The results became sharper.

A local numeric footprint could transfer part of a lived consequence pattern. It worked when the change was close to a linear repair:

increase this value;
change this numeric boundary;
map this local residual to that local patch.

But it failed on cases that required real mechanism:

bit-shift semantics;
branch behavior;
boundary conditions;
state transitions.

That mattered. Passing a generated linear-repair gate does not mean the system learned code. It means the carrier can reuse a local numeric direction.

The criterion became stricter:

a carrier must survive mixed nonlinear executable snippets:

- numeric delta;
- bit shift;
- boundary condition;
- state transition;
- negative abstain.

If it only passes local-linear repair, it is a helper operator, not a candidate core.

7. The Single-File Brain Attempt

One direction looked especially attractive:

What if the entire learned state lived in one mutable file?

The idea was simple:

The system acts.
The world returns consequences.
The state file mutates.
After restart, the file still changes future behavior.

This was not supposed to be a database. It was supposed to be a growing behavioral substrate.

The result was mixed:

transfer cycle: 3/5
blind baseline: 1/5
wrong: 1

So the file was not dead storage. It grew. It mutated. It transferred some lived transition experience. It beat the blind baseline.

But wrong=1 matters more than the improvement.

The system did not merely abstain on unsupported transfer. It sometimes confidently selected a wrong action. For a knowledge carrier, that is a critical failure.

The diagnosis:

bound transition-cells increased capacity,
but did not preserve enough mechanism.

This is a recurring pattern:

more structure != more knowledge

A richer container can still carry the wrong abstraction.

8. The Wall Became a Binding Problem

After many carrier failures, the question changed.

At first I asked:

Which carrier stores knowledge best?

Later, the better question was:

How does a new situation bind to old experience?

The binding is the hard part.

Suppose the system has learned a useful transition in one world:

node A influences node B
changing A repairs failure F

Now it sees a new world with different node names and a different surface form.

Which node corresponds to A?

Which node corresponds to B?

Which local structure is the same mechanism, and which is only superficially similar?

Without an anchor, this turns into graph matching. In one of the gates I attempted an autonomous bijection between train and heldout worlds directly — and failed exactly here: computing the correspondence required already knowing the correspondence. That is graph isomorphism, NP-hard without an anchor. Not an engineering obstacle — a fundamental one.

This produced the anchor trilemma.

9. The Anchor Trilemma

Every attempted solution fell into one of three buckets.

Option 1: The anchor is given

If a human or a hand-written rule tells the system which parts correspond, transfer becomes much easier.

But then the hard part was not learned. It was supplied.

This can still be useful engineering. It is not an alternative knowledge substrate.

Option 2: The anchor is searched

If the system searches over possible bindings, it can sometimes find the match.

But the search explodes quickly. The more nodes, relations, states, and conditions, the less attractive this becomes.

You have not built cheap knowledge. You have moved the cost into combinatorial search.

Option 3: The anchor is learned by a model

If an LLM, an embedding model, an analyzer, or a world-probing system supplies the binding, the system works much better.

But then the symbolic substrate is no longer the source of the core intelligence. It becomes a memory, a cache, a verifier, or an optimizer around another intelligence source.

That may be a good product.

It is no longer the original hypothesis.

The trilemma:

given anchor    -> not learned
searched anchor -> too expensive or unstable
learned anchor  -> not independent from the model/teacher

This was the wall.

10. What Actually Survived

The failed broad claim was:

A symbolic carrier can become alternative weights by storing enough structured consequences.

That did not survive.

What survived was narrower.

1. Consequence loops matter

The system improves only when it receives world feedback:

try -> observe -> mutate -> retry

Static documents, summaries, and graphs are weak unless tied to consequences.

2. Deterministic guards matter

Some fields must not be entrusted to a generative or fuzzy substrate:

IDs;
dates;
amounts;
names;
paths;
exact constraints;
status changes.

Extract them and preserve them explicitly.

3. World judges matter

The compiler, tests, and executable checks were more honest than internal scores.

They do not care how elegant the architecture is. They return:

pass / fail

4. Abstention is a feature

A weak system that abstains is more useful than a weak system that always transfers.

5. Some organs are still valuable

Route-state, scars, typed relations, date guards, append-only memory, cheap probes, and executable judges are not useless.

They are useful organs.

They are not a full brain.

11. What I Am Not Claiming

I am not claiming I built an alternative to LLMs.

I am not claiming symbolic systems cannot work.

I am not claiming all memory systems are useless.

I am not claiming the wall is mathematically insurmountable in every form.

The honest claim is narrower:

In my experiments, every carrier that tried to preserve knowledge as stored structure eventually failed at transferable causal binding.

And there is an even narrower boundary I have to state myself, before someone states it for me: this is one person working for 8 months on a CPU, not the output of a lab. There may be a carrier I never tried. But 150+ gates kept failing at the same point so consistently that the pattern outweighs any single failure — and that point coincides with why symbolic AI historically lost to learning: the correspondence metric has to be learned, not postulated.

That is still useful.

It prevents wasting another cycle on prettier containers.

12. The Test for Any New Approach

After these failures, a new approach is worth testing only if it clears a stricter bar.

It must:

change future behavior without code changes;
survive restart;
beat shuffled and null controls;
transfer to unseen cases;
abstain on unsupported cases;
preserve exact fields separately;
operate in a world with consequences;
handle nonlinear transitions, not only local numeric repair;
show value over a simple baseline.

And most importantly:

it must carry the transition,
not just an aspect of the transition.

This is the line that killed most of my designs.

13. Why This Still Matters

A negative result is not the same as no result.

Before these experiments, the problem looked like:

find a better memory structure

After the failures, it looks like:

store and transfer executable causal transitions with valid binding

That is a much better problem statement.

It also changes product thinking. A useful system does not need to pretend it replaces an LLM. It can be valuable if it provides:

cheaper long-session memory;
evidence-preserving state;
deterministic guards;
world-verified actions;
refusal to overwrite known scars;
test generation from consequences;
bounded automation around external judges.

Those are real mechanisms.

They just should not be sold as "alternative weights" until they pass the binding wall.

14. Conclusion

The original dream was:

build a CPU-only mutable substrate that behaves like alternative weights

The experiments produced something less glamorous and more useful:

storage is easy;
behavioral mutation is possible;
transfer is fragile;
causal binding is the wall.

The wall is not that the system cannot remember.

The wall is that remembering is not enough.

To act intelligently in a new situation, the system must know what old experience corresponds to, which transition applies, and where the analogy breaks.

That is the part I could not solve with another graph, vector, summary, route state, or mutable file.

The final number:

~200 experiments, 30+ candidate carriers, 1 wall.

Series: "Can You Build an Alternative to LLMs? 8 Months of Experiments, 200 Failures, and One Wall" 1

Oleksander — Thu, 02 Jul 2026 12:59:15 +0000

I tested a simple hypothesis: can long LLM sessions be made cheaper by replacing the full transcript with a compact memory state, without losing answer correctness?

On my own synthetic eval, the system passed 30/30. On an external benchmark for long-term conversational memory, the score collapsed to 0.13. The failure was not caused by one bad prompt. It came from the type of memory being preserved: the local eval tested exact facts, while LoCoMo tested episodic memory.

After several failed approaches, one narrow architecture survived: append-only memory plus deterministic guards for exact fields, especially dates. It is not general-purpose memory, but it produced an honest result: 94% retention with 60% per-query saving on 200 QA.

The main lesson: synthetic evals are useful as regression tests, but dangerous as evidence. If an eval is written around the mechanism being tested, the system can look strong exactly where it proves the least.

1. The Problem

Long LLM sessions are expensive. If every new request receives the entire previous transcript, cost grows with conversation length. The obvious engineering response is to compress the history into a compact state and give the model that state plus the most recent turns.

The hypothesis was:

Compact session state can reduce prompt/context usage without losing answer correctness.

This is attractive. It promises smaller prompts, longer sessions, cheaper agents, and less irrelevant context. But it hides a trap: if the state compresses the wrong type of information, it does not optimize memory. It deletes data.

2. The Local Eval Looked Good

I started with my own 30-case corpus. It covered:

exact facts;
support notes;
CRM-style notes;
coding sessions;
RAG-like context;
mixed-language facts;
preferences and decisions;
negative short-context cases.

Example:

{
  "id": "exact_authorization_code",
  "question": "What is the authorization code? Answer only the code.",
  "expected_fragments": ["RX-4471"]
}

On this corpus, the system looked good. But the first pretty numbers were dry-runs. A dry-run does not call the model: the answer equals the expected fragments by construction. That mode is useful for checking the pipeline, but it is not evidence of quality.

The first full real-model run, after fixing checker artifacts and a mixed-language gap, gave:

corpus:       30 cases
noise:        +40 turns
accuracy:     1.000  (30/30)
effectiveness_rate: 1.000
context_window_saved_pct: 82.86%
false savings: 0
GATE: PASS

This was stronger than a dry-run. But the central problem remained: the corpus was mine. It was written around what the mechanism was already good at preserving.

3. Why Synthetic 30/30 Was Not Enough

The local eval mostly tested exact facts and durable decisions:

codes;
dates in strict formats;
IDs;
file paths;
explicit preferences;
short rules;
"do not forget" facts.

That is important for regression testing. But it is not the same as long-term conversational memory.

Real conversational memory often asks:

when something happened;
what "yesterday" meant relative to a specific session;
how events across several dialogues are connected;
who said what;
what two people have in common;
which fact is needed for an answer even though it is not named in the question.

My local eval tested the type of memory I already knew how to compress. The external benchmark tested what I had not shaped around my own mechanism.

4. LoCoMo Broke the Claim

For an external check, I used LoCoMo: a benchmark for very long-term conversational memory. LoCoMo dialogues average around 300 turns, 9K tokens, and up to 35 sessions. It tests long-term memory through QA, event summarization, and multi-session dialogue understanding.

The first result was harsh:

raw              : 8/15 = 0.53
projected_facts  : 2/15 = 0.13
projected_hybrid : 2/15 = 0.13

The optimized state retained only about a quarter of the correct answers preserved by raw context. The context saving looked excellent: 94-99%.

That was not optimization. It was deletion of needed information.

After adding session timestamps to dialogue lines, the raw baseline improved:

raw              : 0.80
projected_hybrid : 0.20
retained_vs_raw  : 0.25

So the issue did not disappear with a better baseline. It became clearer that the projected state was not preserving the required memory type.

There is an important methodological caveat: the early 0.13 used a strict substring/token checker. Such a checker can miss semantically correct date answers. For example, the gold answer may be the sunday before 25 May 2023, while the model answers 20 May 2023.

But the gap was too large to dismiss as a checker artifact: raw had 8 correct answers, projected had 2.

5. What Failed

After LoCoMo, three failures became visible.

5.1 Static Facts Are Not Episodic Memory

The initial state preserved exact facts well:

codes;
paths;
IDs;
strict dates;
explicit user decisions;
preferences;
constraints.

But it poorly preserved:

event sequence;
relative dates;
"yesterday" relative to a session date;
shared interests;
multi-hop links;
evidence-neighbor context.

LoCoMo asked for episodic memory: who did what, when, where, and how it connects across sessions.

5.2 Lexical Retrieval Was Not Enough

I also tested a retrieval-style approach: instead of compressing everything into one state, select relevant chunks for each question.

Result:

append_full      : 32/60 = 0.533, query saving 45.89%
append_retrieved : 25/60 = 0.417, query saving 90.73%

append_retrieved looked better economically, but quality dropped. The reason is simple: lexical overlap fails when the question and the evidence do not share words.

Typical failures:

"What did Caroline research?" did not retrieve adoption agencies;
a shared-destress question did not retrieve dance;
a martial-arts question did not retrieve Kickboxing, Taekwondo;
temporal and multi-hop questions broke more often.

5.3 Dates Were a Separate Failure Class

At 200 QA, the main gap localized to the temporal category:

raw         : 126/200 = 0.63
append-only : 111/200 = 0.56
retention   : 88%

by category:
multi-hop   : 46 / 45   (~98%)
temporal    : 58 / 46   (~79%)
open-domain : 18 / 16
single-hop  : 4 / 4

This distinction mattered. The memory was not uniformly bad. Temporal anchors were bad.

When the diagnosis is precise, the fix can be precise.

6. Ladder of Attempts

There was no direct jump from the first failure to the final result. Several approaches died for specific reasons.

Approach	Result	Why it failed or narrowed
text compression / projected state	`2/15 = 0.13`	kept exact facts, lost episodic memory
MinHash-style lexical retrieval	`3/15 = 0.20`	lexical overlap missed paraphrase; evidence hit about `42%`
evidence oracle	`0.30` under strict checker	even exact evidence lines did not guarantee date-equivalent substring match
recode-to-notation smoke	`3/3 = 1.00`	small smoke was too optimistic
recode-to-notation larger slice	unstable: `0.35` in one slice, `0.70` vs raw `0.75` under LLM judge in another	interesting signal, not stable enough
append-only without date-guard	`111/200 = 0.56`, retention `88%`	most loss concentrated in temporal questions
append-only + date-guard	`94% retention`, `60% per-query saving`	first narrow result that survived scale better

This table is the real research story. The useful mechanism was not guessed. It survived because prettier mechanisms died first.

7. What Survived

The surviving design had two constraints.

First: do not re-summarize the whole state.

Second: protect exact fields deterministically.

The append-only rule is simple:

Compress only the new exchange.
Freeze the compressed chunk.
Append it to memory.
Never re-compress old chunks.

Why this matters: if old facts repeatedly pass through a compressor, small losses accumulate. If each exchange is compressed once and then frozen, loss cannot compound in the same way.

Early append-only results looked even better than raw: 103% retention on 6 conversations. That was a small-sample artifact. At 200 QA, retention fell to 88%. This was useful: it showed that the architecture helped, but the temporal gap was still real.

8. Date-Guard

The fix was deterministic date protection.

The idea:

extract absolute time expressions;
extract relative time expressions;
attach session date to relative expressions;
append these time anchors to the compressed state.

This does not ask the LLM to be careful with dates. It removes the choice. The compressor can shorten prose, but date anchors survive as explicit fields.

Result on the 200 QA setup:

without guard:
retention        88%
temporal cat     79%
per-query saving 45%

with date-guard:
retention        94%
temporal cat     96%
per-query saving 60%

Saving improved because the prose compressor could become more aggressive once dates were protected separately.

Final honest number:

append-only + date-guard: 94% retention, 60% per-query saving

This is not solved memory. It is not a universal alternative to LLM context. It is a narrow result: append-only compression plus deterministic protection for exact fields.

9. Cost and Scope

This mechanism is not useful for short chats.

On short sessions, fixed overhead can exceed savings:

0 added noise exchanges: projected is worse than raw
1 added noise exchange : projected is still usually worse

Another important distinction: context-window saving and API-cost saving are not the same metric.

One real-provider smoke test showed:

context_window_saved_pct_vs_raw_estimate: 14.61
provider_total_saved_pct_vs_raw: -2.83

The final prompt was smaller, but total provider cost was worse because preparation added calls.

For append-only full on 60 QA:

query_saved_pct_vs_raw   : 45.89%
product_saved_pct_vs_raw : 23.92%
break_even_queries       : 28.72
net_saved_pct_at_200     : 39.30%

So the mechanism is only interesting for long sessions where setup cost can be amortized.

10. Lessons

Lesson 1: Synthetic eval is not evidence by itself

Synthetic eval is useful for regression. It is weak evidence for generalization.

If the author writes the eval around their own mechanism, the system can pass by matching the author's blind spots.

Lesson 2: Compression ratio is a vanity metric without retention

99% context saving is meaningless if answer retention collapses.

The key metric is not "how much did we delete?" but "how much correct behavior survived?"

Lesson 3: Memory is not one thing

Exact facts, preferences, episodic events, temporal anchors, multi-hop relations and source evidence are different memory types.

A compressor can preserve one and destroy another.

Lesson 4: Deterministic guards matter

Some fields should not be entrusted to a generative summary:

dates;
amounts;
IDs;
codes;
names;
statuses;
paths;
constraints.

If losing a field breaks correctness, extract it deterministically and preserve it explicitly.

Lesson 5: Small samples lie

The 103% retention result on 6 conversations looked exciting. At 200 QA it became 88%. The useful signal was not the optimistic number, but the category breakdown showing where the loss happened.

11. Limitations

This is not an academic benchmark paper.

Limitations:

one main external conversational-memory benchmark;
small and medium QA slices before the 200-QA run;
no statistical significance analysis;
some early measurements used strict substring checking;
LLM-judge checks reduce one problem but introduce another;
final 94%/60% should be published with a compact appendix table before being treated as a stable claim.

The result is best read as an engineering research note: a failed broad claim, a localized diagnosis, and a narrower mechanism that survived better tests.

12. Conclusion

The local eval said:

30/30

The external benchmark said:

0.13

The final surviving mechanism said:

94% retention / 60% per-query saving

The important result is not that this "solves memory". It does not.

The important result is that an external benchmark forced the system to stop lying through its own eval. The useful architecture appeared only after the original success story failed.

References

LoCoMo: "Evaluating Very Long-Term Conversational Memory of LLM Agents" — https://arxiv.org/abs/2402.17753
Lost in the Middle: "How Language Models Use Long Contexts" — https://arxiv.org/abs/2307.03172
"Investigating Data Contamination in Modern Benchmarks for Large Language Models" — https://arxiv.org/abs/2311.09783
"Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Models" — https://arxiv.org/abs/2311.06233

Why AI Needs an External Cognitive Layer Beyond Memory

Oleksander — Thu, 02 Apr 2026 16:30:15 +0000

Why AI Needs an External Cognitive Layer Beyond Memory

Most AI agents today are still built around a thin pattern:

a large language model,
a prompt,
a tool loop,
and some form of memory or retrieval.

That stack can look impressive in demos, but it breaks down once the agent needs continuity, specialization, self-consistency, and long-lived behavioral control.

Memory alone is not enough.

If an agent only stores past records, it can remember what happened. It still cannot reliably:

form stable beliefs,
build concepts over time,
learn causal structure,
accumulate policies,
generate internal pressure,
anticipate future outcomes,
detect epistemic gaps,
or regulate its own mode of operation.

That is the gap we have been exploring in Aura.

The Core Thesis

AI systems need an external cognitive layer that lives outside model weights.

Not just a vector database.
Not just a chat history.
Not just a memory API.

A real cognitive layer should be able to:

preserve continuity across sessions,
accumulate knowledge in structured form,
survive model upgrades,
support domain specialization,
remain inspectable and governed,
and shape agent behavior over time.

This matters because current LLMs are powerful, but they are still weak at stable long-horizon cognition. They are excellent inference engines. They are not yet sufficient as complete cognitive architectures.

From Memory to Cognition

In Aura, the architecture has gradually moved beyond simple memory.

The working progression is:

Record
Belief
Concept
Causal
Policy

That already changes the role of memory.

The system is no longer just storing facts. It is organizing experience into a structured cognitive state.

And once that structure exists, new layers become possible.

The Next Four Cognitive Functions

The recent evolution of the system can be summarized in four steps:

1. Want

The system should not only react to prompts.

It should also detect internal tensions:

unresolved policy pressure,
contradictions,
unstable structure,
pending cognitive obligations.

Those tensions can flow into drives, goals, and imperative-like internal pressure.

That is the beginning of motivation.

2. Expect

A cognitive system should not only remember the past.

It should form expectations about what should happen next.

From stable causal structure, it can produce predictions.
From mismatches between expectation and observation, it can produce surprise.

That turns cognition from retrospective to anticipatory.

3. Wonder

A capable system should not only repair contradictions.

It should also notice what it does not know.

Epistemic gaps matter:

weakly grounded entities,
missing causal mechanisms,
underspecified policy dependencies,
repeated blind spots,
ambiguous concept boundaries.

That is the beginning of curiosity.

4. Regulate

A cognitive system should not always behave in exactly the same mode.

Under pressure, it may need to become more conservative.
Under stability, it may be able to explore.

This is not about emotional theater.
It is about regulation.

A global modulation layer can shape:

drive thresholds,
curiosity thresholds,
exploration budget,
and behavioral selectivity.

That is the beginning of self-regulation.

Why This Should Live Outside the Model

This is the most important architectural point.

If all cognition lives only inside model weights, you lose too much:

portability,
auditability,
versioning,
organization-level control,
inspectability,
and long-lived continuity across changing model generations.

An external cognitive layer can survive:

model upgrades,
shell changes,
deployment changes,
domain swaps,
and organizational adaptation.

That makes it more durable than any single model interface.

Why This Matters Commercially

This is not only a research direction.
It is also a product direction.

A governed external cognitive layer enables:

specialist cognitive bases,
organization-specific overlays,
persistent agent continuity,
safer multi-step behavior,
and explainable adaptation without retraining.

That creates a path beyond generic chat agents.

Instead of selling only an agent, you can sell:

a cognitive substrate,
a specialist module,
and an organization layer that persists over time.

Why This Matters Even If Model Architectures Change

A common objection is:

What if future models already include better internal cognition?

That does not remove the need for an external layer.

Even if models become far more capable, organizations will still need:

governance,
portability,
ownership,
rollback,
specialist control,
and cognition that survives vendor changes.

So the long-term bet is not:

"models will stay weak."

The better bet is:

"portable, governed cognition will still matter even when models get stronger."

The Direction

The future of agent systems is unlikely to be:

model only,
prompt only,
or memory only.

It will likely require a distinct cognitive layer that can:

accumulate structured knowledge,
generate internal motivational pressure,
anticipate,
explore,
regulate,
and remain externally governed.

That is the direction we think is worth building toward.

Not just better memory for AI.

A real cognitive layer beyond memory.

I am currently building and testing this cognitive architecture in a closed environment. If you are an AI architect, researcher, or founder hitting the limits of RAG and standard agent loops, my DMs are open. I’d love to compare notes on the future of autonomous cognition.

Pennsylvania State found why AI memory fails across models. AuraSDK doesn't have this problem.

Oleksander — Fri, 27 Mar 2026 16:52:05 +0000

Pennsylvania State University just published a paper that exposes a structural flaw in how most AI agent memory systems work.

The paper is called MemCollab: Cross-Agent Memory Collaboration via Contrastive Trajectory Distillation. The findings are uncomfortable if you're building agent memory the conventional way.

The flaw

Most agent memory systems work like this:

Model solves a problem
Memory stores the reasoning trace — what the model did, how it got there
Model retrieves that memory later and performs better

The assumption buried inside this design: the stored knowledge is about the task, not about the model that solved it.

Pennsylvania State tested whether that assumption holds.

They gave a 7B model's memory to a 32B model. MATH500 dropped from 63.8% to 50.6%. HumanEval dropped from 68.3% to 34.1%.

Then they gave the 32B model's memory to the 7B model. Performance dropped again. Both directions failed. Both fell below the zero-memory baseline.

Giving a model someone else's memory made it perform worse than having no memory at all.

Why this happens

A model's reasoning traces don't just capture what the correct answer required. They capture how that specific model thinks — its preferred solving strategies, its heuristic shortcuts, its stylistic patterns.

Memory distilled from those traces encodes the model's reasoning personality alongside the actual task knowledge. When a different model retrieves that memory, it gets handed instructions optimized for a completely different cognitive architecture. The guidance actively interferes.

What MemCollab does

MemCollab fixes this by making memory construction cross-model. Two agents — a smaller and a larger model — independently solve the same problem. One succeeds, one fails. The system contrasts the trajectories and extracts only the abstract invariants:

What reasoning principle was present in the success and violated in the failure?
What error pattern appeared in the failure that the success avoided?

The extracted memory stores only those rules — not the solution, not the reasoning style, not the model-specific heuristics.

Results:

Llama 3 8B: MATH500 from 27.4% → 42.4%
Qwen 7B: MATH500 from 52.2% → 67.0%, HumanEval from 42.7% → 74.4%
Reasoning turns cut from 3.3 → 1.5 on HumanEval (fewer dead ends)

The deeper insight

The efficiency finding is the one that gets overlooked. MemCollab doesn't just improve accuracy — it makes agents reach correct answers in fewer steps. The contrastive memory isn't adding more guidance. It's stripping out the noise that was making agents explore dead ends repeatedly.

By encoding what not to do as explicitly as what to do, the memory prunes the search space before the agent even starts.

Why AuraSDK doesn't have this problem

AuraSDK avoids the contamination problem structurally — by never storing reasoning traces at all.

When you store something in AuraSDK:

brain.store(
    "Staging deploy prevented 3 production incidents",
    semantic_type="fact",
    tags=["workflow", "deployment"]
)

You're storing a claim about the world, not a record of how a model reasoned about it. The cognitive layers — Belief, Concept, Causal, Policy — are derived from the content of what was observed, not from the model's processing of it.

Record → Belief → Concept → Causal → Policy

Each layer is built deterministically from the one below. Beliefs emerge from clusters of records. Causal patterns emerge from temporal co-occurrence and explicit links. Policy hints emerge from repeated causal patterns. None of this touches model internals.

The result: the cognitive layer is model-agnostic by design. Swap GPT-4o for Claude, swap Claude for Llama — the stored memory, the belief structure, the causal patterns, the policy hints all remain valid. There's nothing model-specific to contaminate.

Two different approaches to the same insight

MemCollab and AuraSDK arrive at the same conclusion from different directions:

Memory that encodes how a model thinks is fragile. Memory that encodes what happened is durable.

MemCollab fixes contamination after the fact — by contrasting two models' traces and extracting only what survived.

AuraSDK avoids contamination by construction — by never storing traces in the first place.

	MemCollab	AuraSDK
What's stored	Abstract reasoning invariants across models	Claims, facts, relationships
Requires LLM to build memory	Yes — two models per problem	No
Model-agnostic	Yes — by contrastive distillation	Yes — by design
Works offline	No	Fully
Recall latency	LLM-bound	0.076ms
Cognitive layers	None	Belief → Concept → Causal → Policy
Open source	Research paper	MIT, ships today

What this means for the field

The Pennsylvania State paper validates something important: the right unit of memory is not a reasoning trace. It's the abstract principle that holds regardless of which model does the reasoning.

AuraSDK takes this further: the right unit of memory is a structured observation about the world — a fact, a decision, a contradiction, a preference — that any model can retrieve and use without being handed someone else's cognitive fingerprint.

The field is converging on this. The implementations differ. But the core insight is the same.

pip install aura-memory

GitHub

Google's TurboQuant solves half the AI memory problem. Here's the other half.

Oleksander — Wed, 25 Mar 2026 16:49:10 +0000

This week Google Research published TurboQuant — a two-stage KV-cache quantization algorithm that achieves 6x memory reduction and 8x attention speedup with zero accuracy loss at 3 bits. No training required.

It's genuinely impressive engineering. But it's worth being precise about what problem it solves.

The two AI memory problems

Most people conflate two distinct problems:

Problem A: memory within a session
As context grows, the KV-cache grows. It becomes expensive in RAM and slow in attention computation. TurboQuant solves this — brilliantly.

Problem B: memory between sessions
When the session ends, the KV-cache is gone. The model starts from zero next time. No memory of past interactions, no accumulated patterns, no structured experience. TurboQuant doesn't touch this.

What TurboQuant actually does

TurboQuant is a two-stage pipeline:

PolarQuant — rotates vectors randomly, converts to polar coordinates, quantizes components without needing per-block normalization constants. This eliminates the 1–2 bit overhead that traditional quantization methods carry.
QJL (Quantized Johnson-Lindenstrauss) — encodes residual error with a single sign bit. Zero memory overhead.

Result: 3-bit KV-cache, 6x compression, 8x speedup, zero accuracy degradation on LongBench, Needle-in-a-Haystack, RULER, and ZeroSCROLLS benchmarks.

This makes long-context inference significantly cheaper and faster. Real value.

The gap it leaves open

The moment the session ends — the KV-cache is gone.

Week 1 with any model: average answers.
Week 4 with any model: still average answers. It forgot everything.

Fine-tuning costs thousands of dollars and weeks. RAG gives you retrieval, not cognition. Context windows bill per token and still reset.

What we built for Problem B

AuraSDK is a local cognitive substrate that sits outside model weights.

It accumulates structured experience across sessions through a 5-layer pipeline:

Record → Belief → Concept → Causal → Policy

Each layer is derived deterministically from the one below — no LLM, no embeddings. Policy hints like "deploy to staging first" aren't written by anyone. They emerge from repeated causal patterns in stored experience.

from aura import Aura

brain = Aura("./agent_memory")

brain.store("Staging deploy prevented 3 production incidents", tags=["workflow"])
brain.store("User always deploys to staging first", tags=["workflow"])

# after run_maintenance(), the cognitive stack derives:
hints = brain.get_surfaced_policy_hints()
# → [{"action": "Prefer", "domain": "workflow", "description": "deploy to staging first"}]

What v1.5.4 adds:

Autonomous cognitive plasticity — the substrate observes model output and updates itself. No fine-tuning. Full audit trail.
Salience weighting — what matters persists longer, decays slower
Contradiction governance — conflicting evidence surfaced explicitly, not averaged silently

Performance (1,000 records, Ryzen 7, v1.5.4):

Store: 0.91ms
Recall: 0.076ms (~2,600× faster than Mem0)
Recall (cached): 1.4µs
Maintenance cycle: 15ms median

No API keys. No cloud. No LLM dependency. ~3MB binary. Fully offline. MIT license.

The full picture

	TurboQuant	AuraSDK
Problem	KV-cache overhead within session	No memory between sessions
Approach	Quantization of attention keys/values	Persistent cognitive substrate
Scope	Single inference pass	Cross-session accumulation
Requires LLM	Yes (runs inside it)	No
Works offline	N/A	Fully
Open source	Research paper	MIT, ships today

These are complementary. TurboQuant makes inference cheaper in the moment. AuraSDK makes the model smarter over time.

The field needs both.

pip install aura-memory

GitHub ·

Your AI forgets everything — this layer fixes that without retraining

Oleksander — Tue, 24 Mar 2026 16:19:10 +0000

Your AI model forgets everything after every conversation.

Not because it’s bad — because it has no memory system.

RAG helps retrieve context.
Fine-tuning helps adjust behavior.

But neither actually gives your system memory.

This article shows a different approach:
a cognitive layer that sits outside the model
and gets smarter over time — while the model stays frozen.

What the cognitive layer actually does

AuraSDK builds a 5-layer structure from whatever the model and users store:

Record → Belief → Concept → Causal → Policy

Each layer is derived from the one below — without LLM, without embeddings, locally:

Record: raw stored fact with trust score, provenance, decay rate
Belief: competing hypotheses about the same claim, epistemically weighted
Concept: stable abstractions over repeated beliefs
Causal: learned cause→effect patterns from co-occurring evidence
Policy: advisory hints — Prefer, Avoid, Warn — that emerge from causal structure

Nothing in layers 2–5 is hand-authored. They emerge from what's stored and observed over time.

from aura import Aura, Level

brain = Aura("./memory")
brain.enable_full_cognitive_stack()

brain.store("Staging deploy prevented 3 production incidents", tags=["deploy"])
brain.store("Direct prod deploy caused outage in Q3", tags=["deploy"])

# After maintenance:
hints = brain.get_surfaced_policy_hints()
# → [{"action": "Prefer", "domain": "deploy", "description": "staging before production"}]

That policy hint was not written by anyone. The causal layer found the pattern. The policy layer surfaced it.

v1.5.4: the three things that were missing

1. The substrate now learns from the model's own output

Before v1.5.4, the cognitive layer only knew what you explicitly stored. Now it observes model responses and updates itself.

Claims are extracted. Confirmations strengthen existing beliefs. Contradictions raise volatility. The substrate evolves from inference — without retraining, without an external API.

capture = brain.capture_experience(
    prompt="How should we handle this deploy?",
    retrieved_context=context_ids,
    model_response="Always verify staging health checks before pushing to production.",
    source="model_inference",
)
brain.ingest_experience_batch([capture])
brain.run_maintenance()
# cognitive layer updated — next recall is different

Safety bounds (non-negotiable):

Generated claims capped at 0.70 confidence — cannot overwrite recorded facts
PlasticityMode::Off by default — nothing changes without explicit opt-in
Every mutation writes to an audit trail traceable to the prompt that caused it
purge_inference_records() — clean rollback when needed
freeze_namespace_plasticity("medical") — some domains must never adapt from inference

Recorded facts always win over model inference. Always.

2. The substrate now knows what matters

High-frequency recall and high-significance are not the same thing. A trivial fact mentioned 20 times should not outrank a critical decision mentioned once.

v1.5.4 adds salience weighting:

brain.mark_record_salience(record_id, salience=0.9)
# → this record resists decay, ranks higher, gets preserved longer

Maintenance now also produces bounded reflection summaries: recurring blockers, unresolved tensions, patterns that keep appearing. Not "feelings" — structured synthesis from what's actually stored.

3. Contradictions are now first-class, not silently averaged

Before: conflicting evidence was weighted and averaged. The conflict was invisible.

Now:

clusters = brain.get_contradiction_clusters()
queue = brain.get_contradiction_review_queue()

Recall explanations carry explicit markers: "this recommendation depends on unresolved evidence." The operator sees the friction. The user can be told honestly.

What else ships in v1.5.4

Concept persistence — concepts used to reset on every restart. Now they survive. The 5-layer stack is actually intact across sessions.

Belief reranking active by default — in v1.5.3, BeliefRerankMode::Off was the default. The cognitive stack was engineered but not running. Now it runs.

Production integrity — startup validation, persistence manifest, concept partition cap for large corpora.

Explainability is built in

Every recall decision is traceable:

explanation = brain.explain_recall("deployment decision")
# → which records matched, why, what belief groups they belong to,
#   what salience contributed, whether unresolved evidence is present

chain = brain.provenance_chain(record_id)
# → full trace from policy hint back to source records

This is not logging. It is structural explainability derived from the cognitive layer itself.

Performance

Benchmarked on 1,000 records, Windows 10 / Ryzen 7:

Operation	Latency	vs Mem0
Store	0.09 ms	~same
Recall	0.74 ms	~270× faster
Recall (cached)	0.48 µs	~400,000× faster
Maintenance	1.1 ms	no equivalent

Mem0 recall requires an embedding API call (~200ms+). AuraSDK recall is pure local computation. No embeddings required. No external service.

The positioning in one sentence

AuraSDK is not a vector database. Not a RAG wrapper. Not a fine-tuning platform. Not a generic agent framework.

It is a governable cognitive substrate for frozen AI models — the layer that makes them smarter, more consistent, and more explainable over time, without touching their weights.

Try it

pip install aura-memory

Try in browser (no install): Open in Colab

GitHub: teolex2020/AuraSDK — MIT license, patent pending (US 63/969,703)

Built in Kyiv, Ukraine 🇺🇦

What would you build with a model that actually accumulates structured experience over time?

I tested the same AI model against itself. Memory won 4/5.

Oleksander — Wed, 18 Mar 2026 13:23:16 +0000

The experiment

Same model. Same 5 questions. One difference: one side had persistent memory via AuraSDK, the other had none.

Both sides used Gemini 2.5 Flash-Lite — identical model, identical cost per token.

Result: 4/5 questions won by the side with memory. 48% fewer tokens used.

What the questions tested

Real Python dev scenarios — the kind where generic answers aren't enough:

"I'm writing a new async function to fetch user orders from the DB. What patterns should I follow?"
"We need background email sending when an order is completed. How should we implement it?"
"I'm writing tests for the new payment module. Walk me through the setup."
"Code review: results = list(db.query(Order).filter(...)) — what's wrong?"
"The reports endpoint is slow — 3 seconds for 200 users. How do I debug and fix this?"

These questions have correct generic answers. But they also have specific answers — if you know the team's stack.

What the model with memory knew

AuraSDK had loaded 15 records of accumulated team knowledge:

We use arq for task queues, not Celery — switched because Celery was too heavy
Production incident: missing await on async DB call silently returned a coroutine. 2 hours to debug.
SQLAlchemy lazy loading doesn't work with async sessions — always use selectinload() or joinedload()
Code review caught: someone did list(queryset) and loaded 50k rows into memory. Lint rule added.
N+1 queries found in the user reports endpoint — fixed with a single joined query
Test setup: pytest + pytest-asyncio, transactions that rollback after each test, coverage must stay above 85%

What happened question by question

Q2 — Background email sending:

With memory (1,307 tokens):

"Use arq — we switched from Celery because it was too heavy for our scale. Define an arq job decorated with @arq.func, enqueue it when order completes."

Without memory (3,203 tokens):

"Here are several approaches: FastAPI BackgroundTasks for simple cases, Celery for robust production use..."

Same model. One knew the decision. One gave a textbook overview.

Q4 — Code review:

With memory (829 tokens):

"This is a direct violation of our established practice. We had an incident where this loaded 50,000 rows into memory. A lint rule was added specifically to prevent this."

Without memory (1,604 tokens):

"This is generally considered an anti-pattern in SQLAlchemy. Here's a breakdown of what's wrong..."

How AuraSDK works

from aura import Aura, Level

brain = Aura("./agent_memory")
brain.enable_full_cognitive_stack()

# store team knowledge
brain.store("We use arq, not Celery — switched because Celery was too heavy",
            level=Level.Domain, tags=["python", "dev"])
brain.store("Production incident: list(queryset) loaded 50k rows into memory",
            level=Level.Decisions, tags=["python", "lesson-learned"])

# recall before answering — <1ms, no API call
context = brain.recall("background email sending", token_budget=1000)

# inject into prompt
system = f"TEAM CONTEXT:\n{context}\n\nAnswer using this context."

No embeddings. No vector database. No LLM calls during learning. Pure local Rust computation.

The cognitive pipeline

AuraSDK doesn't just store and retrieve text. Every record goes through 5 layers:

Record → Belief → Concept → Causal → Policy

Belief: groups related observations, resolves contradictions with confidence scores
Concept: discovers stable topic clusters across beliefs
Causal: finds cause-effect patterns from temporal and explicit links
Policy: derives behavioral hints (Prefer / Avoid / Warn) from causal patterns

After enough interactions, the system surfaces this automatically:

hints = brain.get_surfaced_policy_hints()
# [{"action": "Prefer", "domain": "dev", "description": "use arq over celery for task queues"}]

Nobody wrote that rule. The system derived it from the pattern of stored observations.

The token math

	With memory	Without memory
Q1	1,200 tokens	1,545 tokens
Q2	1,307 tokens	3,203 tokens
Q3	1,923 tokens	4,067 tokens
Q4	829 tokens	1,604 tokens
Q5	1,294 tokens	2,155 tokens
Total	6,553 tokens	12,574 tokens

48% fewer tokens. The memory layer doesn't add bloat — it gives the model exactly what it needs.

How it compares

	AuraSDK	Mem0	Zep	Letta
LLM required for learning	No	Yes	Yes	Yes
Works offline	Fully	Partial	No	With local LLM
Recall latency	<1ms	~200ms+	~200ms	LLM-bound
Self-derives behavioral policies	Yes	No	No	No
Binary size	~3MB	~50MB+	Cloud	Python pkg

Try it

pip install aura-memory
python examples/demo.py

Open source: github.com/teolex2020/AuraSDK
Patent pending: US 63/969,703
Built in Kyiv, Ukraine.

I built a cognitive layer for AI agents that learns without LLM calls

Oleksander — Tue, 17 Mar 2026 12:19:42 +0000

The problem

Every time your agent starts a conversation, it starts from zero.

Sure, you can stuff a summary into the system prompt. You can use RAG. You can call Mem0 or Zep.

But all of these have the same problem: they need LLM calls to learn. To extract facts, to build a user profile, to understand what matters — you're paying per token, adding latency, and depending on a cloud service.

What if the learning happened locally, automatically, without any LLM involvement?

What AuraSDK does differently

AuraSDK is a cognitive layer that runs alongside any LLM. It observes interactions and — without any LLM calls — builds up a structured understanding of patterns, causes, and behavioral rules.

from aura import Aura, Level

brain = Aura("./agent_memory")
brain.enable_full_cognitive_stack()

# store what happens
brain.store("User always deploys to staging first", level=Level.Domain, tags=["workflow"])
brain.store("Staging deploy prevented 3 production incidents", level=Level.Domain, tags=["workflow"])

# sub-millisecond recall — inject into any LLM prompt
context = brain.recall("deployment decision")

# after enough interactions, the system derives this on its own:
hints = brain.get_surfaced_policy_hints()
# [{"action": "Prefer", "domain": "workflow", "description": "deploy to staging first"}]

Nobody wrote that policy rule. The system derived it from the pattern of stored observations.

The cognitive pipeline

AuraSDK processes every stored record through 5 layers:

Record → Belief → Concept → Causal → Policy

Each layer is bounded and deterministic:

Belief: groups related observations, resolves contradictions
Concept: discovers stable topic clusters across beliefs
Causal: finds cause-effect patterns from temporal and explicit links
Policy: derives behavioral hints (Prefer / Avoid / Warn) from causal patterns

The entire pipeline runs in milliseconds. No LLM. No cloud. No embeddings required.

Try it in 60 seconds

pip install aura-memory
python examples/demo.py

Output:

Phase 4 - Recall in action

  Query: "deployment decision"  [0.29ms]
    1. Staging deploy prevented database migration failure
    2. Direct prod deploy skipped staging -- caused data loss

  Query: "code review"  [0.18ms]
    1. Code review caught SQL injection before merge
    2. Code review found performance regression early

5 learning cycles completed in 16ms. Recall at 0.29ms.

How it compares

	AuraSDK	Mem0	Zep	Letta
LLM required for learning	No	Yes	Yes	Yes
Works offline	Fully	Partial	No	With local LLM
Recall latency	<1ms	~200ms+	~200ms	LLM-bound
Self-derives behavioral policies	Yes	No	No	No
Binary size	~3MB	~50MB+	Cloud	Python pkg

What's new in v1.5.3

Full 5-layer cognitive pipeline active by default
enable_full_cognitive_stack() — one call to activate everything
Decay now driven by memory level, not manual type labels
Policy hints now work with explicit causal links (link_records())
demo.py — see it working in 60 seconds

Built in Rust, from Kyiv

Pure Rust core. No Python dependencies for the engine. Patent pending (US 63/969,703).

Open source: github.com/teolex2020/AuraSDK
Install: pip install aura-memory
Web: aurasdk.dev

If you're building AI agents and want deterministic, explainable, offline-capable memory — give it a try and tell me what you think.

10x Faster Recall + Memory That Evolves: Aura v1.3 for AI Agents

Oleksander — Thu, 05 Mar 2026 09:42:48 +0000

The Problem

Every AI agent framework has the same weakness: memory is an afterthought. Most solutions dump everything into a vector database and hope cosine similarity finds the right context. This works until it doesn't — when your agent needs to know when it learned something, what changed since last week, or which memories are actually useful.

What Aura Does Differently

Aura is a pure-Rust cognitive memory engine. Instead of embeddings + vector search, it uses:

SDR Encoding (Sparse Distributed Representations) — biologically-inspired, noise-tolerant
RRF Fusion — 4 parallel ranking signals (SDR similarity, MinHash, Tag Jaccard, optional embeddings)
Temporal Decay — memories naturally fade unless reinforced
Graph Connections — associative, causal, and co-activation links between memories

The result: sub-millisecond recall, ~3MB binary, zero external dependencies.

10x Recall Speedup (v1.3.1)

Every recall_structured call was cloning ALL records into a new HashMap to filter by namespace. At 10K records, that's 94ms of pure waste.

Fix: pass the original HashMap through the pipeline. Each signal collector filters by namespace inline with a cheap contains() check. Plus a new StructuredRecallCache for repeated queries.

Records	Before	After	Speedup
1K	15 ms	2.6 ms	5.8x
5K	58 ms	5.1 ms	11.4x
10K	94 ms	8.6 ms	10.9x

Warm recall (cache hit): ~0.07 ms — constant time regardless of record count.

What's New in v1.3.0

1. Temporal Queries

from aura import Aura
import time

brain = Aura("./memory")
brain.store("User prefers dark mode", level="Domain")

# ... days pass, user changes preference ...
brain.supersede(old_id, "User prefers light mode")

# What did we know last week?
old_memories = brain.recall_at("user preferences", last_week_timestamp)

recall_at(query, timestamp) filters records by creation time. history(record_id) shows the full access/strength timeline. This is how you debug agent behavior — "why did it do X on Tuesday?"

2. LangChain Drop-In

from aura import Aura
from aura.langchain import AuraMemory

brain = Aura("./memory")
memory = AuraMemory(brain)

# Works with any LangChain chain
chain = ConversationChain(llm=llm, memory=memory)

AuraChatMessageHistory implements the full BaseChatMessageHistory interface. AuraMemory is duck-type compatible with ConversationBufferMemory. No changes to your existing code.

3. Adaptive Recall

# After recall, tell Aura what was useful
results = brain.recall_structured("deployment steps", top_k=5)
for score, record in results:
    if was_helpful(record):
        brain.feedback(record.id, useful=True)   # +0.1 strength
    else:
        brain.feedback(record.id, useful=False)  # -0.15 strength

Over time, noise naturally decays while valuable memories get reinforced. No other memory SDK has this built-in.

4. Memory Versioning

# Save state before experiment
brain.snapshot("before_refactor")

# ... agent does things ...

# Something went wrong? Roll back
brain.rollback("before_refactor")

# Or compare states
diff = brain.diff("before_refactor", "after_refactor")
print(f"Added: {diff['added']}, Removed: {diff['removed']}")

5. Agent-to-Agent Sharing

# Agent A exports relevant context
fragment = agent_a.export_context("user preferences", top_k=5)

# Agent B imports it (strength halved, tagged "shared")
agent_b.import_context(fragment)

The protocol envelope includes version and provenance metadata. Imported records arrive with reduced trust — they need to prove themselves.

6. C FFI — Aura as a Platform

#include "aura.h"

AuraHandle* h = aura_open("./memory");
aura_store(h, "Remember this", NULL, NULL);

char* result = aura_recall(h, "what to remember", 5);
printf("%s\n", result);
aura_free_string(result);
aura_close(h);

Working examples in Go and C#. Any language with C FFI can use Aura.

7. OpenTelemetry

[features]
telemetry = ["opentelemetry", "opentelemetry_sdk", "opentelemetry-otlp", "tracing-opentelemetry"]

17 key functions instrumented with #[instrument] spans. OTLP export to any collector. Grafana dashboard template included.

The Bug That Took 8 Hours

Fun story: our CI was timing out at 6+ hours. We tried increasing timeouts, switching to release builds, reducing the test matrix. Nothing worked.

Turns out: Aura struct didn't have a Drop implementation. When tests ended without calling close(), internal file handles wouldn't release. Each test hung for 5 minutes waiting for a timeout that never came. 28 tests x 5 min = CI death.

Fix: 9 lines of code.

impl Drop for Aura {
    fn drop(&mut self) {
        self.stop_background();
        let _ = self.flush();
        let _ = self.storage.flush();
        let _ = self.index.save();
    }
}

Now 503 tests pass in 7 minutes. Sometimes the hardest bugs are the simplest ones.

Try It

pip install aura-memory

from aura import Aura

brain = Aura("./my_agent_memory")
brain.store("User prefers concise answers", level="Identity")

context = brain.recall("how should I respond?")
# Returns formatted context for your LLM's system prompt

Star the repo if this is useful. PRs and issues welcome.

Agent memory in 5 lines of Python — no LLM, no cloud, <1ms recall

Oleksander — Mon, 02 Mar 2026 07:32:40 +0000

Last week, my AI agent analyzed 10 competitors in the AI memory market over 3 days. On day 4, I asked it to compare their pricing. It didn't search again — it already knew them all. That's what happens when your agent has real memory, not a chat history.

Your AI agent forgets everything between sessions. Every conversation starts from zero. Every user preference, every decision, every piece of context — gone. You paste old conversations into the system prompt, hit the token limit, and wonder why the agent feels so... stateless.

Most "memory" solutions bolt on a vector database, call an embedding API, and charge you per query. You now have 200ms latency, a cloud dependency, and a monthly bill — for what is essentially a fancy search index.

What if your agent could remember like a human? Important things stick. Trivial things fade. Trusted sources rank higher than random web scrapes. And it all happens in under 1 millisecond, locally, with zero LLM calls.

That's Aura.

What makes Aura different

	Aura	Others (Mem0, Zep, Cognee)
LLM required	No	Yes
Recall latency	<1ms	200ms+ / LLM-bound
Works offline	Yes	No
Binary size	2.7 MB	Heavy
Cost per op	$0	API billing
Source provenance	Built-in	No

Aura is a Rust-native cognitive memory engine with Python bindings. It uses a 4-signal RRF (Reciprocal Rank Fusion) recall system — no embeddings required — and models memory decay, consolidation, and trust scoring inspired by how human memory actually works.

Let's see how it works.

1. Install

pip install aura-memory

That's it. No Docker, no API keys, no cloud account. The entire engine ships as a single 2.7 MB binary.

2. Store & recall — the basics

from aura import Aura, Level

brain = Aura("./agent_memory")

# Store memories at different importance levels
brain.store("User prefers dark mode and Vim keybindings",
            level=Level.Identity, tags=["preference", "ui"])

brain.store("Deploy staging before production, always run tests",
            level=Level.Decisions, tags=["workflow"])

brain.store("Fix login bug - users getting 403 on /api/auth",
            level=Level.Working, tags=["bug", "auth"])

# Recall — returns formatted context ready for LLM injection
context = brain.recall("authentication issues", token_budget=2000)
print(context)

Output:

=== COGNITIVE CONTEXT ===
[IDENTITY]
  - User prefers dark mode and Vim keybindings [preference, ui]

[DECISIONS]
  - Deploy staging before production, always run tests [workflow]

[WORKING]
  - Fix login bug - users getting 403 on /api/auth [bug, auth]

=== END CONTEXT ===

That's it. store() → recall() → inject into your system prompt. Five lines to give your agent persistent memory.

3. Memory levels — not all memories are equal

Aura organizes memory into 4 levels across 2 tiers, modeled after human cognitive architecture:

Tier	Level	Decay rate	Use case
Core	Identity	0.99/cycle	User preferences, personality
Core	Domain	0.95/cycle	Learned facts, domain knowledge
Cognitive	Decisions	0.90/cycle	Choices made, action items
Cognitive	Working	0.80/cycle	Current tasks, recent messages

Core tier = slow decay (weeks to months). Your agent's "personality" and knowledge base.
Cognitive tier = fast decay (hours to days). Ephemeral context that fades naturally.

This means your agent doesn't need explicit "forget" logic. Old tasks decay away. Core knowledge persists. Just like your brain.

# Query only recent, ephemeral memories
recent = brain.recall_cognitive("workflow")

# Query only long-term knowledge
knowledge = brain.recall_core_tier("programming")

4. Trust scoring — the killer feature

Here's where Aura gets interesting. Not all information sources are equally reliable. A user telling you their name is more trustworthy than a web scrape claiming "Python 4.0 is coming soon."

from aura import TrustConfig

tc = TrustConfig()
tc.source_trust = {"user": 1.0, "api": 0.8, "web_scrape": 0.5}
brain.set_trust_config(tc)

# Store from different sources
brain.store("Python 3.13 released October 2024",
            tags=["python"], channel="user")

brain.store("Python 4.0 coming soon",
            tags=["python"], channel="web_scrape")

# Trust-weighted recall ranks user-sourced memory higher
results = brain.recall_structured("python release", top_k=5)
for r in results:
    print(f"  score={r['score']:.3f}  trust={r['trust']:.2f}  {r['content']}")

Output:

  score=0.995  trust=1.00  Python 3.13 released October 2024
  score=0.589  trust=0.50  Python 4.0 coming soon

The user-sourced fact scores 0.995. The web scrape scores 0.589. Your agent now has built-in epistemological hygiene — it knows how much to trust each piece of information.

5. Source provenance — know where every fact came from

Every memory in Aura carries an epistemological tag:

recorded — direct user input (trust × 1.00)
retrieved — fetched from web/API (trust × 0.90)
inferred — LLM conclusion (trust × 0.85)
generated — agent-created (trust × 0.80)

brain.store("BTC at $67k on Feb 21",
            source_type="retrieved", tags=["crypto"])

brain.store("User prefers conservative trading strategy",
            source_type="recorded", tags=["crypto"])

# Recall ranks recorded higher than retrieved
results = brain.recall_structured("crypto strategy", top_k=5)

This prevents a subtle but dangerous problem: agents presenting web search results as their own "memories." With source_type, your agent always knows what it observed vs what it found vs what it guessed.

No other memory SDK tracks this. Not Mem0. Not Zep. Not Cognee.

6. Plug it into any LLM

Aura is LLM-agnostic. The pattern is always the same:

user_message = "What are my UI preferences?"

# 1. Recall relevant context
context = brain.recall(user_message, token_budget=2000)

# 2. Build system prompt
system_prompt = f"""You are a helpful assistant with memory.

{context}

Use the above context to personalize your responses."""

# 3. Send to your LLM of choice:
# Ollama:     requests.post("http://localhost:11434/api/chat", ...)
# OpenAI:     openai.chat.completions.create(messages=[...])
# LangChain:  ChatPromptTemplate with {context}
# Claude:     anthropic.messages.create(...)
# Any HTTP:   just inject system_prompt

No adapters. No framework lock-in. If your LLM takes a string, Aura works with it.

Bonus: structured recall with scores

When you need more than formatted text — for routing, filtering, or debugging:

results = brain.recall_structured("user preferences", top_k=5)
for r in results:
    print(f"  [{r['level']}] score={r['score']:.3f} -- {r['content'][:80]}")

  [IDENTITY] score=0.590 -- User prefers dark mode and Vim keybindings
  [WORKING]  score=0.586 -- Fix login bug - users getting 403 on /api/auth
  [DECISIONS] score=0.581 -- Deploy staging before production, always run tests

Each result includes level, score, trust, tags, timestamps, and source metadata. Use this to build intelligent routing: high-trust Identity memories go straight to the system prompt; low-trust Working memories get verified first.

Performance — sub-millisecond, for real

Benchmarked on a standard machine with 1,000 stored records:

Operation	Latency
Store	0.129 ms/op
Recall (1K records)	0.861 ms/op
Search by tag	0.103 ms/op

For comparison, embedding-based recall typically runs 200ms+ per call. Aura is 200x faster because it uses SDR (Sparse Distributed Representation) encoding + MinHash + tag matching — no neural network inference needed.

You can optionally add embeddings as a 4th signal if you want semantic similarity on top:

from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")
brain.set_embedding_fn(lambda text: model.encode(text).tolist())

But the 3-signal fusion works great without them.

Living memory — decay, reflect, consolidate

Run a single maintenance cycle and Aura handles the rest:

report = brain.run_maintenance()
print(f"Decayed: {report.decay.decayed}")
print(f"Promoted: {report.reflect.promoted}")
print(f"Consolidated: {report.consolidation.native_merged}")
print(f"Archived: {report.records_archived}")

Call this periodically (every N interactions, or on a schedule), and your agent's memory stays clean and relevant without manual curation.

More features you get out of the box

Namespace isolation — keep test/prod/per-user memories separate
Encryption — ChaCha20-Poly1305 + Argon2id, one argument: Aura("./data", password="secret")
MCP server — expose memory as a tool for Claude, GPT, or any MCP-compatible agent
Zero dependencies — pure Rust core, no runtime requirements

Try it now

The fastest way to try Aura is the interactive Colab notebook — zero setup, runs in your browser:

▶ Open in Google Colab

Or install locally:

pip install aura-memory

GitHub — star it if you find it useful ⭐
API docs — full reference for 40+ methods
Examples — Ollama integration, research bot, edge devices

Aura is MIT-licensed. Built by a solo developer in Kyiv, Ukraine — including during power outages. Patent pending (US 63/969,703). If you're building AI agents that need to remember, I'd love to hear what you think.

419 Clones in 48 Hours — What Happened When I Launched an SDK for Offline AI Agent Memory

Oleksander — Thu, 26 Feb 2026 13:05:19 +0000

48 hours after launch. 419 clones. 90 unique developers. 8 stars. Nobody said a word.

That silence told me something important: engineers don't star things — they test them.

Here's the story of what I built, why, and what those numbers actually mean.

The Problem Nobody Talks About

Everyone is building AI agents. Most of them have a memory problem.

The standard approach: use embeddings. Store text as vectors, query them at recall time. Tools like Mem0, Zep, and LangMem all work this way.

The hidden cost:

Every recall = an embedding API call = 150–300ms latency
Every embedding call = money (OpenAI charges per token)
Offline deployment? Impossible — you need the embedding API available

For cloud-based chatbots this is fine. But for local AI agents running on your own hardware — especially with Ollama — this breaks the whole offline-first promise.

If your agent needs to "remember" something, it has to call home first.

That felt wrong to me.

A Different Idea: SDR Instead of Embeddings

I started reading about Sparse Distributed Representations (SDR) — the pattern encoding mechanism used in Hierarchical Temporal Memory (HTM) theory, originally inspired by how the neocortex works.

The core idea: represent any concept as a sparse binary vector (256K bits in Aura's case) where only ~2% of bits are active. Similarity between patterns is computed using Tanimoto coefficient — pure bit math, no neural network needed.

No embedding model. No API call. No GPU.

Just math.

Recall latency: 0.35ms. That's not a typo.

What I Built

Aura — a cognitive memory system for AI agents written in Rust.

Key properties:

Sub-millisecond recall — 0.35ms average, 0.29ms after warm cache
Zero LLM calls for memory operations — the recall itself needs no model
2.7MB binary — the entire memory engine fits in a small file
Fully offline — works with Ollama, any local model, no internet required
Persistent across sessions — brain reloads from disk, all context intact
217 tests, ChaCha20-Poly1305 encryption, patent pending (US 63/969,703)

Four memory levels with different retention weights:

Working Memory    → 0.80 retention  (temporary context)
Decision Memory   → 0.90 retention  (choices made)
Domain Memory     → 0.95 retention  (learned knowledge)
Identity Memory   → 0.99 retention  (core facts)

Integration with Ollama: 3 Lines

from aura_memory import Aura

brain = Aura("./agent_brain")
context = brain.recall(user_input, token_budget=1500)

# inject context into your Ollama system prompt
response = ollama.chat(
    model="gemma3n:e4b",
    messages=[
        {"role": "system", "content": f"Context:\n{context}\n\nYou are a helpful assistant."},
        {"role": "user", "content": user_input}
    ]
)

# store the interaction
brain.store(user_input, response["message"]["content"])

That's it. Your Ollama agent now has persistent memory across sessions — no embedding API, no cloud, no ongoing cost.

Live Demo Output

I ran a 4-phase test with gemma3n:e4b locally. Here's the actual terminal output:

Phase 1: Storing facts
✓ Stored: Name is Aleksander, AI engineer from Ukraine
✓ Stored: Working on AuraSDK — cognitive memory for agents
✓ Stored: Prefers concise technical explanations

Phase 2: Conversations with memory context
[Recall: 0.35ms] Context injected into system prompt
[Recall: 0.48ms] Agent referenced previous preference correctly
[Recall: 0.41ms] Agent remembered project name without being told

Phase 3: Session reload (fresh Python instance)
Brain loaded from disk...
[Recall: 0.29ms] ALL context intact ✅

Total records: 12
Memory persisted: YES
LLM calls for memory: 0

The agent remembered my name, project, and communication preferences across a completely fresh Python instance — without a single LLM or embedding call.

Benchmark vs Embedding-based approach

Metric	Aura	Embedding-based approach
Recall latency	0.35ms	~200ms
Embedding API calls	0	Required
Offline capable	✅	❌
Binary size	2.7MB	N/A (cloud)
Cost per recall	$0	API pricing
Speedup	270x faster	baseline

Why Rust?

Three reasons:

Performance — sub-millisecond recall requires zero garbage collection overhead
Safety — memory systems that corrupt data are worse than no memory at all
Portability — 2.7MB binary runs anywhere: Raspberry Pi, edge devices, air-gapped servers

19,500 lines of Rust. 217 tests. Built during power outages in Kyiv 🇺🇦

The 419 Clones

After posting in the Ollama Discord and commenting on a few Twitter threads about agent memory, the GitHub traffic spiked:

419 clones in 48 hours
90 unique cloners
Zero comments

I think developers are quietly testing it. That's the most honest validation I could ask for — nobody clones a repo to be polite.

If you're one of those 90 people: I'd genuinely love to know what you found. What worked, what didn't, what you were trying to build.

Get Started

pip install aura-memory

📦 PyPI: aura-memory
🔗 GitHub: teolex2020/AuraSDK
🌐 Docs: aurasdk.dev

One Question For You

How are you handling memory in your AI agents right now?

Embeddings? Simple conversation history? Something else entirely?

I'm genuinely curious about the tradeoffs people are navigating — especially for local/offline deployments where latency and API costs actually matter.