In the last part I asked what your AI had remembered wrong. Sixteen of you answered with real cases, and I expected a pile of stale notes. Most of the cases taught me something else: the notes were true, and they still sent the agent the wrong way.
Your cases sorted themselves into three kinds. Each kind needs its own fix, which also explains why a single "freshness" score never worked for me.
1. True once, then replaced
This is where I started. My agent kept quoting "four to one" after I had measured "two to one".
@paullovelace had the same with a renamed API endpoint. The correction happened in chat, and for weeks the agent used the old name, because the correction never reached the memory file. Paul's case sits one step earlier than where I had been looking. Before a correction can win, it has to arrive in the store at all.
Then it has to win. Mine lets a new note name the one it replaces, so the old one stays readable but stops winning. When I counted, 2 of 735 notes carried that link. The mechanism works; agents just rarely use it.
@gimalay (Dmytro) does it differently, and I think that's right. In Dmytro's tool, a note marked as replaced can't be saved without naming its replacement, and sessions load only current notes. Asking nicely gave me 2 of 735, so I'll try it that way: the question becomes part of the write.
@opschainx goes further and deletes the old note outright, and named an honest gap in that, too: the same fact relearned in different words survives the delete. zhang jimmy and @salparvez landed in between, with in-place replacement and a "void" stamp. Side by side, it made sense to me: the old note should stay as history for a human and go quiet for the agent.
2. True for its case, used for another
@galadev had a case I hadn't thought about. Gala's agent keeps a file of corrections. One said never to link a certain subdomain, because that app has its own domain. Eight days later the agent applied the rule to a different site that has no domain of its own. The note was right. It had lost the case it came from.
A replace link can't help, because nothing replaced the note. What helped was writing the scope into the note: which site, which service, which environment. Gala found it by asking the agent where the rule came from, and the pointer led to the wrong site. I'll do that too. A rule that can say where it came from can also say where it stops.
3. True for its data, read without it
@izgorodin (Edward) pointed out that my own two numbers were both honest. "Four to one" and "two to one" were each true of the data they were computed on; the second just rested on more. I think Edward is right: stale facts need a replacement link, stale measurements mostly need their denominator.
So a number now goes into memory with its window and its count next to it: "two to one, N articles, June to August". The agent can then see that a new reading is due once the site has grown, instead of choosing between two equally sure figures.
Two lessons that cut across all three
@kitwisestudio described the workaround for a bug that has since been fixed. It never looks stale, and it keeps getting written back into new code. I checked my own store: it can attach a version range to a note, and recall pushes the note down once the project is past the range. It only helps when someone writes the range at the time, the same weak spot as the link.
@maame-codes, @quantumtechcoder and @halfcycle arrived at the same rule from three directions: for anything that can change, read the live source first. Ports, IDs, versions, config. QuantumTechCoder's migrations ran cleanly against a dead database for 25 minutes; Halfcycle's agents kept hedging about features that had shipped. Memory brings the context; the file on disk brings the value.
What I'm changing
- The "does this replace an older note?" question becomes part of the write, not something the agent may skip.
- Notes get room for their scope, so a rule knows where it stops.
- Numbers get their window and their count.
- Corrections given in chat get caught and offered for storage, so they reach the store at all.
I'll report the replace-link count again in a few weeks. 2 of 735 is the baseline.
Thank you also to Ayush Dhanani, OneLifeVibe, Kevin Hall, Mudassir Khan and everyone else who added a case. Every case helped.
Which of the three have you run into most: the replaced note, the note used outside its case, or the number without its denominator?
I build cachly. It keeps what you explain to your AI: your rules, your decisions and the bugs you already solved. Claude Code, Cursor, Copilot and the others read it in every session, for you alone or for your whole team, and every lesson carries the name of whoever learned it.
Try it, in increasing order of commitment:
-
30 seconds, no account:
npx @cachly-dev/mcp-server@latest demoin any git repo. It reads your log locally and prints what an assistant would already know about the project. After npx fetches the package, the command makes no network calls. -
Claude Code plugin:
/plugin marketplace add cachly-dev/cachly-mcp, then/plugin install cachly-brain@cachly. -
5 minutes, free tier:
npx @cachly-dev/mcp-server@latest autopilotwrites the MCP configuration for whichever assistant you use. - Or from the web: cachly.dev · free tier, servers in Germany.
On Product Hunt from 1 December: cachly on Product Hunt. Follow it there if you want a note on launch day.
Top comments (29)
To your closing question: the second kind — true for its case, used for another — is the one we design against most. HiveMind (the offline Obsidian plugin I build) does its housekeeping sweep as a dry run capped at 50 notes: one review list, nothing written until the user confirms, and it never deletes or moves a note on its own. Those limits exist precisely because a locally-true rule applied vault-wide stops being true fast — the scope has to travel with the action, not just the note. Your "2 of 735" replace-link count doesn't surprise me either; anything that depends on the writer remembering an optional step will round to zero. Making the replacement question part of the write, like Dmytro's tool does, is the only version of that I've seen that would actually move the number.
Gala's case was exactly that. The rule about the subdomain was right for the one site it came from, nothing in the note said which site, and eight days later it got applied to another one. "The scope has to travel with the action" puts it better than I did in the post.
The 50-note cap on the dry run works from the other end: if a rule is wrong, it is wrong for fifty notes, and a person sees all fifty before anything is written.
The eight-days-later part is what makes Gala's case stick for me — the rule didn't fail loudly, it just quietly outlived the context it was true in. That's the failure we kept hitting building HiveMind's memory: a stored note that doesn't carry where it came from reads as universal by default. The 50-note cap is the right shape of brake for the other direction — bounded blast radius, and a human sees the whole batch before anything lands. Is the cap a fixed fifty regardless of the rule, or does a rule with a narrower stated scope get a smaller dry run?
The fifty is yours, from HiveMind's sweep. I only borrowed it in my reply.
If I built it, I'd let the scope set the size. A rule that names one site only ever reaches that site's notes, so its dry run is small by construction. The fixed number then stays as a backstop for rules that name no scope at all. And those should get the smallest batch, not the largest, because a rule without a stated scope is exactly how Gala's case happened.
Fair correction — the fifty was mine, from HiveMind's sweep, and you were borrowing it back at me. Your inversion is the sharper version: in HiveMind the cap is fixed at fifty regardless of scope, because the sweep is a dry run and nothing writes until a human confirms the whole list. But that makes the fixed number a blunt backstop, not a sized one. Letting the stated scope set the batch — one site's notes for a one-site rule — means the dry run is small by construction, and a rule that names no scope at all getting the smallest batch rather than the largest is exactly backwards from how most defaults would land, and exactly right for Gala's case. Would you go one step further for the unscoped ones — no dry run at all until a scope is named — or is a smallest-batch run still useful as a way to show the author what the rule would touch?
I'd keep the small run. When a save is blocked with "scope required", people type in whatever makes the error go away, often something too wide. A preview that shows three notes from a site they never meant tells them what the scope should be. So for a rule without a scope, the small batch is the question, with an example attached.
The preview-as-question framing is better than what I had — the small run stops being a weaker version of enforcement and becomes the thing that teaches the scope at the exact moment someone is tempted to type whatever clears the error. We hit the same shape with fail-closed gates generally: a hard block with no example attached gets routed around, a block that shows the consequence gets answered honestly. Three notes from a site they never meant is a much sharper teacher than a warning sentence. Does the preview show the notes the rule would actually touch first, or the nearest matches outside the intended scope?
The matches outside the intended scope, first. The person already has a picture of the notes they meant, so showing those mostly confirms what they expect. The other site is the surprise, and that is what makes someone narrow the scope instead of widening it to clear the error. I'd still put one match from inside next to them, so the comparison is on screen: this one you meant, these three you didn't.
To be clear, that's a sketch of how I'd build it. cachly doesn't show this preview today.
Outside-first with one inside anchor is the right layout — the surprise does the teaching, and the one match they did mean stops it reading as 'your rule matched nothing you wanted.' Appreciate the honesty that it's a sketch and cachly doesn't show it today; that distinction is exactly what makes the rest of the thread trustworthy. HiveMind's dry-run preview has the opposite problem right now: it lists what would be touched, in-scope only, so there is no surprise on screen to learn from — your layout is the fix I would copy. For the inside anchor, would you pick the nearest in-scope match or the most typical one?
The nearest one, meaning the in-scope match that looks most like the outside ones. A typical match sits in the middle of what they meant and says nothing about where the line runs. The nearest one sits right at the line, so the screen shows the boundary from both sides: this one belongs, these three look almost the same but don't. Sometimes the person then decides the nearest one doesn't belong either. That's a useful result too, because it means the scope they had in mind was narrower than the rule they wrote.
Nearest is the right pick, and the reason is the one you give: the anchor has to sit at the line, not in the middle of the territory. A typical match only confirms the rule works somewhere; the nearest one tests whether it works exactly where it could go wrong. That last outcome is the one I'd underline — when someone looks at the nearest in-scope match and decides it doesn't belong either, the preview has done something a warning never could: it showed them their written rule and their actual intent disagreeing, before anything was touched. HiveMind's dry-run preview lists only what would be touched, so it can't produce that moment yet. When the nearest match gets rejected like that, does the person usually narrow the rule on the spot, or do they abandon the rule and write a different one?
I don't have numbers for that. cachly doesn't show this preview, so I've never watched anyone react to it. My guess is that it depends on where the rule came from. A rule written after one bad incident probably gets dropped, because the rejected match shows the incident was too specific to generalize from. A rule distilled from several cases probably gets narrowed on the spot. But that's a guess, and it's the kind of thing I'd count before building around it.
The origin distinction is the part I'd trust even before the count exists — a rule written after one bad incident is really a note about that incident wearing a rule's clothes, so a rejected match exposing that isn't the preview failing, it's the preview doing the one job it has. We hit the same shape from the other side with our release gate: a check written after a single bad release kept failing closed on changes it was never meant to judge, and it only became useful once we rewrote it around the class of failure instead of the incident. Counting first is the right instinct — when you do count it, would you split the rejections by origin (incident-born vs distilled) from the start, or tally them together and split only if the total looks odd?
From the start. A total can look normal while the two groups move in opposite directions, and then nothing ever tells you to split. Today I made the reason a required field when someone marks a lesson as wrong, mostly so a split like this is still possible later.
From the start is the only version that survives — a total that looks normal while the two groups move opposite is worse than no total, because it tells you not to look. Making the reason a required field at the moment someone marks a lesson wrong is the right place to pay for it: the reason is still in their head then, and it's what makes a later split possible at all instead of archaeology. We learned the same ordering with our release gate's could-not-run log — if the reason isn't captured by the step that failed, the tally a month later is just a number nobody can divide. Is the reason a free-text field, or a small set of categories — free text preserves the story, but a category is what keeps the split countable when you tally it later?
"Three true things that each broke something" is a great finding. True but stale memory is worse than no memory, because the agent acts on it confidently. Expiry dates on remembered facts would help a lot.
A date alone made me hesitate, because some old facts stay true for years and a fixed expiry would throw them away. Under my last post, Edward suggested keeping the count a number was computed on and measuring again once the source has doubled. That ties the expiry to what actually changed, not to the calendar. Have you tried a date-based expiry on remembered facts, and did it remove things you still needed?
Tying expiry to the source changing rather than the calendar makes more sense. A date only tells you when something was learned, not whether the thing it described has moved. Re-checking when the underlying count doubles is a neat trigger.
Thanks for the careful write-up, Heinrich. The second kind is the one I run into most: a rule that was right for one repo and gets applied in another. I'd treat it the way you're treating the replace link. Make scope a required field on rule notes, so a rule that doesn't say which project or service it belongs to can't be saved, and load only the rules whose scope matches where the session is working. On the count, Edward's point holds: links over notes that replaced something is the number to watch, since most of the 735 could never carry one. When Gala's rule got applied to the other site, was there anything in the note the agent could have checked, or did the scope exist only in the chat it came from?
It was Gala's agent, so @galadev knows the details better than I do. From what Gala described, the trail back to the original case did exist: asking the agent where the rule came from is how Gala found the mix-up. But that only worked afterwards, when a human asked. Nothing made the agent check it before acting. Your required field plus loading by scope closes exactly that gap, because the agent never has to remember to check.
On the count, I'll take that over. Most of the 735 never replaced anything, so the next count goes over the notes that did.
The third, and in my case the caveat sat in the same note. My agent's research counted about 11.2M URLs in a site's sitemap and listed the indexed count under what it hadn't verified. The plan then sized the soft 404 bug at about 11.2M URLs, and the PR body called them indexed. When the agent read the indexing report, it showed 959,591 pages indexed and only 24 ever flagged as soft 404s.
Case 2 is the one we hit most: a decision recorded without its scope gets applied everywhere. "A rule that can say where it came from can also say where it stops" is the fix we landed on too. Have you tried making the agent name the note it acted on, so a wrong application shows up in review instead of eight days later?
Not as a rule yet. Each recalled note already comes back with its name, so the agent can say which one it used, but nothing makes it do that. Your version is cheaper than mine: the citation lands in the diff or the PR text, where someone reads it anyway. I'll try it as one line in the agent instructions first and count how often a wrong application gets caught there.
This discussion highlights that reliable AI memory is as much about governance as it is about retrieval. Agents need to understand where a rule applies, whether its source has changed, and when a human should approve an update. At IT Path Solutions, we see these controls as important when developing agentic workflows that operate across multiple projects and systems. Good memory design should prevent incorrect information from becoming an unintended operational rule.
Your 2 of 735 is the third case in your own list: a number without the denominator it needs. Most of the 735 notes replace nothing, so they cannot carry a link in the first place. The count that matters is the notes that do replace an older one. A hand check of 50 random notes gives a rough share, and if a tenth of them replace something, the baseline is 2 of about 74, not 2 of 735. New notes will keep growing 735 whether or not they replace anything, so when you report again in a few weeks, both counts side by side will show whether the write-time question works. The third kind is also the one I run into most.
Fair, and it is my own third case. 2 of 735 mixes notes that could never carry a link with the ones that should. I'll hand-check a random 50 for the share that replaces something and report both counts side by side next time, so the denominator is visible instead of implied.
Nice work! This one actually aims at solving the problem most of us faced for a long time with AI. Will try it! Also thanks for mentioning me in the post :D
Your D1 binding case was the clearest example of the replaced note, so it had to be in there.
If you try it: the quickest path is
npx @cachly-dev/mcp-server setup. If the sign-in step gets in your way, tell me here. That step is what I'm fixing this week.Some comments may only be visible to logged-in visitors. Sign in to view all comments.