AI knowledge reuse: why the best AI answer may already exist

Every day, people ask AI systems questions that have already been answered well: by a colleague last week, by a community last month, or by the same person in a chat they can no longer find. Most AI products answer each one from scratch. A full model run produces the same answer again, and the earlier one — which someone may have checked, corrected and shared — stays buried in a chat history.

Knowledge reuse asks a different first question: does a good answer to this already exist? If it does, and it is safe to use, the most efficient AI request is the one that never runs a model at all.

The most efficient AI request is the one you don't generate

Generating an answer and retrieving one are different operations. Generation runs a language model on specialized accelerators, producing the answer one token at a time. Retrieval looks up a stored answer with database and search infrastructure.

Retrieval is not free. Storage, indexes, embeddings and the servers that run them all use energy, and a knowledge base nobody uses still costs something to keep. But when a sufficiently reliable existing answer can stand in for a new one, retrieving it should generally take far less computation than generating it again. For scale: published estimates for a single text query to a frontier model are on the order of a few tenths of a watt-hour¹².

The value also compounds. A hundred people asking the same timeless question can mean a hundred generations, or one generation and ninety-nine lookups.

Reuse is not RAG, and it is not a cache

Two familiar techniques sound similar but do something different.

Retrieval-augmented generation (RAG) retrieves relevant documents and adds them to the prompt, then generates³. It improves the answer, but a model still runs every time.

Semantic caching stores model responses and returns a stored response when a new prompt looks similar enough, judged by embedding similarity. GPTCache is a widely used open-source example, aimed at cutting cost and latency⁴. It avoids generation, but "similar enough" is a dangerous rule for answers people rely on.

Knowledge reuse sits between them. It avoids generation like a cache, but the stored answers are knowledge objects: deliberately published, attributed, dated, moderated and subject to rules about when they may stand in for a new answer.

Why similarity isn't sameness

The hard part of reuse is deciding when two questions are really the same question. Text embeddings are good at telling you two questions are about the same topic, and much less reliable at telling you they ask the same thing.

In EcoRouter's calibration of its matching thresholds, "hard yolk" versus "soft yolk" egg questions scored 0.975 on embedding similarity, and "minutes in a day" versus "seconds in a day" scored 0.960. Both scored higher than the weakest genuine paraphrase in the test set, at 0.935. A pure similarity threshold would have served the wrong answer.

So similarity has to be backed by guards. EcoRouter's design treats a close match as merely related, not equivalent, when one question contains a negation the other lacks, names different numbers, changes scope ("all" versus "which"), asks for a comparison or criticism the other doesn't, or swaps a single content word. The thresholds favor precision over recall: missing a genuine paraphrase costs one generation; serving the wrong answer costs trust.

Five conditions for safe reuse

Matching is only the start. A reused answer also has to be appropriate in time, context and provenance.

Freshness

Some questions have answers that change: weather, scores, prices, office holders, anything about "today" or "the latest." EcoRouter never reuses a saved answer for a volatile question, and never searches for one. For slower-changing current facts, a fresh, web-grounded saved answer may be offered with its saved date, but never handed over silently. Saved answers that are volatile or expired are never offered at all.

Context

The same words can mean different things mid-conversation. EcoRouter reuses an answer automatically only for a standalone question: the first turn, with nothing attached. Later in a conversation, a match is offered instead. Questions about an attached file are never matched against public knowledge.

Attribution

A reused answer is always labeled — "Reused from saved knowledge," or the name of the Channel it came from — and links back to its source. Knowledge from a Channel carries its origin and contributor; an item whose attribution cannot be read is dropped rather than shown bare. Answers that people imported from other tools are labeled "Imported knowledge" and are offered rather than served automatically.

Privacy and moderation

For public reuse, only knowledge that is published, public and current is eligible. Private entries, a company's workspace knowledge, and entries that have been flagged, hidden or removed are excluded, and everything is re-checked at display time, failing closed. Publishing is deliberate: from a private chat, nothing becomes public until someone reviews exactly what will be shared and confirms it. Matching has a privacy cost of its own: comparing meaning requires an embedding, and EcoRouter's privacy policy discloses that a question's text may be sent to an embedding provider for that purpose.

Choice

The person asking stays in control. When EcoRouter offers saved answers to a similar question, the options are "View answer" or "Generate a new answer," and generating is always one click away.

How EcoRouter puts it together

EcoRouter's methodology describes the path every question takes: check whether useful existing knowledge already meets it; if so, reuse it; if not, route the question to an appropriately capable model. In more detail, in personal chat, knowledge discovery works as a ladder:

1. Exact match to a published answer, for a standalone, timeless question: served directly, with no generation. If the saved answer depends on time-sensitive information, it is offered with its saved date instead of being served.
2. Strong match that is not exact: up to three saved answers are offered before generating, never substituted.
3. No adequate match: a new answer is generated and routed.
4. Related knowledge may be listed after an answer, for exploration.

What this means for teams' knowledge management

Inside organizations the pattern is familiar. Someone works out how to handle an unusual customer request or explain a policy, the answer lives in one person's chat history, and the next person asks the AI again — or interrupts a colleague.

Treating good AI answers as reusable knowledge changes that. With EcoRouter for teams, a company workspace keeps saved answers private to its members. A saved company answer is checked first and shown whole when it closely matches, and company knowledge never flows into public knowledge. Communities can do the same in the open with public Channels, and anyone can browse Public Knowledge.

Whatever tools you use, a few practices make AI answers reusable rather than merely archived:

• Save the timeless, not the volatile. Policies, explanations and how-tos reuse well; prices and status updates don't.
• Keep the source and the date. An answer without provenance is a rumor.
• Review before you share. A short human check is what turns an AI output into team knowledge.
• Let answers expire. Reuse should be easy to refuse when an answer is out of date.

Takeaway

The cheapest, fastest AI answer is often one that already exists. Reuse is only worth it when it is trustworthy: matched carefully, fresh enough, clearly attributed and private where it should be. Aim for maximum trustworthy reuse, not maximum reuse. The FAQ explains how EcoRouter thinks about the difference.

References

1. Google (Elsworth et al.), "Measuring the environmental impact of delivering AI at Google Scale" (2025). https://arxiv.org/abs/2508.15734 (opens in a new tab) ↩

2. Joule (Oviedo et al.), "Energy use of AI inference, efficiency pathways, and test-time scaling" (2026), article 102430. https://doi.org/10.1016/j.joule.2026.102430 (opens in a new tab) (preprint: https://arxiv.org/abs/2509.20241 (opens in a new tab)) ↩

3. NeurIPS (Lewis et al.), "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" (2020). https://arxiv.org/abs/2005.11401 (opens in a new tab) ↩

4. Zilliz, "GPTCache: Semantic cache for LLMs" (open-source project, 2023). https://github.com/zilliztech/GPTCache (opens in a new tab) ↩