Sokudo
Token Cache Hit
Same prefix. Less repeated work.
LLMs process input tokens sequentially. If the beginning of a new request exactly matches a previously processed prefix, cached attention state can be reused.
Cache miss
SystemDocsHistoryQuestion
Compute every input token
Cache hit
System ✓Docs ✓History ✓New question
Reuse prefix; compute only the suffix
It is a computation cache—not semantic memory, retrieval, or answer storage.