Direct recall
Can the system retrieve the stored personal fact when the query explicitly asks for it?
Paper results · 125 tasks
InMind
Leaderboard.
A single view of every backbone and query-time memory configuration reported in the paper's main results table. Rankings use end-to-end Application accuracy.
Reference control
Backbone control (in-context)
The target memory is already visible; no retriever or embedding is used.
Target recall100.0%
Application84.0%
Query-time configurations
Leaderboard
Results are ordered by end-to-end Application. Higher is better for every metric, tied scores share a rank, and evidence badges link to the public paper and code where available.
| Rank | Memory system (embedding) | Direct recallNaive query | Target recallIndirect context | ApplicationEnd-to-end |
|---|---|---|---|---|
| 01 | pd-memory/combined (pplx-embed-v1-0.6b)Memory system | 93.6% | 84.8% | 70.4% |
| 02 | pd-memory/agentic (pplx-embed-v1-0.6b)Memory system | 87.2% | 80.8% | 69.6% |
| 03 | Naive RAG (text-embedding-3-large)Retrieval control | 97.6% | 6.4% | 16.0% |
| 04 | MemoryOS (text-embedding-3-large)Memory system | 96.8% | 7.2% | 14.4% |
| 05 | Naive RAG (MiniLM)Retrieval control | 92.0% | 0.8% | 9.6% |
| 05 | Naive RAG (BM25)Retrieval control | 53.6% | 3.2% | 9.6% |
| 05 | A-Mem (text-embedding-3-large)Memory system | 100.0% | 12.0% | 9.6% |
| 08 | HippoRAG 2 (MiniLM)Memory system | 93.6% | 0.8% | 8.8% |
| 09 | MemoryOS (MiniLM)Memory system | 87.2% | 2.4% | 8.0% |
| 10 | A-RAG (text-embedding-3-large)Memory system | 93.6% | 11.2% | 7.2% |
| 11 | xMemory (text-embedding-3-large)Memory system | 91.2% | 6.4% | 6.4% |
| 11 | Mem0 (text-embedding-3-large)Memory system | 76.8% | 6.4% | 6.4% |
| 11 | A-Mem (MiniLM)Memory system | 99.2% | 2.4% | 6.4% |
| 14 | HippoRAG 2 (text-embedding-3-large)Memory system | 96.0% | 2.4% | 5.6% |
| 15 | A-RAG (MiniLM)Memory system | 97.6% | 5.6% | 4.8% |
| 15 | xMemory (MiniLM)Memory system | 84.8% | 3.2% | 4.8% |
| 17 | Mem0 (MiniLM)Memory system | 76.0% | 2.4% | 3.2% |
Target recall
Does the decisive fact reach GPT-5-mini's context for the indirect query?
Application
Does the final answer correctly apply the personal fact through the required world-knowledge bridge?
MiniLM denotes all-MiniLM-L6-v2 (384 dimensions); text-embedding-3-large uses 3,072 dimensions. pd-memory uses Perplexity pplx-embed-v1-0.6b embeddings. BM25 is lexical retrieval. Scores are percentages on the same 125-task evaluation set.
Submitted answers were independently re-judged.
The InMind team re-judged the submitted contexts and answers with the official prompts, routing GPT-5-mini through Zhizengzeng. Memory construction and answer generation were not rerun, so this cross-provider check is not a protocol-identical rerun. These are the team-verified re-judging scores; differences from the submitter's report may reflect LLM judge variability.
Unverified scores are reported, not guaranteed.
For entries without an InMind team verification badge, we cannot independently confirm the authenticity of the results. Those scores come from the submitting author team's report and are provided for reference only.
Need a spot on the leaderboard?
Contact the InMind team with your system, configuration, and reproducible result package.
imlrz@mail.ustc.edu.cn