Supermemory POC on unRaid
This project is an isolated evaluation service. It does not replace the production Hindsight service or change the default Hermes profile.
Fixed deployment choices
- Supermemory Local
server-v0.0.8, Linux x64 binary, verified by SHA-256. - One self-contained server; state, auth material, uploaded files, and model
cache live under
/mnt/user/appdata/supermemory-poc/data. - A dedicated address on Docker
br0; port6767is not published on the unRaid host address. - Native Gemini embeddings using
gemini-embedding-2at 768 dimensions. - Extraction, summarization, and other intelligent steps use Supermemory's native Gemini provider default. This cleanly removes the LiteLLM and OpenAI-compatible translation layer from the diagnostic path.
- Ingest concurrency remains 1 so the first POC emphasizes correctness and observable cost over throughput.
- Resource ceiling: 4 CPUs and 8 GiB RAM. Supermemory Local keeps its corpus in memory, so RSS must be watched as the sample grows.
Do not change the embedding model or dimensions in place. Use a fresh data directory and re-ingest when comparing a different embedding plan.
Before deployment
- Confirm the chosen
SUPERMEMORY_IPV4_ADDRESSis absent from both Arcane'sbr0attachments and the LAN neighbor/DHCP tables. - Add
SUPERMEMORY_IPV4_ADDRESSandGEMINI_API_KEYto this Project's Arcane environment. Do not put the key in Git, Compose, activity notes, or chat. LegacyOPENAI_*values may remain stored in Arcane for rollback, but this Compose file does not inject them into the container. - Confirm
/mnt/user/appdatahas room for the 298 MiB server binary, the multilingual model cache, data, and rollback copy. - Confirm no other build, image pull, backup, or migration is active on the unRaid Environment.
The supermemory-fetch init service downloads the exact release asset once,
checks its SHA-256, and stores it in the POC appdata directory. Subsequent
starts only verify the existing binary. It also copies the CA bundle from the
pinned fetch image so the slim runtime can download local embedding model
artifacts over verified HTTPS. Both OpenSSL-style clients and the bundled Bun
runtime are pointed at that bundle, which is also mounted at Debian's standard
CA path for embedding workers. TLS verification remains enabled. No custom
image build is required.
First boot and authentication
The server creates its client bearer token at:
/mnt/user/appdata/supermemory-poc/data-gemini-embedding-2-768-r2/api-key
Read it through an authorized unRaid/Arcane console without printing it into
logs. Save it only as SUPERMEMORY_API_KEY in the supermemory-lab Hermes
profile. The extraction LLM key and Supermemory client key are different.
The current candidate address is 192.168.50.13: it is not attached to an
Arcane-managed br0 container and did not answer the pre-deployment neighbor
or ICMP probes. Confirm it is outside the DHCP pool or reserve it before the
Project is started.
Expected Hermes profile file $HERMES_HOME/supermemory.json (also provided as
hermes-supermemory.json.example):
{
"base_url": "http://<dedicated-ip>:6767",
"container_tag": "hermes_supermemory_lab",
"auto_recall": true,
"auto_capture": true,
"max_recall_results": 10,
"profile_frequency": 50,
"capture_mode": "all",
"search_mode": "hybrid",
"api_timeout": 10.0
}
Run poc.py smoke before importing any sampled production material. It checks
the v3 document path plus the v4 search, profile, and conversation endpoints
used by Hermes.
Embedding plan and smoke gate
The earlier local Xenova/bge-m3 and LiteLLM-backed embedding experiments are
preserved under their old data directories because embedding plans cannot be
changed in place. This deployment uses the fresh
data-gemini-embedding-2-768-r2 directory. The r2 store rotates the
auto-generated Supermemory client key after the first native-provider
diagnostic exposed the original key in its startup log. The one-shot supermemory-smoke
service verifies the locked embedding plan, tracks the exact document ID to a
terminal state, observes legacy v3 search, gates on v4 document and Chinese
hybrid search, checks provenance metadata, and exercises the profile and
conversation endpoints. It writes a sentinel after success so ordinary
redeploys do not add duplicate smoke documents. Do not import sampled
Hindsight or document material until this service exits successfully.
On 2026-09-08, the previous remote plan was accepted and locked as
openai / google/gemini-embedding-2 / 1024; the LiteLLM embedding and chat
endpoints also passed independently. Supermemory listed a completed smoke
document, but both v4 and v3 searches returned no hit. The old smoke mixed
list-wide status checks and did not track the newly created document, so this
native Gemini run uses a new store and a stricter document-specific gate.
On 2026-09-09, native Gemini direct checks succeeded for both
gemini-embedding-2 at 768 dimensions and Supermemory's native LLM default.
The v4 documents and hybrid searches returned the expected Chinese marker
and provenance metadata, while legacy /v3/search returned no results. The
self-hosted server-v0.0.8 binary returned 404 for /v4/conversations, even
though the self-hosting overview describes that route as part of the full API.
Keep conversation ingest as a failing smoke gate: document RAG can proceed in
isolation, but Hermes auto-capture/session ingest is not validated on this
binary.
Evaluation guardrails
- Score raw Supermemory API retrieval separately from the Hermes answer. In
Hermes Agent v0.21.1, the provider reads result metadata but omits it from
automatic prefetch text and from the explicit search tool response. A raw
hit with correct
source/original_idfollowed by an uncited Hermes answer is an adapter attribution gap, not a retrieval miss. - Do not run the fixed scored question set with automatic writes enabled. The
default Hindsight profile has
auto_retain: true, and this lab profile hasauto_capture: true; earlier test answers could leak into later questions. Use a read-only Hindsight snapshot/profile and set Supermemoryauto_capture: falsefor the retrieval benchmark. Re-enable capture only for the separately scored session-experience phase. - Hermes built-in
MEMORY.mdandUSER.mdremain active independently of the external provider. Record their presence and size for each arm so built-in context is not credited to Hindsight or Supermemory. - Self-hosted Supermemory has no managed Google Drive, Gmail, Notion, or OneDrive connectors. This POC evaluates explicit file/document ingestion, not hosted connector sync.
Staged data flow
poc.py sample-hindsightreads active observations from the supported Hindsight 0.8.4 list API and writes a local JSONL artifact. It never mutates the source bank.- Manually review that artifact for scope and sensitive content.
poc.py ingest-jsonlwrites the approved rows serially to the isolated Supermemory container using stable custom IDs and provenance metadata.poc.py ingest-filesuploads a small, separately reviewed DEVONthink/Obsidian manifest. Keep the original UUID/path in metadata and treat the source system as authoritative. The default per-file cap is 50 MiB so an accidental library-wide import fails closed.- Run the same questions against the default and
supermemory-labprofiles; record groundedness, source traceability, latency, tokens, and cost.
Artifacts are ignored by Git. Do not put exported observations or document content in this configuration repository.
Hermes sanitizes container tags by replacing hyphens with underscores. Keep
the canonical hermes_supermemory_lab spelling in direct API imports so they
land in the same container queried by the profile.
Rollback
Stop/down only this Project. Keep /mnt/user/appdata/supermemory-poc for later
inspection, or archive it before any separately approved deletion. The default
Hermes profile and Hindsight service require no rollback because this project
does not modify them.