Supermemory POC on unRaid

This project is an isolated evaluation service. It does not replace the production Hindsight service or change the default Hermes profile.

Fixed deployment choices

Supermemory Local server-v0.0.8, Linux x64 binary, verified by SHA-256.
One self-contained server; state, auth material, uploaded files, and model cache live under /mnt/user/appdata/supermemory-poc/data.
A dedicated address on Docker br0; port 6767 is not published on the unRaid host address.
Local multilingual Xenova/bge-m3 embeddings at 1024 dimensions.
One embedding worker and ingest concurrency 1. This avoids treating known concurrent-local-embedding instability as a retrieval-quality result.
Extraction/summarization uses an operator-supplied OpenAI-compatible LLM.
Resource ceiling: 4 CPUs and 8 GiB RAM. Supermemory Local keeps its corpus in memory, so RSS must be watched as the sample grows.

Do not change the embedding model or dimensions in place. Use a fresh data directory and re-ingest when comparing a different embedding plan.

Before deployment

Confirm the chosen SUPERMEMORY_IPV4_ADDRESS is absent from both Arcane's br0 attachments and the LAN neighbor/DHCP tables.
Add SUPERMEMORY_IPV4_ADDRESS, OPENAI_API_KEY, and optional model/base URL overrides to this Project's Arcane environment. Do not put secrets in Git, Compose, activity notes, or chat.
Confirm /mnt/user/appdata has room for the 298 MiB server binary, the multilingual model cache, data, and rollback copy.
Confirm no other build, image pull, backup, or migration is active on the unRaid Environment.

The supermemory-fetch init service downloads the exact release asset once, checks its SHA-256, and stores it in the POC appdata directory. Subsequent starts only verify the existing binary. It also copies the CA bundle from the pinned fetch image so the slim runtime can download local embedding model artifacts over verified HTTPS. Both OpenSSL-style clients and the bundled Bun runtime are pointed at that bundle; TLS verification remains enabled. No custom image build is required.

First boot and authentication

The server creates its client bearer token at:

/mnt/user/appdata/supermemory-poc/data/api-key

Read it through an authorized unRaid/Arcane console without printing it into logs. Save it only as SUPERMEMORY_API_KEY in the supermemory-lab Hermes profile. The extraction LLM key and Supermemory client key are different.

The current candidate address is 192.168.50.13: it is not attached to an Arcane-managed br0 container and did not answer the pre-deployment neighbor or ICMP probes. Confirm it is outside the DHCP pool or reserve it before the Project is started.

Expected Hermes profile file $HERMES_HOME/supermemory.json (also provided as hermes-supermemory.json.example):

{
  "base_url": "http://<dedicated-ip>:6767",
  "container_tag": "hermes_supermemory_lab",
  "auto_recall": true,
  "auto_capture": true,
  "max_recall_results": 10,
  "profile_frequency": 50,
  "capture_mode": "all",
  "search_mode": "hybrid",
  "api_timeout": 10.0
}

Run poc.py smoke before importing any sampled production material. It checks the v3 document path plus the v4 search, profile, and conversation endpoints used by Hermes.

Evaluation guardrails

Score raw Supermemory API retrieval separately from the Hermes answer. In Hermes Agent v0.21.1, the provider reads result metadata but omits it from automatic prefetch text and from the explicit search tool response. A raw hit with correct source/original_id followed by an uncited Hermes answer is an adapter attribution gap, not a retrieval miss.
Do not run the fixed scored question set with automatic writes enabled. The default Hindsight profile has auto_retain: true, and this lab profile has auto_capture: true; earlier test answers could leak into later questions. Use a read-only Hindsight snapshot/profile and set Supermemory auto_capture: false for the retrieval benchmark. Re-enable capture only for the separately scored session-experience phase.
Hermes built-in MEMORY.md and USER.md remain active independently of the external provider. Record their presence and size for each arm so built-in context is not credited to Hindsight or Supermemory.
Self-hosted Supermemory has no managed Google Drive, Gmail, Notion, or OneDrive connectors. This POC evaluates explicit file/document ingestion, not hosted connector sync.

Staged data flow

poc.py sample-hindsight reads active observations from the supported Hindsight 0.8.4 list API and writes a local JSONL artifact. It never mutates the source bank.
Manually review that artifact for scope and sensitive content.
poc.py ingest-jsonl writes the approved rows serially to the isolated Supermemory container using stable custom IDs and provenance metadata.
poc.py ingest-files uploads a small, separately reviewed DEVONthink/Obsidian manifest. Keep the original UUID/path in metadata and treat the source system as authoritative. The default per-file cap is 50 MiB so an accidental library-wide import fails closed.
Run the same questions against the default and supermemory-lab profiles; record groundedness, source traceability, latency, tokens, and cost.

Artifacts are ignored by Git. Do not put exported observations or document content in this configuration repository.

Hermes sanitizes container tags by replacing hyphens with underscores. Keep the canonical hermes_supermemory_lab spelling in direct API imports so they land in the same container queried by the profile.

Rollback

Stop/down only this Project. Keep /mnt/user/appdata/supermemory-poc for later inspection, or archive it before any separately approved deletion. The default Hermes profile and Hindsight service require no rollback because this project does not modify them.

5.8 KiB Raw Blame History