test Supermemory with native Gemini

This commit is contained in:
Codex
2026-09-08 23:55:40 -07:00
parent cc3e05298b
commit e8f98c1ed7
4 changed files with 75 additions and 94 deletions
+24 -26
View File
@@ -10,10 +10,10 @@ production Hindsight service or change the default Hermes profile.
cache live under `/mnt/user/appdata/supermemory-poc/data`.
- A dedicated address on Docker `br0`; port `6767` is not published on the
unRaid host address.
- Remote OpenAI-compatible embeddings through HK LiteLLM using
`google/gemini-embedding-2` at 1024 dimensions.
- Extraction, summarization, and other intelligent steps use
`gemini-3.5-flash-lite` through the same LiteLLM endpoint.
- Native Gemini embeddings using `gemini-embedding-2` at 768 dimensions.
- Extraction, summarization, and other intelligent steps use Supermemory's
native Gemini provider default. This cleanly removes the LiteLLM and
OpenAI-compatible translation layer from the diagnostic path.
- Ingest concurrency remains 1 so the first POC emphasizes correctness and
observable cost over throughput.
- Resource ceiling: 4 CPUs and 8 GiB RAM. Supermemory Local keeps its corpus
@@ -26,11 +26,10 @@ directory and re-ingest when comparing a different embedding plan.
1. Confirm the chosen `SUPERMEMORY_IPV4_ADDRESS` is absent from both Arcane's
`br0` attachments and the LAN neighbor/DHCP tables.
2. Add `SUPERMEMORY_IPV4_ADDRESS`, `OPENAI_API_KEY`,
`OPENAI_BASE_URL=https://api.opfo.xyz/v1`, and
`OPENAI_MODEL=gemini-3.5-flash-lite` to this Project's Arcane environment.
`OPENAI_API_KEY` is the HK LiteLLM bearer key. Do not put it in Git,
Compose, activity notes, or chat.
2. Add `SUPERMEMORY_IPV4_ADDRESS` and `GEMINI_API_KEY` to this Project's
Arcane environment. Do not put the key in Git, Compose, activity notes, or
chat. Legacy `OPENAI_*` values may remain stored in Arcane for rollback,
but this Compose file does not inject them into the container.
3. Confirm `/mnt/user/appdata` has room for the 298 MiB server binary, the
multilingual model cache, data, and rollback copy.
4. Confirm no other build, image pull, backup, or migration is active on the
@@ -49,7 +48,7 @@ image build is required.
The server creates its client bearer token at:
`/mnt/user/appdata/supermemory-poc/data-google-gemini-embedding-2-1024/api-key`
`/mnt/user/appdata/supermemory-poc/data-gemini-embedding-2-768/api-key`
Read it through an authorized unRaid/Arcane console without printing it into
logs. Save it only as `SUPERMEMORY_API_KEY` in the `supermemory-lab` Hermes
@@ -83,24 +82,23 @@ used by Hermes.
### Embedding plan and smoke gate
The earlier local `Xenova/bge-m3` experiment is preserved under the old `data`
directory because embedding plans cannot be changed in place. This deployment
uses the fresh `data-google-gemini-embedding-2-1024` directory. The one-shot
`supermemory-smoke` service verifies the locked embedding plan, performs a
Chinese hybrid-search round trip, checks provenance metadata, and exercises
the profile and conversation endpoints. It writes a sentinel after success so
ordinary redeploys do not add duplicate smoke documents. Do not import sampled
The earlier local `Xenova/bge-m3` and LiteLLM-backed embedding experiments are
preserved under their old data directories because embedding plans cannot be
changed in place. This deployment uses the fresh
`data-gemini-embedding-2-768` directory. The one-shot `supermemory-smoke`
service verifies the locked embedding plan, tracks the exact document ID to a
terminal state, separately checks v3 search, v4 document search, and Chinese
hybrid search, checks provenance metadata, and exercises the profile and
conversation endpoints. It writes a sentinel after success so ordinary
redeploys do not add duplicate smoke documents. Do not import sampled
Hindsight or document material until this service exits successfully.
On 2026-09-08, the remote plan was accepted and locked as
`openai / google/gemini-embedding-2 / 1024`; the LiteLLM embedding and
`gemini-3.5-flash-lite` chat endpoints also passed independently. Supermemory
listed the smoke document and reported a completed document in the test
container, but exact searches for its unique marker returned no hit through
either `/v4/search` or `/v3/search` (`supermemory-smoke` exit 44). Treat this
as a self-hosted indexing/retrieval blocker. Do not import sampled material or
enable the Hermes lab profile until a newer server build or upstream fix makes
the same smoke pass.
On 2026-09-08, the previous remote plan was accepted and locked as
`openai / google/gemini-embedding-2 / 1024`; the LiteLLM embedding and chat
endpoints also passed independently. Supermemory listed a completed smoke
document, but both v4 and v3 searches returned no hit. The old smoke mixed
list-wide status checks and did not track the newly created document, so this
native Gemini run uses a new store and a stricter document-specific gate.
## Evaluation guardrails