test Supermemory with LiteLLM Gemini models

This commit is contained in:
Codex
2026-09-07 23:48:22 -07:00
parent 41ae90f88c
commit 1690f774e9
3 changed files with 118 additions and 25 deletions
+20 -17
View File
@@ -10,10 +10,12 @@ production Hindsight service or change the default Hermes profile.
cache live under `/mnt/user/appdata/supermemory-poc/data`.
- A dedicated address on Docker `br0`; port `6767` is not published on the
unRaid host address.
- Local multilingual `Xenova/bge-m3` embeddings at 1024 dimensions.
- One embedding worker and ingest concurrency 1. This avoids treating known
concurrent-local-embedding instability as a retrieval-quality result.
- Extraction/summarization uses an operator-supplied OpenAI-compatible LLM.
- Remote OpenAI-compatible embeddings through HK LiteLLM using
`google/gemini-embedding-2` at 1024 dimensions.
- Extraction, summarization, and other intelligent steps use
`gemini-3.5-flash-lite` through the same LiteLLM endpoint.
- Ingest concurrency remains 1 so the first POC emphasizes correctness and
observable cost over throughput.
- Resource ceiling: 4 CPUs and 8 GiB RAM. Supermemory Local keeps its corpus
in memory, so RSS must be watched as the sample grows.
@@ -24,8 +26,10 @@ directory and re-ingest when comparing a different embedding plan.
1. Confirm the chosen `SUPERMEMORY_IPV4_ADDRESS` is absent from both Arcane's
`br0` attachments and the LAN neighbor/DHCP tables.
2. Add `SUPERMEMORY_IPV4_ADDRESS`, `OPENAI_API_KEY`, and optional model/base URL
overrides to this Project's Arcane environment. Do not put secrets in Git,
2. Add `SUPERMEMORY_IPV4_ADDRESS`, `OPENAI_API_KEY`,
`OPENAI_BASE_URL=https://api.opfo.xyz/v1`, and
`OPENAI_MODEL=gemini-3.5-flash-lite` to this Project's Arcane environment.
`OPENAI_API_KEY` is the HK LiteLLM bearer key. Do not put it in Git,
Compose, activity notes, or chat.
3. Confirm `/mnt/user/appdata` has room for the 298 MiB server binary, the
multilingual model cache, data, and rollback copy.
@@ -45,7 +49,7 @@ image build is required.
The server creates its client bearer token at:
`/mnt/user/appdata/supermemory-poc/data/api-key`
`/mnt/user/appdata/supermemory-poc/data-google-gemini-embedding-2-1024/api-key`
Read it through an authorized unRaid/Arcane console without printing it into
logs. Save it only as `SUPERMEMORY_API_KEY` in the `supermemory-lab` Hermes
@@ -77,17 +81,16 @@ Run `poc.py smoke` before importing any sampled production material. It checks
the v3 document path plus the v4 search, profile, and conversation endpoints
used by Hermes.
### Current local-embedding blocker
### Embedding plan and smoke gate
On the unRaid POC host, `server-v0.0.8` remains healthy but its local embedding
worker returns `unknown certificate verification error` when first loading
`Xenova/bge-m3`. Supplying a Mozilla CA bundle through `SSL_CERT_FILE`,
`SSL_CERT_DIR`, `NODE_EXTRA_CA_CERTS`, and Debian's standard CA path did not
change the error; TLS verification was never disabled. Do not import sampled
Hindsight or document data until either a remote OpenAI-compatible embedding
endpoint or a separately approved local embedding sidecar passes the smoke
test. The server also returns HTTP 409 when deleting documents stuck in the
`embedding` state, so failed smoke rows may require a fresh POC data directory.
The earlier local `Xenova/bge-m3` experiment is preserved under the old `data`
directory because embedding plans cannot be changed in place. This deployment
uses the fresh `data-google-gemini-embedding-2-1024` directory. The one-shot
`supermemory-smoke` service verifies the locked embedding plan, performs a
Chinese hybrid-search round trip, checks provenance metadata, and exercises
the profile and conversation endpoints. It writes a sentinel after success so
ordinary redeploys do not add duplicate smoke documents. Do not import sampled
Hindsight or document material until this service exits successfully.
## Evaluation guardrails