deploy local retrieval models for Hindsight

2026-09-18 00:29:33 -07:00
parent 26d56f7565
commit 2f8bf0e944
4 changed files with 132 additions and 9 deletions
@@ -0,0 +1,17 @@
+# Hindsight local retrieval models
+
+Arcane-managed TEI services on unRaid GPU0 for the production Hindsight VM.
+
+| Service | Model | LAN endpoint |
+| --- | --- | --- |
+| Embedding | `BAAI/bge-m3` | `http://192.168.50.100:18083` |
+| Reranker | `Alibaba-NLP/gte-multilingual-reranker-base` | `http://192.168.50.100:18084` |
+
+The BGE-M3 cache is reused from the retired unRaid Hindsight stack. The GTE
+reranker has a separate cache. Both services use physical GPU0 and conservative
+TEI concurrency/tokenizer settings.
+
+Hindsight uses the local embedding directly. Hindsight has no native embedding
+provider fallback chain, so OpenRouter BGE-M3 remains a manual rollback rather
+than an automatic fallback. Reranking uses local TEI first, then OpenRouter
+`cohere/rerank-4-fast`, then RRF.