kai/infra-docker-configs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

deploy local retrieval models for Hindsight

Browse Source
This commit is contained in:
Codex
2026-09-18 00:29:33 -07:00
parent 26d56f7565
commit 2f8bf0e944
4 changed files with 132 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
servers/hindsight/hindsight/.env.example
+16 -7
View File
@@ -60,16 +60,25 @@ HINDSIGHT_API_REFLECT_LLM_MODEL=anthropic/claude-sonnet-5
HINDSIGHT_API_REFLECT_LLM_MAX_CONCURRENT=1
HINDSIGHT_API_REFLECT_LLM_TIMEOUT=600
# OpenRouter embeddings. BGE-M3 emits 1024-dimensional multilingual vectors.
# Local TEI embeddings. BGE-M3 emits 1024-dimensional multilingual vectors.
# OpenRouter settings remain available for a manual rollback; Hindsight does not
# provide a native embedding-provider fallback chain.
HINDSIGHT_API_OPENROUTER_API_KEY=replace-with-openrouter-key
HINDSIGHT_API_EMBEDDINGS_PROVIDER=openrouter
HINDSIGHT_API_EMBEDDINGS_PROVIDER=tei
HINDSIGHT_API_EMBEDDINGS_TEI_URL=http://192.168.50.100:18083
HINDSIGHT_API_EMBEDDINGS_OPENROUTER_MODEL=baai/bge-m3
HINDSIGHT_API_EMBEDDINGS_OPENAI_BATCH_SIZE=32
HINDSIGHT_API_EMBEDDINGS_MAX_INPUT_TOKENS=8192
# OpenRouter reranker, with a fail-open RRF fallback.
HINDSIGHT_API_RERANKER_PROVIDER=openrouter
HINDSIGHT_API_RERANKER_OPENROUTER_MODEL=cohere/rerank-4-fast
HINDSIGHT_API_RERANKER_OPENROUTER_TIMEOUT=30
# Local TEI reranker, then OpenRouter, then fail-open RRF.
HINDSIGHT_API_RERANKER_PROVIDER=tei
HINDSIGHT_API_RERANKER_TEI_URL=http://192.168.50.100:18084
HINDSIGHT_API_RERANKER_TEI_BATCH_SIZE=32
HINDSIGHT_API_RERANKER_TEI_MAX_CONCURRENT=4
HINDSIGHT_API_RERANKER_TEI_HTTP_TIMEOUT=30
HINDSIGHT_API_RERANKER_MAX_CANDIDATES=100
HINDSIGHT_API_RERANKER_1_PROVIDER=rrf
HINDSIGHT_API_RERANKER_1_PROVIDER=openrouter
HINDSIGHT_API_RERANKER_1_OPENROUTER_API_KEY=replace-with-openrouter-key
HINDSIGHT_API_RERANKER_1_OPENROUTER_MODEL=cohere/rerank-4-fast
HINDSIGHT_API_RERANKER_1_OPENROUTER_TIMEOUT=30
HINDSIGHT_API_RERANKER_2_PROVIDER=rrf
servers/hindsight/hindsight/README.md
+2 -2
View File
@@ -7,8 +7,8 @@
- Hindsight `0.9.2-slim`；
- PostgreSQL 18 + pgvector；
- OpenRouter `baai/bge-m3` embedding；
- OpenRouter `cohere/rerank-4-fast` reranker，失败时回退到 RRF；
- unRaid GPU0 上的本地 TEI `BAAI/bge-m3` embedding；OpenRouter 同模型保留为手动回滚；
- unRaid GPU0 上的本地 TEI `Alibaba-NLP/gte-multilingual-reranker-base` reranker，失败时依次回退 OpenRouter `cohere/rerank-4-fast` 与 RRF；
- ZenMux Luna 用于 Retain；
- ZenMux Grok 4.6 用于 Consolidation；
- ZenMux Sonnet 5 用于 Reflect。