kai/infra-docker-configs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

同步香港 LiteLLM 镜像与多提供商路由

Browse Source
This commit is contained in:
Codex
2026-10-25 00:42:57 -07:00
parent 7831d5f319
commit 641991043f
6 changed files with 237 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
servers/bwh-hk-cmi/litellm/README.md
+9 -2
View File
@@ -1,4 +1,4 @@
# Hong Kong CMI LiteLLM Vertex AI Relay
# 香港 CMI LiteLLM 多提供商代理
- 公网 OpenAI-compatible 入口为 `https://api.opfo.xyz/v1`，由 Caddy 终止 TLS 后通过外部 Docker 网络 `hk-web` 反向代理到 `litellm-api:1049`。
- LiteLLM 不使用 host network，也不向宿主机发布端口；容器内监听 `0.0.0.0:1049` 仅供 `hk-web` 上的 Caddy 访问。
@@ -8,4 +8,11 @@
- 对外模型别名包括 `gemini-3.6-flash`、`gemini-3.5-flash-lite`、
`google/gemini-embedding-2` 和 `google/gemini-3.8-flash`；后两者仍通过
Vertex AI ADC 路由。
- 后续 Vertex Anthropic 模型在 `config.yaml` 增加独立模型项。
- 2026-10-25 同步 tools-pve 已验证的 LiteLLM 1.104.2 固定镜像，以及 `models.yaml` 中的提供商映射。`latest` 是维护的别名，不是自动发现所有厂商的新模型。
- `provider_routes.py` 是同日 tools-pve 的路由生成器快照；更新提供商逻辑时应同步比对两端。HK 的 `render_config.py` 删除数据库管理、OAuth 和个人词库路线，保留现有主密钥认证。
- Arcane Project 本地变量增加 `ZENMUX_SUBSCRIPTION_API_KEY`、`ZENMUX_PAYG_API_KEY`、`OPENROUTER_API_KEY`、`GOOGLE_AI_STUDIO_API_KEY`。只有相应凭据存在时才生成该组路由；真实值不进入 Git。
- Claude 使用 ZenMux 原生 Anthropic Messages 路径，订阅与 `-payg` 显式分开；开源模型使用 ZenMux 订阅优先、同模型 OpenRouter 后备。不自动回退至 ZenMux PAYG。
- Google AI Studio 的 `gemini-transcribe` 是普通转写路线，不含个人热词；个人词库、OAuth 会话和会议处理程序没有复制至 HK。
- Vertex 继续使用 HK 原有 ADC。旧 `gemini-3.6-flash` 别名保留，但指向当前映射的 `gemini-3.8-flash`；其余旧别名保留。
- 当前实际配置由启动脚本生成至 `/tmp/litellm.yaml`；旧 `config.yaml` 仅作为历史参考，不再挂载。
- 升级只重建 LiteLLM，不变更 Caddy、HAProxy、其他项目及 ADC 卷。失败时回退本次 Git 提交并同步，再按需要 Redeploy，禁止删除卷。
servers/bwh-hk-cmi/litellm/compose.yaml
+12 -3
View File
@@ -14,7 +14,7 @@ services:
printf '%s' "$${GOOGLE_ADC_B64}" | base64 -d > /credentials/application_default_credentials.json
litellm:
image: ghcr.io/berriai/litellm:main-stable@sha256:625981c83410a3ea68eb0697590a57ec1d764d634514d54fa5db0591077ee839
image: ghcr.io/berriai/litellm:main-stable@sha256:3d88bd757134a00e8c00237b85a021ab1f6d54abb9c1df750f411dbe7256a500
restart: unless-stopped
depends_on:
adc-init:
@@ -28,10 +28,19 @@ services:
VERTEXAI_PROJECT: ${VERTEXAI_PROJECT}
VERTEXAI_LOCATION: ${VERTEXAI_LOCATION:-global}
LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}
VERTEXAI_ENABLED: "true"
ZENMUX_SUBSCRIPTION_API_KEY: ${ZENMUX_SUBSCRIPTION_API_KEY:-}
ZENMUX_PAYG_API_KEY: ${ZENMUX_PAYG_API_KEY:-}
OPENROUTER_API_KEY: ${OPENROUTER_API_KEY:-}
GOOGLE_AI_STUDIO_API_KEY: ${GOOGLE_AI_STUDIO_API_KEY:-}
volumes:
- ./config.yaml:/app/config.yaml:ro
- ./models.yaml:/etc/litellm/models.yaml:ro
- ./provider_routes.py:/etc/litellm/provider_routes.py:ro
- ./render_config.py:/etc/litellm/render_config.py:ro
- google-adc:/credentials:ro
command: ["--config", "/app/config.yaml", "--host", "0.0.0.0", "--port", "${LITELLM_PORT:-1049}"]
entrypoint: ["/bin/sh", "-c"]
command:
- python /etc/litellm/render_config.py && exec litellm --config /tmp/litellm.yaml --host 0.0.0.0 --port ${LITELLM_PORT:-1049} --num_workers 1
logging:
driver: json-file
options:
servers/bwh-hk-cmi/litellm/models.yaml
+43
View File
@@ -0,0 +1,43 @@
# 除 Google 自带 latest 别名外，本项目映射不会自行追踪上游发布。
zenmux:
claude-sonnet-latest: anthropic/claude-sonnet-5.5
claude-opus-latest: anthropic/claude-opus-5.5
claude-haiku-latest: anthropic/claude-haiku-5.5
openrouter:
deepseek-flash-latest: deepseek/deepseek-v4.1-flash
deepseek-pro-latest: deepseek/deepseek-v4-pro
glm-latest: z-ai/glm-5.3-prime
glm-flash-latest: z-ai/glm-5.3-flash
kimi-latest: moonshotai/kimi-k3
mimo-flash-latest: xiaomi/mimo-v2.6-flash
# These existing public aliases are subscription-first, not load-balanced.
zenmux_open_models:
deepseek-flash-latest: deepseek/deepseek-v4.1-flash
deepseek-pro-latest: deepseek/deepseek-v4-pro
glm-latest: z-ai/glm-5.3-prime
glm-flash-latest: z-ai/glm-5.3-flash
kimi-latest: moonshotai/kimi-k3
mimo-flash-latest: xiaomi/mimo-v2.6-flash
zenmux_chat:
gemini-flash-zenmux: google/gemini-3.8-flash
google_ai_studio:
gemini-flash-latest: gemini-flash-latest
gemini-flash-lite-latest: gemini-flash-lite-latest
gemini-pro-latest: gemini-pro-latest
gemini-embedding-2-studio: gemini-embedding-2
google_vertex:
gemini-flash-vertex: gemini-3.8-flash
gemini-flash-lite-vertex: gemini-3.5-flash-lite
gemini-pro-vertex: gemini-3.1-pro-preview
google-rerank-default: semantic-ranker-default@latest
google-rerank-fast: semantic-ranker-fast@latest
# 已有 Hindsight 部署用过的候选模型；本代理仍需独立授权与请求验收。
# renderer 在实际 token 出现前排除这些路线。
chatgpt:
gpt-luna-latest: gpt-6-luna
gpt-sol-latest: gpt-6.1-sol
gpt-work-sol: gpt-6-sol
gpt-review-5.5: gpt-5.5
gpt-astra-latest: gpt-6-astra
xai:
grok-latest: grok-4.7
servers/bwh-hk-cmi/litellm/provider_routes.py
+96
View File
@@ -0,0 +1,96 @@
"""Render only credential-ready routes; credentials remain environment references."""
import os
import json
from pathlib import Path
import yaml
def oauth_ready(path):
try:
record = json.loads(path.read_text())
return isinstance(record, dict) and bool(record.get('access_token'))
except (OSError, ValueError):
return False
def build_config(catalog, env, exists):
routes = []
fallbacks = []
def add(alias, model, key=None, **params):
if key:
params['api_key'] = 'os.environ/' + key
routes.append({'model_name': alias, 'litellm_params': {'model': model, **params}})
for key, suffix in [('ZENMUX_SUBSCRIPTION_API_KEY', ''), ('ZENMUX_PAYG_API_KEY', '-payg')]:
if env.get(key):
for alias, model in catalog['zenmux'].items():
# Strip only the LiteLLM provider prefix, retaining ZenMux's vendor prefix.
add(alias + suffix, 'anthropic/' + model, key,
api_base='https://zenmux.ai/api/anthropic')
if env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
for alias, model in catalog.get('zenmux_chat', {}).items():
add(alias, 'openai/' + model, 'ZENMUX_SUBSCRIPTION_API_KEY',
api_base='https://zenmux.ai/api/v1')
if env.get('OPENROUTER_API_KEY'):
for alias, model in catalog['openrouter'].items():
primary = catalog.get('zenmux_open_models', {}).get(alias)
if primary and env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
if primary != model:
raise ValueError('Cross-provider model mismatch: ' + alias)
add(alias, 'openai/' + primary, 'ZENMUX_SUBSCRIPTION_API_KEY',
api_base='https://zenmux.ai/api/v1', max_retries=0)
routes[-1]['model_info'] = {'cooldown_time': 60,
'allowed_fails_policy': {'RateLimitErrorAllowedFails': 0,
'AuthenticationErrorAllowedFails': 0,
'InternalServerErrorAllowedFails': 0,
'ServiceUnavailableErrorAllowedFails': 0,
'TimeoutErrorAllowedFails': 0}}
add(alias + '-openrouter', 'openrouter/' + model, 'OPENROUTER_API_KEY')
fallbacks.append({alias: [alias + '-openrouter']})
else:
add(alias, 'openrouter/' + model, 'OPENROUTER_API_KEY')
elif env.get('ZENMUX_SUBSCRIPTION_API_KEY'):
for alias, model in catalog.get('zenmux_open_models', {}).items():
add(alias, 'openai/' + model, 'ZENMUX_SUBSCRIPTION_API_KEY',
api_base='https://zenmux.ai/api/v1', max_retries=0)
if env.get('GOOGLE_AI_STUDIO_API_KEY'):
add('gemini-transcribe-personal', 'gemini/gemini-3.5-transcribe',
'GOOGLE_AI_STUDIO_API_KEY', personal_vocabulary=True)
for alias, model in catalog['google_ai_studio'].items():
add(alias, 'gemini/' + model, 'GOOGLE_AI_STUDIO_API_KEY')
if env.get('VERTEXAI_ENABLED', '').lower() == 'true' and env.get('VERTEXAI_PROJECT'):
for alias, model in catalog.get('google_vertex', {}).items():
add(alias, 'vertex_ai/' + model,
vertex_project='os.environ/VERTEXAI_PROJECT',
vertex_location='os.environ/VERTEXAI_LOCATION')
for provider, token_var, default_dir in [
('chatgpt', 'CHATGPT_TOKEN_DIR', '/tokens/chatgpt'),
('xai', 'XAI_OAUTH_TOKEN_DIR', '/tokens/xai'),
]:
if exists(Path(env.get(token_var, default_dir)) / 'auth.json'):
for alias, model in catalog[provider].items():
extra = {'use_xai_oauth': True} if provider == 'xai' else {}
add(alias, provider + '/' + model, **extra)
return {
'model_list': routes,
'general_settings': {
'master_key': 'os.environ/LITELLM_MASTER_KEY',
'database_url': 'os.environ/DATABASE_URL',
'store_model_in_db': False,
'disable_env_credential_login': True,
},
'litellm_settings': {'drop_params': False, 'set_verbose': False},
# Only explicitly approved open-model routes have cross-provider fallbacks.
'router_settings': {'num_retries': 1, 'timeout': 180, 'fallbacks': fallbacks},
}
if __name__ == '__main__':
catalog = yaml.safe_load(Path('/etc/litellm/models.yaml').read_text())
config = build_config(catalog, os.environ, oauth_ready)
output = Path('/tmp/litellm.yaml')
output.write_text(yaml.safe_dump(config, allow_unicode=True, sort_keys=False))
output.chmod(0o600)
print('Configured routes:', ', '.join(route['model_name'] for route in config['model_list']))
servers/bwh-hk-cmi/litellm/render_config.py
+41
View File
@@ -0,0 +1,41 @@
"""HK reuses tools-pve provider routing without OAuth, database, or private vocabulary."""
import os
from pathlib import Path
import yaml
from provider_routes import build_config
def build_hk_config(catalog, env):
config = build_config(catalog, env, lambda path: False)
config['general_settings'] = {'master_key': 'os.environ/LITELLM_MASTER_KEY'}
routes = config['model_list']
routes[:] = [route for route in routes if route['model_name'] != 'gemini-transcribe-personal']
if env.get('GOOGLE_AI_STUDIO_API_KEY'):
routes.append({'model_name': 'gemini-transcribe', 'litellm_params': {
'model': 'gemini/gemini-3.5-transcribe',
'api_key': 'os.environ/GOOGLE_AI_STUDIO_API_KEY'}})
if env.get('VERTEXAI_ENABLED', '').lower() == 'true' and env.get('VERTEXAI_PROJECT'):
# Keep old client aliases; the old 3.6 alias now follows the verified Flash map.
legacy = {
'gemini-3.6-flash': catalog['google_vertex']['gemini-flash-vertex'],
'gemini-3.5-flash-lite': catalog['google_vertex']['gemini-flash-lite-vertex'],
'google/gemini-embedding-2': 'gemini-embedding-2',
'google/gemini-3.8-flash': catalog['google_vertex']['gemini-flash-vertex'],
}
for alias, model in legacy.items():
routes.append({'model_name': alias, 'litellm_params': {
'model': 'vertex_ai/' + model,
'vertex_project': 'os.environ/VERTEXAI_PROJECT',
'vertex_location': 'os.environ/VERTEXAI_LOCATION'}})
return config
if __name__ == '__main__':
catalog = yaml.safe_load(Path('/etc/litellm/models.yaml').read_text())
config = build_hk_config(catalog, os.environ)
output = Path('/tmp/litellm.yaml')
output.write_text(yaml.safe_dump(config, allow_unicode=True, sort_keys=False))
output.chmod(0o600)
print('Configured routes:', ', '.join(route['model_name'] for route in config['model_list']))
servers/bwh-hk-cmi/litellm/test_config.py
+36
View File
@@ -0,0 +1,36 @@
import unittest
from pathlib import Path
import yaml
from render_config import build_hk_config
class RoutingTests(unittest.TestCase):
def test_all_providers(self):
catalog = yaml.safe_load(Path(__file__).with_name('models.yaml').read_text())
env = {key: 'synthetic' for key in ['ZENMUX_SUBSCRIPTION_API_KEY',
'ZENMUX_PAYG_API_KEY', 'OPENROUTER_API_KEY', 'GOOGLE_AI_STUDIO_API_KEY',
'VERTEXAI_PROJECT']}
env['VERTEXAI_ENABLED'] = 'true'
result = build_hk_config(catalog, env)
routes = {r['model_name']: r['litellm_params'] for r in result['model_list']}
self.assertEqual(len(routes), len(result['model_list']))
self.assertNotIn('gemini-transcribe-personal', routes)
self.assertNotIn('grok-latest', routes)
self.assertNotIn('gpt-sol-latest', routes)
self.assertEqual(routes['gemini-transcribe']['model'], 'gemini/gemini-3.5-transcribe')
self.assertEqual(routes['gemini-3.6-flash']['model'], 'vertex_ai/gemini-3.8-flash')
self.assertEqual(len(result['router_settings']['fallbacks']), 6)
self.assertEqual(result['general_settings'], {'master_key': 'os.environ/LITELLM_MASTER_KEY'})
self.assertNotIn('synthetic', yaml.safe_dump(result))
def test_missing_credentials(self):
catalog = yaml.safe_load(Path(__file__).with_name('models.yaml').read_text())
result = build_hk_config(catalog, {})
self.assertEqual(result['model_list'], [])
self.assertEqual(result['router_settings']['fallbacks'], [])
if __name__ == '__main__':
unittest.main()