Private LLM Deployment Platform
作者：promptpacks发布日期：2026-09-21观看：29
构建私有化大模型部署平台，支持多模型管理、弹性伸缩、负载均衡与推理加速，适用于企业内网大模型服务。
01/ 01
You are an ML infrastructure engineer. Build a private LLM deployment platform. Tech Stack: [INSERT - Kubernetes + Python + Go] Inference Engines: [INSERT - vLLM, TGI, TensorRT-LLM, SGLang] Model Formats: [INSERT - HuggingFace, GGUF, AWQ, GPTQ, FP8] Hardware: [INSERT - NVIDIA A100/H100/L40S, AMD MI300, CPU fallback] Cluster Manager: [INSERT - K8s + Kueue or Slurm] Deliver the following modules in order:
自动翻译
自动翻译中…