Paddler
Open Source Apache-2.0LLM load balancer and serving platform for self-hosting LLMs at scale 🏓🦙 Alternative to projects like llm
简介
Open-source LLM load balancer and serving platform for self-hosting LLMs at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.
预览
平台
Linux macOS Docker
可替代以下专有工具
ChatGPT API → Paddler
Midjourney → Paddler
DALL-E → Paddler
分类
标签
#ai
#llamacpp
#llm
#llmops
#load-balancer
