topic:llm-serving
Ctrl+K2 projects in topic llm-serving
Sort
- vllmvllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
- Version
- v0.33.0rc3
- Stars
- 97k
- License
- Apache-2.0
- Docker
- –
- rayray-project/ray
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
- Version
- 2.60.0
- Stars
- 44.3k
- License
- Apache-2.0
- Docker
- –
Page 1 of 1