tianyi· @tianyi · X·· 5 小时前AI 评分
DeepSeek 开源高性能大规模推理框架 DeepInfer
DeepSeek 以 Apache-2.0 许可开源了高性能大规模推理框架 DeepInfer，仓库地址为 github.com/deepseek-ai/DeepI…。其支持的特性包括优化的 kernel、模型与采样器运行时，Attention DP / balanced CP 与 MoE EP + EPLB 并行，DSpark 与置信度调度验证，SWA bounded replay，异步调度（零开销批调度器），批不变推理、确定性采样和结构化输出。
DeepSeek just open sourced its high-performance, large-scale inference framework DeepInfer in Apache-2.0 license.
github.com/deepseek-ai/DeepI…
Supported features:
- Optimized kernels, model and sampler runtime
- Parallelism: Attention DP / balanced CP, MoE EP + EPLB
- DSpark and confidence-scheduled verification
- SWA bounded replay
- Asynchronous scheduling (zero-overhead batch scheduler)
- Batch-invariant inference
- Deterministic sampling
- Structured output
来源：tianyi · x.com