inference-bench: serving engine benchmarks on Akamai Cloud GPUs

What is inference-bench: serving engine benchmarks on Akamai Cloud GPUs?

Evaluates six LLM serving engines (vLLM, SGLang, TGI, Ollama, llama.cpp, NVIDIA NIM) on latency, throughput, and cost per million tokens using fixed workloads…

Released

2026-09-27

Evaluates

Language & Knowledge, Software & AI Compute, cs.AI

Openness

unknown

Importer

Claire Radar

Review status

ai-reviewed

Reported scores

0

Source provenance

• Original evidence https://github.com/cmcconnell1/inference-bench

inference-bench: serving engine benchmarks on Akamai Cloud GPUs code

• Code https://github.com/cmcconnell1/inference-bench

Which sources cite inference-bench: serving engine benchmarks on Akamai Cloud GPUs?

• inference-bench: serving engine benchmarks on Akamai Cloud GPUs

Related Language & Knowledge, Software & AI Compute, cs.AI benchmarks

• 2d-fet-bench
• 5-Dialects-BN
• A Large-Scale Dataset and Benchmark
• AARRI-Bench
• ABACUS
• ABC-Bench
• Active-SWE
• ActiveSciBench