Research February 13, 2024 Groq® LPU™ Inference Engine Leads in First Independent LLM Benchmark ArtificialAnalysis.ai Adjusts Chart Axes to Accommodate Groq Performance Levels MOUNTAIN VIEW, CA, February 13, 2024 – Groq® , a generative AI solutions company, is the clear winner in the latest large, the company said on 13 February 2024.
Throughput, Throughput over Time, Total Response Time, and Throughput Variance. The Groq LPU™ Inference Engine performed so well with a leading open source LLM from Meta AI, Llama 2-70b, that axes had to be extended to plot Groq on the Latency vs. Groq participated in its first public LLM benchmark in January 2024 with competition-crushing results.
Groq participated in its first public LLM benchmark in January 2024 with competition-crushing results. “ArtificialAnalysis.ai has independently benchmarked Groq’s Llama 2 Chat (70B) API as achieving throughput of 241 tokens per second, more than double the speed of other hosting providers,” said ArtificialAnalysis.ai Co-creator Micah Hill-Smith.
For enterprise buyers, the useful next step is to map the claim to controls, data residency and measurable outcomes—and to ask for evidence on failure modes, oversight and auditability before treating the release as production-ready fact.