Blog

Blog

Expert guides and engineering deep dives to help you ship faster, scale easier, and learn along the way.

AllModel performanceAI engineeringInfrastructureNewsCommunityAI modelsFoundations
Model performance

How to double tokens per second for Llama 3 with Medusa

Abu Qader
1 other
Double Llama TPS with Medusa
Model performance

How to serve 10,000 fine-tuned LLMs from a single GPU

Pankaj Gupta
1 other
10,000 LoRAs 1 GPU
Model performance

Comparing few-step image generation models

Rachel Rapp
Comparing few-step image generation models
Model performance

How latent consistency models work

Rachel Rapp
How LCMs work
Model performance

Comparing tokens per second across LLMs

Philip Kiely
Comparing TPS across LLMs
Model performance

Continuous vs dynamic batching for AI inference

Matt Howard
1 other
Continuous vs Dynamic batching
Model performance

Benchmarking fast Mistral 7B inference

Abu Qader
3 others
Mistral 7B
Model performance

33% faster LLM inference with FP8 quantization

Pankaj Gupta
1 other
Faster inference with FP8
Model performance

High performance ML inference with NVIDIA TensorRT

Justin Yi
1 other
NVIDIA TensorRT
1234