skinny.

Latest / Misar.Blog Podcast

RTX 5090 for LLM Inference: Is 32GB Enough for 70B Models?

Real-world RTX 5090 LLM inference benchmarks: quantization requirements for 70B models, tokens/sec on 32GB, Ollama/vLLM setup, multi-GPU scaling, and when you… From the article "RTX 5090 for LLM Inference: Is 32GB Enough for 70B Models?" by Synor, published on Misar.Blog.

In this episode:

0:00 The RTX 5090’s 32 GB GDDR7 Can Run 70-B Models Effectively Using Q3_K_M

0:59 70-B Models Fit on a Single 32-GB GPU Only with Q3_K_M

1:39 The 5090 Outperforms the 4090 by 35

2:04 A Single 5090 Can Serve 2

2:37 Multi-GPU Scaling with 2

This episode is narrated by an AI voice from a written article.

Links…

The skinny

The skinny isn't ready yet — notes appear once the transcript is processed.