GPU & NPU Benchmarking Menü ▾
English
Benchmarks Leaderboard FAQ Software

Public GPU and NPU uploads

Find the fastest hardware for prompt processing

Compare GPUs and NPUs by prompt throughput and see which local LLM hardware handles long prompts and large context windows the fastest.

Hardware
VRAM GB
Prompt processing ↓ tok/s
NVIDIA RTX 5090 32 GB
10,381.64 tok/s
NVIDIA RTX 4090 24 GB
9,452.03 tok/s
NVIDIA RTX 5080 16 GB
7,444.99 tok/s
NVIDIA RTX 4080 SUPER 16 GB
7,101.18 tok/s
NVIDIA RTX 5070 Ti 16 GB
6,213.63 tok/s
NVIDIA RTX 4070 Ti SUPER 16 GB
6,099.18 tok/s
AMD RX 9070 XT 16 GB
5,036.04 tok/s
NVIDIA RTX 4070 Ti 12 GB
4,981.44 tok/s
NVIDIA RTX 3090 24 GB
4,298.97 tok/s
NVIDIA RTX 3080 12 GB 12 GB
4,287.11 tok/s
NVIDIA RTX 3080 10 GB 10 GB
4,287.11 tok/s
AMD RX 7900 XTX 24 GB
3,531.93 tok/s
NVIDIA RTX 5060 Ti 16 GB 16 GB
3,460.92 tok/s
NVIDIA RTX 5060 Ti 8 GB 8 GB
3,460.92 tok/s
NVIDIA RTX 4070 12 GB
3,179.37 tok/s
AMD RX 9070 16 GB
3,164.10 tok/s
AMD RX 7900 XT 20 GB
2,941.58 tok/s
AMD RX 7900 GRE 16 GB
2,336.31 tok/s
AMD RX 9060 XT 8 GB 8 GB
2,141.67 tok/s
AMD RX 9060 XT 16 GB 16 GB
2,141.67 tok/s
NVIDIA RTX 3070 8 GB
2,113.02 tok/s
AMD RX 7800 XT 16 GB
2,017.33 tok/s
AMD RX 6900 XT 16 GB
1,901.20 tok/s
NVIDIA RTX 3060 12 GB 12 GB
1,815.70 tok/s
NVIDIA RTX 3060 8 GB 8 GB
1,815.70 tok/s
AMD RX 6800 XT 16 GB
1,752.92 tok/s
AMD RX 6800 16 GB
1,698.69 tok/s
Apple M3 Ultra 80-core GPU 256 GB
1,471.24 tok/s
Apple M3 Ultra 80-core GPU 512 GB
1,471.24 tok/s
Apple M2 Ultra 76-core GPU 64 GB
1,238.48 tok/s
Apple M2 Ultra 76-core GPU 128 GB
1,238.48 tok/s
Apple M2 Ultra 76-core GPU 192 GB
1,238.48 tok/s
Intel Arc A770 8 GB 8 GB
1,073.85 tok/s
Intel Arc A770 16 GB 16 GB
1,073.85 tok/s
Apple M3 Ultra 60-core GPU 96 GB
1,073.09 tok/s
AMD RX 6700 XT 12 GB
1,051.20 tok/s
AMD RX 6750 XT 12 GB
1,040.58 tok/s
Apple M1 Ultra 64-core GPU 64 GB
1,030.04 tok/s
Apple M1 Ultra 64-core GPU 128 GB
1,030.04 tok/s
AMD RX 6650 XT 8 GB
1,029.52 tok/s
Apple M2 Ultra 60-core GPU 64 GB
1,013.81 tok/s
Apple M2 Ultra 60-core GPU 128 GB
1,013.81 tok/s
Apple M2 Ultra 60-core GPU 192 GB
1,013.81 tok/s
Intel Arc B570 10 GB
913.95 tok/s
Apple M4 Max 40-core GPU 48 GB
885.68 tok/s
Apple M4 Max 40-core GPU 64 GB
885.68 tok/s
Apple M4 Max 40-core GPU 128 GB
885.68 tok/s
AMD RX 7600 XT 16 GB
840.85 tok/s
Apple M1 Ultra 48-core GPU 64 GB
772.24 tok/s
Apple M1 Ultra 48-core GPU 128 GB
772.24 tok/s
AMD RX 6600 8 GB
761.89 tok/s
Apple M3 Max 40-core GPU 48 GB
759.70 tok/s
Apple M3 Max 40-core GPU 64 GB
759.70 tok/s
Apple M3 Max 40-core GPU 128 GB
759.70 tok/s
Apple M4 Max 32-core GPU 36 GB
713.93 tok/s
Apple M2 Max 38-core GPU 32 GB
671.31 tok/s
Apple M2 Max 38-core GPU 64 GB
671.31 tok/s
Apple M2 Max 38-core GPU 96 GB
671.31 tok/s
Intel Arc B580 12 GB
620.94 tok/s
AMD RX 6600 XT 8 GB
574.65 tok/s
Apple M3 Max 30-core GPU 36 GB
567.59 tok/s
Apple M3 Max 30-core GPU 96 GB
567.59 tok/s
Apple M2 Max 30-core GPU 32 GB
537.60 tok/s
Apple M2 Max 30-core GPU 64 GB
537.60 tok/s
Apple M1 Max 32-core GPU 32 GB
530.06 tok/s
Apple M1 Max 32-core GPU 64 GB
530.06 tok/s
Apple M4 Pro 20-core GPU 24 GB
439.78 tok/s
Apple M4 Pro 20-core GPU 48 GB
439.78 tok/s
Apple M4 Pro 20-core GPU 64 GB
439.78 tok/s
Apple M1 Max 24-core GPU 32 GB
400.26 tok/s
Apple M1 Max 24-core GPU 64 GB
400.26 tok/s
Apple M4 Pro 16-core GPU 24 GB
364.06 tok/s
Apple M4 Pro 16-core GPU 48 GB
364.06 tok/s
Apple M3 Pro 18-core GPU 18 GB
341.67 tok/s
Apple M3 Pro 18-core GPU 36 GB
341.67 tok/s
Apple M2 Pro 19-core GPU 16 GB
341.19 tok/s
Apple M2 Pro 19-core GPU 32 GB
341.19 tok/s
Intel Arc A750 8 GB
303.37 tok/s
Apple M2 Pro 16-core GPU 16 GB
294.24 tok/s
Apple M2 Pro 16-core GPU 32 GB
294.24 tok/s
Apple M3 Pro 14-core GPU 18 GB
269.49 tok/s
Apple M3 Pro 14-core GPU 36 GB
269.49 tok/s
Apple M1 Pro 16-core GPU 16 GB
266.25 tok/s
Apple M1 Pro 16-core GPU 32 GB
266.25 tok/s
AMD RX 6500 XT 4 GB
255.25 tok/s
Apple M1 Pro 14-core GPU 16 GB
232.55 tok/s
Apple M1 Pro 14-core GPU 32 GB
232.55 tok/s
Apple M4 10-core GPU 16 GB
221.29 tok/s
Apple M4 10-core GPU 24 GB
221.29 tok/s
Apple M4 10-core GPU 32 GB
221.29 tok/s
Apple M3 10-core GPU 8 GB
186.75 tok/s
Apple M3 10-core GPU 16 GB
186.75 tok/s
Apple M3 10-core GPU 24 GB
186.75 tok/s
Apple M2 10-core GPU 8 GB
179.57 tok/s
Apple M2 10-core GPU 16 GB
179.57 tok/s
Apple M2 10-core GPU 24 GB
179.57 tok/s
Apple M1 8-core GPU 8 GB
117.96 tok/s
Apple M1 8-core GPU 16 GB
117.96 tok/s
Apple M1 7-core GPU 8 GB
107.81 tok/s
Apple M1 7-core GPU 16 GB
107.81 tok/s
AMD RX 6400 4 GB
—
AMD RX 6700 10 GB
—
AMD RX 6750 GRE 12 GB 12 GB
—
AMD RX 6750 GRE 10 GB 10 GB
—
AMD RX 6950 XT 16 GB
—
AMD RX 7600 8 GB
—
AMD RX 7700 XT 12 GB
—
AMD RX 9060 8 GB
—
Apple M1 7-core GPU 8 GB
—
Apple M2 8-core GPU 8 GB
—
Apple M2 8-core GPU 16 GB
—
Apple M2 8-core GPU 24 GB
—
Apple M3 8-core GPU 8 GB
—
Apple M3 8-core GPU 16 GB
—
Apple M3 8-core GPU 24 GB
—
Apple M4 8-core GPU 16 GB
—
Apple M4 8-core GPU 24 GB
—
Apple M4 8-core GPU 32 GB
—
Apple M4 9-core GPU 12 GB
—
Apple M5 8-core GPU 16 GB
—
Apple M5 8-core GPU 24 GB
—
Apple M5 8-core GPU 32 GB
—
Apple M5 10-core GPU 16 GB
—
Apple M5 10-core GPU 24 GB
—
Apple M5 10-core GPU 32 GB
—
Apple M5 Max 32-core GPU 36 GB
—
Apple M5 Max 40-core GPU 48 GB
—
Apple M5 Max 40-core GPU 64 GB
—
Apple M5 Max 40-core GPU 128 GB
—
Apple M5 Pro 20-core GPU 24 GB
—
Apple M5 Pro 20-core GPU 48 GB
—
Apple M5 Pro 20-core GPU 64 GB
—
Intel Arc A310 4 GB
—
Intel Arc A380 6 GB
—
Intel Arc A580 8 GB
—
NVIDIA RTX 3050 6 GB 6 GB
—
NVIDIA RTX 3050 8 GB 8 GB
—
NVIDIA RTX 3060 Ti 448 GB/s 8 GB
—
NVIDIA RTX 3060 Ti 608 GB/s 8 GB
—
NVIDIA RTX 3070 Ti 8 GB
—
NVIDIA RTX 3080 Ti 12 GB
—
NVIDIA RTX 3090 Ti 24 GB
—
NVIDIA RTX 4060 8 GB
—
NVIDIA RTX 4060 Ti 16 GB 16 GB
—
NVIDIA RTX 4060 Ti 8 GB 8 GB
—
NVIDIA RTX 4070 SUPER 12 GB
—
NVIDIA RTX 4080 16 GB
—
NVIDIA RTX 5050 8 GB
—
NVIDIA RTX 5060 8 GB
—
NVIDIA RTX 5070 12 GB
—

This website uses technically necessary cookies for the proper operation of the site.