The Impact of Hyperparameters on Large Language Model Inference Performance: An Evaluation of vLLM and HuggingFace Pipelines
Paper
•
2408.01050
•
Published
•
9
Infra • Serving & Optimization
Inference engines, quantization, serving stacks, and perf tooling. Reference list for deployment and latency/cost work.
-
-
Seesaw: High-throughput LLM Inference via Model Re-sharding
Paper • 2503.06433 • Published -
PRIMA.CPP: Speeding Up 70B-Scale LLM Inference on Low-Resource Everyday Home Clusters
Paper • 2504.08791 • Published • 139 -
Evaluation Guidebook
📝267Display benchmark evaluation data for LLMs
-
Open VLM Leaderboard
🌎983VLMEvalKit Evaluation Results Collection
-
Open VLM Video Leaderboard
🌎130VLMEvalKit Eval Results in video understanding benchmark
-
Qwen/Qwen2-7B-Instruct-AWQ
Text Generation • 8B • Updated • 12.5k • 22 -
TheBloke/CodeLlama-7B-Instruct-AWQ
Text Generation • 7B • Updated • 359 • 4 -
TheBloke/Mistral-7B-Instruct-v0.1-AWQ
Text Generation • 7B • Updated • 759 • 38 -
TheBloke/Mistral-7B-Instruct-v0.2-AWQ
Text Generation • 7B • Updated • 12.1k • 52 -
Qwen/Qwen2.5-VL-7B-Instruct-AWQ
Image-Text-to-Text • 8B • Updated • 699k • 99