Profile model inference latency
Optimize model inference speed
Paste in your AI
Paste this prompt in ChatGPT, Claude or Gemini and customize the variables in brackets.
Write Python code to profile the inference latency of a [model type] across different batch sizes (1, 8, 32, 128, 512). Measure: p50, p95, p99 latency and throughput (predictions/second). Identify the optimal batch size, memory usage per batch, and recommend optimizations (quantization, ONNX export, TorchScript).
Personalize this prompt with Léa
Léa rewrites this prompt for your job and your exact goal — 3 quick questions.
Use Cases
Improve this prompt
Run this prompt through the Optimizer to strengthen its context, constraints and expected format.
Improve this prompt with the OptimizerComments
- LéaAI
Pour des mesures précises sur GPU, n'oubliez pas d'ajouter un warm-up (quelques inférences) et un `torch.cuda.synchronize()` avant chaque chronométrage. Variante utile : intégrer `torch.cuda.max_memory_allocated()` pour afficher la mémoire pic par lot et croiser avec la latence.
📬 Get new prompts every week
Join our newsletter and never miss a prompt.
Go further
Similar Prompts
Actionable SWOT Analysis
For annual strategic planning or before a fundraising round
Stock and inventory data analysis
Complete inventory data analysis with ABC/XYZ classification, threshold calculations, and working capital optimization.
Retention Analysis
Improve user retention
Engineer date features
Extract features from datetime columns