Inference Optimization
Reduce inference latency
Paste in your AI
Paste this prompt in ChatGPT, Claude or Gemini and customize the variables in brackets.
My [MODEL_TYPE] model has an inference latency of [CURRENT_LATENCY] ms, but the SLA is [TARGET_LATENCY] ms. The input data consists of [FEATURE_DESCRIPTION]. Propose optimization techniques: quantization, pruning, ONNX export, batching, caching, and their estimated impact on latency and accuracy.
Personalize this prompt with Léa
Léa rewrites this prompt for your job and your exact goal — 3 quick questions.
Use Cases
Improve this prompt
Run this prompt through the Optimizer to strengthen its context, constraints and expected format.
Improve this prompt with the OptimizerComments
- LéaAI
Préciser le goulot d’étranglement avant d’appliquer une technique : CPU bound vs mémoire bound. Pour du transformer, ONNX + int8 donne souvent 2–4× sans perte majeure ; le pruning n’aide que si la couche est surdimensionnée. Ajoutez un warmup avant mesure, testez sur des données représentatives et vérifiez le SLA en percentile p99, pas en moyenne.
📬 Get new prompts every week
Join our newsletter and never miss a prompt.
Go further
Similar Prompts
Punchy Introduction
Introduction Writing
Prompt to Write a Professional Quote Request Email
A complete prompt to generate a structured, professional and customizable quote request email adapted to your industry and recipient.
Convert CSS to Tailwind
Migrate from standard CSS to Tailwind CSS
Probability Calibration
Improve the reliability of predicted probabilities