Optimize robots.txt
Optimize robots.txt
Paste in your AI
Paste this prompt in ChatGPT, Claude or Gemini and customize the variables in brackets.
Review and optimize the following robots.txt file for [domain]: [paste robots.txt]. Check for: accidentally blocked important pages or resources (CSS, JS, images), missing blocks for low-value pages (admin, staging, thank-you pages), incorrect disallow syntax, sitemap declaration, and whether the current rules align with the crawl budget optimization strategy. Provide an updated version.
Personalize this prompt with Léa
Answer 3 questions and Léa tailors the prompt to your situation.
Use Cases
Learn more
Check the full skill on Prompt Guide to master this technique from A to Z.
View on Prompt Guide📬 Get new prompts every week
Join our newsletter and never miss a prompt.
Similar Prompts
Write a Compelling B2B Client Case Study
This prompt generates a structured B2B case study designed to maximize commercial impact, highlighting quantified results and the client voice.
Optimize a Landing Page to Boost Conversions
A comprehensive prompt to analyze and optimize every element of a landing page: headline, copywriting, CTA, social proof and A/B testing plan.
Write a Professional Press Release
This prompt generates a press release in standard journalistic format, ready to send to media outlets, with all sections required by newsrooms.
Create a sunset flow
Clean list with a sunset flow