Robots.txt Generator Web & SEO
Build a robots.txt file that says exactly what you mean.
Robots.txt Generator is a free online tool that build a robots.txt file that says exactly what you mean. It runs entirely in your web browser using plain JavaScript, so files are processed on your own device and never uploaded to a server. There is no sign-up, no file size limit imposed by the site, no watermark and no paid tier.
- Price
- Free — no account, no quota, no watermark
- Category
- Web & SEO
- Where it runs
- In your browser, on your device
- Files uploaded
- None
- Technology
- plain JavaScript
- Settings
- 5
- Works offline
- Yes, after the first visit
About Robots.txt Generator
robots.txt is a request, not a wall — it stops well-behaved crawlers indexing a path but does not secure anything. Never use it to hide sensitive URLs; anyone can read the file.
How to use it
- Enter default policy, paths to disallow and sitemap url.
- Adjust the remaining 2 settings if the defaults do not fit your case.
The 5 settings
| Setting | What it does | Default |
|---|---|---|
| Default policy | Choose from 2: Allow everything, Block everything. | Allow everything |
| Paths to disallow | Free text. | /admin/
/cart/
/search |
| Sitemap URL | Free text. | https://example.com/sitemap.xml |
| Crawl delay (seconds) | Range 0–60. | 0 |
| Block common AI crawlers | On or off. Off by default. | Off |
Under the hood
| Runs on | plain JavaScript — runs the whole thing |
| Controls | Default policy, Paths to disallow, Sitemap URL, Crawl delay (seconds), Block common AI crawlers |
Questions
What are the limits on crawl delay (seconds)?
It accepts 0 to 60, starting at 0.