About MedusaBot

MedusaBot sends this user agent:

MedusaBot/1.0 (+https://hadalmedusa.com/bot/)

The address in it is this page. The robots.txt name it answers to is MedusaBot.

MedusaBot never uses another crawler's name. A request that would carry one is refused before it is sent.

A crawl starts at a site's home page and follows the links it finds to pages on the same host. Every request is a GET. MedusaBot reads the body of a page only when it is HTML; for any other file, a PDF or an image, it reads the headers and closes the connection. It does not fetch scripts, styles or images on its own.

MedusaBot never submits a form, never logs in and never sends a credential. Cookies a site sets are held for that one visit and then dropped.

MedusaBot reads robots.txt before it fetches a site's pages, once per visit, and checks every page it is about to request against it. It follows the group named MedusaBot, in any case, and otherwise the * group. A MedusaBot group replaces the * group entirely, its Crawl-delay included. Rules are matched as RFC 9309 describes: the longest matching path wins, Allow wins a tie, and * and $ work as wildcards.

To keep MedusaBot off your site entirely, two lines in robots.txt: User-agent: MedusaBot and then Disallow: /. To keep it out of one part, name that part instead of /. To slow it, add Crawl-delay. There is no way to allow one of its jobs and refuse another: they share the one name.

Limits of the current code, stated because they are true:

• A redirect to another host is followed without that host's robots.txt being read first.
• A noindex directive is recorded as a fact about the page, not treated as a reason to skip it; nofollow on a link is not honoured.

From each page it reads, MedusaBot keeps the HTML as fetched and the facts read from it. The facts are kept for 400 days. The stored HTML has no automatic expiry in the current code.

Nothing in MedusaBot looks for personal data, and it never goes behind a login; what a public page shows is what it stores. If you would like what we hold of your site deleted, write to us and it is.

There is no published address range and no reverse-DNS scheme for MedusaBot yet, so the user agent is its only identifier, and a user agent can be copied. If a request carrying this name behaves otherwise than this page says, it is not ours or it is our fault. Either way, tell us.

Every statement on this page was read from the instrument's code at commit f739e2b8 on 11 October 2026. The page changes when the code does.