# robots.txt — Defiant Denizens # # Allow general crawlers across the site. The /admin/* surface is # operator-only tooling and behind auth, so we ask crawlers not to spend # budget on it. The AdSense + AdsBot crawlers MUST be allowed everywhere # they want to fetch — required for AdSense approval and quality scoring. User-agent: * Disallow: /admin/ Disallow: /admin User-agent: Mediapartners-Google Allow: / User-agent: AdsBot-Google Allow: / User-agent: AdsBot-Google-Mobile Allow: / Sitemap: /sitemap.xml