# robots.txt for the ARTEX app (Firebase Hosting site artex-490510). # # Served at https://codame.com/robots.txt (and on artex.codame.com, which # redirects). Vite copies this file # from apps/creator/public/ to dist/ root; Firebase Hosting serves an exact # static-file match ahead of the SPA "** -> /index.html" rewrite, so this is # returned verbatim rather than the app shell. # # The sitemap is DYNAMIC: firebase.json rewrites /sitemap.xml (and /llms.txt, # the AI-agent orientation page) to the platform API, which enumerates every # public page — works, artists, orgs, galleries, programs — from live data. # See docs/seo-and-ai-discoverability.md and issue #1724. # # The Sitemap URL below is on codame.com, the canonical public host as of the # domain consolidation (docs/domain-consolidation-plan.md §7/§8). It must name # the SAME host the sitemap body uses, or Google discounts it as a cross-host # sitemap — the bodies of /sitemap.xml and /llms.txt derive their absolute URLs # from the platform API's PUBLIC_SITE_ORIGIN, so the two move together. # # WHY NO Disallow RULES: a `Disallow` does NOT stop indexing — Google still # indexes disallowed URLs it finds via inbound links, then reports the # "Indexed, though blocked by robots.txt" warning because it cannot crawl them # to see the page. To keep a page OUT of the index, let it be crawled and add a # `noindex` (meta robots tag or X-Robots-Tag header) instead of blocking it # here. Non-public surfaces (labs, previews, signed-in tools) carry exactly # that noindex — via firebase.json X-Robots-Tag headers and client meta tags. # Do not add `Disallow` lines to suppress indexing. # # AI agents and assistant crawlers are welcome on the public surface: they are # served the same content shells as social crawlers (real text, Open Graph # meta, Schema.org JSON-LD) and can start from /llms.txt. User-agent: * Allow: / Sitemap: https://codame.com/sitemap.xml