# Disallow stops Googlebot crawling /api/ (crawl budget and 5xx). # X-Robots-Tag noindex on API responses still covers anything that fetches anyway (VOD-88). # Content-Signal: search and live AI answers yes, model training no. # A more specific User-agent group replaces this one, so do not add Allow: / groups for Googlebot or Bingbot. # Named AI search bots must repeat Disallow + Content-Signal or they lose the * rules. User-agent: * Content-Signal: ai-train=no, search=yes, ai-input=yes Disallow: /api/ # ChatGPT Search — citation/search crawl (not model training; that is GPTBot). User-agent: OAI-SearchBot Content-Signal: ai-train=no, search=yes, ai-input=yes Disallow: /api/ # Perplexity answer citations. User-agent: PerplexityBot Content-Signal: ai-train=no, search=yes, ai-input=yes Disallow: /api/ User-agent: MJ12bot Disallow: / Sitemap: https://vodeville.se/sitemap.xml