# Allow all crawlers on content — including ad-revenue "good bots" # (AdsBot-Google, Mediapartners-Google, IAS, DoubleVerify, Criteo, etc.). # Do NOT add Disallow rules for those: blocking them hurts ad categorization # and revenue. Only back-office and internal API routes are excluded below. User-Agent: * Allow: / Disallow: /admin/ Disallow: /api/ Sitemap: https://lab.allkpop.com/sitemap.xml