# robots.txt — MyDigiBiz (https://wa.mydigibiz.com) # Public marketing pages, landing pages, and free tools are open to every # crawler, including AI / LLM crawlers (ChatGPT, Perplexity, Claude, Gemini). # Only private, authenticated app surfaces are disallowed. # --------------------------------------------------------------------------- # All crawlers # --------------------------------------------------------------------------- User-agent: * Allow: / Disallow: /dashboard Disallow: /settings Disallow: /admin Disallow: /billing Disallow: /whatsapp/ Disallow: /contacts Disallow: /messages Disallow: /campaigns Disallow: /group-drip-campaigns Disallow: /auto-responders Disallow: /bot-flows Disallow: /status-schedules Disallow: /templates Disallow: /channels Disallow: /polls Disallow: /api/ Disallow: /store/ Disallow: /register/onboarding Disallow: /verify-email Disallow: /scan_qr/ # --------------------------------------------------------------------------- # AI / LLM crawlers — explicitly welcomed on public content. # (A named group overrides the "*" group for that bot, so the same private # Disallow list is repeated here to keep app surfaces out of AI answers.) # --------------------------------------------------------------------------- User-agent: GPTBot User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: PerplexityBot User-agent: Perplexity-User User-agent: ClaudeBot User-agent: Claude-User User-agent: anthropic-ai User-agent: Google-Extended User-agent: Applebot-Extended User-agent: CCBot User-agent: Amazonbot User-agent: Meta-ExternalAgent Allow: / Disallow: /dashboard Disallow: /settings Disallow: /admin Disallow: /billing Disallow: /whatsapp/ Disallow: /contacts Disallow: /messages Disallow: /campaigns Disallow: /group-drip-campaigns Disallow: /auto-responders Disallow: /bot-flows Disallow: /status-schedules Disallow: /templates Disallow: /channels Disallow: /polls Disallow: /api/ Disallow: /store/ Disallow: /register/onboarding Disallow: /verify-email Disallow: /scan_qr/ Sitemap: https://wa.mydigibiz.com/sitemap.xml