User-agent: GPTBot User-agent: ClaudeBot User-agent: Claude-Web User-agent: anthropic-ai User-agent: CCBot User-agent: Google-Extended User-agent: Applebot-Extended User-agent: Meta-ExternalAgent User-agent: FacebookBot User-agent: Bytespider User-agent: TikTokSpider User-agent: Amazonbot User-agent: YandexAdditional User-agent: cohere-ai User-agent: cohere-training-data-crawler User-agent: AI2Bot User-agent: Ai2Bot-Dolma User-agent: Diffbot User-agent: img2dataset User-agent: omgili User-agent: omgilibot User-agent: Timpibot User-agent: Webzio-Extended User-agent: ImagesiftBot User-agent: PanguBot User-agent: aiHitBot User-agent: MyCentralAIScraperBot User-agent: Quora-Bot User-agent: FirecrawlAgent User-agent: Cotoyogi User-agent: SBIntuitionsBot User-agent: ICC-Crawler User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: Claude-SearchBot User-agent: Claude-User User-agent: PerplexityBot User-agent: Perplexity-User User-agent: DuckAssistBot User-agent: Amzn-SearchBot User-agent: Amzn-User User-agent: Meta-WebIndexer User-agent: meta-externalfetcher User-agent: YouBot User-agent: MistralAI-User Disallow: /sk/ Disallow: /beta/sk/ Disallow: /beta/profiles/ # Squarespace Robots Txt User-agent: AI2Bot User-agent: Ai2Bot-Dolma User-agent: aiHitBot User-agent: Amazonbot User-agent: anthropic-ai User-agent: Applebot-Extended User-agent: Bytespider User-agent: CCBot User-agent: ClaudeBot User-agent: cohere-ai User-agent: cohere-training-data-crawler User-agent: DuckAssistBot User-agent: FacebookBot User-agent: Google-Extended User-agent: GoogleOther User-agent: GoogleOther-Image User-agent: GoogleOther-Video User-agent: GPTBot User-agent: img2dataset User-agent: Meta-ExternalAgent User-agent: MyCentralAIScraperBot User-agent: omgili User-agent: omgilibot User-agent: Quora-Bot User-agent: TikTokSpider User-agent: YouBot User-agent: AdsBot-Google User-agent: AdsBot-Google-Mobile User-agent: AdsBot-Google-Mobile-Apps User-agent: * Disallow: /config Disallow: /search Disallow: /account$ Disallow: /account/ Disallow: /commerce/digital-download/ Disallow: /api/ Allow: /api/ui-extensions/ Disallow: /static/ Disallow:/*?author=* Disallow:/*&author=* Disallow:/*?tag=* Disallow:/*&tag=* Disallow:/*?month=* Disallow:/*&month=* Disallow:/*?view=* Disallow:/*&view=* Disallow:/*?format=json Disallow:/*&format=json Disallow:/*?format=page-context Disallow:/*&format=page-context Disallow:/*?format=main-content Disallow:/*&format=main-content Disallow:/*?format=json-pretty Disallow:/*&format=json-pretty Disallow:/*?format=ical Disallow:/*&format=ical Disallow:/*?reversePaginate=* Disallow:/*&reversePaginate=* Sitemap: https://skillit.com/sitemap.xml Disallow: /beta/profiles/resumes # skillit.com robots.txt User-agent: * Disallow: /beta/profiles/resumes Sitemap: https://skillit.com/sitemap-jobs.xml