# Halvren Capital — robots policy # Plain-language version of this file lives at https://halvrencapital.com/about # (AI & Indexing Policy section). Concatenated long-form text for LLMs is at # https://halvrencapital.com/llms-full.txt. # --------------------------------------------------------------------------- # Search-engine indexing crawlers — allowed in full # --------------------------------------------------------------------------- User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / User-agent: DuckDuckBot Allow: / User-agent: Applebot Allow: / # --------------------------------------------------------------------------- # AI assistant + LLM crawlers — allowed (research is free, attribution requested) # --------------------------------------------------------------------------- User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-Web Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: / User-agent: Applebot-Extended Allow: / User-agent: Bytespider Allow: / User-agent: cohere-ai Allow: / # --------------------------------------------------------------------------- # Aggressive / commercial-scraping crawlers — disallowed # --------------------------------------------------------------------------- User-agent: CCBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: FacebookBot Disallow: / User-agent: Diffbot Disallow: / User-agent: Amazonbot Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: Omgilibot Disallow: / User-agent: omgili Disallow: / User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / # --------------------------------------------------------------------------- # Always block sensitive paths # --------------------------------------------------------------------------- User-agent: * Disallow: /api/ Disallow: /checklist/score/ Allow: / # --------------------------------------------------------------------------- # Sitemap + machine-readable index # --------------------------------------------------------------------------- Sitemap: https://halvrencapital.com/sitemap.xml # llmstxt.org-style index of public surface, with descriptions: # https://halvrencapital.com/llms.txt # concatenated long-form text bundle (founding memo + checklist + operator notes # + recent digest entries) for LLM ingestion: # https://halvrencapital.com/llms-full.txt