Skip to Content

Our Crawler

AlphaScoutBot

What our crawler reads, the rules it follows, and how to block it or have your data removed.

Effective 9 October 2026

AlphaScout is preparing to launch. Our registered business details and contact e-mail will be published here before we accept payments. Until then, privacy requests can be made through the request form.

01What It Is

AlphaScoutBot is the crawler behind AlphaScout, a database of newly launched startups for investors. It reads public pages about new companies: launch platforms and product directories, accelerator and fund portfolio pages, official business registries and filings, public job boards, startup news, and the public pages of the companies' own websites.

It identifies itself with this User-Agent:

Mozilla/5.0 (compatible; AlphaScoutBot/0.1; +https://tryalphascout.com/bot)

The version number changes over time; match on AlphaScoutBot.

02How It Behaves

  • It reads your robots.txt before anything else and follows it, including Crawl-delay. Rules for AlphaScoutBot apply first, then those for all agents (*). robots.txt is re-read at least once a day.
  • If your robots.txt returns a server error or can't be reached, it treats the whole site as off-limits until it can read it again.
  • By default it requests at most one page per second from a site, and slows down further when your server answers 429 or 503, honouring Retry-After.
  • It reads public pages only. It never logs in, never gets past a paywall, and never tries to get around bot protection or CAPTCHAs.
  • It doesn't keep e-mail addresses or phone numbers: they are removed from everything it stores. What we collect, and why, is set out in our Privacy Policy.

03How to Block It

Add this to the robots.txt at the root of your site to keep AlphaScoutBot out entirely:

User-agent: AlphaScoutBot Disallow: /

Or keep it out of part of the site, or slow it down:

User-agent: AlphaScoutBot Disallow: /private/ Crawl-delay: 10

The change takes effect the next time the bot reads your robots.txt, within a day.

04Removing Your Data

Blocking the bot stops new visits but doesn't remove what is already in AlphaScout. To have a company or a person removed, or to object to how it appears, use the request form. A company website is the quickest way for us to find the right record.