About the SpecFlint crawler
If you found this page in your server logs, our crawler visited your site. This page explains what it does and how to reach us.
What it collects
SpecFlint reads published product pages and specification documents from manufacturer websites so that buyers can compare product specifications in one place. It does not collect personal data and it does not fill in forms or sign in.
How it identifies itself
Plain requests carry the user agent SpecExtractBot/1.0 (+https://specai-three.vercel.app/bot). Some pages are rendered in a browser so that product details loaded by scripts can be read; those requests look like an ordinary browser visit.
How it behaves
One request at a time per site, with a delay between requests that grows whenever a site asks for a pause. Daily limits cap how much any site is visited.
Crawl-delay directives in robots.txt are honored up to 10 seconds by default; an administrator may set a different pace for a site. Disallow rules are honored on request for any site; write to us and we will exclude your paths or your whole site.
How to opt out or reach us
Email support@flection.io with your domain. Exclusions take effect within one business day.