Crawler access
VisdendaAuditBot
The crawler behind Visdenda site audits — what it does, how to allow it, and how to block it.
What it fetches
VisdendaAuditBot fetches pages of sites that a Visdenda user has added and verified, for technical SEO audits: on-demand runs, the user's own audit schedules, and sitemap discovery for those sites. It only crawls sites inside a paying-or-free user's verified workspace — it does not follow links into unrelated sites or crawl the open web.
How it identifies itself
Requests are sent with the User-Agent below. Pages are fetched through our rendering infrastructure, which applies the same User-Agent on every lane (plain fetch, headless browser, or edge browser) — so a firewall rule on this string covers the whole audit.
User-Agent
VisdendaAuditBot/1.0
robots.txt
Page crawls honor robots.txt: a Disallow rule for VisdendaAuditBot (or for all user agents) keeps those pages out of audits, and the audit reports them as blocked instead of fetching them anyway.
Allowing it (faster audits)
Sites behind a firewall or CDN bot protection (for example Cloudflare) are still audited — challenged pages fall back to full browser rendering, which works but is slower. Allowing the User-Agent above in your WAF or bot rules skips that fallback and speeds every audit up.
Blocking it
Add a robots.txt rule for VisdendaAuditBot, or block the User-Agent in your firewall — audits will report those pages as blocked rather than fetching them. You can also remove the site from your Visdenda workspace to stop all crawling immediately.
Questions
Write to support@visdenda.com.