About our data-collection bots

CyberInsight Asia uses automated programs to read public web pages, to check the status of laws and to collect public security advisories. They read public pages only; they do not log in or bypass paywalls or access restrictions.

The bots send a User-Agent of the form CyberInsightAsia-role/0.1 (+mailto:contact@cyberinsight.info), which includes a contact address.

The bots read and follow a site's robots.txt first. If robots.txt cannot be read or returns a server error, they treat access as disallowed and do not continue.

The bots rate-limit requests per host and honour a site's declared Crawl-delay; when access is denied or throttled they stop rather than work around it.

To ask us to stop accessing your site, write to contact@cyberinsight.info or say so in your robots.txt.