Crawler information
TimeGraphBot.
TimeGraphBot is the planned crawler for TimeGraph, a project in development that researches publicly accessible information and builds a structured temporal history of public data.
How TimeGraphBot is intended to operate
- Visits publicly accessible web pages and resources needed for the project.
- Does not authenticate or attempt to bypass access controls.
- Does not bypass CAPTCHA or web application firewalls (WAF).
- Respects
robots.txtdirectives. - Uses intentional rate limits to avoid unnecessary load.
- Accepts exclusion and opt-out requests from website operators.
Identification
User-Agent
Planned User-Agent string:
TimeGraphBot/0.3 (+https://timegraph.it/crawler)
The URL in this identifier is the planned crawler information address. This page is available on GitHub Pages while the custom domain is not yet configured.
Contact / Opt-out
Request an exclusion
Website operators can request that TimeGraphBot exclude their site or specific resources. Please include the relevant URL and your request.
crawler@timegraph.it