Research the economy through its websites.
SitesGraph reads public websites and turns each one into a structured company record: what the business does, where it is, what it runs on and how it changes over time. A dataset for researchers who study companies, sectors and technology.
From a website to a research record
Read
SitesGraphBot reads the homepage and a few pages more (contact, menu, careers). It respects robots.txt and visits one page at a time.
Structure
Each site becomes one record with the same fields: whether it is a real business, its category, its location, its technologies and the quality of the site.
Follow over time
Sites are read again about once a month, so you see what changes: new businesses, closures, and the tools they adopt or drop.
Questions you can answer
Sectors
Which sectors grow in a region, and where new businesses open or disappear.
Technology adoption
Which booking tools restaurants use, which shop platforms win, how fast a new technology spreads.
Digital maturity
How findable, accessible and secure the websites of small businesses are, per sector or per city.
The web itself
How sites are hosted, how they handle email and certificates, and how often they change.
Who we work with
Universities, research groups, public bodies and journalists. We share the data for research, and we welcome partners who help with crawling, data or corrections. Website owners can always see how our crawler behaves and opt out on ethics and robots.txt.
Working on a research question?
Tell us what you study and which data you need. We answer within two working days.