User's Guide

RegHub makes our database of collected regulations and legislation available for download. Each country with an associated RegData project will have their own separate pages divided by administrative level. For instance, Australia has both an "Australia Federal" and an "Australia States" page, which will contain the federal and state jurisdictions, respectively. Each page will display rows for each individual jurisdiction, with a total of six columns:

Jurisdiction

Refers to the political territory or entity from which documents were collected. For now, this refers only to the United States, but could also refer to an Australian state or a Canadian province, for instance. Jurisdiction may also refer to

Data Project

Refers to the RegData project this jurisdiction is associated with.

Project Version

Shows the version of the RegData project this jurisdiction is associated with.

Version Date

Shows the year that this version of the RegData project was collected.

Files

Will download the actual text files for that jurisdiction.

Metadata

Will request the metadata associated with these files from out API. See below for a description of available metadata.

Downloading Files

Downloading files for a jurisdiction will return two types of files: raw and cleaned. Raw files contain the text in its original format as collected from that jurisdiction, prior to any cleaning or other processing. This could be, for instance, the text scraped from a webpage, a text file, a Word document, or a PDF.

Cleaned files are the final, cleaned text that QuantGov uses for its word count, restriction counts, and other analysis. These will always be simple text files, converted from the original, raw format and cleaned. Cleaning involves stripping out unnecessary spaces, special characters, and punctuation, with the exception of periods, commas, exclamation marks, and question marks. We clean these files to make them as standardized as possible and prevent introduction of noise into text classification and other types of analysis.

You may also download the document-level metadata for each one of these jurisdictions that will provide our word counts, restriction counts, and additional information for each file included in that jurisdiction. The metadata will download as a .csv file and includes the following information:

documentName identifies the exact file in the file download for that jurisdiction that the row is associated with.
documentTitle refers to the title of the regulation itself.
documentReference provides information on the location of the document in the organizational scheme of that jurisdiction, as represented on the website where the documents were collected. For instance, Title 20, Part X, Chapter 1 means you can find this document in that particular chapter, part, and title of the jurisdiction's organization scheme for their regulations.
collectedDate identifies the date on which these documents were collected.
Total Restrictions represents the summation of all restrictive term counts.
Total Words represents the total count of all words in the document.
Terms - May Not: count of "may not" restrictive term.
Terms - Required: count of "required" restrictive term.
Terms - Must: count of "must" restrictive term.
Terms - Shall: count of "shall" restrictive term.
Terms - Prohibited: count of "prohibited" restrictive term.
jurisdictionName identifies the name of the jurisdiction.
agencyName identifies the agency, office, or department that this document is associated with.
sourceCitation identifies the citation information for this document's associated RegData project.
sourceName identifies the RegData data series that this document is included in.
sourceOrganization identifies the organization responsible for the RegData data series of this document.
sourceUrl provides a Url reference for the organization responsible for this data series, as well as a Url reference for the domain from which the text in this series was collected.
documentationUrl provides a Url reference for the RegData User's Guide where you can find additional information on these data.

Searching AI-Readable Laws & Rules

RegHub also allows users to search across our entire collection of jurisdictional texts at every administrative level, for all countries with an active RegData project. The search functionality can be accessed from the main page or through the menu. Currently, RegHub search is limited to simple keyword queries that will return results by matching against all metadata fields and the entire document text. You can filter your search by jurisdiction by clicking the dropdown menu above the search bar - this is currently limited to single jurisdiction filtering, and will search across all administrative levels for that jurisdiction (federal and state, for instance). In the future, we plan to add more filtering options using the additional document data we have available. Additionally, federal U.S. documents are currently unable to be searched, as our data series contains historical text files that would not be representative of the current state of that jurisdiction.

Search results will return on a page display with four columns telling you the document title, the reference information for that document to assist in locating the document in the jurisdiction's hierarchical code structure, the country the jurisdiction belongs to, and the name of the jurisdiction itself. You can click on the document title to bring you to a page for that specific document, which contains the associated metadata as well as the complete cleaned document text below. You can click the download button at the top to download this individual cleaned text file.

How to Connect Data and Documents

This guide is specifically designed to help individuals who get data from the Bulk Download and Custom Download pages, and documents from the Search for Documents, connect those data and documents. For example, if you downloaded a set of documents from the jurisdiction of Alberta, you might be interested in Shannon Entroy score for each document downloaded.

If you have documents, but need data. Visit either the Bulk Download or Custom Download pages and download the data for the associated jurisdiction(s) and document type(s) combination(s) that you are interested in. If you have data, but need documents. Visit the Search for Documents and download the documents for the associated jurisdiction(s) and document type(s) combination(s) that you are interested in.

QuantGov uses a unique identifier for each document in the QuantGov database called the document id. A document id only ever refers to a single document. Therefore, each data point for a specific document can also be identified via the unique document id.

Find the document id for a specific document by looking at the name of each individual .txt file (example: 2700000753.txt). Find the document id for a specific data point by looking for the document id column in the .csv download.

Read the document ids into the preferred system for analysis and merge the data. Note that there are many easy ways to extract document names from a folder. We suggest following one of these guides:

FAQs

Every time I try to log in, it returns me to the login page. What do I do?

Try closing the page, clearing all cookies for RegHub in your browser, and then reopen the page. If that doesn't work, try using a different browser - RegHub currently works best on Chrome and Microsoft Edge.

Can I download all the text files and metadata returned from a given search?

I'm trying to download metadata for a jurisdiction and it's not working.

My search results aren't refreshing.