Rate Limiting

The rate config option allows you to specify at which rate the scraper should send out requests. The rate is measured in Requests per Minute (RPM).

When no rate is specified, rate limiting is disabled and the scraper will send out requests as fast as it can.

export const options = {
  url: "http://example.com/",
  rate: 100,
};