Jobs have many advanced tools to assist with your scraping needs. We will cover Wait Time Between Scrapes in this section. We would recommend exploring the rest on your own to take full advantage of jobs.
To Find the tools - from the Jobs page, simply click on Advanced Options at the bottom of the page.
Wait Time Between Scrapes, is a setting to use when troubleshooting. Data Miner is a tool that runs only on your computer, which means each scrape is unique to your own Internet speed.
For that reason, if the Internet connection is running slow and Data Miner can’t scrape before the page loads, it will return a failure. To prevent this, simply increase the wait time to 30 or 60 seconds and press “Save”. By default, it’s set to 15 seconds.
Please note: never decrease the wait time below 15 seconds, which could cause the job to fail.