CRAWL_DELAY #14
Description
Not really the best way to avoid incrementing nb_crawl because we can restart the crawler 2 weeks ago and then we mustn't increment nb_crawl. This is not only a delay problem.
{{ message }}
Not really the best way to avoid incrementing nb_crawl because we can restart the crawler 2 weeks ago and then we mustn't increment nb_crawl. This is not only a delay problem.