View definition

(*DefaultExtender).ComputeDelay

Defined in github.com/PuerkitoBio/gocrawl/ext.go

Description

ComputeDelay returns the delay specified in the Crawler's Options, unless a crawl-delay is specified in the robots.txt file, which has precedence.

ComputeDelay is referenced in 0 repositories