Description

HttpClient is the default HTTP client used by DefaultExtender's fetch requests (this is thread-safe). The client's fields can be customized (i.e. for a different redirection strategy, a different Transport object, ...). It should be done prior to starting the crawler.

HttpClient is referenced in 1 repository

github.com/PuerkitoBio/gocrawl