Description

FetchInfo contains the fetch information: the duration of the fetch, the returned status code, whether or not it was a HEAD request, and whether or not it was a robots.txt request.

FetchInfo is referenced in 1 repository

github.com/PuerkitoBio/gocrawl