robots.txt
Authorization
EventManager.close
PersistState
AdaptivePlaywrightCrawler
run()
PydanticAiSelectorExtractor
extract()
BrowserPool
context.request.headers