module documentation

Set User-Agent header per spider or use a default value from settings

Class UserAgentMiddleware This middleware allows spiders to override the user_agent