Dealing with CloudFlare in Scrapy

Date

Problem

When you try to download resources from CloudFlare with Scrapy most of the time you'll get HTTP 403 error. This is CloudFlares' so-called "I'm under attack" mode, when they're restricting downloading resources for "plain" access (like curl http://resource_url).

Solution

Step 1: Cookie

First of all ...

more ...