Creating Rules for your Spider - Scrapy Tutorial Series Part#7

The Web is a large place with all kinds of different components, sections and subsections. Because of it's size and complexity, we often need to create a set of Rules for our Scrapy spider to follow while scraping.
Website Link ###
coderslegacy.c...
Related Links ###
CSS Selectors: • CSS Selectors - Scrapy...
XPath Expressions: • XPath Selectors - Scra...
Link Following: • Automated Link Followi...
Items & Pipelines: • Items, Pipelines and P...
Spider Rules: • Creating Rules for you...
User Agents: • Custom User Agents - S...
Rotating IPs: • Rotating Proxies (IPs)...
Login Automation: • Automating your Logins...
Bypassing Website Security: • Staying Undetected and...
Measuring Performance: • Measuring Performance ...
Cookies: • Cookies - Scrapy Tutor...
Scraping an e-commerce site (p1): • Scraping an Ecommerce ...
Scraping an e-commerce site (p2): • Scraping an Ecommerce ...

[@tobiasdudziak9469]

If you use the big O flag you do not need to delete the json file every time.

[@coderslegacy5661]

Good suggestion!