Questions about scrapy
Read more about scrapy
python (12.9k questions)
javascript (9.2k questions)
reactjs (4.7k questions)
java (4.2k questions)
java (4.2k questions)
c# (3.5k questions)
c# (3.5k questions)
html (3.3k questions)
Scrapy Error Handling for 404,403,301,302
I am trying to scrape data from a list of urls. And for the urls which are not giving response status code 200 I do need some other data.
settings.py
HTTPERROR_ALLOWED_CODES = [404,403,301,302]
Code
...
imhans4305
Votes: 0
Answers: 1
How can I pass items with meta in a 3 page level spider?
How do I pass items using meta in a 3 page level spider?
What am I missing? How to create a new item for every iteration?
Here is the code:
import scrapy
class GsmSpider(scrapy.Spider):
name = ...
Alexander
Votes: 0
Answers: 1
Scrapy rows output is in the wrong order
How can I manipulate the ORDER of the ROWS - or simply output them in the same order as they appear on the website?
(I cannot output the results according to the order on the website in the .csv file....
Alexander
Votes: 0
Answers: 1
Pagination on a website where the URL stays the same python, scrapy, selenium
I am scraping a website that contains companies, each page contains 26 companies, I am able to login and successfully scrape the first page of the website, but have trouble figuring out how to click t...
Berci Vagyok
Votes: 0
Answers: 0