Python Beautiful Soup Only Scraping The Lower Part Of The Page
I'm trying to pull product info from a rather large apparel website page, but soup only appears to scrape a lower half of the html document, at an arbitrary cutoff, so the data tha
Solution 1:
As stated in one of the comments the HTML you are trying to get is added with JavaScript running on the browser.
I recommend you this package Requests-HTML created by the author of very popular requests.
from requests_html import HTMLSession
session = HTMLSession()
r = session.get('https://www.pullandbear.com/rs/man/sale-c1030036006.html')
r.html.render()
print(r.html.html)
Post a Comment for "Python Beautiful Soup Only Scraping The Lower Part Of The Page"