Scraping Coin Gecko With Pandas Report Http Error 403

I have used pd.read_html(url) to scrape tables from url before without problems but with www.coingecko.com I got a HTTPError: HTTP Error 403: Forbidden. Do I really need access per

Solution 1:

You can behave like you are a browser and send request to get the body of the response.

import pandas as pd
import requests
url = "https://www.coingecko.com/en?page=1"
header = {
  "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36",
  "X-Requested-With": "XMLHttpRequest"
}
r = requests.get(url, headers=header)
dfs = pd.read_html(r.text)
df = pd.concat(dfs)
df.head()

Output