Search The Frequency Of Words In The Sub Pages Of A Webpage Using Python

I seek help as I am stuck on how to crawl each and every link (pages or sub pages) in a webpage and find the frequency of any word. I used beautiful soup for scraping but I don't t

Solution 1:

This is what you need to access the href attribute for every link in the page.

import requests
from bs4 import BeautifulSoup

url = "https://www.servicenow.com/solutions-by-category.html"
serviceNow_r = requests.get(url)
sNow_soup = BeautifulSoup(serviceNow_r.text, 'html.parser')

for anchor in sNow_soup.find_all('a', href=True):
    print(anchor['href'])

Solution 2:

You are searching for an href tag. This is wrong!

You should search for an a tag then get the href attribute. This is the url of the linked page.