The Accessing Commented Html Lines With Beautifulsoup

I am attempting to webscrape stats from this specific webpage: https://www.sports-reference.com/cfb/schools/louisville/2016/gamelog/ However, the table for the 'Defensive Game Log'

Solution 1:

Comment object will give you what you want:

from urllib.request import Request,urlopen
from bs4 import BeautifulSoup, Comment

accessurl = 'https://www.sports-reference.com/cfb/schools/oklahoma-state/2016/gamelog/'
req = Request(accessurl)
link = urlopen(req)
soup = BeautifulSoup(link, "lxml")

comments=soup.find_all(string=lambda text:isinstance(text,Comment))
for comment in comments:
    comment=BeautifulSoup(str(comment), 'lxml')
    defensive_log = comment.find('table') #search as ordinary tag
    if defensive_log:
        break