Beautiful Soup not grabbing data from a specific item

Hello there,

I am having trouble understanding how you use beautiful soup to grab data off of a site while scraping.

Here is what I wrote to find a section on the site I am trying to crawl and scrap. But the data will not append to the data= I created above this chunk.

for element in soup.findAll(attrs={‘class’: ‘component–results-info’}):
name = element.find(’#text’)
data.append(name.text)

I think that I am going about this wrong as I am not following how beautiful soup works in this case. Anyone have any references or documentation that could help point me in the right direction?

Hello @nick20

Can you share your notebook?

Python_Scraper_No_passwords.py (5.3 KB)
Attached is my code. I had to remove some stuff for security’s sake. But the issue is trying to get a specific element to be scraped at line 115. It seems that whatever I call will not work. The only thing I have gotten to work on is putting in text=True which pulls all of the text of the page and is not useful.

1 Like

I think findAll has been deprecated. I have checked several resources and they all use find_all. Otherwise, that line is fine. find_all returns a list that is iterable.

1 Like

I will look into it thank you for the heads up!

1 Like

Documentation