Filtering into specific class in BeautifulSoup

Home / Uncategorized / Filtering into specific class in BeautifulSoup

Question:
I was doing something like this using BeautifulSoup:for name in soup.find_all(‘div’,’name’):
When I am using this filter, my understanding is that I will get all the div tags with the attribute or class name name.

However, I don’t want every instance of the div tag with the attribute name. I want certain instances in which they locate in a certain subtree of the HTML file. More specifically, instances within the tag , which is two levels above the tags I am looking for. So my question is, how would I write the filter in soup.find_all() to zoom into a class?

I apologize in advance if I had mixed up any terminology. This is my first time attempting web scraping. I was not sure what to look at within the documentation.

For reference, this is the website I am attempting to do web scraping on: http://pd.appbank.net/ml39


Answer:
You can’t specify a search for tags within a certain parent directly, but you can use a nested loop to get it done. First, get all ul tags with the class list-box mb-3 spacer, and then get all divs located under each one.
div_list = []

for ul in soup.find_all(‘ul’, {‘class’ : ‘list-box mb-3 spacer’}):div_list.extend(ul.find_all(‘div’, {‘class’ : ‘name’}))

print(div_list)
Read more

Leave a Reply

Your email address will not be published. Required fields are marked *