i need parse child elements in parent elements on page.
create list of articles on page
article_elements = driver.find_elements_by_tag_name('article')
and after tying child elements in loop , append results list
for article in article_elements: title = article.find_element_by_xpath('//article/h2').text share_count = article.find_element_by_xpath('//footer/div/a/span').text poinst = article.find_element_by_xpath('//footer/div[2]/div[1]/div[3]').text meta_info_list.append({'title':title, 'share count':share_count, 'points':poinst})
after loop ends got 40 times 1 same article meta (of first article)
{'share count': u'66', 'points': u'53 points', 'title': u'25+ random acts of genius vandalism'} {'share count': u'66', 'points': u'53 points', 'title': u'25+ random acts of genius vandalism'} {'share count': u'66', 'points': u'53 points', 'title': u'25+ random acts of genius vandalism'} {'share count': u'66', 'points': u'53 points', 'title': u'25+ random acts of genius vandalism'} ... 40 times
my whole code
# coding: utf8 selenium import webdriver selenium.webdriver.common.keys import keys import time driver = webdriver.chrome() driver.set_window_size(1024,768) driver.get('http://www.boredpanda.com/') time.sleep(2) meta_info_list = [] article_elements = driver.find_elements_by_tag_name('article') article in article_elements: title = article.find_element_by_xpath('//article/h2').text share_count = article.find_element_by_xpath('//footer/div/a/span').text poinst = article.find_element_by_xpath('//footer/div[2]/div[1]/div[3]').text meta_info_list.append({'title':title, 'share count':share_count, 'points':poinst}) list in meta_info_list: print(list)
the xpath expression in loop has start dot context-specific:
for article in article_elements: title = article.find_element_by_xpath('.//article/h2').text share_count = article.find_element_by_xpath('.//footer/div/a/span').text poinst = article.find_element_by_xpath('.//footer/div[2]/div[1]/div[3]').text meta_info_list.append({'title':title, 'share count':share_count, 'points':poinst})
as side note, can shorten code using list comprehension:
meta_info_list = [{ 'title': article.find_element_by_xpath('.//article/h2').text, 'share count': article.find_element_by_xpath('.//footer/div/a/span').text, 'points': article.find_element_by_xpath('.//footer/div[2]/div[1]/div[3]').text } article in article_elements]
Comments
Post a Comment