python selenium find child elements in loop -


i need parse child elements in parent elements on page.

create list of articles on page

article_elements = driver.find_elements_by_tag_name('article') 

and after tying child elements in loop , append results list

for article in article_elements:     title = article.find_element_by_xpath('//article/h2').text     share_count = article.find_element_by_xpath('//footer/div/a/span').text     poinst = article.find_element_by_xpath('//footer/div[2]/div[1]/div[3]').text     meta_info_list.append({'title':title, 'share count':share_count, 'points':poinst}) 

after loop ends got 40 times 1 same article meta (of first article)

{'share count': u'66', 'points': u'53 points', 'title': u'25+ random acts of genius vandalism'} {'share count': u'66', 'points': u'53 points', 'title': u'25+ random acts of genius vandalism'} {'share count': u'66', 'points': u'53 points', 'title': u'25+ random acts of genius vandalism'} {'share count': u'66', 'points': u'53 points', 'title': u'25+ random acts of genius vandalism'} ... 40 times 

my whole code

 # coding: utf8 selenium import webdriver selenium.webdriver.common.keys import keys import time  driver = webdriver.chrome() driver.set_window_size(1024,768) driver.get('http://www.boredpanda.com/')  time.sleep(2)  meta_info_list = []  article_elements = driver.find_elements_by_tag_name('article')  article in article_elements:     title = article.find_element_by_xpath('//article/h2').text     share_count = article.find_element_by_xpath('//footer/div/a/span').text     poinst = article.find_element_by_xpath('//footer/div[2]/div[1]/div[3]').text     meta_info_list.append({'title':title, 'share count':share_count, 'points':poinst})  list in meta_info_list:     print(list) 

the xpath expression in loop has start dot context-specific:

for article in article_elements:     title = article.find_element_by_xpath('.//article/h2').text     share_count = article.find_element_by_xpath('.//footer/div/a/span').text     poinst = article.find_element_by_xpath('.//footer/div[2]/div[1]/div[3]').text     meta_info_list.append({'title':title, 'share count':share_count, 'points':poinst}) 

as side note, can shorten code using list comprehension:

meta_info_list = [{     'title': article.find_element_by_xpath('.//article/h2').text,     'share count': article.find_element_by_xpath('.//footer/div/a/span').text,     'points': article.find_element_by_xpath('.//footer/div[2]/div[1]/div[3]').text } article in article_elements] 

Comments