How to loop over nodes with xmlfeed using scrapy python
- by Kour ipm
Hi i working on scrapy and trying xml feeds first time, below is my code
class TestxmlItemSpider(XMLFeedSpider):
name = "TestxmlItem"
allowed_domains = {"http://www.nasinteractive.com"}
start_urls = [
"http://www.nasinteractive.com/jobexport/advance/hcantexasexport.xml"
]
iterator = 'iternodes'
itertag = 'job'
def parse_node(self, response, node):
title = node.select('title/text()').extract()
job_code = node.select('job-code/text()').extract()
detail_url = node.select('detail-url/text()').extract()
category = node.select('job-category/text()').extract()
print title,";;;;;;;;;;;;;;;;;;;;;"
print job_code,";;;;;;;;;;;;;;;;;;;;;"
item = TestxmlItem()
item['title'] = node.select('title/text()').extract()
.......
return item
result:
File "/usr/lib/python2.7/site-packages/Scrapy-0.14.3-py2.7.egg/scrapy/item.py", line 56, in __setitem__
(self.__class__.__name__, key))
exceptions.KeyError: 'TestxmlItem does not support field: title'
Totally there are 200+ items so i need to loop over and assign the node text to item
but here all the results are displaying at once when we print, actually how can we loop over on nodes in scraping xml files with xmlfeedspider
