Scrapy spider is not working

Posted by Zeynel on Stack Overflow See other posts from Stack Overflow or by Zeynel
Published on 2009-11-27T05:51:29Z Indexed on 2010/05/22 3:10 UTC
Read the original article Hit count: 450

Filed under:
|

Since nothing so far is working I started a new project with

python scrapy-ctl.py startproject Nu

I followed the tutorial exactly, and created the folders, and a new spider

from scrapy.contrib.spiders import CrawlSpider, Rule
from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor
from scrapy.selector import HtmlXPathSelector
from scrapy.item import Item
from Nu.items import NuItem
from urls import u

class NuSpider(CrawlSpider):
    domain_name = "wcase"
    start_urls = ['http://www.whitecase.com/aabbas/']

    names = hxs.select('//td[@class="altRow"][1]/a/@href').re('/.a\w+')

    u = names.pop()

    rules = (Rule(SgmlLinkExtractor(allow=(u, )), callback='parse_item'),)

    def parse(self, response):
        self.log('Hi, this is an item page! %s' % response.url)

        hxs = HtmlXPathSelector(response)
        item = Item()
        item['school'] = hxs.select('//td[@class="mainColumnTDa"]').re('(?<=(JD,\s))(.*?)(\d+)')
        return item

SPIDER = NuSpider()

and when I run

C:\Python26\Scripts\Nu>python scrapy-ctl.py crawl wcase

I get

[Nu] ERROR: Could not find spider for domain: wcase

The other spiders at least are recognized by Scrapy, this one is not. What am I doing wrong?

Thanks for your help!

© Stack Overflow or respective owner

Related posts about python

Related posts about scrapy