Skip to content Skip to sidebar Skip to footer

Python Shell Not Running Scrapy

I am running Python.org version 2.7 64 bit on Windows Vista 64 bit to use Scrapy. I have some code that is working when I run it via Command Shell (apart from some issues with Comm

Solution 1:

Add from scrapy.cmdline import execute to your imports

Then put execute(['scrapy','crawl','wiki']) and run your script.

from scrapy.spider import Spider
from scrapy.selector import Selector
from scrapy.utils.markup import remove_tags
import re
from scrapy.cmdline import execute
classMySpider(Spider):
    name = "wiki"
    allowed_domains = ["wikipedia.org"]
    start_urls = ["http://en.wikipedia.org/wiki/Asia"]

    defparse(self, response):
        titles = response.selector.xpath("normalize-space(//title)")
        for title in titles:

            body = response.xpath("//p").extract()
            body2 = "".join(body)
            print remove_tags(body2)

execute(['scrapy','crawl','wiki'])

Post a Comment for "Python Shell Not Running Scrapy"