Output Python To Csv Regular

hello i'm new on python/scrapy world, i need to export my list of products to csv like this exemple: what i want but i get this one: what i got ///// spider: ///// import scrapy im

Solution 1:

You don't need to create the csv file yourself when parsing items, scrapy can export by default to a csv file.

so change your parse method to:

def parse(self, response):
    for sel in response.xpath('//*[contains(@class, "ajax_block_product")]'):
        item = EscrapItem()
        item['revendeur'] = '\n'.join(sel.xpath('//*[contains(@class, "center_block")]/h2/a/@href').re('tunisianet'))
        item['produit'] = '\n'.join(sel.xpath('//*[contains(@class, "center_block")]/h2/a/text()').extract())
        item['lien'] = '\n'.join(sel.xpath('//*[contains(@class, "center_block")]/h2/a/@href').extract())
        item['description'] = '\n'.join(sel.xpath('//*[contains(@class, "product_desc")]/a/text()').extract())
        item['prix'] = '\n'.join(sel.xpath('//*[contains(@class, "price")]/text()').extract())
        yield item

later when calling scrapy you can call it with:

scrapy crawl myspider -o output.csv

Now you have all your items exported to a csv file.

If you still want to control it on your own pipeline, check here to create your own exporter. It would like this:

from scrapy import signals
from scrapy.exporters import CsvItemExporter

classCSVExportPipeline(object):

    def__init__(self):
        self.files = {}

     @classmethoddeffrom_crawler(cls, crawler):
         pipeline = cls()
         crawler.signals.connect(pipeline.spider_opened, signals.spider_opened)
         crawler.signals.connect(pipeline.spider_closed, signals.spider_closed)
         return pipeline

    defspider_opened(self, spider):
        file = open('%s_products.csv' % spider.name, 'w+b')
        self.files[spider] = file
        self.exporter = CsvItemExporter(file)
        self.exporter.start_exporting()

    defspider_closed(self, spider):
        self.exporter.finish_exporting()
        file = self.files.pop(spider)
        file.close()

    defprocess_item(self, item, spider):
        self.exporter.export_item(item)
        return item

To create your own pipeline make sure to read this entirely.

Solution 2:

You should probably set the cell where you want to write you data. Something like:

worksheet.write('A1','thing you want to write')

Or it may be default to write content in 'A'

Solution 3:

it export but not with the form i want, i want the form like this one : http://i.imgur.com/r8LaVem.png , but i got this one http://i.imgur.com/8IVnlui.png . here is my final class : def parse(self, response): item = TfawItem() data= [] items = [] out = open('out.csv', 'a') x = response.xpath('//*[contains(@class, "ajax_block_product")]') for i in range(0, len(x)): item['revendeur'] = response.xpath('//*[contains(@class, "center_block")]/h2/a/@href').re('tunisianet')[i] item['produit'] = response.xpath('//*[contains(@class, "center_block")]/h2/a/text()').extract()[i] item['url'] = response.xpath('//*[contains(@class, "center_block")]/h2/a/@href').extract()[i] item['description'] = response.xpath('//*[contains(@class, "product_desc")]/a/text()').extract()[i] item['prix'] = response.xpath('//*[contains(@class, "price")]/text()').extract()[i] data = item['revendeur'], item['produit'], item['url'], item['description'], item['prix'] out.write(str(data)) out.write('\n')