python - Scrapy: How to output to csv without column headings? -


i'm outputting scrape .csv, doing multiple instances of same spider same .csv. causes header repeated, causes problems on database insert.

i found here how rid of headers altogether, solution eliminating them if there's content in file doesn't seem work. i'd comment there, reputation isn't high enough.

does know how can print out column headers file once?

i've tried testing solution, , seems args[0].tell() equal 0, never enters if statement. seems close, have no idea how tell if there content in file.

i change code mentioned answer following:

from scrapy.exporters import csvitemexporter

class headlesscsvitemexporter(csvitemexporter):      def __init__(self, *args, **kwargs):          # args[0] (opened) file handler         # if file not empty skip headers         if os.fstat(args[0].fileno()).st_size > 0:             kwargs['include_headers_line'] = false          super(headlesscsvitemexporter, self).__init__(*args, **kwargs) 

alternatively can without using os module:

from scrapy.exporters import csvitemexporter

class headlesscsvitemexporter(csvitemexporter):      def __init__(self, *args, **kwargs):          # args[0] (opened) file handler         # if file not empty skip headers         args[0].seek(0,2)         if args[0].tell() > 0:             kwargs['include_headers_line'] = false          super(headlesscsvitemexporter, self).__init__(*args, **kwargs) 

in seek function first argument offset seek (in case 0 nothing) , second argument 2 means should start seeking end of file. result file handler position jumps end of file , tell works expected.


Comments

Popular posts from this blog

Ansible - ERROR! the field 'hosts' is required but was not set -

customize file_field button ruby on rails -

SoapUI on windows 10 - high DPI/4K scaling issue -