python - Scrapy: How to output to csv without column headings? -
i'm outputting scrape .csv, doing multiple instances of same spider same .csv. causes header repeated, causes problems on database insert.
i found here how rid of headers altogether, solution eliminating them if there's content in file doesn't seem work. i'd comment there, reputation isn't high enough.
does know how can print out column headers file once?
i've tried testing solution, , seems args[0].tell() equal 0, never enters if statement. seems close, have no idea how tell if there content in file.
i change code mentioned answer following:
from scrapy.exporters import csvitemexporter
class headlesscsvitemexporter(csvitemexporter): def __init__(self, *args, **kwargs): # args[0] (opened) file handler # if file not empty skip headers if os.fstat(args[0].fileno()).st_size > 0: kwargs['include_headers_line'] = false super(headlesscsvitemexporter, self).__init__(*args, **kwargs)
alternatively can without using os
module:
from scrapy.exporters import csvitemexporter
class headlesscsvitemexporter(csvitemexporter): def __init__(self, *args, **kwargs): # args[0] (opened) file handler # if file not empty skip headers args[0].seek(0,2) if args[0].tell() > 0: kwargs['include_headers_line'] = false super(headlesscsvitemexporter, self).__init__(*args, **kwargs)
in seek
function first argument offset seek (in case 0
nothing) , second argument 2
means should start seeking end of file. result file handler position jumps end of file , tell
works expected.
Comments
Post a Comment