Здравствуйте пытаюсь сохранить результат парсинга в CSV файл с помощью Pandas, но выкидывает ошибку:
Traceback (most recent call last):
File "D:/Git/sdvk-oboi.ru/sdvk-oboi.ru.py", line 183, in crawl
urls = await future
File "C:\ProgramData\Anaconda3\lib\concurrent\futures\thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "D:/Git/sdvk-oboi.ru/sdvk-oboi.ru.py", line 173, in parse
pd.DataFrame(all).to_csv(r'products.csv', index=False, encoding='utf-8')
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py", line 348, in __init__
mgr = self._init_dict(data, index, columns, dtype=dtype)
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py", line 459, in _init_dict
return _arrays_to_mgr(arrays, data_names, index, columns, dtype=dtype)
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py", line 7356, in _arrays_to_mgr
index = extract_index(arrays)
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py", line 7402, in extract_index
raise ValueError('arrays must all be same length')
ValueError: arrays must all be same length
Скорее всего это происходит из-за попытки сохранить этот текст в одно из полей (условие заказчика: весь с тегами)
<ul class="list-options">
<li itemprop="ProductModel" itemscope itemtype="http://schema.org/ProductModel">
<table class="attr-line-table"><tr>
<td class="first"><label>Артикул:</label></td>
<td><span itemprop="name">5438</span></td>
</tr></table>
</li>
<li itemprop="brand" itemscope itemtype="http://schema.org/Brand">
<table class="attr-line-table"><tr>
<td class="first"><label>Производитель:</label></td>
<td><a itemprop="name" href="/oboi/Zambaiti_Parati/">Zambaiti Parati</a></td>
</tr></table>
</li>
<li>
<table class="attr-line-table"><tr>
<td class="first"><label>Страна:</label></td>
<td><a itemprop="name" href="/oboi/italiya/">Италия</a></td>
</tr></table>
</li>
<li itemprop="additionalProperty" itemscope itemtype="http://schema.org/PropertyValue">
<table class="attr-line-table"><tr>
<td class="first"><label itemprop="name">Ширина рулона:</label></td>
<td><span itemprop="value" id="a1val">106 см</span></td>
</tr></table>
</li>
<li itemprop="additionalProperty" itemscope itemtype="http://schema.org/PropertyValue">
<table class="attr-line-table"><tr>
<td class="first"><label itemprop="name">Длина рулона:</label></td>
<td><span itemprop="value" id="a2val">10.05 м</span></td>
</tr></table>
</li>
<li itemprop="additionalProperty" itemscope itemtype="http://schema.org/PropertyValue">
<table class="attr-line-table"><tr>
<td class="first"><label itemprop="name">Основа:</label></td>
<td><span itemprop="value" id="a3val">Флизелин</span></td>
</tr></table>
</li>
<li itemprop="additionalProperty" itemscope itemtype="http://schema.org/PropertyValue">
<table class="attr-line-table"><tr>
<td class="first"><label itemprop="name">Материал:</label></td>
<td><span itemprop="value" id="a4val">Винил</span></td>
</tr></table>
</li>
<!-- a65 wuz here -->
</ul>
Как можно исправить? Код:
temp = []
try:
name = html.cssselect("h1[itemprop='name']")[0].text
except:
name = None
temp.append(["Название", name])
# print(name)
try:
price = html.cssselect('#item-price')[0].text
except:
price = None
temp.append(["Цена", price])
# print(price)
ul = html.cssselect('ul.list-options')[0]
print_html(ul)
temp.append(["Характеристики", ul])
try:
url_main_photo = "http:"+html.cssselect("div.main-photo ")[0].get("data-fancybox-href")
except:
url_main_photo = None
# print(url_main_photo)
temp.append(["Основная картинка", url_main_photo])
download_image(url_main_photo)
urls_image_list = []
block_images = html.cssselect('#collection_photo>div>a')
for block in block_images:
urls_image_list.append("http:" + block.get("href"))
# print(urls_image_list)
temp.append(["Изображения", urls_image_list])
all = (dict(temp))
pd.DataFrame(all).to_csv(r'products.csv', index=False, encoding='utf-8')