python爬取次元岛3000多张cosplay图片

henghui2012

用今天python的parsel模块，次大概3500张图片，如下：复制代码，安装所需的库如下，tqdm是一个很不错的库，有运行能力的有效元岛，则需要添加cookie

[Python] 纯文本查看 复制代码

import requests
import re
import parsel
from tqdm import tqdm
#注意headers里面的大小写
headers={#不能爬了，就在这里添加cookie
'Referer':'http://ciyuandao.com/photo/list/0-0-1',
'User-Agent':'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Mobile Safari/537.36'
}
for i in tqdm(range(37, 247)):
try:
url=f'http://ciyuandao.com/photo/list/0-1-{i}'
response=requests.get(url=url,headers=headers)
# print(response.text)
src=re.findall('\s<img src="(.*?)">',response.text)
selector=parsel.Selector(response.text)
a=selector.css('p a ::text').getall()
# print(len(a))
# author=selector.css('p>a[href*="coser"]::text').getall()
Title=[]
for i in range(0,40,2):
Title.append(a[i]+a[i+1])
#创建文件夹
import os
# os.makedirs('次元岛1')
for i in range(0,20):
url1=src[i]
title=Title[i]
response_content=requests.get(url=url1,headers=headers).content
with open(title+'.png',mode='wb') as f:
f.write(response_content)
except FileNotFoundError as f:
pass
except OSError as o:
pass
continue

金坷垃de

感谢楼主的分享

大牛用户名

多谢楼主分享

yyzhongrl

先收藏了，万一用到呢

996533981

感谢楼主分享

z361113602

谢谢大牛

1115406423

6666

00978

谢谢分享