关于python爬虫相关问题。

依旧沉沉

想利用python爬取一个网站的图标，网站地址：https://flaticons.net/customize.php?dir=Application&icon=View-Incident.png
爬到二级页面了，爬三级页面的时候就没头绪了，如下图红框标记的地方，这个是down页面，但是源代码里面没有该地址（但是手动点击会进入三级页面），是该地址js加密了么，有没有破解思路~求大佬指点一二

lql3344521aaa.

https://flaticons.net/icon.php?slug_category=application&slug_icon=view-incident
这个图标吗？

fanvalen

分析不了js就直接分析js请求的网页链接，也就是f12抓包

pzx521521

没有js加密 看html  是一个form, 直接跟

[Asm] 纯文本查看 复制代码

curl "https://flaticons.net/customize.php?dir=Application&icon=View-Incident.png" ^
  -H "authority: flaticons.net" ^
  -H "pragma: no-cache" ^
  -H "cache-control: no-cache" ^
  -H "sec-ch-ua: ^\^" Not A;Brand^\^";v=^\^"99^\^", ^\^"Chromium^\^";v=^\^"90^\^", ^\^"Google Chrome^\^";v=^\^"90^\^"" ^
  -H "sec-ch-ua-mobile: ?0" ^
  -H "upgrade-insecure-requests: 1" ^
  -H "origin: [url=https://flaticons.net]https://flaticons.net[/url]" ^
  -H "content-type: application/x-www-form-urlencoded" ^
  -H "user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36" ^
  -H "accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9" ^
  -H "sec-fetch-site: same-origin" ^
  -H "sec-fetch-mode: navigate" ^
  -H "sec-fetch-user: ?1" ^
  -H "sec-fetch-dest: document" ^
  -H "referer: [img]https://flaticons.net/customize.php?dir=Application&icon=View-Incident.png[/img]" ^
  -H "accept-language: zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7" ^
  -H "cookie: PHPSESSID=p70e3ci2v3j0s2fegn1grok0jn; __gads=ID=758a2093593297c7-226b12d5eac8003a:T=1622021420:RT=1622021420:S=ALNI_MYzuXb51WUtJlauQZ2kSlJstFsUEw" ^
  --data-raw "background=dark&icon_size=256&icon_color=^%^23FFFFFF&icon_rotate=0&icon_flip=n&shape_id=0&shape_size=512&shape_color=^%^23FFFFFF" ^
  --compressed

跟一下很清楚, 往这个地址

[Asm] 纯文本查看 复制代码

https://flaticons.net/customize.php?dir=Application&icon=View-Incident.png

里面post
加数据 "background=dark&icon_size=256&icon_color=^%^23FFFFFF&icon_rotate=0&icon_flip=n&shape_id=0&shape_size=512&shape_color=^%^23FFFFFF"
会返回一个302 location 里面有地址

[Asm] 纯文本查看 复制代码

https://flaticons.net/custom.php?i=v4ETz7TPwnEXizIQIeIOvcWgv5YiE

Ercilan

确实，是提交一个表单。楼主可以稍微百度学习一下 HTML 的 form。

lifeixue

[Python] 纯文本查看 复制代码

import requests
from lxml import etree
import os
head = {
    "User-Agent": "Mozilla/5.0(Windows NT 10.0;Win64;x64) AppleWebKit/537.36(KHTML, likeGecko) Chrome/71.0.3578.98 Safari/537.36"
}
# 创建一个文件夹(用于存储图片)
if not os.path.exists("./icons"):  # 如果images文件夹不存在
    os.mkdir("./icons")  # 创建文件夹

for i in range(1, 2):
    url = "https://flaticons.net/category.php?c=Application&p={}".format(i)
    r = requests.get(url=url, headers=head).text
    html = etree.HTML(r)
    lst = html.xpath("//div[@class='row']/div/a/@href")
    for col in lst:
        data = {
            "background": "dark",
            "icon_size": "256",
            "icon_color":  "# FFFFFF",
            "con_rotate": 0,
            "icon_flip": "n",
            "shape_id": 0,
            "shape_size": 512,
            "shape_color":  "# FFFFFF",
        }
        baseurl = "https://flaticons.net"+col
        response = requests.post(url=baseurl, headers=head, data=data).content.decode("UTF-8")
        tree = etree.HTML(response)
        down = "https://flaticons.net"+tree.xpath("//div[@class='input-group']/button[2]/@data-value")[0]
        # 图标名称
        icon_name = tree.xpath("//section[@id='home']//p/b/text()")[0].replace(" ", "-")+".png"  # icon名称
        icon_data = requests.get(url=down, headers=head).content
        icon_path = "icons/" + icon_name
        with open(icon_path, "wb") as fp:
            fp.write(icon_data)
            print(icon_name, "下载成功")

我也是初学python,这是我写的源代码

刘样andholiday

楼主爬到二级页面的时候可以用下f12

618

import requests
url = "https://flaticons.net/customize.php?dir=Application&icon=View-Incident.png"
form = {
    "background": "dark",
    "icon_size": "256",
    "icon_color": "#FFFFFF",
    "icon_rotate": "0",
    "icon_flip": "n",
    "shape_id": "0",
    "shape_size": "512",
    "shape_color": "#FFFFFF"
}
r = requests.post(url,data = form, allow_redirects=False)
target_url = r.headers['Location']

target_url就是跳转的地址

johngao

lifeixue 发表于 2021-5-26 20:38
[mw_shl_code=python,true]import requests
from lxml import etree
import os

老哥怎么学的python，我自学可达不到你这个程度

毕竟花了19元

LZ大大，您好，您之前做的“批量修改v3.0”那个软件有个问题向您请教下，就是我需要改名的文件中，都有逗号，比如：“2021-06-03 10-31-39 (B,Radius8,Smoothing4).jpg” ，逗号是英文下的逗号，导出表格的时候一切都正常，原始路径及命名里显示正常，但是修改完表格导入软件以后，逗号就变成了英文下的分号，比如：“2021-06-03 10-31-39 (B;Radius8;Smoothing4).jpg”，导致源文件无法通过路径查找到，不能进行批量修改。之前的原帖已经关闭评论了，只能在您最近的帖子里给您回复，问问您有没有什么解决方式。