某视频爬虫批量下载

话痨司机啊

链接地址：aHR0cDovL2YxMDIwLndvcmthcmVhNS5saXZlL3YucGhwP2NhdGVnb3J5PWhvdCZ2aWV3dHlwZT1iYXNpYyZwYWdlPQ==
测试过，可以批量下载！！需要挂Dai_Li
ffmpeg下载链接和M3U8下载器说明：https://github.com/hecoter/m3u8download_hecoter

[Python] 纯文本查看 复制代码

"""
        ************************************
          Description: 好东西，放在这里
          Author: @
          Github: [url=https://github.com/jianala]https://github.com/jianala[/url]
          Date: 2022-06-02 13:57:24
          FilePath:on_download.py
          LastEditors: @
          LastEditTime: 2022-06-02 15:17:53
          善始者实繁，克终者盖寡。

        ************************************
"""

from lxml import etree
import re
import requests
import os
import urllib3
from m3u8download_hecoter import m3u8download
import base64

urllib3.disable_warnings()


# 
# 是否配置代{过}{滤}理，国内访问速度较慢
def proxy_set():
    proxy_set = input('Do you want to use proxy?[y/n]')
    if proxy_set == 'y':
        global my_proxies
        proxies_set = input('input your proxy config ep:"127.0.0.1:7890"')
        my_proxies = {"http": "http://127.0.0.1:7890", "https": "https://127.0.0.1:7890"}
        if proxies_set != '':
            my_proxies['http'] = 'http://'+proxies_set
            my_proxies['https'] = 'https://'+proxies_set

    elif proxy_set == 'n':
        my_proxies = ''
    else:
        proxy_set()

def get_well(response):
    '''
    获取m3u8下载号码
    '''
    try:
        et = etree.HTML(response.text)
        well_list = et.xpath('//div[@class="thumb-overlay"]/@id')[:-1]
        well_title = et.xpath('//div[@class="thumb-overlay"]/../span/text()')[:-1]
    except:
        print('缺少参数')
    finally:    
        m3u8_list = [{'m3u8url':'https://la.killcovid2021.com/m3u8/{num}/{num}.m3u8'.format(num=re.findall('\d+',well_list)[0]),'title':well_title} for well_title,well_list in zip(well_title,well_list)]

    return m3u8_list

# 爬虫主体，flag为页码
def spider(flag):
    # 如果连接访问不了，在这里把base_url替换成你知道的标准地址
    page_url = b'aHR0cDovL2YxMDIwLndvcmthcmVhNS5saXZlL3YucGhwP2NhdGVnb3J5PWhvdCZ2aWV3dHlwZT1iYXNpYyZwYWdlPQ=='
    page_url = base64.b64decode(page_url).decode('utf-8')
    page_url = page_url+ str(flag)
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36Name',
        'Referer': 'http://91porn.com',
        'Accept-Language': 'zh-CN,zh;q=0.9'}
    get_page=requests.get(url=page_url, headers=headers)
    m3u8_list = get_well(get_page)
    dir_download = os.path.join(os.path.dirname(__file__),'_download')
    for m3u8 in m3u8_list:
        m3u8download(m3u8url=m3u8['m3u8url'],title = m3u8['title'],enable_del=True,proxies=my_proxies,work_dir=dir_download)


if __name__ == '__main__':
    proxy_set()
    for i in range(1,4):
        spider(i)

特别注意的点有2个，第一个是m3u8download 源码没挂代{过}{滤}理，需要手动改一下，第二个urllib3的版本必须是urllib3==1.25.11

话痨司机啊

Ace803 发表于 2022-6-7 11:00
import requests.packages.urllib3
urllib3标示红线，显示为没有名称为urllib3的模块，后续也导致reques ...

[Python] 纯文本查看 复制代码

import urllib3
urllib3.disable_warnings()

Ace803

话痨司机啊 发表于 2022-6-7 11:20
不给你们提示怎么知道这是一个比较符合大家口味的批量下载器~

挂了dl，选择y或者n都是同样的错误
Traceback (most recent call last):
  File "D:\python******91爬取.py", line 70, in <module>
    spider(i)
  File "D:\python******91爬取.py", line 63, in spider
    m3u8download(m3u8url=m3u8['m3u8url'], title=m3u8['title'], enable_del=True, proxies=my_proxies,
TypeError: m3u8download() got an unexpected keyword argument 'proxies'

Ace803

话痨司机啊 发表于 2022-6-7 10:56
改成requests.urllib3
代码里
[mw_shl_code=python,true]requests.urllib3.disable_warnings() # 防止H ...

import requests.packages.urllib3
urllib3标示红线，显示为没有名称为urllib3的模块，后续也导致requests.packages.urllib3.disable_warnings()有问题
py是3.10，requests库是2.27.1

lc123cctv

m3u8download_hecote这个是本地库吗

iawyxkdn8

但凡看到代{过}{滤}理，我一般就路过~~~~~~~~~~~~~

chengccc

Referer亮眼

hack528

91真是厉害。。居然通过了

福森108

厉害，谢谢分享

没人用的id

厉害&#128077;

武汉小Liz

这个收藏了

XiaoZouYu

iawyxkdn8 发表于 2022-6-2 16:51
但凡看到代{过}{滤}理，我一般就路过~~~~~~~~~~~~~

没这么绝对嘛

kds0221

感谢分享。