吾爱破解 - 52pojie.cn

 找回密码
 注册[Register]

QQ登录

只需一步,快速开始

查看: 1942|回复: 25
收起左侧

[学习记录] 17k小说网爬取书架中的小说

[复制链接]
原野OK 发表于 2023-10-13 14:12
本帖最后由 原野OK 于 2023-10-14 08:34 编辑

#代码主要是下载的17k小说网添加到书架内的小说,只需要把账号和密码替换成自己的账号和密码就可以了,然后在pycharm里运行就可以了,不知道怎么安装pycharm的,去B站学一下,本贴,仅限交流
import requests
from lxml import etree
import os

session = requests.session()
accountNumber = input("请输入你的17K小说网账号:")
password = input("请输入你的登录密码:")
def login():
    session.post("https://passport.17k.com/ck/user/login",data={
                            "loginName": accountNumber,
                            "password": password
                    })


def get_shelf_books():
    '''

    :return:
    '''
    res = session.get("https://user.17k.com/ck/author2/shelf?page=1&appKey=2406394919")
    res.encoding="utf8"
    data = res.json().get("data")
    return data


def get_books(data):

    for bookDict in data:
        # print(bookDict)
        bookID = bookDict.get("bookId")
        bookName = bookDict.get("bookName")
        book_path = os.path.join(root_path,bookName)
        if not os.path.exists(book_path):
            os.mkdir(book_path)
        get_chapter(bookID,book_path,bookName)
def get_chapter(bookID,book_path,bookName):
    res = requests.get(f"https://www.17k.com/list/{bookID}.html")
    # print("res",res)
    res.encoding = "utf8"

    selector = etree.HTML(res.text)
    items = selector.xpath('//dl[@class="Volume"]/dd/a')
    # print(items)


    for item in items:
        chapter_href = item.xpath('./@href')[0]
        chapter_title = item.xpath('./span/text()')[0].strip()
        print("chapter_href", chapter_href)
        print("chapter_text", chapter_title)

        res = requests.get("https://www.17k.com" + chapter_href)
        res.encoding = "utf8"
        chapter_html = res.text
        print(chapter_html)
        selector = etree.HTML(res.text)
        chapter_text = selector.xpath('//div[contains(@class,"content")]/div[@class="p"]/p[position()<last()]/text()')
        # ret = selector.xpath('//div[contains(@class,"content")]/div[@class="p"]/p[position()<last()]/text()')
        print(chapter_text)
        download(book_path, chapter_title,chapter_text)

        print(f'{bookName}书籍的{chapter_title}章节已经下载完成')

def download(book_path, chapter_title,chapter_text):
    chapter_path = os.path.join(book_path, chapter_title)
    with open(chapter_path, "w", encoding="utf8") as f:
        for line in chapter_text:
            f.write(line + "\n")

login()
data = get_shelf_books()
root_path = "我的书架"
if  not os.path.exists(root_path):
    os.mkdir(root_path)

get_books(data)




免费评分

参与人数 6吾爱币 +6 热心值 +6 收起 理由
heavenman + 1 + 1 实战贴,谢谢!
wanfon + 1 + 1 热心回复!
klxx1314520 + 1 + 1 谢谢@Thanks!
woyucheng + 1 + 1 谢谢@Thanks!
杨辣子 + 1 + 1 谢谢@Thanks!
Quincy379 + 1 + 1 谢谢@Thanks!

查看全部评分

发帖前要善用论坛搜索功能,那里可能会有你要找的答案或者已经有人发布过相同内容了,请勿重复发帖。

syhui 发表于 2023-10-13 14:30
谢谢分享
xiaoysm 发表于 2023-10-13 14:30
林泽西 发表于 2023-10-13 14:42
yjn866y 发表于 2023-10-13 14:55
这样发更好

[Python] 纯文本查看 复制代码
import requests
from lxml import etree
import os
headers  = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36",
        }
session = requests.session()

def login():
    session.post("https://passport.17k.com/ck/user/login",data={
                            "loginName": "账号",
                            "password": "密码"
                    },headers=headers)


def get_shelf_books():
    '''

    :return:
    '''
    res = session.get("https://user.17k.com/ck/author2/shelf?page=1&appKey=2406394919")
    res.encoding="utf8"
    data = res.json().get("data")
    return data


def get_books(data):

    for bookDict in data:
        # print(bookDict)
        bookID = bookDict.get("bookId")
        bookName = bookDict.get("bookName")
        book_path = os.path.join(root_path,bookName)
        if not os.path.exists(book_path):
            os.mkdir(book_path)
        get_chapter(bookID,book_path,bookName)
def get_chapter(bookID,book_path,bookName):
    res = requests.get(f"https://www.17k.com/list/{bookID}.html")
    # print("res",res)
    res.encoding = "utf8"

    selector = etree.HTML(res.text)
    items = selector.xpath('//dl[@class="Volume"]/dd/a')
    # print(items)


    for item in items:
        chapter_href = item.xpath('./@href')[0]
        chapter_title = item.xpath('./span/text()')[0].strip()
        print("chapter_href", chapter_href)
        print("chapter_text", chapter_title)

        res = requests.get("https://www.17k.com" + chapter_href)
        res.encoding = "utf8"
        chapter_html = res.text
        print(chapter_html)
        selector = etree.HTML(res.text)
        chapter_text = selector.xpath('//div[contains(@class,"content")]/div[@class="p"]/p[position()<last()]/text()')
        # ret = selector.xpath('//div[contains(@class,"content")]/div[@class="p"]/p[position()<last()]/text()')
        print(chapter_text)
        download(book_path, chapter_title,chapter_text)

        print(f'{bookName}书籍的{chapter_title}章节已经下载完成')

def download(book_path, chapter_title,chapter_text):
    chapter_path = os.path.join(book_path, chapter_title)
    with open(chapter_path, "w", encoding="utf8") as f:
        for line in chapter_text:
            f.write(line + "\n")

login()
data = get_shelf_books()
root_path = "我的书架"
if  not os.path.exists(root_path):
    os.mkdir(root_path)

get_books(data)

免费评分

参与人数 1吾爱币 +1 收起 理由
Outis404 + 1 谢谢@Thanks!

查看全部评分

52bojie 发表于 2023-10-13 15:17
谢谢分享
w12928013 发表于 2023-10-13 15:54
谢谢分享,如何下载视频啊
18691616928 发表于 2023-10-13 15:58
w12928013 发表于 2023-10-13 15:54
谢谢分享,如何下载视频啊

下载视频首先要看下是哪个网站,针对性的去下载
zqtzheng 发表于 2023-10-13 16:05
毛泽西 发表于 2023-10-13 14:42
小白不知道咋用呀

哈哈,不止你一个,我也没搞明白
TL1ng 发表于 2023-10-13 16:15
学习学习,谢谢了
您需要登录后才可以回帖 登录 | 注册[Register]

本版积分规则

返回列表

RSS订阅|小黑屋|处罚记录|联系我们|吾爱破解 - LCG - LSG ( 京ICP备16042023号 | 京公网安备 11010502030087号 )

GMT+8, 2024-11-1 07:40

Powered by Discuz!

Copyright © 2001-2020, Tencent Cloud.

快速回复 返回顶部 返回列表