好友
阅读权限25
听众
最后登录1970-1-1
|
本帖最后由 txq0211 于 2023-3-15 11:14 编辑
业余爱好者,好久没写爬虫了,近期世界经济变化莫测,
突然看到一篇“你需要知道的隔夜全球要闻”,忍不住写个爬虫把它爬下来每天看看。
第一步 找到通用接口
第二步 测试接口
第三步 尝试编写代码,反复测试
[Python] 纯文本查看 复制代码 import requests
import json
import re
import time
from lxml import etree
if __name__ == '__main__':
url = 'https://www.cls.cn/api/sw?app=CailianpressWeb&os=web&sv=7.7.5&sign=bf0f367462d8cd70917ba5eab3853bce'
headers = {"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/110.0"}
data = {"type":"telegram","keyword":"你需要知道的隔夜全球要闻","page":0,"rn":20,"os":"web","sv":"7.7.5","app":"CailianpressWeb"}
response = requests.post(url=url,headers=headers,data=data)
news = json.loads(response.text)['data']['telegram']['data'][0]['descr']
timeStamp = json.loads(response.text)['data']['telegram']['data'][0]['time']
timeArray = time.localtime(timeStamp)
formatTime = time.strftime("%Y年%m月%d日", timeArray)
news = re.split(r'\d+、',news)
title = ''.join(etree.HTML(news[0]).xpath('//text()'))[1:-2]
print(formatTime,title)
for i in range(1, len(news)):
new = '%s、%s'%(i,news[i])
print(new)
第四步 迭代完善(2023.3.14修改)。
最初编写代码过程中存在问题,本想通过正则表达式在"。数字、"数字前面通过正则替换内容替换新增换行,没想到输出的时候数字被\d+替换了。感谢大神@Arcticlyc指点,优化正则替换。
如有不足,欢迎大家提出意见
[Python] 纯文本查看 复制代码 import requests
import json
import re
import time
from lxml import etree
if __name__ == '__main__':
url = 'https://www.cls.cn/api/sw?app=CailianpressWeb&os=web&sv=7.7.5&sign=bf0f367462d8cd70917ba5eab3853bce'
headers = {"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/110.0"}
data = {"type":"telegram","keyword":"你需要知道的隔夜全球要闻","page":0,"rn":20,"os":"web","sv":"7.7.5","app":"CailianpressWeb"}
response = requests.post(url=url,headers=headers,data=data)
news = json.loads(response.text)['data']['telegram']['data'][0]['descr']
timeStamp = json.loads(response.text)['data']['telegram']['data'][0]['time']
timeArray = time.localtime(timeStamp)
formatTime = time.strftime("%Y年%m月%d日", timeArray)
news = re.sub(r'(\d+、)', r'\n\1', news)
formatNews = ''.join(etree.HTML(news).xpath('//text()'))
print(formatTime,formatNews)
第五步 发送至微信(2023.3.15修改)
发现一款新的微信插件wxauto支持最新的微信3.9+。
地址https://github.com/cluic/wxauto
[Python] 纯文本查看 复制代码 import requests
import json
import re
import time
from lxml import etree
from wxauto import *
if __name__ == '__main__':
url = 'https://www.cls.cn/api/sw?app=CailianpressWeb&os=web&sv=7.7.5&sign=bf0f367462d8cd70917ba5eab3853bce'
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/110.0"}
data = {"type": "telegram", "keyword": "你需要知道的隔夜全球要闻", "page": 0, "rn": 20, "os": "web", "sv": "7.7.5",
"app": "CailianpressWeb"}
response = requests.post(url=url, headers=headers, data=data)
news = json.loads(response.text)['data']['telegram']['data'][0]['descr']
timeStamp = json.loads(response.text)['data']['telegram']['data'][0]['time']
timeArray = time.localtime(timeStamp)
formatTime = time.strftime("%Y年%m月%d日", timeArray)
news = re.sub(r'(\d+、)', r'\n\1', news)
formatNews = ''.join(etree.HTML(news).xpath('//text()'))
# 获取当前微信客户端
wx = WeChat()
# 获取会话列表
wx.GetSessionList()
# 向某人发送消息(以`文件传输助手`为例)
msg = formatTime + '\n' + formatNews
who = '文件传输助手'
WxUtils.SetClipboard(msg) # 将内容复制到剪贴板,类似于Ctrl + C
wx.ChatWith(who) # 打开`文件传输助手`聊天窗口
wx.SendClipboard() # 发送剪贴板的内容,类似于Ctrl + V
|
免费评分
-
查看全部评分
|
发帖前要善用【论坛搜索】功能,那里可能会有你要找的答案或者已经有人发布过相同内容了,请勿重复发帖。 |
|
|
|
|