Dissionpage实现 Molecular Plant文献获取

13955925361

co中的参数请填写自己浏览器所在地址
page.get()的网址请填写自己想请求的具体网址（https://www.sciencedirect.com  Molecular plant 主站中） 关键词自选

[Python] 纯文本查看 复制代码

import numpy as np
import pandas as pd

from DrissionPage import ChromiumPage
from DrissionPage import ChromiumPage,ChromiumOptions
co=ChromiumOptions().set_paths(browser_path=r'C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe')
page = ChromiumPage(co)
page.get('https://www.sciencedirect.com/search?qs=wheat&pub=Molecular%20Plant&cid=311979&sortBy=date')
a=page.eles('.ResultItem col-xs-24 push-m')
weblist=[]
for i in a:
    t=i.ele('t:a').text
    ppp=str(i.ele('.srctitle-date-fields').text)
    pp = ppp.split(' ')[-2:]
    p=' '.join(pp)
    w=i.ele('t:a').attr('href')
    j = 'Molecular Plant'
    data = {
        'Title': t,
        'Pubdate': p,
        'Journal': j,
        'Web': w
    }
    weblist.append(w)
    df = pd.DataFrame(data,index=[0])
    df.to_csv('test1.csv',mode='a',index=False,header=False)
for i in weblist:
    page.get(i,retry=3,interval=2,timeout=10)
try:
    d = page.ele('.anchor doi anchor-default').attr('href')
except:
    d = 'None'
try:
    Ab = page.ele('#abspara0010').text
except:
    Ab = 'None'
lis = [i, d, Ab]
df = pd.DataFrame(lis)
df.to_csv('test2.csv', mode='a', index=False, header=False)

Cacarot

感觉Dissionpage比selenium强

feiyu361

Dissionpage很不错的