运用Python爬取百度风云榜
2021/7/3 12:21:27
本文主要是介绍运用Python爬取百度风云榜,对大家解决编程问题具有一定的参考价值,需要的程序猿们随着小编来一起学习吧!
import requests
import xlwt
from bs4 import BeautifulSoup
import DBUtils
def getCid():
hd = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36 SLBrowser/7.0.0.4071 SLBChan/21"}
url = "http://top.baidu.com/boards?fr=topindex"
page = requests.get(url,headers=hd).content.decode("GBK")
soup = BeautifulSoup(page, 'html.parser')
dIv = soup.find('div', attrs={'class':"links"})
a = dIv.find_all('a')
code = []
type_L = ["爱情","喜剧","惊悚","科幻","剧情"]
for i in range(0,len(a)-1):
src = a[i]['href']
code.append(src.strip('.'))
return code,type_L
def getItem(page,type_L):
hd = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36 SLBrowser/7.0.0.4071 SLBChan/21"}
url = "http://top.baidu.com%s"
url = url%page
page = requests.get(url,headers=hd).content.decode("GBK")
soup = BeautifulSoup(page, 'html.parser')
type_M = type_L
这篇关于运用Python爬取百度风云榜的文章就介绍到这儿,希望我们推荐的文章对大家有所帮助,也希望大家多多支持为之网!
- 2024-10-04Python外包有哪些常见的主要库-icode9专业技术文章分享
- 2024-10-02基于深度学习的钢铁缺陷检测系统(yolov8、注意力机制、PyQt5界面、Python)
- 2024-10-01怎么用python搭建一个系统-icode9专业技术文章分享
- 2024-09-30Python基础入门教程
- 2024-09-30获取参数学习:Python中的基础教程
- 2024-09-30Python编程基础详解
- 2024-09-29点击加载学习:Python编程基础教程
- 2024-09-29数据科学五大Python前端库:第二部分
- 2024-09-27使用python 将ETH账户的资产打散
- 2024-09-26Python编程基础