Python爬蟲采集微博視頻數據
前言
隨時隨地發現新鮮事!微博帶你欣賞世界上每一個精彩瞬間,瞭解每一個幕後故事。分享你想表達的,讓全世界都能聽到你的心聲!今天我們通過python去采集微博當中好看的視頻!
沒錯,今天的目標是微博數據采集,爬的是那些好看的小姐姐視頻
知識點
requests
pprint
開發環境
版 本:python 3.8
-編輯器:pycharm 2021.2
爬蟲原理
作用:批量獲取互聯網數據(文本, 圖片, 音頻, 視頻)
本質:一次次的請求與響應
案例實現
1. 導入所需模塊
import requests import pprint
2. 找到目標網址
打開開發者工具,選中Fetch/XHR,選中數據所在的標簽,找到目標所在url
https://www.weibo.com/tv/api/component?page=/tv/channel/4379160563414111/editor
3. 發送網絡請求
headers = { 'cookie': '', 'referer': 'https://weibo.com/tv/channel/4379160563414111/editor', 'user-agent': '', } data = { 'data': '{"Component_Channel_Editor":{"cid":"4379160563414111","count":9}}' } url = 'https://www.weibo.com/tv/api/component?page=/tv/channel/4379160563414111/editor' json_data = requests.post(url=url, headers=headers, data=data).json()
4. 獲取數據
json_data_2 = requests.post(url=url_1, headers=headers, data=data_1).json()
5. 篩選數據
dict_urls = json_data_2['data']['Component_Play_Playinfo']['urls'] video_url = "https:" + dict_urls[list(dict_urls.keys())[0]] print(title + "\t" + video_url)
6. 保存數據
video_data = requests.get(video_url).content with open(f'video\\{title}.mp4', mode='wb') as f: f.write(video_data) print(title, "爬取成功................")
完整代碼
import requests import pprint headers = { 'cookie': '添加自己的', 'referer': 'https://weibo.com/tv/channel/4379160563414111/editor', 'user-agent': '', } data = { 'data': '{"Component_Channel_Editor":{"cid":"4379160563414111","count":9}}' } url = 'https://www.weibo.com/tv/api/component?page=/tv/channel/4379160563414111/editor' json_data = requests.post(url=url, headers=headers, data=data).json() print(json_data) ccs_list = json_data['data']['Component_Channel_Editor']['list'] next_cursor = json_data['data']['Component_Channel_Editor']['next_cursor'] for ccs in ccs_list: oid = ccs['oid'] title = ccs['title'] data_1 = { 'data': '{"Component_Play_Playinfo":{"oid":"' + oid + '"}}' } url_1 = 'https://weibo.com/tv/api/component?page=/tv/show/' + oid json_data_2 = requests.post(url=url_1, headers=headers, data=data_1).json() dict_urls = json_data_2['data']['Component_Play_Playinfo']['urls'] video_url = "https:" + dict_urls[list(dict_urls.keys())[0]] print(title + "\t" + video_url) video_data = requests.get(video_url).content with open(f'video\\{title}.mp4', mode='wb') as f: f.write(video_data) print(title, "爬取成功................")
以上就是Python爬蟲采集微博視頻數據的詳細內容,更多關於Python采集視頻數據的資料請關註WalkonNet其它相關文章!
推薦閱讀:
- Python爬蟲實戰之虎牙視頻爬取附源碼
- 用Python采集《雪中悍刀行》彈幕做成詞雲實例
- Python實現微博動態圖片爬取詳解
- Python requests.post()方法中data和json參數的使用方法
- Python爬取股票交易數據並可視化展示