場景

在寫爬蟲進(jìn)行request請求時(shí)，如果不能正常請求就要添加請求頭。

常用的是修改User-Agent來偽裝瀏覽器。

我們除了復(fù)制之外，這里可以參照一個(gè)庫叫fake-useragent。

網(wǎng)址：

https://pypi.org/project/fake-useragent/

安裝

pip install fake-useragent

使用

官方示例：

from fake_useragent import UserAgent ua = UserAgent()ua.ie # Mozilla/5.0 (Windows; U; MSIE 9.0; Windows NT 9.0; en-US); ua.msie # Mozilla/5.0 (compatible; MSIE 10.0; Macintosh; Intel Mac OS X 10_7_3; Trident/6.0)' ua['Internet Explorer'] # Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; GTB7.4; InfoPath.2; SV1; .NET CLR 3.3.69573; WOW64; en-US) ua.opera # Opera/9.80 (X11; Linux i686; U; ru) Presto/2.8.131 Version/11.11 ua.chrome # Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.2 (KHTML, like Gecko) Chrome/22.0.1216.0 Safari/537.2' ua.google # Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.13 (KHTML, like Gecko) Chrome/24.0.1290.1 Safari/537.13 ua['google chrome'] # Mozilla/5.0 (X11; CrOS i686 2268.111.0) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11 ua.firefox # Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:16.0.1) Gecko/20121011 Firefox/16.0.1 ua.ff # Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:15.0) Gecko/20100101 Firefox/15.0.1 ua.safari # Mozilla/5.0 (iPad; CPU OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5355d Safari/8536.25# and the best one, random via real world browser usage statistic ua.random

?

實(shí)現(xiàn)

自己經(jīng)常使用的代碼：

'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36',

比如說我們可以這樣使用：

def Get_data():headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36','cookie': 'acw_tc=6f3e47cc15498749418558688e3d6410ea97be9a552b180841c5079605; PHPSESSID=s2hjdg3slh32omchfsib4himk0; UM_distinctid=168dbbfe8c533e-0e42df11c1c0a7-9393265-100200-168dbbfe8c71088; CNZZDATA1256319371=345851669-1549873534-%7C1549873534; CNZZDATA1257119496=134880590-1549870452-%7C1549870452; Hm_lvt_a48e6ab107a4e68d47e6fdb5d83961e5=1549875015; Hm_lvt_3c8ecbfa472e76b0340d7a701a04197e=1549875021; CNZZDATA1254708131=653924416-1549874976-%7C1549874976; CNZZDATA1275922735=1673340420-1549874690-%7C1549874690; CNZZDATA1257131565=1820590917-1549873827-%7C1549873827; CNZZDATA1257125147=1301871275-1549871347-https%253A%252F%252Fwww.duanwenxue.com%252F%7C1549871347; Hm_lpvt_a48e6ab107a4e68d47e6fdb5d83961e5=1549875906; Hm_lpvt_3c8ecbfa472e76b0340d7a701a04197e=1549875913; ajax_award_timestamp=1549875887; ajax_award_timestamp__ckMd5=706904497e9c8dfd; ajax_award_key=94a17405d22ab8aacbabbbfdb9d4740c; ajax_award_key__ckMd5=3e97bde02ab1998d','referer': 'https://www.duanwenxue.com/huayu/tianyanmiyu/list_69.html','upgrade-insecure-requests' : '1',}for i in range(1,70):url = 'https://www.duanwenxue.com/huayu/tianyanmiyu/list_{}.html'.format(i)response = requests.get(url,headers=headers)parse_text(response.text)

?

總結(jié)

以上是生活随笔為你收集整理的爬虫进行request请求时User-Agent怎样写的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。