# 獲取相應(yīng)內(nèi)容 import requests r = requests.get('http://www.santostang.com/') print("文本編碼：",r.encoding) print("相應(yīng)狀態(tài)碼：",r.status_code)print("字符串方式的響應(yīng)體：",r.text) # 傳遞URL參數(shù) import requests key_dict = {'key1':'value1','key2':'value2'} r = requests.get('http://httpbin.org/get',params=key_dict) print("URL 已經(jīng)正確編碼：",r.url) print("字符串方式的響應(yīng)體：\n",r.text) # 定制請求頭 import requests headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.90 Safari/537.36 2345Explorer/9.3.3.17611','Host':'www.santostang.com'}r = requests.get('http://www.santostang.com/',headers = headers)print("相應(yīng)狀態(tài)碼：",r.status_code) # 發(fā)送post請求 import requests key_dict = {'key1':'value1','key2':'value2'} r = requests.get('http://httpbin.org/post',data = key_dict) print(r.text) # 獲取豆瓣前250個(gè)排名的電影 import requests from bs4 import BeautifulSoup def get_movies():headers = {'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.90 Safari/537.36 2345Explorer/9.3.3.17611','Host':'movie.douban.com'}movie_list = []for i in range(0,10):link = 'https://movie.douban.com/top250?start=' + str(i*25)r = requests.get(link, headers = headers, timeout = 10)print(str(i+1),"網(wǎng)頁相應(yīng)狀態(tài)碼：", r.status_code)soup = BeautifulSoup(r.text,'lxml')div_list = soup.find_all('div',class_='hd')for each in div_list:movie = each.a.span.text.strip()movie_list.append(movie)return movie_listmovies = get_movies() print(movies)

輸出結(jié)果為：
1 網(wǎng)頁相應(yīng)狀態(tài)碼： 200
2 網(wǎng)頁相應(yīng)狀態(tài)碼： 200
3 網(wǎng)頁相應(yīng)狀態(tài)碼： 200
4 網(wǎng)頁相應(yīng)狀態(tài)碼： 200
5 網(wǎng)頁相應(yīng)狀態(tài)碼： 200
6 網(wǎng)頁相應(yīng)狀態(tài)碼： 200
7 網(wǎng)頁相應(yīng)狀態(tài)碼： 200
8 網(wǎng)頁相應(yīng)狀態(tài)碼： 200
9 網(wǎng)頁相應(yīng)狀態(tài)碼： 200
10 網(wǎng)頁相應(yīng)狀態(tài)碼： 200
[‘肖申克的救贖’, ‘霸王別姬’, ‘這個(gè)殺手不太冷’, ‘阿甘正傳’, ‘美麗人生’, ‘泰坦尼克號’, ‘千與千尋’, ‘辛德勒的名單’, ‘盜夢空間’, ‘機(jī)器人總動(dòng)員’, ‘忠犬八公的故事’, ‘三傻大鬧寶萊塢’, ‘海上鋼琴師’, ‘放牛班的春天’, ‘大話西游之大圣娶親’, ‘楚門的世界’, ‘教父’, ‘龍貓’, ‘星際穿越’, ‘熔爐’, ‘無間道’, ‘觸不可及’, ‘亂世佳人’, ‘當(dāng)幸福來敲門’, ‘怦然心動(dòng)’, ‘天堂電影院’, ‘瘋狂動(dòng)物城’, ‘蝙蝠俠：黑暗騎士’, ‘十二怒漢’, ‘鬼子來了’, ‘活著’, ‘少年派的奇幻漂流’, ‘指環(huán)王3：王者無敵’, ‘搏擊俱樂部’, ‘天空之城’, ‘控方證人’, ‘大話西游之月光寶盒’, ‘飛屋環(huán)游記’, ‘羅馬假日’, ‘竊聽風(fēng)暴’, ‘兩桿大煙槍’, ‘飛越瘋?cè)嗽骸? ‘哈爾的移動(dòng)城堡’, ‘聞香識女人’, ‘辯護(hù)人’, ‘摔跤吧！爸爸’, ‘V字仇殺隊(duì)’, ‘死亡詩社’, ‘海豚灣’, ‘教父2’, ‘指環(huán)王2：雙塔奇兵’, ‘美麗心靈’, ‘指環(huán)王1：魔戒再現(xiàn)’, ‘飲食男女’, ‘情書’, ‘美國往事’, ‘獅子王’, ‘素媛’, ‘鋼琴家’, ‘小鞋子’, ‘七宗罪’, ‘被嫌棄的松子的一生’, ‘致命魔術(shù)’, ‘天使愛美麗’, ‘本杰明·巴頓奇事’, ‘西西里的美麗傳說’, ‘音樂之聲’, ‘拯救大兵瑞恩’, ‘勇敢的心’, ‘黑客帝國’, ‘讓子彈飛’, ‘低俗小說’, ‘看不見的客人’, ‘剪刀手愛德華’, ‘沉默的羔羊’, ‘蝴蝶效應(yīng)’, ‘大鬧天宮’, ‘末代皇帝’, ‘入殮師’, ‘春光乍泄’, ‘心靈捕手’, ‘瑪麗和馬克思’, ‘哈利·波特與魔法石’, ‘陽光燦爛的日子’, ‘布達(dá)佩斯大飯店’, ‘幽靈公主’, ‘禁閉島’, ‘第六感’, ‘重慶森林’, ‘貓鼠游戲’, ‘狩獵’, ‘致命ID’, ‘?dāng)啾成健? ‘大魚’, ‘穿條紋睡衣的男孩’, ‘加勒比海盜’, ‘射雕英雄傳之東成西就’, ‘告白’, ‘甜蜜蜜’, ‘一一’, ‘陽光姐妹淘’, ‘摩登時(shí)代’, ‘阿凡達(dá)’, ‘上帝之城’, ‘愛在黎明破曉前’, ‘消失的愛人’, ‘風(fēng)之谷’, ‘側(cè)耳傾聽’, ‘愛在日落黃昏時(shí)’, ‘超脫’, ‘倩女幽魂’, ‘恐怖直播’, ‘紅辣椒’, ‘小森林 夏秋篇’, ‘喜劇之王’, ‘菊次郎的夏天’, ‘馴龍高手’, ‘幸福終點(diǎn)站’, ‘螢火蟲之墓’, ‘借東西的小人阿莉埃蒂’, ‘神偷奶爸’, ‘歲月神偷’, ‘七武士’, ‘殺人回憶’, ‘怪獸電力公司’, ‘電鋸驚魂’, ‘貧民窟的百萬富翁’, ‘諜影重重3’, ‘喜宴’, ‘真愛至上’, ‘東邪西毒’, ‘記憶碎片’, ‘黑天鵝’, ‘瘋狂原始人’, ‘海洋’, ‘雨人’, ‘小森林 冬春篇’, ‘盧旺達(dá)飯店’, ‘哈利·波特與死亡圣器(下)’, ‘7號房的禮物’, ‘英雄本色’, ‘螢火之森’, ‘請以你的名字呼喚我’, ‘心迷宮’, ‘虎口脫險(xiǎn)’, ‘燃情歲月’, ‘傲慢與偏見’, ‘荒蠻故事’, ‘超能陸戰(zhàn)隊(duì)’, ‘海邊的曼徹斯特’, ‘教父3’, ‘唐伯虎點(diǎn)秋香’, ‘蝙蝠俠：黑暗騎士崛起’, ‘戀戀筆記本’, ‘縱橫四海’, ‘完美的世界’, ‘花樣年華’, ‘時(shí)空戀旅人’, ‘玩具總動(dòng)員3’, ‘魂斷藍(lán)橋’, ‘猜火車’, ‘穿越時(shí)空的少女’, ‘達(dá)拉斯買家俱樂部’, ‘雨中曲’, ‘二十二’, ‘我是山姆’, ‘無人知曉’, ‘人工智能’, ‘冰川時(shí)代’, ‘血戰(zhàn)鋼鋸嶺’, ‘浪潮’, ‘爆裂鼓手’, ‘香水’, ‘朗讀者’, ‘羅生門’, ‘未麻的部屋’, ‘阿飛正傳’, ‘被解救的姜戈’, ‘頭腦特工隊(duì)’, ‘可可西里’, ‘恐怖游輪’, ‘一次別離’, ‘追隨’, ‘撞車’, ‘戰(zhàn)爭之王’, ‘房間’, ‘地球上的星星’, ‘你的名字。’, ‘魔女宅急便’, ‘夢之安魂曲’, ‘模仿游戲’, ‘諜影重重’, ‘牯嶺街少年殺人事件’, ‘忠犬八公物語’, ‘諜影重重2’, ‘一個(gè)叫歐維的男人決定去死’, ‘驚魂記’, ‘完美陌生人’, ‘青蛇’, ‘哪吒鬧海’, ‘再次出發(fā)之紐約遇見你’, ‘小蘿莉的猴神大叔’, ‘黑客帝國3：矩陣革命’, ‘東京物語’, ‘源代碼’, ‘新龍門客棧’, ‘終結(jié)者2：審判日’, ‘末路狂花’, ‘秒速5厘米’, ‘綠里奇跡’, ‘碧海藍(lán)天’, ‘步履不停’, ‘勇闖奪命島’, ‘城市之光’, ‘這個(gè)男人來自地球’, ‘初戀這件小事’, ‘海街日記’, ‘無恥混蛋’, ‘海盜電臺(tái)’, ‘無敵破壞王’, ‘卡薩布蘭卡’, ‘變臉’, ‘愛在午夜降臨前’, ‘E.T. 外星人’, ‘瘋狂的石頭’, ‘發(fā)條橙’, ‘黃金三鏢客’, ‘美國麗人’, ‘荒野生存’, ‘英國病人’, ‘彗星來的那一夜’, ‘遷徙的鳥’, ‘血鉆’, ‘國王的演講’, ‘聚焦’, ‘非常嫌疑犯’, ‘燕尾蝶’, ‘勇士’, ‘叫我第一名’, ‘穆赫蘭道’, ‘遺愿清單’, ‘黑鷹墜落’, ‘我愛你’, ‘槍火’, ‘荒島余生’, ‘上帝也瘋狂’, ‘大衛(wèi)·戈?duì)柕囊簧? ‘千鈞一發(fā)’, ‘2001太空漫游’, ‘藍(lán)色大門’]

總結(jié)

以上是生活随笔為你收集整理的Python 网络爬虫第二讲的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。