?

幫朋友抓了一些代理IP，并根據(jù)測(cè)試聯(lián)的通性，放在了不通的文件夾下。特將源碼分享

注意：

1，環(huán)境Python3.5

2，安裝BeautifulSoup4 ?requests

?

代碼如下:

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66

#-*-?coding:gb18030?-*- from?bs4?import?BeautifulSoup import?requests import?time import?os,sys all_url_add={ ??????'url2':'http://ip84.com/gn/', ??????} def?func(url): ????r?=?requests.get(url) ????content?=?r.text ????soup?=?BeautifulSoup(content,?"html.parser") ????ListTable?=?soup.find_all("table",?class_="list") ????for?table?in?ListTable: ????????ListTr?=?table.find_all("tr") ????????for?tr?in?ListTr: ????????????try: ????????????????ListTd?=?tr.find_all("td") ????????????????ipaddress?=?str(ListTd[0].get_text()).strip() ????????????????port?=?str(ListTd[1].get_text()).strip() ????????????????city?=?str(ListTd[2].get_text()).strip().replace("\n",?"") ????????????????leixing?=?str(ListTd[3].get_text()).strip() ????????????????xieyi?=?str(ListTd[4].get_text()).strip() ????????????????shudu?=?str(ListTd[5].get_text()).strip() ????????????????time1?=?str(ListTd[6].get_text()).strip() ????????????????f?=?open("ip"?+?'.txt',?'a') ????????????????f.write(ipaddress+":"+port+'\n') ????????????????f.close() ????????????????print('地址:'+ipaddress?+?"端口:"?+?port?+?"地區(qū):"?+?city?+?"類(lèi)型:"?+?leixing?+?"協(xié)議"?+?xieyi?+?"速度"?+?shudu?+?"時(shí)間:"?+?time1) ????????????except?Exception?as?e: ????????????????print?(u"-------------------程序異常-----------------------") ????????return?'success' ????print?(u'本頁(yè)抓取結(jié)束，正在跳轉(zhuǎn)下一頁(yè)') def?pin(): ????f2?=?open('ip.txt',?'r') ????count?=?len(open('ip.txt',?'rU').readlines()) ????for?x?in?range(count): ????????ip?=?f2.readline().split(':')[0] ????????return1?=?os.system('ping?-n?5?-w?5?%s'?%?ip)?? ????????if?return1: ????????????print('測(cè)試失敗') ????????else: ????????????print('測(cè)試成功，正在寫(xiě)入新文件') ????????????f3?=?open('SuccessIp.txt',?'a') ????????????f3.write(f2.readline()?+?'\n') ????????????f3.close() ????f2.close() ????print('程序結(jié)束，可用IP已放在SuccessIp中') if??__name__=='__main__': ????????for?x?in?all_url_add: ????????????print?(x) ????????????for?y?in?range(1,50): ????????????????url=all_url_add[x]+str(y) ????????????????print?(url) ????????????????status=func(url) ????????????????if?status=='success': ????????????????????print(y,'頁(yè)結(jié)束') ????????print?(u'****程序抓取運(yùn)行結(jié)束,正在檢查所得IP連通性，請(qǐng)勿關(guān)閉窗口*****') ????????pin()

?

有點(diǎn)亂，有時(shí)間將數(shù)據(jù)存儲(chǔ)在數(shù)據(jù)庫(kù)，再將這個(gè)功能集成在博客當(dāng)中。

?

?

Rex博客保留所有權(quán)利

轉(zhuǎn)載于:https://www.cnblogs.com/rexyan/p/6212923.html

總結(jié)

以上是生活随笔為你收集整理的Python 爬虫抓取代理IP，并检测联通性的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。