四要素：

首先，確定首頁URL? ? ?【谷歌瀏覽器 F12】【https://new.qq.com/ch/ent/】

?

---

?

代碼展示：

實體類News?:

public class News {public String appId;//app_idpublic String title;//titlepublic String intro;//intropublic String source;//sourcepublic String url; //urlpublic String updateTime ;//update_time@Overridepublic String toString() {return "News{" +"appId='" + appId + '\'' +", title='" + title + '\'' +", intro='" + intro + '\'' +", source='" + source + '\'' +", url='" + url + '\'' +", updateTime='" + updateTime + '\'' +'}';}public String getTitle() {return title;}public void setTitle(String title) {this.title = title;}public String getIntro() {return intro;}public void setIntro(String intro) {this.intro = intro;}public String getSource() {return source;}public void setSource(String source) {this.source = source;}public String getUrl() {return url;}public void setUrl(String url) {this.url = url;}@JSONField(name="update_time",format = "yyyy-MM-dd HH:mm:ss")public String getUpdateTime() {return updateTime;}@JSONField(name="update_time",format = "yyyy-MM-dd HH:mm:ss")public void setUpdateTime(String updateTime) {this.updateTime = updateTime;}@JSONField(name="app_id")public String getAppId() {return appId;}@JSONField(name="app_id")public void setAppId(String appId) {this.appId = appId;} }

---

集合類：

public class NewsCollection {//data與json中data相對應public List<News> data;public List<News> getData() {return data;}public void setData(List<News> data) {this.data = data;} }

---

?爬蟲類：

?

import com.alibaba.fastjson.JSON; import org.apache.http.HttpEntity; import org.apache.http.client.methods.CloseableHttpResponse; import org.apache.http.client.methods.HttpGet; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClients; import org.apache.http.util.EntityUtils; import org.jsoup.Jsoup; import org.jsoup.nodes.Document; import org.jsoup.nodes.Element; import org.jsoup.select.Elements; import org.junit.Test;import java.io.File; import java.io.IOException; import java.util.List;public class TestQQ {@Test //根據標簽名稱public void jsoupParse01() throws IOException {String url="https://pacaio.match.qq.com/irs/rcd?cid=146&token=49cbb2154853ef1a74ff4e53723372ce&ext=ent" +"&page=1&callback=__jp6";CloseableHttpClient closeableHttpClient = HttpClients.createDefault();HttpGet httpGet=new HttpGet(url);//執行請求 user center pageCloseableHttpResponse closeableHttpResponse = closeableHttpClient.execute(httpGet);HttpEntity entity = closeableHttpResponse.getEntity();String content = EntityUtils.toString(entity, "GB2312");String jsonContent= StringToJson(content);List<News> news=jsonToObject(jsonContent);}/*** string轉換成json工具方法* @param srcJson* @return*/public String StringToJson(String srcJson){int start=srcJson.indexOf('(')+1;int end=srcJson.lastIndexOf(')');String jsonContent= srcJson.substring(start,end);return jsonContent;}public List<News> jsonToObject(String srcJson){//通過fastjson把json字符串轉換成java對象NewsCollection newsCollection = JSON.parseObject(srcJson, NewsCollection.class);List<News> news = newsCollection.getData();for (News newsMessage : news) {System.out.println(newsMessage);System.out.println("---------------------------------------------------------");}return news;} }

?結果展示：

---

【注】根據不同頁數爬取多頁信息?

?

?

?

?

總結

以上是生活随笔為你收集整理的爬虫 spider07——爬取腾讯娱乐新闻的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。