HttpClient

網絡爬蟲就是用程序幫助我們訪問網絡上的資源，我們一直以來都是使用HTTP協議訪問互聯網的網頁，網絡爬蟲需要編寫程序，在這里使用同樣的HTTP協議訪問網頁。

這里使用Java的HTTP協議客戶端 HttpClient這個技術，來實現抓取網頁數據。

---

GET請求?

package cn.itcast.crawler.test;import org.apache.http.client.methods.CloseableHttpResponse; import org.apache.http.client.methods.HttpGet; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClients; import org.apache.http.util.EntityUtils;import java.io.IOException;public class HttpGetTest {public static void main(String[] args) {//創建HttpClient對象CloseableHttpClient httpClient = HttpClients.createDefault();//創建HttpGet對象，設置url訪問地址HttpGet httpGet = new HttpGet("http://www.itcast.cn");CloseableHttpResponse response = null;try {//使用HttpClient發起請求，獲取responseresponse = httpClient.execute(httpGet);//解析響應if (response.getStatusLine().getStatusCode() == 200) {String content = EntityUtils.toString(response.getEntity(), "utf8");System.out.println(content.length());}} catch (IOException e) {e.printStackTrace();}finally {//關閉responsetry {response.close();} catch (IOException e) {e.printStackTrace();}try {httpClient.close();} catch (IOException e) {e.printStackTrace();}}} }

---

---

GET請求

帶參數的GET請求

---

package cn.itcast.crawler.test;import org.apache.http.client.methods.CloseableHttpResponse; import org.apache.http.client.methods.HttpGet; import org.apache.http.client.utils.URIBuilder; import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClients; import org.apache.http.util.EntityUtils;import java.io.IOException; import java.net.URISyntaxException;public class HttpGetParamTest {public static void main(String[] args) throws Exception {//創建HttpClient對象CloseableHttpClient httpClient = HttpClients.createDefault();//設置請求地址是：http://yun.itheima.com/search?keys=Java//創建URIBuilderURIBuilder uriBuilder = new URIBuilder("http://yun.itheima.com/search");//設置參數uriBuilder.setParameter("keys","Java");//創建HttpGet對象，設置url訪問地址HttpGet httpGet = new HttpGet(uriBuilder.build());System.out.println("發起請求的信息："+httpGet);CloseableHttpResponse response = null;try {//使用HttpClient發起請求，獲取responseresponse = httpClient.execute(httpGet);//解析響應if (response.getStatusLine().getStatusCode() == 200) {String content = EntityUtils.toString(response.getEntity(), "utf8");System.out.println(content.length());}} catch (IOException e) {e.printStackTrace();}finally {//關閉responsetry {response.close();} catch (IOException e) {e.printStackTrace();}try {httpClient.close();} catch (IOException e) {e.printStackTrace();}}} }

總結

以上是生活随笔為你收集整理的HttpClient ||GET请求||带参数的GET请求的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。