HttpClient使用代理

1、为什么使用代理

当爬取网页的时候，有的目标站点有反爬虫机制，对于频繁访问站点以及规则性访问站点的行为，会采用屏蔽IP的措施，这时候代理IP就派上用场了。

2、代理的分类

（一）透明代理(Transparent Proxy)-透明代理虽然可以直接隐藏你的IP地址，但是还是从HTTP_X_FORWARD_FOR 来查到你是谁

REMOTE_ADDR= Proxy IP
HTTP_VIA = Proxy IP
HTTP_X_FORWORAD_FOR= YOUR IP

（二）匿名代理(Anonymous Proxy)-匿名代理比透明代理进步一点，别人只能知道你用了代理，无法知道你是谁

REMOTE_ADDR= proxy Ip
HTTP_VIA = proxy IP
HTTP_X_FORWARD_FOR = proxy_IP

（三）混淆代理(Distorting Proxies)-与匿名代理相同，如果使用了混淆代理，别人还是能知道你在用代理，但是会得到一个假的IP地址，伪装的更逼真

REMOTE_ADDR=PROXY_IP
HTTP_VIA =PROXY IP
HTTP_X_FOREARD_FOR=Random IP ADDRESS

（四）高匿代理(Elite Proxy 或High Anonymity Proxy)-可以看出，高匿代理让别人无法发现你是在用代理，是爬虫最好的选择

REMOTE_ADDR=PROXY_IP
HTTP_VIA = not determined
HTTP_X_FORWARD_FOR= not determined

3、代理IP的获取

@Test
public void testHttpProxy() throws  Exception{HttpClient httpClient = HttpClients.createDefault();HttpGet httpGet = new HttpGet("http://www.baidu.com");//使用代理服务器HttpHost httpHost = new HttpHost("220.194.55.160",3128);RequestConfig config = RequestConfig.custom().setProxy(httpHost).build();httpGet.setConfig(config);CloseableHttpResponse response = (CloseableHttpResponse) httpClient.execute(httpGet);HttpEntity entity = response.getEntity();//输出网页内容System.out.println("网页内容:");System.out.println(EntityUtils.toString(entity,"utf-8"));response.close();
}

4、HttpClient代理配置
HttpClient支持复杂的路由方案和代理链，同样也支持直接或者只通过一跳的连接。

（一）使用代理服务器最简单的方式，执行一个默认的默认的代理。

HttpHost proxy = new HttpHost("someproxy", 8080);
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
CloseableHttpClient httpclient = HttpClients.custom()  .setRoutePlanner(routePlanner)  .build();

（二） HttpClient使用jre代理服务器

SystemDefaultRoutePlanner routePlanner = new SystemDefaultRoutePlanner(  ProxySelector.getDefault());
CloseableHttpClient httpclient = HttpClients.custom()  .setRoutePlanner(routePlanner)  .build();

手动配置RoutePlanner，这样就可以完全控制Http路由的过程

HttpRoutePlanner routePlanner = new HttpRoutePlanner() {    public HttpRoute determineRoute(  HttpHost target,  HttpRequest request,  HttpContext context) throws HttpException {  return new HttpRoute(target, null,  new HttpHost("someproxy", 8080),  "https".equalsIgnoreCase(target.getSchemeName()));  }
};
CloseableHttpClient httpclient = HttpClients.custom()  .setRoutePlanner(routePlanner)  .build();  }
}

例子

package com.book.common.utils;import com.alibaba.fastjson.JSONObject;
import com.book.common.constant.StaticConstant;
import com.book.common.result.Failed;
import com.google.common.collect.Lists;
import com.google.common.collect.Maps;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.HttpHost;
import org.apache.http.NameValuePair;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.*;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.entity.ByteArrayEntity;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.ssl.SSLContexts;
import org.apache.http.util.EntityUtils;
import org.springframework.util.CollectionUtils;import javax.net.ssl.SSLContext;
import java.io.*;
import java.net.*;
import java.nio.charset.StandardCharsets;
import java.security.KeyManagementException;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Random;/*** Description：* HTTP调用；https路径会转化为http，所以需要能使用http调用访问的路径才可以使用* 用法：HttpUtils.doGet(HttpUtils.create().addHostPort("http://suyun.nevtsp.com/").addProxy("116.30.240.246:80").build());** @Author： leo.xiong* @CreateDate： 2021/11/30 9:43* @Email： leo.xiong@suyun360.com* @Since：*/
@Slf4j
public final class HttpUtils {private static final int DOWN_LOAD_READ_BYTE_BUFFER = 1024;private static final int SO_TIMEOUT_MS = 8000;private static final int CONNECTION_TIMEOUT_MS = 3000;private static final int SUCCESS = 200;private static final int HTTP_PORT = 80;private static final int HTTPS_PORT = 443;private static final Random RANDOM = new Random();/*** 创建参数对象信息** @return*/public static HttpClientParam create() {return new HttpClientParam();}/*** get** @param httpClientParam* @return*/public static String doGet(final HttpClientParam httpClientParam) {HttpGet request = new HttpGet(httpClientParam.getUrl());buildHeaderProxy(request, httpClientParam);return execute(httpClientParam.getCloseableHttpClient(), request);}/*** Post From** @param httpClientParam* @return*/public static String doPost(final HttpClientParam httpClientParam) {HttpPost request = new HttpPost(httpClientParam.getUrl());buildHeaderProxy(request, httpClientParam);if (!CollectionUtils.isEmpty(httpClientParam.getBodyMap())) {List<NameValuePair> nameValuePairList = Lists.newArrayList();for (String key : httpClientParam.getBodyMap().keySet()) {nameValuePairList.add(new BasicNameValuePair(key, httpClientParam.getBodyMap().get(key)));}UrlEncodedFormEntity formEntity = new UrlEncodedFormEntity(nameValuePairList, StandardCharsets.UTF_8);formEntity.setContentType("application/x-www-form-urlencoded; charset=UTF-8");request.setEntity(formEntity);} else if (!StringUtils.isEmpty(httpClientParam.getBodyStr())) {request.setEntity(new StringEntity(httpClientParam.getBodyStr(), StandardCharsets.UTF_8));} else if (httpClientParam.getBody() != null) {request.setEntity(new ByteArrayEntity(httpClientParam.getBody()));}return execute(httpClientParam.getCloseableHttpClient(), request);}/*** Put Stream** @param httpClientParam* @return*/public static String doPut(final HttpClientParam httpClientParam) {HttpPut request = new HttpPut(httpClientParam.getUrl());buildHeaderProxy(request, httpClientParam);if (StringUtils.isNotEmpty(httpClientParam.getBodyStr())) {request.setEntity(new StringEntity(httpClientParam.getBodyStr(), StandardCharsets.UTF_8));} else if (httpClientParam.getBody() != null) {request.setEntity(new ByteArrayEntity(httpClientParam.getBody()));}return execute(httpClientParam.getCloseableHttpClient(), request);}/*** Delete Stream** @param httpClientParam* @return*/public static String doDelete(final HttpClientParam httpClientParam) {HttpDelete request = new HttpDelete(httpClientParam.getUrl());buildHeaderProxy(request, httpClientParam);return execute(httpClientParam.getCloseableHttpClient(), request);}/*** 执行信息** @param httpClient* @param request* @return*/private static String execute(CloseableHttpClient httpClient, HttpUriRequest request) {CloseableHttpResponse response = null;try {response = httpClient.execute(request);if (response == null) {return JSONObject.toJSONString(new Failed("100000", "返回信息为空"));} else {if (response.getStatusLine().getStatusCode() == SUCCESS) {return EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);} else {return JSONObject.toJSONString(new Failed(response.getStatusLine().getStatusCode() + "", "返回信息失败"));}}} catch (IOException e) {log.warn("执行失败", e);return JSONObject.toJSONString(new Failed("100001", e.getMessage()));} finally {try {if (response != null) {response.close();}if (httpClient != null) {httpClient.close();}} catch (IOException e) {}}}/*** 根据URL下载文件信息，保存在指定的目录下面** @param urlStr* @param fileName* @param savePath*/public static void downLoadFromUrl(String urlStr, String fileName, String savePath) {downLoadFromUrl(urlStr, fileName, savePath, null, null);}/*** 从网络Url中下载文件* http://suyun-nevtsp-foreverfile.oss-cn-shenzhen.aliyuncs.com/vehicle_admin/dbc_excel/2020/11/16/L16_V1.1_20201116095257890.xlsx** @param urlStr   下载URL* @param fileName 下载文件名* @param savePath 保存的路径* @param tokeName Token 名称* @param token    需要的TOKEN新* @throws IOException*/public static void downLoadFromUrl(String urlStr, String fileName, String savePath, String tokeName, String token) {FileOutputStream fos = null;InputStream is = null;try {URL url = new URL(urlStr);HttpURLConnection conn = (HttpURLConnection) url.openConnection();//设置超时间为3秒conn.setConnectTimeout(CONNECTION_TIMEOUT_MS);//防止屏蔽程序抓取而返回403错误conn.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)");if (StringUtils.isNotEmpty(token)) {conn.setRequestProperty(tokeName, token);}//得到输入流is = conn.getInputStream();//获取自己数组byte[] getData = readInputStream(is);//文件保存位置File saveDir = new File(savePath);if (!saveDir.exists()) {saveDir.mkdir();}File file = new File(saveDir + File.separator + fileName);fos = new FileOutputStream(file);fos.write(getData);} catch (IOException e) {e.printStackTrace();} finally {try {if (fos != null) {fos.close();}if (is != null) {is.close();}} catch (IOException e) {}}}/*** 添加头部信息，判断是否使用代理** @param httpRequestBase* @param httpClientParam*/private static void buildHeaderProxy(HttpRequestBase httpRequestBase, final HttpClientParam httpClientParam) {addHeader(httpRequestBase, httpClientParam.getHeaderMap());if (StringUtils.isEmpty(httpClientParam.getProxy()) || !httpClientParam.getProxy().contains(StaticConstant.Common.COLON)) {httpRequestBase.setConfig(RequestConfig.custom().setSocketTimeout(SO_TIMEOUT_MS).setConnectTimeout(CONNECTION_TIMEOUT_MS).build());return;}Proxy proxy = null;if (httpClientParam.getProxy().startsWith(StaticConstant.Common.HTTP)) {proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(httpClientParam.getProxy().replace(StaticConstant.Common.HTTP, StaticConstant.Common.EMPTY_STR), HTTP_PORT));} else if (httpClientParam.getProxy().startsWith(StaticConstant.Common.HTTPS)) {proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(httpClientParam.getProxy().replace(StaticConstant.Common.HTTPS, StaticConstant.Common.EMPTY_STR), HTTPS_PORT));} else {String[] ipPort = httpClientParam.getProxy().split(StaticConstant.Common.COLON);proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(ipPort[0], Integer.parseInt(ipPort[1])));}InetSocketAddress socketAddress = (InetSocketAddress) proxy.address();String ip = socketAddress.getAddress().getHostAddress();Integer port = socketAddress.getPort();if (StringUtils.isNotEmpty(ip) && port != null) {HttpHost httpHost = new HttpHost(ip, port);httpRequestBase.setConfig(RequestConfig.custom().setProxy(httpHost).setSocketTimeout(SO_TIMEOUT_MS).setConnectTimeout(CONNECTION_TIMEOUT_MS).build());} else {httpRequestBase.setConfig(RequestConfig.custom().setSocketTimeout(SO_TIMEOUT_MS).setConnectTimeout(CONNECTION_TIMEOUT_MS).build());}if (httpRequestBase.getHeaders("X-Forwarded-For") == null || httpRequestBase.getHeaders("X-Forwarded-For").length == 0) {httpRequestBase.addHeader("X-Forwarded-For", httpClientParam.getForwardIp());}}private static void addHeader(HttpRequestBase httpRequestBase, Map<String, String> headerMap) {if (!CollectionUtils.isEmpty(headerMap)) {for (Map.Entry<String, String> header : headerMap.entrySet()) {httpRequestBase.addHeader(header.getKey(), header.getValue());}}if (headerMap.containsKey("User-Agent")) {return;}httpRequestBase.addHeader("User-Agent", USER_AGENT_ALL[RANDOM.nextInt(USER_AGENT_ALL.length - 1)]);}/*** 从输入流中获取字节数组** @param inputStream* @return* @throws IOException*/private static byte[] readInputStream(InputStream inputStream) throws IOException {byte[] buffer = new byte[DOWN_LOAD_READ_BYTE_BUFFER];int len = 0;ByteArrayOutputStream bos = new ByteArrayOutputStream();while ((len = inputStream.read(buffer)) != -1) {bos.write(buffer, 0, len);}bos.close();return bos.toByteArray();}/*** 访问参数*/@Getterpublic static class HttpClientParam {/*** 拼装之后的URL信息*/private String url;/*** 访问客户端*/private CloseableHttpClient closeableHttpClient;/*** 访问协议IP和端口例如：http://10.0.0.5:8080 不能为空*/private String hostPort;/*** 访问的方法 例如： /a/query 可以为空*/private String path;/*** URL带的参数信息 key 为name value为值 可以为空 ，host+path+url会拼接成真正的URL*/private Map<String, String> urlParamMap = Maps.newHashMap();/*** 头部参数 key 为name value为值 可以为空*/private Map<String, String> headerMap = Maps.newHashMap();/*** FormData 参数信息*/private Map<String, String> bodyMap = Maps.newHashMap();/*** body字符串*/private String bodyStr;/*** body byte数组 bodyMap，bodyStr，body三个只能使用一个，优先级 bodyMap>bodyStr>body*/private byte[] body;/*** 是否使用代理，如果为空，表示不代理 10.0.0.1:8080*/private String proxy;private String forwardIp;private HttpClientParam() {}/*** 访问协议IP和端口例如：http://10.0.0.5:8080 不能为空*/public HttpClientParam addHostPort(String hostPort) {this.hostPort = hostPort;return this;}/*** 访问的方法映射 例如： /a/query 可以为空*/public HttpClientParam addPath(String path) {this.path = path;return this;}/*** URL带的参数信息 key 为name value为值 可以为空 ，host+path+url会拼接成真正的URL** @param key* @param value* @return*/public HttpClientParam addQuery(String key, String value) {this.urlParamMap.put(key, value);return this;}/*** URL带的参数信息 key 为name value为值 可以为空 ，host+path+url会拼接成真正的URL** @param queryMap* @return*/public HttpClientParam addQueryAll(Map<String, String> queryMap) {this.urlParamMap.putAll(queryMap);return this;}/*** 头部参数 key 为name value为值 可以为空** @param key* @param value* @return*/public HttpClientParam addHeader(String key, String value) {this.headerMap.put(key, value);return this;}/*** 头部参数 key 为name value为值 可以为空** @param headerMap* @return*/public HttpClientParam addHeaderAll(Map<String, String> headerMap) {this.headerMap.putAll(headerMap);return this;}/*** FormData 参数信息 bodyMap，bodyStr，body三个只能使用一个，优先级 bodyMap>bodyStr>body** @param key* @param value*/public void addBodyMap(String key, String value) {this.bodyMap.put(key, value);}/*** FormData 参数信息 bodyMap，bodyStr，body三个只能使用一个，优先级 bodyMap>bodyStr>body** @param bodyMap*/public void addBodyMapAll(Map<String, String> bodyMap) {this.bodyMap.putAll(bodyMap);}/*** body字符串 bodyMap，bodyStr，body三个只能使用一个，优先级 bodyMap>bodyStr>body** @param bodyStr* @return*/public HttpClientParam addBodyStr(String bodyStr) {this.bodyStr = bodyStr;return this;}/*** byte数组 bodyMap，bodyStr，body三个只能使用一个，优先级 bodyMap>bodyStr>body** @param body* @return*/public HttpClientParam addBody(byte[] body) {this.body = body;return this;}/*** 是否使用代理，如果为空，表示不代理 10.0.0.1:8080** @param proxy* @return*/public HttpClientParam addProxy(String proxy) {this.proxy = proxy;return this;}/*** 创建对象信息*/public HttpClientParam build() {boolean isNoValid = StringUtils.isEmpty(this.hostPort) || (!this.hostPort.startsWith(StaticConstant.Common.HTTP) && !this.hostPort.startsWith(StaticConstant.Common.HTTPS));if (isNoValid) {log.warn("路径信息错误 hostPort:{}", this.hostPort);return null;}if (this.hostPort.endsWith(StaticConstant.Common.SLASH)) {this.hostPort = this.hostPort.substring(0, this.hostPort.lastIndexOf(StaticConstant.Common.COLON));}if (StringUtils.isNotEmpty(this.path) && !this.path.startsWith(StaticConstant.Common.SLASH)) {this.path = StaticConstant.Common.SLASH + this.path;}try {if (this.proxy != null) {String tempProxy = null;this.forwardIp = this.proxy.substring(0, this.proxy.lastIndexOf(StaticConstant.Common.COLON));if (this.hostPort.startsWith(StaticConstant.Common.HTTP)) {tempProxy = this.hostPort.replace(StaticConstant.Common.HTTP, StaticConstant.Common.EMPTY_STR);this.hostPort = StaticConstant.Common.HTTP + this.proxy;} else {tempProxy = this.hostPort.replace(StaticConstant.Common.HTTPS, StaticConstant.Common.EMPTY_STR);this.hostPort = StaticConstant.Common.HTTPS + this.proxy;}this.proxy = tempProxy;}this.url = buildUrl(this.hostPort, this.path, this.urlParamMap);this.closeableHttpClient = wrapClient(this.hostPort);} catch (Exception e) {log.warn("创建访问对象错误", e);return null;}return this;}/*** 拼接URL** @param host* @param path* @param queryMap* @return* @throws UnsupportedEncodingException*/private String buildUrl(String host, String path, Map<String, String> queryMap) {StringBuilder sbUrl = new StringBuilder();sbUrl.append(host);if (!StringUtils.isBlank(path)) {sbUrl.append(path);}if (CollectionUtils.isEmpty(queryMap)) {return sbUrl.toString();}StringBuilder sbQuery = new StringBuilder();for (Map.Entry<String, String> query : queryMap.entrySet()) {if (sbQuery.length() > 0) {sbQuery.append(StaticConstant.Common.AND);}if (StringUtils.isBlank(query.getKey()) && !StringUtils.isBlank(query.getValue())) {sbQuery.append(query.getValue());}if (!StringUtils.isBlank(query.getKey())) {sbQuery.append(query.getKey());if (!StringUtils.isBlank(query.getValue())) {sbQuery.append("=");try {sbQuery.append(URLEncoder.encode(query.getValue(), StandardCharsets.UTF_8.name()));} catch (UnsupportedEncodingException e) {log.warn("编码失败 value: {}", query.getValue());}}}}if (sbQuery.length() > 0) {sbUrl.append(StaticConstant.Common.UN_KNOW).append(sbQuery);}return sbUrl.toString();}private CloseableHttpClient wrapClient(String host) {CloseableHttpClient httpClient = HttpClients.createDefault();if (host.startsWith(StaticConstant.Common.HTTPS)) {SSLContext sslContext = null;try {sslContext = SSLContexts.custom().loadTrustMaterial(null, (x509Certificates, s) -> {return true;}).build();} catch (NoSuchAlgorithmException | KeyManagementException | KeyStoreException e) {e.printStackTrace();}//创建httpClientreturn HttpClients.custom().setSSLContext(sslContext).setSSLHostnameVerifier(new NoopHostnameVerifier()).build();}return httpClient;}}private static final String[] USER_AGENT_ALL = {"Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36","Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36","Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2224.3 Safari/537.36","Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:21.0.0) Gecko/20121011 Firefox/21.0.0","Mozilla/5.0 (Windows NT 6.1; WOW64; rv:64.0) Gecko/20100101 Firefox/64.0","Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10; rv:33.0) Gecko/20100101 Firefox/33.0","Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/22.0.1207.1 Safari/537.1","Mozilla/5.0 (X11; CrOS i686 2268.111.0) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11","Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1092.0 Safari/536.6","Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1090.0 Safari/536.6","Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/19.77.34.5 Safari/537.1","Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.9 Safari/536.5","Mozilla/5.0 (Windows NT 6.0) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.36 Safari/536.5","Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1063.0 Safari/536.3","Mozilla/5.0 (Windows NT 5.1) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1063.0 Safari/536.3","Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_0) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1063.0 Safari/536.3","Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1062.0 Safari/536.3","Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1062.0 Safari/536.3","Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1 Safari/536.3","Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1 Safari/536.3","Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1 Safari/536.3","Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.0 Safari/536.3","Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.24 (KHTML, like Gecko) Chrome/19.0.1055.1 Safari/535.24","Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/535.24 (KHTML, like Gecko) Chrome/19.0.1055.1 Safari/535.24","Mozilla/5.0 (Macintosh; U; Mac OS X Mach-O; en-US; rv:2.0a) Gecko/20040614 Firefox/3.0.0 ","Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9.0.3) Gecko/2008092414 Firefox/3.0.3","Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1) Gecko/20090624 Firefox/3.5","Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.14) Gecko/20110218 AlexaToolbar/alxf-2.0 Firefox/3.6.14","Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9.2.15) Gecko/20110303 Firefox/3.6.15","Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0.1) Gecko/20100101 Firefox/4.0.1","Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10","Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/533.17.8 (KHTML, like Gecko) Version/5.0.1 Safari/533.17.8","Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5","Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0","Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)","Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)","Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"};
}

private static void proxy() {String result = HttpUtils.doGet(HttpUtils.create()//访问的服务器IP.addHostPort("http://116.30.240.246:8080")//访问的方法.addPath("/a/login")//代理IP和端口

效果：

使用代理

不用代理

StaticConstant.Common.COLON=“:”;
StaticConstant.Common.HTTP=“http”;
StaticConstant.Common.HTTPS=“https”;
StaticConstant.Common.EMPTY_STR=“”;
StaticConstant.Common.SLASH=“/”;
StaticConstant.Common.AND=“&”;
StaticConstant.Common.UN_KNOW=“?”;
Filed类里面有code errorMsg2属性；

HttpClient使用代理相关推荐

java httpClient使用代理实现互联网公网访问
项目生产环境往往会有这样的需求,内网的生产环境的应用需要访问互联网上第三方的服务,但又出于安全考虑,不允许该生产机直接访问互联网,这时我们往往通过代理方式来实现网外服务的访问:我们可以在可访问互联网的 ...
httpclient动态代理ip
在爬取网页的时候,有的目标站点有反爬虫机制,对于频繁访问站点以及规则性访问站点的行为,会采集屏蔽IP措施. 这时候,代理IP就派上用场了. 关于代理IP的话也分几种透明代理.匿名代理.混淆代理.高 ...
android HttpClient 设置代理
HttpClient client = new HttpClient();//设置代理.转发到哪个 ip 和 port , 192.168.3.196 是我本机的charles 地址和端口clien ...
httpclient设置代理
原来用gae的时候折腾过java自带代理.然而并没有什么卵用. 现在用gae也时常不稳定,处于基本报废状态.今天找到了另外一个可用的,并且发现httpclient也有自带的设置代理功能.代码如下: H ...
HttpURLConnection和httpclient添加代理IP和端口
/*代理设置*/ HttpClient httpclient = new DefaultHttpClient(); HttpHost proxy = new HttpHost(System.getPr ...
.NetCore HttpClient Proxy 设置全局代理
.NetCore HttpClient Proxy 设置全局代理环境 .net6.0,使用的是 HttpClient 在Program.cs里面 service.AddHttpClient<H ...
HttpClient使用Ip代理访问目标IP
一.HttpClient使用代理IP 1.1.前言在爬取网页的时候,有的目标站点有反爬虫机制,对于频繁访问站点以及规则性访问站点的行为,会采集屏蔽IP措施. 这时候,代理IP就派上用场了.可以使用代 ...
HttpClient 教程 (二)
转自:http://www.cnblogs.com/loveyakamoz/archive/2011/07/21/2112832.html 第二章连接管理 HttpClient有一个对连接初始化和终 ...
httpclient 学习
Http协议的重要性相信不用我多说了,HttpClient相比传统JDK自带的URLConnection,增加了易用性和灵活性,它不仅是客户端发送Http请求变得容易,而且也方便了开发人员测试接口(基 ...

HttpClient使用代理

HttpClient使用代理相关推荐

最新文章

热门文章