【Redis连接超时】io.lettuce.core.RedisConnectionException: Unable to connect to 192.168.x.x:7000

「扫码关注我，面试、各种技术（mysql、zookeeper、微服务、redis、jvm）持续更新中～」

今天临近下班了，线上开始频繁报警，各种Redis连接超时，顿时慌的一批，因为最近在优化系统高频查询时用到了Redis作为缓存，难道要出生产事故，额～～～一首凉凉送给自己。。。。。。
于是马上联系下运维看下什么情况，运维看了下监控情况，OPS（operation per second）确实增加了不少，见下图：

于是乎发现确实是自己的锅，二话不说，先回复线上优化查询前的版本，保证线上能够正确运行，减少事故造成的影响，发完版本后，报警顿时停止，好了，开始找BUG吧～
先去ELK楼下日志，发现日志如下：

org.springframework.data.redis.RedisSystemException: Redis exception; nested exception is io.lettuce.core.RedisException: io.lettuce.core.RedisConnectionException: Unable to connect to 192.168.x.x:7000at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:74)at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:41)at org.springframework.data.redis.PassThroughExceptionTranslationStrategy.translate(PassThroughExceptionTranslationStrategy.java:44)at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:42)at org.springframework.data.redis.connection.lettuce.LettuceConnection.convertLettuceAccessException(LettuceConnection.java:257)at org.springframework.data.redis.connection.lettuce.LettuceHashCommands.convertLettuceAccessException(LettuceHashCommands.java:445)at org.springframework.data.redis.connection.lettuce.LettuceHashCommands.hGet(LettuceHashCommands.java:171)at org.springframework.data.redis.connection.DefaultedRedisConnection.hGet(DefaultedRedisConnection.java:855)at org.springframework.data.redis.connection.DefaultStringRedisConnection.hGet(DefaultStringRedisConnection.java:429)at org.springframework.data.redis.core.DefaultHashOperations.lambda$get$0(DefaultHashOperations.java:52)at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:224)at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:184)at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:95)at org.springframework.data.redis.core.DefaultHashOperations.get(DefaultHashOperations.java:52)at com.xes.merchant.service.impl.InvoiceServiceImpl.getCacheByRegIdAndStagesId(InvoiceServiceImpl.java:336)at com.xes.merchant.service.impl.InvoiceServiceImpl$$FastClassBySpringCGLIB$$d92d6c6c.invoke(<generated>)at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:684)at com.xes.merchant.service.impl.InvoiceServiceImpl$$EnhancerBySpringCGLIB$$cf8e4582.getCacheByRegIdAndStagesId(<generated>)at com.xes.merchant.service.impl.ForOrdersServiceImpl.queryFinanceListByRegistId(ForOrdersServiceImpl.java:269)at com.xes.merchant.controller.ForOrdersController.queryFinanceListByRegistId(ForOrdersController.java:182)at sun.reflect.GeneratedMethodAccessor130.invoke(Unknown Source)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209)at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102)at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:877)at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:783)at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991)at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925)at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974)at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:877)at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851)at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:158)at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.filterAndRecordMetrics(WebMvcMetricsFilter.java:126)at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:111)at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)at org.springframework.boot.actuate.web.trace.servlet.HttpTraceFilter.doFilterInternal(HttpTraceFilter.java:90)at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)at com.xes.payment.safe.filter.XssFilter.doFilter(XssFilter.java:64)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:130)at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:66)at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:105)at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:123)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:650)at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800)at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806)at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)at java.lang.Thread.run(Thread.java:745)
Caused by: io.lettuce.core.RedisException: io.lettuce.core.RedisConnectionException: Unable to connect to 192.168.x.x:7000at io.lettuce.core.LettuceFutures.awaitOrCancel(LettuceFutures.java:125)at io.lettuce.core.cluster.ClusterFutureSyncInvocationHandler.handleInvocation(ClusterFutureSyncInvocationHandler.java:118)at io.lettuce.core.internal.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:80)at com.sun.proxy.$Proxy156.hget(Unknown Source)at org.springframework.data.redis.connection.lettuce.LettuceHashCommands.hGet(LettuceHashCommands.java:169)... 76 common frames omitted

异常已经很明显了，那么接下来就是如何去解决问题了，但是自己也好奇，Redis的性能应该非常高啊，但是为什么会出现连接超时呢？
首先描述下业务背景，该接口是查询财务交易的接口，该接口中包含财务支付核心信息还有是否开过发票，并且该接口还是个批量查询接口，就是一次性可以传递不超过50个报名ID，根据报名ID集合去查询数据库中的财务信息和发票信息。

由于是以传过来报名ID集合拼接作为Redis的key，所以当发票状态改变时无法找到对应的报名ID的缓存，因此，基于以上情况，设置为2个缓存来分别存放财务信息和发票信息，财务记录存取采用Redis的String结构来存放，发票信息财务Hash结构存放。

当第一次查询财务发票接口时，首先在缓存中获取，如果缓存中没有，则直接查询数据库，然后将查询当财务信息存取到缓存中，同时将发票信息也存储到对应的缓存中（单条操作），这样如果一次传过来多个报名ID，同样需要设置多次发票缓存。

情况已经比较明了了，经过分析发现存在以下几方面的问题：
1.Redis缓存key是报名ID拼接的，每一个报名ID是32位，那么如果传过来是批量报名ID，那么一次行传50个的话，拼接起来作为key，那么key就会有1600字符，如果缓存数量过多，那么在缓存Get时，就不利于缓存命中，严重影响OPS，所以将key进行MD5加密后进行存储，减小key的长度；
2.排查SpringBoot整合Redis时，发现配置文件中并没有重新设置连接池大小，而是使用的默认值，同时连接超时时间（连接断开等待时间）也没有进行设置，当请求量上来也会影响Redis的OPS，所以重新配置了下，如下：

# 连接池最大连接数（使用负值表示没有限制）
spring.redis.lettuce.pool.max-active=300
# 连接池中的最大空闲连接
spring.redis.lettuce.pool.max-idle=100
# 连接池中的最小空闲连接
spring.redis.lettuce.pool.min-idle=8
# 连接池最大阻塞等待时间（使用负值表示没有限制）
spring.redis.lettuce.pool.max-wait=-1ms
# 连接超时时间（毫秒）
spring.redis.timeout=12000ms

3.采用管道方式（executePipelined）批量设置发票缓存，同时获取也采用批量的方式（multiGet），代码如下：

//批量设置
redisTemplate.executePipelined(new RedisCallback<Object>() {@Overridepublic Object doInRedis(RedisConnection connection) throws DataAccessException {for(FinanceVO financeVO : financeVOList){String key = StringUtils.join(RedisConstant.INVOICE_BY_REGIDANDSTAGESID,financeVO.geRegtId(),financeVO.getStagesId());//获取序列化器RedisSerializer<String> keySerializer = (RedisSerializer<String>) redisTemplate.getKeySerializer();RedisSerializer<String> hashKeySerializer = (RedisSerializer<String>) redisTemplate.getHashKeySerializer();RedisSerializer<String> hashValueSerializer = (RedisSerializer<String>) redisTemplate.getHashValueSerializer();byte[] rawKey = keySerializer.serialize(key);byte[] invoiceIdField = hashKeySerializer.serialize("id");byte[] invoiceStatusField = hashKeySerializer.serialize("status");byte[] invoiceIdValue = hashValueSerializer.serialize(financeVO.getId());byte[] invoiceStatusValue = hashValueSerializer.serialize(financeVO.getStatus());Map<byte[],byte[]> hashes = new HashMap<>();hashes.put(invoiceIdField,invoiceIdValue);hashes.put(invoiceStatusField,invoiceStatusValue);connection.hMSet(rawKey,hashes);connection.expire(rawKey,invoiceTimeout*24*60*60);}return null;}});//批量获取List<String> values = redisTemplate.opsForHash().multiGet(key,hashKeys);

经过改造之后，重新上线，OPS值正常了，说明以上优化是起作用的，同时，也在以后的开发工作中积累了经验，但是缓存数据也会出现各种个样的问题，主要是数据库缓存不一致等问题，所以，缓存数据应该是基本保持不变的数据，不然可能导致很多问题。

【Redis连接超时】io.lettuce.core.RedisConnectionException: Unable to connect to 192.168.x.x:7000相关推荐

unable to connect to 192.168.1.110:5555 解决办法
问题记录测试想要adb wife连接Android设备时,经常碰到如下错误 unable to connect to 192.168.x.xxx:5555: cannot connect to 19 ...
unable to connect to 192.168.1.110:5555 解决办法（不用RootAndroid设备）
问题 adb通过wifi调试Android设备,在连接过程中经常显示以下错误: Windows下:unable to connect to 192.168.1.110:5555: cannot con ...
springboot连接redis 没有权限 io.lettuce.core.RedisCommandExecutionException: NOAUTH Authentication requir
io.lettuce.core.RedisCommandExecutionException: NOAUTH Authentication required 解决方案: 1.检查一下你linux端口有 ...
无线调试和unable to connect to 192.168.2.245:5555由于目标计算机积极拒绝，无法连接。 (10061)
去年调试设备使用了无线调试,主要是设备也没有usb线,想要有线调试也不行,就去找了相应的博客来看了看,前段时间有人问了我一些这方面的问题,这里记录一下命令,希望能帮助到有需要的人.一般来说只需要以 ...
springboot:redis报错io.lettuce.core.RedisCommandExecutionException: ERR Client sent AUTH, but no passw
一.报错原因:下载安装的redis默认是没有密码的. 所以在项目中配置文件中,连接redis,配置了密码.就会报错. 二.解决:给redis设置密码 1.命令窗口,进入到redis所在的目录下 2.然 ...
RedisConnectionException: Unable to connect to localhost:6379
本地项目启动,没有配置redis,自动连接了本地地址, org.springframework.data.redis.RedisConnectionFailureException: Unable t ...
解决 cannot connect to 192.168.1.136:5555: 由于目标计算机积极拒绝，无法连接。 (10061)
通过WiFi连接adb调试解决 cannot connect to 192.168.1.136:5555: 由于目标计算机积极拒绝,无法连接. (10061) 解决办法1 (Windows下cmd环 ...
Redis - Redis command timed out nested exception is io.lettuce.core.RedisCommandTimeoutException
Redis - Redis command timed out nested exception is io.lettuce.core.RedisCommandTimeoutException 这个问 ...
解决阿里云服务器redis连接超时
我的问题背景 SpringBoot项目部署到linux,需要使用服务器本机的redis(自己写的一个小网站),阿里云开放redis 6379端口.redis配置也没问题,但是连接不上原因: 当你修改 ...

【Redis连接超时】io.lettuce.core.RedisConnectionException: Unable to connect to 192.168.x.x:7000

【Redis连接超时】io.lettuce.core.RedisConnectionException: Unable to connect to 192.168.x.x:7000相关推荐

最新文章

热门文章