org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet

hbase 进行 javaAPI 操作时报了如下错误 :

org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=32, exceptions:
Tue Dec 17 10:21:47 CST 2019, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=68473: row 'myUser,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=hadoop01,60020,1576455547324, seqNum=0

Caused by: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet

报错详情如下:

org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=32, exceptions:
Tue Dec 17 10:21:47 CST 2019, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=68473: row 'myUser,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=hadoop01,60020,1576455547324, seqNum=0at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.throwEnrichedException(RpcRetryingCallerWithReadReplicas.java:320)at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:247)at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:62)at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:327)at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:302)at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:167)at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:162)at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:862)at org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:421)at com.czxy.HBaseAPI.createTable(HBaseAPI.java:40)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:498)at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)at org.junit.runners.ParentRunner.run(ParentRunner.java:363)at org.junit.runner.JUnitCore.run(JUnitCore.java:137)at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
Caused by: java.net.SocketTimeoutException: callTimeout=60000, callDuration=68473: row 'myUser,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=hadoop01,60020,1576455547324, seqNum=0at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:169)at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:80)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yetat org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1099)at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2311)at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33648)at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2191)at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:183)at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:163)at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:423)at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:332)at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:408)at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:204)at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:65)at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:397)at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:371)at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:136)... 4 more
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.ipc.ServerNotRunningYetException): org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yetat org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1099)at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2311)at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33648)at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2191)at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:183)at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:163)at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1272)at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:34094)at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:400)... 10 moreProcess finished with exit code -1

问题排查 :

1.一开是看到如下这段

org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=32, exceptions:
Tue Dec 17 10:21:47 CST 2019, null, java.net.SocketTimeoutException: callTimeout=60000, callDuration=68473: row 'myUser,,' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=hadoop01,60020,1576455547324, seqNum=0

去百度找,百度上都说是hosts文件的问题,但经过实际排查却发现跟hosts文件没一点关系

2.后来看到了 Server is not running yet 这个错,发现确实是集群处于安全模式,不知道什么时候namenode掉了,然后重启了一下集群,并查看集群状态直到集群退出了安全模式再次运行,发现还是报错.
3.经过排查,发现还需要重启hbase(保险起见顺便把zookeeper一起重启了)
然后问题解决.

报错原因:

集群处于安全模式

解决 :

1.查看集群状态

hdfs dfsadmin -safemode get
//如果返回Safe mode is OFF 就说明没问题
//如果返回Safe mode is ON 就说明集群正处于安全模式(强制退出即可)
hdfs dfsadmin -safemode leave//如果返回其他就说明集群出问题了 例如 返回如下内容
safemode: Call From hadoop01/192.168.100.100 to hadoop01:8020 failed on connection exception: java.net.ConnectException: 拒绝连接; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused检查之后发现是namenode掉了,重启了一下集群.建议执行完上述的操作之后重启hbase

org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet相关推荐

【hbase问题】org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not run
启动后无论什么命令都会报Server is not run,查询日志后发现各种问题都有,但是没得都不能解决问题.遂一气之下重装hbase,然后重新启动,依旧报错,查看日志,发现日志有新的报错信息经排 ...
解决Hbase输入list，创建表报错，输入status可以，org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: S
今天恢复了一下快照,然后hbase出现了问题 2019-08-25 00:59:55,810 INFO [RpcServer.default.FPBQ.Fifo.handler=28,queue=1, ...
HBase报错server is not running yet解决方法
问题描述尝试安装Hadoop 3.3.1与HBase 2.3.5,安装并配置完成后,Hadoop正常运行,HBase启动Shell后运行list指令测试,出现如下错误 ERROR: org.apac ...
org.apache.hadoop.hbase.PleaseHoldException: Master is initializing(解決方案汇总＋自己摸索)
#--------------------------------------环境(start)---------------------------------------------------- ...
hbase报错：org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
报错 hbase(main):002:0> status 1 active master, 0 backup masters, 2 servers, 0 dead, 0.0000 average ...
org.apache.hadoop.hbase.NotServingRegionException: hbase:meta,,1 is not online问题（暂时没有解决）
在试图使用 sqoop从mysql导数据到hbase的过程中,发生下面报错: 2020-06-30 11:53:38,636 ERROR [main] tool.ImportTool (ImportT ...
启动HBase抛出org.apache.hadoop.hbase.ClockOutOfSyncException异常：hmaster正常，节点hregionserver启动失败
启动HBase抛出org.apache.hadoop.hbase.ClockOutOfSyncException异常原因节点间时间不一致,时间同步出了问题: 解决进行时间同步: [root@cm ...
org.apache.hadoop.hbase.ClockOutOfSyncException
如果集群节点的系统时间和master的时间不一致的话,从节点启动不起来,报错如下: org.apache.hadoop.hbase.ClockOutOfSyncException: org.apach ...
HBase出现java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration问题
问题:Hbase在集群上运行报错:NoClassDefFoundError:org/apache/hadoop/hbase/HBaseConfiguration 需求:HBase使用Java创建表,打 ...