Spark1.4 和 Hive 1.1.1 启动错误

启动Spark-sql

[jifeng@feng03 spark-1.4.0-bin-hadoop2.6]$ ./bin/spark-sql  --master spark://feng03:7077
Exception in thread "main" java.lang.RuntimeException: java.lang.NumberFormatException: For input string: "1s"at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346)at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:109)at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:606)at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:664)at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:169)at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:192)at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:111)at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.NumberFormatException: For input string: "1s"at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)at java.lang.Integer.parseInt(Integer.java:492)at java.lang.Integer.parseInt(Integer.java:527)at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1134)at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:1211)at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:1220)at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:58)at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453)at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465)at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340)... 11 more
15/09/05 14:47:31 INFO util.Utils: Shutdown hook called
15/09/05 14:47:31 INFO util.Utils: Deleting directory /tmp/spark-3fabb880-f749-41d9-9a4f-09e704e65cf3
[jifeng@feng03 spark-1.4.0-bin-hadoop2.6]$ ./bin/spark-sql --master spark://feng03:7077 --driver-class-path /home/jifeng/hive/mysql-connector-java-5.1.34.jar
Exception in thread "main" java.lang.RuntimeException: java.lang.NumberFormatException: For input string: "1s"at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346)at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:109)at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:606)at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:664)at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:169)at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:192)at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:111)at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.NumberFormatException: For input string: "1s"at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)at java.lang.Integer.parseInt(Integer.java:492)at java.lang.Integer.parseInt(Integer.java:527)at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1134)at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:1211)at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:1220)at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:58)at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453)at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465)at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340)... 11 more
15/09/05 14:48:18 INFO util.Utils: Shutdown hook called
15/09/05 14:48:18 INFO util.Utils: Deleting directory /tmp/spark-72aa6ae8-5bba-437f-8636-fd0a2fe72d25

启动Spark-shell

[jifeng@feng03 hive]$ ./bin/spark-shell --master spark://feng03:7077 --driver-class-path /home/jifeng/hive/mysql-connector-java-5.1.34.jar
-bash: ./bin/spark-shell: No such file or directory
[jifeng@feng03 hive]$ cd ..
[jifeng@feng03 ~]$ cd spark-1.4.0-bin-hadoop2.6/
[jifeng@feng03 spark-1.4.0-bin-hadoop2.6]$ ./bin/spark-shell --master spark://feng03:7077 --driver-class-path /home/jifeng/hive/mysql-connector-java-5.1.34.jar
15/09/05 14:50:29 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/09/05 14:50:29 INFO spark.SecurityManager: Changing view acls to: jifeng
15/09/05 14:50:29 INFO spark.SecurityManager: Changing modify acls to: jifeng
15/09/05 14:50:29 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(jifeng); users with modify permissions: Set(jifeng)
15/09/05 14:50:29 INFO spark.HttpServer: Starting HTTP Server
15/09/05 14:50:30 INFO server.Server: jetty-8.y.z-SNAPSHOT
15/09/05 14:50:30 INFO server.AbstractConnector: Started SocketConnector@0.0.0.0:44520
15/09/05 14:50:30 INFO util.Utils: Successfully started service 'HTTP class server' on port 44520.
Welcome to____              __/ __/__  ___ _____/ /___\ \/ _ \/ _ `/ __/  '_//___/ .__/\_,_/_/ /_/\_\   version 1.4.0/_/Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_79)
Type in expressions to have them evaluated.
Type :help for more information.
15/09/05 14:50:41 INFO spark.SparkContext: Running Spark version 1.4.0
15/09/05 14:50:41 INFO spark.SecurityManager: Changing view acls to: jifeng
15/09/05 14:50:41 INFO spark.SecurityManager: Changing modify acls to: jifeng
15/09/05 14:50:41 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(jifeng); users with modify permissions: Set(jifeng)
15/09/05 14:50:41 INFO slf4j.Slf4jLogger: Slf4jLogger started
15/09/05 14:50:42 INFO Remoting: Starting remoting
15/09/05 14:50:42 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriver@192.168.0.110:45394]
15/09/05 14:50:42 INFO util.Utils: Successfully started service 'sparkDriver' on port 45394.
15/09/05 14:50:42 INFO spark.SparkEnv: Registering MapOutputTracker
15/09/05 14:50:43 INFO spark.SparkEnv: Registering BlockManagerMaster
15/09/05 14:50:43 INFO storage.DiskBlockManager: Created local directory at /tmp/spark-21bdebec-d78f-4491-bd03-52defdf21f68/blockmgr-7206fb4f-673f-40dd-b8dd-3b178ee39938
15/09/05 14:50:43 INFO storage.MemoryStore: MemoryStore started with capacity 267.3 MB
15/09/05 14:50:43 INFO spark.HttpFileServer: HTTP File server directory is /tmp/spark-21bdebec-d78f-4491-bd03-52defdf21f68/httpd-8c3af065-b7fe-4fb4-a5bd-8dde0b236698
15/09/05 14:50:43 INFO spark.HttpServer: Starting HTTP Server
15/09/05 14:50:43 INFO server.Server: jetty-8.y.z-SNAPSHOT
15/09/05 14:50:43 INFO server.AbstractConnector: Started SocketConnector@0.0.0.0:37858
15/09/05 14:50:43 INFO util.Utils: Successfully started service 'HTTP file server' on port 37858.
15/09/05 14:50:43 INFO spark.SparkEnv: Registering OutputCommitCoordinator
15/09/05 14:50:44 INFO server.Server: jetty-8.y.z-SNAPSHOT
15/09/05 14:50:44 INFO server.AbstractConnector: Started SelectChannelConnector@0.0.0.0:4040
15/09/05 14:50:44 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
15/09/05 14:50:44 INFO ui.SparkUI: Started SparkUI at http://192.168.0.110:4040
15/09/05 14:50:45 INFO client.AppClient$ClientActor: Connecting to master akka.tcp://sparkMaster@feng03:7077/user/Master...
15/09/05 14:50:45 INFO cluster.SparkDeploySchedulerBackend: Connected to Spark cluster with app ID app-20150905145045-0000
15/09/05 14:50:46 INFO client.AppClient$ClientActor: Executor added: app-20150905145045-0000/0 on worker-20150905101207-192.168.0.110-58490 (192.168.0.110:58490) with 1 cores
15/09/05 14:50:46 INFO cluster.SparkDeploySchedulerBackend: Granted executor ID app-20150905145045-0000/0 on hostPort 192.168.0.110:58490 with 1 cores, 512.0 MB RAM
15/09/05 14:50:46 INFO client.AppClient$ClientActor: Executor updated: app-20150905145045-0000/0 is now RUNNING
15/09/05 14:50:46 INFO client.AppClient$ClientActor: Executor updated: app-20150905145045-0000/0 is now LOADING
15/09/05 14:50:46 INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 57911.
15/09/05 14:50:46 INFO netty.NettyBlockTransferService: Server created on 57911
15/09/05 14:50:46 INFO storage.BlockManagerMaster: Trying to register BlockManager
15/09/05 14:50:46 INFO storage.BlockManagerMasterEndpoint: Registering block manager 192.168.0.110:57911 with 267.3 MB RAM, BlockManagerId(driver, 192.168.0.110, 57911)
15/09/05 14:50:46 INFO storage.BlockManagerMaster: Registered BlockManager
15/09/05 14:50:47 INFO cluster.SparkDeploySchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0
15/09/05 14:50:47 INFO repl.SparkILoop: Created spark context..
Spark context available as sc.
15/09/05 14:50:50 INFO hive.HiveContext: Initializing execution hive, version 0.13.1
java.lang.RuntimeException: java.lang.NumberFormatException: For input string: "1s"at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346)at org.apache.spark.sql.hive.client.ClientWrapper.<init>(ClientWrapper.scala:105)at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:163)at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:161)at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:167)at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:526)at org.apache.spark.repl.SparkILoop.createSQLContext(SparkILoop.scala:1028)at $iwC$$iwC.<init>(<console>:9)at $iwC.<init>(<console>:18)at <init>(<console>:20)at .<init>(<console>:24)at .<clinit>(<console>)at .<init>(<console>:7)at .<clinit>(<console>)at $print(<console>)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:606)at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1338)at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:857)at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:902)at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:130)at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply(SparkILoopInit.scala:122)at org.apache.spark.repl.SparkIMain.beQuietDuring(SparkIMain.scala:324)at org.apache.spark.repl.SparkILoopInit$class.initializeSpark(SparkILoopInit.scala:122)at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:64)at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1$$anonfun$apply$mcZ$sp$5.apply$mcV$sp(SparkILoop.scala:974)at org.apache.spark.repl.SparkILoopInit$class.runThunks(SparkILoopInit.scala:157)at org.apache.spark.repl.SparkILoop.runThunks(SparkILoop.scala:64)at org.apache.spark.repl.SparkILoopInit$class.postInitialization(SparkILoopInit.scala:106)at org.apache.spark.repl.SparkILoop.postInitialization(SparkILoop.scala:64)at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:991)at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:945)at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:945)at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)at org.apache.spark.repl.Main$.main(Main.scala:31)at org.apache.spark.repl.Main.main(Main.scala)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:606)at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:664)at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:169)at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:192)at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:111)at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.NumberFormatException: For input string: "1s"at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)at java.lang.Integer.parseInt(Integer.java:492)at java.lang.Integer.parseInt(Integer.java:527)at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1134)at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:1211)at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:1220)at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:58)at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453)at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465)at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340)... 56 more15/09/05 14:50:58 INFO cluster.SparkDeploySchedulerBackend: Registered executor: AkkaRpcEndpointRef(Actor[akka.tcp://sparkExecutor@192.168.0.110:44198/user/Executor#934818115]) with ID 0
15/09/05 14:50:59 INFO storage.BlockManagerMasterEndpoint: Registering block manager 192.168.0.110:35246 with 267.3 MB RAM, BlockManagerId(0, 192.168.0.110, 35246)
<console>:10: error: not found: value sqlContextimport sqlContext.implicits._^
<console>:10: error: not found: value sqlContextimport sqlContext.sql^scala> exit

Spark1.4 和 Hive 1.1.1 启动错误相关推荐

Hive 1.1.1 启动错误
Hive 下载 [jifeng@feng03~]$ mkdir hive [jifeng@feng03~]$cd hive/ [jifeng@feng03 hive]$ wget http://ap ...
HIVE 1.1.0 启动时报错： Unsupported major.minor version 51.0
JDK 1.6 + HIVE 1.1.0 启动时抛异常: root@ubuntu:~# hive Logging initialized using configuration in jar:file ...
启动ipython_ipython,_ipython 启动错误，ipython - phpStudy
ipython 启动错误 [paulsweet123@localhost ~]$ ipython notebook [TerminalIPythonApp] WARNING | Subcommand ...
mysql启动错误1067进程意外终止的解决方法
mysql启动错误1067进程意外终止的解决方法参考文章: (1)mysql启动错误1067进程意外终止的解决方法 (2)https://www.cnblogs.com/huhyoung/p/960 ...
Jmeter启动错误及解决方案
最近使用Jmeter做压力测试时,遇到了几个启动错误,如下(mac环境) 第一个:Unrecognized VM option 'MaxMetaspaceSize=256m' 原先以为是Jmeter的 ...
终极解决方案：Emacs+Slime+Lisp启动错误：Polling /tmp/slime.50
2019独角兽企业重金招聘Python工程师标准>>> 终极解决方案:Emacs+Slime+Lisp启动错误:Polling "/tmp/slime.5000 .. 25 ...
hive遇到FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask错误...
hive遇到FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask错误起因 ...
MFC的exe启动时提示应用程序配置不正确，应用程序未能启动错误
MFC的exe启动时提示应用程序配置不正确,应用程序未能启动错误老姐笔记本中毒太深于病入膏肓,无奈迫写一程序"曲线救国".刚启动自己写的程序就出现应用程序配置不正确,应用程序未能 ...
Grub error17:Cannot mount selected partition启动错误的解决
下午在window下用pqmagic把移动硬盘重新分区后,重启选择进入ubuntu的时候出现启动错误: Grub Error17:Cannot mount selected partition.... ...

Spark1.4 和 Hive 1.1.1 启动错误

Spark1.4 和 Hive 1.1.1 启动错误相关推荐

最新文章

热门文章