None.org.apache.spark.api.python.PythonAccumulatorV2

完整报错如下：

2019-05-21 15:19:00 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Traceback (most recent call last):File "WordCount.py", line 48, in <module>sc=CreateSparkContext()File "WordCount.py", line 37, in CreateSparkContextspark = SparkSession.builder.appName("Intro").config("spark.master", "local").getOrCreate()File "/home/appleyuchi/.virtualenvs/python2.7/local/lib/python2.7/site-packages/pyspark/sql/session.py", line 173, in getOrCreatesc = SparkContext.getOrCreate(sparkConf)File "/home/appleyuchi/.virtualenvs/python2.7/local/lib/python2.7/site-packages/pyspark/context.py", line 367, in getOrCreateSparkContext(conf=conf or SparkConf())File "/home/appleyuchi/.virtualenvs/python2.7/local/lib/python2.7/site-packages/pyspark/context.py", line 136, in __init__conf, jsc, profiler_cls)File "/home/appleyuchi/.virtualenvs/python2.7/local/lib/python2.7/site-packages/pyspark/context.py", line 207, in _do_initself._javaAccumulator = self._jvm.PythonAccumulatorV2(host, port, auth_token)File "/home/appleyuchi/.virtualenvs/python2.7/local/lib/python2.7/site-packages/py4j/java_gateway.py", line 1525, in __call__answer, self._gateway_client, None, self._fqn)File "/home/appleyuchi/.virtualenvs/python2.7/local/lib/python2.7/site-packages/py4j/protocol.py", line 332, in get_return_valueformat(target_id, ".", name, value))
py4j.protocol.Py4JError: An error occurred while calling None.org.apache.spark.api.python.PythonAccumulatorV2. Trace:
py4j.Py4JException: Constructor org.apache.spark.api.python.PythonAccumulatorV2([class java.lang.String, class java.lang.Integer, class java.lang.String]) does not existat py4j.reflection.ReflectionEngine.getConstructor(ReflectionEngine.java:179)at py4j.reflection.ReflectionEngine.getConstructor(ReflectionEngine.java:196)at py4j.Gateway.invoke(Gateway.java:237)at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80)at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69)at py4j.GatewayConnection.run(GatewayConnection.java:238)at java.lang.Thread.run(Thread.java:748)

解决方案如下:
.py文件开头加入：

# -*- coding: UTF-8 -*-
import findspark
findspark.init()

None.org.apache.spark.api.python.PythonAccumulatorV2相关推荐

Py4JError: org.apache.spark.api.python.PythonUtils.getPythonAuthSocketTimeout does not exist in the
背景就是版本没统一 spark和pyspark的版本需要统一我的spark是3.0.3的然后使用的是anaconda内的python,安装pyspark的时候没指定版本,默认安装最新的3.2 导 ...
An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext.
最近公司上云,所有大数据环境都迁移到云上.在进行项目迁移时候发现运行报错 py4j.protocol.Py4JJavaError: An error occurred while calling No ...
spark提交python程序_pyspark：带有spark提交的jars依赖
我编写了一个pyspark脚本,该脚本读取两个json文件,coGroup然后将它们发送到elasticsearch集群.当我在本地运行该elasticsearch- hadoop文件时,一切都会正常 ...
分享一个.NET平台开源免费跨平台的大数据分析框架.NET for Apache Spark
今天早上六点半左右微信群里就看到张队发的关于.NET Spark大数据的链接https://devblogs.microsoft.com/dotnet/introducing-net-for-apac ...
Apache Spark 3.0 DStreams-Streaming编程指南
目录总览一个简单的例子基本概念连结中初始化StreamingContext 离散流(DStreams) 输入DStreams和接收器基本资料进阶资源自订来源接收器可靠性 DStrea ...
Apache Spark 3.0 结构化Streaming流编程指南
目录总览快速范例 Scala语言 Java语言 Python语言 R语言程式设计模型基本概念处理事件时间和延迟数据容错语义使用数据集和数据帧的API 创建流数据框架和流数据集流数据帧/ ...
Apache Spark 3.0 SQL DataFrame和DataSet指南
目录简介 SQL 数据集和数据框入门起点:SparkSession Scala语言 Java语言 Python语言 R语言创建DataFrame Scala语言 Java语言 Python语言 ...
Apache Spark【从无到有从有到无】【编程指南】【AS5】结构化流编程指南
目录 1.概观 2.快速示例 3.编程模型 3.1.基本概念 3.2.处理事件时间和延迟数据 3.3.容错语义 4.使用数据集和数据框架的API 4.1.创建streaming DataFrames ...
ERROR SparkContext: Error initializing SparkContext. org.apache.spark.SparkException: Could not pars
ERROR SparkContext: Error initializing SparkContext. org.apache.spark.SparkException: Could not pars ...

None.org.apache.spark.api.python.PythonAccumulatorV2

None.org.apache.spark.api.python.PythonAccumulatorV2相关推荐

最新文章

热门文章