Spark 读 CSV格式文件，报错UnicodeDecodeError: 'utf8' codec can't decode byte 0xca in position 17: invalid c

今天在Windows10系统下，跑Spark Python脚本，执行collect()时报下面的错误

births.select(s[0]) \
            .distinct() \
            .rdd \
            .map(lambda row: row[0]) \
            .collect()

尝试把csv文件编码改为utf-8也没有用，代码中加入编码格式也没有效果。

import sys
reload(sys)
sys.setdefaultencoding('ISO-8859-1') #改成utf8也没有用。

原来我犯了一个错误，python脚本的名称包含中文导致的。(⊙ ︿ ⊙)

Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Traceback (most recent call last):
File "36-38(数据预处理_逻辑回归).py", line 83, in <module>
    print births.rdd.collect()
File "e:\anaconda2\lib\site-packages\pyspark\rdd.py", line 815, in collect
    with SCCallSiteSync(self.context) as css:
File "e:\anaconda2\lib\site-packages\pyspark\traceback_utils.py", line 72, in __enter__
    self._context._jsc.setCallSite(self._call_site)
File "e:\anaconda2\lib\site-packages\py4j\java_gateway.py", line 1277, in __call__
    args_command, temp_args = self._build_args(*args)
File "e:\anaconda2\lib\site-packages\py4j\java_gateway.py", line 1247, in _build_args
    [get_command_part(arg, self.pool) for arg in new_args])
File "e:\anaconda2\lib\site-packages\py4j\protocol.py", line 292, in get_command_part
    command_part = STRING_TYPE + escape_new_line(parameter)
File "e:\anaconda2\lib\site-packages\py4j\protocol.py", line 187, in escape_new_line
    return smart_decode(original).replace("\\", "\\\\").\
File "e:\anaconda2\lib\site-packages\py4j\protocol.py", line 219, in smart_decode
    return unicode(s, "utf-8")
UnicodeDecodeError: 'utf8' codec can't decode byte 0xca in position 17: invalid continuation byte

Spark 读 CSV格式文件，报错UnicodeDecodeError: 'utf8' codec can't decode byte 0xca in position 17: invalid c相关推荐

pandas 读csv 报错 UnicodeDecodeError: 'utf-8' codec can't decode byte 0xca in position 0: invalid cont
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xca in position 0: invalid continuation byte 解码 ...
pandas读文件UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8e in position 17: invalid start byte
@ 解决pandas读文件报错 UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8e in position 17: invalid sta ...
pandas读取数据时，报错UnicodeDecodeError: 'utf-8' codec can't decode byte 0xce in position 0: invalid contin
1.我的代码: df_train=pd.read_csv("C:\\Users\\15520\\Desktop\\AI\\阿里天池\\幸福感挖掘\\happiness_train_compl ...
【pandas问题】UnicodeDecodeError: ‘utf-8‘ codec can‘t decode byte 0xca in position 0: invalid continuati
出错位置: data = pd.read_csv('../data/user.csv') print(data.head()) 报错信息: UnicodeDecodeError: 'utf-8' co ...
解决UnicodeDecodeError: ‘utf-8‘ codec can‘t decode byte 0xca in position 0: invalid continuation byte
英文字母,数字,符号都用特定的二进制编码方式计算机才能识别."UnicodeDecodeError: 'utf-8' codec can't decode byte 0xca in posi ...
UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0xca in position 24: invalid continuation byte
Flask报错:UnicodeDecodeError: 'utf-8' codec can't decode byte 0xca in position 24: invalid continuatio ...
python 报错 UnicodeDecodeError: ‘utf-8‘ codec can‘t decode byte 0xd3 in position 解决方法
在使用urllib获取reqest的response的时候,还要进行解码 content = urllib.request.urlopen(request).read().decode() 当执行该语 ...
(20200921Solved)UnicodeDecodeError: ‘utf-8‘ codec can‘t decode byte 0xca in position 0: invalid cont
问题描述通过pandas.read_csv读取.csv.gz文件出现上述问题. 解决方案 encoding='ISO-8859-1' 但是这种编码会导致中文乱码,改用GB18030,顺利解决. Re ...
python读取字符报错：‘utf-8‘ codec can‘t decode byte 0xbd in position 2: invalid start byte
python读取字符报错:'utf-8' codec can't decode byte 0xbd in position 2: invalid start byte 原因解决方法原因 utf-8 ...

Spark 读 CSV格式文件，报错UnicodeDecodeError: 'utf8' codec can't decode byte 0xca in position 17: invalid c

Spark 读 CSV格式文件，报错UnicodeDecodeError: 'utf8' codec can't decode byte 0xca in position 17: invalid c相关推荐

最新文章

热门文章

Spark 读 CSV格式文件 ，报错UnicodeDecodeError: 'utf8' codec can't decode byte 0xca in position 17: invalid c

Spark 读 CSV格式文件 ，报错UnicodeDecodeError: 'utf8' codec can't decode byte 0xca in position 17: invalid c相关推荐

最新文章

热门文章

Spark 读 CSV格式文件，报错UnicodeDecodeError: 'utf8' codec can't decode byte 0xca in position 17: invalid c

Spark 读 CSV格式文件，报错UnicodeDecodeError: 'utf8' codec can't decode byte 0xca in position 17: invalid c相关推荐