解决output directory already exists

解决output directory already exists的问题

在eclipse里面运行Hadoop，报错文件已经存在

package myMapReduce;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.HashMap;
import java.util.Map;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.util.GenericOptionsParser;public class JoinMap {public static class MyMapper extends Mapper<Object, Text, Text, Text> {private Map<String, String> dict = new HashMap<>();@Overrideprotected void setup(Context context) throws IOException, InterruptedException {String fileName = context.getLocalCacheFiles()[0].getName();System.out.println(fileName);BufferedReader reader = new BufferedReader(new FileReader(fileName));String codeandname = null;while (null != (codeandname = reader.readLine())) {String str[] = codeandname.split(",");if (!Character.isDigit(str[0].charAt(0))) continue;dict.put(str[0], str[2] + "," + str[3]);}reader.close();}@Overrideprotected void map(Object key, Text value, Context context)throws IOException, InterruptedException {String[] kv = value.toString().split(",");if (dict.containsKey(kv[1])) context.write(new Text(kv[1]), new Text(dict.get(kv[1]) + "," + kv[2]));}}public static class MyReducer extends Reducer<Text, Text, Text, Text> {@Overrideprotected void reduce(Text key, Iterable<Text> values, Context context)throws IOException, InterruptedException {for (Text text : values) context.write(key, text);}}public static void main(String[] args) throws ClassNotFoundException,IOException, InterruptedException, URISyntaxException {Configuration joinmap = new Configuration(); Job job = Job.getInstance();job.setJobName("joinmap");job.setJarByClass(JoinMap.class);job.setMapperClass(MyMapper.class);job.setReducerClass(MyReducer.class);job.setOutputKeyClass(Text.class);job.setOutputValueClass(Text.class);// 判断output文件夹是否存在，如果存在则删除Path in = new Path("hdfs://aaaa:9000/order_items");Path out = new Path("hdfs://aaaa:9000/out/JoinMap");FileSystem fileSystem = out.getFileSystem(joinmap);// 根据path找到这个文件if (fileSystem.exists(out)) {fileSystem.delete(out, true);// true的意思是，就算output有东西，也一带删除}FileInputFormat.addInputPath(job, in);FileOutputFormat.setOutputPath(job, out);URI uri = new URI("hdfs://aaaa:9000/orders");job.addCacheFile(uri);System.out.println("Start");System.exit(job.waitForCompletion(true) ? 0 : 1);}
}

其中起作用的是这几句：

 // 判断output文件夹是否存在，如果存在则删除Path in = new Path("hdfs://aaaa:9000/order_items");Path out = new Path("hdfs://aaaa:9000/out/JoinMap");FileSystem fileSystem = out.getFileSystem(joinmap);// 根据path找到这个文件if (fileSystem.exists(out)) {fileSystem.delete(out, true);// true的意思是，就算output有东西，也一带删除}

改完之后运行，还是会报错：
failed to create symlink…
这时候，关闭eclipse，重新以管理员身份运行即可。

修改前：找不到文件

修改后：

参考文章：
改代码
连接错误

解决output directory already exists相关推荐

idea 报错Output directory is not specified错误
在启动tomcat的时候会出现一个***报错Output directory is not specified***这个错误 Output directory is not specified该报错的 ...
Cannot create resource output directory
现象 IDEA执行install报错:Cannot create resource output directory,然后点击target目录报错无权限,拒绝访问.如图解决方式关闭向日葵
useradd: warning: the home directory already exists. Not copying any file from skel directory intoit
1.问题描述笔者想要创建一个用户,并把该该用户的家目录指定到一个目录下,使用 sudo useradd -d /data/tyb -m tyb 提示, useradd: warning: the h ...
通过迁移的方式解决Active Directory服务器问题之6
(接上一篇) (21)打开"欢迎使用证书颁发机构还原向导"对话框,如图6-21所示. 图6-21 还原向导 (22)在"要还原的项目"对话框,选择要还原的项目, ...
warning MSB8004: Output Directory does not end with a trailing slash.
当在VC里编译时,发现这个警告,就是说设置的目录参数不是以反斜杠结束的目录名称,如下: 1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V ...
maven打包项目时报错：Cannot create resource output directory
前言:如今使用开发工具打包java项目是一件常用且方便的事情,但是当我们使用maven打包项目出错时,就会十分让人郁闷,今天我就遇到了这件事,大家来看看报错原因:打包项目根目录下的target文件夹 ...
Output directory not specified for Module
Error:Kotlin:Output directory not specified for Module 'XXX' production 第一步点击左上角File打开Project Struc ...
ubuntu下面解决mysqld_safe Directory '/var/run/mysqld' for UNIX socket file don't exists
事情是这样的,原先创建MySQL中的root用户,指定的host为localhost,就想改一下为某个网段的,结果改完之后就退出了,然后就悲剧了,无法连接了.网上找的大部分解决办法是: ubuntu下 ...
【clickhouse】未解决 ClickHouse exception, code: 1002 DB::Exception: Directory already exists
1.概述 clickhouse 报错如下 SQL 错误 [1002]: ClickHouse exception, code: 1002, host: localhost, port: 8123; C ...
运行VINS-mono：/home/tony-ws1/output/pose_graph/ not exists, trying to create it /home/tony-ws1/ou错误解决
问题:运行VINS-mono时出现错误解决方法:在home下创建这些文件(sudo 命令行方式)

解决output directory already exists

解决output directory already exists的问题

解决output directory already exists相关推荐

最新文章

热门文章