案例：手机流量的统计

对于记录用户手机信息的文件，得出统计每一个用户（手机号）所耗费的总上行流量、下行流量，总流量结果。

分析

1. 实现自定义的 bean 来封装流量信息，使用手机号码作为Key,Bean作为value。这个Bean的传输需要实现可序    列化,因此我们需要实现MapReduce的序列化接口Writable,重写相关方法。
2. 计算上行流量、下行流量、计费流量
3. <k1,v1>的分析：取一行TextInputFormat类去读取，offset做key,一行数据做value，offset:phoneNum,upflow,downflow
4. 拆分，取出倒数第二倒数第三段，map端读取数据然后输出

打开IDEA新建maven工程

  • 导入依赖
<dependencies><dependency><groupId>org.apache.hadoop</groupId><artifactId>hadoop-common</artifactId><version>2.7.6</version></dependency><!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client --><dependency><groupId>org.apache.hadoop</groupId><artifactId>hadoop-client</artifactId><version>2.7.6</version></dependency><!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-hdfs --><dependency><groupId>org.apache.hadoop</groupId><artifactId>hadoop-hdfs</artifactId><version>2.7.6</version></dependency>
</dependencies>
  • 将数据放在当前项目下resources文件夹里

文件名为：HTTP_20130313143750.dat

1363157985066    13726230503 00-FD-07-A4-72-B8:CMCC  120.196.100.82  i02.c.aliimg.com        24  27  2481    24681   200
1363157995052   13826544101 5C-0E-8B-C7-F1-E0:CMCC  120.197.40.4            4   0   264 0   200
1363157991076   13926435656 20-10-7A-28-CC-0A:CMCC  120.196.100.99          2   4   132 1512    200
1363154400022   13926251106 5C-0E-8B-8B-B1-50:CMCC  120.197.40.4            4   0   240 0   200
1363157993044   18211575961 94-71-AC-CD-E6-18:CMCC-EASY 120.196.100.99  iface.qiyi.com  视频网站    15  12  1527    2106    200
1363157995074   84138413    5C-0E-8B-8C-E8-20:7DaysInn  120.197.40.4    122.72.52.12        20  16  4116    1432    200
1363157993055   13560439658 C4-17-FE-BA-DE-D9:CMCC  120.196.100.99          18  15  1116    954 200
1363157995033   15920133257 5C-0E-8B-C7-BA-20:CMCC  120.197.40.4    sug.so.360.cn   信息安全    20  20  3156    2936    200
1363157983019   13719199419 68-A1-B7-03-07-B1:CMCC-EASY 120.196.100.82          4   0   240 0   200
1363157984041   13660577991 5C-0E-8B-92-5C-20:CMCC-EASY 120.197.40.4    s19.cnzz.com    站点统计    24  9   6960    690 200
1363157973098   15013685858 5C-0E-8B-C7-F7-90:CMCC  120.197.40.4    rank.ie.sogou.com   搜索引擎    28  27  3659    3538    200
1363157986029   15989002119 E8-99-C4-4E-93-E0:CMCC-EASY 120.196.100.99  www.umeng.com   站点统计    3   3   1938    180 200
1363157992093   13560439658 C4-17-FE-BA-DE-D9:CMCC  120.196.100.99          15  9   918 4938    200
1363157986041   13480253104 5C-0E-8B-C7-FC-80:CMCC-EASY 120.197.40.4            3   3   180 180 200
1363157984040   13602846565 5C-0E-8B-8B-B6-00:CMCC  120.197.40.4    2052.flash2-http.qq.com 综合门户    15  12  1938    2910    200
1363157995093   13922314466 00-FD-07-A2-EC-BA:CMCC  120.196.100.82  img.qfc.cn      12  12  3008    3720    200
1363157982040   13502468823 5C-0A-5B-6A-0B-D4:CMCC-EASY 120.196.100.99  y0.ifengimg.com 综合门户    57  102 7335    110349  200
1363157986072   18320173382 84-25-DB-4F-10-1A:CMCC-EASY 120.196.100.99  input.shouji.sogou.com  搜索引擎    21  18  9531    2412    200
1363157990043   13925057413 00-1F-64-E1-E6-9A:CMCC  120.196.100.55  t3.baidu.com    搜索引擎    69  63  11058   48243   200
1363157988072   13760778710 00-FD-07-A4-7B-08:CMCC  120.196.100.82          2   2   120 120 200
1363157985066   13726238888 00-FD-07-A4-72-B8:CMCC  120.196.100.82  i02.c.aliimg.com        24  27  2481    24681   200
1363157993055   13560436666 C4-17-FE-BA-DE-D9:CMCC  120.196.100.99          18  15  1116    954 200
  • 导入log4j日志配置文件到resources
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.# Define some default values that can be overridden by system properties
hadoop.root.logger=INFO,console
hadoop.log.dir=.
hadoop.log.file=hadoop.log# Define the root logger to the system property "hadoop.root.logger".
log4j.rootLogger=${hadoop.root.logger}, EventCounter# Logging Threshold
log4j.threshold=ALL# Null Appender
log4j.appender.NullAppender=org.apache.log4j.varia.NullAppender#
# Rolling File Appender - cap space usage at 5gb.
#
hadoop.log.maxfilesize=256MB
hadoop.log.maxbackupindex=20
log4j.appender.RFA=org.apache.log4j.RollingFileAppender
log4j.appender.RFA.File=${hadoop.log.dir}/${hadoop.log.file}log4j.appender.RFA.MaxFileSize=${hadoop.log.maxfilesize}
log4j.appender.RFA.MaxBackupIndex=${hadoop.log.maxbackupindex}log4j.appender.RFA.layout=org.apache.log4j.PatternLayout# Pattern format: Date LogLevel LoggerName LogMessage
log4j.appender.RFA.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n
# Debugging Pattern format
#log4j.appender.RFA.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} (%F:%M(%L)) - %m%n#
# Daily Rolling File Appender
#log4j.appender.DRFA=org.apache.log4j.DailyRollingFileAppender
log4j.appender.DRFA.File=${hadoop.log.dir}/${hadoop.log.file}# Rollover at midnight
log4j.appender.DRFA.DatePattern=.yyyy-MM-ddlog4j.appender.DRFA.layout=org.apache.log4j.PatternLayout# Pattern format: Date LogLevel LoggerName LogMessage
log4j.appender.DRFA.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n
# Debugging Pattern format
#log4j.appender.DRFA.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} (%F:%M(%L)) - %m%n#
# console
# Add "console" to rootlogger above if you want to use this
#log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{2}: %m%n#
# TaskLog Appender
##Default values
hadoop.tasklog.taskid=null
hadoop.tasklog.iscleanup=false
hadoop.tasklog.noKeepSplits=4
hadoop.tasklog.totalLogFileSize=100
hadoop.tasklog.purgeLogSplits=true
hadoop.tasklog.logsRetainHours=12log4j.appender.TLA=org.apache.hadoop.mapred.TaskLogAppender
log4j.appender.TLA.taskId=${hadoop.tasklog.taskid}
log4j.appender.TLA.isCleanup=${hadoop.tasklog.iscleanup}
log4j.appender.TLA.totalLogFileSize=${hadoop.tasklog.totalLogFileSize}log4j.appender.TLA.layout=org.apache.log4j.PatternLayout
log4j.appender.TLA.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n#
# HDFS block state change log from block manager
#
# Uncomment the following to suppress normal block state change
# messages from BlockManager in NameNode.
#log4j.logger.BlockStateChange=WARN#
#Security appender
#
hadoop.security.logger=INFO,NullAppender
hadoop.security.log.maxfilesize=256MB
hadoop.security.log.maxbackupindex=20
log4j.category.SecurityLogger=${hadoop.security.logger}
hadoop.security.log.file=SecurityAuth-${user.name}.audit
log4j.appender.RFAS=org.apache.log4j.RollingFileAppender
log4j.appender.RFAS.File=${hadoop.log.dir}/${hadoop.security.log.file}
log4j.appender.RFAS.layout=org.apache.log4j.PatternLayout
log4j.appender.RFAS.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n
log4j.appender.RFAS.MaxFileSize=${hadoop.security.log.maxfilesize}
log4j.appender.RFAS.MaxBackupIndex=${hadoop.security.log.maxbackupindex}#
# Daily Rolling Security appender
#
log4j.appender.DRFAS=org.apache.log4j.DailyRollingFileAppender
log4j.appender.DRFAS.File=${hadoop.log.dir}/${hadoop.security.log.file}
log4j.appender.DRFAS.layout=org.apache.log4j.PatternLayout
log4j.appender.DRFAS.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n
log4j.appender.DRFAS.DatePattern=.yyyy-MM-dd#
# hadoop configuration logging
## Uncomment the following line to turn off configuration deprecation warnings.
# log4j.logger.org.apache.hadoop.conf.Configuration.deprecation=WARN#
# hdfs audit logging
#
hdfs.audit.logger=INFO,NullAppender
hdfs.audit.log.maxfilesize=256MB
hdfs.audit.log.maxbackupindex=20
log4j.logger.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=${hdfs.audit.logger}
log4j.additivity.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=false
log4j.appender.RFAAUDIT=org.apache.log4j.RollingFileAppender
log4j.appender.RFAAUDIT.File=${hadoop.log.dir}/hdfs-audit.log
log4j.appender.RFAAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.RFAAUDIT.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
log4j.appender.RFAAUDIT.MaxFileSize=${hdfs.audit.log.maxfilesize}
log4j.appender.RFAAUDIT.MaxBackupIndex=${hdfs.audit.log.maxbackupindex}#
# mapred audit logging
#
mapred.audit.logger=INFO,NullAppender
mapred.audit.log.maxfilesize=256MB
mapred.audit.log.maxbackupindex=20
log4j.logger.org.apache.hadoop.mapred.AuditLogger=${mapred.audit.logger}
log4j.additivity.org.apache.hadoop.mapred.AuditLogger=false
log4j.appender.MRAUDIT=org.apache.log4j.RollingFileAppender
log4j.appender.MRAUDIT.File=${hadoop.log.dir}/mapred-audit.log
log4j.appender.MRAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.MRAUDIT.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
log4j.appender.MRAUDIT.MaxFileSize=${mapred.audit.log.maxfilesize}
log4j.appender.MRAUDIT.MaxBackupIndex=${mapred.audit.log.maxbackupindex}# Custom Logging levels#log4j.logger.org.apache.hadoop.mapred.JobTracker=DEBUG
#log4j.logger.org.apache.hadoop.mapred.TaskTracker=DEBUG
#log4j.logger.org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit=DEBUG# Jets3t library
log4j.logger.org.jets3t.service.impl.rest.httpclient.RestS3Service=ERROR# AWS SDK & S3A FileSystem
log4j.logger.com.amazonaws=ERROR
log4j.logger.com.amazonaws.http.AmazonHttpClient=ERROR
log4j.logger.org.apache.hadoop.fs.s3a.S3AFileSystem=WARN#
# Event Counter Appender
# Sends counts of logging messages at different severity levels to Hadoop Metrics.
#
log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter#
# Job Summary Appender
#
# Use following logger to send summary to separate file defined by
# hadoop.mapreduce.jobsummary.log.file :
# hadoop.mapreduce.jobsummary.logger=INFO,JSA
#
hadoop.mapreduce.jobsummary.logger=${hadoop.root.logger}
hadoop.mapreduce.jobsummary.log.file=hadoop-mapreduce.jobsummary.log
hadoop.mapreduce.jobsummary.log.maxfilesize=256MB
hadoop.mapreduce.jobsummary.log.maxbackupindex=20
log4j.appender.JSA=org.apache.log4j.RollingFileAppender
log4j.appender.JSA.File=${hadoop.log.dir}/${hadoop.mapreduce.jobsummary.log.file}
log4j.appender.JSA.MaxFileSize=${hadoop.mapreduce.jobsummary.log.maxfilesize}
log4j.appender.JSA.MaxBackupIndex=${hadoop.mapreduce.jobsummary.log.maxbackupindex}
log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
log4j.appender.JSA.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{2}: %m%n
log4j.logger.org.apache.hadoop.mapred.JobInProgress$JobSummary=${hadoop.mapreduce.jobsummary.logger}
log4j.additivity.org.apache.hadoop.mapred.JobInProgress$JobSummary=false#
# Yarn ResourceManager Application Summary Log
#
# Set the ResourceManager summary log filename
yarn.server.resourcemanager.appsummary.log.file=rm-appsummary.log
# Set the ResourceManager summary log level and appender
yarn.server.resourcemanager.appsummary.logger=${hadoop.root.logger}
#yarn.server.resourcemanager.appsummary.logger=INFO,RMSUMMARY# To enable AppSummaryLogging for the RM,
# set yarn.server.resourcemanager.appsummary.logger to
# <LEVEL>,RMSUMMARY in hadoop-env.sh# Appender for ResourceManager Application Summary Log
# Requires the following properties to be set
#    - hadoop.log.dir (Hadoop Log directory)
#    - yarn.server.resourcemanager.appsummary.log.file (resource manager app summary log filename)
#    - yarn.server.resourcemanager.appsummary.logger (resource manager app summary log level and appender)log4j.logger.org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary=${yarn.server.resourcemanager.appsummary.logger}
log4j.additivity.org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary=false
log4j.appender.RMSUMMARY=org.apache.log4j.RollingFileAppender
log4j.appender.RMSUMMARY.File=${hadoop.log.dir}/${yarn.server.resourcemanager.appsummary.log.file}
log4j.appender.RMSUMMARY.MaxFileSize=256MB
log4j.appender.RMSUMMARY.MaxBackupIndex=20
log4j.appender.RMSUMMARY.layout=org.apache.log4j.PatternLayout
log4j.appender.RMSUMMARY.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n# HS audit log configs
#mapreduce.hs.audit.logger=INFO,HSAUDIT
#log4j.logger.org.apache.hadoop.mapreduce.v2.hs.HSAuditLogger=${mapreduce.hs.audit.logger}
#log4j.additivity.org.apache.hadoop.mapreduce.v2.hs.HSAuditLogger=false
#log4j.appender.HSAUDIT=org.apache.log4j.DailyRollingFileAppender
#log4j.appender.HSAUDIT.File=${hadoop.log.dir}/hs-audit.log
#log4j.appender.HSAUDIT.layout=org.apache.log4j.PatternLayout
#log4j.appender.HSAUDIT.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
#log4j.appender.HSAUDIT.DatePattern=.yyyy-MM-dd# Http Server Request Logs
#log4j.logger.http.requests.namenode=INFO,namenoderequestlog
#log4j.appender.namenoderequestlog=org.apache.hadoop.http.HttpRequestLogAppender
#log4j.appender.namenoderequestlog.Filename=${hadoop.log.dir}/jetty-namenode-yyyy_mm_dd.log
#log4j.appender.namenoderequestlog.RetainDays=3#log4j.logger.http.requests.datanode=INFO,datanoderequestlog
#log4j.appender.datanoderequestlog=org.apache.hadoop.http.HttpRequestLogAppender
#log4j.appender.datanoderequestlog.Filename=${hadoop.log.dir}/jetty-datanode-yyyy_mm_dd.log
#log4j.appender.datanoderequestlog.RetainDays=3#log4j.logger.http.requests.resourcemanager=INFO,resourcemanagerrequestlog
#log4j.appender.resourcemanagerrequestlog=org.apache.hadoop.http.HttpRequestLogAppender
#log4j.appender.resourcemanagerrequestlog.Filename=${hadoop.log.dir}/jetty-resourcemanager-yyyy_mm_dd.log
#log4j.appender.resourcemanagerrequestlog.RetainDays=3#log4j.logger.http.requests.jobhistory=INFO,jobhistoryrequestlog
#log4j.appender.jobhistoryrequestlog=org.apache.hadoop.http.HttpRequestLogAppender
#log4j.appender.jobhistoryrequestlog.Filename=${hadoop.log.dir}/jetty-jobhistory-yyyy_mm_dd.log
#log4j.appender.jobhistoryrequestlog.RetainDays=3#log4j.logger.http.requests.nodemanager=INFO,nodemanagerrequestlog
#log4j.appender.nodemanagerrequestlog=org.apache.hadoop.http.HttpRequestLogAppender
#log4j.appender.nodemanagerrequestlog.Filename=${hadoop.log.dir}/jetty-nodemanager-yyyy_mm_dd.log
#log4j.appender.nodemanagerrequestlog.RetainDays=3# WebHdfs request log on datanodes
# Specify -Ddatanode.webhdfs.logger=INFO,HTTPDRFA on datanode startup to
# direct the log to a separate file.
#datanode.webhdfs.logger=INFO,console
#log4j.logger.datanode.webhdfs=${datanode.webhdfs.logger}
#log4j.appender.HTTPDRFA=org.apache.log4j.DailyRollingFileAppender
#log4j.appender.HTTPDRFA.File=${hadoop.log.dir}/hadoop-datanode-webhdfs.log
#log4j.appender.HTTPDRFA.layout=org.apache.log4j.PatternLayout
#log4j.appender.HTTPDRFA.layout.ConversionPattern=%d{ISO8601} %m%n
#log4j.appender.HTTPDRFA.DatePattern=.yyyy-MM-dd
  • 编写FLowBean类型
package mr.phoneflow;import org.apache.hadoop.io.Writable;import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;public class FLowBean implements Writable {private long upflow;private long downflow;private long sumflow;//如果空参构造函数被覆盖，一定要显示定义一下，否则在反序列化时会抛出异常public FLowBean() {}public FLowBean(long upflow, long downflow) {this.upflow = upflow;this.downflow = downflow;this.sumflow = upflow + downflow;}public long getUpflow() {return upflow;}public void setUpflow(long upflow) {this.upflow = upflow;}public long getDownflow() {return downflow;}public void setDownflow(long downflow) {this.downflow = downflow;}public long getSumflow() {return sumflow;}public void setSumflow(long sumflow) {this.sumflow = sumflow;}@Overridepublic String toString() {return upflow + "\t" + downflow + "\t" + sumflow;}//序列化，将对象的字段信息写入输出流@Overridepublic void write(DataOutput dataOutput) throws IOException {dataOutput.writeLong(upflow);dataOutput.writeLong(downflow);dataOutput.writeLong(sumflow);}//反序列化，从输入流读取各字段的信息@Overridepublic void readFields(DataInput dataInput) throws IOException {upflow = dataInput.readLong();downflow = dataInput.readLong();sumflow = dataInput.readLong();}
}
  • 编写FlowMapper类型
package mr.phoneflow;import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;import java.io.IOException;public class FlowMapper extends Mapper<LongWritable, Text, Text, FLowBean> {@Overrideprotected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {String line = value.toString();String[] fields = line.split("\t");String phoneNum = fields[1];long upFlow = Long.parseLong(fields[fields.length - 3]);long downFlow = Long.parseLong(fields[fields.length - 2]);FLowBean bean = new FLowBean(upFlow, downFlow);context.write(new Text(phoneNum), bean);}
}
  • 编写FlowReduce类型
package mr.phoneflow;import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Reducer;import java.io.IOException;
import java.util.Iterator;public class FlowReduce extends Reducer<Text, FLowBean, Text, FLowBean> {@Overrideprotected void reduce(Text key, Iterable<FLowBean> values, Context context) throws IOException, InterruptedException {Iterator<FLowBean> iterator = values.iterator();long upFlow = 0;long downFlow = 0;while (iterator.hasNext()) {FLowBean bean = iterator.next();upFlow += bean.getUpflow();downFlow += bean.getDownflow();}FLowBean total = new FLowBean(upFlow, downFlow);context.write(key, total);}
}
  • 编写FlowDriver类型
package mr.phoneflow;import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;public class FlowDriver {public static void main(String[] args) throws Exception {//1、配置连接hadoop集群的参数Configuration conf = new Configuration();conf.set("fs.defaultFS", "file:///");conf.set("mapreduce.framework.name", "local");//2、获取job对象实例Job job =Job.getInstance(conf,"FLOWCOUNT");//3、驱动job.setJarByClass(FlowDriver.class);//4、设置mapper和reducer类型job.setMapperClass(FlowMapper.class);job.setReducerClass(FlowReduce.class);//5、设置k2,v2,k3,v3的泛型job.setMapOutputKeyClass(Text.class);job.setMapOutputValueClass(FLowBean.class);job.setOutputKeyClass(Text.class);job.setOutputValueClass(FLowBean.class);//6、设置reduceTask的个数，默认值是1job.setNumReduceTasks(2);//7、设置job要处理的数据的输入源FileInputFormat.setInputPaths(job,new Path("D:\\IDEA\\MapReducdDemo1\\src\\main\\resources\\HTTP_20130313143750.dat"));Path outPath = new Path("D:/output");FileSystem fs = FileSystem.get(conf);//判断输出目录是否存在，如果存在，则删除之if(fs.exists(outPath)){fs.delete(outPath,true);}FileOutputFormat.setOutputPath(job,outPath);//8、提交jobSystem.exit(job.waitForCompletion(true)?0:1);}
}
  • 运行成功后找到D:/output目录下的文件用Notepad++打开
part-r-00000文件13480253104    180 180 360
13760778710 120 120 240
13826544101 264 0   264
13922314466 3008    3720    6728
15989002119 1938    180 2118part-r-00001文件13502468823   7335    110349  117684
13560436666 1116    954 2070
13560439658 2034    5892    7926
13602846565 1938    2910    4848
13660577991 6960    690 7650
13719199419 240 0   240
13726230503 2481    24681   27162
13726238888 2481    24681   27162
13925057413 11058   48243   59301
13926251106 240 0   240
13926435656 132 1512    1644
15013685858 3659    3538    7197
15920133257 3156    2936    6092
18211575961 1527    2106    3633
18320173382 9531    2412    11943
84138413    4116    1432    5548

MapReduce案例：手机流量的统计相关推荐

  1. MapReduce--实现手机流量分析

    实现手机流量分析 1.需求 2.分析:逻辑:设计 (1)==需求一== step1:`不论是写SQL还是写MR,一般都先把结果的格式先列出来` step2:`有没有分组或者排序:决定Map输出的Key ...

  2. 一脸懵逼学习Hadoop中的序列化机制——流量求和统计MapReduce的程序开发案例——流量求和统计排序...

    一:序列化概念 序列化(Serialization)是指把结构化对象转化为字节流. 反序列化(Deserialization)是序列化的逆过程.即把字节流转回结构化对象. Java序列化(java.i ...

  3. 15.大数据---Mapreduce案例之---统计手机号耗费的总上行流量、下行流量、总流量

    Mapreduce案例之-统计手机号耗费的总上行流量.下行流量.总流量 1.需求: 统计每一个手机号耗费的总上行流量.下行流量.总流量 2.数据准备: 2.1 输入数据格式: 时间戳.电话号码.基站的 ...

  4. 实现用户手机流量统计（ReduceTask并行度控制）

    需求:1.实现用户手机流量统计(ReduceTask并行度控制) 数据如下:保存为.dat文件(因为以\t切分数据,文件格式必须合适) 13726230503 00-FD-07-A4-72-B8:CM ...

  5. android统计app流量的软件,流量控(手机流量统计)app

    流量控(手机流量统计)app,为你带来准确,及时的流量通知,帮助用户时刻查询自己的手机流量和使用状况,管理程序的流量使用,全方面保证您的流量使用 流量控(手机流量统计)app介绍 流量控是一款统计安卓 ...

  6. 手机上网流量统计_数据统计 | 上半年手机流量同比增110.2%，你贡献了多少？

    来源:工信部网站.中新经纬 版权申明:内容来源网络,版权归原创者所有.除非无法确认,我们都会标明作者及出处,如有侵权烦请告知我们,我们会立即删除并表示歉意.谢谢! 7月25日,工信部网站公布了2019 ...

  7. android统计流量,Android 获取手机整体流量使用情况以及某个应用的流量的统计

    很多安全卫士类软件都实现了网速监测功能,也算是一个比较实用的功能.Android下,TrafficStats类实现了对流量的统计. /proc/uid_stat/uid/tcp_send        ...

  8. Hadoop快速入门——第三章、MapReduce案例(字符统计)

    Hadoop快速入门--第三章.MapReduce案例 目录 环境要求: 1.项目创建: 2.修改Maven 3.编码 4.本地文件测试 5.修改[Action]文件(修改测试文件路径) 6.导出ja ...

  9. 手机上网流量统计_手机流量上网课花掉800元话费？“助学流量包”来了！湖南送流量达2880万G...

    图片来源:视觉中国 " 请问是彭仁哥家吗?"3 月 9 日,怀化分公司城东分局杨村支局局长向纲跃和网格经理梁平云,来到河西街道方石坪村彭仁哥家中,进行宽带网络安装. 近日,记者从省 ...

最新文章

  1. linux图形化卡在开机界面,linux怎么在开机时进入图形界面
  2. 任意进制转换简单理解
  3. 用Quartus II Timequest Timing Analyzer进行时序分析 ：实例讲解
  4. 软件测试的缺陷管理系统有哪些,简述：一款优秀的缺陷管理系统有哪些功能特点！...
  5. 使用StyleCop 进行代码评审
  6. oracle一些基本命令
  7. 朋友圈文字怎么到中间_怎样查看微信朋友圈访客记录
  8. froala富文本编辑器与golang、beego，脱离ueditor苦海
  9. 【数据库】Oracle更改时间显示格式
  10. 电源线径大小与用电负荷的关系
  11. heka 介绍，以及编译，备忘
  12. Java同步锁对比synchronized 和ReentrantLock 的区别--超级详细权威
  13. 2022中国新时代100大建筑公布，重庆来福士、北京大兴机场、港珠澳大桥等杰出工程入选 | 美通社头条...
  14. m对比PSO,WPA,GWPA以及GWO四种优化算法的优化性能,优化目标函数为10个来自CEC2017的标准测试函数
  15. Es 超时设置 high-level-client
  16. 怎样用计算机设置隐私空间,华为手机的三种隐私设置，开放的秘密空间，永远不用担心女友检查手机！...
  17. UML(Jude) | UseCase Diagram
  18. TPO， toefl practice online
  19. MPC（3）常用车辆模型
  20. 皇家贝贝《秦俑情》开机 杜淳安以轩“接棒”张艺谋巩俐

热门文章

  1. 计算机组成原理实验一报告——运算器
  2. 数据结构二叉树的链式存储
  3. 如何用70行Java代码实现深度神经网络算法
  4. WRF模型模拟时所遇到的问题及解决方法
  5. T7983 大芳的逆行板载
  6. 论文笔记：残差神经网络(ResNet v1)
  7. #双11故事联播#守护篇| 支付王牌军-我们如何从容应对双11？
  8. stm32 mp3软件音频解码案例分析流程（一）
  9. 智能信用卡还款软件是什么？和传统代还平台有什么区别？
  10. H5 3d立体相册 CSS3特性