Hadoop ecosystem

How did it all start- huge data on the web!
Nutch built to crawl this web data
Huge data had to saved- HDFS was born!
How to use this data?
Map reduce framework built for coding and running analytics – java, any language-streaming/pipes
How to get in unstructured data – Web logs, Click streams, Apache logs, Server logs – fuse,webdav, chukwa, flume, Scribe
Hiho and sqoop for loading data into HDFS – RDBMS can join the Hadoop band wagon!
High level interfaces required over low level map reduce programming– Pig, Hive, Jaql
BI tools with advanced UI reporting- drilldown etc- Intellicus
Workflow tools over Map-Reduce processes and High level languages
Monitor and manage hadoop, run jobs/hive, view HDFS – high level view- Hue, karmasphere, eclipse plugin, cacti, ganglia
Support frameworks- Avro (Serialization), Zookeeper (Coordination)
More High level interfaces/uses- Mahout, Elastic map Reduce
OLTP- also possible – Hbase

转载于:https://www.cnblogs.com/AloneSword/p/3789835.html

Hadoop ecosystem相关推荐

Hadoop Ecosystem解决方案---数据仓库
个人总结的一套基于hadoop的海量数据挖掘的开源解决方案. BI系统: Pentaho pentaho是开源的BI系统中做得算顶尖的了. 提供的核心功能如下: 报表功能: 可视化(clien ...
Hadoop生态圈一览
根据Hadoop官网的相关介绍和实际使用中的软件集,将Hadoop生态圈的主要软件工具简单介绍下,拓展对整个Hadoop生态圈的了解. 这是Hadoop生态从Google的三篇论文开始的发展历程,现已 ...
几篇关于Hadoop+Hive数据仓库的入门文章
Hadoop计算平台和Hadoop数据仓库的区别 http://datasearch.ruc.edu.cn/~boliangfeng/blog/?tag=%E6%95%B0%E6%8D%AE%E4%B ...
Hadoop体系结构– YARN，HDFS和MapReduce
Before reading this post, please go through my previous post at "Hadoop 1.x: Architecture and H ...
Hadoop 1.x：体系结构，主要组件以及HDFS和MapReduce的工作方式
Before reading this post, please go through my previous post at "Introduction to Hadoop" t ...
Hadoop大数据采集
Hadoop HDFS 1.基于HDFS的云盘系统设计与实现: 1 系统基本功能模块设计 HDFS是一个高度容错性的系统, 它是运行在通用硬件上的分布式文件系统, 适合部署在廉价的机器上.HDFS能提 ...
大数据概况以及Hadoop生态系统
一,什么是大数据? 大数据是一个描述大量高速,复杂和可变数据的术语,需要先进的技术来实现信息的捕获,存储,分发,管理和分析.大数据包含四个特性(4v),体量(Volume).速度(Veloci ...
hadoop的组件有哪些
Apache Hadoop是一个分布式计算系统,它主要由以下几个组件组成: Hadoop Distributed File System(HDFS):这是Hadoop的分布式文件系统,负责存储大量的数 ...
Hadoop 生态圈 ————Hbase数据库
HBase是一个分布式的,面向列的开源数据库,该技术来源于Fay Chang所撰写的Google论文"Bigtable:一个结构化数据的分布式存储系统".HBase不同于一般的关系 ...

Hadoop ecosystem

Hadoop ecosystem相关推荐

最新文章

热门文章