HBase中region的数目可以根据Section 3.6.5, “更大的 Regions”调整.也可以参见 Section 12.3.1, “Region大小”

对于大型的系统，你需要考虑管理压缩和分割

参见hbase.regionserver.handler.count.这个参数的本质是设置一个RegsionServer可以同时处理多少请求。 如果定的太高，吞吐量反而会降低;如果定的太低，请求会被阻塞，得不到响应。你可以打开RPC-level日志读Log，来决定对于你的集群什么值是合适的。(请求队列也是会消耗内存的)

参见 hfile.block.cache.size. 对于区域服务器进程的内存设置。

参见 hbase.regionserver.global.memstore.upperLimit. 这个内存设置是根据区域服务器的需要来设定。

参见 hbase.regionserver.global.memstore.lowerLimit. 这个内存设置是根据区域服务器的需要来设定。

参见hbase.hstore.blockingStoreFiles. 如果在区域服务器的Log中block,提高这个值是有帮助的。

参见 hbase.hregion.memstore.block.multiplier. 如果有足够的RAM，提高这个值。

HBase does not start when deployed as Amazon EC2 instances. Exceptions like the below appear in the Master and/or RegionServer logs:

2009-10-19 11:52:27,030 INFO org.apache.zookeeper.ClientCnxn: Attempting connection to server ec2-174-129-15-236.compute-1.amazonaws.com/10.244.9.171:2181 2009-10-19 11:52:27,032 WARN org.apache.zookeeper.ClientCnxn: Exception closing session 0x0 to sun.nio.ch.SelectionKeyImpl@656dc861 java.net.ConnectException: Connection refused

Security group policy is blocking the ZooKeeper port on a public address. Use the internal EC2 host names when configuring the ZooKeeper quorum peer list.

Questions on HBase and Amazon EC2 come up frequently on the HBase dist-list. Search for old threads using Search Hadoop

参考 Andrew's answer here, up on the user list: Remote Java client connection into EC2 instance.

HBase 0.90.x does not ship with hadoop-0.20.205.x, etc. To make it run, you need to replace the hadoop jars that HBase shipped with in its libdirectory with those of the Hadoop you want to run HBase on. If even after replacing Hadoop jars you get the below exception:

sv4r6s38: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration sv4r6s38: at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMetricsSystem.java:37) sv4r6s38: at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<clinit>(DefaultMetricsSystem.java:34) sv4r6s38: at org.apache.hadoop.security.UgiInstrumentation.create(UgiInstrumentation.java:51) sv4r6s38: at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:209) sv4r6s38: at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:177) sv4r6s38: at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:229) sv4r6s38: at org.apache.hadoop.security.KerberosName.<clinit>(KerberosName.java:83) sv4r6s38: at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:202) sv4r6s38: at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:177)

you need to copy under hbase/lib, the commons-configuration-X.jar you find in your Hadoop's lib directory. That should fix the above complaint.

The following is an exchange from the user dist-list regarding a fairly common question: how to handle per-user list data in HBase.

*** QUESTION ***

We're looking at how to store a large amount of (per-user) list data in HBase, and we were trying to figure out what kind of access pattern made the most sense. One option is store the majority of the data in a key, so we could have something like:

<FixedWidthUserName><FixedWidthValueId1>:"" (no value) <FixedWidthUserName><FixedWidthValueId2>:"" (no value) <FixedWidthUserName><FixedWidthValueId3>:"" (no value)

The other option we had was to do this entirely using:

<FixedWidthUserName><FixedWidthPageNum0>:<FixedWidthLength><FixedIdNextPageNum><ValueId1><ValueId2><ValueId3>... <FixedWidthUserName><FixedWidthPageNum1>:<FixedWidthLength><FixedIdNextPageNum><ValueId1><ValueId2><ValueId3>...

where each row would contain multiple values. So in one case reading the first thirty values would be:

scan { STARTROW => 'FixedWidthUsername' LIMIT => 30}

And in the second case it would be

get 'FixedWidthUserName\x00\x00\x00\x00'

The general usage pattern would be to read only the first 30 values of these lists, with infrequent access reading deeper into the lists. Some users would have <= 30 total values in these lists, and some users would have millions (i.e. power-law distribution)

The single-value format seems like it would take up more space on HBase, but would offer some improved retrieval / pagination flexibility. Would there be any significant performance advantages to be able to paginate via gets vs paginating with scans?

My initial understanding was that doing a scan should be faster if our paging size is unknown (and caching is set appropriately), but that gets should be faster if we'll always need the same page size. I've ended up hearing different people tell me opposite things about performance. I assume the page sizes would be relatively consistent, so for most use cases we could guarantee that we only wanted one page of data in the fixed-page-length case. I would also assume that we would have infrequent updates, but may have inserts into the middle of these lists (meaning we'd need to update all subsequent rows).

Thanks for help / suggestions / follow-up questions.

*** ANSWER ***

If I understand you correctly, you're ultimately trying to store triples in the form "user, valueid, value", right? E.g., something like:

"user123, firstname, Paul", "user234, lastname, Smith"

(But the usernames are fixed width, and the valueids are fixed width).

And, your access pattern is along the lines of: "for user X, list the next 30 values, starting with valueid Y". Is that right? And these values should be returned sorted by valueid?

The tl;dr version is that you should probably go with one row per user+value, and not build a complicated intra-row pagination scheme on your own unless you're really sure it is needed.

Your two options mirror a common question people have when designing HBase schemas: should I go "tall" or "wide"? Your first schema is "tall": each row represents one value for one user, and so there are many rows in the table for each user; the row key is user + valueid, and there would be (presumably) a single column qualifier that means "the value". This is great if you want to scan over rows in sorted order by row key (thus my question above, about whether these ids are sorted correctly). You can start a scan at any user+valueid, read the next 30, and be done. What you're giving up is the ability to have transactional guarantees around all the rows for one user, but it doesn't sound like you need that. Doing it this way is generally recommended (see here #schema.smackdown).

Your second option is "wide": you store a bunch of values in one row, using different qualifiers (where the qualifier is the valueid). The simple way to do that would be to just store ALL values for one user in a single row. I'm guessing you jumped to the "paginated" version because you're assuming that storing millions of columns in a single row would be bad for performance, which may or may not be true; as long as you're not trying to do too much in a single request, or do things like scanning over and returning all of the cells in the row, it shouldn't be fundamentally worse. The client has methods that allow you to get specific slices of columns.

Note that neither case fundamentally uses more disk space than the other; you're just "shifting" part of the identifying information for a value either to the left (into the row key, in option one) or to the right (into the column qualifiers in option 2). Under the covers, every key/value still stores the whole row key, and column family name. (If this is a bit confusing, take an hour and watch Lars George's excellent video about understanding HBase schema design: http://www.youtube.com/watch?v=_HLoH_PgrLk).

A manually paginated version has lots more complexities, as you note, like having to keep track of how many things are in each page, re-shuffling if new values are inserted, etc. That seems significantly more complex. It might have some slight speed advantages (or disadvantages!) at extremely high throughput, and the only way to really know that would be to try it out. If you don't have time to build it both ways and compare, my advice would be to start with the simplest option (one row per user+value). Start simple and iterate! :)

Investigation results of a self-described "we're not sure what's wrong, but it seems slow" problem. http://gbif.blogspot.com/2012/03/hbase-performance-evaluation-continued.html

Investigation results of general cluster performance from 2010. Although this research is on an older version of the codebase, this writeup is still very useful in terms of approach. http://hstack.org/hbase-performance-testing/

Case study of configuring xceivers, and diagnosing errors from mis-configurations. http://www.larsgeorge.com/2012/03/hadoop-hbase-and-xceivers.html

参考 also Section 2.3.2, “dfs.datanode.max.xcievers”.

主压缩可以通过HBase shell 或 HBaseAdmin.majorCompact 进行。

注意：主压缩不进行区域合并。更多关于压缩的信息，参考 Section 9.7.5.5, “Compaction”

Merge is a utility that can merge adjoining regions in the same table (see org.apache.hadoop.hbase.util.Merge).

$ bin/hbase org.apache.hbase.util.Merge <tablename> <region1> <region2>

If you feel you have too many regions and want to consolidate them, Merge is the utility you need. Merge must run be done when the cluster is down. 参考 the O'Reilly HBase Book for an example of usage.

Additionally, there is a Ruby script attached to HBASE-1621 for region merging.

你可以在HBase的特定的节点上运行下面的脚本来停止RegionServer:

$ ./bin/hbase-daemon.sh stop regionserver

RegionServer会首先关闭所有的region然后把它自己关闭，在停止的过程中，RegionServer的会向Zookeeper报告说他已经过期了。master会发现RegionServer已经死了，会把它当作崩溃的server来处理。他会将region分配到其他的节点上去。

RegionServer下线有一个缺点就是其中的Region会有好一会离线。Regions是被按顺序关闭的。如果一个server上有很多region,从第一个region会被下线，到最后一个region被关闭，并且Master确认他已经死了，该region才可以上线，整个过程要花很长时间。在HBase 0.90.2中，我们加入了一个功能，可以让节点逐渐的摆脱他的负载，最后关闭。HBase 0.90.2加入了 graceful_stop.sh脚本，可以这样用，

$ ./bin/graceful_stop.sh Usage: graceful_stop.sh [--config &conf-dir>] [--restart] [--reload] [--thrift] [--rest] &hostname> thrift If we should stop/start thrift before/after the hbase stop/start rest If we should stop/start rest before/after the hbase stop/start restart If we should restart after graceful stop reload Move offloaded regions back on to the stopped server debug Move offloaded regions back on to the stopped server hostname Hostname of server we are to stop

要下线一台RegionServer可以这样做

$ ./bin/graceful_stop.sh HOSTNAME

这里的HOSTNAME是RegionServer的host you would decommission.

graceful_stop.sh 脚本会一个一个将region从RegionServer中移除出去，以减少改RegionServer的负载。他会先移除一个region,然后再将这个region安置到一个新的地方，再移除下一个，直到全部移除。最后graceful_stop.sh脚本会让RegionServer stop.,Master会注意到RegionServer已经下线了，这个时候所有的region已经重新部署好。RegionServer就可以干干净净的结束，没有WAL日志需要分割。

hbase(main):001:0> balance_switch false true 0 row(s) in 0.3590 seconds

hbase(main):001:0> balance_switch true false 0 row(s) in 0.3590 seconds

你还可以让这个脚本重启一个RegionServer,不改变上面的Region的位置。要想保留数据的位置，你可以依次重启(Rolling Restart),就像这样:

$ for i in `cat conf/regionservers|sort`; do ./bin/graceful_stop.sh --restart --reload --debug $i; done &> /tmp/log.txt &

Tail /tmp/log.txt来看脚本的运行过程.上面的脚本只对RegionServer进行操作。要确认load balancer已经关掉。还需要在之前更新master。下面是一段依次重启的伪脚本,你可以借鉴它：

参见 Metrics 可以获得一个enable Metrics emission的指导。

下面的度量方法对每个区域服务器的宏观监控被证明是是最重要的，特别是在像 OpenTSDB这样的系统中。如果你的集群具有性能问题，你可能得参考本组信息。

HBase:

OS:

Java:

HBase度量的更多信息，参考 Section 14.4, “HBase Metrics”.

HBase查询太慢的日志由可分析的 JSON结构描述。客户端操作 (Gets, Puts, Deletes, 等)的属性， 要么运行太久，或产生输出太多。“运行太久”和“输出太多”的门限可配置，如后面所述。输出产生在主区域服务器日志中，以便和其他日志事件一起发现更多细节。它也前置区分标签(responseTooSlow), (responseTooLarge),(operationTooSlow)和(operationTooLarge)，以便当用户只希望看到慢查询时，用grep过滤。

2011-09-08 10:01:25,824 WARN org.apache.hadoop.ipc.HBaseServer: (operationTooSlow): {"tables":{"riley2":{"puts":[{"totalColumns":11,"families":{"actions":[{"timestamp":1315501284459,"qualifier":"0","vlen":9667580},{"timestamp":1315501284459,"qualifier":"1","vlen":10122412},{"timestamp":1315501284459,"qualifier":"2","vlen":11104617},{"timestamp":1315501284459,"qualifier":"3","vlen":13430635}]},"row":"cfcd208495d565ef66e7dff9f98764da:0"}],"families":["actions"]}},"processingtimems":956,"client":"10.47.34.63:33623","starttimems":1315501284456,"queuetimems":0,"totalPuts":1,"class":"HRegionServer","responsesize":0,"method":"multiPut"}

一些环境可以容忍暂时停止 HBase 集群，如用于后台容量分析，并不提供前台页面。 好处是 NameNode/Master 和 RegionServers是停止的， 不会有任何机会丢失正在处理改变的保存文件或元数据。明显的坏处是集群被 关闭。步骤包括：

这种方法假设有另一个集群。参考HBase的 replication 。

14.1.6节, “CopyTable” 工具，即可用于将一个表复制到同集群的另一个表，也可将表复制到另一个集群的另一个表。

由于集群在工作，有丢失正在改变的数据的风险。

14.1.7节, “Export” 是一种将 HDFS 内容导出到同一集群的方法。 恢复数据， 14.1.8节, “Import” 工具可以使用。

由于集群在工作，有丢失正在改变的数据的风险。

一个常见问题是HBase管理员需要估算一个HBase集群要用多大存储量。可以通过几个方面去考虑，最重要的是集群要加载什么数据。 开始于对HBase内部处理数据(KeyValue)的可靠了解。

Another common question for HBase administrators is determining the right number of regions per RegionServer. This affects both storage and hardware planning. 参考 Section 11.4.1, “Number of Regions”.

Pass -Dsnappy to trigger the snappy maven profile for building snappy native libs into hbase.

Do the following to build the HBase tarball. Passing the -Drelease will generate javadoc and run the RAT plugin to verify licenses on source.

% MAVEN_OPTS="-Xmx2g" mvn clean site install assembly:single -Dmaven.test.skip -Prelease

Follow the instructions at Publishing Maven Artifacts. The 'trick' to making it all work is answering the questions put to you by the mvn release plugin properly, making sure it is using the actual branch AND before doing the mvn release:perform step, VERY IMPORTANT, check and if necessary hand edit the release.properties file that was put under ${HBASE_HOME} by the previous step, release:perform. You need to edit it to make it point at right locations in SVN.

Use maven 3.0.x.

At the mvn release:perform step, before starting, if you are for example releasing hbase 0.92.0, you need to make sure the pom.xml version is 0.92.0-SNAPSHOT. This needs to be checked in. Since we do the maven release after actual release, I've been doing this checkin into a particular tag rather than into the actual release tag. So, say we released hbase 0.92.0 and now we want to do the release to the maven repository, in svn, the 0.92.0 release will be tagged 0.92.0. Making the maven release, copy the 0.92.0 tag to 0.92.0mvn. Check out this tag and change the version therein and commit.

Here is how I'd answer the questions at release:prepare time:

What is the release version for "HBase"? (org.apache.hbase:hbase) 0.92.0: : What is SCM release tag or label for "HBase"? (org.apache.hbase:hbase) hbase-0.92.0: : 0.92.0mvnrelease What is the new development version for "HBase"? (org.apache.hbase:hbase) 0.92.1-SNAPSHOT: : [INFO] Transforming 'HBase'...

A strange issue I ran into was the one where the upload into the apache repository was being sprayed across multiple apache machines making it so I could not release. 参考 INFRA-4482 Why is my upload to mvn spread across multiple repositories?.

Here is my ~/.m2/settings.xml.

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"> <servers> <!- To publish a snapshot of some part of Maven --> <server> <id>apache.snapshots.https</id> <username>YOUR_APACHE_ID </username> <password>YOUR_APACHE_PASSWORD </password> </server> <!-- To publish a website using Maven --> <!-- To stage a release of some part of Maven --> <server> <id>apache.releases.https</id> <username>YOUR_APACHE_ID </username> <password>YOUR_APACHE_PASSWORD </password> </server> </servers> <profiles> <profile> <id>apache-release</id> <properties> <gpg.keyname>YOUR_KEYNAME</gpg.keyname> <!--Keyname is something like this ... 00A5F21E... do gpg --list-keys to find it--> <gpg.passphrase>YOUR_KEY_PASSWORD </gpg.passphrase> </properties> </profile> </profiles> </settings>

When you run release:perform, pass -Papache-release else it will not 'sign' the artifacts it uploads.

If you see run into the below, its because you need to edit version in the pom.xml and add -SNAPSHOT to the version (and commit).

[INFO] Scanning for projects... [INFO] Searching repository for plugin with prefix: 'release'. [INFO] ------------------------------------------------------------------------ [INFO] Building HBase [INFO] task-segment: [release:prepare] (aggregator-style) [INFO] ------------------------------------------------------------------------ [INFO] [release:prepare {execution: default-cli}] [INFO] ------------------------------------------------------------------------ [ERROR] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] You don't have a SNAPSHOT project in the reactor projects list. [INFO] ------------------------------------------------------------------------ [INFO] For more information, run Maven with the -e switch [INFO] ------------------------------------------------------------------------ [INFO] Total time: 3 seconds [INFO] Finished at: Sat Mar 26 18:11:07 PDT 2011 [INFO] Final Memory: 35M/423M [INFO] -----------------------------------------------------------------------

If you see Unable to find resource 'VM_global_library.vm', ignore it. Its not an error. It is officially ugly though.

As of 0.96, HBase is split into multiple modules which creates "interesting" rules for how and where tests are written. If you are writting code for hbase-server, see Section 15.6.2, “Unit Tests” for how to write your tests; these tests can spin up a minicluster and will need to be categorized. For any other module, for example hbase-common, the tests must be strict unit tests and just test the class under test - no use of the HBaseTestingUtility or minicluster is allowed (or even possible given the dependency tree).

mvn test

mvn clean test -Dskip-server-tests

mvn clean test -Dskip-server-tests -Dskip-common-tests

HBase unit tests are subdivided into three categories: small, medium and large, with corresponding JUnit categories: SmallTests, MediumTests,LargeTests. JUnit categories are denoted using java annotations and look like this in your unit test code.

... @Category(SmallTests.class) public class TestHRegionInfo { @Test public void testCreateHRegionInfoName() throws Exception { // ... } ... @org.junit.Rule public org.apache.hadoop.hbase.ResourceCheckerJUnitRule cu = new org.apache.hadoop.hbase.ResourceCheckerJUnitRule(); }

The above example shows how to mark a test as belonging to the small category. The @org.junit.Rule lines on the end are also necessary. Add them to each new unit test file. They are needed by the categorization process. HBase uses a patched maven surefire plugin and maven profiles to implement its unit test characterizations.

mvn test

mvn test -P security

mvn test -P runAllTests

mvn test -P localTests -Dtest=MyTest

mvn test -P localTests -Dtest=MyTest1,MyTest2,MyTest3

mvn test -P localTests -Dtest=org.apache.hadoop.hbase.client.*

mvn test -P security,localTests -Dtest=TestGet

mvn test -P runSmallTests

mvn test -P runMediumTests

mvn test -P runLargeTests

./dev-support/hbasetests.sh

./dev-support/hbasetests.sh runAllTests

./dev-support/hbasetests.sh replayFailed

mvn compile

参考 the Section 15.6.2.4, “Running tests” section above in Section 15.6.2, “Unit Tests”

As of 0.96, HBase supports building against hadoop versions: 1.0.3, 2.0.0-alpha and 3.0.0-SNAPSHOT. By default, we will build with Hadoop-1.0.3. To change the version to run with Hadoop-2.0.0-alpha, you would run:

mvn -Dhadoop.profile=2.0 ...

That is, designate build with hadoop.profile 2.0. Pass 2.0 for hadoop.profile to build against hadoop 2.0. Tests may not all pass as of this writing so you may need to pass -DskipTests unless you are inclined to fix the failing tests.

Similarly, for 3.0, you would just replace the profile value. Note that Hadoop-3.0.0-SNAPSHOT does not currently have a deployed maven artificat - you will need to build and install your own in your local maven repository if you want to run against this profile.

In earilier verions of HBase, you can build against older versions of hadoop, notably, Hadoop 0.22.x and 0.23.x. If you are running, for example HBase-0.94 and wanted to build against Hadoop 0.23.x, you would run with:

mvn -Dhadoop.profile=22 ...

Sign up for the dev-list and the user-list. 参考 the mailing lists page. Posing questions - and helping to answer other people's questions - is encouraged! There are varying levels of experience on both lists so patience and politeness are encouraged (and please stay on topic.)

Check for existing issues in Jira. If it's either a new feature request, enhancement, or a bug, file a ticket.

{code} code snippet {code}

Most development is done on TRUNK. However, there are branches for minor releases (e.g., 0.90.1, 0.90.2, and 0.90.3 are on the 0.90 branch).

If you have any questions on this just send an email to the dev dist-list.

In HBase we use JUnit 4. If you need to run miniclusters of HDFS, ZooKeeper, HBase, or MapReduce testing, be sure to checkout theHBaseTestingUtility. Alex Baranau of Sematext describes how it can be used in HBase Case-Study: Using HBaseTestingUtility for Local Testing and Development (2010).

TODO...

参考 Section 15.2.1.1, “Code Formatting” and Section 15.10.5, “Common Patch Feedback”.

Also, please pay attention to the interface stability/audience classifications that you will see all over our code base. They look like this at the head of the class:

@InterfaceAudience.Public @InterfaceStability.Stable

If the InterfaceAudience is Private, we can change the class (and we do not need to include a InterfaceStability mark). If a class is marked Public but its InterfaceStability is marked Unstable, we can change it. If it's marked Public/Evolving, we're allowed to change it but should try not to. If it'sPublic and Stable we can't change it without a deprecation path or with a really GREAT reason.

When you add new classes, mark them with the annotations above if publically accessible. If you are not cleared on how to mark your additions, ask up on the dev list.

This convention comes from our parent project Hadoop.

If you are developing HBase, frequently it is useful to test your changes against a more-real cluster than what you find in unit tests. In this case, HBase can be run directly from the source in local-mode. All you need to do is run:

${HBASE_HOME}/bin/start-hbase.sh

This will spin up a full local-cluster, just as if you had packaged up HBase and installed it on your machine.

Keep in mind that you will need to have installed HBase into your local maven repository for the in-situ cluster to work properly. That is, you will need to run:

mvn clean install -DskipTests

to ensure that maven can find the correct classpath and dependencies. Generally, the above command is just a good thing to try running first, if maven is acting oddly.

Patch files can be easily generated from Eclipse, for example by selecting "Team -> Create Patch". Patches can also be created by git diff and svn diff.

Please submit one patch-file per Jira. For example, if multiple files are changed make sure the selected resource when generating the patch is a directory. Patch files can reflect changes in multiple files.

Make sure you review Section 15.2.1.1, “Code Formatting” for code style.

The patch file should have the HBase Jira ticket in the name. For example, if a patch was submitted for Foo.java, then a patch file calledFoo_HBASE_XXXX.patch would be acceptable where XXXX is the HBase Jira number.

If you generating from a branch, then including the target branch in the filename is advised, e.g., HBASE-XXXX-0.90.patch.

Yes, please. Please try to include unit tests with every code patch (and especially new classes and large changes). Make sure unit tests pass locally before submitting the patch.

Also, see Section 15.9.2.1, “Mockito”.

If you are creating a new unit test class, notice how other unit test classes have classification/sizing annotations at the top and a static method on the end. Be sure to include these in any new unit test files you generate. 参考 Section 15.5, “Tests” for more on how the annotations work.

The patch should be attached to the associated Jira ticket "More Actions -> Attach Files". Make sure you click the ASF license inclusion, otherwise the patch can't be considered for inclusion.

Once attached to the ticket, click "Submit Patch" and the status of the ticket will change. Committers will review submitted patches for inclusion into the codebase. Please understand that not every patch may get committed, and that feedback will likely be provided on the patch. Fear not, though, because the HBase community is helpful!

The following items are representative of common patch feedback. Your patch process will go faster if these are taken into account beforesubmission.

参考 the Java coding standards for more information on coding conventions in Java.

if ( foo.equals( bar ) ) { // don't do this

if (foo.equals(bar)) {

foo = barArray[ i ]; // don't do this

foo = barArray[i];

public void readFields(DataInput arg0) throws IOException { // don't do this foo = arg0.readUTF(); // don't do this

public void readFields(DataInput di) throws IOException { foo = di.readUTF();

Bar bar = foo.veryLongMethodWithManyArguments(argument1, argument2, argument3, argument4, argument5, argument6, argument7, argument8, argument9); // don't do this

Bar bar = foo.veryLongMethodWithManyArguments( argument1, argument2, argument3,argument4, argument5, argument6, argument7, argument8, argument9);

Bar bar = foo.getBar(); <--- imagine there's an extra space(s) after the semicolon instead of a line break.

/** * * @param bar <---- don't do this!!!! * @return <---- or this!!!! */ public Foo getFoo(Bar bar);

Larger patches should go through ReviewBoard.

For more information on how to use ReviewBoard, see the ReviewBoard documentation.

Committers do this. 参考 How To Commit in the HBase wiki.

Commiters will also resolve the Jira, typically after the patch passes a build.

一个分布式运行的HBase依赖一个zookeeper集群。所有的节点和客户端都必须能够访问zookeeper。默认的情况下HBase会管理一个zookeep集群。这个集群会随着HBase的启动而启动。当然，你也可以自己管理一个zookeeper集群，但需要配置HBase。你需要修改conf/hbase-env.sh里面的HBASE_MANAGES_ZK 来切换。这个值默认是true的，作用是让HBase启动的时候同时也启动zookeeper.

当HBase管理zookeeper的时候，你可以通过修改zoo.cfg来配置zookeeper，一个更加简单的方法是在 conf/hbase-site.xml里面修改zookeeper的配置。Zookeep的配置是作为property写在 hbase-site.xml里面的。option的名字是 hbase.zookeeper.property. 打个比方， clientPort 配置在xml里面的名字是hbase.zookeeper.property.clientPort. 所有的默认值都是HBase决定的，包括zookeeper, 参见 Section 2.6.1.1, “HBase 默认配置”. 可以查找hbase.zookeeper.property 前缀，找到关于zookeeper的配置。 ^[13]

对于zookeepr的配置，你至少要在 hbase-site.xml中列出zookeepr的ensemble servers，具体的字段是 hbase.zookeeper.quorum. 该这个字段的默认值是 localhost，这个值对于分布式应用显然是不可以的. (远程连接无法使用).

举个例子，HBase管理着的ZooKeeper集群在节点 rs{1,2,3,4,5}.example.com, 监听2222 端口(默认是2181)，并确保conf/hbase-env.sh文件中 HBASE_MANAGE_ZK的值是true ，再编辑 conf/hbase-site.xml 设置 hbase.zookeeper.property.clientPort 和 hbase.zookeeper.quorum。你还可以设置 hbase.zookeeper.property.dataDir属性来把ZooKeeper保存数据的目录地址改掉。默认值是 /tmp ，这里在重启的时候会被操作系统删掉，可以把它修改到 /user/local/zookeeper.

<configuration> ... <property> <name>hbase.zookeeper.property.clientPort</name> <value>2222</value> <description>Property from ZooKeeper's config zoo.cfg. The port at which the clients will connect. </description> </property> <property> <name>hbase.zookeeper.quorum</name> <value>rs1.example.com,rs2.example.com,rs3.example.com,rs4.example.com,rs5.example.com</value> <description>Comma separated list of servers in the ZooKeeper Quorum. For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com". By default this is set to localhost for local and pseudo-distributed modes of operation. For a fully-distributed setup, this should be set to a full list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in hbase-env.sh this is the list of servers which we will start/stop ZooKeeper on. </description> </property> <property> <name>hbase.zookeeper.property.dataDir</name> <value>/usr/local/zookeeper</value> <description>Property from ZooKeeper's config zoo.cfg. The directory where the snapshot is stored. </description> </property> ... </configuration>

... # Tell HBase whether it should manage it's own instance of Zookeeper or not. export HBASE_MANAGES_ZK=false

${HBASE_HOME}/bin/hbase-daemons.sh {start,stop} zookeeper

Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=false useTicketCache=true; };

Server { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="$PATH_TO_ZOOKEEPER_KEYTAB" storeKey=true useTicketCache=false principal="zookeeper/$HOST"; }; Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true useTicketCache=false keyTab="$PATH_TO_HBASE_KEYTAB" principal="hbase/$HOST"; };

export HBASE_OPTS="-Djava.security.auth.login.config=$CLIENT_CONF" export HBASE_MANAGES_ZK=true export HBASE_ZOOKEEPER_OPTS="-Djava.security.auth.login.config=$HBASE_SERVER_CONF" export HBASE_MASTER_OPTS="-Djava.security.auth.login.config=$HBASE_SERVER_CONF" export HBASE_REGIONSERVER_OPTS="-Djava.security.auth.login.config=$HBASE_SERVER_CONF"

<configuration> <property> <name>hbase.zookeeper.quorum</name> <value>$ZK_NODES</value> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <property> <name>hbase.zookeeper.property.authProvider.1</name> <value>org.apache.zookeeper.server.auth.SASLAuthenticationProvider</value> </property> <property> <name>hbase.zookeeper.property.kerberos.removeHostFromPrincipal</name> <value>true</value> </property> <property> <name>hbase.zookeeper.property.kerberos.removeRealmFromPrincipal</name> <value>true</value> </property> </configuration>

bin/hbase zookeeper start bin/hbase master start bin/hbase regionserver start

Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true useTicketCache=false keyTab="$PATH_TO_HBASE_KEYTAB" principal="hbase/$HOST"; };

export HBASE_OPTS="-Djava.security.auth.login.config=$CLIENT_CONF" export HBASE_MANAGES_ZK=false export HBASE_MASTER_OPTS="-Djava.security.auth.login.config=$HBASE_SERVER_CONF" export HBASE_REGIONSERVER_OPTS="-Djava.security.auth.login.config=$HBASE_SERVER_CONF"

<configuration> <property> <name>hbase.zookeeper.quorum</name> <value>$ZK_NODES</value> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> </configuration>

authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider kerberos.removeHostFromPrincipal=true kerberos.removeRealmFromPrincipal=true

Server { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="$PATH_TO_ZOOKEEPER_KEYTAB" storeKey=true useTicketCache=false principal="zookeeper/$HOST"; };

SERVER_JVMFLAGS="-Djava.security.auth.login.config=$ZK_SERVER_CONF" bin/zkServer start

bin/hbase master start bin/hbase regionserver start

11/12/05 22:43:39 INFO zookeeper.Login: successfully logged in. 11/12/05 22:43:39 INFO server.NIOServerCnxnFactory: binding to port 0.0.0.0/0.0.0.0:2181 11/12/05 22:43:39 INFO zookeeper.Login: TGT refresh thread started. 11/12/05 22:43:39 INFO zookeeper.Login: TGT valid starting at: Mon Dec 05 22:43:39 UTC 2011 11/12/05 22:43:39 INFO zookeeper.Login: TGT expires: Tue Dec 06 22:43:39 UTC 2011 11/12/05 22:43:39 INFO zookeeper.Login: TGT refresh sleeping until: Tue Dec 06 18:36:42 UTC 2011 .. 11/12/05 22:43:59 INFO auth.SaslServerCallbackHandler: Successfully authenticated client: authenticationID=hbase/ip-10-166-175-249.us-west-1.compute.internal@HADOOP.LOCALDOMAIN; authorizationID=hbase/ip-10-166-175-249.us-west-1.compute.internal@HADOOP.LOCALDOMAIN. 11/12/05 22:43:59 INFO auth.SaslServerCallbackHandler: Setting authorizedID: hbase 11/12/05 22:43:59 INFO server.ZooKeeperServer: adding SASL authorization for authorizationID: hbase

11/12/05 22:43:59 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=ip-10-166-175-249.us-west-1.compute.internal:2181 sessionTimeout=180000 watcher=master:60000 11/12/05 22:43:59 INFO zookeeper.ClientCnxn: Opening socket connection to server /10.166.175.249:2181 11/12/05 22:43:59 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 14851@ip-10-166-175-249 11/12/05 22:43:59 INFO zookeeper.Login: successfully logged in. 11/12/05 22:43:59 INFO client.ZooKeeperSaslClient: Client will use GSSAPI as SASL mechanism. 11/12/05 22:43:59 INFO zookeeper.Login: TGT refresh thread started. 11/12/05 22:43:59 INFO zookeeper.ClientCnxn: Socket connection established to ip-10-166-175-249.us-west-1.compute.internal/10.166.175.249:2181, initiating session 11/12/05 22:43:59 INFO zookeeper.Login: TGT valid starting at: Mon Dec 05 22:43:59 UTC 2011 11/12/05 22:43:59 INFO zookeeper.Login: TGT expires: Tue Dec 06 22:43:59 UTC 2011 11/12/05 22:43:59 INFO zookeeper.Login: TGT refresh sleeping until: Tue Dec 06 18:30:37 UTC 2011 11/12/05 22:43:59 INFO zookeeper.ClientCnxn: Session establishment complete on server ip-10-166-175-249.us-west-1.compute.internal/10.166.175.249:2181, sessionid = 0x134106594320000, negotiated timeout = 180000

git clone git://git.apache.org/hbase.git cd hbase mvn -Psecurity,localTests clean test -Dtest=TestZooKeeperACL

bin/hbase zookeeper & bin/hbase master & bin/hbase regionserver &

echo `find ~/.m2 -name "*hadoop-core*7070*SNAPSHOT.jar"` ':' `cat target/cached_classpath.txt` | sed 's/ //g' > target/tmp.txt mv target/tmp.txt target/cached_classpath.txt

[33] For the full list of ZooKeeper configurations, see ZooKeeper's zoo.cfg. HBase does not ship with a zoo.cfg so you will need to browse the conf directory in an appropriate ZooKeeper download.