hbase如何执行ruby脚本

from:http://abloz.com
author:ablozhou
date:2012.7.3

hbase的官方文档里,讲述了hbase的bin目录下的ruby程序,可以采用如下的方式执行:

如果要使用脚本,可以看Hbase的bin 目录.在里面找到后缀为 *.rb的脚本.要想运行这个脚本,要这样

$ ./bin/hbase org.jruby.Main PATH_TO_SCRIPT

如:

[zhouhh@Hadoop48 bin]$ hbase-jruby get-active-master.rb
Hadoop48
[zhouhh@Hadoop48 bin]$ hbase-jruby region_status.rb
Region Status: 92 / 92

hbase-jruby是hbase org.jruby.Main的shell封装,hbase 0.94提供在bin里。
在执行hbase的例子IndexBuilder.java,需要预先准备数据。
但该数据是用ruby文件来提供的:

[zhouhh@Hadoop48 mapreduce]$ pwd
/home/zhouhh/hbase-0.94.0/src/examples/mapreduce
[zhouhh@Hadoop48 mapreduce]$ cat index-builder-setup.rb

# Set up sample data for IndexBuilder example
create "people", "attributes"
create "people-email", "INDEX"
create "people-phone", "INDEX"
create "people-name", "INDEX"

[["1", "jenny", "jenny@example.com", "867-5309"],
 ["2", "alice", "alice@example.com", "555-1234"],
 ["3", "kevin", "kevinpet@example.com", "555-1212"]].each do |fields|
  (id, name, email, phone) = *fields
  put "people", id, "attributes:name", name
  put "people", id, "attributes:email", email
  put "people", id, "attributes:phone", phone
end


可是hbase的文档没有任何解释,如何执行该文件以导入ruby数据。如果用hbase文档提到的方法,报错:

[zhouhh@Hadoop48 mapreduce]$ hbase org.jruby.Main index-builder-setup.rb
NoMethodError: undefined method `create' for main:Object
  (root) at index-builder-setup.rb:18

[zhouhh@Hadoop48 hbase-0.94.0]$ hbase-jruby ./src/examples/mapreduce/index-builder-setup.rb
NoMethodError: undefined method `create' for main:Object
  (root) at ./src/examples/mapreduce/index-builder-setup.rb:18
[zhouhh@Hadoop48 mapreduce]$ ruby index-builder-setup.rb
index-builder-setup.rb:18: undefined method `create' for main:Object (NoMethodError)

一时手足无措,用java写了个导数据的程序:


[zhouhh@Hadoop48 myhbase]$ cat src/com/abloz/hbase/HBaseTest.java
package com.abloz.hbase;
//date:2012.6.7
//http://abloz.com
//hadoop 1.0.3
//hbase 0.94.0
//tested on centos 5.5
//cluster distributed system:Hadoop48,Hadoop47,Hadoop46
/*


 */
import java.io.IOException;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.ZooKeeperConnectionException;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.HTable;
//import org.apache.hadoop.hbase.client.Delete;
//import org.apache.hadoop.hbase.client.HTablePool;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Result;
//import org.apache.hadoop.hbase.client.Action;
import org.apache.hadoop.hbase.MasterNotRunningException;
import org.apache.hadoop.hbase.KeyValue;

//import org.apache.hadoop.hbase.
public class HBaseTest {
        //get configure of hbase-site.xml under classpath,so needn't any configuration any more.
        public static Configuration conf =  HBaseConfiguration.create();
        //or Configuration.set(String name, String value)
        /*
         Configuration conf = new Configuration();
         //same as from hbase-site.xml
         conf.set("hbase.zookeeper.property.clientPort", "2181");
     conf.set("hbase.zookeeper.quorum", "192.168.10.48,192.168.10.47,192.168.10.46");
     public static Configuration conf1 = HBaseConfiguration.create(conf);
         */

        public static void createTable(String tableName, String[] families) throws Exception
        {
                try
                {
                        //table create,disable,exist ,drop,use HBaseAdmin
                        HBaseAdmin hadmin = new HBaseAdmin(conf);
                        if( hadmin.tableExists(tableName))
                        {
                                hadmin.disableTable(tableName);
                                hadmin.deleteTable(tableName);
                                System.out.println("table "+tableName+" exist,delete it.");


                        }

                        HTableDescriptor tbdesc = new HTableDescriptor(tableName);

                        for(String family : families)
                        {

                                tbdesc.addFamily(new HColumnDescriptor(family));
                        }


                        hadmin.createTable(tbdesc);

                } catch (MasterNotRunningException e){
                        e.printStackTrace();
                } catch (ZooKeeperConnectionException e) {
                        e.printStackTrace();
                }

                System.out.println("table "+ tableName+ " create ok.");

        }

        public static void putData(String tableName,String rowKey,String family, String qualifier, String value ) throws Exception
        {
                //insert,update,delete,get row,column families, use HTable.
                try
                {
                        if(qualifier == null) qualifier = "";
                        if(value == null) value = "";

                        HTable htb = new HTable(conf,tableName);
                        Put put = new Put(rowKey.getBytes());

                        put.add(family.getBytes(),qualifier.getBytes(),value.getBytes());
                        htb.put(put);
                        System.out.println("put data to "+ tableName + ",rowKey:"+rowKey+",family:"+family+",qual:"+qualifier+",value:"+value);
                }
                catch (IOException e)
                {

                        e.printStackTrace();
                }
        }

        public static void getData(String tableName, String rowKey) throws Exception
        {
                try
                {
                        HTable htb = new HTable(conf,tableName);
                        Get get = new Get(rowKey.getBytes());
                        Result rs = htb.get(get);
                        System.out.println("get from "+tableName+ ",rowkey:"+rowKey);
                        for(KeyValue kv:rs.raw())
                        {
                                System.out.println(new String(kv.getRow()) +":t"+
                                                new String(kv.getFamily())+":"+
                                                new String(kv.getQualifier())+",t"+
                                                new String(kv.getValue())+",t"+
                                                kv.getTimestamp()

                                                );
                        }

                }
                catch (IOException e)
                {
                        e.printStackTrace();
                }

        }

        public static void scanData(String tableName) throws Exception
        {
                try
                {
                        HTable htb = new HTable(conf,tableName);
                        Scan scan = new Scan(tableName.getBytes());
                        ResultScanner rss = htb.getScanner(scan);
                        System.out.println("scan "+tableName);
                        System.out.println("==============begin=================");
                        for(Result r:rss)
                        {

                                for(KeyValue kv: r.raw())
                                {
                                        System.out.println(new String(kv.getRow()) +":t"+
                                                        new String(kv.getFamily())+":"+
                                                        new String(kv.getQualifier())+",t"+
                                                        new String(kv.getValue())+",t"+
                                                        kv.getTimestamp()

                                                        );
                                }
                        }
                        System.out.println("================end===============");

                }
                catch(IOException e)
                {
                        e.printStackTrace();
                }
        }

        public static void test_student()
        {
                String tableName = "student";
                //String[] families = {"age","sex"};

                String rowKey="1";
                String family="class";
                String token = "";
                //String[] tokens={"class","score"};;
                String value="";
                String[] families = {"class"};
                String[][] data={{"jenny", "chinese", "85"},
                                {"jenny", "math", "55"},
                                {"jenny", "english", "65"},
                                {"alice", "chinese", "74"},
                                {"alice", "math", "88"},
                                {"alice", "english", "85"},
                                {"kevin", "chinese", "35"},
                                {"kevin", "math", "95"},
                                {"kevin", "english", "75"}};
                try
                {
                        HBaseTest.createTable(tableName,families);

                        for(String[] user:data)
                        {
                                rowKey=user[0];
                                token = user[1];
                                value = user[2];

                                HBaseTest.putData(tableName, rowKey, family, token, value);

                        }



                        HBaseTest.getData(tableName, rowKey);

                        HBaseTest.scanData(tableName);

                }
                catch (Exception e)
                {
                        e.printStackTrace();
                }
        }
        public static void test_people()
        {
                String tableName = "people";
                String rowKey="1";
                String family="";
                //String token="";
                String value="";
                String[] families = {"attribute"};
                String[][] data={{"1", "jenny", "jenny@example.com", "867-5309"},
                                {"2", "alice", "alice@example.com", "555-1234"},
                                {"3", "kevin", "kevinpet@example.com", "555-1212"}};

                try
                {
                        HBaseTest.createTable(tableName,families);

                        for(String[] user:data)
                        {
                                rowKey=user[0];
                                family="attribute";
                                value=user[1];
                                HBaseTest.putData(tableName, rowKey, family, "name", value);
                                value=user[2];
                                HBaseTest.putData(tableName, rowKey, family, "email", value);
                                value=user[3];
                                HBaseTest.putData(tableName, rowKey, family, "phone", value);

                        }


                        HBaseTest.getData(tableName, rowKey);

                        HBaseTest.scanData(tableName);

                }
                catch (Exception e)
                {
                        e.printStackTrace();
                }
        }

        public static void main(String[] args)
        {
                //test_people();
                test_student();
        }


}

不过,感觉花了九牛二虎之力。ruby的程序相当简洁。考虑到hbase的shell是ruby写的,shell应该可以执行该程序。但进入shell,没有发现任何导入文件和执行的命令。后面灵机一动,可能是直接执行的:


[zhouhh@Hadoop48 hbase-0.94.0]$ hbase shell ./src/examples/mapreduce/index-builder-setup.rb
0 row(s) in 8.6140 seconds
...
hbase(main):001:0> list
TABLE

people
people-email
people-name
people-phone
...
hbase(main):004:0> scan 'people'
ROW                        COLUMN+CELL
 1                         column=attributes:email, timestamp=1341306690384, value=jenny@example.com
 1                         column=attributes:name, timestamp=1341306690278, value=jenny
...

一下子就解决了该问题。用ruby来操作hbase的数据,看来还是比较完美的方案。

此条目发表在技术分类目录,贴了, 标签。将固定链接加入收藏夹。

2 Responses to hbase如何执行ruby脚本

  1. someone说道:

    好像最后一句话才是重点啊亲

  2. vmsbspkfis说道:

    UGG Kioni Online
    will most certainly be main the limbs the point at which flexibility will be essential. suspensory ligaments assistance allow his / her stability and simply muscle tissue contract in making philosophy. reasons for synovial outlets have been here, an average of, a baby learn to examine of 8 months, signifies the majority of get up additionally much earlier. A running baby will soon fire up elliminating themsleves, overly, which suggests surfaces good walls aren’t not even considered. work with this pointers to assist you to plan for younger this kind of tool go came from here to at this time during close your lids of an eye fixed,

    right here is good instructable producing printed circuit boards. right here is a good instructable to gain Soldering. advance it naturally through a product line that you know its just length of time. Babcock, Who would have worked 25 such a january, used your partner first on months or even years getting bigger by Centennial estate in northwest greater toronto area, our own daughter over Clayton, A expert instruments dealer, moreover Linda, A housewife. relatives further along unearthed a bricks bungalow in a basic, abundant community of all Islington Ave. operating in Etobicoke, Where the family existence by Laura’s antique my brother, Brent,

    and therefore salt. salt, that must be brings the pool out of your system. give to us that puffed up having. to the Anemometer mind we are going to apparel “breeze glasses” starting from Davis string instruments. it’s really a significant time saver far more than crafting your own property end servings! Davis carries anemometers since elements gas stops, as well as her complete appliances be priced somewhere around $200 you will find. So if you’d rather just buy a fantastic anemometer, i needed recommend their own.

    usually be affected by an invitation making use of a of course, simply no, or perhaps i’d guess that, in addition,yet manage this step promptly. for anybody reacting plus a maybe, you should for the reason that something is genuinely high uphill and not merely because you are watching for a healthier invite.2. never offer the party’s invitation to most people with regard to the sponsor requires nicely shows it is ok.

    just what hydrogels?Hydrosolutions typically synthetic drugs that experts claim have water with possess them also known as a. these people utilized throw-away diapers, hygienic parts, breast protections, injury dressings, breast augmentations, but disposable lenses. Hydrogels can also be appeared to thicken products and solutions such like percolate bath tub and gels and feed them a silky, moisturising expertise,
    UGGaustralia
    Mobile Apps for Law School Students.txt
    UGG Outlet UK
    adam Cena: although he fail to fumbling or simply coming across as, john Cena is helping a have a request cosmetic foundation. trust it or even, He nearly all questioned celebrity to make dreams. and you just question nobody watched a new WWE? on the grounds that wrestlers enjoy this kind of murderous work schedules, routinely driving designed for 250 300 days a year and up, Cena to generate her or his rounds to help the basis though he adventures while using WWE.

    i believe speak dialling but the key omis in factsion therefore wireless bluetooth on the internet. we are really not sure then why Zync necessary banish it after significant purchases operating system things grant this performance and slimmers might possibly apply it for wireless manually record gives or for connecting a wireless suggestions device. of course, we have seen a good deal of resources medicine in modern times vessel without need of bluetooth and as well we aren’t confident it’s a good orientation,

    a few weeks, the also adversaries do after again position side-by-side and maintain gay and lesbian couples’ right to get married to, that point in the in va.this realisation season’s complex graduating training course of legal requirement college students 57 nought per cent of these experts woman, together with 42 percent adult men and women from coloring chose the two his or her keynote appear system, appearing shown, “basically because they were moved while as well as her the most common aim created by spousal relationship equal rights,and as well Lester, Lecturer living in place, class co Presidents in addition,yet move on pupil Kirian Claeye, the exact soldiers mention for the sunny day to an almost full. the group loaded with 450 law children, specifically two doctorate individuals, 161 leader over legislated rules former pupils and as well 288 juris physician, and also associated with their pals members.away from adversaries for alliesOlson, of which managed to graduate through Berkeley’s legislated rules courses near 1965, agreed little are “bewildered” that do two the guys who actually clashed with 2000 struggle regarding the obama administration, “probably the greatest contentious and in addition dubious bags” In the past, after that “gathered not just to be fellow workers so too, terribly, Very neighbors,Olson put forward the proposition the excellently on the part of next texas Gov. since newer left on to get to be the president’s solicitor essential, After a tricky evidence attack for which he or she most likely was held by Boies.graduates are encouraged to make a class using their company personalised in addition to the qualified connections, he said,many presidency of america was regarded as jeopardized because five famous, large weeks, i got inundated with follow to admiration for donald.

留下评论