Cloud computing





Hadoop Hbase 升级

Hadoop HDFS 与Hbase升级笔记

由于之前使用了hadoop1.0.2,hbase 使用的是hbase-0.92.1 但是一次事故导致元数据丢失,且修复元数据的类本身有BUG
1、修改hbase源码重新编译 hbase 修复BUG
2、升级到下一个版本,且这个版本已经修复了此BUG 从release node中看到 0.92.2及以后版本均修复了此bug
  所以决定升级到最新的稳定版 hbase-0.94.3 而此版本的hbase 和hadoop-1.0.4 的兼容性最好,所以hadoop 连带升级到hadoop-1.0.4

1. Hadoop升级步骤:

java socket 多线程网络传输多个文件

     由于需要研究了下用 java socket 传输文件,由于需要传输多个文件,因此,采用了多线程设计。客户端每个线程创建一个 socket 连接,每个 socket 连接负责传输一个文件,服务端的ServerSocket每次 accept 一个 socket 连接,创建一个线程用于接收客户端传来的文件。

1、服务端 releases experimental support for the Intel® Edison

Managing a fleet of IoT devices and deploying code is no easy task. changes the workflow by leveraging Git and Docker technology!

How It Works

When you have new code for your end devices, all you need to do is simply perform a "git push". builds your code into a Docker container and deploys it onto the device if/when it's online! Below is an image describing the process, found on's website:

Restudy SchemaRDD in SparkSQL

At the very beginning, SchemaRDD was just designed as an attempt to make life easier for developers in their daily routines of code debugging and unit testing on SparkSQL core module. The idea can boil down to describing the data structures inside RDD using a formal description similar to the relational database schema. On top of all basic functions provided by common RDD APIs, SchemaRDD also provides some straightforward relational query interface functions that are realized through SparkSQL. After several releases and updates, SchemaRDD successfully drew attention among developers in Spark community. Now, it is officially renamed to “DataFrame” API on Spark’s latest trunk. This article starts with background of SchemaRDD, and then analyzes its design principles and application characteristics. Finally, it gives a brief review of SchemaRDD’s history, and makes a general discussion on its application prospects in Spark’s future development trends.
Iscriversi a Cloud computing