云计算

【原创】SAE本地环境与真实环境的差别

其实从刚开始用的时候就发现本地环境和真实环境有不同,导致本地环境我都没怎么用。

1、本地环境的php是32bit版本的,真实环境是64bit的。这是我发现的第一个不相同的地方,就是因为这个,直接导致我的应用绝大部分调试工作都是在把代码上传到服务器后直接在线测试的。因为我做那个应用的时候需要用到64位整数,本地环境不支持。

2、saemysql使用的不同,今天在本地用了一下saemysql,本来以为直接拿以前线上的代码直接在本地环境中运行就可以的,结果发现不行。在线上的版本,只要直接

Hadoop Hbase 升级

Hadoop HDFS 与Hbase升级笔记

由于之前使用了hadoop1.0.2,hbase 使用的是hbase-0.92.1 但是一次事故导致元数据丢失,且修复元数据的类本身有BUG
所以摆在眼前的只有两条路:
1、修改hbase源码重新编译 hbase 修复BUG
2、升级到下一个版本,且这个版本已经修复了此BUG 从release node中看到 0.92.2及以后版本均修复了此bug
  所以决定升级到最新的稳定版 hbase-0.94.3 而此版本的hbase 和hadoop-1.0.4 的兼容性最好,所以hadoop 连带升级到hadoop-1.0.4

1. Hadoop升级步骤:
 (1)停止集群上的所有MR任务,包括Hbase(如果Hbase在使用中,先停掉,接着是zookeeper)
 (2)停止DFS(1、2两步也可以在hbase和zookeeper关闭后使用stop-all.sh脚本直接关闭)
 (3)删除临时数据,即在core-site.xml中设置的hadoop.tmp.dir的value所文件目录下的文件
 (4)备份HDFS元数据

java socket 多线程网络传输多个文件

     由于需要研究了下用 java socket 传输文件,由于需要传输多个文件,因此,采用了多线程设计。客户端每个线程创建一个 socket 连接,每个 socket 连接负责传输一个文件,服务端的ServerSocket每次 accept 一个 socket 连接,创建一个线程用于接收客户端传来的文件。

1、服务端

Getting Started with Tachyon by Use Cases

In-memory computing has become an irreversible trend in big data technology, for which the wide popularity of Spark provides a good evidence. Meanwhile, memory storage and management for large data sets are still posing challenges. Out of numerous solutions, Tachyon, a memory-centric distributed storage, well solves the problems faced by many application scenarios. For example, it avoids severe GC issues due to large in-memory data being stored in JVM heap, enables data sharing across applications/jobs through memory, and reduces overhead caused by JVM heap memory allocation and on-heap data management, which help improve the computing engine’s stability and availability. This article uses application examples to demonstrate how to solve real-world big data issues with Tachyon, and to help understand what Tachyon can offer in various big data scenarios.

英特尔物联网入门代码案例:供水系统

From this exercise, developers will learn how to: Connect the Intel® Edison development platform, a computing platform designed for prototyping and producing IoT and wearable computing products. Interface with the Intel® Edison platform IO and sensor repository using MRAA and UPM from the Intel® IoT Developer Kit, a complete hardware and software solution to help developers explore the IoT and implement innovative projects. Run this code sample in Intel® XDK IoT Edition, an IDE for creating applications that interact with sensors and actuators, enabling a quick start for developing software for the Intel® Edison or Intel® Galileo board. Set up a web application server to store watering system data using Azure Redis Cache* from Microsoft* Azure*, cloud services for connecting IoT solutions including data analysis, machine learning, and a variety of productivity tools to simplify the process of connecting your sensors to the cloud and getting your IoT project up and running quickly. Invoke the services of the Twilio* API for sending text messages.
  • 开发人员
  • 专业版
  • Yocto 项目
  • HTML5
  • 物联网
  • HTML5
  • JavaScript*
  • Node.js*
  • 高级
  • 入门级
  • 蓝牙*
  • Wi-Fi*
  • 云计算
  • 开发工具
  • 物联网
  • 传感器
  • Connecting Sensor Networks and Devices to the Cloud in Just Minutes: Solution Brief

    This article provides the steps to connect an Intel© IoT gateway to the IBM Bluemix* cloud service using Python and MQTT client library in just minutes.

    For more information about this article, please see the IoT forum for Q&A.

  • 开发人员
  • 专业版
  • Linux*
  • Microsoft Windows* (XP, Vista, 7)
  • Microsoft Windows* 10
  • Microsoft Windows* 8.x
  • 物联网
  • Python*
  • 云计算
  • IBM Bluemix*
  • 英特尔® 物联网网关
  • 物联网
  • IoT Device Telemetry using Azure* Event Hubs with .NET: Solution Brief

    This article describes how-to create a .NET IoT device telemetry using the Azure* Event Hub service. It provides the steps to setup an Event Hub on Azure cloud and the .NET C# telemetry sample code for the client IoT device to communicate to the Azure cloud.

    For more information about this article, please see the IoT forum for Q&A.

  • 开发人员
  • 专业版
  • Microsoft Windows* (XP, Vista, 7)
  • Microsoft Windows* 10
  • Microsoft Windows* 8.x
  • 物联网
  • .NET*
  • C/C++
  • 云计算
  • 英特尔® 物联网网关
  • 物联网
  • Microsoft Azure*
  • 订阅 云计算