Chromebooks*: Developing the user experience of the future

By Richard Winterton, Sarah Han, and Georgene Dee

Introduction

Chromebooks have established a new computer use paradigm, leveraging the internet, and are fast becoming the platform of choice for collaboration at work and school via use of applications such as Google Docs* and Google Hangouts*. Chromebooks provide users with a variety of applications including ones for writing documents, creating presentation, spreadsheets, and creating drawings, without the need to install a suite of applications. Users can also listen to their favorite music or watch movies with the option to easily connect to a larger screen via Chromecast*. The emergence of WebRTC (Web Real Time Communications) technology and Google Hangouts, makes the Chromebook a platform of choice for real-time audio and video communication and collaboration. Google Docs makes virtual group collaboration simple and easy. The growth and acceptance of the Internet and its corresponding technologies such as HTML, JavaScript, Cascade Style Sheets (CSS), and WebGL have made all this possible. With all these advantages comes challenges. Users have grown accustomed to a rich and responsive user interface and delivering a comparable user experience on Chromebooks has its challenges. One key factor regards creating a seamless experience both online and offline to meet user expectations of having applications that run well even without an internet connection. Traditional applications and their associated technologies have had the advantage of decades of improvements and developer tools whereas Internet based applications and technologies have only come to prominence debatably over the last ten years. This paper presents methods, tools and technologies that are helping to enable Chromebook applications, and explains some of the challenges and opportunities we face in changing the way people use computers.

Exceeding User Expectations on a Chromebook

The enhanced capabilities of internet technologies have enabled various appealing key Chromebook features such as:

  • more secure
  • fast boot time
  • easy or no provisioning and deployment
  • team collaboration with office applications
  • open source sharing of JavaScript, CSS techniques
  • rapid prototyping of applications

The same technologies that make these great features possible for Chromebooks also creates some difficult and challenging problems for Chromebook application developers. Tools and technologies used in traditional application development, such as DirectX, OpenGL, vectorizing compilers, and integrated developer environments for debugging, profiling and optimizing have evolved over the years to help developers create a great user experience. Intel continues to push efforts to enable Internet and Chromebook application developers in this evolving developer environment transformation from traditional applications to internet applications.

Most users have developed a set of expectations regarding web or internet based applications and application capabilities. In order for a web technology based application to exceed user expectations the developer needs a detailed understanding of the application being developed, the Chrome* browser, the Chromebook operating system, and the available tools.

This paper is not intended to be an in-depth or all inclusive paper on how to design applications for Chromebooks, but will provide some in-sights and an overview of the following:

  • Chromebook architecture and the Chrome browser
  • V8 JavaScript engine
  • Blink
  • SKIA
  • WebGL
  • Chrome Tools and other developer tools and optimization techniques

Chromebook Architecture Overview

To meet the expectations users have grown accustomed to with native applications, Chromebook applications should have a similar experience to that of native or traditional applications which includes:

  • A responsive user interface
  • Simple program control flow
  • Connected and non-connected functionality
  • Engaging user experience with graphics and multi-media

To provide these capabilities the developer needs to have a good understanding of the components and architecture of Chromebooks, the Chrome browser, and the runtime extensions that provide these capabilities.

The diagram below shows a high level overview of different Chromebook components from the end user application to the hardware.


Chromebook high level block diagram

The organization will start simple and at the top of the stack and move down exposing details that will be helpful to a developer in creating an application that exceeds the user’s expectations.

Difference between an Internet/Web Application and a Chrome Runtime Extension

Web and Internet Applications that run on a Chromebook are based on several key technologies.

A Chromebook application consists of three core technologies:

  • Hypertext Markup Language – HTML and its corresponding tags and objects
  • Cascade Style Sheets – CSS allows developers to create selectors and properties for the selectors and values for the properties. Creating good CSS pages can effectively implement classes and attributes that may be inherited by other selectors greatly minimizing the work and special effects in the application.
  • JavaScript – “Current programming language of the internet”

Additional functional components in the Chrome Runtime Extensions (CRX) are unique to Chrome and Chromebooks. Many of these API’s are documented at:

                https://developer.chrome.com/api_index

Application developers that are familiar with native application programming will see a lot of the same functionality available to CRX Chromebook developers. Some of the more interesting ones are:

  • Messaging Events and Alarms
  • Application Runtime Control
  • Window Customization
  • Local File System Access, Chrome Storage and Google Drive* Sync functionality (allowing applications to have seamless offline-online-offline transitions and functionality)
  • I18N (Internationalization assistance)
  • Authentication assistance (e.g. authenticating to OAuth2 access tokens)
  • Multimedia access
  • System / application power information
  • Independent network access (UDP and TCP chrome.socket API’s)
  • System information such as CPU, memory, storage
  • Information from the Chrome WebStore

This paper will also touch on other technologies such as WebGL, and future functionality such as direct Single Instruction Multiple Data (SIMD) programming that will leave you thinking “is there really anything else I need”. The more complex applications you write, debug and optimize you will find yourself saying, “If I could only do this”, or “if I could debug faster”. That is one of the beauties Intel has found with open source technologies. You can get your hands dirty and contribute. It is time to start breaking down the layers.

Web / Internet Applications

Web / Internet applications typically contain three basic components: HTML, CSS and JavaScript. Moving down the layers, this paper will reference several components within the Chrome browser. The following is a high level block diagram of components contained within the Chrome browser.

Chromebooks rely on the Chrome browser and its infrastructure to do a lot of the work. Most HTML pages and applications contain objects that are tagged by either name and/or an ID. These identifiers are places within the HTML document that must be accessed or recognized by the DOM or Document Object Model software. By using Intel® VTune™ Amplifier1, a profiler available from Intel at http://intel.ly/vtune-amplifier-xe, and a simple HTML file we can see the code flow within the browser infrastructure. I intentionally want to keep the example extremely simple to emphasize the Chrome components and what the HTML, JavaScript and CSS functions are doing. The HTML looks like the following:

<html>
  <head>
    <meta charset="UTF-8">
  </head>
  <body style="width: 1024px">
    <div id="demo">Chromebook-Opt:</div>
    <hr>
  </body>
</html>

When we use Intel® VTune™ Amplifier to look at the profile we see the components within Chrome doing their job. You see SKIA the Chrome graphic software (S32_AOpaque_BlitRow32_SSE2, SkLumaColorFilter and SkA8_Shader_Bliter). There is not much else happening on the page, but serves as a building block to understand how components are used to render the HTML pages.

In Chrome much of the work is done by a component named “Blink,” formerly Webkit. The DOM associates the objects with the actions that need to be taken and are defined in the CSS and JavaScript files. Blink helps to define what parts of the Window need to be drawn and redrawn. Let’s include a css file to our example and set a Chrome developer option to make this concept clearer.

<html>
<head>
  <meta charset="UTF-8">
  <link rel="stylesheet" type="text/css" href="chromebook-opt1.css">
  </script-->
  </head>
<body style="width: 1024px">
  <div id="demo"><font color="lightgray">Chromebook-Opt:</font></div>
  <hr>
</body>
</html>

 

The CSS file contains the following:

html {
  overflow-x: hidden;
}
body {
  font-family: Helvetica, Arial, sans-serif;
  background-color:black;
}
}

After these modifications are made you will see a black screen and the light grey letter, which will make the next steps easier to see. Chrome browser has a developer feature that allows you to dynamically visualize which components will be repainted by the render. In Chrome version 35+ you can enable this feature via the following steps.

  • In the omnibox enable the developer experiments by typing in chrome://flags and then searching for developer.
  • Do a <control shift i> to pop up the developer tools select the tools cog and disable cache, show rendering view in console drawer
  • Click on the console in the developer view and look for the console drawer. You will see Console, Search, Emulation and Rendering.
  • Click on the Rendering tab. You will see show paint rectangles, show composite layer and borders, fps meter, enable continuous page repainting and potential scroll bottlenecks.

After you do this, re-launch Chrome with the button at the bottom prompting you to re-launch Chrome and you should see the following:

It will contain squares showing where Chrome decides to partition the screen to invalidate and redraw content. Now let’s add in a <div> that will show us an update of a real time clock updating once a second. We will add the following to the HTML:

<div id=”demo”> chromebook example: </div>

Inside of the header we also need to add a JavaScript file:

<script src=”chromebook-example2.js”></script>

Inside of the Javascript file we will have this:

function doUpdate(t, addTime) {
  var tmp;
  startTime += addTime;
  tmp = startTime.toFixed(1);
  outputInfo = "chromebook example: Current time: " + t + " Time running (s): " + tmp.toString(10);
  $("demo").innerHTML= outputInfo;
}

function myTimer() {
  var runningTime;
  var CurrentTime;
  var d = new Date();
  var result;

  CurrentTime = new Date().getTime();
  runningTime = (CurrentTime - PrevTime)/1000;
  PrevTime = CurrentTime;
  doUpdate(CurrentTime, runningTime);

  });

}

function init() {
  var myVar=setInterval(function() {
    myTimer()
  },1000);
  }, true);
}

document.addEventListener('DOMContentLoaded', function(tab) {
  init();
});

This will create a running time at the top of the screen that will show the following:

chromebook example: Current time: Current Time xxxxxx. Time running (s) xx.x

And will continuously update every second as specified in the init function by setinterval. Note that if you are only displaying seconds there is no reason to update the <div> section at a faster rate. Updating the <div> section more frequently will waste CPU and GPU processing and will cause the platform to use more power unnecessarily.

A real life example of mismatching the content with the refresh or update rate was found on a well-known financial website. Stock information in a section of the screen was being updated at a rate of 30 times per second. Since stock value don’t change at a rate of 30 times a second there is no apparent need to update the screen that frequently. Understanding how Chrome redraws and the corresponding refresh rates of page elements is an important concept.

Now let’s take a look at something significantly more complex. This example comes from Paul Neave and bouncyballs.org found from the http://www.chromeexperiments.com site. This site is maintained by Google* and features several good examples which utilize Canvas, WebGL and other web technologies available on Chromebooks. Without the Chrome Tools developer option enabled, the balls look as follows:

With the developer option enabled you are able to see some of the complexity behind the page. Notice that each one of the balls is a <div>. The lighting effect is done by creating a separate <div> and layering it on top of the other <div> with a transparency. This implementation perhaps provides the developer the greatest flexibility in moving the light source around but if the light source is static as in this case above, having an extra <div> will cause Chrome and the DOM to do extra work keeping track of where these objects are on the screen and how they move. It may also cause an unnecessary refresh of the screen. These are all factors to consider when writing an application, as it could affect power efficiency and the responsiveness of the application or website.

Now let’s take a look at what the Chrome Developer Tools profiler tells us about this site, followed by a look at Intel® VTune™ Amplifier for further insight.

The Chrome Dev Tools Profile provides information into what is going on. The letters are functions within the JavaScript code generated by the V8 engine. You can see a significant amount of the work load, but the Chrome profiler only provides information on the V8 engine and a few other functions. The garbage collector is present in every profile, indicating a fair amount of memory is being allocated and released. The Chrome profiler does not show you what other components of Chrome are involved or information on how you can improve your application at the platform level. Furthermore, the function requestAnimationFrame is being used significantly.

HTML animation frames usually involve a timer look, as previously mentioned. Now browsers have an API that can do some of the optimization for you. It is a basic API that works with the DOM, canvas or WebGL.

The use of requestAnimationFrame allows the browser to consider all the animations together in a single reflow and results in a single loop to repaint the screen. This allows the CPU and GPU to go into an extended lower power state, thus saving power as well as potentially allowing you to do more with the saved CPU and GPU cycles if you are CPU or GPU bound.

Now let’s take a look at what additional information Intel® VTune™ Amplifier provides. Due to the graphics focus on this site, we would expect to see heavy use of Chrome’s graphics components. This is exactly what we see. Most of the functions shown deal with SKIA, a graphics component of Chrome, or with memory. I will skip the Outside of any known module function for now and will return to this later in a different example to point out the Clockticks Per Instruction is 2.017. But for now let’s focus on the SKIA and memory functions. SKIA graphics functions play a significant role in this website. Notice the tc_realloc and tc_free functions which could account for the garbage collector we saw in the Chrome Developer Tools Profile. Notice that the CPI (Clocks per Instruction Retired) are below 2 with the exception of two sections the Outside any known module and the _memcpy_ssse3_back. This is due to the Outside any known module section code being generated by the V8 JavaScript engine. The code generated is not as efficient as the natively compiled Chrome code and on an average takes more time per instruction retired than the native code. The memory copy function inside of libc-2.17.so _memcpy_ssse3_back is uses SIMD vector instructions. There are two reasons this code may be less efficient, although it may be very good code. This could be due to the use of a rep prefix instruction to copy memory until the memory locations are properly aligned to take advantage of SIMD. The rep prefix will show up as one instruction although it may be copying significant sizes of memory. Another possibility is that the written SIMD code is not well suited for the sizes of memory being copied in this application. Using Intel® VTune™ Amplifier you can provide an application developer with options for how to improve the code. One option could be to change the balls so that the light shining on the ball is all one image.

WebGL

WebGL is another developer tool that can be used to help generate a great user experience. WebGL is based on a JavaScript API implementation of OpenGL ES 2.0. WebGL API’s can be easily translated to OpenGL API’s to take advantage of hardware accelerated rendering via the graphics driver.

Typically within the application you create a <canvas> element with an id you can reference from JavaScript similar to the following:

    <canvas id="webgl" width="400" height="400">
    Please use a browser that supports "canvas"
    </canvas>

Inside of the JavaScript code you can then access the canvas object using the ID in the main function similar to this:

function main() {
  // Retrieve <canvas> element
  var canvas = document.getElementById('webgl');

After you have access to the object, you can reference the canvas area within your JavaScript and use WebGL API, similar to how you would program in OpenGL ES.

The following is a simple example of how to create a shader program using Google’s webgl-utils.js

var FragShander = getShader(gl, “shader-fs”);
var vertexShader = getShader(gl, “shader-vs”);
 
shaderProgram = gl.createProgram();
gl.attachShader(shaderProgram, vertexShader);
gl.attachShader(shaderProgram, fragmentShader);
gl.linkProgram(shaderProgram);

The code shown above is a simple example using WebGL and generates a green square inside of a black square:

The actual machine code generated looks very similar to that of code generated by the bouncy balls html page shown above. The key difference is that this examples uses WebGL to generate the image whereas Bouncy Balls uses a Canvas and moving objects using the DOM.

Using Intel® VTune™ Amplifier provides a profile of a simple WebGL object:

Notice again that the top function is the Outside of any known module. The other functions listed are almost the same functions as the ones from the bouncy balls website. Notice that there is a new mesa (graphics driver) function called. Also notice that the memory copy is the same function and the skia functions are also significant in this profile.

Determining whether to use canvas and moving DOM objects or WebGL will depend on how dynamic the website or Chrome application is. The key take way from these examples is that the method used for graphics generation can significantly impact the CPU/GPU and power of your application.

Chrome Extension Applications

We covered graphics and profiling applications in both Chrome Developer tools and Intel® VTune™ Amplifier. Next we will look at the difference between a Chromebook and Chrome extension (CRX) and a regular HTML, CSS, and JavaScript page.

Chrome Extensions are basically HTML, CSS and JavaScript applications with special privileges provided by the Chrome Runtime Extensions. At the start of this paper many of the extensions capabilities were listed.

In order to create a Chromebook Chrome Runtime Extension, the first thing you do is create a manifest for the application. A manifest is simply a CRX json file and will be modified as you change your code. A manifest for the application provides Chrome with the information it needs to grant the application specific rights. The following is an example I wrote that gives the application right to the tabs, management, storage and background functionality provided by the Chrome Runtime Extensions.

{
  "name": "rrw",
  "version": "1.1",
  "description": "Simple Manifest.",
  "permissions": [
     "tabs",
     "management",
     "storage",
     "background"
  ],
  "content_scripts": [
    {
      "matches": ["http://*/*"],
      "css": ["rrw.css"],
      "js": ["rrw.js"],
      "run_at": "document_end"
    }
  ],
  "browser_action": {
      "default_title": "rrw",
      "default_icon": "rrw.png",
      "default_popup": "rrw.html"
  },
  "background": {
     "persistent": true,
     "page": "rrw.html"
     },
  "manifest_version": 2,
  "content_security_policy": "script-src 'self' https://ajax.googleapis.com; object-src 'self'"
}

Once you create your manifest you can create your application that can use most of the functionality of HTML, JavaScript and CSS as well as the extra functionality provided by the CRX and privileges requested in the manifest.

Google has made it very simple to install the Chrome Extension for development. In the Chrome Browser go to the tools extensions drop menu and click on the Developer mode checkbox. Then you can click on the Load unpacked extension… button and point to the directory in which your manifest, HTML, CSS and JavaScript files reside. Chrome will load the extension you are want to test.

With the privileges provided in the manifest you can access functionality that would normally not be available from a website. For example, in the image below the CRX extension running reads the tabs that are open as well as the Chrome Extensions loaded. This is done by calling into the JavaScript functions provided by the CRX. The chrome.tabs API provides the open tab information and the chrome.management API provides the extension information.

  chrome.tabs.query({}, function(tabs) {
  table = outputTabsToList(tabs, addTime);
  tabsDiv.innerHTML = table;
  });
 
  chrome.management.getAll(function(info) {
  if (info.length == 0) {
                $('search').style.display = 'none';
                $('appstore_link').style.display = '';
                return;
                }

Then the information in JavaScript can be formatted into an HTML table, in this case a sorted hash table, for display. In the example there were two tabs open, the WebGL example and another tab open to http://www.intel.com. The table below the tabs table shows all of the Chrome Extensions installed on the Chromebook.

Improving Chromebook and the Extension Development Environment

Intel is working to improve the Chromebook and the Chrome runtime extensions development and optimization environment. For example, Intel has integrated Just-In-Time (JIT) profiling into the Chromium browser open source project. The previous profiles shown had identified code generated by the V8 engine within Chrome as “Outside any known module”. The Chromium browser source code has an option flag inside of the src\content\content.gyp file, ‘chromium_enable_vtune_jit_for V8’. The flag is set to 0 by default, but changing the gyp flag to 1 and recompiling Chromium on Linux or Windows will provide integrated JIT support for Intel® VTune™ Amplifier so enable machine code to JavaScript JIT profiling. We are currently investigating with Google to make this change a possible runtime option for Chrome, without requiring a recompile. To make building and testing Chromium with Intel® VTune™ Amplifier JIT more reliable there is a buildbot that developers can check to make sure that the modifications to the V8 engine for JIT profiling are in synch with the modification within the Chrome browser. The buildbot for Linux may be found at http://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20vtunejit

An example of how to use this V8 / Intel® VTune™ Amplifier JIT capability can be seen by running Google V8 benchmark Octane found at http://octane-benchmark.googlecode.com/svn/latest/index.html. Running Octane and profiling the benchmark using the Chrome Developer Tools gives you the following:

You will notice that montReduce is the hottest self-timed section of code within the benchmark. Clicking on the code will take you to the location in the JavaScript code.

You can drill down to the function that is going to be JIT’ed by the V8 engine and you know it is a function you may want to take a look at because of its significance in self-timed code. What you cannot tell as easily is the quality of the code being generated. With the Intel® VTune™ Amplifier profile you can. The hotspots found by a Intel® VTune™ Amplifier profile are essentially the same JavaScript hotspots. Notice in this case with the JIT option turned on in Chromium you actually see the Module [Dynamic code] instead of [Outside any known module].

Both Intel® VTune™ Amplifier profiles and Chrome Developer Tool profiles allow you to drill down to the JavaScript source code but Intel® VTune™ Amplifier will also show you where you are spending time within the function. This is a useful feature for longer functions where it is hard to identify where time is spent.

As you recall in some of the previous Intel® VTune™ Amplifier profiles the CPI of the JITed code was around 2.3. You can see in the case of the montReduction function the CPI is 0.3. This means that the average time spent on an instruction in Octane is about 7.5X less than the time spent on instructions in the example application. Of course this does not mean that this is really as inefficient as pointed out with the rep mov prefixed instruction but it is a good indication of how well the code generation works and if it is ideal for optimizing. In this case the montReduce CPI of 0.3 is fairly good with an average of 3 instructions per clock.

Chromebook and Chrome Runtime Extension in the future

Having applications that run as web or Internet applications with a seamless online and offline experience is essential to having a great user experience. Using the Chrome Runtime extension provides developers with an easier method for implementation. Furthermore, web applications require little to no deployment. As the prominence of web and internet applications and development continues to rise, creating the best user experience is key. Writing and debugging Web applications via the browser console log output is not efficient or effective. This is something we as a community and Intel as a company are working to fix. Intel continues to contribute to the Chrome and Chromebook open source projects, improving tools to help developers debug and optimize code for the CPU and GPU. Intel is also very involved in helping to define computing standards and is working with Google and Mozilla to introduce Single Instruction Multiple Data instructions directly into EcmaScript. Intel has worked with Google and Mozilla to create a prototype of this technology that has already landed in the repositories described in a paper written by Mohammad Haghighat at Intel, https://01.org/blogs/tlcounts/2014/bringing-simd-javascript. With the continued improvements in Internet technologies and tools there is no reason why Chromebooks and Chrome Runtime Extension applications can’t provide an outstanding user experience.

 

1 Today this capability is in beta and is available under NDA. Recompilation of the OS is required for symbolic support of JavaScript. Contact your chrome_developer@intel.com for access to the software and other technical issues.

For more complete information about compiler optimizations, see our Optimization Notice.
Tags: