By Chris Walley, Director, Escapist Games Ltd.
Escapist Games Limited recently collaborated with Intel to enhance their popular Star Chart application to take advantage of the latest Intel® chipsets and devices. This included tailoring Star Chart for the 3rd generation Intel® Core™ processor on Ultrabook™ devices and the Intel® Atom™ processor on tablet devices. During development, Escapist Games was able to give valuable feedback to Intel on the efficacy of the sensors and hardware in a real-world software use scenario.
This case study examines various issues that were overcome as a result of this collaboration, as well as the highlights of the engagement.
Released in 2009, initially for the smartphone market, Star Chart is phenomenally popular, available on 7 operating systems and downloaded onto over 7 million devicesi, proving that the stars continue to fascinate and inspire. Star Chart’s core experience of pointing a smart device at the sky and using internal sensors to track movements and orientation—providing a compelling virtual window into the night sky—has given users a glimpse into the future of augmented reality. Points of light in the night become labeled and quantified, allowing everyone with the technology access to a plethora of real-time information, previously the exclusive domain of professional astronomers.
To develop this enhanced version of Star Chart, Escapist Games decided to concentrate on three main areas:
- Content - expansion of content by taking advantage of the fast loading times and large storage space of Intel® architecture-based devices.
- Graphics - enhancing the real-time graphics by making the most of the powerful Intel GPUs and latest version of DirectX*.
- UX - tailoring the user experience to meet a set of predefined and known minimum device specifications, while ensuring that the additional content did not equate to additional UX complexity.
Conversely, during development Escapist Games was able to provide valuable feedback and insight to Intel on the usability of their hardware, particularly with regard to sensor functionality (see the section: User Interface Design and Sensors).
Approximately five thousand stars are visible in the night sky to the unaided eye. Until recently, Star Chart shipped with a database of ~125 thousand stars, taken from the HIPPARCOS star catalogii. With the zoom feature in Star Chart allowing users to dynamically narrow the field of view so that even this number of stars quickly thins out, it was decided to radically expand the star database. The PPM-Extended star catalogiii of approximately 18 million stars was selected for this purpose. This data, pre-processed into a proprietary format, is streamed in from storage on-demand as the user interacts with the software. Performance is achieved by using a binary space partitioning algorithmiv to cull stars that are outside the camera frustum, ensuring the software remains performant regardless of zoom level.
A much-requested feature by the Star Chart community has been the ability to view satellites as they orbit the Earth. Escapist Games decided to take this opportunity to add satellites, including the International Space Station(ISS)v and the Hubble Space Telescopevi, providing additional depth and content to the application. Star Chart not only visualizes the orbits of these satellites, but for the ISS and Hubble a detailed 3D mesh, along with materials, was authored at a resolution suitable for real-time display. Additionally, the history of the ISS was captured, showing the various stages of its in-situ construction.
For the Hubble Space Telescope, Star Chart parses an RSS feed managed by the European Space Agency. This enables images of Hubble’s latest deep-sky photography to be retrieved and embedded alongside the rest of the satellite’s information.
An interesting point to note, is that by adding satellite meshes that users can travel to and explore, it became apparent that the current Earth visuals, now able to be seen in much closer detail, also needed improving. This was done in various ways, as described in the next section.
Final planet materials were meticulously crafted using a variety of sources, predominantly from NASA’s huge public archive. DEM datavii was converted into normal maps. Flow maps were hand authored using the freely available Flowmap Painterviii software.
Furthermore, constellation imagery and the full set of Messier object textures were enhanced using Photoshop*. Every single star was removed using a combination of filters and good old fashioned hard work, ready to be put back into the sky and overlaid with our twinkling real-time and data-driven star field (Figure 3).
The graphical showcase continues as we explore the planets of our solar system. Upon entering the ring systems of Saturn and Uranus, the user is greeted with thousands of dynamic, tumbling rocks ( Figure 4), rendered on the GPUix using geometry shaders and incorporating fractal noise, ambient occlusion, and subsurface scattering effects. Volumetric planet atmospheres are ray-traced, again solely on the GPU using optimized pixel shaders.
Finally, the Earth was significantly improved in various ways. Aurora were added at the poles, dynamically modeled and animated. Gently pulsing city lights and global lightning storms augment the night-side. All this, combined with HDR effectsx, anti-aliasing, and significantly increased texture and geometry resolutions make Star Chart’s Earth aesthetically very appealing.
User Interface Design and Sensors
An overriding concern when designing a user interface experience is ensuring that the application is responsive to user input. With this is mind, a minimum goal of 30 fpsxi was set. Optimizations, both from a high-level design point, through to optimization of assets, and all the way down to saving CPU/GPU cycles by tweaking code logic were employed. This meant that the user experience was always a smooth and fluid one.
One of Star Chart’s most exciting features is its Augmented Reality (or AR) mode. Point the device to the sky and the display orients the camera to that part of the sky, creating a virtual window into the universe; it’s an ideal way of intuitively interacting with the application.
To get this working correctly requires various data points, listed here in inverse order of reliability:
- A 3D vector from the magnetometer sensor (aka digital compass) pointing north.
- A 3D vector from the accelerometer sensor that provides, when the device is not accelerating, a fairly constant vector pointing in the direction of the pull of gravity, i.e., ‘down’.
- The correct latitude and longitude derived either from GPS satellites or correlated from Wi-Fi* access points.
- The correct date/time + timezone offset, from which UTC-0 can be obtained.
Combining all these allows Star Chart to accurately display the sky in the direction you are pointing your device. However, if any of these data points are inaccurate, Star Chart is unable to reliably achieve AR mode, in which case what you see on Star Chart’s display will not match what you see in reality.
The following table illustrates each data source required by AR Mode along with its most likely error cases:
MagnetometerElectro-magnetic interferenceNearby electronic equipment, or electric cabling / power lines. Poorly shielded sensors.AccelerometerAcceleration in addition to the constant pull of gravityDevice motion. Poorly shielded sensors.GPS sensor / Wi-Fi*Incorrect latitude/longitude coordinatesNo connection to GPS satellites or Wi-Fi. Coordinates manually entered incorrectly.System clockSystem time not set correctly. Timezone not set correctly.A new system incorrectly set up. No Wi-Fi connection.
|Data source||Error||Most likely cause(s) of error|
|Magnetometer||Electro-magnetic interference||Nearby electronic equipment, or electric cabling / power lines. Poorly shielded sensors.|
|Accelerometer||Acceleration in addition to the constant pull of gravity||Device motion. Poorly shielded sensors.|
|GPS sensor / Wi-Fi*||Incorrect latitude/longitude coordinates||No connection to GPS satellites or Wi-Fi. Coordinates manually entered incorrectly.|
|System clock||System time not set correctly. Timezone not set correctly.||A new system incorrectly set up. No Wi-Fi connection.|
The Intel Ultrabook test devices were found to give generally very reliable sensor readings across the board. ‘Jitter’ caused by poorly shielded or substandard electronics was virtually non-existent. This contrasts sharply with hardware such as the Microsoft Surface* RT, in which reliable magnetometer readings proved, at best, problematic to obtain. A device specification sheet may tick all the boxes when it comes to sensor capabilities, but if costs have been saved by sacrificing shielding or the quality of the sensor electronics, it will certainly affect the end-user experience and consumers will notice.
Star Chart’s Augmented Reality Mode relies on being able to ascertain the orientation of a device’s screen. In tablets and smartphones, which consist of one solid block, this is not a problem. However, laptops like the Ultrabook consist of a screen that hinges out from the main body and exact placement of the sensor board becomes important. It was found that the sensors, being located in the main body, had no relation to the orientation of the screen. Therefore due to this disconnect, Star Chart could determine the orientation of the devices body, but not of the screen, and this is a very important distinction! After feedback from Escapist Games, Intel issued updated guidelines regarding ideal sensor placement, and subsequent devices such as the Lenovo Yoga were able to implement these improvements. In the Yoga, the sensor board is located behind the devices screen and the problem was solved.
The Augmented Reality mode in Star Chart is a good example of intuitive and invisible UX design. The input experience is simplified to such a point the user barely has to think about how to interact with the application. Taking this concept further still, the developers implemented Automatic Night-mode on devices that contain an ambient light sensor. As light levels drop below a certain threshold, Star Chart automatically switches to ‘Night Mode’, where the interface becomes entirely red-shaded in order to preserve the users night vision. This is an extremely useful feature when using Star Chart outside looking up at the real stars.
For the final and most radical extension to the input system, the developers implemented what was, again, driven by a desire to simplify the way users interact with the application: voice control. Star Chart was programmed to respond to a multitude of aural commands, such as “Fly to Mars” or “Look at Orion.” For Android devices the Google Speech APIxii was adopted. For the Windows 8 version, the Microsoft Speech APIxiii proved up to the task.
In developing the Intel enhanced version of Star Chart, Escapist Games crafted a piece of software that very effectively showcases the power and capabilities of a range of Intel devices. Star Chart for the latest Intel Ultrabooks is the most aesthetically and technically superb version of the software available.
A wise man once said “Space is bigxiv.” Not only does there remain a vast variety of exciting content and visual splendor to add to Star Chart, but new technology allows users to interact and experience this like never before. To ensure the continued success of Star Chart and to maintain their place at the forefront of educational and entertainment software, Escapist Games aims to further enhance and expand Star Chart indefinitely.
About Escapist Games
Founded in early 2009, Escapist Games mission is to create the highest quality games and game-like applications for the latest technology, while taking advantage of direct-to-consumer digital marketplaces.
With a host of highly rated products under their belt, the most successful being the worldwide, best-selling, and highly acclaimed Star Chart, Escapist Games now finds itself at the forefront of the digital marketplace revolution, working closely with such distinguished companies as Intel, Samsung, Google, and Microsoft.
“We make the software that sells the hardware.”
i Current as of 16-September 2013 and collated from the following primary app stores:
ix Feature only possible on 3rd generation Intel Core processor and later.
xi 30 fps minimum average across all devices. However, on Ultrabook, Star Chart runs at a consistent 60 fps.
xiv Adams, Douglas (1979). “A Hitchhikers Guide To The Galaxy.” Random House Publishing Group, 2007 ISBN 0307417131, 9780307417138