The MID (Mobile Internet Device) is the latest innovation in the PC family that gives the consumer a low power mobile device with the complete capabilities of a PC. On the basis of MID’s features and applications, many issues should be taken into account to provide users with a great experience on MID’s UI. This whitepaper gives an overview of the methods of UI design for MID. The Hildon user interface is task oriented, so it needs special considerations. This whitepaper also offers the BKMs of Hildon UI design.
1. MID Introduction
The MID is the latest innovation in the PC family that gives the consumer a low power mobile device with complete capabilities of a PC. A MID is small form factor device which allows the consumer to carry it conveniently wherever they go and provides a facility to access digital data and be connected 24x7.
Intel is an active partner in developing and delivering the technology that addresses the needs of a MID consumer. One of the major contributions of Intel in MID revolution is Integrated Graphics Software, through its integrated graphics driver, Intel® Graphics Media Accelerator.
MID has a relatively small LCD (4.5” – 6”) with high resolution, touch screen and reduced keypad. In order to bring a device-like experience to the user, the content must be easy to read and the UI must be easy to operate by finger, which means the UI of the OS and applications needs to be highly customized.
2. Midinux Overview
Linux is an open source OS, which can be highly customized from the system level, such as quick bootup and quick resume from standby, to the application level, such as UI style and theme.
Midinux is the first Linux based OS, which fully supports the Intel 2007 MID platform. It has a GTK-based mobile application framework that is optimized for MID screen size and resolution. Figure 1 shows the MID application framework stack.
Figure 1. MID application framework stack
2.1 Hildon Overview
The Hildon user interface, as well as Maemo as a whole, is based on open source components. Hildon is based on GTK+, the toolkit used in Gnome desktop, with several tweaks to make it more suitable for mobile devices. The first product using the Hildon UI style is the Nokia 770 Internet Tablet. The Hildon user interface is highly driven by touch screen operations, meaning the user interacts with the device by using a stylus, finger or thumb. Even though there are a few mandatory hardware buttons, users are mostly interacting with the device by using the touch screen. And Hildon supports multitasking, meaning that several applications can be running at the same time and the user can switch between them.
There are 4 basic components of Hildon UI: Task Navigator, Title Area, Status indicator Area and Application Area. Task Navigator is a global screen element located on the top of the screen. Task Navigator is used to launch applications and to switch between them. Title area contains the application title which opens application specific menu. The right side of the title area contain s minimize and close buttons. Status indicator area contains icons which indicate system states and behavior. Status bar applets are located here. Application Area can be used by applications in normal mode. In full screen mode all other parts disappear and this application area expands to fill the whole screen.
Figure 2 is the general MID application UI layout.
Figure 2. MID Application UI Layout
3. UI Design Challenges and BKMs for MID
Although PC architecture and full-function PC OS will reduce efforts from developers to port applications from desktop PC to MID, the MID form factor does introduce a number of unique considerations in application UI design. And Hildon application framework is a device-like application framework, which also adds some special considerations in application UI design.
3.1.Screen Size Consideration
Because the MID screen is much smaller than a traditional screen, the size of graphical elements must be handled with some care. Developers must avoid making scaled-down graphical elements too small, as well as allowing elements that are left the same physical size to consume too much screen space. Specifically, text and icons are often hard to see clearly, some buttons or units are difficult to click reliably, and some game windows do not fit entirely onto the screen.
3.1.1.Text and Icon Sizes
Issue: In the interfaces of applications that are ported to the MID, text or icons may shrink to a size where it becomes difficult to see them clearly. Text that appears reasonably sized on a 15” screen can easily become too small when shrunk to a five- to seven-inch screen that you might find on a MID. Aside from the actual font size of text, chat and other text windows may become too small. Accommodating smaller window sizes by decreasing the font size can make the text difficult to read.
Figure 3. Proper text size for reading(a) and unreadable text without suitable scale(b).
Best Practice: Ideally, applications designed with the MID in mind should use text sparingly and consider the MID screen size when choosing a font. Likewise, icons should not rely heavily on fine detail s, so different objects are different enough to be easily distinguished from one another, even on the smaller MID screen. In places where increasing text size enough would not compromise other elements of the application, allowing text size to be adjustable may be a good solution, allowing individual players to decide for themselves the ideal size for the text.
3.1.2.Clickability of Buttons and Other Elements
Issue: Similar to the issue of small text, buttons and other clickable elements add complexity to porting software to the MID. That is, while the overall interface on the MID must be smaller than the corresponding standard PC version, the actual buttons in the MID interface must be larger, in order to accommodate being accurately clicked by a stylus or finger, rather than the more precise mouse that is used in the standard PC version. Even when it is possible to select the correct element with some care, this issue can significantly detract from the user experience.
Figure 4. Clickability of Buttons. On a 20-inch PC monitor (left), the user can easily identify and select a domino using the arrow cursor. When the domino images are scaled to fit on a five-inch UMPC screen (right), however, it becomes very difficult for users to select individual dominoes, or even to identify how many dots are on each domino.
Best Practice: As with the size of text and icons, developers should avoid this problem by using large, distinct buttons and other elements. These elements should also have enough space between them, making it less likely that the user will inadvertently select the wrong one. In those cases where a text label appears adjacent to a button or other element, that label should be part of the clickable area associated with the element, making it easier to click without having to devote any additional space.
Issue: If application windows use a fixed size, rather than automatically resizing themselves to fill the full MID screen, parts of them may not be visible all at once, and, in some cases, certain parts may not be reachable at all. This situation has made it more complex to use the 800x480 or 1024x600 resolution enabled by the MID's wide screen, since those aspect ratios must be matched to the standard 4:3 PC aspect ratio.
Figure 5. Visibility of Window. On a standard PC (left), the user is able to see the entire play area, including informational parts of the player interface. If the game window is cropped to fit the UMPC’s smaller size and different aspect ratio (center), part of the game will be hidden, represented here by the translucent parts of the screen at the edges. If the game window is scaled to fit the UMPC screen (right), all parts of the game can readily be made visible, with a level of visual distortion that may be acceptable.
Best Practice: The key to resolving this issue is to consider the 800x480 and 1024x600 resolutions as they develop UI for the MID, scaling the entire window to fit on the screen or rearranging the interface to take full a dvantage of the wide screen area. For example, providing scrollbars and allowing the window to be resized (manually or automatically) may be sufficient in some cases to allow the user access to all parts of the screen and to provide an acceptable user experience.
3.2.Touch Screen Consideration
The use of a touch screen instead of a mouse on the MID adds another layer of complexity to porting applications. For most software purposes, the touch screen behaves like a mouse, except that instead of registering a linear track of movement as the user drags the cursor, it generates a series of button-down events with a location each time the user touches the screen. The inability to move the cursor without clicking and the relationship between left-clicking and right-clicking also causes issues.
3.2.1.Accurate Interpretation of Tap-Only Touch Screen Input
Issue: MID applications must be able to interpret cursor input provided by the user as a series of points (corresponding to a series of touch-screen taps), rather than requiring a linear pattern of movement (like that provided by mouse movement controlling the cursor). This issue tends to arise particularly often, for example, in games that shift the user's perspective so that the cursor is always at the center of the screen (which is common in first-person shooters). In such a scenario, when the player touches the screen, the cursor may jump around the screen erratically, instead of moving to where the user touched.
Best Practice: If the game tracks movement of the cursor from point A to point B, it must be able to interpret a click at point A, followed by a click at point B, without the cursor first moving between point A and point B. Requiring an external mouse to be plugged into the MID will typically correct this problem as well, although that solution is clearly sub-optimal from a user-experience point of view.
3.2.2.Accurate Touch-Screen Mapping
Issue (Window Resolution Smaller than Physical Screen): When the application believes it is operating in full screen mode at a lower resolution than the physical screen, the UI display may appear centered with space on either side (as in the case of a 640x480 window being centered on a 800x480 screen or an 800x600 window being centered on a 1024x600 screen, with black bars on each side), mismapping may occur between the two, with the entire touch-screen surface being mapped to the relatively small display area. That mismapping results in inappropriate response of the interface to user interaction, such as a button's clickable screen area being located away from the visual representation of the button.
Figure 6. If the game window is centered with black bars on each side, the game logic that interprets screen taps must avoid mapping the entire physical screen area to the smaller game window. Such incorrect mapping can cause a screen tap at one position (represented by the yellow arrow) to be interpreted by the game as occurring at a different position (represented by the red burst).
Best Practice: Applications can be centered with black bars at the sides without causing the touch screen to be mismapped, as long as the mapping logic considers the black bars as part of the screen. With this adjustment, touches on the screen map to exactly the location touched.
Issue (Window Resolution Larger than Screen): In those cases where the window takes up more than the full vertical space of the screen, the touch screen may be incorrectly mapped vertically instead of horizontally. This scenario typically causes the touch screen to be mapped to the whole game window, rather than just to the visible section. Again, clicks do not correspond to the area of the game window that the user intends.
Figure 7. Having part of the game window (represented by the translucent area) cropped from the physical screen may also result in a screen tap at one position (represented by the yellow arrow) being interpreted by the game as occurring at a different position (represented by the red burst), due to mismapping between the different shapes of the game window and the physical display.
Best Practice: Developers should ensure that the operating system maps the touch screen exactly to the entire visible portion of the screen (including portions not being used by the game), and not including portions of the game window that are not visible. Another solution is for developers to natively support the 800x480 and 1024x600 screen resolutions as a user-configurable option. In some cases, it might be possible to automatically stretch the window to fit the full screen, if the resulting dimensional distortion of elements is acceptable.
3.2.3.Alternatives to Hover-Over Effects
Issue: Many games use a hover-over feature that enables the user to position the mouse over an object or location in the game (without clicking on it) to trigger an informative animation or tooltip. This feature, which is often used to provide instructions for novice players, is incompatible with touch screens, since a touch screen cannot move the cursor without clicking.
Figure 8. Hover-Over Effects.
Best Practice: Since the hover-over feature is not compatible with current MID hardware, developers should choose alternative events to trigger the animation or tooltip. For example, it could be triggered when the player reaches the part of game play when the subject of the animation or tooltip becomes relevant. Another possibility is to design the interface with a means of temporarily interpreting click events as mouse over events, such as a button that can be held down to allow touches on the touch screen to move the cursor without registering a click.
3.2.4.Accommodating Right-Click Functionality
Issue: On a typical MID touch-screen, users perform right-clicks by holding down the click on the screen for a longer time than for a left-click, which can cause users to inadvertently left-click when they mean to right-click. Moreover, it is impossible to perform a left-click and right-click simultaneously.
Best Practice: Developers should provide alternative user interactions to replace right-cl ick functionality, such as using double-clicks or clickable controls on the screen that take place of right-clicking. Alternatively, game developers can simply require the use of an external mouse or other pointing device, although that requirement may detract from the user experience.
3.3.Form Factor Considerations
A number of issues relate directly to hardware differences in the MID form factor, relative to standard PCs. For example, unlike a PC, a MID may have a touch screen, joystick, user-programmable buttons, and dedicated buttons such as a menu button, but it will typically not have a keyboard or CD-ROM drive. Although it may be possible to add a keyboard, CD-ROM drive, or other peripheral devices, doing so reduces the portability of the MID. Moreover, different models of MIDs may have different elements included in their designs, which causes the limitations associated with the form factor to vary by device. That variability typically requires developers to focus on the lowest common denominator of hardware features when designing applications, in terms of core requirements.
3.3.1.Limiting the Need for Diverse Command Inputs
Issue: The MID form factor provides for a limited number of inputs, relative to the full keyboard and mouse available from a standard PC. Even those games that are designed to be run on a PC using a joystick may require the use of multiple buttons that are included on a typical joystick controller, which may not be available for the MID. MIDs also have issues with games that incorporate keyboard shortcuts as a key game play component. Notably, games that are primarily mouse-driven are not affected by form-factor limitations in this area.
Best Practice: Developers can address limited input options on the MID by decreasing the number of commands that are actually required to play the game, as opposed to being optional conveniences such as shortcuts for menu options. A somewhat more flexible solution, however, is to create buttons on the screen, particularly if the buttons are context-sensitive, appearing only when they are relevant. This can provide an open-ended solution for input requirements.
3.3.2.Addressing Keyboard Requirements
Issue: In addition to issuing commands, many games also use the keyboard for input such as naming characters, creating profiles, saving games, or supporting a chat feature for online games. For example, it is common for games to require a keyboard to enter a profile name at the beginning of a gaming session, but not to require the use of a keyboard at any other time during the session. Some games also require players to use the keyboard to name their saved games. In many cases the on-screen keyboard will not run on top of the game interface, so it cannot resolve these issues.
Best Practice: Developers should either design games explicitly not to conflict with the onscreen keyboard (e.g., by not defaulting to full-screen mode) or provide an on-screen keyboard within the interface itself that appears only when needed. One simple means of minimizing the need for keyboard input is to provide default values for text strings such as profile names, saved game names, etc. and to allow them to be selected using screen-tap input. Additional values for these text strings can a lso be provided by adding an interface button that randomly selects a new value from a list of character names created by the developer. Saved games can also be identified using a screen capture of where the player left off, together with a date and time stamp. Since chat features are very rarely a core aspect of game play, they can generally be disabled on the MID without negatively impacting the user experience.
Figure 9. An onscreen keyboard inside a full-screen mode.
4. BKMs of Hildon UI design
The Hildon user interface is task oriented, meaning that it is designed to help the users do common tasks as actions by using a focused and optimized set of applications and features. Examples of this kind of task are opening bookmarked browser sites by only 2 taps and sending new email with only 2 taps.
There are several differences between a PC's mouse and keyboard -based interaction and the Hildon interaction model that must be taken into account when designing application user interfaces. While on the move it’s more difficult to tap precisely on and interact with small on-screen controls. Users may also cover areas of the screen, UI controls or displayed content. Optimize your UI design so that only a few taps are needed to accomplish a task, avoiding unnecessary detours. Task flows in your UI should follow standard flows like from the left to the right or from the top to the bottom, instead of forcing the user to go back and forth.
Menu commands should be structured according to their priority and functional relationship. If you have a lot of commands, try to group functionally related commands to submenus and use only submenu headers at the highest menu level, like 'File'->'New' and 'File'->'Save'.
In the Hildon UI the menu layout is vertical as opposed to the horizontal layout of menus in PC environment. Most commonly used commands or submenus should be placed higher in the menu and less important on the bottom so that users will reach the frequently used features easily. For grouping the related menu items within each menu, separators shou.d be used. Avoid using more than seven commands or submenu headers vertically or three menus horizontally.
Commands which are not currently usable should be dimmed. One unique feature in Hildon is that applications can also react to selection of dimmed menu item selections, usually by showing an information banner which explains why the feature is not available, e.g., “Nothing to undo.” This way user can be guided to use the application correctly.
For the first application menu title, use a term that is related to the main idea of the application, for example ‘Document’, ‘Image’ or ‘Card,’ instead of ‘File.’
Figure 10. Menu on Midinux.
The toolbar should provide the user easy access to most-used and most important functions in your application. Here are some rules for designing and using toolbars:
- The number of items that can be placed on a toolbar depends on the width of each item and whether the Task Navigator is shown or not (switching from normal screen to full screen).
- If your application is using several views, then consider changing and optimizing the displayed functions on the toolbar for each view.
- The most common icons for the basic functionalities should be provided.
- The metaphors of toolbar icons should be understandable, intuitive, and closely related to the desired action. Icons should preferably be taken from everyday situations so that users can associate tasks on the device to those that are familiar to them from other situations.
- All displayed commands should be available in the application menu
- Because of limited screen space, there should be a choice to hide a toolbar from the menu.
- Separator line should be used for grouping icons when necessary.
- Avoid using text which needs to be localized on the toolbar as different lengths may mess up the toolbar layout. Icons should be used instead of text whenever possible to save the toolbar space.
Figure 11. Toolbar on Midinux. (browser, player, E-mail, GPS, Office tool, Phone Book…)
Dialogs or longer dialog paths can be used to guide the user to finish certain tasks. Do not constitute long chains of dialogs and keep tasks simple. Sometimes it might be a good idea to use a new window for more complex tasks. If the task needs information from other applications and the information is not provided as a pre-set list, it is more advisable to implement the task with a window than a modal dialog. Dialogs are displayed centered over the application area.
Figure 12. Dialog in the office tool.
Notifications are used to provide feedback for the user. There are two types of notifications: notes and banners.
Notes are displayed in the center of the screen and usually contain a small amount of information with buttons to react. Applications are stopped until the user acknowledges the note dialog.
Banners do not require action from the user, and don't take focus. They are displayed in the top-right corner of the application area.
Figure 3 shows the difference between Notes and Banners.
Figure 13. Note (Left) and Banner (Right)
Some main issues are put forward on the design of the UI for MID and methods of UI design are given in terms of screen size, touch screen and form factor. Since Hildon user interface is task oriented, it needs special considerations. The whitepaper also offers the BKMs of Hildon UI design.
 Best Practices in Game Design for the Ultra-Mobile PC by Matt Gillespie, Michael Finkel and Victoria Bailey
 Hildon User Interface Style Guide Summary Version 1.1