Krita* Gemini* - Twice as Nice on a 2 in 1

Download PDF

Why 2 in 1?

A 2 in 1 computer (2 in 1) is a laptop computer that can be transformed into a tablet by manipulating the body of computer. Laptop mode (also referred to as desktop mode) allows a keyboard and mouse to be used as the primary input devices. Tablet mode relies on a touch screen, thus requiring finger or stylus interaction. A 2 in 1, such as the Intel® Ultrabook™, offers precision and control with multiple input options that allow you to type when you need to work and touch when you want to play.

Developers must consider multiple scenarios in modifying their applications to take advantage of and conform to this new type of transformable computer. Some applications may keep their existing menus and appearance nearly identical in both modes. While others, like Krita Gemini for Windows* 8 [11], may carefully select what is highlighted and make it available differently in each user interface mode. Krita is a program for sketching and painting that offers an end-to-end solution for creating digital painting files from scratch [12]. This article explains how the Krita developers added 2 in 1 mode awareness—including implementation of both automatic and user-selected mode switching—and some of the areas developers should be considering when creating applications for the 2 in 1 user experience.

Introduction

Over the years, computers have used a variety of input methods from punch cards to command lines to point-and-click with a mouse or other devices. With the adoption of touch screens, we can now point-and-click with a mouse, stylus, or fingers. Most of us are not yet ready to perform every interaction with touch and, with mode-aware applications like Krita Gemini, we don’t have to. The new 2 in 1s, like an Intel® Ultrabook™, can deliver the user interface mode that offers the best UI experience possible, on one device.

There are multiple ways that a 2 in 1 computer can transform between laptop and tablet modes (Figure 1 and Figure 2). There are many more examples of 2 in 1 computers on the Intel website [2]. The Ultrabook computer can transform into tablet-mode from laptop-mode by detaching the screen from the keyboard or using another means (such as folding the screen on top of the keyboard) to disable the keyboard and make the screen the primary input device (such as folding the screen on top of the keyboard). Computer manufacturers are now starting to provide this hardware transition information to the operating system. For example, the Windows* 8 API event WM_SETTINGCHANGE and the “ConvertibleSlateMode” text parameter signal the automatic laptop-to-tablet and tablet-to-laptop mode changes. It is also helpful for developers to include a manual mode change button for users’ convenience as well.

Just as there are multiple ways that the 2 in 1 can transform between laptop and tablet modes, software can be designed in different ways to respond to the transformation. In some cases it may be desirable to keep the UI as close to the laptop mode as possible, while in other cases you may want to make more significant changes to the UI. For example, Intel helped KO GmBH combine the functionality of their Krita Touch application with their popular Krita open source painting program (laptop application) in the new Krita Gemini application. The Krita project is an active development community, welcoming new ideas and maintaining quality support. The team added the mechanisms required to provide seamless transition from the laptop “mouse and keyboard” mode to the touch interface for tablet mode. See Krita Gemini’s user interface (UI) transformations in action in the short video in Figure 3.


Figure 3: Video - Krita Gemini UI Change – click icon to run

Intel has worked with many vendors to help them add 2 in 1 awareness to their applications. The Intel Developer Zone provides significant resources to help developers build applications, including 2 in 1, touch sensors, graphics, and more. Please refer to the Related Links at the end of this article.

Create in Tablet Mode, Refine in Laptop Mode

The Gemini team set out to maximize the user experience in the two modes of operation. In Figure 4 and Figure 5 you can see that the UI changes from one mode to the other are many and dramatic. This allows the user to move seamlessly from drawing “in the field” in tablet mode to touch-up and finer detail work when in laptop mode.


Figure 4: Krita Gemini tablet user interface


Figure 5: Krita Gemini laptop user interface

There are three main steps to making an application transformable between the two modes of operation.

Step 1: The application must be ”touch aware.” We were fortunate that the touch aware movement was started well in advance of the current 2 in 1 activity, as that would have been a much more challenging transition that the simple to-and-from tablet mode. Intel has published articles on adding touch input to a Windows 8 application [9].

Step 2: Add 2 in 1 awareness. The first part of the video (Figure 3) demonstrates the automatic sensor activated mode change—a rotation in this case (Figure 6)—followed by the user-initiated transition via a button in the application (Figure 7).


Figure 6: Sensor-state activated 2 in 1 mode transition


Figure 7: Switch to Sketch transition button – user initiated action for laptop to tablet mode

Support for automatic transitions requires the sensor state to be defined, monitored, and appropriate actions to be taken once the state is known. In addition, a user initiated mode transition [Figure 3] should be included as a courtesy to the user should they wish to be in the tablet mode when the code defaults to laptop mode. You can reference the Intel article “How to Write a 2 in 1 Aware Application” for an example approach to adding the sensor-based transition [8]. Krita’s code for managing the transitions from one mode to the other can be found in their source code by searching for “SlateMode” [13]. Krita is released under a GNU Public License. Please refer to source code repository for the latest information [14].

// Snip from Gemini - Define 2-in1 mode hardware states:

#ifdef Q_OS_WIN
#include <shellapi.h>
#define SM_CONVERTIBLESLATEMODE 0x2003
#define SM_SYSTEMDOCKED 0x2004
#endif

Not all touch-enabled computers offer the automatic transition, so we suggest you do as the Krita Gemini team did and include a button in your application to allow the user to manually initiate the transition from one mode to the other. Gemini’s button is shown in Figure 7. The button-initiated UI transition performs the same functions as the mechanical-sensor-initiated transition. The screen information and default input device will change from tablet mode touch input with large icons to laptop mode with smaller icons and input from keyboard and mouse. However, since the sensor path is not there, the button method must perform the screen, icon, and default input device changes without the sensor-state information. Therefore, developers should provide a path for the user to change from one mode to the other with either touch or mouse regardless of the state of the button-initiated UI state in case the user chooses an inappropriate mode.

The button definition Kaction() - as well as its states and actions - are shown in the code below [13]:

// Snip from Gemini - Define 2-in1 Mode Transition Button:

         toDesktop = new KAction(q);
         toDesktop->setEnabled(false);
         toDesktop->setText(tr("Switch to Desktop"));
SIGNAL(triggered(Qt::MouseButtons,Qt::KeyboardModifiers)), q, SLOT(switchDesktopForced()));
         connect(toDesktop,
SIGNAL(triggered(Qt::MouseButtons,Qt::KeyboardModifiers)), q, SLOT(switchToDesktop()));
sketchView->engine()->rootContext()->setContextProperty("switchToDesktop
sketchView->Action", toDesktop);

Engineers then took on the task of handling the events triggered by the button. Checking the last known state of the system first, since the code cannot assume it is on a 2 in 1 system, and then changing the mode [13]:

// Snip from Gemini - Perform 2-in1 Mode Transition via Button:

#ifdef Q_OS_WIN
bool MainWindow::winEvent( MSG * message, long * result ) {
     if (message && message->message == WM_SETTINGCHANGE && message->lParam)
     {
         if (wcscmp(TEXT("ConvertibleSlateMode"), (TCHAR *) message->lParam) == 0)
             d->notifySlateModeChange();
         else if (wcscmp(TEXT("SystemDockMode"), (TCHAR *) 
message->lParam) == 0)
             d->notifyDockingModeChange();
         *result = 0;
         return true;
     }
     return false;
}
#endif

void MainWindow::Private::notifySlateModeChange()
{
#ifdef Q_OS_WIN
     bool bSlateMode = (GetSystemMetrics(SM_CONVERTIBLESLATEMODE) == 0);

     if (slateMode != bSlateMode)
     {
         slateMode = bSlateMode;
         emit q->slateModeChanged();
         if (forceSketch || (slateMode && !forceDesktop))
         {
             if (!toSketch || (toSketch && toSketch->isEnabled()))
                 q->switchToSketch();
         }
         else
         {
                 q->switchToDesktop();
         }
         //qDebug() << "Slate mode is now" << slateMode;
     }
#endif
}

void MainWindow::Private::notifyDockingModeChange()
{
#ifdef Q_OS_WIN
     bool bDocked = (GetSystemMetrics(SM_SYSTEMDOCKED) != 0);

     if (docked != bDocked)
     {
         docked = bDocked;
         //qDebug() << "Docking mode is now" << docked;
     }
#endif
}

Step 3: Fix issues discovered during testing. While using the palette is fairly easy in either touch or mouse mode, the workspace itself needs to hold focus and zoom consistent with the user’s expectations. Therefore, making everything bigger was not an option. Controls have been enlarged for touch interaction in tablet mode, but the screen image itself needed to be managed at a different level to maintain an expected user experience. Notice in the video (Figure 3) that the image in the edit pane stays the same size on the screen from one mode to the other. This was an area that required creative solutions from the developers to reserve screen real estate to hold the image consistent. Another issue was that an initial effort had both UIs running concurrently, which adversely affected performance due to both UIs sharing the same graphics resources. Adjustments were made in both UIs to keep the allotted resource requirements as distinct as possible and prioritize the active UI wherever possible.

Summary

As you can see, adding 2 in 1 mode awareness to your application is a fairly straightforward process as long as you consider how your users will interact with your application when in one interactive mode versus the other. Read the Intel article “Write Transformational Applications for 2 in 1 Devices Based on Ultrabook™ Designs“ for more information on creating an application with a transforming user interface [10]. For Krita Gemini, the decision was made to make creating drawings and art simple while in tablet mode and add the finishing touches to those creations while in the laptop mode. As you are developing your 2 in 1 modes, consider what you wish to highlight in your application when presenting it to users in tablet mode versus laptop mode.

Related Links:

  1. Intel.com: Introducing the Intel Developer Zone
  2. Intel.com: 2 in 1 Information
  3. Intel.com: Touch Developer Guide for Ultra Mobile Devices
  4. Intel.com: Developer's Guide for Intel® Processor Graphics for 4th Generation Intel® Core™ Processor
  5. Intel.com: Ultrabook and Tablet Windows* 8 Sensors Development Guide
  6. Intel® Article: Ideum GamePlay: Touch Controls for Your Favorite Games
  7. Intel® Article: Designing for Ultrabook Devices and Touch-enabled Desktop Applications
  8. Intel® Article: How to Write a 2 in 1 Aware Application by Stephan Rogers
  9. Intel® Article: Mixing Stylus and Touch Input on Windows* 8 by Meghana Rao
  10. Intel® Developer Forum 2013 Presentation (PDF): Write Transformational Applications for 2 in 1 Devices Based on Ultrabook™ Designs by Meghana Rao
  11. Krita Gemini*: General Information
  12. Krita Gemini: Executable download (scroll to Krita Gemini link)
  13. Krita Gemini Mode Transition: Source Code Download
  14. KO GmbH Krita Gemini: Source Code and License Repository

Related Intel Articles:

Ultrabook Device and Tablet Windows Touch Developer Guide
All-in-One PC: What are the Developer Possibilities?
Windows 8* Store vs Desktop App Development

Additional Intel Resources:

Intel® Developer Zone 
Intel® Graphics Performance Analyzers  
Developing Power-Efficient Apps for Ultrabook™ Devices 
Ultrabook™ App Lab 
Windows* 8.1 Preview – What’s New for Developers
Ultrabook™ and Tablet Windows* 8 Sensors Development Guide

About the Author

Tim Duncan is an Intel engineer described by friends as “Mr. Gidget-Gadget,” with a more recent description as an ”Evangineer” for Intel. Currently helping developers integrate technology into solutions, Tim has decades of industry experience, from chip manufacturing to systems integration. Visit Tim’s blog at the Intel® Developer Zone at Tim Duncan (Intel), contact him via email at timothy.c.duncan@intel.com,or tweet @tcduncan.

Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.
Copyright © 2013-2014 Intel Corporation. All rights reserved.
*Other names and brands may be claimed as the property of others.

 

Reportez-vous à notre Notice d'optimisation pour plus d'informations sur les choix et l'optimisation des performances dans les produits logiciels Intel.