Intel Visual Computing Institute, Saarbrücken
MPI Informatik, Saarbrücken
Télécom ParisTech (CNRS), Paris
The limited spatial resolution of current displays makes the depiction of very fine spatial details difficult. For example, there is not yet a display that is able to render human hair at the right scale, simply because spatial resolution is to low, i.e., pixels are too big. High–quality digital images, in particular panoramic images that arise from combinations of many individual images, also have to be down-sampled to fit on a display, resulting in a noticeable loss of fine detail.
We have developed a system that conveys the impression of enhanced spatial resolution on a conventional display for an important class of animated images. The system works best for static images moving across the display in a simple camera “pan” as found in slide shows, or for simple animations with simple motion.
Application: Hair Rendering
Using our approach, individual hairs appear finer on a monitor. In fact, they appear finer than a pixel itself. This is another step towards the photorealistic depiction of human characters, where it is of high importance to remove small, but persistent errors that compromise naturalness. Without doing so, photo-realistic depiction of humans quickly is perceived as worse than most stylized depictions (the so called “uncanny valley”).
Application: Image Slide Shows
Simply sliding an image in slow motion across the screen can result in a sharper, more natural and more pleasant experience then looking at a static image. This finding – the so-called “Ken Burns effect”, is named after Ken Burns, a director, who first used sliding historic photographs in his award-winning documentations (“The Civil War”). Using our approach, we can improve upon any other known image filtering and reproduction technique, resulting in sharper and even more natural image for this type of motion. Remarkably, the speed at which the Ken Burns effect works best is also the speed where our approach has the most significant results.
Application: Scrolling Text
One particular application that requires optimal reproduction of fine details is the display of text. Ideally, given enough space, text simply has to be printed big enough to be legible. However, space is limited in practice and tricks are used to place more and longer text into a limited amount of space, e.g., stock tickers. Such a ticker consists of moving images that are ideal for our enhancement technique. Using our approach we were able to produce a 3x3 pixel-sized font, which is still proven to be legible in a user study.
How it Works
The basic principle behind our approach is a computational model of the tempo-spatial signal integration in the human retina. When an observer is exposed to a high-resolution image shown on a low-resolution display the result is a low-resolution image on the person’s retina, where we want to achieve the impression of high-resolution. The display forms the bottleneck. As a solution we rapidly vary pixel intensities on the display, to send more information and – exploiting time – can overcome this bottleneck. Loosely speaking, we leverage high temporal resolution to boost perception of spatial resolution. A related technique is commonly used in TV sets that compute and display interpolated video frames to create the perception of smooth movement on LCD screens.
The underlying model of the human visual system is simple, yet efficient enough to be implemented in graphics hardware to run close to real-time rates. In addition, many modern display devices do support high refresh rates – TV sets commonly use refresh rates of 200 Hertz or higher to provide a good video experience, and 3D computer monitors can show at least 120 Hertz.
While mobile devices are slowly catching up with the resolution of stationary displays, they are even more limited resolution wise. Our approach can bring significant benefit to these systems, as increasing the refresh rate consumes much less power than cranking up the screen resolution.
In future work, we seek to identify other shortcoming in image reproduction in time, space, color and brightness and to develop novel algorithms that exploit human perception to overcome them.
Piotr Didyk, Elmar Eisemann, Tobias Ritschel, Karol Myszkowski, Hans-Peter Seidel. Apparent Display Resolution Enhancement for Moving Images. ACM Transactions on Graphics (Proc. SIGGRAPH 2010, Los Angeles) 29(4), 2010.
Krzysztof Templin, Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, Hans-Peter Seidel. Apparent Resolution Enhancement for Animations. Proc. 27th Spring Conference on Computer Graphics, 28-30 April 2011, Vinicne, Slovak Republic.