To use a shader in OpenGL ES, the program will typically:
1. Load a file
2. Compile the shader
3. Link the shader

Before compiling the shader, the program calls glShaderSource to set the shader source. The real trick is with glShaderSource. This function can take multiple strings and combines them for compilation.

Here is a code sample that compiles a shader given a string (ShaderData) and the size of the string (Size). The function is passed a type (Type) which is defined as GL_VERTEX_SHADER or GL_FRAGMENT_SHADER. This function uses an #ifdef to compile the source as a vertex or pixel shader:

The documentation for glShaderSource says that the function can be called with string lengths of 0 for NULL terminated strings. However, for my implementation of OGLES 2 this did not work, so I had to specify the individual string lengths (as defined in ShaderStringLengths for this example).

With #ifdef’s, an application can modify a shader at runtime. It’s possible to enable/disable features like post processing effects, dynamic lighting, and more based on #defines. Best of luck!

Since I work at Intel, I'll obviously be discussing some experiences porting ARM apps to x86 :)
Session Description
By GDC 2012, there will be 3 major ABIs (armeabi, armeabi-v7a, and x86), several GPU architectures (PVR, Mali, Tegra), and a wide range of Android OS versions.  Without careful planning and some tricks, validating your game on a variety of hardware and software configurations will be time consuming and costly.
This session presents best practices for implementing truly cross-platform Android applications and how to properly enable low-level hardware optimizations.  This goes beyond high level issues such as appropriate screen sizing and handling different form factors. Attendees will learn:
- How to package multiple platform specific .so files in a .APK file (“fat binaries”)
- Understand the memory alignment differences between ARM and x86, and how to exploit them
- How to use processor specific SIMD intrinsics (NEON and SSE)
- How to properly use OpenGL extensions
- Understand the best compiler flags for different ABIs, and
- How to abstract out hardware and when does it makes sense to do so

At the end of this session, attendees will know how to create highly optimized native applications on Android that follow best practices for cross-platform development. The audience should already understand Android development, including use of the NDK.

Please come join me at GDC 2012! See you there.

One difference between x86 and ARM is the memory alignment requirements for data. Let’s look at a simple example:

This example just logs the size and offset of variables in TestStruct. The output for this program isn’t too surprising:

ARM
I/libtestjni( 5025): TestStruct (size: 12)
I/libtestjni( 5025): -- Var1 offset: 0
I/libtestjni( 5025): -- Var2 offset: 4
I/libtestjni( 5025): -- Var3 offset: 8

x86
I/libtestjni( 4175): TestStruct (size: 12)
I/libtestjni( 4175): -- Var1 offset: 0
I/libtestjni( 4175): -- Var2 offset: 4
I/libtestjni( 4175): -- Var3 offset: 8

But now, let’s change TestStruct to the following:

The output is now:

ARM
I/libtestjni( 4675): TestStruct (size: 24)
I/libtestjni( 4675): -- Var1 offset: 0
I/libtestjni( 4675): -- Var2 offset: 8
I/libtestjni( 4675): -- Var3 offset: 16

x86
I/libtestjni( 4079): TestStruct (size: 16)
I/libtestjni( 4079): -- Var1 offset: 0
I/libtestjni( 4079): -- Var2 offset: 4
I/libtestjni( 4079): -- Var3 offset: 12

The 8-byte (64-bit) mVar2 results in different layout for TestStruct. This is because ARM requires 8-byte alignment for 64-bit variables like mVar2. In most cases, this won’t cause problems because building for x86 vs ARM requires a full rebuild.

However, if an application serializes class or structures, this could cause a size mismatch. For example, say you create a save file on an ARM application and it writes TestStruct to a file. If you later load this file on an x86 platform, the class size in the application will be different than the saved file. As you can imagine, similar memory alignment issues can happen for network traffic that expects a specific memory layout.

The GCC compiler option “-malign-double” will generate the same memory alignment on x86 and ARM. However, since the OS was not built with this flag, it will break some OS calls.

You can control the alignment of variables through compiler attributes. So, if we tell GCC to align(8) for mVar2, x86 and ARM will have the same alignment:

The output is now:

ARM
I/libtestjni( 4675): TestStruct (size: 24)
I/libtestjni( 4675): -- Var1 offset: 0
I/libtestjni( 4675): -- Var2 offset: 8
I/libtestjni( 4675): -- Var3 offset: 16

x86
I/libtestjni( 4678): TestStruct (size: 24)
I/libtestjni( 4678): -- Var1 offset: 0
I/libtestjni( 4678): -- Var2 offset: 8
I/libtestjni( 4678): -- Var3 offset: 16

Once you understand the memory alignment difference between x86 and ARM, rebuilding your ARM Android NDK application for x86 should be pretty simple! Go grab the latest NDK and give it a try.

Topics covered include:
- Intel® Atom processor optimizations
- Understanding graphics packaged with Inte®l Atom processor based platforms
- Tools to help optimize and profile game/graphic applications
- Microsoft DirectX* optimizations (most of the focus is on Microsoft Windows* for this revision of the guide)

Most of these tips and tricks are for game developers targeting Intel® Atom processors. However, anyone working with graphics should find some useful information. Please let us know what you think! We will use your feedback to help improve the next revision.

Task-based threading
A particle system is ideal for threading because it’s essentially a big loop that operates on a bunch of independent objects. Because the objects don’t need to interact (they don’t write to shared data), they can easily be spread across multiple threads.

Here is an example of a loop that updates all particles:

for( unsigned int i = 0; i < NumParticles; i++ )
{

UpdateForces( g_Particle[i], DeltaTime );
UpdateCollision( g_Particle[i], DeltaTime );
UpdatePosition( g_Particle[i], DeltaTime );

}

Threading these loops is trivial. OpenMP is supported by all major compilers and allows simple for-loops to be parallelized. OpenMP will automatically divide the loop and run it across all available cores on the machine.

#pragma omp parallel for
for( unsigned int i = 0; i < NumParticles; i++ )
{

UpdateForces( g_Particle[i], DeltaTime );
UpdateCollision( g_Particle[i], DeltaTime );
UpdatePosition( g_Particle[i], DeltaTime );

}

However, this is not an ideal approach. This method does not support good load balancing in a complex system such as a real game. For example, if spawned threads generate more threads (nested threading), then it’s possible to oversubscribe the system. Oversubscription causes a performance hit because there is an overhead associated with swapping the execution of the threads.

There is a better way to thread particles than the simple fork-and-join approach of OpenMP’s parallel for. It’s very simple to divide the work to run as tasks. Using tasks provides several benefits. Once you have a tasking system set up, it’s easier to add new tasks to increase parallelism throughout the code. Also, it’s easier to load balance and be platform-agnostic. If the task scheduler manages all parallel tasks, the program will avoid oversubscription. In this example, we don’t have to wait for all the particles for a given emitter to finish before moving to the next emitter and scheduling more tasks.

To convert the loop above to use tasks, the code needs to divide the work into several tasks and submit them to a task scheduler. These tasks will define the range of particles to update and includes all required information.

unsigned int Start = 0;
unsigned int End = 0;
unsigned int ParticlesPerTask = NumParticles / NumTasks;

for( unsigned int i = 0; i < NumTasks; i++ )
{

// Determine the range of particles to update
// (the last task might be bigger)
End = (i < (NumTasks-1)) ? uStart + ParticlesPerTask : NumParticles;

// Build and submit the task
ParticleTask* pTask = &g_ParticleTasks[i];
pTask->m_DeltaTime = DeltaTime;
pTask->m_Start = Start;
pTask->m_End = End;

g_TaskScheduler->addTask(pTask);

// Move to the next set of particles
Start = End;

}

...

void ParticleTask::Run()
{

for( unsigned int i = m_Start; i < m_End; i++ )
{
UpdateForces( g_Particle[i], m_DeltaTime );
UpdateCollision( g_Particle[i], m_DeltaTime );
UpdatePosition( g_Particle[i], m_DeltaTime );

}

}

Using SSE
Tasking is a great way to get scaling with a particle system. However, it’s also important to make sure the code fully uses the CPU cores it is running on. Developers should consider using SIMD (single instruction, multiple data) with SSE instructions. For floating point, SSE instructions operate on 4 floating points in a single instruction. Obviously, this has the potential to increase throughput by up to 4x.

There are multiple ways to use SSE. For developers interested in maximum control, intrinsics are the best way to utilize SSE. Intrinsics are compiler specific functions that generate inlined highly efficient machine instructions. For developers targeting DirectX on PC or Xbox, the XNA Math library wraps the use of intrinsics in a library that already supports vectors and matrices.

There are a few things to keep in mind when use the XNA Math library. First, be careful accessing individual elements. Getting and setting elements inside an SSE vector isn’t free. It’s best to put data into XMVECTORS and keep it there as long as possible. Also, make sure you are using properly aligned data.

For a more in-depth discussion on cross-platform SIMD, check out Gustavo Oliveira’s well written article.

Sample application
A lot of this article is based on the learnings from my cube-mate, Quentin Froemke (lovingly referred to as Q-Ball). Quentin created a tech sample called Ticker Tape to showcase some best know practices for creating a highly scalable particle system. To learn more about Ticker Tape, check out the associated article and download the source code.

Ticker Tape simulates the physics behavior of fluttering and tumbling. This simulation showcases an interesting and complex particle behavior. The performance benefits for utilizing multi-core and SSE are apparent in Ticker Tape:

Both multi-core and SSE give significant benefits. The benefits of Intel® Hyper-Threading Technology are much more prevalent with SSE. Hyper-threading takes advantage of the fact that all execution units for a CPU core might not be fully utilized by a single thread. Multiple execution units allow multiple instructions to be executed simultaneously and be pipelined. Execution units perform operations such as loads, stores, integer operations, floating-point operations, and SSE operations. With more SSE instructions, there is a better utilization of all the processors resources and therefore better use of hyper-threading.

If you are afraid that adding a highly parallel particle system will Duke Nuke your schedule, you can always investigate middleware options. For example, the team at Fork Particle has a parallel particle systems backed with solid content creation tools.

Summary/ Conclusion
With higher core counts, it’s possible to scale with compute power and show a larger number of particles. This would give users a perceivable difference for high-end machines, without punishing players with lesser gaming hardware.

Particle systems are ideal for threading and a good wading pool for people new to threading. Future blogs will go off the deep end and explore more complex problems like crowd simulation AI. In the meantime, check out Ticker Tape and let me know what you think!

Update: Fixed source code

Some think programming for multi-core is a punishment reserved for the eighth ring of hell (I recently beat EA's Dante’s Inferno, so it’s on my mind). But you don’t need to fight your way through purgatory to reach multi-core heaven. If you keep two concepts in mind, things get easier:

1. Use data decomposition – A game can’t scale just by dividing subsystems onto separate threads (sometimes referred to as "functional decomposition"). It has to divide data intelligently to run across multiple cores. The ever-insightful Mike Acton has a great article on this very subject over at Insomniac Games’ R&D page.

2. Use tasks, not threads directly – To scale on an arbitrary number of cores and be truly cross-platform, work should be divided into tasks. A task is a unit of work (for example a function pointer and data) that can run without (or with very limited) synchronization. These units of work are processed by a thread pool which is scaled appropriately to the available parallelism in the hardware.

By utilizing tasking and data decomposition together, it is possible to take advantage of multi-core for all subsystems. Prior to DX11, rendering was still the locked gateway to heaven. With DX11, it’s now possible to divide the work of rendering into multiple tasks using deferred contexts. Jérôme Muffat-Méridol recently delivered a presentation about DX11 multithreaded rendering at GDC Europe 2010 which detailed methods for doing this. Jérôme lovingly refers to this project as Nulstein. In previous versions of Nulstein, Jérôme also explored the requirements of building a task scheduler from scratch, but the focus for GDC Europe was multithreaded DX11.

It’s interesting to note that DX11’s multithreaded goodness can run on DX10 hardware. DX11 supports “feature level,” which allows the latest API to be used on a wide range of hardware (provided the application is running on Vista/Win7) by emulating any missing behavior in software. To learn more, check out the articles and presentations associated with Nulstein.

There is a growing uptake of task-based architectures in game engine design. Mark Randel’s implementation of physics and AI in Terminal Reality’s Infernal Engine is nothing short of amazing. The fellows over at BitSquid are building a new game engine written with solid support for multi-core. Furthermore, the Civ5 team at Firaxis is getting great performance out of a task-based approach (check out their GDC 2010 presentation: “Firaxis’ Civilization V: A Case Study in Scalable Game Performance”).

Intel recently introduced its 2nd Generation Intel® Core™ processors (codenamed Sandy Bridge). This generation of CPUs really ushers in the era of processor graphics. There is a lot to explore with Sandy Bridge! Check out Onloaded Shadows if you want to see how the close combinations of CPU and GPU can lead to new and interesting areas.

At CES this year, Intel introduced the 2nd Generation Intel® Core™ processors (codenamed “Sandy Bridge”). This processor is Intel’s first new microarchitecture utilizing 32nm technology. Smaller transistors and new architectural design result in higher performance at lower power.

One advantage of working at Intel is having early access to leading-edge technology. I’m writing this blog on my Sandy Bridge machine. Right now it may be another big dev box on my desk, but this chip is now reaching numerous mainstream laptops. The processor graphics are pretty good. It runs one of my favorite games, Bioware’s Mass Effect* 2, at a solid fps in glorious detail (by the way, my mission was truly suicidal).

One key feature of this architecture is this tighter integration of graphics into the processor. The ring architecture that connects the processor cores (x86 cores) together is now connected to the processor graphics. This ring interconnect enables high-speed and low-latency communication between the processors cores, processor graphics, and other integrated components, such as the memory controller and the display. Basically, the processor cores and graphics engine communicate through a shared cache, creating some interesting possibilities for tight CPU/GPU interaction. We’ll explore some of these topics in future articles.

The integration of process components also provides some new improvements to Intel® Turbo Boost Technology. Intel® Turbo Boost Technology enables the processor to adjust the processor core and processor graphics frequencies to increase performance and maintain the allotted power/thermal budget. This means the processor can increase individual core speed or graphics speed as the workload dictates.

During performance analysis, it’s important to pay attention to dynamic frequencies. A typical game will load down the CPU and GPU with the processor finding a good balance. However, if you are playing back a captured frame, the CPU workload might not be as high and the graphics dynamic frequency might affect your results. This should be a relative scaling within the frame, but the time to complete the frame might be unexpected.

Since Intel® Turbo Boost Technology is automatically controlled by the CPU, a developer cannot directly control it. However, it’s important to understand how it works. Most games I have investigated benefit well from the graphics turbo scaling.

The addition of Intel® Advanced Vector Extensions (Intel® AVX) is another interesting Sandy Bridge feature. AVX extends SIMD (single instruction multiple data) instructions from 128 bits to 256 bits. For applications that are floating point intensive, AVX enables a single instruction to work on eight floating points at a time instead of the four that the current SIMD provides.

It’s important to note other hardware vendors have also announced support for AVX.

Most developers will just use the latest Microsoft® Visual Studio compiler or Intel’s C/C++ Compiler to take advantage of AVX. But, for the clock counting, bit shifting, tech heads out there, you can learn more at the Intel’s AVX website.

The best way to work with AVX is through intrinsics, which are supported by both the Microsoft and Intel compilers. Anyone familiar with programming SSE or the PlayStation* 3’s SPUs will be good “frenemies” with intrinsics. Intrinsics are compiler specific functions that usually compile down to highly efficient inlined machine instructions. Since the compiler has a strong understanding of intrinsics, it will often generate code faster than inlined assembly. Intrinsics are the best way to write high-throughput, compute-intensive code on the CPU (intrinsics are an acquired taste, like wine or Remedy Entertainment’s Alan Wake*).

Intel engineers and performance hungry developers are already exploring ways AVX can benefit game and graphics applications. For example, my coworker Stan Melax, just wrote a great article presenting a programming pattern to improve the performance of geometry computations by transposing packed 3D data on-the-fly.

AVX is interesting, but let’s shift our focus to the graphics engine.

Remember, Sandy Bridge is a DirectX 10.1 part. I like the DX11 multithreaded API, so most of my current code is DX11 with the proper DX10.1 “feature level” set for Sandy Bridge. With full DX10.1 support, there are no real major surprises when programming for Intel graphics. However, there are few things to keep in mind.

The memory layout for processor graphics is different than it is for a discrete card. Graphics applications often check for the amount of available free video memory early in execution. As a result of the dynamic allocation of graphics memory performed by the Intel® HD Graphics devices (based on application requests), you need to know the total amount of memory that is truly available to the graphics device. Memory checks that supply only the amount of “local” or “dedicated” graphics memory available do not supply an appropriate value for the Intel® HD Graphics devices.

All video memory on Intel® HD Graphics and earlier generations, including Intel® Graphics Media Accelerator Series 3 and 4, use Dynamic Video Memory Technology (DVMT). DVMT memory is considered “local memory.” “Non-local video memory” will show as ZERO (0). This should not be used to determine compatibility with Accelerated Graphics Port (AGP) or PCI Express*.

To accurately detect the amount of memory available, you’ll need to check the total video memory availability. The Microsoft DirectX* SDK (June 2010) includes the VideoMemory sample code and describes five commonly used methods to detect the total amount of video memory. Applications targeting Microsoft Windows Vista* and Microsoft Windows* 7, should reference GetVideoMemoryViaDXGI. For Microsoft Windows* XP applications, GetVideoMemoryViaWMI is a good starting place. For more information, see the Microsoft sample code site.

The best place to get started with Intel processor graphics is to check out the Intel Graphics Developer’s Guides

With nearly a million PCs shipped each day, the available market of processor graphics is growing quickly. It’s worthwhile to understand and validate on processor graphics. Soon, processor graphics will be everywhere.

I often ask game developers “what do you want?”
In other words, I really want to know what technologies game developers are interested in. More to the point, I am interested in technologies that they wish someone else would helped implement. One common answer is “give me a file/memory manager… no one wants to write a file/memory manager.” I’ll write another blog about the memory manager part ;)

I am researching the needs of a good generic file manager for games.
Here is what I think is required:
- Designed for games
- Platform independent
- Multithreaded
- Open source
- Support streaming content
- Support archives

There are some good articles about file systems on the internet.
Michael Walter wrote a good article on FlipCode about virtual file systems.
However, there is little about open source file systems and desired features for a file manager for games.

I need your help.
What do you want?
Is there a good open/closed source project already in the works?
What features would make a great file system for games?

Thanks,
Orion

Writing threaded AI (Artificial Intelligence) is epic?
It’s easier than you would think!

I am going to give a presentation about multithreaded AI at GDC this year. We will examine how AI can be threaded and live in a highly parallel environment. How can you thread AI? How can AI talk to physics running on another thread or device? Is deferred processing worthwhile? Do you have to thread your designers? My presentation will hopefully answer these questions and more. The presentation will end with a quick overview of Intel’s Smoke demo; Smoke is a n-way threaded framework that includes source code for highly parallel AI (you can download it at Whatif.intel.com).

Here is my tentative outline:
• The challenge – Why thread AI
• Define a Simple AI – Let’s start by defining a simple game AI
• Threading AI – Let’s thread our simple AI
• Working in a Multithread Land – Let’s look at how AI can work and live in a highly parallel environment
• Smoke: An Example – A quick overview of Smoke and it’s multithreaded AI
• Summary - Threaded AI for the win!!

Please let me know what you think.
Is there anything you want to hear about that I’m not covering?

If you are interested or working with AI/Smoke, I hope to see you at GDC!

EDIT: You can get the Smoke demo here.

Jeff Andrews wrote a great article about multithreaded game engines over on Whatif.intel.com. These are the concepts that the Smoke (n-way threaded game framework) demo was built on. One of the readers, Josh, brought up a good comment (check out the comments at the bottom of the article). One of the questions in his comment really got me thinking about lock step versus free step. Which is better? Which is easier? Which does Smoke use?

The last question is the easiest: Smoke uses lock step. Jeff goes into some good detail in section 2.1 about these two execution modes. Lock step just means all systems update at the same rate. For each frame, all systems update and sync. If a system takes longer to run, the other systems have to wait for it to finish. This is the biggest complaint against lock step. In free step, all systems run at separate frequencies… they update and sync based on events.

Lock step does have its advantages. It’s easier to understand and code. It’s definitely easier to debug (you can check for sanity at each sync). As Josh points out, lock mode doesn't suffer from the possibly increased frame latency of free step. But lock step can waste resources if other systems are waiting around… or does it?

First attempts at threading games mostly involved functional decomposition; just put each system (graphics, IO, physics, etc) on a separate thread. If a system finishes updating and has to wait for another system, that system’s thread would sleep. This is wasting resources because that thread could be doing more work… this is the major fault of lock step. However, Smoke uses a job pool and worker threads to support functional and data decomposition. So… if a system finishes its work, that thread can work on jobs for other systems! Now we are not wasting resources on lock step ^_^ Score! There are a few exceptions, if a system doesn’t divide its work properly and takes a long period of time to finish… then the worker threads could end up unemployed.

I have want to rework Smoke to run in free step mode. But I am comfortable with lock step. It’s easy for me to understand and explain… and I can easily map out the latency between the systems. I wonder if anyone will take up the challenge and get Smoke working in a free step mode… I’d love to hear if anyone out there gives it a try ^_^ I’d also like to hear more about peoples’ experience on free or lock step. Which do you like? Is free step worth the possible headaches (especially with a large team of developers)?

While running free sounds nice. I think I’ll keep my projects locked down for the near future.

This is the first of a series of blogs I am going to post about Smoke.
Smoke is a demo developed by Intel to show n-way threading in a game framework… that’s a mouth full! Basically, it shows one way games can maximize the CPU. To maximize CPU utilization, a game needs to use all available cores. By properly threading a game it can have more accurate physics, smarter AI, more particles, and/or a faster frame-rate. Smoke demonstrates one way to achieve better games.

Smoke has been in development for over a year and a half! I’m glad it’s finally released.
There is a ton of content about Smoke here (including all the source code).

Since this is my first blog on ISN, let me introduce myself...
I’m a senior engineer with Intel’s Visual Computing Software Division. I am the tech lead on the Smoke project. Prior to joining Intel in 2007, I worked on several PlayStation 3 titles as a senior programmer for Insomniac Games. My most recently published titles are Resistance: Fall of Man and Ratchet and Clank Future: Tools of Destruction.

In my next post, I’ll talk about some of the work I did on Smoke (e.g. multithreaded AI).

- Orion