Intel® TBB Helps Total War: Shogun 2 Run Faster and Better

If there’s a word that describes the design philosophy at Creative Assembly, it’s “flexible.” That’s not to say the Horsham-based team isn’t hardworking, rather that an atmosphere where ideas can develop is valued more than delivering on an original vision for a game.

Design documents here are continual works in progress, rather than set-in-stone templates for a final product. There’s a constant conversation going on between designers and engineers about what should and shouldn’t be included in a game—no feature is too precious to be dropped. Plus, because the various teams are organized by game area rather than skill set, ideas flow freely between software engineers, designers, and artists.

If it weren’t for this type of studio culture in which nothing is sacred, the gaming world might have been denied one of its most important and revolutionary moments.

“Our original intention was not to do a strategy game at all,” explained Creative Director Mike Simpson, who joined the company in 1996. “We were going to do an RPG based on the TV series Monkey, and we were going to do it in Singapore.”

Fortunately, his embryonic team was easily distracted from their mission. The brand new approach to real-time strategy of Command & Conquer* was the first thing to catch their eye, and then came 3D graphics hardware.

“At that point,” Simpson said, “doing a spline-based, curved surface landscape became possible in a way that had never been done before. And that gave us the opportunity to put the camera in the eyes of a general watching a battle landscape.

“That was the point when we realized this wasn’t a B-grade RTS clone any more, but something new. Shogun came out of that.”

Total War: Shogun arrived in the year 2000 and was suitably millennial in the way that it changed everything about strategy games. It brought together two previously separate concepts: a turn based, 2D campaign game that could rival many a Civilization* V clone for depth, and a real-time battlefield sim, which was far more epic than any predecessor in terms of on-screen landscapes and troops. And it threw in some RPG-lite character development for good measure.

The rest, as they say, is history. Medieval Japanese history, to be precise.

Ten years, five sequels, and seven expansions later, Creative Assembly is revisiting the Sengoku Jidai period to rework its masterpiece in a way that shows not only how far Total War has developed over time, but how PCs and technology have changed too.

“There were a number of things that we were looking for with a period and a setting,” Simpson said. “The first was a situation where many different factions were vying for supremacy in a context where any one of them could have won. The second was a technology race, and this period had a very fast-moving tech race fuelled by gunpowder weapons from the Europeans.

“The third thing we looked for was cool content, and all of the samurai, the ninjas, the geishas, the Japanese cultural elements associated with that period—it’s fascinating. It satisfied all the requirements of a Total War game probably more than any of the other periods.”

Previously each new Total War game moved on to a different point in time and geography, reinvigorating the core mechanics with the unique properties of medieval Europe, the Roman Empire, the colonization of the New World, and the Napoleonic era. However, Creative Assembly decided it wanted to return to its original source of inspiration.

“There are so many things that we can do now with the game engine and all of the technology that we’ve built,” Simpson said. “It seemed like the right time to revisit it.”

The Total War Machine

With each new game, new features and technical abilities have been added to the core engine of Total War. It has also been completely rewritten twice, once for Rome and once for Empire. The man who’s been responsible for overseeing its development from the start is Technical Director Richard Broadhurst.

“The game is too big now to rewrite it again,” Broadhurst said. “So we’re revolutionizing and evolving parts of the game with each iteration. For Shogun 2, there have been massive changes on the campaign map and the technology behind it. Now there’s more code shared with the battle map, so we can show a lot more detail.” (See the Campaign Trail sidebar.)

Other new features include Microsoft DirectX* 11 support, which will be matured thoroughly with the next title in the series, and a deferred renderer. The latter is being used to light dramatic night-time battles, in which the only sources of illumination are paper lanterns held aloft on sticks. In one awe-inspiring sequence, the pre-fight camera pans high over the heads of several thousand neatly aligned silhouettes marked out only by small pools of light.

Since the Empire rewrite, Broadhurst has been especially keen to refine the use of multi-threaded code in the game engine. In this respect Shogun 2 represents significant progress over its immediate predecessor, with an improved ability to scale effects and details according to the number of processor cores available.

“We’ve kept the minimum spec where it was with Napoleon,” explained Mike Simpson. “Keeping the spec the same helps to retain our customer base. At the top end, you want to make sure that if somebody buys an absolutely state-of-the-art machine that they’re going to get something out of it that they wouldn’t otherwise.”

Similarly with graphic effects and environmental details, players will be able to tune the number of soldiers in each unit to make sure that Shogun 2 can play on low-end machines. They may miss out on the spectacle of 2,000 infantrymen charging down a hill at once, but the fundamental game remains intact.

Every samurai and ashiguru on the battlefields of Shogun 2 has a polygon count to rival the main characters in a first-person shooter. But getting all the details right can be challenging.

“With Napoleon,” explained Mike Simpson, “there was complete outrage because we had the wrong type of moustaches on British troops. Our moustaches were big handlebar affairs, which apparently were too Victorian. We had to change it.”

As a result, the team now makes every effort to be as authentic as possible. They even invite academics to talk with the artists about a specific time period. But something, says Simpson, always slip through.

“We will get exactly the same kind of stuff with Shogun 2, because the fans really care about those levels of detail,” he said. “We possibly have a slight advantage with the Japanese stuff because there’s not as much detailed reference printed in English.”

He sighs, somewhat resignedly. “There’s more now than there was ten years ago, though.”

The Tools Behind the Team

Over the years, a close working relationship has developed between Creative Assembly’s senior programmers and Intel’s software engineers. Intel Senior Application Engineer Steve Hughes has been working with Creative Assembly on the Total War engine since mid-2009 during the development of Total War: Napoleon. Hughes’ role is to work alongside the team using the suite of Intel® tools, including Intel® VTune™ Performance Analyzer and Intel® Graphics Performance Analyzers (Intel® GPA), to help identify performance issues and bottlenecks.

More recently, during the development of Total War: Shogun 2, Hughes was a regular presence at the Creative Assembly studio. Starting in October 2010, Hughes spent a couple of days a month with Broadhurst and the team, increasing that time to a day a week in the latter stages, running tests using Intel GPA and other Intel tools.

“The Intel GPA tool gave us a clear picture of what graphics were costing,” said Broadhurst, “and we were then able to use the information to make important value judgments as to what is displayed on screen.”

“If something was too expensive and causing slowdown, we could quickly pinpoint the bottleneck, decide on a course of action, and tweak the code accordingly,” continued Broadhurst. “We were able to use the data that [Intel] GPA provided to help optimize the performance of the game on any graphics card.”

During testing, Hughes found a number of issues in the graphics pipeline that were affecting performance. Broadhurst’s team spotted the issues in parallel, and together they were swiftly able to identify the causes and apply code fixes. “Using Intel GPA alongside our own in-house tools helps us make sure nothing slips through the net,” said Broadhurst.

The most intensive period of testing happened at the end of the development cycle, at which point the team ran a series of replay tests where they simulated an accurate representation of a typical user experience to identify any lingering issues.

“We ran the replay tests using Fraps* benchmarking software and Intel GPA System Monitor to measure the frame rate and spot any slowdowns,” said Broadhurst. “Then we used [Intel] GPA to analyze the bottlenecks, and the data was combined with our own testing results to help us make informed decisions on what optimizations to apply and how.”

Using [Intel] GPA, Hughes and Broadhurst also analyzed the game’s performance on the Intel® HD Graphics of the 2nd generation Intel® Core™ processors. Happily, they didn’t encounter any problems in running the game. “I was pleasantly surprised by the performance of the game on Intel HD Graphics,” said Broadhurst. “In my experience, the performance of the new Intel Core processors with these built-in graphics has consistently exceeded expectations.”

Another important asset has been Intel® Threading Building Blocks (Intel® TBB), the multi-threaded code library, which has been imported into the game engine.

“Before we started using Intel TBB,” explained Broadhurst, “we used a threaded core library that I wrote based on some papers that Intel published 15 years ago. That was great for giving work to cores, but it couldn’t handle how to split up one job and balance it across multiple processors, which programmers still had to work out.”

By switching to Intel TBB, however, every last drop of performance is eked out of available resources, helping make the game run faster and better.

“We use Intel TBB to take our simple, straightforward serial code that we’ve been writing for years and years,” said Broadhurst, “and—by changing only a few lines—Intel TBB will distribute big chunks of processing over the cores in a way that uses them all efficiently.”

The practical upshot is that even though there are thousands of soldiers on the battlefield, they can be rendered in unprecedented detail and individually animated. Intel TBB takes care of balancing the workload that makes this possible in the background, leaving programmers to concentrate on implementing new features and nailing the smallest of details.

“It translates into things like trees,” Broadhurst explained. “[Previously] we could have had a thousand trees on the battlefield all standing stock still and ignoring the gale that’s blowing across the scene. Now we can drive the rain, drive the trees, and have thousands of trees all swaying. We couldn’t do that before.”

With its changing seasons, waterfalls, lightning storms, and curtains of cherry blossoms blowing over the screen, Shogun 2 is as evocative of the heavily stylized movies of Ang Lee as it is the blood-soaked battlefields of Akira Kurosawa. Thanks to the abilities of the multi-threaded engine, art has become a key part of the design agenda.

Campaign Trail

Total War* fans will see a big difference in the way Shogun 2 handles the divide between the campaign map and the battlefield game. By streamlining the game engine using Intel® Threading Building Blocks it’s become possible to share more of the core technology between the two, with the result that extra details are achieved in both.

“There was a clear divide in the development team between wanting to go back to the original Shogun map and have that parchment style,” explained Jamie Ferguson, “and other people who wanted to bring in the iconography and feel of medieval Japan, the colors and the saturation, the wood-block prints and so on.”

“So in the end,” James Russell interjected, “we did all of it. The parchment represents a lack of information. It doesn’t mean you don’t know what’s there, because you have a map. Then as you explore, the map blossoms into a 3D representation of the world. You see the landscape forming as your character explores, and the map becomes real.”

In addition to enabling players to zoom from a top-level view to a close-up of cities and units, the campaign map is used to create the topography of the battlefields.

“We’re now capable of creating a battlefield from almost every area of Japan,” Ferguson said. “When you go down from the campaign map to the battlefield, you’ll see something that is a pretty close approximation of what’s in that area. At the beginning of Rome, we had to make maps by hand. Now there are large areas of that which are performed by code.”

“We want to make it exquisitely beautiful and really immersive,” explained James Russell, lead designer for the Total War series. “Everything you do in the game should make you feel like a Japanese warlord.”

What Does it All Mean?

Shogun 2 has been described by members of the team as the “Zen of Total War.” That’s not because the battlefield action is especially serene: One key design goal was to strip out “some of the baroque bits that have accumulated over the years,” as Mike Simpson put it.

Jamie Ferguson, Shogun 2’s lead designer, explained that the amount of processing power available as a result of multicore processors, such as the Intel Core processor family, allows designers to increase the level of detail in the game world, and therefore reduces the need to rely on artificial feedback systems to let the player know what’s going on.

“Some say that graphics aren’t important to them,” he said. “But I believe that when you’re in a situation where you feel like you’re really seeing something occur, that element of realism adds extra authenticity to the game play.”

There are many advantages to evolving the Total War engine over a long period of time. Refining rather than rebuilding with each iteration means that it’s possible to achieve levels of detail that developers who make a one-off game or who use middleware graphics or physics engines simply won’t have time to implement. Arrows, for example, are persistent, and hatboxes are painstakingly accurate. If you hit an opponent in Total War, there’s no dice-roll to see if you connect—if you see someone struck in the game, they stay struck. There’s no need to layer on more information.

“When you slam your cavalry really hard into the side of a troop formation, and you see the soldiers recoil or get knocked to the ground,” said Ferguson, “well, these are things that help you identify and understand the effects of the units that you’re using. There’s a very strong game play message there. They’re not just eye candy.”

Shogun 2, though, is eye candy. While its visual effects may be the result of technology, tools, and clever design, they’re stunning nonetheless. And it’s only just hitting its alpha build. There’s much, much more to go into the game before it’s finally released, which should be some time in 2011.

And to think, it might have been an RPG based on Monkey.

About the Author

Adam Oxford is a freelance technology journalist. He edits the monthly hardware section for PC Gamer UK and blogs regularly for the magazine’s Web site. Before going freelance, Adam edited the UK’s leading PC entertainment publication, PC Format, for three years. He’s also written for The Sunday Times and The Guardian.

----------------------------------------------------

Sign up today for Intel® Visual Adrenaline magazine: http://va.softwaredispatch.intel.com