The Multi-Core Dilemma - By Patrick Leonard

By Steve Pitzel (Intel) (32 posts) on March 14, 2007 at 8:27 pm

Guest Blogger Bio: Patrick Leonard

Hardware Evolution

Throughout the history of modern computing, enterprise application developers have been able to rely on new hardware to deliver significant performance improvements while actually reducing costs at the same time. Unfortunately, increasing difficulty with heat and power consumption along with the limits imposed by quantum physics has made this progression increasingly less feasible.

There is good news. Hardware vendors recognized this several years ago, and have introduced multi-core hardware architectures as a strategy for continuing to increase computing power without having to make ever smaller circuits.

Sounds Good, So What's the Dilemma?

The "dilemma" is this: a large percentage of mission-critical enterprise applications will not "automagically" run faster on multi-core servers. In fact, many will actually run slower.

There are two main reasons for this:

  1. The clock speed for each "core" in the processor is slower than previous generations. This is done primarily to manage power consumption and heat dissipation. For example, a processor with a single core from a few years ago that ran at 3.0 Ghz is being replaced with a dual or quad-core processor with each core running in the neighborhood of 2.6 Ghz. More total processing power, but each one is a bit slower.
  1. Most enterprise applications are not programmed to be multi-threaded. A single-threaded application cannot take advantage of the additional cores in the multi-core processor without sacrificing ordered processing. The result is idle processing time on the additional cores. Multi-threaded software should do better, but many people are finding that their multi-threaded code behaves differently in a multi-core environment than it did on a single core, so even these should be tested.

Won't my application server or operating system take care of this for me?

One of the key considerations here is the order of processing. A single threaded application that needs to ensure that A happens before B cannot run multiple instances concurrently on multiple cores and still ensure a particular order.

Application servers and operating systems are generally multi-threaded themselves, but unfortunately their multi-threaded nature does not necessarily extend to the applications that run on them. The app server and OS don't know what the proper order is for your particular business logic unless you write code to tell them. In fact, they are designed to simply process any thread as soon as possible, potentially disastrous in a business application. SMP (symmetric multiprocessing) has similar limitations.

So we are back to the same problem -- how to run multiple instances concurrently on multiple cores and still ensure a particular order.

Intel's 45nm announcement

Intel recently announced that they will have chips in the near future with 45nm features, a significant advance from the 60-65nm that is prevalent today. The company has also made it clear that this is not reducing the need for multi-core.

Around the same time as this announcement, Intel announced that they have an 80 core processor in the works. Power and heat will have to be addressed for a processor with 80 cores to come to market. So 45nm may mean some increase in clock speeds for future processors, but its primary use will be enablement of a higher number of cores.

Concurrent Computing

There is no easy solution, but there are several options, and they all involve bringing concurrency to your software. Concurrent computing (or parallel programming, as many refer to it) is likely to be a very hot topic in the coming years, so it's a good idea to start preparing now.

Since multi-core servers already make up most of the new servers shipments, concurrent computing in the enterprise will quickly become a way of life. So we need to put some thought into two things: how to make existing applications run concurrently, and how to build new systems for concurrency.

More people are talking now than any time in recent memory about how to do multi-threaded software development as the primary answer to concurrency. However, instead of writing our application code to be multi-threaded, we should consider how to abstract threading out of application code. Multi-threaded code is difficult to write and difficult to test, which is why many people have avoided it in the first place.

At Rogue Wave Software we have been working for several years in the area of "Software Pipelines". Software Pipelines* is an approach that can be used to abstract the threading model out of the application code. As software developers, you would not mix your UI code with your business logic, and for good reasons. A similar principle should apply for programming for concurrency -- the threading model should not be driven from within the application logic.

There are several important benefits to this approach. Removing threading from application code means:· The application developer doesn't have to own the threading model· Existing applications can move into a concurrent environment with much less effort· Makes it easier to scale to additional computing resources without modifying the application· If done right, the application can continually be tuned for performance without modifying application code

This approach does not allow the application developer to wash their hands entirely of concurrency. Application code needs to be written to be thread-aware, but does not need to have threads written into it. For more information on Software Pipelines, you can read this white paper (you have to log in to webservices.org to download it).

Consider how to abstract your threading model from your application logic, and you may find a smoother (concurrent) path ahead.

* Software Pipelines is a general term, not owned or trademarked by Rogue Wave or anyone as far as I'm aware. It also does not require the use of any of our technology. Software Pipelines borrows conceptually from hardware pipelines and also from fluid dynamics, which has interesting parallels to software systems.

Categories: Parallel Programming

Comments (11)

March 15, 2007 8:30 AM PDT


Stan Jones		 Great article - will check out Software Pipelines. But you do realize that most Java developers believe that the application servers will automatically maximize application performance on multicore processors?
March 15, 2007 10:15 AM PDT


Stan Jones		 So I've read about Software Pipelines. The problem Java developer that don't think that app servers will solve this problem probably think that the JVM will.

Software Pipelines looks a bit like multi-threading at a business component level.

The only people talking about this new methodology at the moment seem to be Intel and Rogue Wave.
March 15, 2007 11:03 AM PDT


Patrick Leonard		 Stan, you hit it exactly. Most java developers that we talk to do assume that this will be handled for them. And there is some truth to that - certain aspects can be handled "automagically". But there are some significant use cases where existing apps will not scale when moved to multi-core hardware.

For example, any application that has to ensure ordered processing will have this issue. The JVM, OS, app server, etc. don't know the business logic of the application, so they cannot execute parallel tasks in a way that is consistent with the application's ordering requirements. In fact, they may execute the tasks in a way that is at odds with the application.
March 16, 2007 8:13 AM PDT


Emilio		 I would like a compare/contrast between Rouge's stuff and Pervasive DataRush. Are both 100% Java? Do both use dataflow graph theory to structure pipelines? Are both targeting data-intensive BULK data processing apps? Do both auto-scale up and down as processor and memory are modified on the fly by the VM?

When to use which?

Let's cut through the marketing b.s. and talk turkey so developers will learn something in this blog. I dare ya :-)

PS: I totally do not get the comments about multicore + app servers = slight problem. I don't see the issue because the OS and SMP multicore server abstract any/all issues from the JVM that spins up concurrent threads. There are 1000 "contracts" all the way up and down the stack that would prevent surprises. So I am most definitely missing the point here. Can we talk a couple levels deeper starting from EJB bean pooling and driving down the stack? I need to see where the issue is for J2EE-like use-cases.

PPS: I see all the issue with batch data processing apps. Not with OLTP or ESB/SOA apps. Just don't see it...
March 16, 2007 4:37 PM PDT


Patrick Leonard		 Emilio,

I will try to "de-marketing-ize" as much as possible ;) I'm not familiar with DataRush (apart from what i just got from the web site) so I can't really comment on what they have done.

The limitation in app servers with multi-core is primarily around applications with ordering requirements. If the app doesn't care about order, then the OS, app server, SMP, etc. can spawn new threads as needed and it should scale fine. But if the application logic requires ordering ('A' must happen before 'B'), then they have no way of knowing how to avoid race conditions or other problematic behavior.

This problem is not unique to java. It is a basic issue of concurrency - how to do more than one thing at a time and still ensure that order happens as required by the business logic.

Yes, it can be done with multi-threaded programming, and many apps are already set up for this. But many are not, so the question is how to get them ready for concurrency without significant rewrites.

There is another thing to consider - do you want your concurrency model embedded in your application logic? It's a bit like databases - the data model doesn't reside inside the application logic (although people used to do that way back when). Abstracting your concurrency model out of the application code can have benefits, such as being able to performance tune without modifying source code.

BTW, to answer your question, the Rogue Wave implementation is multi-language, so it supports Java, C++, .NET, BPEL. We have used it for bulk batch apps and for transactional.
March 16, 2007 4:47 PM PDT


Patrick Leonard		 One more thing that may be of interest - just so you know that I'm not the only one who thinks this (is that the definition of insanity?...), a few other postings on concurrency from SDTimes.

Threading Maturity Model
http://ztrek.blogspot.com/2007/02/responses-to-threading-maturity-model.html

The Personal Threading Maturity Model
http://www.knowing.net/default,month,2007-01.aspx
March 21, 2007 9:51 AM PDT


Emilio		 I guess I understand exactly what you are saying, but I only take exception to the premise that J2EE and .NET -- when used to create OLTP threaded Web Apps -- would see any impact going to multicore. You see, the J2EE container contracts already assume unordered, concurrent "beans" and any shared memory issues are built into the framework, so it's really not an issue.

I agree, however, that if you want to sync threads MANUALLY, and you go beyond the standard J2EE and .NET API contracts, or you throw them aside all together, then you're in deep doo doo (technical term).

I really hope PeakStream and others don't wade into OLTP, SOA and Enterprise Service Bus "blueprints", because I think the value of your platform will diminish to near-zero. It's actually the poor sods who are forced to use straight C, Java, COBOL or C# that have a huge uphill battle on multicore -- and you will only find those developers in very specific markets solving very specific problems AWAY from OLTP, SOA, Web Apps etc...

Of course, it just so happens that DataRush crushes these non-J2EE data parallel problems http://www.pervasivedatarush.com/benchmarks

:-)
March 30, 2007 5:09 PM PDT


Henry Cobb		 Gosh, Software Pipelines, do you mean like using the pipe symbol to connect two processes so that each can run at full speed with say a shell buffering the output of one into the input of the other?

If the next version of WinDos includes this advanced new feature then Microsoft will only be like three decades behind the state of the art.

-HJC
April 3, 2007 5:05 PM PDT


Patrick Leonard		 DataRush is also working to bring concurrency to the application server, but they seem to be focused more on the dtraditional compute grid problem - taking an algorithm or other large compute task like monte carlo or pricing algorithm and breaking it up into pieces to run across a grid. I haven't tested DataRush, so I can't say for sure, but that's what it seems from their web site.

Software Pipelines is an approach for bringing concurrency to an application along with order and control of the threads - from the application layer. Most business applications can't run in a compute grid because they have these types of requirements (FIFO ordering, human interaction, splits/joins, etc.) that they can't hand over to the app server or OS. This typically gets solved with multi-threaded programming. Of course you can still write multi-threaded code to solve this, but most people don't want to.
June 8, 2007 7:19 AM PDT


Matt		 DataRush does not currently operate on a "grid"---it requires an SMP machine since it resides entirely within one JVM. Ordering is explicit: you plug your Java code into operators which you then arrange into a pipeline using a coordination language called DFXML. More generally, you can arrange the nodes of the computation in a directed graph (not just pipelines, but also fan-out structures through partitioning, replicated output, or operators that map one input to many outputs and fan-in structures through unpartitioning or operators that map many inputs to one output). All threading is managed by the runtime system, responsible for scheduling the execution of the various operators when data is available for them to process. The system is quite analogous to pipes in Unix, but with strict type enforcement, configurable properties at each operator, and the portability of the JVM.

From what I can tell, Software Pipelines operates at a courser level, wrapping an entire application in each node, then providing an API for nodes to communicate with one another either on the same machine or across a network via web services. Does it provide a standard library of operators, or simply a contract which you must abide by to take advantage of the coordination system?
June 27, 2008 2:18 AM PDT


Ondrej Spanel		 Quote: "For example, a processor with a single core from a few years ago that ran at 3.0 Ghz is being replaced with a dual or quad-core processor with each core running in the neighborhood of 2.6 Ghz. More total processing power, but each one is a bit slower."

This does not reflect for the very important fact the work per clock has significantly increased meanwhile. I have Intel Core 2 Quad at 2.4 GHz now in my computer. Even its performance as a single core is higher than what I had with 3.6 GHz Intel Pentium 4, while the power consumption is much lower.

This is one thing I see as a good with the multicore revolution - nobody seems to be obsessed with the GHz any more, and it is the performance which is increased instead.

Trackbacks (1)


Leave a comment  

To obtain technical support, please go to Software Support.
Name (required)*

Email (required; will not be displayed on this page)*

Your URL (optional)


Comment*