English | 中文 | Русский | Français

2,590 Posts served

8,335 Conversations started

I Cuda written more, but unlike Polonius, I actually will be brief

By wolfmurphy (9 posts) on June 16, 2009 at 10:58 am

Thanks to Wen-Mei for a delightful chat. I found your site, http://courses.ece.illinois.edu/ece498/al/, with the curricula for your Programming Massively Parallel Processors course. This curricula fosters students acquiring practical experience, typically learned toiling hours to days in trenches with little sleep, and less coffee. I plan to look over his mathematically prodigious mini-case studies to seek ones more accessible to undergraduate students, and not surprisingly I seek your help with this effort. I hope to have this be a very long chain of messages, with at least one entry from Craig Murphy, not known to me as a relative, who asked a great question during the talk.

... Tom

Categories: Academic, Parallel Programming

Comments (4)

June 17, 2009 4:18 PM PDT Kenneth Lloyd Total Points: 5 Registered User

After having been programming in CUDA since beta 0.8, Dr. Hwu's course is an excellent resource for MPP on GPUs. There are many rewarding opportunities exploring various computational fabrics between multiple CPU cores, multiple GPUs with hundreds of cores, and clusters of these hybrid, heterogenous devices. By exploring the possibilities of dynamic IB interconnections, in parallel addition to the other various connection channels (xGigE, PCIe), where do you think MPP will be in 5 years? And what ramifications does this hold for traditional clustered supercomputers?

June 17, 2009 4:49 PM PDT wolfmurphy Total Points: 805 Status Points: 805 Black Belt

I think we will end up morphing the first M in MPP from massively to many; the 'P' from processing to processors. Actually, immediately after I wrote that, I realized I didn't believe it at all. I do think some other MLA (many letter acronym) will surface. Please be aware my current crystal ball is cracked and very hazy; I am looking to trade it in. In five years, I expect we will be programming a distributed collection of shared memory many-core processors. I wouldn't be surprised if there were chips that could masquerade as hundreds of shared memory cores or as a large array of devices lock-step executing the same instruction. The masquerade would likely reduce heat by powering down unused silicon. I certainly agree with you that hybrid is going to be a key phrase. We will need to have hybrid codes that can stride between shared and distributed memory architectures. We will need hybrid algorithms. I would not at all be surprised to see all this resting on OpenMP, MPI, and OpenCL. I would expect that underlying plumbing will rely on dynamic IP connections. Bandwidths and correct authorization plumbing may finally allow for essentially synchronous, geographically distributed grid processing. Underlying all this will be the need for portable software. I think it will be essential to avoid going back to the days when it was cumbersome to even port code between platforms from the same manufacturer. Luckily, my almost broken crystal ball does not disturb me. I have consistently been more delighted with what actually is around the corner, than what I think might be around the corner.

June 18, 2009 7:22 AM PDT Ken Lloyd

Interesting vision! I have given up on crystal balls, and to paraphrase: "If I see further, it is by standing on the shoulders of giants." I see a future that the role of genetic evolutionary algorithms will play in configuring the compute fabrics in hybrid, heterogeneous clusters. The reason for this statement is I know how hard it is to write software to optimally compute solutions to different problems using SIMD (or SPMD). The nature of the computational paths varies, as does the data partitioning schema - not to mention that pre-existing schedules, priorities, and the vagaries of the cluster configuration itself - changes in time. In other words, the computer will use part of its own resources to configure itself, meaning learning to configure its compute fabric in the most efficient, effective manner. This is not to say that programmers aren't smart enough to program that hard-coded logic in their softare, it is that the code for discovering those computational configurations is so much smaller, faster and adaptible for conditions unknown by the programmer a-priori. Finally, since we can't see what lies over the hill, we may use cycles of forward and inverse Bayesian reasoning in our configuration code to better prepare us (meaning the computational fabric) for what we might find when the time arrives and we get there. So no need for a foggy, old crystal ball. The only really scary thought is that you (Tom) and I could be twins ...

June 18, 2009 10:40 AM PDT wolfmurphy Total Points: 805 Status Points: 805 Black Belt

I enjoyed twins, I'd prefer to be Danny, especially with educational crisis in California augmented by Arnold. It will be an interesting to see what emerges. I haven't played much with genetic algorithms, but my friend Charlie has at Earlham college. Bayesian reasoning sounds interesting. Many people say I would benefit from improving my reasoning. My experience has been my algorithmic knowledge wins. For instance, we are running a student contest at Teragrid, where we've prepared students with challenges of the week. One was to calculate the circular primes less than a million. A prime is circular if every rotation of its digits are also a prime, such as 197, 971, and 719. A student wrote and compiled a code to generate them that ran in 300 seconds. I wrote a 26 line program in an interpreted language (Scheme) that generated all primes less than a million and then selected the circular ones. It ran in 7 seconds on my Mac. I am a CS geek depending on knowledge of data structures and computer architecture to tame problems. I suspect these will be the mainstay of CS for a long time to come, as they have been with all the other hardware and software improvements we've seen through the years. It is sometime scary to recognize a twin, and often delightful.

Trackbacks (0)

Leave a comment