4,391 Posts served

10,712 Conversations started

Easy HPC for bioinformatics - Build your own cluster

By Paul Guermonprez (Intel) (2 posts) on May 29, 2008 at 4:38 am

Easy HPC for bioinformatics - Build your own cluster

Research and industry need huge amounts of computing power, especially in fields like life sciences, where the need to understand life mechanisms and cure diseases is endless. In a time where research is funded on a per project or lab basis, sharing large HPC (High Performance Computing) centers with a large number of users for small workloads is becoming complex. But you can install a cluster yourself in your company department or your research lab. The software tools now exist and we have the right hardware.

One lead is to train students, researchers and engineers to build their own cluster from off the shelf parts and open source tools. Not only to plug cables, but install the operating system, a cluster toolkit like OSCAR and some real life applications like mpiBLAST. You can find attached to this post pdf slides (under GNU Free Documentation License) and files for a 1 or 2 day course with many screenshots and detailed instructions. Starting from hardware you will (hopefully) have at the end a working cluster with running real life applications.

And because some applications require low latency communications, this setup was built with infiniband. If you are a HPC software developer, you will probably be interested in the Intel Cluster Tools part : have a data driven approach to diagnose bottlenecks and enhance performance.

Another complementary lead is to work closely with leading HPC Independent Software Vendors like Accelrys to standardize the deployment of their software stacks with the Intel Cluster Ready initiative. It is now easier to understand what hardware and software you need to run a specific software suite like Accelrys Materials Studio or Discovery Studio.

The goal is of course not to replace large HPC centers, on the contrary, if you are able to build a small cluster yourself, you will probably use a large one more efficiently and quickly. This training was presented in a previous version during a 3 day session for master students and researchers, together with Intel Software optimization courses. Feel free to contact me if you are interested (the documentation is released nuder GNU Free Documentation License).

Regards, paul

paul.guermonprez@intel.com

File : PDF Slides

File : Tarball of files needed

Categories: Academic, Intel SW Partner Program, Open Source, Parallel Programming, Software Tools
Tags: bioinformatics, cluster, cluster tools, hpc, life sciences, oscar

For more complete information about compiler optimizations, see our Optimization Notice.

Comments (8)

May 29, 2008 9:32 AM PDT Pierre

Cool, I want one ! :-)

June 3, 2008 11:50 AM PDT Paul Guermonprez (Intel) Total Points: 2,817 Brown Belt

now with file links ... and insidehpc.com post : http://insidehpc.com/2008/06/02/build-your-own-cluster-presentati on-from-intel

July 16, 2009 9:23 PM PDT Ankush Sharma

very nice website and very informative article . Thanks for the information regards Ankush Sharma

September 13, 2010 11:03 PM PDT bioinformatics training

I have also the opinion that everyone should have the knowledge to build their own cluster from off the shelf parts and open source tool because now is the timewhere technology is used in every field included pharmaceutical companies..so it is great.

September 15, 2010 5:25 AM PDT bioinformatics project training

Thanks for the information, I like the cluster making, and i am also trying to do this.

September 20, 2010 9:01 PM PDT bioinformatics training

Yah......... this training is really very nice and the conversation isdone is also very good .. I like it so much..

September 21, 2010 1:49 AM PDT jasmine25

very nice and informative article..i am also interested in cluster making and i will trying to do this

September 26, 2010 4:19 AM PDT maverick258 Total Points: 5 Registered User

oh thanks for giving this useful information...nice man...

Trackbacks (2)

• Bioinformatics » Post-doc position in Bioinformatics and Computational Systems ...
  July 26, 2009 1:37 AM PDT
• Twitter Trackbacks for Intel Software Network Blogs » Easy HPC for bioinformatics - Build your own cluster [intel.com] on Topsy.com
  August 31, 2009 4:30 PM PDT

Leave a comment