Best way to build large applications in parallel

Best way to build large applications in parallel

Hello everyone,
At the HPC site where I worked before joining Intel, I used a program called pmake to build applications in parallel. Pmake worked on distributed-memory parallel systems. One of the proprietary UNIX systems also had a regularmake program with a -P option for parallel builds.

What is the best way to build large applications in parallel on a Linux cluster? A Google search on 'pmake' turned up a couple of candidates: Parallel Make and Berkeley Pmake. Does anyone have other recommendations?

Thanks,
Henry

4 posts / novo 0
Último post
Para obter mais informações sobre otimizações de compiladores, consulte Aviso sobre otimizações.

Henry -

The GNU make utility allows for parallel compilation with the "-j" flag.

gmake -j 2 MyApp will spawn off two threads to divide up the building of multiple files and libraries. Unfortunately, this only works within a single node, not across a cluster.

--clay

Message Edited by ClayB on 08-24-2004 10:55 AM

I'm also interested in this, but on Windows. We are using the cons
build system, which has a pcons flavor http://www.baldmt.com/parallel-cons/.
We are not using pcons yet, and also might need a better cluster file system
than just a share on windows.

There is also the newer http://www.scons.org/build system that supports parallel build on Linux and Windows.

I think there has been cluster versions of either cons or scons, but I cannot
find them now.

cons/scons have the advantage over make that dependencies are automatically generated, including what libs to build before what
execiutables and what source code needs to be generated before
compilation starts. It also handles changes using MD5 checksums
which means that things will rebuild properly even if time stamp
of file doesn't change or goes backwards (like if you undo a version
control checkout).

Jonas

I believe you are looking for:

http://distcc.samba.org/

It is very easy to setup and use.

RPMS for RHEL or CentOS can be found at:

http://dag.wieers.com/packages/distcc/

Deixar um comentário

Faça login para adicionar um comentário. Não é membro? Inscreva-se hoje mesmo!