November 14, 2008 12:16 PM PST
Need "wrapper" to compile the Linux kernel with ICC
I am a developer with an idea to start a new kernel source for Gentoo that is specifically for the Intel ICC compiler. Evidentally there is a "wrapper" that is needed to be applied to the kernel source first before ICC can correctly complie the Linux kernel. Where can I get this wrapper? (I have looked all over the web but the only yhing I can find is old documentation)
You could find 'Linux kernel build white paper with 10.0 compiler' in below URLs. In the bottom of that page, there is another paper discussion the compatibility with GNU Compilers may be also useful for you.
You could find 'Linux kernel build white paper with 10.0 compiler' in below URLs. In the bottom of that page, there is another paper discussion the compatibility with GNU Compilers may be also useful for you.
I am a developer with an idea to stat a new kernel source for Gentoo that is specifically for the Intel ICC compiler. Evidentally there is a "wrapper" that is need to be applied to the kernel source first before ICC can correctly complie the Linux kernel. Wher can I get this wrapper? (I have looked all over the web but the only yhing I can find is old documentation)
-------- Nescire autem quid ante quam natus sis acciderit, id est semper esse puerum. Quid enim est aetas hominis, nisi ea memoria rerum veterum cum superiorum aetate contexitur?
I am a developer with an idea to stat a new kernel source for Gentoo that is specifically for the Intel ICC compiler. Evidentally there is a "wrapper" that is need to be applied to the kernel source first before ICC can correctly complie the Linux kernel. Wher can I get this wrapper? (I have looked all over the web but the only yhing I can find is old documentation)
Hi thaidog, were you asking for a wrapper for 2.6.xx kernel on IA32? I can share one for 10.x compiler with you. I didn't test with 11.0 compiler yet.
I tried to compile the linux kernel with ICC and got into a horrible mess. I think it is way too embedded with GCC for it to work but if you managed to get it working, I would be VERY interested.
If you need a hand, please ask.
-------- Nescire autem quid ante quam natus sis acciderit, id est semper esse puerum. Quid enim est aetas hominis, nisi ea memoria rerum veterum cum superiorum aetate contexitur?
Well I've got version 10.0. Not sure where to find 10.1. I got this error when trying to do it with 10.0:
gentoo_x86_32 ~ # cd /usr/src/linux gentoo_x86_32 linux # nano intelwrapper gentoo_x86_32 linux # make HOSTCC=intelwrapper CC=intelwrapper /usr/src/linux-2.6.25-gentoo-r7/scripts/gcc-version.sh: line 25: intelwrapper: command not found /usr/src/linux-2.6.25-gentoo-r7/scripts/gcc-version.sh: line 26: intelwrapper: command not found make: intelwrapper: Command not found CHK include/linux/version.h CHK include/linux/utsrelease.h HOSTCC scripts/basic/fixdep /bin/sh: intelwrapper: command not found make[1]: *** [scripts/basic/fixdep] Error 127 make: *** [scripts_basic] Error 2 gentoo_x86_32 linux #
EDIT: whoops... I did not put it in my path... that's what that error is.
I tried to compile the linux kernel with ICC and got into a horrible mess. I think it is way too embedded with GCC for it to work but if you managed to get it working, I would be VERY interested.
If you need a hand, please ask.
A hand would be great. I have version 10 not 10.1. Do you know where to get 10.1? I got the wrapper working... but after a few min it errored:
I successfully built a kernel (that is very close to RHEL5 kernel) with the above wrapper script. A few source code patches are needed. Could you please show me a few more lines in your build log, so that I can see what the problem is?
Let me try to list all patches you might need to apply to your kernel source code.
add the following line at the end of include/linux/compiler-intel.h. #undef __compiler_offsetof
arch/i386/kernel/apm.c, remove "static" staticstruct { unsigned long offset; unsigned short segment; } apm_bios_entry;
I successfully built a kernel (that is very close to RHEL5 kernel) with the above wrapper script. A few source code patches are needed. Could you please show me a few more lines in your build log, so that I can see what the problem is?
Let me try to list all patches you might need to apply to your kernel source code.
add the following line at the end of include/linux/compiler-intel.h. #undef __compiler_offsetof
arch/i386/kernel/apm.c, remove "static" staticstruct { unsigned long offset; unsigned short segment; } apm_bios_entry;
Makefile, add libirc_s.a as follow. --start-group $(vmlinux-main) /opt/intel/cc/10.1.021/lib/libirc_s.a --end-group \
I got the update to 10.1 but it died again (I have not had a chance to try those changes you added in the post above yet though) Here is the build log about 50 lines to the bottom:
arch/x86/boot/boot.h(174): remark #593: parameter "len" was set but never used static inline int memcmp(const void *s1, const void *s2, size_t len) ^
arch/x86/boot/boot.h(182): remark #593: parameter "s1" was set but never used static inline int memcmp_fs(const void *s1, addr_t s2, size_t len) ^
arch/x86/boot/boot.h(182): remark #593: parameter "s2" was set but never used static inline int memcmp_fs(const void *s1, addr_t s2, size_t len) ^
arch/x86/boot/boot.h(182): remark #593: parameter "len" was set but never used static inline int memcmp_fs(const void *s1, addr_t s2, size_t len) ^
arch/x86/boot/boot.h(189): remark #593: parameter "s1" was set but never used static inline int memcmp_gs(const void *s1, addr_t s2, size_t len) ^
arch/x86/boot/boot.h(189): remark #593: parameter "s2" was set but never used static inline int memcmp_gs(const void *s1, addr_t s2, size_t len) ^
arch/x86/boot/boot.h(189): remark #593: parameter "len" was set but never used static inline int memcmp_gs(const void *s1, addr_t s2, size_t len) ^
arch/x86/boot/main.c(66): remark #593: variable "ax" was set but never used u16 ax = 0x0305; ^
arch/x86/boot/main.c(67): remark #593: variable "bx" was set but never used u16 bx = 0; ^
arch/x86/boot/main.c(122): warning #1079: return type of function "main" must be "int" void main(void) ^
IPO: WARNING: File scope asm disables -ipo CC arch/x86/boot/mca.o icc: command line warning #10006: ignoring unknown option '-fno-toplevel-reorder' arch/x86/boot/boot.h(174): remark #593: parameter "s1" was set but never used static inline int memcmp(const void *s1, const void *s2, size_t len) ^
arch/x86/boot/boot.h(174): remark #593: parameter "s2" was set but never used static inline int memcmp(const void *s1, const void *s2, size_t len) ^
arch/x86/boot/boot.h(174): remark #593: parameter "len" was set but never used static inline int memcmp(const void *s1, const void *s2, size_t len) ^
arch/x86/boot/boot.h(182): remark #593: parameter "s1" was set but never used static inline int memcmp_fs(const void *s1, addr_t s2, size_t len) ^
arch/x86/boot/boot.h(182): remark #593: parameter "s2" was set but never used static inline int memcmp_fs(const void *s1, addr_t s2, size_t len) ^
arch/x86/boot/boot.h(182): remark #593: parameter "len" was set but never used static inline int memcmp_fs(const void *s1, addr_t s2, size_t len) ^
arch/x86/boot/boot.h(189): remark #593: parameter "s1" was set but never used static inline int memcmp_gs(const void *s1, addr_t s2, size_t len) ^
arch/x86/boot/boot.h(189): remark #593: parameter "s2" was set but never used static inline int memcmp_gs(const void *s1, addr_t s2, size_t len) ^
arch/x86/boot/boot.h(189): remark #593: parameter "len" was set but never used static inline int memcmp_gs(const void *s1, addr_t s2, size_t len) ^
arch/x86/boot/mca.c(29): error: register "ax" used more than once : "=acd" (err), "=acdSD" (es), "=b" (bx) ^
arch/x86/boot/mca.c(29): error: register "cx" used more than once : "=acd" (err), "=acdSD" (es), "=b" (bx) ^
arch/x86/boot/mca.c(29): error: register "dx" used more than once : "=acd" (err), "=acdSD" (es), "=b" (bx) ^
I don't have arch/x86/boot/main.c in my source code. It looks the error are coming from an inline assembly code. If you can extract that piece of code and create a small test case for me to reproduce the error, that'll be great.
If there is no way to create a smaller test case, could you please preprocess it with proper command line and upload the preprocessed code, so that I can look into it?
In order to extract the command line for arch/x86/boot/main.c, just issue "make HOSTCC=intelwrapper CC=intelwrapper V=1". It will print command lines along with various warnings & errors. Find the command line for arch/x86/boot/main.c and replace -c with -E. Re-run the command line and the compiler will print preprocessed C code.
I don't have arch/x86/boot/main.c in my source code. It looks the error are coming from an inline assembly code. If you can extract that piece of code and create a small test case for me to reproduce the error, that'll be great.
If there is no way to create a smaller test case, could you please preprocess it with proper command line and upload the preprocessed code, so that I can look into it?
In order to extract the command line for arch/x86/boot/main.c, just issue "make HOSTCC=intelwrapper CC=intelwrapper V=1". It will print command lines along with various warnings & errors. Find the command line for arch/x86/boot/main.c and replace -c with -E. Re-run the command line and the compiler will print preprocessed C code.
Are these the lines you are talking about?
arch/x86/boot/main.c(66): remark #593: variable "ax" was set but never used u16 ax = 0x0305; ^
arch/x86/boot/main.c(67): remark #593: variable "bx" was set but never used u16 bx = 0; ^
arch/x86/boot/main.c(122): warning #1079: return type of function "main" must be "int" void main(void) ^ ############################################
I did a: cat arch/x86/boot/main.c but I did not see a -c to replace with the -E
Maybe I am not understanding you correcty?
EDIT: If you would like I can give you access to the development box via ssh and sudo acct?
You need to do "make HOSTCC=intelwrapper CC=intelwrapper V=1". In the build log, try to find a command line like
intelwrapper <a lot of compiler options> arch/x86/boot/main.c
change -c to -E in the above command line and append > main-prep.c at the end. Re-run it. main-prep.c should contain preprocessed source code, which is what I need.
intelwrapper <a lot of compiler options> -E arch/x86/boot/main.c > main-prep.c
You need to do "make HOSTCC=intelwrapper CC=intelwrapper V=1". In the build log, try to find a command line like
intelwrapper <a lot of compiler options> arch/x86/boot/main.c
change -c to -E in the above command line and append > main-prep.c at the end. Re-run it. main-prep.c should contain preprocessed source code, which is what I need.
intelwrapper <a lot of compiler options> -E arch/x86/boot/main.c > main-prep.c
Thanks,
Feilong
I do not see that line from your command - here is the output:
gentoo_x86_32 bin # cd /usr/src/linux gentoo_x86_32 linux # make HOSTCC=intelwrapper CC=intelwrapper V=1 /usr/src/linux-2.6.25-gentoo-r7/scripts/gcc-version.sh: line 25: intelwrapper: command not found /usr/src/linux-2.6.25-gentoo-r7/scripts/gcc-version.sh: line 26: intelwrapper: command not found make: intelwrapper: Command not found rm -f include/config/kernel.release echo 2.6.25-gentoo-r7 > include/config/kernel.release set -e; :; mkdir -p include/linux/; (echo \#define LINUX_VERSION_CODE 132633; echo '#define KERNEL_VERSION(a,b,c) (((a) << 16) + ((b) << 8) + (c))';) < /usr/src/linux-2.6.25-gentoo-r7/Makefile > include/linux/version.h.tmp; if [ -r include/linux/version.h ] && cmp -s include/linux/version.h include/linux/version.h.tmp; then rm -f include/linux/version.h.tmp; else :; mv -f include/linux/version.h.tmp include/linux/version.h; fi set -e; :; mkdir -p include/linux/; if [ `echo -n "2.6.25-gentoo-r7" | wc -c ` -gt 64 ]; then echo '"2.6.25-gentoo-r7" exceeds 64 characters' >&2; exit 1; fi; (echo \#define UTS_RELEASE \"2.6.25-gentoo-r7\";) < include/config/kernel.release > include/linux/utsrelease.h.tmp; if [ -r include/linux/utsrelease.h ] && cmp -s include/linux/utsrelease.h include/linux/utsrelease.h.tmp; then rm -f include/linux/utsrelease.h.tmp; else :; mv -f include/linux/utsrelease.h.tmp include/linux/utsrelease.h; fi set -e; asmlink=`readlink include/asm | cut -d '-' -f 2`; \ if [ -L include/asm ]; then \ if [ "$asmlink" != "x86" ]; then \ echo "ERROR: the symlink include/asm points to asm-$asmlink but asm-x86 was expected"; \ echo " set ARCH or save .config and run 'make mrproper' to fix it"; \ exit 1; \ fi; \ else \ echo ' SYMLINK include/asm -> include/asm-x86'; \ if [ ! -d include ]; then \ mkdir -p include; \ fi; \ ln -fsn asm-x86 include/asm; \ fi mkdir -p .tmp_versions ; rm -f .tmp_versions/* make -f scripts/Makefile.build obj=scripts/basic (cat /dev/null; ) > scripts/basic/modules.order make -f scripts/Makefile.build obj=. (cat /dev/null; ) > modules.order mkdir -p arch/x86/kernel/ intelwrapper -Wp,-MD,arch/x86/kernel/.asm-offsets.s.d -nostdinc -isystem -D__KERNEL__ -Iinclude -include include/linux/autoconf.h -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Os -msoft-float -mregparm=3 -freg-struct-return -march=i686 -ffreestanding -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -Iinclude/asm-x86/mach-generic -Iinclude/asm-x86/mach-default -fno-omit-frame-pointer -fno-optimize-sibling-calls -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(asm_offsets)" -D"KBUILD_MODNAME=KBUILD_STR(asm_offsets)" -fverbose-asm -S -o arch/x86/kernel/asm-offsets.s arch/x86/kernel/asm-offsets.c /bin/sh: intelwrapper: command not found make[1]: *** [arch/x86/kernel/asm-offsets.s] Error 127 make: *** [prepare0] Error 2 gentoo_x86_32 linux #
I looked at http://www.tyler.mcadams.com/build.txt. It looks that the error occured at arch/x86/boot/mca.c. I copied the command line and changed -c to -E. Please run the following command line and upload mca-prep.c for me.
I looked at http://www.tyler.mcadams.com/build.txt. It looks that the error occured at arch/x86/boot/mca.c. I copied the command line and changed -c to -E. Please run the following command line and upload mca-prep.c for me.
Only half way done (obviously) so far but the benchmarks with gcc are already smashing Windows! What do you think?
here is a screenhot of what could be the gui: http://www.tyler.mcadams.com/optimized.png
I will give you full credit for all your gracious help on this project! I just talked with Fabio Erculiani the inventor of Sabayon Linux and he is very interested in this little project also. (Sabayon is a Getnoo Linux based distro)
It looks that the problem is coming from the inline assembly code in the function query_mca(). Could you please upload the original arch/x86/boot/mca.c (don't preprocess) and the object file compiled with gcc (arch/x86/boot/mca.o)? Probably we need to do some small modification to that piece of inline asm code.
I didn't keep the RH kernel built with icc. It will take some time if I need to re-compile one. And will it fit into your OS? I don't think so...
Also, if you get a chance, can you post the older redhat kernel that you got to compile? I'd like to download it and try it also if possible.
thaidog, I hope it doesn't hurt to ask one obvious thing -- have you succesfully compiled the kernel with gcc on your computer before trying to compile with icc? Do you use patched gentoo kernel source or you are compiling vanilla kernel source tree? Which gcc and glibc versions you use? Perhaps that info will help.
-------- If you find my post helpfull, please rate it and/or select it as a best answer where applies. Thank you.
Also, if you get a chance, can you post the older redhat kernel that you got to compile? I'd like to download it and try it also if possible.
thaidog, I hope it doesn't hurt to ask one obvious thing -- have you succesfully compiled the kernel with gcc on your computer before trying to compile with icc? Do you use patched gentoo kernel source or you are compiling vanilla kernel source tree? Which gcc and glibc versions you use? Perhaps that info will help.
Oh yeah... I've got kernel after kernel that I have been optimizing with gcc 4.3.2 ad 4.3.1. I think I'll zip the /usr/src/linux up so you guys and check it out your self. One thing I did though is over the weekend I downloaded the latest vanilla-sources version in an attmept to see if it was just the gentoo-sources patches that was screwing everything up, but they did not compile with ICC either... I could have missed something though. I'll zip up the kernel-2.6.25-7 with all the mods we made to it so that you guys have better access to it. Right now it is pretty much striped bare... I dorpped everything I could in an attempt to see if "bleeding edge" stuff was causing issues. this is it: www.tyler.mcadams.com/linux-2.6.25-gentoo-r7.tar.gz It's 12:15EST - give it about 20min to upload. It has the current .config and all the previous changes we made to the verious files to see if that woud help as well. If you want an particular kernel for me to build with gcc or any already built I can give those too if it would help.