Compiling programs on Big Red at IU

Two compilers are available on Big Red at Indiana University, the Gnu compilers (GCC) and IBM compilers.

On this page:

GCC

  • Presently at version 3.3.3

  • Works the same as GCC everywhere else

  • For 64-bit compilations, use the -mpowerpc64 compiler switch.

  • Other handy compiler options for Big Red include:

    -mcpu=970 Identifies our processor type
    -mabi=altivec Needed to enable the next switch (-maltivec)
    -maltivec For C codes using 4-byte real numbers, will attempt to vector stream the computations
    -mfused-madd Turns on the IBM-specific "multiply-add" instruction during optimizations (where possible)

IBM compilers

  • Generally compile faster executables than GCC for the same level of optimization

  • A little stricter regarding standards than GCC

  • One basic compiler is available in two packages, with different front ends for each language (C, C++, Fortran variants). Therefore, many optimization switches are the same across all supported languages.

For the compilers listed below, the following types of switches are discussed:

  • Machine switches: These switches are always the same for the platform.

  • User switches: These switches vary with the job. In general, switches prefixed with -q are sent to the compiler, while those prefixed with -b are sent to the loader. Some switches (for example, -E and -o) are exceptions due to historical precedents.

  • Optimization switches: These are the switches most people think about when they think about compiler switches. Optimizations can be both positive (faster executables) or negative (slower executables). Some switches, like -Q, are almost always positive (-Q attempts to place functions in line with the main body and remove the overhead of the function call). Others, like -g, are almost always negative (-g creates debug symbol tables). Various levels of -O are offered. Without a number following, -O defaults to -O2. Generally, the higher the number, the more risks the compiler is willing to take. At -O0, absolutely no optimization occurs. At -O5, everything from loop fusion to cross-source code optimizations (for multiple source code files) is considered. Often, more is not better. Testing and experimenting is generally the only way to properly tune an executable.

C compiler

The compiler is called xlc. A CC compiler is also available. On AIX systems, it's embedded in the OS; results can be unpredictable. On Big Red, CC has been soft-linked to GCC. The recommended C compiler is xlc.

  • Machine switches: For Big Red, -qarch=ppc970 -qtune=ppc970 -qenablevmx -qaltivec is a good choice. If the altivec processors are unusable, there is no penalty for setting the switch.

  • User switches: If addressing is expected above 2GB, a 64-bit compile will be needed with the -q64 switch. If you want to use only the C preprocessor, use the -E switch.

On Big Red, even if you have MANPATH problems, you can use the following command to see the xlc man page for other options:

man -M /opt/ibmcmp/vacpp/8.0/man/en_US/ xlc

C++ compiler

The IBM compiler for C++ codes is called xlC. This compiler used to have real trouble with many C++ source codes. It has improved (as has C++ standardization) quite a bit in the last several years. Most of the switches available to xlc are also available with xlC. In addition, many switches have been added to help with GCC compatibility. One example is -qlanglvl=gnu_complex. This switch instructs the compiler to recognize GNU complex data types and related keywords.

The man path is the same as that for xlc:

man -M /opt/ibmcmp/vacpp/8.0/man/en_US/ xlC

Fortran compiler

Fortran IV (66) and Fortran 77 are no longer supported. Use xlf90 -qfixed instead. Dropped syntax includes a few "computed goto" options.

Support for Fortran 2003 is nearly complete.

  • Machine switches: -qarch=ppc970 -qtune=ppc970 is the same as in xlc and xlC. However, the altivec switches are not directly available. C routines that are altivec-enabled (such as FFTW) are linked to the Fortran programs.

  • User switches: For Fortran, UITS highly recommends a 64-bit compile in most cases, to take advantage of the memory management functionality available in Fortran. 64-bit compilations are designated with the -q64 switch. However, additional "safety" switches, such as -C for testing array parameters, are also available.

  • Optimization switches: The xlf90 optimization switches are nearly identical to the xlc optimizations. Switches like -Q are almost always positive and switches like -g are almost always negative. The same levels of -O are offered but with slightly different effects. One throttle on risky optimizations is the -qstrict option. This is available only at -O3 and higher. Generally, peace of mind comes at a significant performance cost. The warning messages are very prolific. Again, testing and experimenting is generally the only way to properly tune an executable.

On Big Red, even if you have MANPATH problems, you can use the following command to see the xlf90 man page for other options: man -M /opt/ibmcmp/xlf/10.1/man/en_US/ xlf90

Threads and system math libraries

On all compilers, adding a _r suffix enables threads, for example, xlC_r. Often threads need enabling that runs behind the scenes.

An example is compiling and linking in IBM's Engineering and Scientific Subroutine Libraries (ESSL). If you are using actual threads (like OpenMP or POSIX), the compile switch is -qesslsmp. Even serial code requires the _r suffix, as in xlf90_r -q64 -qessl... or xlc_r -qessl... . For xlC_r, you must add -lessl -qnocinc=/usr/include/essl as well to redefine the include files.

This document was developed with support from the National Science Foundation (NSF) under Grant No. 0503697 to the University of Chicago and subcontracted to Indiana University. Additional support was provided by IU through its participation in the TeraGrid, which is supported by the NSF under Grants No. 0833618, SCI451237, SCI535258, and SCI504075. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.