Build and configuration

External dependencies

To build EDDL you will need a C++11 compiler

If you want to compile with CUDA support, install:

  • NVIDIA CUDA 10 or above

  • NVIDIA cuDNN to accelerate primitfives (optional)

Note

CUDA 10 and 11 does not support GCC versions later than 8. (Ubuntu 20.04 comes with GCC 9.3.0 by default)

Also, we highly recommend installing an Anaconda environment to manage the external dependencies. You will get controlled dependency versions regardless of your system.

Once you have Anaconda installed, you can create and activate our environment by running the following commands from the source directory:

conda env create -f environment.yml
conda activate eddl

You can also update your environment with:

conda env update -f environment.yml

If you decide to manually install these dependencies in your system (make sure they are at standard paths):

# Conda versioning: https://docs.conda.io/projects/conda-build/en/latest/resources/package-spec.html
- cmake=3.19
- eigen=3.3|3.4
- protobuf==3.11.4
- libprotobuf==3.11.4
- zlib==1.2.11
- openssl==1.1.1i
- gtest=1.10
- graphviz=2.42|2.47 # Build & Run
- wget=1.20|1.16
- doxygen=1.9  # Docs
- python=3.8
- pip=21.0
- pip:  # Pip versioning: https://www.python.org/dev/peps/pep-0440/#version-specifiers
    - sphinx==4.3.*
    - sphinx_rtd_theme==1.0.*
    - sphinx-tabs==3.2.*
    - breathe==4.31.*
    - onnx-simplifier==0.3.*
    - protobuf==3.19.*

Note

  • You can double-check your dependency and versions using this reference conda list file: Requirements

  • When using apt-get, the installed version of the package depends on the distro version (by default). This is important to known, because for instance, on Ubuntu 18.04 apt-get install libeigen3-dev installs Eigen 3.3.4-4, when the EDDL needs Eigen 3.3.7.

Build and optimization

Build

To build the EDDL, you will need a recent version of cmake. Then, run the following commands from the source directory:

mkdir build/
cd build/
cmake ..
make install

Note

If you are using the conda environment to manage the dependencies, remember to activate it by typing: conda activate eddl

Backend support

You can choose the hardware for which the EDDL will be compiled. By default it is compiled for GPU but if it is not not found (or CUDA), the EDDL will automatically fallback to CPU.

  • CPU support: If you want to compile it for CPU, use the following cmake option:

-DBUILD_TARGET=CPU

Note

Backup option when cuDNN or CUDA is not found

  • GPU (CUDA) support: If you want to compile it for GPU (CUDA), use the following cmake option:

-DBUILD_TARGET=GPU  # or "CUDA", both are equivalent

Note

Fallback to CPU. To use a specific CUDA version you only need to specify the NVCC location: -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc

  • GPU (cuDNN) support: If you want to compile it for GPU (cuDNN), use the following cmake option:

-DBUILD_TARGET=CUDNN

Note

Enabled by default. If cuDNN is not installed, we will fallback to GPU (CUDA), or to CPU if CUDA is not installed. To use a specific CUDA version you only need to specify the NVCC location: -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc

  • FPGA support: If you want to compile it for FPGA, use the following cmake option:

-DBUILD_TARGET=FPGA

Additional flags

These flags can enable/disable features of the EDDL so that you can optimize and troubleshoot the compilation process (see: Troubleshoot).

  • Prefix path: Semicolon-separated list of directories specifying installation prefixes to be searched by the find_package(), find_program(), find_library(), find_file(), and find_path() commands.

-DCMAKE_PREFIX_PATH=/path/to/dir

Note

If using conda, get the path by activating the environment, and typing echo $CONDA_PREFIX

  • Installation paths: To change the installation paths, use the following cmake option:

-DCMAKE_INSTALL_PREFIX=/path/to/dir

Note

Defaults to /usr/local on UNIX and c:/Program Files on Windows. If using conda, get the path by activating the environment, and typing echo $CONDA_PREFIX

  • C/C++ compiler:

-DCMAKE_CXX_COMPILER=/path/to/c++compiler  # /usr/bin/g++-8
-DCMAKE_C_COMPILER=/path/to/c compiler  # /usr/bin/gcc-8

Note

The default compiler in MacOS has problems with OpenMP. We recommend to install either gcc or clang using brew.

  • CUDA compiler:

-DCMAKE_CUDA_COMPILER=/path/to/cuda compiler  #/usr/bin/nvcc

Note

This flag is needed to known which CUDA Toolkit the user wants to use. By default cmake looks in the PATH.

  • CUDNN ROOT DIR:

--DCUDNN_ROOT_DIR=/path/to/cuda  #/usr/local/cuda

Note

This flag is needed to known where to look for the cuDNN libraries. By default cuda is expected to be installed in along with the CUDA toolkit.

  • CUDA host compiler:

-DCMAKE_CUDA_HOST_COMPILER=/path/to/host compiler  # /usr/bin/g++-8

Note

You can also create a symbolic link: (unix) sudo ln -s usr/local/cuda-{VERSION} /usr/local/cuda

  • Eigen3: At the core of many numerical operations, we use Eigen3. If CMake is unable to find Eigen3 automatically, try setting Eigen3_DIR, such as:

-DEigen3_DIR=/path/to/eigen  # /usr/lib/cmake/eigen3

  • Use OpenMP: To enable/disabled OpenMP, use the setting BUILD_OPENMP, such as:

-DBUILD_OPENMP=ON

Note

Enabled by default. The default compiler in MacOS has problems with OpenMP. We recommend to install either gcc or clang using brew.

  • Use HPC: To enable/disabled HPC flags, use the setting BUILD_HPC, such as:

-DBUILD_HPC=ON

Note

Enabled by default. This enables flags such as: -march=native -mtune=native -Ofast -msse -mfpmath=sse -ffast-math -ftree-vectorize, that might cause some units tests to fail due to numerical errors (minor deviations from the value asserted)

  • Use protobuf: Protobuf allows you to use the ONNX import/export functions, to use them, use the setting BUILD_PROTOBUF, such as:

-DBUILD_PROTOBUF=ON

Note

Enabled by default

  • Build tests: To compile the tests, use the setting BUILD_TESTS, such as:

-DBUILD_TESTS=ON

Note

Enabled by default. The flag BUILD_HCP needs to be disabled. If not, some tests might not pass due to numerical errors.

  • Build examples: To compile the examples, use the setting BUILD_EXAMPLES, such as:

-DBUILD_EXAMPLES=ON

Note

Enabled by default

  • Build shared library: To compile the EDDL as a shared library:

-DBUILD_SHARED_LIBS=ON

Note

Enabled by default

  • Build target: Specifies the build type on single-configuration generators.

-DCMAKE_BUILD_TYPE=Release

Note

“Release” by default.

Possible values are empty, Debug, Release, RelWithDebInfo, MinSizeRel,… (Read more: here)

  • Superbuild: To let the EDDL manage its dependencies automatically:

-DBUILD_SUPERBUILD=ON

Note

Disabled by default. If OFF, cmake will look at your CMAKE_PREFIX_PATH

If you want to distribute the resulting shared library, you should use the flag -DBUILD_SUPERBUILD=ON so that we can make specific tunings to our dependencies.

  • Build distributed: To let the EDDL work in a distributed mode, use the setting BUILD_DIST:

-DBUILD_DIST=ON

Note

Enabled by default.