### Scripts for building FEniCS on the Peregrine cluster at RUG ###
_forked from [fenics-gaia-cluster](https://bitbucket.org/unilucompmech/fenics-gaia-cluster)_
@ -9,10 +9,14 @@ This collection of scripts will automatically build current releases of
@@ -9,10 +9,14 @@ This collection of scripts will automatically build current releases of
### python 3 ###
Python 3 with [virtualenv](http://docs.python-guide.org/en/latest/dev/virtualenvs) is required.
This build uses `python 3.6.4` from the Lmod modules. All FEniCS related python modules will be installed within an virtual environment.
In particular, if you want to install different builds of FEniCS (different versions, packages (e.g., OpenMPI vs. MPICH, Intel MKL vs. OpenBLAS), or for different processor micro-architectures, see below), this will be helpful.
Here, [pew](https://github.com/berdario/pew) is used to manage the virtualenvs. To install `pew` follow these steps:
This build uses `python 3.6.4` from the Lmod modules.
All FEniCS related python modules will be installed within an virtual environment.
In particular, if you want to install different builds of FEniCS (different versions,
packages (e.g., OpenMPI vs. MPICH, Intel MKL vs. OpenBLAS), or for different processor
micro-architectures, see below), this will be helpful.
Here, [pew](https://github.com/berdario/pew) is used to manage the virtualenvs.
To install `pew` follow these steps:
```shell
$ module load Python/3.6.4-intel-2018a
$ pip3 install pew --user
@ -24,18 +28,22 @@ First clone this repository, for example to the location `$HOME/dev`.
@@ -24,18 +28,22 @@ First clone this repository, for example to the location `$HOME/dev`.
The folder `intel` contains the build scripts using the intel toolchain (compilers, impi, intel MKL). The folder `foss` contains build scripts using the foss toolchain (gcc, openblas, openmpi). Select the folder according to which toolchain you want to use.
The main file is `build_all.sh`. Modify it to set the FEniCS version to be used. The `$TAG` variable (can be changed) specifies the directories FEniCS and its dependencies are installed to and the name of the virtual environment.
The folder `intel` contains the build scripts using the intel toolchain (compilers, impi, intel MKL).
The folder `foss` contains build scripts using the foss toolchain (gcc, openblas, openmpi).
Select the folder according to which toolchain you want to use.
The main file is `build_all.sh`. Modify it to set the FEniCS version to be used. The `$TAG` variable
(can be changed) specifies the directories FEniCS and its dependencies are installed to and the name of the virtual environment.
It is recommended to set `continue_on_key=true` for the first build in order to check that each dependency was installed correctly!
The script calls a number of `build_xxx.sh` scripts which download and install the corresponding applications. Edit these files to change the version, compile flags, etc.
The calls can be commented if it was already built, e.g., if a number of programs was built correctly until an error occurred in, say, `build_dolfin.sh`, because of a network timeout.
Note that if you want to rebuild everything the `$PREFIX` and the `$BUILD_DIR` dirs should be deleted, also the python virtual environment with `pew rm $TAG`.
Note that if you want to rebuild everything the `$PREFIX` and the `$BUILD_DIR` dirs should be deleted,
also the python virtual environment with `pew rm $TAG`.
Make sure the environments are correctly set in `env_build.sh` and `setup_env_fenics.env`. Also revise the individual build files.
@ -46,21 +54,28 @@ In order to build FEniCS run
@@ -46,21 +54,28 @@ In order to build FEniCS run
```shell
$ ./build_all.sh |& tee -a build.log
```
on the compute node inside the `fenics-peregrine-build` directory.
on the compute node inside the `FEniCS-Peregrine/intel` directory.
**Remark** There is an outdated folder with foss compilers. Currently is it not supported and expected to be deprecated in the future.
Wait for the build to finish. The output of
the build will be stored in `build.log` as well as printed on the screen.
## Running FEniCS ##
To activate a FEniCS build you need to source the environment file created by the build script and activate the virtualenv. The corresponding lines are printed after building is completed.
To activate a FEniCS build you need to source the environment file created by the build script and activate the virtualenv.
The corresponding lines are printed after building is completed.
```shell
$ source <prefix>/bin/env_fenics_run.sh
$ pew workon fenics-<tag>
```
Now you can run python/ipython. `python -c "import dolfin"` should work. Try running `python poisson.py` and `mpirun -n 4 python poisson.py`.
Now you can run python/ipython. `python -c "import dolfin"` should work.
Try running `python poisson.py` and `mpirun -n 4 python poisson.py`.
**Remark** If installed successfully, the fenics environment created will be named *fenics-2019.1.0.post0-intel2018a*.
## Submit jobs to Peregrine (RUG) ##
To submit jobs to Peregrine, you need to provide a minimal configuration using sbatch, activate FEniCS environment and export the MPI library, e.g.
To submit jobs to Peregrine, you need to provide a minimal configuration using sbatch,
activate FEniCS environment and export the MPI library, e.g.
@ -24,9 +24,9 @@ echo "Downloading and building pybind11 $PYBIND11_VERSION"
@@ -24,9 +24,9 @@ echo "Downloading and building pybind11 $PYBIND11_VERSION"
reidmen
4 months ago