Opticks Externals
====================
.. contents:: Table of Contents
:depth: 2
Three type of externals : system, foreign and automated
----------------------------------------------------------
1. **system** externals: NVIDIA GPU Driver, NVIDIA CUDA, NVIDIA OptiX must be installed
following the instructions from NVIDIA and assistance from your system administrator.
The envvars OPTICKS_CUDA_PREFIX and OPTICKS_OPTIX_PREFIX exported from `~/.opticks_config` communicate
the install locations to Opticks.
2. **foreign** externals are no longer automatically installed by `opticks-full` in order to facilitate integration of Opticks with detector
simulation frameworks.
The location of these foreign externals is communicated to the Opticks CMake machinery (`om-cmake` and `om-cmake-okconf`)
via the CMAKE_PREFIX_PATH envvar. This envvar and other PATH envvars such as LD_LIBRARY_PATH are
appended and exported by the `~/.opticks_config` lines starting `opticks-prepend-prefix`
These foreign externals can optionally be installed using::
opticks-foreign # list them, currently : boost, clhep, xercesc, g4
opticks-foreign-install
3. **automated** externals are automatically installed by `opticks-full`. To list them::
opticks-externals # currently: bcm, glm, glfw, glew, gleq, imgui, plog, nljson
Brief description of foreign and automated externals
------------------------------------------------------
================= ===================== ==============================================================================
precursor pkg name notes
================= ===================== ==============================================================================
bcm- BCM My fork of Boost CMake Modules, which eases use of modern CMake target import/export
glm- GLM OpenGL mathematics, 3D transforms
glfw- GLFW Interface between system and OpenGL, creating windows and receiving input
glew- GLEW OpenGL extensions loading library, cmake build didnt work, includes vc12 sln for windows
gleq- GLEQ Keyboard event handling header from GLFW author, header only
imgui- ImGui OpenGL immediate mode GUI, depends on glfw and glew
plog- PLog Header only logging, supporting multi dll logging on windows
xercesc XercesC XML handling dependency of Geant4, required for GDML parsing
g4- Geant4 The preeminent simulation toolkit
================= ===================== ==============================================================================
Separate installation of externals : useful for debugging the build
----------------------------------------------------------------------
The *opticks-externals* function lists current precursor names, *opticks-externals-install* runs each
of the precursor functions in turn. To rerun a single external install, use the below pattern of running
the precursor function and then the installer function.
::
glew-
glew--
After installation has been done rerunning *opticks-externals-install* completes quickly,
and does no harm.
Foreign Externals
------------------
Listing the foreign externals with bash function **opticks-foreign**::
epsilon:opticks blyth$ opticks-foreign
boost
clhep
xercesc
g4
What these do:
boost
system, program_options, filesystem, regex
clhep
optionally needed by geant4
xercesc
XML parsing needed by g4 GDML functionality
g4
geant4
Automated Externals
--------------------
Listing the automated externals with bash function **opticks-externals**::
epsilon:opticks blyth$ opticks-externals
bcm
glm
glfw
glew
gleq
imgui
plog
What these do is described in the below sections.
Base externals
----------------
bcm
boost CMake modules, target export/import for CMake 3.5+
allows config to direct dependencies only, the rest of the tree
gets configured automatically
glm
vector, matrix, 3D projection mathematics
plog
logging
Visualization externals
-------------------------
glfw
cross platform OpenGL and system events : keyboard, mouse
gleq
event queue for glfw
glew
OpenGL extension wrangler, providing access to OpenGL symbols
imgui
immediate mode OpenGL GUI
Boost C++ Libraries
----------------------
The Boost components listed in the table need to be installed.
These are widely available via package managers. Use the standard one for
your system. The FindBoost.cmake provided with cmake is used to locate the installation.
===================== =============== ============= ==============================================================================
directory precursor pkg name notes
===================== =============== ============= ==============================================================================
boost boost- Boost components: system thread program_options log log_setup filesystem regex
===================== =============== ============= ==============================================================================
The recommended minimum boost version is 1.53 as that is what I am using.
You might be able to survive with an earlier version,
but anything before 1.41 is known not to work.
Updating Boost
~~~~~~~~~~~~~~~~
If your version of Boost is not recent enough the cmake configuring
step will yield errors like the below.::
CMake Error at /home/blyth/local/env/tools/cmake/cmake-3.5.2-Linux-x86_64/share/cmake-3.5/Modules/FindBoost.cmake:1657 (message):
Unable to find the requested Boost libraries.
Boost version: 1.41.0
If possible use your system package manager to update Boost. If that is
not possible then do a local Boost install. Opticks includes bash functions
starting *boost-* that can get and install Boost locally.
Opticks Pre-requisites : NVIDIA OptiX and NVIDIA CUDA
-----------------------------------------------------------
OptiX requires your system to have a fairly recent NVIDIA GPU of CUDA compute capability 3.0 at least.
To download OptiX you need to join the NVIDIA Developer Program.
Use the links in the table to register, it is free but may take a few days to be approved.
Follow the NVIDIA instructions to download and install CUDA and OptiX.
Thrust is installed together with CUDA.
===================== =============== ============= ==============================================================================
directory precursor pkg name notes
===================== =============== ============= ==============================================================================
cuda cuda- CUDA https://developer.nvidia.com/cuda-downloads (includes Thrust)
optix optix- OptiX https://developer.nvidia.com/optix
===================== =============== ============= ==============================================================================
CUDA installation guides:
* http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
* http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html
Finding CUDA
~~~~~~~~~~~~~
Opticks uses the `FindCUDA.cmake` supplied by CMake to, eg
on macOS at `/opt/local/share/cmake-3.12/Modules/FindCUDA.cmake`.
Quoting from that::
29 # The script will prompt the user to specify ``CUDA_TOOLKIT_ROOT_DIR`` if
30 # the prefix cannot be determined by the location of nvcc in the system
31 # path and ``REQUIRED`` is specified to :command:`find_package`.
Thus check that `nvcc` is in your PATH, and preferably compile some CUDA examples
on your system before installing Opticks.::
epsilon:opticks blyth$ which nvcc # macOS
/Developer/NVIDIA/CUDA-9.1/bin/nvcc
[blyth@localhost ~]$ which nvcc # Linux
/usr/local/cuda-9.2/bin/nvcc
Versions of CUDA and OptiX
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I recommend you start your installation attempt with OptiX 6.5
together with the version of CUDA that it was built against, as stated in
the OptiX release notes.
This version pinning between CUDA and OptiX is because Opticks links against
both the OptiX library and the CUDA runtime.
If you cannot use the latest CUDA (because of kernel incompatibility) you will need to
use an older OptiX version contemporary with the CUDA version that your kernel supports.
Version combinations that have been used:
*current*
CUDA 10.1, OptiX 6.5.0
*previous*
CUDA 9.1, OptiX 5.1.1
CUDA 9.1, OptiX 5.1.0
*earlier*
CUDA 7.0, OptiX 3.80
The *current* version combination is regularly tested, the *previous* one
relies on your bug reports https://groups.io/g/opticks/topics to keep it working.
Any issues with *earlier* version combinations will not be addressed.
The reason for the extremes of caution regarding version combinations of drivers
is that the interface to the GPU is via kernel extensions where if anything goes
wrong there is no safety net. A bad kernel extension will cause kernel panics,
your machine crashes and continues to crash until the bad driver is removed
(on macOS the removal can be done by resetting NVRAM).
NVIDIA Driver Versions
~~~~~~~~~~~~~~~~~~~~~~~~
======== =============== =================== ============================================ ===================================
OptiX Date Driver (Linux) Working Problems Reported
======== =============== =================== ============================================ ===================================
6.5.0 Aug 26, 2019 435.17 435.21 CUDA 10.1 TITAN RTX, TITAN V Sajan: 440.33.01 and CUDA 10.2
7.0.0 July 29, 2019 435.12
6.0.0 Feb 2018 418.30
======== =============== =================== ============================================ ===================================
The release notes from every version of OptiX states the
required minimum version of the NVIDIA Driver that must be used
for that version of OptiX. In recent releases that driver version
has been from the so called short-lived series.
From https://www.nvidia.com/en-gb/drivers/unix/ on April 29, 2020::
Latest Long Lived Branch version: 440.82
Latest Short Lived Branch version: 435.21
Note that the long-lived series may have version numbers that exceed those
of the short-lived series but the features needed for OptiX take much longer
to appear in that series.
The releases from the longer-lived driver branches are intended for
users who do not need the latest and greatest features.
If you cannot change your driver version this sometimes means that
an older version of OptiX must be used to work with your driver.
OptiX 6.5.0 (August 26, 2019)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Quote from release notes::
OptiX 6.5.0 requires that you install the 436.02 driver on Windows or the 435.17 Driver for linux. Operating System:
* Windows 7/8.1/10 64-bit
* Linux RHEL 4.8+ or Ubuntu 10.10+ 64-bit
Problems with OptiX 6.5.0, Driver 440.33.01, CUDA 10.2
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* https://www.nvidia.com/en-gb/drivers/unix/
Search the OptiX forum for "driver" and looking for Linux reports:
* https://forums.developer.nvidia.com/search?q=driver%20%20category%3A167
* https://forums.developer.nvidia.com/t/optix7-and-game-ready-driver-440-97/83907
From droettger of NVIDIA on Nov 20, 2019 regarding a problem with optix7::
Yes, there was a bug in R440 drivers and a serious test escape.
This has been fixed in the meantime. The just released Windows driver 441.28 has picked up the fix already.
Unfortunately Linux 440.31 drivers have been cut before the fix. The next 441 Linux drivers should have it.
If you already hacked that integer field to 0 when preTransform is null, the expected value when preTransform is actually containing 3x4 matrices is 0x21E1.
OptiX 7.0.0 (July 29, 2019)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
**An entirely new API : not yet supported by Opticks.**
Quote from release notes::
OptiX 7.0.0 requires that you install the 435.80 driver on Windows or the 435.12 Driver for linux.
Note OptiX dll from the SDK are no longer needed since the symbols are loaded from the driver.
* Windows 7/8.1/10 64-bit;
* Linux RHEL 4.8+ or Ubuntu 10.10+ 64-bit
OptiX 6.0.0 (February 2018)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
**first version of OptiX with support for Turing GPUs and RT Cores**
Quote from release notes::
Graphics Driver:
* Windows: driver version 418.81 or later is required.
* Linux: driver version 418.30 or later is required.
OS:
* Windows 7/8.1/10 64-bit
* Linux RHEL 4.8+ or Ubuntu 10.10+ 64-bit
Using a non-standard OptiX version
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Opticks tends to adopt new OptiX versions very soon after they become
available approximately twice per year. This is because OptiX continues
to improve rapidly. Updating OptiX typically also requires an update of
the CUDA version and the NVIDIA driver.
Sometimes due to the suitable NVIDIA driver not yet being installed by
your system admin it is necessary to use older OptiX+CUDA versions.
Opticks aims to allow this for recent version combinations only.
Building Opticks against "foreign" externals such as Geant4, Boost
-------------------------------------------------------------------
When integrating Opticks with a detector simulation framework
it is important that externals that are in common between Opticks and the framework
are one and the same to avoid symbol inconsistency between different versions of libraries.
The most likely packages to be in common are::
Boost
Geant4
XercesC
GLEW
The Opticks build is sensitive to the CMAKE_PREFIX_PATH envvar
allowing Opticks to build against "foreign" externals. To check which external that
the CMake based build will pick use the find_package.py script::
epsilon:opticks blyth$ find_package.py Geant4
Geant4 : /usr/local/foreign/lib/Geant4-10.5.1/Geant4Config.cmake
Geant4 : /usr/local/opticks/externals/lib/Geant4-10.4.2/Geant4Config.cmake
If you can integrate Opticks with your framework using CMake then the non-CMake
opticks-config system which is based on pkg-config pc files is not relevant to you.
Conversely if you need to integrate with legacy build systems such as CMT
then it is necessary to arrange consistency between the two config systems.
To check which external that non-CMake pkg-config based builds will pick use the
pkg_config.py script::
epsilon:~ blyth$ pkg_config.py Geant4
geant4 : /usr/local/foreign/lib/pkgconfig/geant4.pc
geant4 : /usr/local/opticks/externals/lib/pkgconfig/geant4.pc
And also directly with opticks-config (or shorthand oc)::
opticks-config --cflags Geant4
opticks-config --help
To keep consistency between the CMake and pkg-config configuration systems it is
necessary for do several things:
1. ensure that the original CMAKE_PREFIX_PATH and PKG_CONFIG_PATH are consistent, for example::
export CMAKE_PREFIX_PATH=/usr/local/foreign
export PKG_CONFIG_PATH=/usr/local/foreign/lib/pkgconfig
2. ensure that pc files are present for relevant packages in the lib/pkgconfig
or lib64/pkgconfig directories beneath all relevant prefix dirs, for example::
/usr/local/foreign/lib/pkgconfig/geant4.pc
/usr/local/foreign/lib/pkgconfig/boost.pc
3. generate any missing pc files with::
g4-pcc-all
boost-pcc-all
These use find_package.py which iterates over prefixes in CMAKE_PREFIX_PATH
writing .pc files.
4. check consistency with::
find_package.py Boost
pkg_config.py Boost
5. do cleaninstalls following changes to CMAKE_PREFIX_PATH and PKG_CONFIG_PATH with::
cd ~/opticks
om-
om-cleaninstall
Testing CUDA and OptiX Installs and nvcc toolchain
-------------------------------------------------------
Before trying to install Opticks check your CUDA and OptiX installs:
1. run the precompiled CUDA and OptiX sample binaries
2. compile the CUDA and OptiX samples
3. run your compiled samples
Testing Thrust
----------------
Thrust provides a higher level C++ template approach to using CUDA that is used extensively
by Opticks. The Thrust headers are installed by the CUDA toolkit installater, eg at `/usr/local/cuda/include/thrust`.
You are recommended to try some of the Thrust examples to check your nvcc toolchain.
* http://docs.nvidia.com/cuda/thrust/index.html
* https://github.com/thrust/thrust/tree/master/examples
Geant4
---------
Geant4 is no longer automatically installed by *opticks-full* it can be installed with *g4--*.
The *g4-* precursor selects a version of Geant4. Currently a bit dated, this is intended to be brought uptodate sometime.
The coupling between Opticks and Geant4 is intended to be weak : so a range of
recent versions of Geant4 are intended to be supported.