If you just want to use tensorflow, nothing can be easier than running pip
install tensorflow to install it. If for any reason you need to recompile it
from source code (in Linux), this is what to do.
Dependencies
You will depend on some libraries and tools. Most importantly, you need to add CUDA sources from nVidia. But for Debian, you won’t get as much CUDA packages as Ubuntu. But it is fine, you can still install packages for Ubuntu 20.04 on Debian 11. Hence just add Ubuntu repository to the apt-get system.
The packages to install are
apt install bazel bazel-3.7.2
apt install libnvinfer-dev libnvinfer-plugin-dev libnccl-dev
Source
To get tensorflow source, you can simply
git clone https://github.com/tensorflow/tensorflow
But remember to check out the tag for the version to compile. For example,
git checkout v2.7.0
Build process
First, we need to run
./configure
and set to use CUDA, TensorRT, but do not use clang as it seems can’t compile
successfully in Debian. If not the case, remember to symlink python to
python3 as it is required for the build script.
Then the actual compilation is from the following command:
TMP=/tmp bazel build --config=mkl --config=cuda --config=opt --verbose_explanations --verbose_failures --jobs=6 //tensorflow/tools/pip_package:build_pip_package
In my system of 16GB memory, we need --jobs=6 or the compilation will run out of memory by having too many jobs running concurrently.
The compilation will take hours to complete. Afterwards, we can verify the executable is built with
ls ./bazel-bin/tensorflow/tools/pip_package/build_pip_package
and then we can run the following to build the wheel package (add
--nightly_flag if not on a tagged version):
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
which the wheel package is stored at /tmp/tensorflow_pkg and we can install it with pip install
After this is done, we can clean up the file generated by the build process at
~/.cache, which takes around 20GB in space.
References
It is useful to reference to the following official documentation on how to build Tensorflow:
- https://www.tensorflow.org/install/gpu#linux_setup
- https://www.tensorflow.org/install/source