How to install Tensorflow GPU with CUDA 10.0 for python on Windows

This is going to be a tutorial on how to install tensorflow 1.12 GPU version. We will also be installing CUDA 10.0 and cuDNN 7.3.1 along with the GPU version of tensorflow 1.12. At the time of writing this blog post, the latest version of tensorflow is 1.12. This tutorial is for building tensorflow from source. If you want to use the official pre-built pip package instead, I recommend another post, How to install Tensorflow using official pip package. If you are using Ubuntu instead of Windows, you may want to refer to our another article, How to install Tensorflow GPU with CUDA 10.0 for python on Ubuntu.  Tensorflow is an open source software library developed and used by Google that is fairly common among students, researchers, and developers for deep learning applications such as neural networks. It has both the CPU as well as GPU version available and although the CPU version works quite well, realistically, if you are going for deep learning, you will need GPU. In order to use the GPU version of TensorFlow, you will need an NVIDIA GPU with a compute capability > 3.0. Using latest version of Tensorflow provides you latest features and optimization, using latest CUDA Toolkit provides you speed improvement with latest gpu support and using latest CUDNN greatly improves deep learing training time. There must be 64-bit python installed tensorflow does not work on 32-bit python installation. Goto here for instruction on how to install python 3.6 on Windows. Also make sure python is set to %PATH% environment. Also make sure if you have enough ~ 15 GB of free space in C drive. You can use another drive as well but need to change path.

Step 1: Verify you have a CUDA-Capable GPU:

Before doing anything else, you need to verify that you have a CUDA-Capable GPU in order to install Tensorflow GPU. You can verify that you have a CUDA-capable GPU through the Display Adapters section in the Windows Device Manager. Here you will find the vendor name and model of your graphics card(s). The Windows Device Manager can be opened via the following steps: Open a run window from the Start Menu or (Win+R) Run:
control /name Microsoft.DeviceManager
Then goto Display Adapters. If your graphics card is from NVIDIA then go to http://developer.nvidia.com/cuda-gpus and verify if listed in CUDA enabled GPU list. Note down its Compute Capability. eg. GeForce 840M 5.0

Step 2: Install Visual C++ Build Tools 2015

We need Visual Studio 2015 stuffs here. If you have installed Visual Studio 2015 or 2017 no problem run the following setup and be okay with instructions on screen Goto https://visualstudio.microsoft.com/vs/older-downloads and move to end you will find “Redistributables and Build Tools” Download and install following :
1. Microsoft Visual C++ 2015 Redistributable Update 3 x64
2. Microsoft Build Tools 2015 Update 3

Step 3: Download the NVIDIA CUDA Toolkit:

If you have already CUDA stuffs other than CUDA 10 then uninstall all one by one from control panel. Go to https://developer.nvidia.com/cuda-downloads and download Installer for Windows [your version][network]. For me, version is Windows 10. I recommend you to use network installer. Install it in default location with default settings. It will update your GPU driver if required.

Step 4: Reboot the system to load the NVIDIA drivers.

Step 5: Check Cuda Toolkit:

Go to run (Win + R) type cmd The following command will check for nvcc version and insure that it is set in path environment variable.
nvcc --version
You will see something like:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:04_Central_Daylight_Time_2018
Cuda compilation tools, release 10.0, V10.0.130
Now test a sample to see setup is working by running following Go to run (Win + R) type
"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\extras\demo_suite\nbody.exe"

Step 6: Install MSYS2:

MSYS2 provides you basic useful command-line Linux programs to your windows. Download http://repo.msys2.org/distrib/x86_64/msys2-x86_64-20180531.exe and install with all default settings Goto run [Win+R] and enter following command:
"C:\msys64\msys2_shell.cmd" -use-full-path
MSYS2 shell will opened update it using:
pacman -Syu
It will take some time to get updated if all completed then close the shell. Again goto run [Win+R] and enter following command:
"C:\msys64\msys2_shell.cmd" -use-full-path
MSYS2 shell will opened update it using:
pacman -Su

pacman -S git patch unzip zip ed
Complete all installation with [y]

Step 7: Install cuDNN 7.3.1:

Goto https://developer.nvidia.com/cudnn (Membership required) After login Download the following: cuDNN v7.3.1 Library for Windows [your version] for me Windows 10 Goto downloaded folder and extract cudnn-10.0-windows10-x64-v7.3.1.20.zip Go inside extracted folder and copy all files and folder from cuda folder (eg. Bin, include, lib) and paste to “C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0”.

Step 8: Install bazel build:

Goto run [Win+R] and enter following command:
"C:\msys64\msys2_shell.cmd" -use-full-path
MSYS2 shell will opened, then do following:
cd /c/

mkdir bazel

cd bazel/

wget https://github.com/bazelbuild/bazel/releases/download/0.17.2/bazel-0.17.2-windows-x86_64.exe -O bazel.exe

Step 9: Set Windows to use long path :

Go to run (Win + R) enter following:
gpedit.msc
Then goto Computer Configuration -> Administrative Templates -> System -> Filesystem Configure “Enable Win32 long paths” to “Enabled”

16 Comments on How to install Tensorflow GPU with CUDA 10.0 for python on Windows

  1. I am following the guide but I am getting this error. See if anyone can help.
    p_packageuild –config=opt –config=cuda //tensorflow/tools/pip_package:build_pip
    Starting local Bazel server and connecting to it…
    WARNING: The following configs were expanded more than once: [cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior.
    WARNING: Option ‘experimental_shortened_obj_file_path’ is deprecated
    Loading:
    Loading: 0 packages loaded
    Analyzing: target //tensorflow/tools/pip_package:build_pip_package (1 packages loaded)
    Analyzing: target //tensorflow/tools/pip_package:build_pip_package (41 packages loaded)
    Analyzing: target //tensorflow/tools/pip_package:build_pip_package (95 packages loaded)
    Analyzing: target //tensorflow/tools/pip_package:build_pip_package (159 packages loaded)
    Analyzing: target //tensorflow/tools/pip_package:build_pip_package (223 packages loaded)
    Analyzing: target //tensorflow/tools/pip_package:build_pip_package (241 packages loaded)
    Analyzing: target //tensorflow/tools/pip_package:build_pip_package (268 packages loaded)
    Analyzing: target //tensorflow/tools/pip_package:build_pip_package (272 packages loaded)
    Analyzing: target //tensorflow/tools/pip_package:build_pip_package (293 packages loaded)
    Analyzing: target //tensorflow/tools/pip_package:build_pip_package (293 packages loaded)
    Analyzing: target //tensorflow/tools/pip_package:build_pip_package (294 packages loaded)
    ERROR: Analysis of target ‘//tensorflow/tools/pip_package:build_pip_package’ failed; build aborted: no such package ‘@png_archive//’: Traceback (most recent call last):
    File “C:/tensorflow/third_party/repo.bzl”, line 106
    _apply_patch(ctx, ctx.attr.patch_file)
    File “C:/tensorflow/third_party/repo.bzl”, line 73, in _apply_patch
    _execute_and_check_ret_code(ctx, cmd)
    File “C:/tensorflow/third_party/repo.bzl”, line 52, in _execute_and_check_ret_code
    fail(“Non-zero return code({1}) when …))
    Non-zero return code(256) when executing ‘C:\msys64\usr\bin\bash.exe -l -c “patch” “-p1” “-d” “C:/users/kailok/_bazel_kailok/xv6zejqw/external/png_archive” “-i” “C:/tensorflow/third_party/png_fix_rpi.patch”‘:
    Stdout:
    Stderr: Timed out
    INFO: Elapsed time: 29.694s
    INFO: 0 processes.
    FAILED: Build did NOT complete successfully (296 packages loaded)
    FAILED: Build did NOT complete successfully (296 packages loaded)

    • vim third_party/repo.bzl
      Press “a” on your keyboard. It will go in Insert mode look around def _wrap_bash_cmd(ctx, cmd):
      In that function you have to comment a line by adding # at first (may in line 29). See below
      Before:
      def _wrap_bash_cmd(ctx, cmd):
      if _is_windows(ctx):
      bazel_sh = _get_env_var(ctx, “BAZEL_SH”)
      if not bazel_sh:
      fail(“BAZEL_SH environment variable is not set”)
      cmd = [bazel_sh, “-l”, “-c”, ” “.join(cmd)]
      return cmd
      After:

      def _wrap_bash_cmd(ctx, cmd):
      if _is_windows(ctx):
      bazel_sh = _get_env_var(ctx, “BAZEL_SH”)
      if not bazel_sh:
      fail(“BAZEL_SH environment variable is not set”)
      # cmd = [bazel_sh, “-l”, “-c”, ” “.join(cmd)]
      return cmd
      Press ESE after edit then type
      :wq
      And press ENTER

  2. I tried that.
    It gave me error:
    Solution copy NUL WORKSPACE
    I tried again.
    It gave me error: no member sort in __init__ of some package.
    I found solution.
    I tried again: now I get
    WARNING: The following configs were expanded more than once: [cuda]. For repeatable flags, repeats are counted twice and may lead to unexpected behavior.
    WARNING: Option ‘experimental_shortened_obj_file_path’ is deprecated
    Loading:
    Loading: 0 packages loaded
    ERROR: Skipping ‘//tensorflow/tools/pip_package:build_pip_package’: error loading package ‘tensorflow/tools/pip_package’: Extension file not found. Unable to load package for ‘@local_config_rocm//rocm:build_defs.bzl’: The repository could not be resolved
    WARNING: Target pattern parsing failed.
    ERROR: error loading package ‘tensorflow/tools/pip_package’: Extension file not found. Unable to load package for ‘@local_config_rocm//rocm:build_defs.bzl’: The repository could not be resolved
    INFO: Elapsed time: 2.914s
    INFO: 0 processes.
    FAILED: Build did NOT complete successfully (0 packages loaded)
    FAILED: Build did NOT complete successfully (0 packages loaded)
    To what stage should I return?
    Thanks

  3. If you have an error with eigen (something like execroot\org_tensorflow\external\eigen_archive\eigen\src/Core/arch/CUDA/Half.h(212)), which may occurs because you have Compute capability above 5.3, do the following steps before step 12:

    1. download https://github.com/amsokol/tensorflow-windows-build-tutorial
    2. copy eigen_half.patch to tensorflow/third_party
    3. add Add patch_file = clean_dep(“//third_party:eigen_half.patch”), line to eigen_archive section to tensorflow/tensorflow/workspace.bzl

    it should looks like:
    tf_http_archive(
    name = “eigen_archive”,
    build_file = clean_dep(“//third_party:eigen.BUILD”),
    patch_file = clean_dep(“//third_party:eigen_half.patch”),
    sha256 = “d956415d784fa4e42b6a2a45c32556d6aec9d0a3d8ef48baee2522ab762556a9”,
    strip_prefix = “eigen-eigen-fd6845384b86”,
    urls = [
    “https://mirror.bazel.build/bitbucket.org/eigen/eigen/get/fd6845384b86.tar.gz”,
    “https://bitbucket.org/eigen/eigen/get/fd6845384b86.tar.gz”,
    ],
    )

    This bug is reported at https://github.com/tensorflow/tensorflow/issues/22715

  4. The result of this building is original tensorflow but the one I needed is tensorflow with gpu support, is there any fix to the instructions?

      • After the installation, I open up CMD and type in “pip list” but there isn’t any sign of tensorflow-gpu but the regular tensorflow is there waiting for me to use it, is there any instructions that differentiate the results as tensorflow or tensorflow-gpu?

        • Run the following to check tf gpu:

          python -c "import tensorflow as tf; print(tf.contrib.eager.num_gpus())"

          • It shows the gpu is there but when I use tensorflow for python it does not show any usage on the gpu…

  5. Windows 10 Home Edition user doesn’t have gpedit, do we need to spend $200 on it to use tensorflow? 😀

2 Trackbacks & Pingbacks

  1. How to install Tensorflow GPU on Windows | Python 3.6
  2. How to install Tensorflow 1.7.0 using official pip package | Python 3.6

Leave a Reply

Your email address will not be published.




This site uses Akismet to reduce spam. Learn how your comment data is processed.