This is going to be a tutorial on how to install tensorflow GPU on Windows OS. We will be installing tensorflow 1.5.0 along with CUDA Toolkit 9.1 and cuDNN 7.0.5. At the time of writing this blog post, the latest version of tensorflow is 1.5.0. We also did the installation guide for tensorflow 1.5.0 GPU on ubuntu 16.04 which you can find in this post here. This tutorial is for building tensorflow from source. If you want to use the official pre-built pip package instead, I recommend another post, How to install Tensorflow 1.5.0 using official pip package.
Tensorflow is an open source software library developed and used by Google that is fairly common among students, researchers, and developers for deep learning applications such as neural networks. It has both the CPU as well as GPU version available and although the CPU version works quite well, realistically, if you are going for deep learning, you will need GPU. In order to use the GPU version of TensorFlow, you will need an NVIDIA GPU with a compute capability > 3.0.
To install tensorflow GPU on Windows is complicated especially when compared to Mac or Linux OS. Even the tensorflow’s official website states, “We don’t officially support building TensorFlow on Windows; however, you may try to build TensorFlow on Windows if you don’t mind using the highly experimental Bazel on Windows or TensorFlow CMake build.” However, here is a complete step by step working tutorial to install Tensorflow GPU on Windows (64-bit only) OS using Visual Studio 2015 Update 3 and CMake.
The steps you need to take in order to install Tensorflow GPU on Windows OS are as follows:
Step 1: Verify you have a CUDA-Capable GPU:
Before doing anything else, you need to verify that you have a CUDA-Capable GPU in order to install Tensorflow GPU. You can verify that you have a CUDA-capable GPU through the Display Adapters section in the Windows Device Manager. Here you will find the vendor name and model of your graphics card(s).
The Windows Device Manager can be opened via the following steps:
Open a run window from the Start Menu or (Win+R)
control /name Microsoft.DeviceManager
If your graphics card is from NVIDIA then go to http://developer.nvidia.com/cuda-gpus and verify if listed in CUDA enabled GPU list.
Note down its Compute Capability. eg. GeForce 840M 5.0
Step 2: Install Visual Studio 2015 Update 3:
You will need Visual Studio 2015 in order to install tensorflow GPU on Windows. We tried installing with Visual Studio 2017 but it seems as if currently, Visual Studio 2017 is not fully supported to build tensorflow-gpu from source. So we recommend installing Visual Studio 2015 Update 3 by going to https://my.visualstudio.com/Downloads (Sign In required with Developer Account). Search for “Visual Studio Community 2015 with Update 3”. Download x64 version.
Goto Custom installation and make sure to install Visual C++ and Python Tools for Visual Studio inside Programming Language. Finish Installation.
Step 3: Download the NVIDIA CUDA Toolkit:
Go to https://developer.nvidia.com/cuda-downloads and download Installer for Windows [your version][network]. For me, version is Windows 10. We recommend network installer.
Install it in default location with default settings. It will update your GPU driver if required.
Step 4: Reboot the system to load the NVIDIA drivers.
Step 5: Check Cuda Toolkit is set to path:
Go to run (Win + R) type cmd
The following command will check for nvcc version and insure that it is set in path environment variable.
You will see something like:
nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2017 NVIDIA Corporation Built on Fri_Nov__3_21:08:12_Central_Daylight_Time_2017 Cuda compilation tools, release 9.1, V9.1.85
Step 6: Install cuDNN 7.0.5:
Goto https://developer.nvidia.com/cudnn (Membership required)
Download the following:
cuDNN v7.0.5 Library for Windows [your version] for me Windows 10
Goto downloaded folder and extract cudnn-9.1-windows[your version]-x64-v7.zip
Go inside extracted folder and copy all files and folder from cuda folder (eg. Bin, include, lib) and paste to “C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.1”.
Step 7: Install Dependencies
If you have already git then skip this else do this.
go to https://git-scm.com/download/win download git and install with all default settings.
Goto https://cmake.org/download/ and download Windows win64-x64 Installer.
Install it but make sure to check Add Cmake to system path for all users.
goto http://www.swig.org/download.html and download swigwin.
Extract it to C:\
Step 8: Verify if your CPU support AVX/ AVX2:
AVX/AVX2 optimization make tensorflow perform faster.
You can verify for AVX/ AVX2 support for your CPU by using Coreinfo tool.
Goto https://docs.microsoft.com/en-us/sysinternals/downloads/coreinfo to get it.
Extract it to C:\
Goto run (Win+R) and type cmd
Run following command:
Look for AVX
AVX * Supports AVX intruction extensions
If you got * then it supports AVX.
Step 9: Configure Tensorflow from source using CMake:
Start the process of building TensorFlow by downloading latest tensorflow 1.5.0
Goto run (Win+R) and copy paste following:
"C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Visual Studio 2015\Visual Studio Tools\Windows Desktop Command Prompts\VS2015 x64 Native Tools Command Prompt.lnk"
cd c:\ git clone https://github.com/tensorflow/tensorflow cd tensorflow git checkout r1.5 cd tensorflow\contrib\cmake mkdir build cd build
Activate your virtual environment eg. Virtualenv, conda, anaconda with python version 3.5. This makes CMake detect python automatically with python related to your virtual environment.
Install numpy (required):
pip install --upgrade numpy pip install --upgrade wheel
If AVX is supported by your CPU then do following:
cmake -G "Visual Studio 14 2015 Win64" -T host=x64 -DCMAKE_BUILD_TYPE=Release -DSWIG_EXECUTABLE=c:/swigwin-3.0.12/swig.exe -Dtensorflow_ENABLE_GPU=ON -Dtensorflow_CUDA_VERSION=9.1 -Dtensorflow_CUDNN_VERSION=7 -Dtensorflow_WIN_CPU_SIMD_OPTIONS=/arch:AVX2 ..
Else do following:
cmake -G "Visual Studio 14 2015 Win64" -T host=x64 -DCMAKE_BUILD_TYPE=Release -DSWIG_EXECUTABLE=c:/swigwin-3.0.12/swig.exe -Dtensorflow_ENABLE_GPU=ON -Dtensorflow_CUDA_VERSION=9.1 -Dtensorflow_CUDNN_VERSION=7 ..
You can see:
-- Configuring done -- Generating done -- Build files have been written to: C:/tensorflow/tensorflow/contrib/cmake/build
Goto “C:\tensorflow\third_party\gpus\cuda\ ” and open “cuda_config.h” with any editor or notepad.
Edit this line with your CUDA capability noted in step 1:
#define TF_CUDA_CAPABILITIES CudaVersion("3.0"),CudaVersion("3.5"),CudaVersion("5.2")
For my GPU, CUDA capability is 5.0
#define TF_CUDA_CAPABILITIES CudaVersion("5.0")
Verify following files with numbers as specified below:
#define TF_CUDA_VERSION "64_91" #define TF_CUDNN_VERSION "64_7" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.1\bin\cudart64_91.dll" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.1\bin\cudnn64_7.dll"
If all good then we are ready to build Tensorflow.
Step 10: Build Tensorflow using MSbuild
Copy this file (required):
"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.1\include\crt\math_functions.hpp" to "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.1\include\"
Disable real-time protection of Windows Defender or any other antivirus. We found that doing this dramatically decreases build time by a few hours.
Build using MSbuild to create whl file (pip package) using the following command:
MSBuild /p:Configuration=Release /verbosity:detailed tf_python_build_pip_package.vcxproj
It will take 4 -5 hours if antivirus disabled.
Step 11: Install Tensorflow GPU on Windows
Finally… Activate virtual environment here where you want to install tensorflow.
To install tensorflow gpu on windows with pip:
pip install "C:\tensorflow\tensorflow\contrib\cmake\build\tf_python\dist\tensorflow_gpu-1.5.0-cp35-cp35m-win_amd64.whl"
note- correct whl filename if not found.
Step 12: Verify Tensorflow installation
Run in terminal
python import tensorflow as tf hello = tf.constant('Hello, TensorFlow!') sess = tf.Session() print(sess.run(hello))
If the system outputs the following, then you are ready to begin writing TensorFlow programs:
Success! You have now successfully installed tensorflow GPU on windows machine.
Since it is experimental as stated on the official website, comment below if any errors occurred.
If you can’t make it, then go to How to install Tensorflow 1.5.0 using official pip package. for alternative.
For prebuilt wheel with optimization for AVX2 and cuda 9.1, cudnn 7.0.5, compute capability 5.0 go to this link .
Do let us know in the comments below if it worked for you. Or if you got any errors. Cheers!