Set Up Your Own Keras with Tensorflow GPU Deep Learning Environment on Windows 8.1 and 10

Keras Deep Learning Library

There have been many strides and advances made in machine learning over the last decade. For the first time, we have just witnessed an intelligent computer program beat a professional player at the game of Go (read more). Machine learning has been successfully applied to a wide range of fields and industries. It goes without saying that the finance world is one of its many early-adopters.

Today I’m going to show you how you can set up your own GPU-based deep learning environment on Windows using Keras with a Tensorflow backend. At the end, we’ll also train a simple convolutional neural network to recognize handwritten characters!

What is Machine Learning?

Without going into too much detail, machine learning is a type of artificial intelligence (AI) that allows computers to learn from vast volumes of data without being explicitly programmed. Machine learning algorithms are able to model the patterns and trends found within the data which can then be used to classify, forecast, and detect anomalies (ex. credit card fraud). Some popular machine learning algorithms are:

  • Artificial neural networks
  • Clustering (k-NN)
  • Decision trees
  • Support vector machines

What is Deep Learning?

Deep learning is a branch within machine learning that uses specific algorithms to model high level abstractions in data. Generally, it is associated with neural networks with multiple hidden layers and more neurons. Until recently, training neural networks with more than 1 hidden layer had been very difficult (vanishing gradient problem) and time-consuming. Some applications of deep learning include natural language processing (automatic translation, sentiment analysis), image classification, and time-series forecasting.

Keras and Tensorflow

There have been many machine learning libraries published, but for today’s article we will be talking about Keras and Tensorflow.

Tensorflow

Made by researchers and developers at Google, Tensorflow is an “open-source software library for machine intelligence”. Tensorflow supports computing using one or many CPUs or GPUs via CUDA. GPU performance is dramatically faster than CPU, sometimes by factors of 30 times and more! So if you have a GPU that supports CUDA, now is the time to take advantage of it! To check if your GPU supports CUDA, see list of supported GPUs. You can also find more information about Tensorflow from their website.

Keras

As the developers at Keras explains it, Keras is a “high-level neural networks library, written in Python and capable of running on top of either TensorFlow or Theano”. Keras makes building neural networks extremely easy, and with just a few lines of code anyone can get started with their first neural network. Written in Python, Keras is modular, minimalist, and yet still very powerful. Keras documentation is also very clear and concise, making the learning curve very smooth.

Setting Up Keras with Tensorflow

We will be setting up a Keras (using Tensorflow as the backend) machine learning environment today on Windows 8, 8.1 and 10. The reason I chose the Windows operating system is because like most of you, I prefer to use Windows as my daily driver and I was not able to find a good tutorial for Windows. At the same time, the installation process is already simple enough for those on Linux/Mac.

Requirements

To follow along with this step-by-step guide, you’ll need to meet the following requirements:

  • Windows 8.1 or 10 64-bit
  • NVIDIA Graphics Card with CUDA support (this is only if you want GPU-accelerated computing, see list of supported GPUs)

If you’re all ready, let’s get started!

Download and Install CUDA Toolkit 8.0 and cuDNN v5.1

CUDA Toolkit and cuDNN are required for GPU computing. If you do not want to enable GPU computing, or your current GPU does not support CUDA, you can skip this step.

First, visit https://developer.nvidia.com/cuda-downloads and grab download CUDA Toolkit 8.0. You will need to register an account with Nvidia’s Accelerated Computing Developer Program. Select Windows as your operation system, choose the version of Windows you’re on (8.1 or 10), and select your installer type.

CUDA Toolkit Download Page

Run the installer (you may have to run it as adminstrator) and follow the on-screen prompts. Remember the location that it was installed at, you will need it for the next step (default is “C:\Program Files\NVIDIA GPU Computing Toolkit”).

Next, we’ll download the cuDNN files from https://developer.nvidia.com/rdp/cudnn-download. Make sure to select the correct operating system.

Extract the files from the zip and you should see a single directory named “cuda”, inside you will find a few more directories: “bin”, “include”, and “lib” (there is another directory inside “lib” called “x64”). Each directory contains a file: “cudnn64_5.dll”, “cudnn.h”, and “cudnn.lib”. Locate the path that CUDA Toolkit was installed at and go into “CUDA\v8.0” (ex. “C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0”).

Copy and paste “cudnn64_5.dll” into “\bin”, “cudnn.h” into “\include”, and “cudnn.lib” into “\lib\x64”.

You’ve now got CUDA all set up!

Anaconda – Python 3.6

Now we’re going to install our Python environment. I prefer Anaconda because it bundles all the common packages that you’ll need in a single convenient download. Keras requires h5py, HDF5, NumPy, PyYAML, and SciPy, which luckily Anaconda comes with.

Visit https://www.continuum.io/downloads, and download the latest version of Anaconda for Windows. At the time of this writing, it is Anaconda 4.3.0 Python 3.6 Version. Again, launch the installer and follow the prompts (keep the Advanced Options checked).

Anaconda Python Installer Advanced Options

Create and Activate Conda Environment

Once Anaconda has been successfully installed, launch the Anaconda Prompt with administrator privileges and type in the following command:

conda create -n KerasEnv python=3.5 anaconda

NOTE: Make sure that the python version is set to 3.5. Tensorflow currently only supports python 3.5 on Windows.

Feel free to replace KerasEnv with a name of your choosing. Press y and then enter to continue and Anaconda will create a new virtual environment with all the included packages for you at “C:\ProgramData\Anaconda3\envs\KerasEnv. To activate the environment, use the command:

activate KerasEnv

Replace KerasEnv with the name you gave your environment when it was created. Your prompt should now display:

(KerasEnv) C:\WINDOWS\system32>_

To deactivate it, simply enter:

deactivate

To delete your environment, enter the following:

conda remove -n KerasEnv --all

Again, replace KerasEnv with the name of your environment.

Install Keras and Tensorflow with Pip

Make sure the environment is activated before proceeding with the next steps.

Pip is packaged with Anaconda so we can directly install Keras and Tensorflow using pip install. Let’s start with Tensorflow.

We will use pip to install Tensorflow. Although conda can be used to install Tensorflow, only the CPU version is available.

To install the GPU version of Tensorflow, use the following command:

pip3 install --upgrade tensorflow-gpu

If you do not have a GPU that supports CUDA, you can install the CPU-only version of Tensorflow using the command below:

pip3 install --upgrade tensorflow

Once completed, we can go ahead and install Keras using the following simple command:

pip install keras

At this point, Keras and Tensorflow has been successfully installed on your machine! The next couple of steps will go through how to set up PyCharm as our IDE and training an example neural network to make sure everything is working.

NOTE: TensorFlow requires MSVCP140.DLL, which may not be installed on your system. If, when you import tensorflow as tf, you see an error about No module named “_pywrap_tensorflow” and/or DLL load failed, check whether MSVCP140.DLL is in your %PATH% and, if not, you should install the Visual C++ 2015 redistributable (x64 version).

March 21, 2017: With the release of Keras 2.0, you can easily update by uninstalling your current Keras package through the command

pip uninstall keras

then re-installing with the command above.

Setting Up PyCharm

For our IDE, we will be using PyCharm Community (it’s free!), so let’s go ahead and download the Windows version from https://www.jetbrains.com/pycharm/download/#section=windows.

Once the download is completed, launch the installer and follow the prompts to install it.

Now, let’s launch PyCharm and create a new project. You can save your project in any location you’d like. For the Interpreter, we’re going to select our conda environment we made earlier. Click the gear on the right side and select “Add Local”, find the location of the conda environment you created earlier and select the “python.exe” file. The full path might look like the following:

C:\ProgramData\Anaconda3\envs\KerasEnv\python.exe

It may take a few minutes for PyCharm to update the interpreter afterwards.

PyCharm Splash

PyCharm New Project Window

Once that’s done and the PyCharm has launch, we can create a new python file. To create a new file, simply go to File -> New… -> Python File. Let’s name it “main.py”. It’s time to run our first neural network!

Running Our First Neural Network!

Keras offers an entire directory of example models made to demonstrate the features of the library which you can find here. You should be able to run any one of them now on your new Keras environment! I’ll use “mnist_cnn.py” as an example, which you can find here. This particular example builds and trains a convolutional neural network (CNN) on the MNIST dataset. The MNIST dataset is a very popular dataset comprised on images of handwritten characters. We will use a CNN to find features in the dataset which allows the CNN to recognize the characters. Since CNNs have wide applications in image and video recognition, it is the perfect tool to solve this type of problem.

Some samples of the MNIST dataset

The code is below:

'''Trains a simple convnet on the MNIST dataset.

Gets to 99.25% test accuracy after 12 epochs
(there is still a lot of margin for parameter tuning).
16 seconds per epoch on a GRID K520 GPU.
'''

from __future__ import print_function
import numpy as np
np.random.seed(1337)  # for reproducibility

from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import Convolution2D, MaxPooling2D
from keras.utils import np_utils
from keras import backend as K

batch_size = 128
nb_classes = 10
nb_epoch = 12

# input image dimensions
img_rows, img_cols = 28, 28
# number of convolutional filters to use
nb_filters = 32
# size of pooling area for max pooling
pool_size = (2, 2)
# convolution kernel size
kernel_size = (3, 3)

# the data, shuffled and split between train and test sets
(X_train, y_train), (X_test, y_test) = mnist.load_data()

if K.image_dim_ordering() == 'th':
    X_train = X_train.reshape(X_train.shape[0], 1, img_rows, img_cols)
    X_test = X_test.reshape(X_test.shape[0], 1, img_rows, img_cols)
    input_shape = (1, img_rows, img_cols)
else:
    X_train = X_train.reshape(X_train.shape[0], img_rows, img_cols, 1)
    X_test = X_test.reshape(X_test.shape[0], img_rows, img_cols, 1)
    input_shape = (img_rows, img_cols, 1)

X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
X_train /= 255
X_test /= 255
print('X_train shape:', X_train.shape)
print(X_train.shape[0], 'train samples')
print(X_test.shape[0], 'test samples')

# convert class vectors to binary class matrices
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)

model = Sequential()

model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1],
                        border_mode='valid',
                        input_shape=input_shape))
model.add(Activation('relu'))
model.add(Convolution2D(nb_filters, kernel_size[0], kernel_size[1]))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=pool_size))
model.add(Dropout(0.25))

model.add(Flatten())
model.add(Dense(128))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(nb_classes))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='adadelta',
              metrics=['accuracy'])

model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epoch,
          verbose=1, validation_data=(X_test, Y_test))
score = model.evaluate(X_test, Y_test, verbose=0)
print('Test score:', score[0])
print('Test accuracy:', score[1])

Copy the code to the “main.py” file you just created in your PyCharm project and save it. Open the terminal by clicking the “Terminal” button on the bottom left of the IDE. You will notice that the terminal has automatically activated the environment you set up earlier.

(C:\ProgramData\Anaconda3\envs\KerasEnv) C:\Users\Clark\Documents\KerasExample>

To run the code, we simply run the command “python main.py” in our terminal and watch the magic happen!

python main.py

Executing main.py

Training Keras CNN on MNIST

If everything is working correctly, we’ll see the network train a total of 12 epochs and finally reaching an accuracy of 98.9%! Feel free to test any of the other models from Keras Examples.

I strongly encourage you to go through Keras documentation and try to make your own models. If you enjoyed this post and want to see more like this or if you have any feedback, be sure to let me know in the comments.

Until next time!

PS. For anyone interested in reading more about machine learning or deep learning, I strongly suggest Jason Brownlee’s website/blog at http://machinelearningmastery.com/. I have personally found Jason’s practical style of teaching to be incredibly effective.

2 thoughts on “Set Up Your Own Keras with Tensorflow GPU Deep Learning Environment on Windows 8.1 and 10”

  1. Thanks for sharing this! I’ve been looking for a windows tutorial for ages!

    Quick Q though: when running the pip commands, are those done in the anaconda prompt or a normal cmd prompt?

    Thanks!

Leave a Reply

Your email address will not be published. Required fields are marked *