Microsoft Cognitive Toolkit (CNTK), formerly known as Computational Network Toolkit, is a free, easy-to-use, open-source, commercial-grade toolkit that enables us to train deep learning algorithms to learn like the human brain. It enables us to create some popular deep learning systems likeĀ feed-forward neural network time series prediction systems and Convolutional neural network (CNN) image classifiers.
Audience
This tutorial will be useful for graduates, post-graduates, and research students who either have an interest in Deep learning or Artificial Neural Networks or have this subject as a part of their curriculum. The reader can be a beginner or an advanced learner.
Prerequisites
The reader must have basic knowledge about Neural Networks. He/she should also be aware about basic terminologies used in Python programming concepts.
Introduction
In this chapter, we will learn what is CNTK, its features, difference between its version 1.0 and 2.0 and important highlights of version 2.7.
What is Microsoft Cognitive Toolkit (CNTK)?
Microsoft Cognitive Toolkit (CNTK), formerly known as Computational Network Toolkit, is a free, easy-to-use, open-source, commercial-grade toolkit that enables us to train deep learning algorithms to learn like the human brain. It enables us to create some popular deep learning systems like feed-forward neural network time series prediction systems and Convolutional neural network (CNN) image classifiers.
For optimal performance, its framework functions are written in C++. Although we can call its function using C++, but the most commonly used approach for the same is to use a Python program.
CNTKās Features
Following are some of the features and capabilities offered in the latest version of Microsoft CNTK:
Built-in components
- CNTK has highly optimised built-in components that can handle multi-dimensional dense or sparse data from Python, C++ or BrainScript.
- We can implement CNN, FNN, RNN, Batch Normalisation and Sequence-to-Sequence with attention.
- It provides us the functionality to add new user-defined core-components on the GPU from Python.
- It also provides automatic hyperparameter tuning.
- We can implement Reinforcement learning, Generative Adversarial Networks (GANs), Supervised as well as Unsupervised learning.
- For massive datasets, CNTK has built-in optimised readers.
Usage of resources efficiently
- CNTK provides us parallelism with high accuracy on multiple GPUs/machines via 1-bit SGD.
- To fit the largest models in GPU memory, it provides memory sharing and other built-in methods.
Express our own networks easily
- CNTK has full APIs for defining your own network, learners, readers, training and evaluation from Python, C++, and BrainScript.
- Using CNTK, we can easily evaluate models with Python, C++, C# or BrainScript.
- It provides both high-level as well as low-level APIs.
- Based on our data, it can automatically shape the inference.
- It has fully optimised symbolic Recurrent Neural Network (RNN) loops.
Measuring model performance
- CNTK provides various components to measure the performance of neural networks you build.
- Generates log data from your model and the associated optimiser, which we can use to monitor the training process.
Version 1.0 vs Version 2.0
Following table compares CNTK Version 1.0 and 2.0:
| Version 1.0 | Version 2.0 |
|---|---|
| It was released in 2016. | It is a significant rewrite of the 1.0 Version and was released in June 2017. |
| It used a proprietary scripting language called BrainScript. | Its framework functions can be called using C++, Python. We can easily load our modules in C# or Java. BrainScript is also supported by Version 2.0. |
| It runs on both Windows and Linux systems but not directly on Mac OS. | It also runs on both Windows (Win 8.1, Win 10, Server 2012 R2 and later) and Linux systems but not directly on Mac OS. |
Important Highlights of Version 2.7
Version 2.7 is the last main released version of Microsoft Cognitive Toolkit. It has full support for ONNX 1.4.1. Following are some important highlights of this last released version of CNTK.
- Full support for ONNX 1.4.1.
- Support for CUDA 10 for both Windows and Linux systems.
- It supports advance Recurrent Neural Networks (RNN) loop in ONNX export.
- It can export more than 2GB models in ONNX format.
- It supports FP16 in BrainScript scripting languageās training action.