Microsoft Cognitive Toolkit CNTK Introduction

Abhijeet P

a month ago

In this chapter, we will learn what is CNTK, its features, difference between its version 1.0 and 2.0 and important highlights of version 2.7.

What is Microsoft Cognitive Toolkit (CNTK)?

Microsoft Cognitive Toolkit (CNTK), formerly known as Computational Network Toolkit, is a free, easy-to-use, open-source, commercial-grade toolkit that enables us to train deep learning algorithms to learn like the human brain. It enables us to create some popular deep learning systems like feed-forward neural network time series prediction systems and Convolutional neural network (CNN) image classifiers.
For optimal performance, its framework functions are written in C++. Although we can call its function using C++, but the most commonly used approach for the same is to use a Python program.

CNTK’s Features

Following are some of the features and capabilities offered in the latest version of Microsoft CNTK:

Built-in components

  • CNTK has highly optimised built-in components that can handle multi-dimensional dense or sparse data from Python, C++ or BrainScript.
CNTK has highly optimised built-in components that can handle multi-dimensional dense or sparse data from Python, C++ or BrainScript.
  • We can implement CNN, FNN, RNN, Batch Normalisation and Sequence-to-Sequence with attention.
We can implement CNN, FNN, RNN, Batch Normalisation and Sequence-to-Sequence with attention.
  • It provides us the functionality to add new user-defined core-components on the GPU from Python.
It provides us the functionality to add new user-defined core-components on the GPU from Python.
  • It also provides automatic hyperparameter tuning.
It also provides automatic hyperparameter tuning.
  • We can implement Reinforcement learning, Generative Adversarial Networks (GANs), Supervised as well as Unsupervised learning.
We can implement Reinforcement learning, Generative Adversarial Networks (GANs), Supervised as well as Unsupervised learning.
  • For massive datasets, CNTK has built-in optimised readers.
For massive datasets, CNTK has built-in optimised readers.

Usage of resources efficiently

  • CNTK provides us parallelism with high accuracy on multiple GPUs/machines via 1-bit SGD.
CNTK provides us parallelism with high accuracy on multiple GPUs/machines via 1-bit SGD.
  • To fit the largest models in GPU memory, it provides memory sharing and other built-in methods.
To fit the largest models in GPU memory, it provides memory sharing and other built-in methods.

Express our own networks easily

  • CNTK has full APIs for defining your own network, learners, readers, training and evaluation from Python, C++, and BrainScript.
CNTK has full APIs for defining your own network, learners, readers, training and evaluation from Python, C++, and BrainScript.
  • Using CNTK, we can easily evaluate models with Python, C++, C# or BrainScript.
Using CNTK, we can easily evaluate models with Python, C++, C# or BrainScript.
  • It provides both high-level as well as low-level APIs.
It provides both high-level as well as low-level APIs.
  • Based on our data, it can automatically shape the inference.
Based on our data, it can automatically shape the inference.
  • It has fully optimised symbolic Recurrent Neural Network (RNN) loops.
It has fully optimised symbolic Recurrent Neural Network (RNN) loops.

Measuring model performance

  • CNTK provides various components to measure the performance of neural networks you build.
CNTK provides various components to measure the performance of neural networks you build.
  • Generates log data from your model and the associated optimiser, which we can use to monitor the training process.
Generates log data from your model and the associated optimiser, which we can use to monitor the training process.

Version 1.0 vs Version 2.0

Following table compares CNTK Version 1.0 and 2.0:

Important Highlights of Version 2.7

Version 2.7 is the last main released version of Microsoft Cognitive Toolkit. It has full support for ONNX 1.4.1. Following are some important highlights of this last released version of CNTK.
  • Full support for ONNX 1.4.1.
Full support for ONNX 1.4.1.
  • Support for CUDA 10 for both Windows and Linux systems.
Support for CUDA 10 for both Windows and Linux systems.
  • It supports advance Recurrent Neural Networks (RNN) loop in ONNX export.
It supports advance Recurrent Neural Networks (RNN) loop in ONNX export.
  • It can export more than 2GB models in ONNX format.
It can export more than 2GB models in ONNX format.
  • It supports FP16 in BrainScript scripting language’s training action.
It supports FP16 in BrainScript scripting language’s training action.

Submit Review

BOT
Agent(Online)
We're Online!

Chat now for any query