Blog

To support the research and development of Deep Learning applications across many fields, Apache MXNet provides us a rich ecosystem of toolkits, libraries and many more. Let us explore them −

ToolKits

Following are some of the most used and important toolkits provided by MXNet −

GluonCV

As name implies GluonCV is a Gluon toolkit for computer vision powered by MXNet. It provides implementation of state-of-the-art DL (Deep Learning) algorithms in computer vision (CV). With the help of GluonCV toolkit engineers, researchers, and students can validate new ideas and learn CV easily.

Given below are some of the features of GluonCV −

• It trains scripts for reproducing state-of-the-art results reported in latest research.
• More than 170+ high quality pretrained models.
• Embrace flexible development pattern.
• GluonCV is easy to optimize. We can deploy it without retaining heavy weight DL framework.
• It provides carefully designed APIs that greatly lessen the implementation intricacy.
• Community support.
• Easy to understand implementations.

Following are the supported applications by GluonCV toolkit:

• Image Classification
• Object Detection
• Semantic Segmentation
• Instance Segmentation
• Pose Estimation
• Video Action Recognition

We can install GluonCV by using pip as follows −

pip install --upgrade mxnet gluoncv

GluonNLP

As name implies GluonNLP is a Gluon toolkit for Natural Language Processing (NLP) powered by MXNet. It provides implementation of state-of-the-art DL (Deep Learning) models in NLP.

With the help of GluonNLP toolkit engineers, researchers, and students can build blocks for text data pipelines and models. Based on these models, they can quickly prototype the research ideas and product.

Given below are some of the features of GluonNLP:

• It trains scripts for reproducing state-of-the-art results reported in latest research.
• Set of pretrained models for common NLP tasks.
• It provides carefully designed APIs that greatly lessen the implementation intricacy.
• Community support.
• It also provides tutorials to help you get started on new NLP tasks.

Following are the NLP tasks we can implement with GluonNLP toolkit −

• Word Embedding
• Language Model
• Machine Translation
• Text Classification
• Sentiment Analysis
• Natural Language Inference
• Text Generation
• Dependency Parsing
• Named Entity Recognition
• Intent Classification and Slot Labeling

We can install GluonNLP by using pip as follows −

pip install --upgrade mxnet gluonnlp

GluonTS

As name implies GluonTS is a Gluon toolkit for Probabilistic Time Series Modeling powered by MXNet.

It provides the following features −

• State-of-the-art (SOTA) deep learning models ready to be trained.
• The utilities for loading as well as iterating over time-series datasets.
• Building blocks to define your own model.

With the help of GluonTS toolkit engineers, researchers, and students can train and evaluate any of the built-in models on their own data, quickly experiment with different solutions, and come up with a solution for their time series tasks.

They can also use the provided abstractions and building blocks to create custom time series models, and rapidly benchmark them against baseline algorithms.

We can install GluonTS by using pip as follows −

pip install gluonts

GluonFR

As name implies, it is an Apache MXNet Gluon toolkit for FR (Face Recognition). It provides the following features −

• State-of-the-art (SOTA) deep learning models in face recognition.
• The implementation of SoftmaxCrossEntropyLoss, ArcLoss, TripletLoss, RingLoss, CosLoss/AMsoftmax, L2-Softmax, A-Softmax, CenterLoss, ContrastiveLoss, and LGM Loss, etc.

In order to install Gluon Face, we need Python 3.5 or later. We also first need to install GluonCV and MXNet first as follows −

pip install gluoncv --pre
pip install mxnet-mkl --pre --upgrade
pip install mxnet-cuXXmkl --pre –upgrade # if cuda XX is installed

Once you installed the dependencies, you can use the following command to install GluonFR −

From Source

pip install git+https://github.com/THUFutureLab/gluon-face.git@master

Pip

pip install gluonfr

Ecosystem

Now let us explore MXNet’s rich libraries, packages, and frameworks −

Coach RL

Coach, a Python Reinforcement Learning (RL) framework created by Intel AI lab. It enables easy experimentation with State-of-the-art RL algorithms. Coach RL supports Apache MXNet as a back end and allows simple integration of new environment to solve.

In order to extend and reuse existing components easily, Coach RL very well decoupled the basic reinforcement learning components such as algorithms, environments, NN architectures, exploration policies.

Following are the agents and supported algorithms for Coach RL framework −

Value Optimization Agents

• Deep Q Network (DQN)
• Double Deep Q Network (DDQN)
• Dueling Q Network
• Mixed Monte Carlo (MMC)
• Persistent Advantage Learning (PAL)
• Categorical Deep Q Network (C51)
• Quantile Regression Deep Q Network (QR-DQN)
• N-Step Q Learning
• Neural Episodic Control (NEC)
• Normalized Advantage Functions (NAF)
• Rainbow

Policy Optimization Agents

• Policy Gradients (PG)
• Asynchronous Advantage Actor-Critic (A3C)
• Deep Deterministic Policy Gradients (DDPG)
• Proximal Policy Optimization (PPO)
• Clipped Proximal Policy Optimization (CPPO)
• Generalized Advantage Estimation (GAE)
• Sample Efficient Actor-Critic with Experience Replay (ACER)
• Soft Actor-Critic (SAC)
• Twin Delayed Deep Deterministic Policy Gradient (TD3)

General Agents

• Direct Future Prediction (DFP)

Imitation Learning Agents

• Behavioral Cloning (BC)
• Conditional Imitation Learning

Hierarchical Reinforcement Learning Agents

• Hierarchical Actor Critic (HAC)

Deep Graph Library

Deep Graph Library (DGL), developed by NYU and AWS teams, Shanghai, is a Python package that provides easy implementations of Graph Neural Networks (GNNs) on top of MXNet. It also provides easy implementation of GNNs on top of other existing major deep learning libraries like PyTorch, Gluon, etc.

Deep Graph Library is a free software. It is available on all Linux distributions later than Ubuntu 16.04, macOS X, and Windows 7 or later. It also requires the Python 3.5 version or later.

Following are the features of DGL −

No Migration cost − There is no migration cost for using DGL as it is built on top of popular exiting DL frameworks.

Message Passing − DGL provides message passing and it has versatile control over it. The message passing ranges from low-level operations such as sending along selected edges to high-level control such as graph-wide feature updates.

Smooth Learning Curve − It is quite easy to learn and use DGL as the powerful user-defined functions are flexible as well as easy to use.

Transparent Speed Optimization − DGL provides transparent speed optimization by doing automatic batching of computations and sparse matrix multiplication.

High performance − In order to achieve maximum efficiency, DGL automatically batches DNN (deep neural networks) training on one or many graphs together.

Easy & friendly interface − DGL provides us easy & friendly interfaces for edge feature access as well as graph structure manipulation.

InsightFace

InsightFace, a Deep Learning Toolkit for Face Analysis that provides implementation of SOTA (state-of-the-art) face analysis algorithm in computer vision powered by MXNet. It provides −

• High-quality large set of pre-trained models.
• State-of-the-art (SOTA) training scripts.
• InsightFace is easy to optimize. We can deploy it without retaining heavy weight DL framework.
• It provides carefully designed APIs that greatly lessen the implementation intricacy.
• Building blocks to define your own model.

We can install InsightFace by using pip as follows −

pip install --upgrade insightface

Please note that before installing InsightFace, please install the correct MXNet package according to your system configuration.

Keras-MXNet

As we know that Keras is a high-level Neural Network (NN) API written in Python, Keras-MXNet provides us a backend support for the Keras. It can run on top of high performance and scalable Apache MXNet DL framework.

The features of Keras-MXNet are mentioned below −

• Allows users for easy, smooth, and fast prototyping. It all happens through user friendliness, modularity, and extensibility.
• Supports both CNN (Convolutional Neural Networks) and RNN (Recurrent Neural Networks) as well as the combination of both also.
• Runs flawlessly on both Central Processing Unit (CPU) and Graphical Processing Unit (GPU).
• Can run on one or multi GPU.

In order to work with this backend, you first need to install keras-mxnet as follows −

pip install keras-mxnet

Now, if you are using GPUs then install MXNet with CUDA 9 support as follows −

pip install mxnet-cu90

But if you are using CPU-only then install basic MXNet as follows −

pip install mxnet

MXBoard

MXBoard is logging tool, written in Python, that is used to record MXNet data frames and display in TensorBoard. In other words, the MXBoard is meant to follow the tensorboard-pytorch API. It supports most of the data types in TensorBoard.

Some of them are mentioned below −

• Graph
• Scalar
• Histogram
• Embedding
• Image
• Text
• Audio
• Precision-Recall Curve

MXFusion

MXFusion is a modular probabilistic programming library with deep learning. The MXFusion allows us to fully exploited modularity, which is a key feature of deep learning libraries, for probabilistic programming. It is simple to use and provides the users a convenient interface for designing probabilistic models and applying them to the real-world problems.

MXFusion is verified on Python version 3.4 and more on MacOS and Linux OS. In order to install MXFusion, we need to first install the following dependencies −

• MXNet >= 1.3
• Networkx >= 2.1

With the help of following pip command, you can install MXFusion −

pip install mxfusion

TVM

Apache TVM, an open-source end-to-end deep learning compiler stack for hardware-backends such as CPUs, GPUs, and specialized accelerators, aims to fill the gap between the productivity-focused deep-learning frameworks and performance-oriented hardware backends. With the latest release MXNet 1.6.0, users can leverage Apache(incubating) TVM to implement high-performance operator kernels in Python programming language.

Apache TVM actually started as a research project at the SAMPL group of Paul G. Allen School of Computer Science & Engineering, University of Washington and now it is an effort undergoing incubation at The Apache Software Foundation (ASF) which is driven by an OSC (open source community) that involves multiple industry as well as academic institutions under the Apache way.

Following are the main features of Apache(incubating) TVM −

• Simplifies the former C++ based development process.
• Enables sharing the same implementation across multiple hardware backends such as CPUs, GPUs, etc.
• TVM provides compilation of DL models in various frameworks such as Kears, MXNet, PyTorch, Tensorflow, CoreML, DarkNet into minimum deployable modules on diverse hardware backends.
• It also provides us the infrastructure to automatically generate and optimize tensor operators with better performance.

XFer

Xfer, a transfer learning framework, is written in Python. It basically takes an MXNet model and train a meta-model or modifies the model for a new target dataset as well.

In simple words, Xfer is a Python library that allows users to quick and easy transfer of knowledge stored in DNN (deep neural networks).

Xfer can be used −

• For the classification of data of arbitrary numeric format.
• To the common cases of images or text data.
• As a pipeline that spams from extracting features to training a repurposer (an object that performs classification in the target task).

Following are the features of Xfer:

• Resource efficiency
• Data efficiency
• Easy access to neural networks
• Uncertainty modeling
• Rapid prototyping
• Utilities for feature extraction from NN

Next Topic:-Click Here