Apache MXNet’s module API is like a FeedForward model and it is easier to compose similar to Torch module. It consists of following classes −
BaseModule([logger])
It represents the base class of a module. A module can be thought of as computation component or computation machine. The job of a module is to execute forward and backward passes. It also updates parameters in a model.
Methods
Following table shows the methods consisted in BaseModule class−This method will get states from all devices
Methods | Definition |
---|---|
backward([out_grads]) | As name implies this method implements the backward computation. |
bind(data_shapes[, label_shapes, …]) | It binds the symbols to construct executors and it is necessary before one can perform computation with the module. |
fit(train_data[, eval_data, eval_metric, …]) | This method trains the module parameters. |
forward(data_batch[, is_train]) | As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes. |
forward_backward(data_batch) | It is a convenient function, as name implies, that calls both forward and backward. |
get_input_grads([merge_multi_context]) | This method will gets the gradients to the inputs which is computed in the previous backward computation. |
get_outputs([merge_multi_context]) | As name implies, this method will gets outputs of the previous forward computation. |
get_params() | It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device. |
get_states([merge_multi_context]) | |
init_optimizer([kvstore, optimizer, …]) | This method installs and initialize the optimizers. It also initializes kvstore for distribute training. |
init_params([initializer, arg_params, …]) | As name implies, this method will initialize the parameters and auxiliary states. |
install_monitor(mon) | This method will install monitor on all executors. |
iter_predict(eval_data[, num_batch, reset, …]) | This method will iterate over predictions. |
load_params(fname) | It will, as name specifies, load model parameters from file. |
predict(eval_data[, num_batch, …]) | It will run the prediction and collects the outputs as well. |
prepare(data_batch[, sparse_row_id_fn]) | The operator prepares the module for processing a given data batch. |
save_params(fname) | As name specifies, this function will save the model parameters to file. |
score(eval_data, eval_metric[, num_batch, …]) | It runs the prediction on eval_data and also evaluates the performance according to the given eval_metric. |
set_params(arg_params, aux_params[, …]) | This method will assign the parameter and aux state values. |
set_states([states, value]) | This method, as name implies, sets value for states. |
update() | This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch. |
update_metric(eval_metric, labels[, pre_sliced]) | This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation. |
backward([out_grads]) | As name implies this method implements the backward computation. |
bind(data_shapes[, label_shapes, …]) | It set up the buckets and binds the executor for the default bucket key. This method represents the binding for a BucketingModule. |
forward(data_batch[, is_train]) | As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes. |
get_input_grads([merge_multi_context]) | This method will get the gradients to the inputs which is computed in the previous backward computation. |
get_outputs([merge_multi_context]) | As name implies, this method will get outputs from the previous forward computation. |
get_params() | It gets the current parameters especially those which are potentially copies of the actual parameters used to do computation on the device. |
get_states([merge_multi_context]) | This method will get states from all devices. |
init_optimizer([kvstore, optimizer, …]) | This method installs and initialize the optimizers. It also initializes kvstore for distribute training. |
init_params([initializer, arg_params, …]) | As name implies, this method will initialize the parameters and auxiliary states. |
install_monitor(mon) | This method will install monitor on all executors. |
load(prefix, epoch[, sym_gen, …]) | This method will create a model from the previously saved checkpoint. |
load_dict([sym_dict, sym_gen, …]) | This method will create a model from a dictionary (dict) mapping bucket_key to symbols. It also shares arg_params and aux_params. |
prepare(data_batch[, sparse_row_id_fn]) | The operator prepares the module for processing a given data batch. |
save_checkpoint(prefix, epoch[, remove_amp_cast]) | This method, as name implies, saves the current progress to the checkpoint for all buckets in BucketingModule. It is recommended to use mx.callback.module_checkpoint as epoch_end_callback to save during training. |
set_params(arg_params, aux_params[,…]) | As name specifies, this function will assign parameters and aux state values. |
set_states([states, value]) | This method, as name implies, sets value for states. |
switch_bucket(bucket_key, data_shapes[, …]) | It will switche to a different bucket. |
update() | This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch. |
update_metric(eval_metric, labels[, pre_sliced]) | This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation. |
Attributes
Following table shows the attributes consisted in the methods of BaseModule class −
Attributes | Definition |
---|---|
data_names | It consists of the list of names for data required by this module. |
data_shapes | It consists of the list of (name, shape) pairs specifying the data inputs to this module. |
label_shapes | It shows the list of (name, shape) pairs specifying the label inputs to this module. |
output_names | It consists of the list of names for the outputs of this module. |
output_shapes | It consists of the list of (name, shape) pairs specifying the outputs of this module. |
symbol | As name specified, this attribute gets the symbol associated with this module. |
data_shapes: You can refer the link available at https://mxnet.apache.org for details. output_shapes: More
output_shapes: More information is available at https://mxnet.apache.org/api/python
BucketingModule(sym_gen[…])
It represents the Bucketingmodule class of a Module which helps to deal efficiently with varying length inputs.
Methods
Following table shows the methods consisted in BucketingModule class −
Attributes
Following table shows the attributes consisted in the methods of BaseModule class −
Attributes | Definition |
---|---|
data_names | It consists of the list of names for data required by this module. |
data_shapes | It consists of the list of (name, shape) pairs specifying the data inputs to this module. |
label_shapes | It shows the list of (name, shape) pairs specifying the label inputs to this module. |
output_names | It consists of the list of names for the outputs of this module. |
output_shapes | It consists of the list of (name, shape) pairs specifying the outputs of this module. |
Symbol | As name specified, this attribute gets the symbol associated with this module. |
data_shapes − You can refer the link at https://mxnet.apache.org/api/python/docs for more information.
output_shapes− You can refer the link at https://mxnet.apache.org/api/python/docs for more information.
Module(symbol[,data_names, label_names,…])
It represents a basic module that wrap a symbol.
Methods
Following table shows the methods consisted in Module class −
Methods | Definition |
---|---|
backward([out_grads]) | As name implies this method implements the backward computation. |
bind(data_shapes[, label_shapes, …]) | It binds the symbols to construct executors and it is necessary before one can perform computation with the module. |
borrow_optimizer(shared_module) | As name implies, this method will borrow the optimizer from a shared module. |
forward(data_batch[, is_train]) | As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes. |
get_input_grads([merge_multi_context]) | This method will gets the gradients to the inputs which is computed in the previous backward computation. |
get_outputs([merge_multi_context]) | As name implies, this method will gets outputs of the previous forward computation. |
get_params() | It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device. |
get_states([merge_multi_context]) | This method will get states from all devices |
init_optimizer([kvstore, optimizer, …]) | This method installs and initialize the optimizers. It also initializes kvstore for distribute training. |
init_params([initializer, arg_params, …]) | As name implies, this method will initialize the parameters and auxiliary states. |
install_monitor(mon) | This method will install monitor on all executors. |
load(prefix, epoch[, sym_gen, …]) | This method will create a model from the previously saved checkpoint. |
load_optimizer_states(fname) | This method will load an optimizer i.e. the updater state from a file. |
prepare(data_batch[, sparse_row_id_fn]) | The operator prepares the module for processing a given data batch. |
reshape(data_shapes[, label_shapes]) | This method, as name implies, reshape the module for new input shapes. |
save_checkpoint(prefix, epoch[, …]) | It saves the current progress to checkpoint. |
save_optimizer_states(fname) | This method saves the optimizer or the updater state to a file. |
set_params(arg_params, aux_params[,…]) | As name specifies, this function will assign parameters and aux state values. |
set_states([states, value]) | This method, as name implies, sets value for states. |
update() | This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch. |
update_metric(eval_metric, labels[, pre_sliced]) | This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation. |
Attributes
Following table shows the attributes consisted in the methods of Module class −
Attributes | Definition |
---|---|
data_names | It consists of the list of names for data required by this module. |
data_shapes | It consists of the list of (name, shape) pairs specifying the data inputs to this module. |
label_shapes | It shows the list of (name, shape) pairs specifying the label inputs to this module. |
output_names | It consists of the list of names for the outputs of this module. |
output_shapes | It consists of the list of (name, shape) pairs specifying the outputs of this module. |
label_names | It consists of the list of names for labels required by this module. |
data_shapes: Visit the link https://mxnet.apache.org/api/python/docs/api/module for further details.
output_shapes: The link given herewith https://mxnet.apache.org/api/python/docs/api/module/index.html will offer other important information.
PythonLossModule([name,data_names,…])
The base of this class is mxnet.module.python_module.PythonModule. PythonLossModule class is a convenient module class which implements all or many of the module APIs as empty functions.
Methods
Following table shows the methods consisted in PythonLossModule class:
Methods | Definition |
---|---|
backward([out_grads]) | As name implies this method implements the backward computation. |
forward(data_batch[, is_train]) | As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes. |
get_input_grads([merge_multi_context]) | This method will gets the gradients to the inputs which is computed in the previous backward computation. |
get_outputs([merge_multi_context]) | As name implies, this method will gets outputs of the previous forward computation. |
install_monitor(mon) | This method will install monitor on all executors. |
PythonModule([data_names,label_names…])
The base of this class is mxnet.module.base_module.BaseModule. PythonModule class also is a convenient module class which implements all or many of the module APIs as empty functions.
Methods
Following table shows the methods consisted in PythonModule class −
Methods | Definition |
---|---|
bind(data_shapes[, label_shapes, …]) | It binds the symbols to construct executors and it is necessary before one can perform computation with the module. |
get_params() | It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device. |
init_optimizer([kvstore, optimizer, …]) | This method installs and initialize the optimizers. It also initializes kvstore for distribute training. |
init_params([initializer, arg_params, …]) | As name implies, this method will initialize the parameters and auxiliary states. |
update() | This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch. |
update_metric(eval_metric, labels[, pre_sliced]) | This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation. |
Attributes
Following table shows the attributes consisted in the methods of PythonModule class −
Attributes | Definition |
---|---|
data_names | It consists of the list of names for data required by this module. |
data_shapes | It consists of the list of (name, shape) pairs specifying the data inputs to this module. |
label_shapes | It shows the list of (name, shape) pairs specifying the label inputs to this module. |
output_names | It consists of the list of names for the outputs of this module. |
output_shapes | It consists of the list of (name, shape) pairs specifying the outputs of this module. |
data_shapes − Follow the link https://mxnet.apache.org for details.
output_shapes − For more details, visit the link available at https://mxnet.apache.org
SequentialModule([logger])
The base of this class is mxnet.module.base_module.BaseModule. SequentialModule class also is a container module that can chain more than two (multiple) modules together.
Methods
Following table shows the methods consisted in SequentialModule class
Methods | Definition |
---|---|
add(module, **kwargs) | This is most important function of this class. It adds a module to the chain. |
backward([out_grads]) | As name implies this method implements the backward computation. |
bind(data_shapes[, label_shapes, …]) | It binds the symbols to construct executors and it is necessary before one can perform computation with the module. |
forward(data_batch[, is_train]) | As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes. |
get_input_grads([merge_multi_context]) | This method will gets the gradients to the inputs which is computed in the previous backward computation. |
get_outputs([merge_multi_context]) | As name implies, this method will gets outputs of the previous forward computation. |
get_params() | It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device. |
init_optimizer([kvstore, optimizer, …]) | This method installs and initialize the optimizers. It also initializes kvstore for distribute training. |
init_params([initializer, arg_params, …]) | As name implies, this method will initialize the parameters and auxiliary states. |
install_monitor(mon) | This method will install monitor on all executors. |
update() | This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch. |
update_metric(eval_metric, labels[, pre_sliced]) | This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation. |
Attributes
Following table shows the attributes consisted in the methods of BaseModule class −
Attributes | Definition |
---|---|
data_names | It consists of the list of names for data required by this module. |
data_shapes | It consists of the list of (name, shape) pairs specifying the data inputs to this module. |
label_shapes | It shows the list of (name, shape) pairs specifying the label inputs to this module. |
output_names | It consists of the list of names for the outputs of this module. |
output_shapes | It consists of the list of (name, shape) pairs specifying the outputs of this module. |
output_shapes | It consists of the list of (name, shape) pairs specifying the outputs of this module. |
data_shapes − The link given herewith https://mxnet.apache.org will help you in understanding the attribute in much detail.
output_shapes − Follow the link available at https://mxnet.apache.org/api for details.
Implementation Examples
In the example below, we are going create a mxnet module.
import mxnet as mx input_data = mx.symbol.Variable('input_data') f_connected1 = mx.symbol.FullyConnected(data, name='f_connected1', num_hidden=128) activation_1 = mx.symbol.Activation(f_connected1, name='relu1', act_type="relu") f_connected2 = mx.symbol.FullyConnected(activation_1, name = 'f_connected2', num_hidden = 64) activation_2 = mx.symbol.Activation(f_connected2, name='relu2', act_type="relu") f_connected3 = mx.symbol.FullyConnected(activation_2, name='fc3', num_hidden=10) out = mx.symbol.SoftmaxOutput(f_connected3, name = 'softmax') mod = mx.mod.Module(out) print(out)
Output
The output is mentioned below −
<Symbol softmax>
Example
print(mod)
Output
The output is shown below −
<mxnet.module.module.Module object at 0x00000123A9892F28>
In this example below, we will be implementing forward computation
import mxnet as mx from collections import namedtuple Batch = namedtuple('Batch', ['data']) data = mx.sym.Variable('data') out = data * 2 mod = mx.mod.Module(symbol=out, label_names=None) mod.bind(data_shapes=[('data', (1, 10))]) mod.init_params() data1 = [mx.nd.ones((1, 10))] mod.forward(Batch(data1)) print (mod.get_outputs()[0].asnumpy())
Output
When you execute the above code, you should see the following output −
[[2. 2. 2. 2. 2. 2. 2. 2. 2. 2.]]
Example
data2 = [mx.nd.ones((3, 5))] mod.forward(Batch(data2)) print (mod.get_outputs()[0].asnumpy())
Output
Given below is the output of the code −
[[2. 2. 2. 2. 2.] [2. 2. 2. 2. 2.] [2. 2. 2. 2. 2.]]
Pingback: Apache MXNet - Python API Symbol - Adglob Infosystem Pvt Ltd