AutoPool¶
AutoPool is an adaptive pooling function with a learnable parameter that allows it to smoothly interpolate between min-, mean-, softmax- and max-pooling. It is implemented as Keras or Tensorflow-Keras layers for easy integration in deep learning architectures.
AutoPool was originally developed for multiple instance learning from weakly labeled time-series data as detailed in the AutoPool paper.
Getting Started¶
Installation instructions¶
pypi¶
The simplest way to install autopool is through the Python Package Index (PyPI). This will ensure that all required dependencies are fulfilled. This can be achieved by executing the following command:
python -m pip install autopool[keras]
which installs autopool for the Keras backend. If you would rather use tensorflow, you can do the following:
python -m pip install autopool[tf]
Source¶
If you’ve downloaded the archive manually from the releases page, you can install using the setuptools script:
tar xzf autopool-VERSION.tar.gz
cd autopool-VERSION/
python setup.py install
If you intend to develop autopool or make changes to the source code, you can install with pip install -e to link to your actively developed source tree:
tar xzf autopool-VERSION.tar.gz
cd autopool-VERSION/
python -m pip install -e .
Alternately, the latest development version can be installed via pip:
python -m pip install git+https://github.com/marl/autopool
Definitions¶
AutoPool¶
AutoPool extends softmax-weighted pooling by adding a trainable parameter α to be learned jointly with all other trainable model parameters:
Note that when α = 0 this reduces to an unweighted mean; when α = 1 this simplifies to soft-max pooling; and when α → ∞ this approaches the max operator. Hence the name: AutoPool.
Constrained and Regularized AutoPool (CAP & RAP)¶
In the paper we show there can be benefits to either constraining the range α can take, or, alternatively, applying l2 regularization on α; this results in constrained AutoPool (CAP) and regularized AutoPool (RAP) respectively. Since AutoPool is implemented as a keras layer, CAP and RAP can be can be achieved through the layer’s optional arugments:
CAP with non-negative α:
bag_pred = AutoPool1D(axis=1, kernel_constraint=keras.constraints.non_neg())(instance_pred)
CAP with α norm-constrained to some value alpha_max:
bag_pred = AutoPool1D(axis=1, kernel_constraint=keras.constraints.max_norm(alpha_max, axis=0))(instance_pred)
RAP with l2 regularized α:
bag_pred = AutoPool1D(axis=1, kernel_regularizer=keras.regularizers.l2(l=1e-4))(instance_pred)
CAP and RAP can be combined, of course, by applying both a kernel constraint and a kernel regularizer.
SoftMaxPool¶
SoftMaxPool is a special case of AutoPool where α is fixed to α = 1. This layer has no trainable parameters.
API Documentation¶
API Reference¶
Keras implementation¶
- class autopool.keras.AutoPool1D(axis=0, kernel_initializer='zeros', kernel_constraint=None, kernel_regularizer=None, **kwargs)[source]¶
Automatically tuned soft-max pooling. (Keras implementation)
This layer automatically adapts the pooling behavior to interpolate between mean- and max-pooling for each dimension.
- __init__(axis=0, kernel_initializer='zeros', kernel_constraint=None, kernel_regularizer=None, **kwargs)[source]¶
- Parameters
- axisint
Axis along which to perform the pooling. By default 0 (should be time).
- kernel_initializer: Initializer for the weights matrix
- kernel_regularizer: Regularizer function applied to the weights matrix
- kernel_constraint: Constraint function applied to the weights matrix
- kwargs
Tensorflow implementation¶
- class autopool.tf.AutoPool1D(axis=0, kernel_initializer='zeros', kernel_constraint=None, kernel_regularizer=None, **kwargs)[source]¶
Automatically tuned soft-max pooling. (tensorflow.keras implementation)
This layer automatically adapts the pooling behavior to interpolate between mean- and max-pooling for each dimension.
- __init__(axis=0, kernel_initializer='zeros', kernel_constraint=None, kernel_regularizer=None, **kwargs)[source]¶
- Parameters
- axisint
Axis along which to perform the pooling. By default 0 (should be time).
- kernel_initializer: Initializer for the weights matrix
- kernel_regularizer: Regularizer function applied to the weights matrix
- kernel_constraint: Constraint function applied to the weights matrix
- kwargs
Examples¶
Example¶
Coming soon…