Edit on GitHub

enlr.jl

This unit implements the elastic net logistic regression (ENLR) machine learning model on the tangent space for symmetric positive definite (SDP) matrices, i.e., real PD matrices. This model features two hyperparameters: a user-defined alpha hyperparameter, in range $[0, 1]$, where $α=0$ allows a pure Ridge LR model and $α=1$ a pure lasso LR model and the lambda (regularization) hyperparameter. When the model is fitted, we can request to find the optimal lambda hyperparameter for the given training data using cross-validation and/or to find the regularization path.

The lasso model (default) has enjoyed popularity in the field of brain-computer interaces due to the winning score obtained in six international data classification competitions.

The ENLR model is implemented using the Julia GLMNet.jl package. See 🎓 for resources on GLMNet.jl and learn how to use purposefully this model.

The fit, predict and cvAcc functions for the ENRL models are reported in the cv.jl unit, since those are homogeneous across all machine learning models. Here it is reported the ENLRmodel abstract type and the ENLR structure.

PosDefManifoldML.ENLR — Type
mutable struct ENLR <: ENLRmodel
	metric      :: Metric = Fisher;
	alpha       :: Real = 1.0
	standardize :: Bool
	intercept   :: Bool
	meanISR     :: Union{ℍVector, Nothing}
	vecRange    :: UnitRange
	featDim     :: Int
	# GLMNet Models
	path        :: GLMNet.GLMNetPath
	cvλ         :: GLMNet.GLMNetCrossValidation
	best        :: GLMNet.GLMNetPath
end

ENLR machine learning models are incapsulated in this mutable structure. Fields:

.metric, of type Metric, is the metric that will be adopted to compute the mean used as base-point for tangent space projection. By default the Fisher metric is adopted. See mdm.jl for the available metrics. If the data used to train the model are not positive definite matrices, but Euclidean feature vectors, the .metric field has no use. In order to use metrics you need to install the PosDefManifold package.

.alpha is the hyperparameter in [0, 1] trading-off the elestic-net model. α=0 requests a pure ridge model and α=1 a pure lasso model. By default, α=1 is specified (lasso model). This argument is usually passed as parameter to the fit function, defaulting therein to α=1 too. See the examples here below.

All other fields do not correspond to arguments passed upon creation of the model by the default creator. Instead, they are filled later when a model is created by the fit function:

For the content of fields standardize, intercept, meanISR and vecRange, please see the documentation of the fit function.

if the data used to train the model are positive definite matrices, .featDim is the length of the vectorized tangent vectors. This is given by n(n+1)÷2 (integer division), where n is the dimension of the original PD matrices on which the model is applied once they are mapped onto the tangent space. If feature vectors are used to train the model, .featDim is the length of these vectors. If for fitting the model you have provided an optional keyword argument vecRange, .featDim will be reduced accordingly.

.path is an instance of the following GLMNetPath structure of the GLMNet.jl package. It holds the regularization path that is created when the fit function is invoked with optional keyword parameter fitType = :path or = :all:

struct GLMNetPath{F<:Distribution}
    family::F                        # Binomial()
    a0::Vector{Float64}              # intercept values for each solution
    betas::CompressedPredictorMatrix # coefficient values for each solution
    null_dev::Float64                # Null deviance of the model
    dev_ratio::Vector{Float64}       # R^2 values for each solution
    lambda::Vector{Float64}          # lambda values for each solution
    npasses::Int                     # actual number of passes over the
                                     # data for all lamda values
end

.cvλ is an instance of the following GLMNetCrossValidation structure of the GLMNet.jl package. It holds information about the cross-validation used for estimating the optimal lambda hyperparameter by the fit function when this is invoked with optional keyword parameter fitType = :best (default) or = :all:

struct GLMNetCrossValidation
    path::GLMNetPath            # the cv path
    nfolds::Int                 # the number of folds for the cv
    lambda::Vector{Float64}     # lambda values for each solution
    meanloss::Vector{Float64}   # mean loss for each solution
    stdloss::Vector{Float64}    # standard deviation of the mean losses
end

.best is an instance of the GLMNetPath structure (see above). It holds the model with the optimal lambda parameter found by cross-validation that is created by default when the fit function is invoked.

Examples:

# Note: creating models with the default creator is possible,
# but not useful in general.

using PosDefManifoldML, PosDefManifold

# create an empty lasso model
m = ENLR(Fisher)

# since the Fisher metric is the default metric,
# this is equivalent to
m = ENLR()

# create an empty ridge model using the logEuclidean metric
m = ENLR(logEuclidean; alpha=0)

# Empty models can be passed as first argument of the `fit` function
# to fit a model. For instance, this will fit a ridge model of the same
# kind of `m` and put the fitted model in `m1`:
m1=fit(m, PTr, yTr)

# in general you don't need this machinery for fitting a model,
# since you can specify a model by creating one on the fly:
m2=fit(ENLR(logEuclidean; alpha=0), PTr, yTr)

# which is equivalent to
m2=fit(ENLR(logEuclidean), PTr, yTr; alpha=0)

# note that, albeit model `m` has been created as a ridge model,
# you have passed `m` and overwritten the `alpha` hyperparameter.
# The metric, instead, cannot be overwritten.
source