user defined loss function liblinear

Question

In the Java version of LIBLINEAR there is a class called 'SolverType' in which one can choose type of the loss function to which they want to optimize the function. For example 'SolverType.L2LOSS_SVM_DUAL'. Is there any way to define a user-defined loss function?

Answer 1

The short answer is no.

The "loss function" defines the optimization problem, in fact this parameter changes (in particular) this model to

• linear regression
• logistic regression
• support vector machine

While first two are quite similar, third requires completely different machinery to solve it, much more complex methods. In particular one can define very arbitrary functions, which fall into "linear models" category, which are unsolvable (are solvable by very complex techniques).

On the other hand, if the function is very simple, ie. it is a differentiable function, without any bounds (optimization is performed on the whole parameters space) then (assuming you know analytical form of the derivatives) you can plug it in into any steepest descent algorithm implementation (there are dozens of such solvers avaliable).

Answer 2

SVM is formulated as a QP problem.

minimize ||w|| w.r.t

y * (w'x) >= 1 for all (x, y) in the training dataset

This is the dual form of the problem and the objective is to minimize the L2 norm of the weight w.

If you change the objective ||w|| then it is no longer SVM. However, you can change the weight of training examples. You can find a tutorial here:

http://scikit-learn.org/stable/modules/svm.html#unbalanced-problems