To Optimize Four Parameters In Python Scipy.optimize.fmin_l_bfgs_b, With An Error

I am coding an algorithm for active learning, using L-BFGS algorithm from scipy.optimize. I need to optimize four parameters: alpha,beta,W and gamma. However, it does not work, wi

Solution 1:

The scipy optimization routines can only optimize a one-dimensional vector of parameters. It looks like you're trying to optimize a tuple of values which contain a mix of scalars, vectors, and matrices.

What you'll want to do is flatten all the relevant parameter values into a one-dimensional array, and have your func unwrap and set these appropriately.

---

Edit: I'd proceed by creating a convenience function which extracts your parameters; for example:

def get_params(para):
    # extract parameters from 1D parameter vector
    assert len(para) == 22
    alpha = para[0:4]
    beta = para[4]
    W = para[5:17].reshape(3, 4)
    gamma = para[17:]
    return alpha, beta, gamma, W

def func(para, *args):
    #the function to minimize
    #parameters
    alpha, beta, gamma, W = get_params(para)

    #args
    X = args [0]
    Y = args[1]
    Z = args[2]        
    return  np.sum(np.sum(log_p_y_xz(Y[i][t], Z[i], sigma_eta_ti(X[i],W[t],gamma[t]))+log_p_z_x(alpha, beta, X[i]) for t in range(num_labeler)) for i in range (num_instance))


def func_grad(para, *args):
    #para have 4 values
    alpha, beta, gamma, W = get_params(para)

    #args
    X = args [0]
    Y = args[1]
    Z = args[2]
    #gradiants
    d_f_a = df_alpha(X,Y,Z,W,alpha,beta,gamma)
    d_f_b = df_beta(X,Y,Z,W,alpha,beta,gamma)
    d_f_w = df_w(X,Y,Z,W,alpha,beta,gamma)
    d_f_g = df_gamma(X,Y,Z,W,alpha,beta,gamma)
    return np.array([d_f_a, d_f_b,d_f_w,d_f_g])


x0 = np.concatenate([np.ravel(alpha), np.ravel(beta), np.ravel(W), np.ravel(gamma)])
optimLogitLBFGS = sp.optimize.fmin_l_bfgs_b(func, x0=x0, fprime=func_grad)

It doesn't run because your func() and func_grad() require additional arguments which are not specified in your code snippet, but this change solves the specific problem you were asking about.