Optimization again



Robert Dodier wrote:
> On Dec 14, 2007 12:28 PM, Daniel Lakeland <dlakelan at street-artists.org> wrote:
> 
>> For some decision theory type stuff I tried to use maxima to fit a
>> polynomial to the piecewise linear tax code.
> 
> It should work OK to phrase an interpolation problem as a general
> minimization problem, but doesn't Maxima have a function
> to construct an interpolating polynomial or (probably better)
> a polynomial spline?

Maxima does, but it's not terribly convenient to work with those forms 
in maxima (for example, differentiating a 1000 point spline 
interpolating function isn't convenient). I am willing to have a bit 
more error in the approximation in order to get a form that I can run 
through an optimizer so that in a 32 dimensional optimization (32 years 
of payments) it will have a relatively easy time converging on a global 
optimum scheduling of the payments taking taxes into account.

> I've found that rescaling the variables often helps LBFGS.
> But since the FOM isn't differentiable everywhere, that might
> not make any difference.
> 
>> It's come up several times before that numerical optimizations are not
>> terribly successful in maxima.
> 
> Has it? Maybe I have different expectations or something ...

I've had a lot of luck for numerical optimizations in maxima when the 
figure of merit is a relatively manageable symbolic expression. I LOVE 
the augmented_lagrangian code for these types of situations, but when 
the figure of merit doesn't have a simple, differentiable, symbolic 
representation (such as working with splines and with max and min type 
functions) the numerical optimization routines don't offer a method that 
is robust for those types of cases.

> Maybe we could throw in a gradient-free minimization method.

I think this would be a very advantageous addition, and I'd be willing 
to work on it this holiday season if we can come up with a suggested 
method on the list.