Subject: LBFGS for use in large maximum likelihood problem
From: Robert Dodier
Date: Sat, 9 Aug 2008 08:12:12 -0600
On 8/8/08, dlakelan <dlakelan at street-artists.org> wrote:
> loglikelihood:'lsum(log(subst('x=x,pdf)),x,data_test);
>
> lbfgs(-loglikelihood,[m,s],[9,2],.00025,[1,0]); /* ** THIS
> FAILS ** */
>
> /* the error above is something like "non variable 2nd argument to diff
> 9.0" */
Dan, I think the problem in this case is that diff doesn't treat
lsum the same as sum --- specifically that diff distributes over
sum but not lsum.
'sum (f (a, i), i, 1, n);
diff (%, a);
=> 'sum('diff(f(a,i),a,1),i,1,n)
'lsum (f (a, x), x, L);
diff (%, a);
=> 'diff('lsum(f(a,x),x,L),a,1)
Seems to me that diff should treat sum and lsum the same.
I'll put it on my to-do list unless someone wants to convince me otherwise.
Here's a formulation with sum which avoids the explicit "+" expression:
load(distrib);
load(lbfgs);
data_test : random_normal (10, 1, 100);
pdf (x) := exp(-(x - m)^2/(2*s^2)) / (s*sqrt(2*%pi));
logexpand : super;
nll : - 'sum (log (pdf ('data_test [i])), i, 1, 100);
=> -'sum(-log(s) - ('data_test[i]-m)^2/(2*s^2) - log(%pi)/2 -
log(2)/2, i, 1, 100)
lbfgs (nll, [m, s], [9, 2], .00025, [1, 0]);
(... chug chug chug ...)
=> [m = 9.979185712412182, s = .9223923146774927]
The need for 'data_test[i] instead of data_test[i] is another issue ---
Maxima barfs on foo[i] when foo is a literal list or matrix and i is
a symbol. I have a patch to make Maxima happy with that, maybe
I'll get around to committing it, unless of course someone wants to
talk me out of it.
Anyway hope this sheds some light on the problem.
Robert Dodier