psyche

You may remember Approval and Optimisation, in which I minimised the error between a function and a candidate function, linearly parameterised in one variable.

By linearly parameterised, I mean the function would be something like y = mx^n or mx + sinx or something with m the parameter. If it were non-linearly done, like y = nx^m or x + sin(mx), then my quadratics ‘n’ completing the square approach wouldn’t work (although you could just use derivative = zero stuff).

I’ll just summarise the general process before writing up the obvious extension which Bryant and I both considered. The parameter in your candidate function is m.

1. Get expression (candidate function - target function)^2 in integrable form
2. Let E be the integral of that expression between whatever limits you want
3. Find m such that E is a minimum (the most robust way is to solve dE/dm = 0 but I used a completing-the-square shortcut last time as I was only using a linearly parameterised thing)

For a candidate function that is a polynomial, remember that it must approach some kind of infinity as x goes to ±infinity, so there is no maximum error, so dE/dm can be rigorously shown to give the minimum.

Ok, now the important stuff. As I understand it, partial differentiation is just treating everything except the thing you’re differentiating with respect to as a constant. To illustrate, consider z = x^2 + xy + y^2

Implicit differentiation: dz/dx = 2x + (x dy/dx + y) + 2y dy/dx (having y as a function of x and chaining/producting it as necessary - the bracket in the middle is just the product rule)

Partial differentiation: delz/delx = 2x + y (as if y were a constant)

A quick scout online showed that this business of finding “critical points” i.e. maxima and minima of multivariable functions simply involves setting all the partial derivatives to zero.

The extended method is pretty similar, but here’s an example. I can’t be bothered to type up all the working - it won’t look nice anyway - so I’ve skipped all the simplification/algebra. Note to self: use The Integrator more often…

Candidate function: ax + b

Target function: x^2 on [0, t]

E = integral from 0 to t of ( (ax+b) - x^2 )^2 dx

= (a^2)/3t^3 + abt^2 + tb^2 - (at^4/2) - (2bt^3)/3 + (t^5)/5

Thus delE/dela = (2/3)at^3 + bt^2 - (t^4)/2

Setting this equal to zero leads to 4at + 6b = 3t^2 (EQ 1)

and delE/delb = at^2 + 2bt - (2/3)t^3

Setting this equal to zero leads to 3at + 6b = 2t^2 (EQ 2)

Solving EQs 1 and 2 simultaneously gives a = t, b = -(t^2)/6, so y = t(x - t/6). Oddish but niceish result I guess. Someone please confirm.

I sanity checked this with ax + b and mx + c and thankfully a = m and b = c dropped out fairly easily despite some difficult-looking but actually tolerable multiplying-out.

Infinitely more promising is the fact that your function doesn’t have to be linearly parameterised or just a straight line or any of that batshit insane stuff. You can do an arbitrarily high-order polynomial approximation… or something. Cool. Going to do that for some badass functions and see how they compare with Taylor/Maclaurin expansions.

Pax