Computational statistics -OR - identity, ego and self-esteem (BUT NOT MINE!)

22 02 2009

My A2 computing project involves downloading stock prices and related news and showing it in a VB6-made executable. Most of the work is actually getting VB to co-operate with filetypes and graphing modules and whatnot, but it does raise some questions (or maybe I just randomly raised them). Mr Barker explained that Charlie Paterson’s algorithm for predicting trends involved inspecting the last five (or some similar number) changes in price and then looking back for a bunch of five similarly-sized consecutive changes and seeing what the sixth change was and then predicting that that sixth change would happen this time around. It sounded quite interesting to me but apparently it was rather unsuccessful. My dad mentioned “trend-following” ones that predict the price will go down if it’s gone down the last n times. The smaller n is, the more “sensitive” the thing is - obviously if n is 1, it’ll just predict the last change which is probably around 50% useful… but the larger n is, the more “behind the trend” you’ll be when you pick it up.

I kind of dream of fitting polynomials to the last 100 data points or something - maybe this would help predict minima and maxima as well? I think someone once mentioned Fourier analysis of the stock market to me, years and years ago… … … I don’t know. Sounds very interesting.

Then I started thinking in very general terms - think about the deterministic universe idea that if you know its initial state, you can predict its whole future. Obviously you wouldn’t expect to be able to predict the entire progress of the stock market from its initial state (!) because not only is it externally-influenced, it’s completely controlled by external forces. Now, if we could model the behaviour of lots and lots of investors… :) :) :)

I’m going to try to have a look at some stats/compsci mashups when I have the change.

The second part of this high-variance post is about some probably useless things that I’ve been thinking about. In a culture of mass consumption (blah blah) perhaps we slide towards defining ourselves through what we buy etc. … perhaps this is unavoidable, but I hope it’s understandable (in some sense). What I’ve been worrying about is the idea of being not just emotionally invested in endeavours and inanimate items (or even other people) but really, really relying on external things for validation. Maybe it’s insane to think you can fuel yourself - maybe that’s a microneurosis of my own, the belief that someone can function alone - but sometimes it seems insane to think that your environment totally controls you and you are just living “in reaction” to the world. I chanced upon the print version of this article, possibly exaggeratedly (I’m not very good at reading between lines etc.) mocking Twitter users’ need for constant attention/validation… its criticisms seem to ring true in some sense… … I suppose everyone who tries to maximise their “connectability” (*cough, shuffle*) is guilty of trying to “feel alive” by connecting to stuff. Is that bad? Possibly.

I think there must be an optimum point between being {selfish, obstinate, unreactive} and {personalityless, fickle, reactive}. Some particularly cool people seem to just settle near this optimum while others like myself flounder in a state of extreme self-consciousness, spewing a narrative of the slightly odd journey as they (I…) go. I’d like to think I’m slowly migrating towards a healthy emotional state - I’m definitely very happy but it feels a little bit fragile, like if I hit my head and was paralysed and couldn’t do any maths or guitar or exercise anymore I would become depressed. I don’t want that kind of mental breakdown to be possible ever.

What of the idea that one’s ego and one’s self-esteem aren’t the same, and that one’s ego is a big rationalisation (”I’m Farhan, I go to a good school, I have {some skills}, I’m a sexy beast” and one’s self-esteem is one’s natural will to survive? In that case, some people will be successful in maintaining their ego and have that at their core and let their self-esteem wither and with good luck will remain happy for their lives and be so confident that they can be happy that they will always somehow manufacture the situation necessary to reinforce their ego and assure them that they are a functioning life-form. I suspect that these people, if confronted by a terrible accident, would “lose it”, so to speak. Because their ego generally works for them and the world has tricked them into believing that “that is the way happiness works”, they never confront the strange damage that has taken place to their self-esteem as a result of the way modern human civilisation plays itself out.

On the other hand, cool mofos who have worked to genuinely improve themselves as people (I don’t mean in a “LOL LERN NEW SKILLZ” but in a “understand what it means to be human and not a robot or an animal or a brick”) and have avoided or got wise to the hidden and never-discussed subtle, indirect and psychological pitfalls of consumerism have their bodies compeltely crippled by fate and respond amazingly well.

I suspect/hope that for the majority of people, this works itself out - as they “mature” they somehow “find themselves”, some later than others. Fun.

So much/many lulz! countable(lulz)?

Pax

EDIT:

The obvious thing of taking the mean change for the last five and extrapolating is probably ill-fated, so I asked my dad about weighting the means so the ones closer are much more important. I thought that was quite nifty. He laughed in my face and suggested that I read up about the tremendous amount of stuff that’s been done and about how complex and, occasionally, effective it was. Interesting…



Approval and optimisation

11 02 2009

Slowly detaching myself from the need to gain approval from others and losing the arbitrary restrictions I have placed on myself is an enjoyable process.

Now, optimisation. Initially I wanted something like “What is the optimum quadratic ax2 + bx + c to model the cubic px3 + qx2 + rx + s on the interval [x0, x1]?” but realised that this was rather ambitious.

I tried a crippled version of this, which was kind of fun. Say I have a function f(x) and want to approximate it with a straight line g(x) = mx on [0, t] (mx because I want g(x) to be strictly of the form m*h(x)… you’ll see why later).

The integral of f(x) - g(x) dx from 0 to t will give, in some sense, the “error” between the two - the distance. Unfortunately, the distance of one line below the other cancels out with the distance of that one above the other. In order to skip out integrals of |f(x)| (…), I’ll go for int[0, t]{(f(x)-g(x))2}dx. The integrand is a quadratic in m, as is the integral. If I call this integral sigma e2 (”sum of infinitesimal square errors” in a handwavy way), … nothing. That naming was pointless. Anyway, I now rewrite the evaluated definite integral by completing the square with respect to m. From this I can see the value m must take to set the newly-created (m - k)2 term to 0 and thus minimise the expression. Thus I have my line y = mx.

I understand that I just botched the explanation, so I’ll give an example with f(x) = sqrt(x).

You can evaluate the integral term by term by multiplying it out (it’s just polynomial in x) but I used The Integrator (I’LL BE BACK!!) to speed this along.

sigma e2 = int[0 to t]{(sqrt[x] - mx)2} = (1/30)t2(10tm2 - 24m.sqrt(t) + 15)

I don’t even need to fully complete the square… just take that 10t outside the bracket: (10/3)t3(m2 - (12/5)m/sqrt(t) + 3/(2t)) and then we know the (m - k)^2 bit is (m - 6/(5sqrt(t)))2 so to zero that term, m = 6/(5sqrt(t))…

…so in conclusion, the line y = 6x/(5sqrt(t)) best approximates y = sqrt(x) on [0, t]. Try it in Autograph. Lulz.

Yes, it’s useless, but it… is something that should work and it… does (?). It highlights the interpretation of the integral as an infinitesimal sum rather than “the area under a graph” or whatever. Eh.

Pax



Music

18 01 2009

Here’s my guess for the importance of chords in a key, in descending order:

I, V, IV, vi, iii, ii, VII (this is in the key of I major with i meaning I minor)

Little fills like ii V7 I or IVm V IVm I seem to be falling out of favour. There seem to be two types of songs: I IV V vi songs and “minor key” songs which are all like vi III ii (or II) and then V IV I. Lots of songs masquerade as minor key songs while actually being mainly iv IV V. Meh.

Of course, everyone who attempts to write music seriously figures this out pretty quickly and some choose to capitalise on how weird it seems to people who don’t think about it (1,2) while others just keep quiet and write hits and everyone wonders “Wow, so catchy. But how?”… lulz.

Pax



Responsibility

7 01 2009

Should people have some kind of responsibility to contribute proportionally to their ability?* Is there some significance behind the discrepancy in the definitions of genius?

Look at the difference between, say, Richard Feynman and George Dantzig. Dantzig didn’t seem to be a prodigy and, although he won prizes, has never really been acclaimed as a genius. However, as a grad student he found high-profile unsolved statistics questions merely “a little harder than usual”. Should people who work hard and, as a result of this, achieve fantastic things be respected more than those who do it with little effort? Should they be respected in a different way?

I’m slowly but surely becoming someone else. I’m not certain it’s who I want to be or who I wanted to be or who I will want to be but it’s like, the only solution (like). It’s all about… something.

I feel like disastrous climate change etc. is going to cut short our progress as a species. I think we should go into space. Maybe we can’t solve problems apparently intrinsic to human society - but if we can, it’d be almost unforgivable to give up. What a dilemma, ha ha ha!

Just concentrate on your breathing and slow it down and don’t think about anything. Initially, your mind will wander as it doesn’t usually get the chance, I guess (unless it does, in which case you’re ahead of me and like, should forget this). After a while you’ll have it, you know, down. You’ll make patterns with your thoughts and then you can be really calm and just sort of forget about the petty things that might ruin you.

Yes - I’m testing my theory.

Pax

*socialism? meritocracy?



Root one minus x squared

11 12 2008
Place Course Qualification Length /years Code Offer
Cambridge CS BA 3 G400 If I get one, AAA11*
Imperial JMC MEng 4 GG41 AAAA
Edinburgh CS&M BSc 4 GG14 ABB
Warwick CS MEng 4 G403 AAB
Southampton CS MEng 4 G401 AAB

Cambridge interview seemed to go well (…) - as my mum and then Dr Eves - and then even a hallucination of Dr Eves - said, fingers crossed. Discussed P =? NP and other such lulz, as well as some memorable maths questions (I have an infinite string of bits, initially all zeroed. I increment this binary number repeatedly for a long time. In the long run, if altering a bit is one operation, what is the mean number of operations per increment?)**.

Algorithmics from now on:

Conditional Firm ~ 1st choice Conditional Insurance ~ 2nd choice
if(CambridgeOffer) then Cambridge, else Imperial Edinburgh

* that’s two grade 1s in STEP
** my guess is two, but… uh…

Pax



Southampton computing interview day

11 11 2008

The main highlight was Will revealing that he was stalking one of the students who was giving us a tour. However, the interview was also pretty good. We discussed - very informalulzily, sort of how my conversations with friends go if they’re in a good mood - my A-level computing project, the Collatz conjecture (my interviewer’s wife had worked on it) and Last.fm. It turns out that Dr Paul Garatt, the admissions tutor who happened to sit in on my interview, supervised Richard Jones’ third year project - Audioscrobbler. Also, Tim Berners-Lee* lectures at Southampton occasionally and has set up web groups etc. there. There was some pretty awesome-sounding semantic web stuff there.

It made me think: what if universities integrated web science properly with informatics? Wouldn’t that be better? Otherwise, it takes extreme determination on the part of web developers to fashion themselves into experts. While people like Jeremy Keith are testament to how motivated - and as a consequence, skilful - people get with (semantic) web stuff, it’d be nice if people had their eyes opened without having to start from the angle of commercial web design. Also I wonder: if people like me, who are genuinely interested in the semantic web and aren’t either i) ignorant or ii) extremely dismissive, don’t work on it, who will? Perhaps someone will come along with a web messiah complex and fix it for us.

Pretty good!

Pax

* am I omitting the “Sir” as a sign of respect?



(A little bit of [an attempt at]) relativistic SUVAT

9 11 2008

Not sure if this is correct, but I thought this was quite good (done while procrastinating about doing Mr Miller’s Cosmology prep):

Consider an object whose initial speed, u or v0, is zero. A constant force F then acts on it (and continues acting on it). I will (attempt) to find an expression for the speed v(t) of the object. I will use u’ to mean du/dt. Please ignore terrible interchanges between functions of stuff and such like - don’t really know what I’m doing.

Remember, v0 is zero.

First, use the crunched version Newton’s second law: F = ma, so a = F/m, so v’ = F/m. Integrating both sides with respect to t gives v(t) = Ft/m + k. Since v(0) = 0, k = 0 so v = tv’ (cf. suvat stuff, v = u + at when u = 0… it’s all fine so far).

Now consider the relativistically tweaked version of Newton II: F = ma/sqrt(1-(v/c)^2). Using the same juggling as before, this gives:

v’ = Fsqrt(1-(v/c)^2)/m

Rearrange and use separation of variables/whatever to get:

int(1/sqrt(1-(v/c)^2))dv = int(F/m)dt

which gives (remembering the boundary condition v(0) = 0)

csin-1(v/c) = Ft/m

sin-1(v/c) = Ft/mc

v/c = sin(Ft/mc)

finally:

v(t) = csin(Ft/mc)

This looked promising (note the maximum value of v is c)…

To try to convince myself that this was correct, consider the sin small angle approximation: sin(x) is approx. equal to x for small values of x. Well, Ft/mc is certainly small for most F and t when you consider how large c is, so: for values of F that are small compared to c (and small t but not sure what that means physically), we get v(t) is approx. equal to c*Ft/mc = Ft/m (which = at for small v as this means we can ignore the Lorentz factor), which is the Newtonian one… which seems quite interesting. Tell me where I’ve messed up.

I almost did it with a non-zero initial speed but I saw a bunch of sqrt(1-(v0/c)^2) blah (because of the trig addition formulae and Pythagorean identities) in there so I avoided that and then gave up entirely (good times).

Pax



Imperial Computing Interview

5 11 2008

Met Erroll on tube - he seemed cool and his interview went well. After the general talk (which I’ve now seen three times) there was one for just JMC people (about nine of us). Randomly, maths person pointed to Sierpinski triangle and asked if any of us knew what it was. I said “Sierpinski” in a mid-volume high-pitched whine. The speaker then said “BRILLIANT!”, looked at some admissions guy and shouted “TAKE THAT GUY!”. Wikipedia browsing has its benefits.

Random highlights included a candidate’s unassuming parent who had actually supplied the electromagnets for the LHC (!!), free food, the lulziest tour guide students in the history of lulz and some pretty good talks.

I applied for JMC 4-year but my interviewer didn’t know until I told him. He didn’t seem to have read my personal statement or know my A-levels either, so I spent about two minutes telling him a condensed version of my entire application.

He then gave me a wooden Towers of Hanoi thing, asked me if I knew what it was called (I did) and then told me to find out the number of moves to solve it. He gave me a piece of paper and a pen and for a second I wasn’t sure if he wanted me to just start writing. He didn’t do anything, so I solved it physically (it came out easily). He then asked about the number of moves. I said that since I had gone from a tower of 4 to a base and a sub-tower of 3 during the solution, it might be to do with recursion. He seemed surprised/happy for a moment - I scribbled a bit and got the recurrence relation and then pattern-spotted the closed form thing (it was some obvious thing of O(2n)). He asked me how I would prove it, to which I replied induction, which he then made me do. He seemed pretty happy after that and implied that that question was meant to take longer.

He then asked me to integrate 1/x2 from -1 to 1. I became extremely suspicious because of how easy antidifferentiating it was which led me to realise that the point was that it was undefined at zero. I said this and he just didn’t respond, so I evaluated it (initially making a sign error, but then fixing it to get -2). He asked me what I’d said about undefinedness before and I restated it. He then asked me what the actual value of the integral was, and I non-comittally said it could be infinite but might converge. He didn’t do anything, so I sketched it and then re-said that it could be infinite but might converge. He agreed and then asked how I’d evaluate it, and I said by symmetry, integrating from 0 to 1 and then doubling by using a variable as the limit for the bit near the origin and making it tend to a value. He nodded “OK” and then said I’d be well-suited to the JMC course and shook my hand. The interview was about seven minutes of frantic scribbling and stammering, but I got the questions so I don’t care how crack-addicted I appeared.

They’re going to mail us offers/rejections within three weeks (probably within two).

The JMC course looks really good and it’ll be weird to choose between top universities if I have to. I was thinking that depending on how other people do, I may be able to ditch all of my friends in one fell swoop. Joy. As we (my mother insisted on accompanying me) trudged back down Birch Grove, she told me to “stop taking school and teachers so seriously” and “just coast”. It was strange to hear that type of psycho-social engineering.

It was almost as if she was reading my mind*.

Pax

* blog?



Interviews

1 11 2008

I can’t remember the stuff that’s happened, but I’ll quickly outline what’s happening with me: AAB offer from Warwick (computer science), awaiting response from Edinburgh (maths with computer science), interview at Southampton on 22nd (computer science), interview at Imperial on 5th (joint maths and computing [4 year]), awaiting response from Cambridge (computer science with first year 50% maths, Trinity - hopefully invitation to interview sometime in December).

Expecting rather technical interview from Imperial, not sure about Southampton but will try to have wits about me. Trinity will likely sit me in a room with an hour long test and then discuss it with me for the duration of the interview. Trying to ignore uncertainty. Went to a Bat Mitzvah today where the golden girl namechecked my mother (she’s golden girl’s science teacher) and launched into a fatalistic (wasn’t trying to be, just the way she put it was depressing - she acknowledged this and pointed out lulz of networking, but I couldn’t shake the feeling) talk about global warming. Jokes included drinking water of Athabasca Glacier to look ten years younger (have done that, incidentally) to which her mother responded with retorts about turning taps off after showers etc.. Strange experience overall - nice people, I couldn’t understand singing or prayers, all speeches were surprisingly insightful. Awkward shoe twisting ensued as enthusiastic family friends tracked down Cambridge graduates to “convince” me to go there (?) but overall everyone seemed to have good intentions.

When I got back, to compound global doom and gloom, saw this - maybe Gavin Starks touched a nerve.

Never looked at school references; don’t want to. Had enough of that. Oh, incidentally golden girl’s father is a Watchmen fan - asked him to convince parents to read. He agreed on the merit of my fabulous (not ashamed to say it) skinnier-than-you-thought-possible red tie.

Got some fun mini-reports as well: Mr Slay politely said my mechanics preps lack focus while in fact I am simply behind. Mr Rokison interestingly commented that I lack “intuitive grasp” of material that I had last year, by which he means I did badly on “can you remember what we’ve learned” paper whereas last year I generally did quite well (no intuition involved - learned facts, this year failed). I suppose the general point here is that my talent is always rather forced - important to remember, maybe.

Dr Bullett points out a change in attitude to prepared work required. Short comment is that he’s right - I give up too easily sometimes, and not easily enough at other times. However, I do know what to do.

I’m certain that everything will work out acceptably. Global problems will have to be addressed somehow, or they’ll change into different problems. Interesting. Remembering Metroid Fusion, which told me that experiences are limits of consciousness. Must find a way to experience war, death etc. more so can appreciate horrors of global catastrophes. Am still a child in that respect. No understanding of suffering. Tempting fate?

I’ll post anything useful I learn during interview process.

I just remembered that golden girl’s mother is the one who recommended my counsellor, who I need to get back in touch with to end phase. Good times, as they say*. Good times.

Pax

* Now, is that what they say?



Internet

20 08 2008

Some people are so smart that whatever choices they make, they’ll be celebrated. Some people are smart enough to make good choices without being all-round successes. I think I’m going to choose the Internet, and lulz it up. I don’t have the luxury of… choicelessness. I have to be good by choosing things, not by indiscriminately lulzing - because I just don’t have the ability to.

Pax