Computational statistics -OR - identity, ego and self-esteem (BUT NOT MINE!)

22 02 2009

My A2 computing project involves downloading stock prices and related news and showing it in a VB6-made executable. Most of the work is actually getting VB to co-operate with filetypes and graphing modules and whatnot, but it does raise some questions (or maybe I just randomly raised them). Mr Barker explained that Charlie Paterson’s algorithm for predicting trends involved inspecting the last five (or some similar number) changes in price and then looking back for a bunch of five similarly-sized consecutive changes and seeing what the sixth change was and then predicting that that sixth change would happen this time around. It sounded quite interesting to me but apparently it was rather unsuccessful. My dad mentioned “trend-following” ones that predict the price will go down if it’s gone down the last n times. The smaller n is, the more “sensitive” the thing is - obviously if n is 1, it’ll just predict the last change which is probably around 50% useful… but the larger n is, the more “behind the trend” you’ll be when you pick it up.

I kind of dream of fitting polynomials to the last 100 data points or something - maybe this would help predict minima and maxima as well? I think someone once mentioned Fourier analysis of the stock market to me, years and years ago… … … I don’t know. Sounds very interesting.

Then I started thinking in very general terms - think about the deterministic universe idea that if you know its initial state, you can predict its whole future. Obviously you wouldn’t expect to be able to predict the entire progress of the stock market from its initial state (!) because not only is it externally-influenced, it’s completely controlled by external forces. Now, if we could model the behaviour of lots and lots of investors… :) :) :)

I’m going to try to have a look at some stats/compsci mashups when I have the change.

The second part of this high-variance post is about some probably useless things that I’ve been thinking about. In a culture of mass consumption (blah blah) perhaps we slide towards defining ourselves through what we buy etc. … perhaps this is unavoidable, but I hope it’s understandable (in some sense). What I’ve been worrying about is the idea of being not just emotionally invested in endeavours and inanimate items (or even other people) but really, really relying on external things for validation. Maybe it’s insane to think you can fuel yourself - maybe that’s a microneurosis of my own, the belief that someone can function alone - but sometimes it seems insane to think that your environment totally controls you and you are just living “in reaction” to the world. I chanced upon the print version of this article, possibly exaggeratedly (I’m not very good at reading between lines etc.) mocking Twitter users’ need for constant attention/validation… its criticisms seem to ring true in some sense… … I suppose everyone who tries to maximise their “connectability” (*cough, shuffle*) is guilty of trying to “feel alive” by connecting to stuff. Is that bad? Possibly.

I think there must be an optimum point between being {selfish, obstinate, unreactive} and {personalityless, fickle, reactive}. Some particularly cool people seem to just settle near this optimum while others like myself flounder in a state of extreme self-consciousness, spewing a narrative of the slightly odd journey as they (I…) go. I’d like to think I’m slowly migrating towards a healthy emotional state - I’m definitely very happy but it feels a little bit fragile, like if I hit my head and was paralysed and couldn’t do any maths or guitar or exercise anymore I would become depressed. I don’t want that kind of mental breakdown to be possible ever.

What of the idea that one’s ego and one’s self-esteem aren’t the same, and that one’s ego is a big rationalisation (”I’m Farhan, I go to a good school, I have {some skills}, I’m a sexy beast” and one’s self-esteem is one’s natural will to survive? In that case, some people will be successful in maintaining their ego and have that at their core and let their self-esteem wither and with good luck will remain happy for their lives and be so confident that they can be happy that they will always somehow manufacture the situation necessary to reinforce their ego and assure them that they are a functioning life-form. I suspect that these people, if confronted by a terrible accident, would “lose it”, so to speak. Because their ego generally works for them and the world has tricked them into believing that “that is the way happiness works”, they never confront the strange damage that has taken place to their self-esteem as a result of the way modern human civilisation plays itself out.

On the other hand, cool mofos who have worked to genuinely improve themselves as people (I don’t mean in a “LOL LERN NEW SKILLZ” but in a “understand what it means to be human and not a robot or an animal or a brick”) and have avoided or got wise to the hidden and never-discussed subtle, indirect and psychological pitfalls of consumerism have their bodies compeltely crippled by fate and respond amazingly well.

I suspect/hope that for the majority of people, this works itself out - as they “mature” they somehow “find themselves”, some later than others. Fun.

So much/many lulz! countable(lulz)?

Pax

EDIT:

The obvious thing of taking the mean change for the last five and extrapolating is probably ill-fated, so I asked my dad about weighting the means so the ones closer are much more important. I thought that was quite nifty. He laughed in my face and suggested that I read up about the tremendous amount of stuff that’s been done and about how complex and, occasionally, effective it was. Interesting…



Root one minus x squared

11 12 2008
Place Course Qualification Length /years Code Offer
Cambridge CS BA 3 G400 If I get one, AAA11*
Imperial JMC MEng 4 GG41 AAAA
Edinburgh CS&M BSc 4 GG14 ABB
Warwick CS MEng 4 G403 AAB
Southampton CS MEng 4 G401 AAB

Cambridge interview seemed to go well (…) - as my mum and then Dr Eves - and then even a hallucination of Dr Eves - said, fingers crossed. Discussed P =? NP and other such lulz, as well as some memorable maths questions (I have an infinite string of bits, initially all zeroed. I increment this binary number repeatedly for a long time. In the long run, if altering a bit is one operation, what is the mean number of operations per increment?)**.

Algorithmics from now on:

Conditional Firm ~ 1st choice Conditional Insurance ~ 2nd choice
if(CambridgeOffer) then Cambridge, else Imperial Edinburgh

* that’s two grade 1s in STEP
** my guess is two, but… uh…

Pax



Imperial Computing Interview

5 11 2008

Met Erroll on tube - he seemed cool and his interview went well. After the general talk (which I’ve now seen three times) there was one for just JMC people (about nine of us). Randomly, maths person pointed to Sierpinski triangle and asked if any of us knew what it was. I said “Sierpinski” in a mid-volume high-pitched whine. The speaker then said “BRILLIANT!”, looked at some admissions guy and shouted “TAKE THAT GUY!”. Wikipedia browsing has its benefits.

Random highlights included a candidate’s unassuming parent who had actually supplied the electromagnets for the LHC (!!), free food, the lulziest tour guide students in the history of lulz and some pretty good talks.

I applied for JMC 4-year but my interviewer didn’t know until I told him. He didn’t seem to have read my personal statement or know my A-levels either, so I spent about two minutes telling him a condensed version of my entire application.

He then gave me a wooden Towers of Hanoi thing, asked me if I knew what it was called (I did) and then told me to find out the number of moves to solve it. He gave me a piece of paper and a pen and for a second I wasn’t sure if he wanted me to just start writing. He didn’t do anything, so I solved it physically (it came out easily). He then asked about the number of moves. I said that since I had gone from a tower of 4 to a base and a sub-tower of 3 during the solution, it might be to do with recursion. He seemed surprised/happy for a moment - I scribbled a bit and got the recurrence relation and then pattern-spotted the closed form thing (it was some obvious thing of O(2n)). He asked me how I would prove it, to which I replied induction, which he then made me do. He seemed pretty happy after that and implied that that question was meant to take longer.

He then asked me to integrate 1/x2 from -1 to 1. I became extremely suspicious because of how easy antidifferentiating it was which led me to realise that the point was that it was undefined at zero. I said this and he just didn’t respond, so I evaluated it (initially making a sign error, but then fixing it to get -2). He asked me what I’d said about undefinedness before and I restated it. He then asked me what the actual value of the integral was, and I non-comittally said it could be infinite but might converge. He didn’t do anything, so I sketched it and then re-said that it could be infinite but might converge. He agreed and then asked how I’d evaluate it, and I said by symmetry, integrating from 0 to 1 and then doubling by using a variable as the limit for the bit near the origin and making it tend to a value. He nodded “OK” and then said I’d be well-suited to the JMC course and shook my hand. The interview was about seven minutes of frantic scribbling and stammering, but I got the questions so I don’t care how crack-addicted I appeared.

They’re going to mail us offers/rejections within three weeks (probably within two).

The JMC course looks really good and it’ll be weird to choose between top universities if I have to. I was thinking that depending on how other people do, I may be able to ditch all of my friends in one fell swoop. Joy. As we (my mother insisted on accompanying me) trudged back down Birch Grove, she told me to “stop taking school and teachers so seriously” and “just coast”. It was strange to hear that type of psycho-social engineering.

It was almost as if she was reading my mind*.

Pax

* blog?



Going

18 07 2008

Finished work experience at Fabric; was good overall, despite some exposure to terrible retardation of some bits of IT. Fabric’s Raymond Serville [sic] was shockingly intelligent and knowledgeable (a good combination) and has set me up to rip apart some computers and then eventually build my own.

Packing for Arizona tomorrow. Taking books to blitz (Dewdney and Körner for the nine-hour flight) and computer for lulz.

Pax



Java

9 07 2008

I tried to teach myself some Java from a book I’ve had for years (I remember buying it at Waterstone’s in Ealing Broadway back when I didn’t exclusively use the internet to accumulate stuff). I’ve done some simple things like porting binary (which wasn’t too hard), making a program for finding the arithmetic, geometric or harmonic mean of an indefinitely long list of numbers (will adjust that to perform a quick iteration to find the arithmetic-geometric mean) and an iterative factorial thing.

What I want to do eventually is write some routines to a) solve a system of linear equations in 3 variables via matrices b) absorb plotter and c) do splines.

I’ve made headway with the algorithmic techniques already. I made a spreadsheet to solve the equations in a way similar to my sum of r4 thing so when you enter two points (x co-ord, y co-ord, gradient at that point), it tells you the coefficents of the cubic required to draw smoothly through those points.

I tested it with Grapher (a superb application). For people unfamiliar with cubic splines, a different function is plotted between each pair of points that joins up with the next one at the same gradient, so as to create the appearance of a smooth curve. For this, the gradient at each point, as explained in the picture, was just the gradient between that point and the next. Probably not the best way to do it, but it seemed to work, kind of… I believe you can do some magic with the second derivative too. I’ll look into it. Have a look.

If the gradient at the point (xn, yn) is mn, maybe I should’ve said:

mn = ((yn - yn-1)/(xn - xn-1) + (yn+1 - yn)/((xn+1 - xn))/2

That is, the mean of the gradient between the previous point and this one, and the gradient between this one and the next.

I played around a little with Swing but have so far only made an infinite number of replicating, unclosable windows.

I’m also finding Coda even more useful than before. I can just work from a split-screen of its inbuilt terminal and the .java file I’m working on (it highlights Java syntax - how nice!).

Pax



Cambridge Open Day

5 07 2008

On the train, I read the most recent Wired. It was particularly interesting. They’re focussing in on the important consequences of our vast storage capacity and the global network becoming what it ought to be. It hasn’t directly affected everyone yet, but the cloud is looming on the horizon. Grid/cloud computing should be very interesting indeed.

I was amused, however, when I read this:

Moreover, the number of hyperlinks in the World Wide Web is approaching that of synapses in the human brain.

I think something similar was in Serial Experiments Lain. As Mr Smith has my copy, maybe he should WATCH LAIN to confirm it (hint, hWATCH LAIN!!int)…

This guy doesn’t seem to like the idea of emergent properties, but he seems balanced.

The Cambridge thing yesterday was good. The course looks good. The colleges look good. The people were good. The research was good.

I talked to one guy - soon-to-be-Dr Jean Martina (blog, CL site) - who seemed to have an up-and-running automated theorem prover (+ until yesterday I didn’t know that the sum of consecutive odd numbers was always a perfect square…). My dad asked him if it had implications for machine intelligence and he smiled and said no. I thought that could be a cop-out so asked if he could use automatic genetic programming to make a computer come up with theorems and tests them and maybe come up with new mathematical resutls. Again, he smiled and said no… Later he mentioned his wife, also doing a PhD, was using software to model the human brain at the cellular level and then look for emergent properties. Like intelligence. He seemed like a cool guy and I was pretty happy that there was SOME theory being explored!

I nearly met someone I’d spoken to from the Computer Lab’s IRC channel (Malte Schwarzkopf). I didn’t realise it was him until later, but he was busy explaining his state diagram tool to an American girl anyway so I didn’t get a chance to see what it was all about. I would have, actually - I was seeking out theoretical core research, which is why seeing that theorem prover was so good.

The talk by Neil Dodgson was entertaining and informative (and for bonus points he repeatedly mocked Mirosoft - brave, as we were in the William Gates Building and M$FT seemed to fund lots of research etc.). I felt a little uncomfortable when he called computers “stupid” - I hope he never has to deal with HAL 9000.

At the end, there were questions (”I have a Scottish accent, will I be okay?”, “Is IB okay?”, “Can you interview me by phone bcause I’m overseas?” etc.). I asked what kind of theoretical research went on (apart from proving programming languages correct, which he’d mentioned). He smirked a little so I thought he was going to incite nerd-mocking and mentioned concurrency (from one of his slides). He then said “good question” and said something I can’t remember and then transferred it to his assistant guy (I’m not doing assistant guy any favours, he was helpful and smart but I just don’t know his name or position), who mentioned compilers. Later I asked if there were any College quirks we should know about. He said no.

His assistant guy finally mentioned the 2minutechallenge and then we were free.

We saw a bunch of colleges and I decided I wanted to be in the town centre because otherwise I’d be (more) disturbed by the potential peace and quiet [also consider Churchill's EXTREME DISTANCE (15 MINS BUS OH MY GOD) from town centre... although it was close to CL].

I met up with Elliot there and we ran into Will who knew lots about Queens’ so took us around there and explained everything. As a result of that and later bus issues (…) I didn’t manage to see Trinity Hall but I think as long as I go central, it’ll be fine.

BTW in this issue of New Scientist, there’s an article about Abdus Salam by Jim Al-Khalili. My grandparents hung out with Salam in that dingy house *shudder* and I’m reading Al-Khalili’s Quantum. It’s good. Connected!

Pax



241

3 07 2008

I think we’ve all seen the original seam carving video, but I hadn’t seen the following. It deals with problems in the original algorithm stemming from removing low-energy seams causing there to be more edges in the retargeted image - this roughly means that you can potentially make an image more jagged despite only altering the least edgy parts. Their solution was to consider “forward energy” and remove seams which, when removed, would insert the least amount of energy/edginess/contrast, which means generally the retargeted images are now even better quality.

They also treated 2D video as 3D objects, with the third dimension as time. Have a look:

I still find their work strikingly elegant. Not only is it effective, with many potential applications, but it’s also clear enough for us to get the gist of without reading their paper or doing any deep thought. It’s fantastic! If it’s proliferated, it’ll be even harder to tell what’s real and what’s not (e.g. is that field really that big, or are those inserted seams? Is that street really that narrow, or…?) unless it’s carefully managed. Right now we only have to contend with Photoshop (I say only but that’s only because people don’t abuse it as much as it could be abused, or with as much skill [generally] as possible)…

Pax



Imperial College Taster Course (”Future of Computing”)

2 07 2008

Watching Federer. They like his footwork. Jim (Philbin, fencing coach) likes footwork. He draws parallels with boxing. Personal trainer (Patrick Sago - awesome guy) is an amateur boxer. Lulz.

The Imperial taster was cool. We had lectures - some cool AI. Unfortunately they missed out the crypto talk but it was pretty good nonetheless. Did some programming. Once Vivan had plugged in my monitor (Dr Zetie’s report was right, I realised in shock: I do give up easily) the exercises were okay. They were more an exercise in figuring out the convoluted documentation and Linux keyboard shortcuts (Mac OS X has taught me some faux-Linux bad habits, I think).

I don’t really know why it was called the future of computing, but both the talks were essentially on AI (face tracking and emotion recognition, and then computational creativity - the Painting Fool). They linked the painting to the emotion thing - I’d already seen that in New Scientist or something similar, I think. I’d also seen the augmented reality thing they showed in the intro talk. The guy spoke very quietly but the videos he showed us were definitely ones I’d seen on YouTube - tracking the environment and then putting virtual items in. Lain. Lulz.

Pretty good. Vivan reckoned people there were smarter than the ones at the Royal Holloway one, although one of them who’d been to the RH one denied that he’d been when asked, so perhaps he wasn’t so smart after all (???). Java seems cool. Linux isn’t as bad as I thought.

Lulz.

Pax



Canvas

30 06 2008

It looks like graphics is well and truly useless now. I’ve been reading through the Mozilla Developer Center’s Canvas Tutorial and it’s clear that it’s quite powerful (e.g. automatic bezier and quadratic curves and all that). It’s quite a cool new element and I’ll have to check it out at some point.

Unrelatedly, Vivan suggested that I make a last minute application for the Imperial College IC125 Future Computing Taster Course, which I did and - surprisingly - got a place at. So we’ll be there, wrecking Art and Evolution, Enigma - the cryptographer’s battle, Lunch will be provided*, Gestures and artificial intelligence and Image manipulation in Java. Sounds good. Artificial intelligence and quantum computing, ha ha! Ha ha ha! And I actually know where the Huxley Building is now. We’ll have some stuff to talk about at university interviews. Lulz.

Also, I’m going to take back Ealing Broadway. I’m going to make it more fun! Preparing for Arizona by buying lots of reflective clothing (yes). Trying to make main site more coherent. Failing.

Trying to get Wikipedia SEL people to accept that the Knights guy was running HotSauce, and that lambda calculus is quite important.

Trying to think of massive hack for Lain’s tenth anniversary. Have a few ideas.

*may not be a lecture, not sure

Pax



Hero

29 06 2008

m35 of jpsxdec and #lain just found out why my site hadn’t been rendering in IE or Firefox: you can’t end script tags with “/>”, even though it’s valid XHTML. Oh well.

What a hero!

Also, this renders graphics obsolete… although I knew about the canvas element when I was writing it.

I finished lain.

Pax