don't uplift them

This window at work today looks like the setup for an Adrian Tchaikovsky novel...

at least they're on the outside ... for now

book review: The Book of Why

Judea Pearl, Dana Mackenzie.
The Book of Why: the new science of cause and effect.
Penguin. 2018

---

We have all heard the old saying “correlation is not causation”. This is a problem for statistics, since all it can measure is correlation. Pearl here argues that this is because statisticians are restricting themselves too much, and that it is possible to do more. There is no magic; to get this more, you have to add something into the system, but that something is very reasonable: a causal model.

He organises his argument using the three-runged “ladder of causation”. On the bottom rung is pure statistics, reasoning about observations: what is the probability of recovery, found from observing these people who have taken a drug. The second rung allows reasoning about interventions: what is the probability of recovery, if I were to give these other people the drug. And the top rung includes reasoning about counterfactuals: what would have happened if that person had not received the drug?
 
Intervention (rung 2) is different from observation alone (rung 1) because the observations may be (almost certainly are) of a biassed group: observing only those who took the drug for whatever reason, maybe because they were already sick in a particular hospital, or because they were rich enough to afford it, or some other confounding variable. The intervention, however, is a different case: people are specifically given the drug. The purely statistical way of moving up to rung 2 is to run a randomised control trial (RCT), to remove the effect of confounding variables, and thereby to make the observed results the same as the results from intervention. The RCT is often known as the “gold standard” for experimental research for this reason.

But here’s the thing: what is a confounding variable, and what is not? In order to know what to control for, and what to ignore, the experimenter has to have some kind of implicit causal model in their head. It has to be implicit, because statisticians are not allowed to talk about causality! Yet it must exist to some degree, otherwise how do we even know which variables to measure, let alone control for? Pearl argues to make this causal model explicit, and use it in the experimental design. Then, with respect to this now explicit causal model, it is possible to reason about results more powerfully. (He does not address how to discover this model: that is a different part of the scientific process, of modelling the world. However, observations can be used to test the model to some degree: some models are simply too causally strong to support the observed situation.)

Pearl uses this framework to show how and why the RCT works. More importantly, he also shows that it is possible to reason about interventions sometimes from observations alone (hence data mining pure observations becomes more powerful), or sometimes with fewer controlled variables, without the need for a full RCT. This is extremely useful, since there are many cases where RCTs are unethical, impractical, or too expensive. RCTs are not the “gold standard” after all; they are basically a dumb sledgehammer approach. He also shows how to use the causal model to calculate which variables do need to be controlled for, and how controlling for certain variables is precisely the wrong thing to do.

Using such causal models also allows us to ascend to the third rung: reasoning about counterfactuals, where experiments are in principle impossible. This gives us power to reason about different worlds: What’s the probability that Fred would have died from lung cancer if he hadn’t smoked? What’s the probability that heat wave would have happened with less CO2 in the atmosphere?

[p51] probabilities encode our beliefs about a static world, causality tells us whether and how probabilities change when the world changes, be it by intervention or by act of imagination. 

This is a very nicely written book, with many real world examples. The historical detail included shows how and why statisticians neglected causality. It is not always an easy read – the concepts are quite intricate in places – but it is a crucially important read. We should never again bow down to “correlation is not causation”: we now know how to discover when it is.

Highly recommended.

---

For all my book reviews, see my main website.

Wicklow Mountains coach tour

The Worldcon is over, and we have an extra day to enjoy Ireland.  Yesterday evening we went on a bus tour of Dublin; today we went on a longer coach tour, to the Wicklow mountains.  We had tried to book the Newgrange tour, but it was sold out.  As we were wondering whether to take the Wicklow tour instead, another customer in the Tourist Office, who had also been at the Worldcon, highly recommended it.  Thanks!  You were right!

First stop was the Glencree Cemetry and Barracks, for a cuppa and a loo break.  There was also the Glencree Grotto down in a beautiful small valley.

everything is so green

Next stop was a photo-op at the "P.S. I Love You" bridge.

the bridge that features in a film I have never heard of

Then another photo-op, at the Guinness Lake.

The Guinness Lake -- so called not because it is made of Guinness, or used to make Guinness, but because the land was owned by the Guinness family (until recently)

Then the main stop -- Glendalough ("Glen of the two Loughs").  We had two hours to walk round the loughs; it didn't start raining until we were nearly back!

walking along the woodland path, along the south side of the smaller Lower Lough

between the loughs: view across the larger Upper Lough, with the rain clouds beginning to thicken

view from the boardwalk along the north side of the Lower Lough: a magnificent Rowan, laden with brilliant red berries, with the Lough in the background

mossy branches; the boardwalk was well clear of the very boggy ground

an impressive range of lichens

The rain set in quite seriously at this point, but we were nearly back at the coach, so no problem.

Then off to our final stop at Avoca Village, and a late lunch at Fitzgerald's pub.  The exterior of this pub starred in the TV series Ballykissangel (which I have heard of, and seen, although long enough ago I have no memory of the pub).

After that, we returned to Dublin along the motorway, rather than via the windy backroads we had travelled down.

A great day out.

how tall is that?

Walking back and forth in Dublin over the last couple of days, we noticed a strange construction in O’Connell Strret.

It’s difficult to get a picture of the whole thing on a mere phone camera.  Here’s the base:

a spike with a shiny base (and, apparently, a bit of my finger).

And here’s the rest of it:

all the way to the top

How tall is that?  It seems to go up forever.  It’s a cunning optical illusion, though: The Spire of Dublin is “only” 120m tall, but the way it tapers gives a false perspective view that makes it look a lot taller from the base.  Neat.

view from a hotel window

We have arrived in Dublin, for the Science Fiction Worldcon.  We didn’t manage to book a room in a nearby hotel (they were sold out about 10 minutes after booking opened), so have a bit of a trek from Parnell Street.

We got the airport bus to the hotel, and checked in.  We then walked to the Convention Centre, to pre-emptively register for tomorrow’s start, and then walked on to the Science Galley for a panel discussion: “Oppy or Armstrong? Autonomous vs human space exploration”.  The panel unanimously agreed we need to send both humans and robots.  We then walked back to the hotel.  We had also done a bit of walking around airports earlier in the day, and my phone’s step counter is registering an impressive (for me) 15100 steps.

That local Tesco will be useful for stocking up on carbs needed to keep going through the weekend.  The trams clang their warning bells as they go past: I hope that doesn’t go on all night!

invasion of the butterflies

There are supposed to be millions of Painted Lady butterflies this year.  We saw this somewhat bedraggled one in our garden today.