Black Swan

At some point I overheard Buster (my PM on analytics.twitter.com) raving about Nassim Nicholas Taleb’s books. I was intrigued without knowing too much and dove into Black Swan (even though Buster recommended Antifragile as his favorite) because I wanted to see what the deal was.

Nassim describes himself as a quant but writes like a philosopher. The book jumps around between his experiences during the Lebanese Civil War, Gaussian and Zipf distributions, blatant attacks on specific economists and thinkers and a long section in praise of Benoit Mandelbrot. Throughout are descriptions and examples of the central premise for which the book is named.

A Black Swan - an event of such low probability that it is ignored by risk models but has orders of magnitude more impact on a system than the average event - is a very satisfying idea to delve. Nassim points out how market crashes are always unforeseen but wind up having the most prominent impact on value over a long period. He talks about structuring income to be scalable according to Black Swans, his personal example being that insuring against them may only be profitable a few days out of decades, but pays off enough to make optimizing for them worthwhile.

Indeed, in some domains—such as scientific discovery and venture capital investments—there is a disproportionate payoff from the unknown, since you typically have little to lose and plenty to gain from a rare event. We will see that, contrary to social-science wisdom, almost no discovery, no technologies of note, came from design and planning—they were just Black Swans

One of the most compelling ideas I found was the idea that in many industries success itself is a random Black Swan. We hold up entrepreneurs, celebrities, musicians, authors as having some inherent virtues which put them ahead of their competition, but we never check to see whether their contemporaries who failed have those same qualities. Survivorship bias is brought up a lot.

A great deal of empiricism has been done on the subject, most notably by Art De Vany, an insightful and original thinker who singlemindedly studied wild uncertainty in the movies. He showed that, sadly, much of what we ascribe to skills is an after-the-fact attribution. The movie makes the actor, he claims—and a large dose of nonlinear luck makes the movie.

Ultimately:

So I disagree with the followers of Marx and those of Adam Smith: the reason free markets work is because they allow people to be lucky, thanks to aggressive trial and error, not by giving rewards or “incentives” for skill.

As part of a framework for talking about a Black Swan, Nassim brings up Platonic idealism in the frame of our simplified internal models for the world.

What I call Platonicity, after the ideas (and personality) of the philosopher Plato, is our tendency to mistake the map for the territory, to focus on pure and well-defined “forms,” whether objects, like triangles, or social notions, like utopias (societies built according to some blueprint of what “makes sense”), even nationalities… Platonicity is what makes us think that we understand more than we actually do.

Platonicity is what you get when you watch a Wes Anderson movie. Everything has been meticulously constructed and all actions and reactions are internally consistent to the rules of the universe. It’s what we think of as stylistic in movies, but I could certainly imagine the exposure to risk by having such a model of the real world in your head while trying to make investments or business decisions.

The Platonic fold is the explosive boundary where the Platonic mind-set enters in contact with messy reality, where the gap between what you know and what you think you know becomes dangerously wide. It is here that the Black Swan is produced.

At various points in my career I’ve certainly been exposed to this, particularly in the form of stories told about businesses. “We’re successful because we always do X”, “we didn’t perform as well this quarter because the market did Y”, “users will use our product more because users want X and Y delivers X”. These kinds of analysis are regarded poorly in the book - they may be descriptive in light of past events, but aren’t always successful as predictive guides (and can cause Black Swans when they fail).

These stories are incredibly sticky to us. Part of the book discusses the inherent nature of humans to try and reinforce such stories by looking for supporting instead of contradictory evidence:

The first experiment I know of concerning this phenomenon was done by the psychologist P. C. Wason. He presented subjects with the three-number sequence 2, 4, 6, and asked them to try to guess the rule generating it. Their method of guessing was to produce other three-number sequences, to which the experimenter would respond “yes” or “no” depending on whether the new sequences were consistent with the rule… The correct rule was “numbers in ascending order,” nothing more. Very few subjects discovered it because in order to do so they had to offer a series in descending order (that the experimenter would say “no” to). Wason noticed that the subjects had a rule in mind, but gave him examples aimed at confirming it instead of trying to supply series that were inconsistent with their hypothesis. Subjects tenaciously kept trying to confirm the rules that they had made up.

I find myself doing this often - for example when we deploy a stability fix for our service at work I tend to ignore the first couple of errors as spurious and wait to see if they’ll go away as opposed to getting detailed logs about what went wrong because I assume that the fix should work. I think I could be better off by training myself to think more critically about such assumptions.

Platonic ideals also came up in the last book I (re-)read, but in a more positive light - Neal Stephenson built a very cool religion around the idea of a perfect triangle in the alternate universe story Anathem. So I’d been thinking about them for a bit before reading Black Swan, but this book contextualized them for me very well. It would be wonderful to live in an understandable world. I’m pretty sure I love making games because you get to create a very specific world based in such ideals - enemies always move in defined ways, players have limited actions with predictable results. There feels like depth but not uncertainty. There are no Black Swans in Super Mario.

Except of course, that there are - in the form of glitches and bugs in the program itself. Many world record speed runs rely on such Platonic exploits to beat games (usually much) faster than would be possible by adhering to the model.

Comments? In the interest of limiting third-party Javascript running on this site, I've disabled all commenting plugins. If you have feedback, please share it with me on Twitter.

Elsewhere

Twitter (@kurrik) Github (kurrik) Google+ (+kurrik) Linkedin (kurrik)

Tags

arne (10) reviews (9) work (7) chrome (7) twitter (7) extensions (6) cinemaclub (5) games (5) html (4) javascript (3) google (3) go (3) presentations (3) books (3) algorithms (3) appengine (2) internet (2) estonia (2) ludumdare (2) product (1) space (1) recipes (1) http (1) art (1) management (1) readinglist (1)