<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Leafletters.  By Brightleaf. &#187; Document parsing</title>
	<atom:link href="http://www.brightleaf.com/blog/category/document-parsing/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.brightleaf.com/blog</link>
	<description>Thoughts on law, business, technology, and economics.</description>
	<lastBuildDate>Fri, 03 Feb 2012 17:32:29 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.2.1</generator>
		<item>
		<title>A Drunkard&#8217;s Walk: how our brains are lying to us about the numbers around us.</title>
		<link>http://www.brightleaf.com/blog/document-parsing/a-drunkards-walk-how-our-brains-are-lying-to-us-about-the-numbers-around-us/</link>
		<comments>http://www.brightleaf.com/blog/document-parsing/a-drunkards-walk-how-our-brains-are-lying-to-us-about-the-numbers-around-us/#comments</comments>
		<pubDate>Tue, 16 Feb 2010 04:37:54 +0000</pubDate>
		<dc:creator>Luke O'Brien</dc:creator>
				<category><![CDATA[Book review]]></category>
		<category><![CDATA[Document parsing]]></category>
		<category><![CDATA[Brightleaf Interpreter]]></category>
		<category><![CDATA[Document analysis]]></category>
		<category><![CDATA[Heuristics]]></category>
		<category><![CDATA[Leonard Mlodinow]]></category>

		<guid isPermaLink="false">http://www.brightleaf.com/blog/?p=145</guid>
		<description><![CDATA[Perhaps because of Brightleaf&#8217;s cool Interpreter technology, we&#8217;re fascinated with the science of pattern recognition.  If you’ve ever gotten this piece of junk email, and were able to comprehend it quickly, you’ll probably agree with me when I say that our brains are basically big, whompin’, lightning-fast pattern-matching engines.  “Aoccdrnig to a rscheearch at an [...]]]></description>
			<content:encoded><![CDATA[<p>Perhaps because of Brightleaf&#8217;s cool Interpreter technology, we&#8217;re fascinated with the science of pattern recognition. </p>
<p>If you’ve ever gotten this piece of junk email, and were able to comprehend it quickly, you’ll probably agree with me when I say that our brains are basically big, whompin’, lightning-fast pattern-matching engines. </p>
<p><em>“Aoccdrnig to a rscheearch at an Elingsh uinervtisy, it deosn&#8217;t mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht frist and lsat ltteer is at the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit porbelm. Tihs is bcuseae we do not raed ervey lteter by it slef but the wrod as a wlohe….”</em></p>
<p>While it has been <a href="http://www.snopes.com/language/apocryph/cambridge.asp">widely debunked</a> that this email actually originated from any “Elingsh uonervtisy rscheearch,” it does point out something very interesting about how we recognize patterns:  we cheat.  In the interests of speed, our brains employ sets of heuristic processes—shortcuts designed to accelerate processing time—so we can make our identification and move onto the next task.</p>
<p>Evolutionarily speaking, this probably got us to where we are today.  Back when your ancestors were little baby humanoids, huddled in the tall grass of some prehistoric African veldt, nothing was as important to them (and by progeniture, to you) as being able to evaluate the facial curves and planes and topography of the nearby larger humanoids and to discern Mommy from not-Mommy quickly…because Mommy would feed you and not-Mommy would eat you.  Perhaps for this reason, huge portions of our brains engage when we perform any kind of facial recognition task (One interesting note, described <a href="http://psycnet.apa.org/index.cfm?fa=search.displayRecord&amp;uid=2001-07565-002">here</a>:  boys use the right hemispheres of their brains to recognize faces; girls use the left).</p>
<p>But as demonstrated by the jumbled text of the debunked email above, we’re often too good at pattern recognition…we tend to over-recognize; we ignore mistakes and omissions; we jump to conclusions that really aren’t supported by our observations.  This is certainly true when it comes to the visual topography of words and faces, but it’s also true in a larger sense.  In our heuristic rush to make sense of the environments around us, we cheat.  We create patterns where statistically none exist. We confuse weak correlations with strong ones; we mistake correlation for causation altogether.</p>
<p>Here&#8217;s a simple example.  Growing up, how many times were you told that if you went outside when it was cold you would catch a cold?    (I can still hear my mother saying to me, and the echoes of her mother saying to her, “Put on a coat…you’ll catch your death of [insert name of largely defunct malady here]).  It seems likely that this is where the affliction got its name:  you caught a cold by being cold…a correlation completely unsupported by any medical research.  Lower temperatures don’t cause colds, rhinoviruses do.  About the closest you can come to an actual nexus between temperature and disease here is that when it’s cold, it tends to be dry…and when it tends to be dry your nostrils become irritated and run more, making you a more effective vector of viral retransmission.  Also, when it’s cold, you stay inside more, in closer contact with other people, raising your chances of exposure to whatever they’re carrying.  So because it’s cold outside, we become statistically more likely to exchange viruses.  And yet when most of us get sick during the winter, we still blame the temperature.</p>
<p>The best book I’ve read in a while, “A Drunkard’s Walk,” by Leonard Mlodinow, follows the history of the developments of probability and statistics.  It makes the case that while valid statistical evidence permeates every aspect of our lives, we treat that evidence much the same way that we treat the words in the debunked email above, by skimming over it and making unsupported conclusions from it, ascribing patterns where there is only randomness.  If you meet twenty people on the street of a town of 30,000, and all twenty of them tell you that they are voting for John Smith for Mayor, you invariably think that Mr. Smith is riding a tsunami of universal support all the way to City Hall.  In fact, your woefully inadequate empirical sampling makes your survey close to meaningless.  (Mlodinow terms this “The Law of Small Numbers,” the tendency for people to give disproportionately large impact to small sets of data).</p>
<p>Mlodinow goes on, pointing out over and over how our innate innumeracy gets in the way of our understanding of how the world around us really works.  Check out “A Drunkard’s Walk.”  It’s fascinating, much more so that you might think possible for a book on how the disciplines of probability and statistics came into existence.  </p>
<p>I’m 95% sure, with a margin of error of +/- 40%, that the 80 percent of you will really like it.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.brightleaf.com/blog/document-parsing/a-drunkards-walk-how-our-brains-are-lying-to-us-about-the-numbers-around-us/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

