In Episode 47 of Hidden Forces, Demetri Kofinas speaks with Jimmy Soni about the father of Information Theory, Claude Shannon, and Shannon’s foundational work, A Mathematical Theory of Communication.

The 20th century is known as the information age, and for a good reason. It is a period that is dominated by knowledge and data. It’s an era in which the economy is no longer driven by traditional industries — such as construction, manufacturing, or agriculture — but by advanced information technologies that store, retrieve, transmit, and manipulate data.

This revolution finds its roots in Information Theory. And remarkably, it is a theory that was developed by one man: Claude Shannon.

Before Shannon, society had a rather immature understanding of what information was. Information was understood as something immaterial and intangible. It was not seen as something that could be touched or manipulated. It was assumed that the only way to send information (intelligence, as it was then referred to) across a greater distance was to “boost” the signal by using more power. This was a notoriously imperfect system, as it increased the amount of “noise” that was received and made the message more difficult to discern.

In his foundational work, A Mathematical Theory of Communication, Shannon solved this issue and presented a completely new way of understanding information. He showed that information isn’t insubstantial, but something that we can measure and manipulate — something that has physical characteristics and can be quantified. Shannon also created a diagram which showed that all information has certain, set components — such as a source, a transmitter, a recipient, and so on. As such, not only did he show that information is something that can be made material, through his work, Shannon proved that all information (be it a radio signal, a photo, or a song) can be governed through a set of common laws.

In short, he turned information into something that can be computed and reliably transmitted, laying the foundation for the digital revolution.

Producer & Host: Demetri Kofinas

Editor & Engineer: Stylianos Nicolaou

Join the conversation on FacebookInstagram, and Twitter at @hiddenforcespod

United States


00:00:09what's up everybody welcome to this week's episode of hidden forces with me the metric of Venus today I speak with Jimmy Sony a New York based author editor and speech writer is the co author of Rome's last citizens a biography the ancient Roman center Kato and a mind
00:00:27to play a biography of the late mathematician engineer and father of information theory Claude Shannon the latter one the Norman prize for the top book in the history of mathematics for twenty seventeen and was named one of the best books of the year by nature and Bloomberg Jimmy
00:00:45has also served as an editor at The Washington Post The New York observer and the Washington examiner Jimmy walking in forces thank you for having me you wrote this book with rob Goodman right you and rob wrote one book before this how do you guys know each other
00:01:00so we were debate partners at Duke in college and we just became really good friends and we loved a lot of the same things a lot of the same authors the writers we both did a bit of speech writing in our past and when we did our first
00:01:14book we're both you know early to mid twenties and we kind of figuring it out it's remarkable man well you can I ask you something how how would you get off no seriously where you get off like where do you get the idea in your head first we'll
00:01:27give you the locally speaking the balls to think that you could write a book at that age is my first question to how did maybe being the bay partners help you accomplish a how to that train you and maybe also what it reflect about your nature that made
00:01:41you do that go on to the second verse which we are both huge on repent and nerds and nerds then nerds now so you know candidly on the first question when robin I started to do rooms last citizen which is our first book I had come that same
00:01:55sort of deal I'd gone on Amazon and look for a biography of Kato because I'd bought biographies of Caesar and Cicero and all these other famous Roman figures and I just assumed just assume that someone had done one of Kato and when they hadn't I just kind of
00:02:09called robin I said Hey man look no one's done this we could do a proposal here's the best case scenario best case scenario publisher buys a proposal and we get to do a book worst case scenario spend two three months writing a proposal it sits on a shelf
00:02:24somewhere and that said there was very limited opportunity cost because we were essentially working on nights and weekends on this thing during the academic year well actually we were we had just both sort of in graduate school and I was working he was working so this was while
00:02:38we were working full time and you know this is how things are made you see something that's an unmet need you go out you try to figure it out and sometimes you strike sparks help me though because this is interesting that because what you do when you write
00:02:49a book like that you're on steroids doing what I do every week right right you're exhausting and doing tremendous amount of research and you're taking on a tremendous responsibility and you've incurred a huge future cost your life right now I'm just curious where did you develop the skills
00:03:05to do that we're going way off topic but I don't know of a cure is I love talking about this stuff so the skills so a lot of it is like you learn in college right essentially how to write papers now they're not exactly writing a book but
00:03:18you learn how to look up information find information source inside right basic skills then rose robin I are avid readers and my favorite favorite books to Rita biographies and so I was kinda had in the back of my head like maybe one day I'll get lucky enough to
00:03:32do this like I love biographies and so I I just kind of have the the back of my head the topic came around and then it was just a matter of like we had a whole stack of books I still remember those that were our models for how
00:03:44we did this right so like you're building a company we find a company that sort of looks feels acts like yours you Sir to learn some things from them for us there were two or three really important books that helped us figure out how to write rooms US
00:03:55citizen it was a biography of Cicero by Anthony ever it and it was this book Rubicon by Tom Holland and I can tell you I still have the copies of both of those books and they're basically like husks of books like that their pages torn up the bindings
00:04:08are because I'd spend time I mean to give your listeners a sense like I actually counted the number of words in the chapters I counted the number of chapters I put things together just to understand how it was done in a sense that's not random that your next
00:04:23book was on college and in that sense I mean everyone else could have had the same epiphany but you really squeezed as much as you could out of that text you extracted as much meaning out of that information will get to that yeah defining the counter intuitive definition
00:04:37of what information is as we proceed through this but that's interesting so that's how you got the idea of the book your process I think in some ways for writing Kato sounds like get informed your later interest and Shannon some degree and you saw the opportunity of course
00:04:51in the fact that there was a gap in the market which is a traditionally entrepreneurial thing to do what's also by the way it's incredibly in a way it's almost stupidly simple to do because if you go on Amazon you can identify right away what can apps there
00:05:03are in the books you want to read and ultimately for us this was just an act of all there's a but we want to read doesn't exist let's created and we figured that there would be a market given that we knew that there were markets for books are
00:05:15very similar so look I am sort of an army to be falsely humble robin I knew we were doing we had written professionally for a while at the same time all books are acts of faith you find something that isn't out there you go ahead and try to
00:05:28put together and again I'm their book ideas that don't go anywhere but these two happening %HESITATION so yeah and I should also mention to the audience that they should check out any of of rob's interviews you guys do them together your fantastic we can't replicate that type of
00:05:40dynamic here your great team and unfortunately he's in Montreal now we will give his exact location out all right so let's get the Shannon because this is why I brought you here I want to talk about Shannon and really want to talk about information there to the extent
00:05:53that we can but %HESITATION you make a point very early on the book and this is also it comes across in the title of the book minded play you make the point that Shannon was a born tanker that's lifted from your text why did you feel that that
00:06:07was an important thing to state out right in the book because he was the kind of person who wandered across so many different fields and everyone of the fields that he's in he doesn't have a precise sense of where his ideas for his ambition are going to take
00:06:24them he's tinkering he's playing we call the book a mind play because in everything really does you get the sense that this isn't a guy who's out to you know hit a certain threshold or capture certain trophy or when a certain warder get a position at a certain
00:06:38university he's just playing with ideas with concepts in some cases with objects into the tinkering being a born tanker is actually a reference to his childhood growing up in Michigan in northern Michigan where he's playing with broken radios and telegraph lines he's making barbed wire you know barber
00:06:59communication system for himself in a friend so he somebody who had a very very young age is using his hands to construct things to make things and that's not every childhood and so we felt like that was important enough to state out right we also think that that
00:07:14tinkering instinct plays into what he does later he someone who always is very gifted not just seeing ideas in his head you know Einstein was brilliant at seeing ideas in his head Shannon was also brilliant and bring those ideas to life with physical objects on is it fair
00:07:29to say that the Midwest at the time was in some ways the frontier of America %HESITATION very much I mean I you know I don't there was a frontier American logical frontier of America to some degree yeah certainly in the kind of engineering sense now he's you know
00:07:44born in northern Michigan he grows up in a town called Gaylor and his dad is a builder his dad the tanker he in addition to being a judge he's also he makes coffins he makes furniture is actually full page ads that we found in these papers from eighteen
00:07:59hundreds that say you know Claude Shannon the furniture man and he'll build things for you right but where Claude Shannon grows up it's actually at the intersection of a bunch of railroad lines and so he is in a place that has a ton of of industry and activity
00:08:15and commerce he's also an area that's packed with force so the first settlers moved northern Michigan because there's a lot of wood there and that's very valuable and so you can imagine a town that's role that's building things they're making wheels for you know slays they're making ten
00:08:31pens are making all kinds of stuff and into this you drop a boy who loves to play with things to build things of his own and so his childhood is really the story of taking these physical devices and making them better improving them playing with them and you
00:08:45know for us that we feel like you can't have Claude Shannon the engineer and information theorist without having clutch and the boy grows up in northern Michigan that's a big statement the other thing I'm thinking about as you're talking because complexity theory complexity science comes up in the
00:08:59work of Shannon it's sort of intersex like a lot of different fields there something about that that rings true in the sense that the east coast was heavily industrialized and so very structured when your physician we passed the railroads you're in the territory of the barb wired you
00:09:15know farmers who are you you'll breaking their fences and you're pretty much on your own and what you're describing there's that culture is what the culture in which chan and grew up in but he had the intersection the railway lines of the intersection of commerce and there was
00:09:27a sort of budding middle ground of complexity that allowed him to that he was able to feed off of %HESITATION a thousand percent I mean I this is the way I think about it let's say Shannon grown up around the same era in a brownstone in New York
00:09:40right he would not have had a barbed wire fence into which she could have run a signal to communicate with a friend a half mile away now he may have found other ways to satisfy those curiosities but the fact is you know he builds in a friend's backyard
00:09:55a makeshift barn elevator in New York he would have had a barn much less the ability to build a barn elevator right so I do think that people are products of that early environment in a way and Shannon in a way is influenced by northern Michigan and everything
00:10:11it's building and everything that he gets a chance to build and if he was living in let's say Boston or New York in the early twentieth century I can seem exposed to a wider range of ideas and people but I can't necessarily see him being exposed to the
00:10:26same kind of practical hard headed tinkering mitigation is John necessity is the mother of invention is this and what about his character's personalities disposition was you quiet child what was he like he was introverted any was for the balance of his life he was not a social butterfly
00:10:43he was someone who spent a lot of time his own head he loved games puzzles you love code breaking even as a kid his favorite home or if it's a it short story was %HESITATION the gold Bob by Edgar Allen Poe which is actually the story of a
00:10:57very complex code but he somebody who is very comfortable alone and that's important it's important because later he's not a frequent collaborator his paper information their **** and it is is we run information during published alone his master's famous master's thesis published alone no co authors he's not
00:11:14a co author or co founder kind of guy but he's a loner he's also somebody who loves music so in his childhood he learns to play believe it's the French horn and he maintains an interest and the clarinet throughout his life in jazz music throughout his life he
00:11:29grows up playing music and it it's an important part of his both his boyhood and later his adult life the whole family orients itself from music but he's the kind of kid who from the best we can get some you can only make estimations of this right we're
00:11:40talking about a child that that you know was early twentieth century but from what we can gather he was a smart student he wasn't somebody that anybody said wow that boys a genius right I mean there's a funny enough they used to publish grades like in the local
00:11:53newspaper because they had nothing else to publish so they listed the students who are straight a students one year and Claude Shannon is not on the list and this is like one of the giants of the twentieth century intellectually did not get straight it so for all you
00:12:05kids out there you know since when %HESITATION you know just to drive that point home about him being a tanker and being %HESITATION innovative talked about music he created me create a like a flame thrower trumpet yeah it's a vibrating trumpet it's much later in life money just
00:12:21a small refund this so his this is much later when he's older and his son Andrew has to perform a like a school musical or something and Claude Shannon apparently this is the story we got directly from Andrew he just looks at me says wouldn't it be great
00:12:35if the trumpet the you played could also breathe fire and often built a fire breathing from raising and we've seen a video a representation of it actually we've seen the video of the thing being played and no joke you play the trumpet and it spits out fire from
00:12:48the value had pretty good access to this family we did it took three years to get access we patiently had to communicate with them send them tidbits we found out about their dad if you guys will permit me it's a great story because basically what we had to
00:13:01do was earn their trust so this is a family that was very private just like their dad didn't chase fame that entries fortune they were scientific family they'd prefer to keep their own kind of councils they don't go out and advertise it there the Shannon family and when
00:13:16we first contacted Betty is wife was in her eighties we basically got the polite brush off she said well wonderful that you're doing a biography best of luck to you and then we contacted the family in overtime we said Hey look at this cool thing we found out
00:13:30about your dad smart look at this school thing found out about your dad Hey did you know that I was profiled in vogue here's a vogue profile of him and over time I think they saw wow you know these guys are exhaustive we want to tell the story
00:13:40the right way and finally and this X. have never told her before but the way that I got the long sit down interview with his daughter was that I had emailed her I said Hey Peggy I'm going to be in the Massachusetts area you know in the Boston
00:13:52area sometime the next couple weeks any chance you're around his ago great when we need %HESITATION in Amherst you know at Bubba and of course I had no plans to be in the Boston area I was just so far line as bite for sure and he was very
00:14:04gracious and she met with me for a number of hours and that's how we got a lot of information job man that's fantastic I totally relate with that and it sounds like you're sort of strategy you and rob is just to share your enthusiasm yeah genuine love for
00:14:15the material to be straight up and honest people respond to that people like when you genuinely care about what they're doing I hear I just like trying to manipulate your way through to get what you want that's right and it's also you know with any of these projects
00:14:27are any project like if you're building a company I mean if you if you have that quality I find that the world sort of organizes itself to help you you know along the way is we've managed to even track down an ex girlfriend one of his closest friends
00:14:42running buddy you know somebody share an office with him as a who was a hundred and one years old when we talk to him and so the the world's or find a way to help you out as soon as you demonstrate your actually doing the right who was
00:14:52it that said man the information gather was that number that was I don't think it was no I mean maybe it was so let's get back to Shannon now because this will never gonna get through all the things that matter here but %HESITATION let's start with Michigan so
00:15:06we got through is used to the extent that we have the time to get through that he goes to Michigan and Michigan he studies both mathematics and electrical engineering first of all an odd pairing not by today's standards but I wonder by those standards how odd was that
00:15:20pairing you know at the time at the university of Michigan the two curriculums are actually starting to overlap part of the reason is because there's a really enterprising engineering team named Mortimer Coley who essentially like does a you can call it a hostile take over but it was
00:15:37a takeover and retrofitting of the engineering department he's got a lot of bravado he's very %HESITATION confident guy so he gets the university give a lot of money to build out the engineering program and he does know what happens is the mathematics department in the engineering department and
00:15:50overlapping so when Shannon goes out to do both degrees he actually jokes later that the reason that he did it was it was it was really easy to get adultery you didn't actually to take them any additional courses but at the same time would expose them to is
00:16:03engineers who at the time or by the way working on very very practical problems how do you transmit a wireless signal how do you build a ship that's better for the navy then in mathematics he's doing much more kind of theoretical work is exposed to boolean logic for
00:16:17example comes in portent later later for him but he managed to get both degrees the university Michigan its its first experience publishing something and he actually also manages to do reasonably well in his studies there so let's talk about that so he studied bullion algebra which was essential
00:16:34to the formulation information theory first of all give our audience a basic sense of what boolean algebra is billion logic is and how sort of important do you think that was for the later work that he did with electrical switching and information theory so in the middle eighteen
00:16:53hundreds eighteen forty seven George bowl creates an algebra in which the values that are concerned are sort of the true what I call the sordid truth value service zero one true false right so instead of using numbers or even serving all numbers he's using zero and one as
00:17:11stand ins for true and false and it gives you the ability to do logical processes right so true false and if or not and so with that study in the eighteen hundreds is taught to Claude Shannon in the early twentieth century and it stays with him and it
00:17:30becomes one of the foundations for a paper that he writes later that becomes incredibly important was anyone using boolean logic for anything other than philosophical thinking I mean it's foundational set theory for example and Bertrand Russell did no not as far as I know and again I'm not
00:17:45an expert on the topic but so far as I know no one was using it and it's why when Shannon saw the application of it to engineering it was this astonishing thing right through mental break through yeah right I mean because we're talking about something that wasn't being
00:17:58applied in electrical engineering also and we'll get into this but I actually have some statistics here that by nineteen forty eight when he published information we're gonna get to that right now we're around nineteen thirty two to nineteen thirty six when he's in Michigan in nineteen forty eight
00:18:12the same year which and published on a medical tier of communication more than one hundred twenty five million conversations past daily through the bell system's one hundred and thirty eight million mines of cable and thirty one million telephone sets so by the time you publish that paper there
00:18:27was all this stuff for even called stuff would be a misnomer that was passing through the lines but how did you measure that right and this is why his application of boolean logic and the way he formulated information %HESITATION was so essential because it helped to make sense
00:18:44of this world that was emerging that we couldn't make sense of we couldn't wrap our hands around but I wanted to make that about billion logic amble in algebra so then he eventually was off to him I. T. in nineteen thirty six there he forms a relationship of
00:19:00Vandiver bush how important was that relationship to Shannon's life and how did that also take what he'd already learned and what he had studied in Michigan and the experiences of his error life and build on it so when Shannon is getting ready to graduate the university of Michigan
00:19:18you've got to remember this is a kid who now has spent his entire life in Michigan you know parents are northern Michigan they expect him most likely to come back and join the family business you know making furniture and he's got an engineering degree so that would be
00:19:32a plausible next up he's walking around Michigan and he looked to the Bolton born he sees a postcard tacked to it and it's a postcard that is looking for recruits to come to in my tea and help to work on a machine called the differential analyzer this is
00:19:47a room sized computer and he decides to send in an application and he's the person he's sending is application to is at the time one of the most important faculty members at MIT it man name Vannevar bush who later becomes probably one of the most important scientists of
00:20:03the twentieth century an adviser to presidents a wide ranging lecture one of the true geniuses of the twentieth century and he sees something we don't know what but he sees something insurance application and invite them to come out east and my tea and I think of this as
00:20:18one of the seminal moments and Janice I for two reasons the first is it takes him out of the Midwest and into a much more diverse energetic %HESITATION mainly because the big leagues the other is that he connects with someone Vannevar bush who is in many ways everything
00:20:33chan is not so van bush you know he knows how to network with presidents he knows how to talk to people he knows how to organize things these are all things that chain and has no interest in but the other biggest thing is Shannon is now entering M.
00:20:45I. T. to work on the most significant computer of that day an analog computer meaning it's different than our digital computers but the most sophisticated thinking machine of that time and Claude Shannon is there to learn from this machine and to learn from band bush it's analog in
00:21:04the sense that it acts out the algorithms of the world in a sense can you give us some examples are one example of what would be let's say a algorithmic computation in and I think the machine was called the brain I think that's what they tell us if
00:21:17you look at videos from back in the day like the brain actually does this and that and then it spits out this computation yeah it's actually we call it an analog computer because it's actually an analogy right so it's analogies what it's doing is it's actually acting out
00:21:32the answers to the kinds of things that it's being asked to compute so if you put in a graph actually the output is not a series of numbers the output is grass right that's an interpreted by the people running the machine so it has these we call it
00:21:45imagine a hundred ton giants foosball table that it's this gigantic series of disks and levers and pulleys and it's actually writing out graphically what the answers to our questions are acting out equations in trying to answer them and so it's one of the most unique machines of its
00:22:00time this was the foremost computing machine of its time and collision and is very lucky to get to spend time in this room where these gears are humming any does this for weeks and weeks and weeks in that's it's it's continuous waves as opposed to discreet and I
00:22:14come at significant you write in the book this action according to hear that quote in the midst of his work he came to understand that he knew another way of automating thought one that would ultimately prove far more powerful than the analog machine you then side of quote
00:22:30by certain logician at the turn of the twentieth century guarding chan's insights about building a general purpose machine and you say quote as a material machine is an instrument for economizing the exertion of force so a symbolic calculus is an instrument for economizing the exertion of intelligence logic
00:22:49just like a machine you right was a tool for democratizing force built with enough precision and skill it could multiply the power of the gifted and the average alike I love that quote and I think it speaks to another insight which was this idea of moving from a
00:23:05special purpose to a general purpose machine something that we all are devices are today this is what the personal computer is what was that Ross S. like from your research for Shannon and while he was at MIT applying his ideas of logic this analog computer into these notions
00:23:23of computation in the sense and how did he begin to arrive at this notion of discrete ness in messaging and information so he is spending you know many months with the machine that has switches right open closed open closed open close any seeing these switches actually control the
00:23:43flow of information the flow of life love something yeah there's something I saw something glowing right he then remembers his undergraduate study in bullion logic and remembers that there's zero one market open close true false and the insight that he comes up with which Walter Isaacson later called
00:24:03the underlying foundation for all digital computing people refer to it as the most important master's thesis of the century which is really saying something is that you can use this kind of logic to more elegantly and easily design circuits right so you can break down what at the
00:24:20time was a really convoluted trial and error process in the very very very simple math did that not exist before nope that was a total break through was a complete break there it was a break you know also part of it is a product of happenstance it was
00:24:34very rare to have somebody who was an expert in switching also studied mathematical logic and the logicians warned studying switching right so you had somebody for whom actually engineering this room this room size computer the differential analyzer proved essential to this masters thesis and you know even at
00:24:52the time when it was published people recognize that this was something significant Claude Shannon is invited to go to Washington DC to give a speech about it he's given the noble prize and this is not the Nobel Prize it but it's the most important prize for young engineer
00:25:07so he's marked out as somebody who is able to combine field and really interesting ways and I think that for me for somebody who doesn't come this discipline as a mathematician one of the most important things I learned from doing a life of Shannon is how important it
00:25:20is to study diverse fields to look at like places that are completely unfamiliar to you to understand something because had it not been for the fact that he studied George bowl you know digital computing may have been set back god decades that's the single biggest value proposition of
00:25:34the show this is why we cover so many different topics it's exactly for that reason because there's no one feel that's going to cover all the different things that might give you the inspiration need to help you solve that problem bring that creative approach that's actually right and
00:25:48I think that there's value in that not just from the perspective of my understanding Claude Shannon but in trying to improve ourselves right and I think some of the finest thinkers that I know tend not to be one trick ponies clutch and certainly in that in that field
00:26:00as well that that he's got a diverse array of interest and it leaves as best a few points to drive home one is the paper that you're referring to is a symbolic analysis of relay switching circuits such as master's thesis remarkable even more remarkable to me that paper
00:26:13the fact that I didn't know about that paper other than glanced over inflexible but even like I don't know that he made a big deal of it it was a big deal enough that I didn't know about a mathematical theory of communication the fact that I didn't know
00:26:25about this I found this is down in the fact that he wrote this back then and whether he's twenty one as remote I'm is actually remarkable after remarkable they push anyone to shame but then another point I really want to drive home here which is that he's looking
00:26:37at this computer this analog computer and it's doing all this stuff and it's creating this output and what he's saying is that there is value in the in between and that the in between carries the capacity for expressing computing and storing information and of course who knows where
00:26:54if you thought of it as informational time how old this is all coming together in his head I find astounding but it just shows you how many years it took between a clear articulation of some foundational ideas that were incorporated in his nineteen forty eight paper how many
00:27:09years between that and that paper was published and so much else must of got into it and that baking and that simmering of thoughts and to that point about interdisciplinary approach to learning after M. I. T. van bush put him at Cold Spring harbor laboratory let's just briefly
00:27:27sort of touch on that really to drive that point home and he was studying Mendelian genetics I believe yeah this is one of the least remembered parts of Shannon's life it's ironic because it's actually the subject of his PhD dissertation but van bush thought it was very important
00:27:42for mathematicians and for scientists and he was training to be flexible to be open to studying your field so he urges Claude Shannon to go to New York to go to Cold Spring harbor which at the time is the world's biggest warehouse of genetic information as the Rockefellers
00:27:59eugenics experiments makes him is has a it has an interest in its origins and some frankly someone fortune destinations and it's a really I visited and you can actually still see the cards where they try to predict which genes will show success in chess or will show infidelity
00:28:17or also a love of the sea they still have these cards and they're you know by science than bunk science now but it's a really really useful collection of data so Claude Shannon is there and what he's trying to do is essentially do what he did with his
00:28:29master's thesis apply algebra two genetics to see if you can figure out when with a gene show up in a population to look at the start to look at alleles and crumbs I'm very closely to say is there a way to deduce mathematically what we are trying to
00:28:42look at biologically and so he's applying algebra two genetics in a way that's never been done before and while it's not quite as astounding breakthrough there are scholars later said you know he was about a decade ahead of the field and this is a guy who'd never studied
00:28:55biology remarkable wife told you have as a kid I did a summer camp there is %HESITATION but I think in early high school or something like that it's an amazing place and it's a beautiful little small intimate campus yeah it's incredible to right on the water and it's
00:29:08one of these places where you just get the sense that it's almost set apart from the world so that people can think about big important things like population genetics %HESITATION at the same time you know this that history Israel %HESITATION shadow is well that was a time when
00:29:22you Gen X. and you Janice ism as a movement was really on the rise both across the Atlantic both in the United States and in Europe soap and speaking of that this was during World War two right now we're touching on this is exactly why this was happening
00:29:35at the time this is in the middle of World War two right when the United States is getting into the war right so this is an important thing I think to touch on even though it's not do your earlier point that knowing something the biography of a scientist
00:29:49the biography of mathematician of these intellectuals is important certainly insofar as informing us about the way that the person thought the things that compelled them towards the breakthroughs that they had this is an essential time for Shannon and would be good I think for all of us to
00:30:03sort of just take a moment appreciate what it means to be a young man living in the ad states the Japanese have just bombed Pearl Harbor right before right after he went to Cold Spring harbor is after right after okay so but it was while he was there
00:30:17another was while I was there because the only spend the summer okay spring and then he goes to Princeton right after that yeah so he finishes up in my tea and heads to prince in on a fellowship okay said he had actually gotten that fills before with the
00:30:27Cold Spring right that was like yeah it was it was about the same time that he was having the Cold Spring is when they apply it was around that time is applying for it and he wins what is it NSF National Science Foundation fellowship procedures than the procedures
00:30:40now and he is able to go to the institute for advanced study which is located in Princeton New Jersey now people take great pains to remind you that the institute for advanced study is actually distinct from Princeton University there on the same campus but they're very different but
00:30:57institute allowed people to basically come and work in a free wheeling environment it also became a place where refugees from Europe the most famous scientists of that day sure going to the I SO Herman while is there Kurt girdle is there Albert Einstein is there and into that
00:31:14milieu drops Claude Shannon complete with a fellowship with a great you know list of mentors in a couple of papers that are pretty significant and interestingly enough despite being drop in that milieu is you appropriately call it I think I draw this inference from your work as a
00:31:29result of the fact that he was a partly as a result of the fact certainly that he was not a gregarious guy that was not necessarily advantageous for him and he was also going to a divorce the time right to his first wife was married to about a
00:31:42year on character for the time certainly the war the uncertainty of what would happen to him the real possibility he might be drafted that he might lose his life or lose a limb and the uncertainty the natural normal uncertainty of being a young person at the beginning of
00:31:58your career all those things come together and this person was otherwise playful and light hearted seems throughout most of his life went through a period of melancholy yeah you nailed it on the head there's a dark period in Shannon's life the time at Princeton is the dark period
00:32:13and one of the reasons the clouds hanging over him and all young men from that period is the war so it is just as he arrives at Princeton that the draft order is gives given by President Roosevelt and so all young men are of a certain age or
00:32:28are eligible to be drafted and Shannon is in that group and it is a source of huge anxiety he believes that he is frail he doesn't believe you'll be well suited for army life he is someone who believes that he could put his mind to use in the
00:32:43war effort but not necessarily as body and ease just afraid of being drafted afraid of having a scientific life which is now spend you eight nine years cultivating derailed by the war and so in a stroke of incredible fortune his mentor's find him a contract working for bell
00:33:00laboratories on the war effort Danny's not drafted but he's put to work on what's called fire control which is basically the science of how you should things down from the sky anti aircraft guns yeah exactly yeah there's an interesting passage about that I don't know that who was
00:33:14in your broker from confusing it were something else where they actually went into specifics about the level of mathematical precision that went into designing the anti aircraft weaponry which is just fasting right to think about that I mean this is just remarkable you wanna talk about being on
00:33:28the frontier of again this on the cusp of the transition action not even really on the cusp this the germination of the digital revolution right the informational structure the road map which is what Shannon did so he goes to bell labs and bell labs is really interesting anecdote
00:33:48of him having tea time for about two months is that right with during a nineteen forty three now as I understand it touring and my correct was in nineteen forty three nineteen forty three from my understanding touring was essentially set the U. S. in order to kick the
00:34:03tires on the US cryptography team the team involved in building the site for of Roosevelt and Churchill square or whatever other green trip didn't communication across the Atlantic because the Brits were worried that the Yankees were going to drop the ball on it yeah tells a little bit
00:34:20about this because these are two giants obviously Alan Turing more notably known for his role in computing and computation and of course and is the father nation there yeah it's an extraordinary moment in twentieth century history and it's one that's overlooked turning it is sent by the British
00:34:37to the United States to essentially make sure the communications coming from the US to top level leaders in Britain are going to be secure you do this through the science and mathematics behind cryptographic analysis in cryptography specifically how do you encrypt a phone call so that the Germans
00:34:54can't decipher it and then use that information changes to an English you know they did it was in any case they did in the longer is that yeah I went though yeah the a command she's looking right language yeah but basically what trying is here to do is
00:35:08to test our systems to see will they be secure and safe one of the places that he has to go is bell laboratories and one of the people he meets they're also working on cryptography is Claude Shannon and it's an extraordinary moment for a couple reasons the first
00:35:23is they do end up getting together for tea every day in the Bell Labs cafeteria and there was something really remarkable about the idea that these two giants of the information age we're having tea and talking together you know at a modest cafeteria at bell laboratories and the
00:35:37other reason it's remarkable is because neither of these guys have a ton of friends and so the fact that they made friends with each other I'd like to think suggested they recognize just how smart the other one was in college and even later would say that he was
00:35:49blown away by turnings intellect they stayed friends collection also invites turning over to his home in the west village which collagen invited no one orders on the west village so again it is saying something that they became friendly they would reunite after the war and without missing a
00:36:06beat the Shannons are visiting during it is home entering invites Claude down to go see a computer that is building downstairs is like one of those great so far you know bromance is in in technological history so I think this is appropriate moment to talk about the bomb
00:36:23as you well yeah I think that's actually lifted from a quote of one of his colleagues John hires because he said it came as a bomb is the way he described in the paper the nineteen forty eight paper they wrote and then I think that's actually the title
00:36:35of your chapter on that why the bomb I think you've sort of illuminated us a bit about that by referencing how he works in solitude it just kind of came out but why the bomb and what was the immediate sort of reaction when it dropped yeah so we
00:36:52in the book we talk about how the paper the nineteen forty eight paper which is called a mathematical theory of communication collagen is most important work it's why he's called the father of information theory it was stunning in how surprising it was and there's part of that is
00:37:06Shannon didn't talk about the paper with anyone before it was published in nineteen forty eight which is just not done I mean in scientific circles you're constantly trading ideas you're talking other people but he worked on this paper in private essentially for ten years before he ever let
00:37:21anyone else have access to some of the ideas he was always in limited discussion with a couple of other very smart people but nothing like the collaboration that other academics and researchers were engaged in the other reason called the bomb not just that it came as a surprise
00:37:35but the scope of this work it not only asked but roughly answered all of the significant questions within the field of information very so he invents a field and then answered all the questions and so it's an incredible stroke in technological history and this paper is even today
00:37:53regarded as one of the true master pieces of the twenty century okay I want to nerd out with you right now by okay I want to get totally unstructured right now okay because this is an area that I've amateurish Lee read about and thought about for the last
00:38:08number of years maybe three to I rarely get to discuss it and I think the nature of the subject is such that even brilliant geniuses mathematicians philosophers who understand this field of study that far more than you are I still probably struggle with it in some sense and
00:38:24perhaps in the same way that physicists struck with the the notion of quantum theory right now as a sort of what is it that we can understand a mathematically but what is it I want to try and for our audience to get some sense of what is it
00:38:38we're talking about we we unite we mentioned James Gleick James Gleick refers to this in his book as a fulcrum that the nineteen forty paper was a fulcrum around which the world began to turn I like that expression because there is a sense in which things really began
00:38:57to change when we look back retrospectively the way we think about the world the way we think about information right and mentioned before the number of electrical lines that ran across the United States the telephones like thirty one million telephones the conversations are world was beginning to transform
00:39:16in terms of where and how it was happening how would you explain I know rob has talked about information as being well he I think he cites someone that or statement from Shannon chance said information resolved our uncertainty I think that's a good way to approach it there
00:39:34are many ways to approach it talking about in terms of entropy talking about in terms of randomness talking about in terms of computability how do you try to explain this to people who have let's say no idea of what we're talking about here yeah I throw out a
00:39:50few things so this is one of the more incredible papers in the twentieth century so I'll do my best to give the sound bites that are most helpful for people to understand it but I do think it's actually worth going back and reading it yourself even as a
00:40:02non mathematician non engineer you actually find it to be a very readable paper that's one of the things that makes the most remarkable is that even a lay person could get a rough sense of what the heck he's talking about when he talks about information the big thing
00:40:16to think about is at this time our understanding at the time machine is running our understanding of information is very immature there's a sense in which the only way that you can make a signal go from one place to the other if it's a little unclear if dole
00:40:33noisy is just by shouting louder by increasing the power within a message you just yell right %HESITATION there were there any analog systems yeah and what Shannon showed the repeaters the way you got a message across this unit of the distance had to travel and yep Oosting that
00:40:47boosting at all you do is just try to boost the power so got louder so that he could make it clear in you to accept that noise was just an inevitability and you created more noise of course in terms of boosting exactly five eight iron right what Shannon
00:40:59does a few things the first thing that he shows is that you can actually quantify information so Shannon is responsible for inventing the bit the runners are eight bits ten bits twelve beds Janet invents the bit it short for binary digit prime and it's a way of measuring
00:41:13information which turns information from something ethereal and a little hard to pin down into something that mathematicians physicists and engineers can actually use connection compute right that's sort of big breakthrough in site number one is let's quantify informational in algebra boom right there yeah then he has a
00:41:31diagram a famous diagram that shows the different components in any kind of information source so this is like the transmitter should have it in front of you obviously but it's a famous diagram that shows any kind of information has an information source a message transmitter signal noise a
00:41:47recipient and a destination and so he actually makes it so that anything like a radio signal a telegraph signal of photo a song bird song they're all the same they can all be brought under the same rules which is really important one of the things that defines field
00:42:05is when people can actually bring rules to bear on those fields of Shannon finally brings information he can govern information with these common laws one of his other insights is it information is probabilistic by which we mean that when he says the information is the resolution of uncertainty
00:42:22what he means is that you can predict with some accuracy how much redundancy there is in a language so good example of this that I like to think about is the letter Q. in the English language is almost always followed by the letter you write he makes of
00:42:37something like seventeen percent of all the letters used in any kind of English writing right there are these sort of rules he's like predictable patterns and words as well Sir the proceeds let's say subjects are setting the Malaga's and what that allows you to do if your Claude
00:42:51Shannon and if you're trying to think about the transmission of information is if you wanted to communicate a sentence you could strip away all the vowels in a sentence and still have someone understand it right so let's say a sentence at a hundred characters and you strip out
00:43:06all the vowels and now let's say it's seventy characters well you've actually just compress to that message without losing any of the meaning why is that important because that means you don't have to shout louder to get those hundred across yet you just have to say less you
00:43:20to say it more economically so that was one of his big break their insights is because of how redundant languages are he actually estimates that English is like seventy five or eighty percent or done it it's kind of funny when you think about it but he estimates a
00:43:32because of the redundancy in language we can say less we can say things with economy and thus transmit them with more accuracy so that's one of his big insights I was gonna say also with the corollary of that be that if you want to say things with more
00:43:46certainty that instead of amplifying the message you would increase the redundancy in the message in our to account for packet loss yep so that was the next big breakthrough in sight wasn't just that you can compress it was actually that you can use redundancy to create where called
00:44:00error correcting codes so for example if you wanted to make sure that a certain message like let's say the letter a is represented with like let's say a is zero one right the first already off at zero one it's one in order to make sure that that's transmitted
00:44:15with a hundred percent fidelity you could adizero an ATA one SO zero zero one one meaning that if like you are transmitting across the channel and one of his heroes got locked off for one of the ones got lopped off you would still have the fundamental lessons zero
00:44:29and one and what he realizes that you could use redundancy this kind of redundancy to protect messages that were being sent and his big breakthrough insight is that you can send any message with almost perfect accuracy over a channel that that is actually theoretically plus channel limit you
00:44:45can to the Shannon limit the thing is that was not an idea that anyone had everyone assumed that noise was just a problem that had to be dealt with in that the only way to do it was to shout louder Shannon realizes you can both compressed and code
00:45:00messages to transmit them with perfect fidelity and that is something that is the underpinning architecture for everything it's the reason you can watch a YouTube video on iPhone and it doesn't take seven years to watch the video because it can be compressed because he be communicated with error
00:45:15correcting codes that allow it to be presented with accuracy will that third thing you mentioned so we talked about bullying algebra we talked about electrical relays and switches the third one was the statistics statistical structure understanding that there was a statistical structure that underlay I don't know how
00:45:33he would say that everything in terms of of information the waves thinking about it there are many other things I want to say before I do I want to actually pull it just pulled up the paper here and for me there's this paragraph is a second paragraph in
00:45:44the paper inside the most cited sentence and then that actually had a couple of sentences to it this is Claude Shannon speaking now introducing information there in this paper the fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at
00:46:03another point frequently the messages have indentation here meaning they refer to or are correlated according to some system with certain physical and conceptual entities the semantic aspects of communication are irrelevant to the engineering problem the significant aspect is that the actual message is one selected from a set
00:46:24of possible messages I mean that capture so much right there not only is to talk about the statistics but is also talking about the physical fact that information does not mean a physical we have this metaphysical notion of information but it ultimately resides in a physical reality which
00:46:40lays the groundwork for the work that's been done in information theory and entropy and thinking about information there in terms of the physical sciences which it is entropy it is disorder in a physical system as the universe expands from the Big Bang if we take that theory the
00:46:56universe generates more information and that a fully entropic universe at completely sort of thermal death universe has the most information and its caption the sense and this also I'd love to ask Jimmy we think about this because this is all over the place now but it's %HESITATION just
00:47:12gonna get got I warned you hear it's the encyclopedia says the admin right it's an inch there's another great quote by %HESITATION Henry punk care here I've mentioned before chance is only the measure of our own ignorance fortuitous phenomena are by definition those whose laws we do not
00:47:29know this idea of do we live in a deterministic universe or do we not and is complexity science and complexity theory is complexity just simply V. reflection or the theory created by a mind that doesn't have the capacity to compute the universe doesn't have the full level of
00:47:49intelligence to be able to see all the causality in the world which generates the sort of you know dry hump open not glazing over here now I don't know if you think about all that I just threw a mishmash of ideas here dozer series of big and very
00:48:04related questions all offer a couple of insights the first is that there was a reference someone called Shannon's paper coupon again so I can partake in in the sense that before Kaepernick as it was just assumed that the sun orbited the earth an afterburner guess we know that
00:48:20the earth orbits the sun in the same way Shannon's findings were component can he turn the entire idea of information on its head and took it from something that we couldn't measure to something we could measure something we couldn't manipulate to something we could manipulate something that had
00:48:37very few physical properties are ideas that seem like physical properties to something that could be very carefully statistically calculated so that just give your readers a broad sense that's why this matters is because before Shannon information was not this kind of hard science engineers did not have the
00:48:54conceptual tools to think about communication like this right that the thing I'll say about and should be in about the probabilistic work the best analogy the chanting uses in this paper is the flipping of a coin right and he's as a fair coin that's weighted equally to heads
00:49:09an equally details if you flip it it actually communicates a lot of information you don't know what's gonna happen you're uncertain about what the outcome is going to be so when the outcome is either heads or tails you have one bit worth of information right he says a
00:49:22coin where both sides are heads communicates no information because it's entirely predictable biggest a hundred percent predictable it communicates zero bits of information and vat idea it can be a little you know complicated to think about but if you just say it that way let me finish choice
00:49:40if the rate some choice and it makes a lot of sense means information is the resolution of uncertainty it means it's something like a coin it's weighted with both sides of our has two sides of heads is actually communicating nothing it's looking utterly nothing it has no bits
00:49:53of information to it right whereas uncertainty not knowing is what we're trying to resolve and we communicate also I think in the way that's interesting there so many examples that are useful and I think all of them are useful in something like this because redundancy in the message
00:50:07right you want to get this redundancy in your head is music jazz if you have an hour of jazz if you want to compress that that takes up way more information way more space on the hard drive then let's say one hour of I don't know what's that
00:50:23song from the south like Dixie my does your something some simple you know mundane you know percussion of a drummer something because it there's predictability statistical structure in that other our in so you can compress it there's far less information yeah it's appropriate by the way you chose
00:50:40jazz because this was among the things a cloud chanting care a lot about when he would get into from the debates he cared nothing about politics but he would argue with you to the bone about who is the best jazz musician and he would go to clubs in
00:50:53the west village and just sit there and and sort of nurses cigarette nurse to drink when you could smoke indoors he would sit there and just watch jazz musicians for hours on end lose himself inside the music and so it is actually we think it's important and interesting
00:51:07that jazz is a central part of his life because there is so much uncertainty and complexity in that kind of music that maybe there isn't another styles and again I'm not making a judgment about the music comes in talking about the kind of the order of the chaos
00:51:20within jazz as compared to other styles of music you know again I'm thinking of Brownian motion I'm thinking of Heisenberg's uncertainty principle I'm thinking about to take the opposite side of that our capacity to see patterns where patterns don't exist right and to infer meaning where none exists
00:51:41all of these are so interesting you and I were speaking before what went on there about what is a random number well to choose a number by inherently means that it's not random I hadn't %HESITATION they're actually been books written that produce random numbers that were created by
00:51:54algorithms which I like St hi sin so these notions of in computability incompleteness uncertainty randomness intelligence these are so interesting to me it's at the intersection of so many different fields I guess what I want to do with this I took it as far as I could take
00:52:12it because it you get kinda lost when you try to mention Brownian motion that has to do with again entropy in the random motion of particles in a fluid but there are so many interesting ways to approach a subject it's so valuable to study and I do implore
00:52:27anyone and everyone to kind of look into it and begin to get into it I didn't even mention girls and completeness they are there some aspect of that as well that that feeds into it you can feel it when you study these different theories and and touring talked
00:52:38about an algorithm being a way of expressing information like the shortest algorithm is the shortest way of expressing that and anyway it's all part of that so I wanted to make that point we're not gonna have a chance to get into his later years which are fascinating I
00:52:55think from the standpoint of how much of his life happened after he both labor and %HESITATION he of course he went back to MIT and he taught and I think he retired in nineteen seventy eight but he died in two thousand and one Italian one it's really wonderful
00:53:12Jimmy I'm remarkably impressed by not just your coherence in being able to talk about the subject but the extent to which you understand it is I think remarkable especially given the fact you're only what thirty two I mean I just think it's I mean for me to say
00:53:29that I I'm thirty two now but you look at what Shannon had done by the age of thirty two by the age of thirty two he published the most important masterpieces of the century and then at twenty thirty two that he publishes the information theory paper and I
00:53:41have written a couple good books Ivan and that's got a hold on those everyone of course so that's cool that the fact that you study that but it's it's really great but I want to really emphasize that point because it's valuable for anyone is going to be reading
00:53:53your book to understand that you understand the stuff we've thought about it to this degree because you're not just writing about a life or writing about a scientific life and I think these things are relevant I appreciate that in a minute means a lot to hear you say
00:54:03that I think the other thing about the about it is that I think I'm not a scientist don't play when TV my math studies and did in high school and part of the reason I think non experts could approach this book and read it and not get bogged
00:54:20down in the terms of the equations is because neither Robb nor I were experts when we wrote it so we had to learn the stuff from the ground up and then communicated as learners not as experts and so for your audience and for other people it's actually one
00:54:34of the virtues of scientific biography in a way is that if you read for example I I don't mean to just brag on our book but is a really wonderful book called a beautiful mind and it's by a woman named Sylvia noster was one of our models for
00:54:45this book and one of things it's great about the book is you don't have to be an expert in game theory or an economics to read that book and get a lot out of it right and we like to think that part of our book yes it's an
00:54:56introduction to Claude Shannon the man it's also an introduction information theory the idea introduction to circuit design the idea introduction to you know how to program a computer to play chess which is one of the things that Shannon did so I hope and it sounds like we did
00:55:10get to a place where the ideas in the life balance each other out is I think one of the virtues of starting someone like Janice for and you know how that someone like this come up with these ideas yeah I love it man I appreciate you writing it
00:55:22and rob as well he's here in spirit the non physical body tell our audience your Twitter handle how they can learn more about you that paper back is coming out in like a month right coming out in July %HESITATION an excellent anniversary the seventieth anniversary of the question
00:55:35is is very appropriate at the at the seventieth anniversary of the information theory paper this year and actually the other thing your audience should know is there's a movie coming out so it's called a bit player and it's by a film making a mark Levinson whose other film
00:55:48was a film called particle fever which is all about the search for the Higgs bosons a guy that was a great %HESITATION documentary so his next documentaries on college item entry yeah %HESITATION you work with you guys %HESITATION yeah he did so with such a mix of documentary
00:55:59antibiotic he got an actor to play Claude Shannon and this actor looks exactly like Claude Shannon and then part of it is interviews with experts robin are both in the movie bunch of others people from his family and it's so good I mean just as good as particle
00:56:15fever if not better I'm so jealous when is that coming out that's fantastic he's debuting at a conference in a couple weeks and then he's gonna probably look at you know it's gonna go to film vessels the whole process I don't know anything about but it's the first
00:56:25feature length look at clouds hands life and I think I'll introduce a lot more people to own one we get you robin him the director here will do a Charlie Rose situation I'll I'll gladly come back any podcast it's talking about Brownian motion our audience geeks man I'm
00:56:41I really wish that they could they could meet %HESITATION rob as well because he's such a geek he's such a geek so I really appreciate you coming on that thanks so much right yeah it was great to talk to and that was my purse so with Jimmy Sony
00:56:56I want to thank Jimmy for being on the program today's episode was produced by me edited by the end of the fallout four more episodes you can check out our website at hidden forces dot I %HESITATION follow us on Twitter Facebook and Instagram act and forces pot for
00:57:13send me an email at D. K. hidden forces dot heiau thanks for listening we'll see you next week

Transcribed by algorithms. Report Errata
Disclaimer: The podcast and artwork embedded on this page are from Demetri Kofinas, which is the property of its owner and not affiliated with or endorsed by Listen Notes, Inc.


Thank you for helping to keep the podcast database up to date.