Tuesday, September 8, 2015

Fermi Questions: Creating Intelligence Without Collection

Collection is, for many, a fundamental part of and, in extreme cases, the essential purpose of, intelligence.  What would we be without all our drones and spies and sensors?

What if I told you that you can do intelligence without any collection at all?

You probably wouldn't believe me ... but ... you'd likely admit that the advantages would be substantial.  It would be blazingly fast - no waiting around for satellites to come into position or agents to report back.  It would be mindnumbingly safe - virtually no footprint, no assets to risk, no burn notices to issue.  It could reduce as much as 90% of the uncertainty in any given intelligence problem at essentially zero cost.

What is this prodigious procedure, this miracle methodology, this aspirational apex of analytic acumen?

Fermi questions.

Enrico Fermi was a mid-twentieth century physicist who created the first nuclear reactor.  He also taught physics at the University of Chicago.  He liked to ask his students questions like, "How many piano tuners are there in Chicago?"  

In the pre-internet days, this kind of question required a tedious trip through the phone book to determine the number.  Even today, using brute force to answer this question is not a trivial exercise.  Students almost always balked at the work involved.

Fermi's approach, however, was different.  He wasn't asking, "What is the most direct route to the answer to this problem?"  Instead he asked a slightly different and, for intelligence purposes, vastly more useful, question: "How close can I get to the answer with what I already know?"

So.  What did Fermi already know?  Well, the population of Chicago is about 3 million and from this he could immediately devise that there could be no more than 3 million piano tuners and that the minimum was none.  That may not sound particularly useful but just recognizing these facts already limits the problem in useful ways and points the way towards how to make the estimate better.

We know, for example, that the number of piano tuners has to be driven by the number of pianos in Chicago.  How many of those 3 million people have pianos?  Here we could tap into our own experience.  How many people do you know?  How many of them have pianos in their houses?

Some will say 1 in 10.  Some might say 1 in 100.  Even this wide range is very useful.  Not only does it narrow the problem significantly but also it highlights one way in which we could get a better estimate if we absolutely have to (i.e get a more exact number of people with pianos in their houses).  But we want to do this without collection so let's carry on!

With the average household being a shade under 4 people, we can estimate that there are about 750,000 households in Chicago.  We can further refine that to between 75,000 and 7500 pianos (depending on whether you thought 1 in 10 households had a piano or 1 in 100).

Oh, I know what you  are thinking!  What about all the non-household pianos - at schools and such - that you are conveniently leaving out.  I would say that my high end estimate of the number of pianos includes them and my low end estimate does not so they are in there somewhere.  It is a "good enough" answer for right now for me.  For you that might not be the case, however, so you can make your own estimates about what these numbers might be and put them into the mix.

Working about 250 days a year (weekends, vacation and holidays excluded) on about 2 pianos a day means that Chicago needs between 150 and 15 piano tuners.  

How many piano tuners are there really in Chicago?  Wolfram Alpha is one of the best search engines to use to answer these kinds of questions.  It permits users to ask natural language questions and then dips deeply into public databases to extract precise answers.  When asked, "How many piano tuners are there in Chicago?" this is what you get:

Note that Wolfram gives us the number of all musical instrument repairers and tuners - 290 as of 2009.  Certainly not all of them are piano tuners.  In fact, once you consider just how many instruments need to be professionally tuned besides pianos and you subtract the number of repairers of all kinds of instruments that do not tune pianos, you are lucky to have a third of these musical instrument repairers and tuners who actually can tune a piano.

More importantly a third of 290 falls comfortably within the 15-150 limits derived from our Fermi process.

Without leaving our chairs.

Intelligence without collection.

What if relying on Fermi questions results in really wrong answers?  First, I could say the same thing about any intelligence methodology.  Very few of them have been tested to see if they actually improve forecasting accuracy and all of them take time and resources to implement.  All of them can be wrong.  Here, at least, both the logic chain and the path to improving the estimate is obvious.

Second, I would ask, what level of precision do you actually need?  Norm Augustine, former CEO of Lockheed Martin used to say, "The last 10 percent of performance generates one-third of the cost and two-thirds of the problems."  Augustine was talking about airplanes but he could have just as well been speaking of intelligence analysis.  Getting ever more narrow estimates costs time and money.  Good enough is often - in fact, surprisingly often - good enough.

Third, it is unlikely to give you really wrong answers - say one or two orders of magnitude off.  This is one of the best benefits of going through the Fermi process.  It allows you to have a good sense of the range in which the right answer will likely fall.   For example, if, before you had done a Fermi analysis, someone came up to you and said that there are 100,000 piano tuners in Chicago, you might not question it.  A Fermi analysis, however, suggests that either something is really wrong with your logic or, more likely, that the person does not know what they are talking about.  Either way, the red flag is up and that might be just enough to prevent a disastrous mistake.

You can easily try this method yourself.  Pick a country that you know little about and try to estimate the size of its military based on just a few easily found facts such as population and GDP.  Once you have gone through the process, check your answer with an authoritative source such as Janes - oh! - and please do not hesitate to post your results in the comments!

By the way, I routinely use this method to get students to answer all sorts of interesting and seemingly intractable problems like the number of foreign government spies working within the US Intelligence Community.  The answer we get is usually right around 100 which always seems to surprise them.

Finally, if you are interested in integrating Fermi Problems into your tradecraft, there are lots of good resources available.  One of the best has been put together by the Science Olympiad, which actually holds a Fermi Problem competition each year.

Friday, August 7, 2015

Cheap, Re-usable Cell Phone Microscope? Yeah, It's A Thing...

For the last couple of years I have been exploring the idea of intelligence support to entrepreneurs.  The cool thing about this is that I get exposed to lots of new ideas.  The most recent - and one of the most interesting - products I have seen is the Button Microscope.

This is a microscope that you can attach to the lens of any cell phone with a camera.  It immediately turns it into a powerful microscope.  To be honest, others have done much the same thing but their products tend to be clunky, DIY projects that require far more patience than I have for that sort of thing.

The Button Microscope just works.  More importantly, it is going to be pretty inexpensive to produce and re-usable as well.  

I can't show you the prototypes I have been playing around with this morning (top secret, hush-hush stuff, you know) but I can show you some of the pics I took with them (with zero training I should add).

This first pic is one of a piece of graph paper I had lying around.  I edited both the left and the right image for size and brightness in the online photo editor, PicMonkey, but other than that both images are straight from my cell phone.

You can get a little bit better feel for the power of the microscope in this image.  On the left is the venerable Intelligence Analyst's Deck Of Cards (still available for sale...ahem...).  On the right is a close up of the box (focused on the "L" in "Analyst's").  You can see that the microscope has a distinct focal point and that the image blurs some at the margins.  That may be an existential feature of the device or it may just be that I am a pretty poor photographer.  I'll need to play around with it some more to see.

To me, this is the most impressive image set.  On the left you see one of the playing pieces from my game, Cthulhu Vs. Vikings.  On the right you see an image taken using the Button Microscope from the top down.  These pieces were all printed on the 3D printer and the macro view allows you to see every layer quite clearly and captures a surprising amount of detail even as the playing piece recedes from the focal point.

The broader intel/investigative implications of a device like this are pretty interesting to contemplate.  Clandestine collectors who are looking to get extreme closeups of, I don't know, circuit boards and such will love it.  Investigators look for trace evidence or fingerprints are going to love it too (If you have a clever idea for something like this, drop it in the comments!).

When can you get one of these amazing devices for your own cell phone?  Well, we hope to launch a Kickstarter campaign in October to fund the initial production run.  

Next we will add a mass spectrometer (currently available for $249 - no shit) and we will be well on our way to a tricorder.  Oh, wait.  That's due in January.

Tuesday, July 14, 2015

How Did I Miss This? YouTube Now Has A 360 Degree Video(!) From North Korea(!)

You read that title right, sports fans!  360 degree videos.  As in you can now decide where you want to look, left, right, up or down in a video.  Take a look at this recent video shot by a couple of guys visiting North Korea...

How does it work?  Incredibly simply!  Just click the arrows in the circle in the upper right hand corner of the video image.  Take a look at the annotated screenshot to the right if I am not being clear enough.

Right now it appears to only work on Chrome or Android devices and I found that other videos (and there are a growing number of them) often had to pause to buffer.

The North Korean video was shot with an Etaniya camera and some specialized software.  Apparently YouTube (via Google) is working with the software and the hardware manufacturers to make it easier.

In the interim, there are some guides starting to be produced to help you get going making your own 360 degree video.

(H/T to WK!)

Tuesday, June 16, 2015

Why The Most Important Question In Game-based Learning Is "Who Will Fund The Game Genome Project?" (Part 2 of 2)

What's Missing From This Picture?

I hate Monopoly.  If there was a time when I liked Monopoly, I can't remember it.  Even today, when I dream of hell it features an endless game of Monopoly played with Hitler, Stalin and Pol Pot (Don't ask...).

What if game C in the image above (and also featured prominently in Part 1 of this series) is Monopoly?

All the cool databasing and meeting and organizing in the world aren't going to help me learn if I absolutely hate the game that is supposed to teach me.  Resolving this problem is tricky and it starts with the question, "What is a game?"

I am a big fan of Bernard Suits definition of a game: "Games are a voluntary attempt to overcome unnecessary obstacles."  
Note:  At this stage, it is typically obligatory to write a lengthy discussion about all the other definitions of "game" and how Suits succeeds in part and fails in part...blah, blah, blah.  You can find this sort of stuff anywhere - just Google it.  So, in the interest of time, let's just pretend I have already written this essay (OK, OK, "brilliant essay", if you insist).  Now we can get to the point.  
The key thing that the Suits' definition adds to the discussion of games in the context of learning is that games are voluntary.  Think about it.  If, at some level, the learner is not motivated to play the game by the game itself, it isn't really a game for that learner (kind of like The Hunger Games aren't really games for Katniss Everdeen...).

What Is The Game Genome Project?

If the missing piece from the picture above is the preference of the learner/player, then the question becomes, "How do we determine those preferences?"  To put it another way, if Rock and Country and Classical were insufficient to define musical preferences, why should we think that Role-playing, Collectible Card or First Person Shooter are good enough to define game preferences?

The truth is, we shouldn't.  The Game Genome Project would seek to do to games what the Music Genome Project did to music - break games down into their component parts, validate the relevance of those parts in determining player preferences and then test that system so that we can reliably predict game preferences across learners/players and genres.

Some of this kind of work is already being done, albeit without the focus on education.  Take a look at BoardGameGeek, for example.  BGG is arguably the web's best resource for tabletop games and its advanced search feature allows users to search by hundreds of categories, subcategories and mechanics as well as by number of players and playing time.  

The tens of thousands of amateurs and professionals who have contributed to BGG over the years have done very good work in crafting all these elements of board games but which of these categories actually matter?  And what about video games?  Do any of these categories and subcategories cross over?  

Yes, there is a lot of work to do but imagine if such a system were fully realized.  Teachers could go to one site, input their students preferences and the teacher's learning objectives and a list of games would pop up.  Even more important, a student, faced with a learning challenge could input his or her preferences and the learning objectives and find a list of games that would make the effort not only fruitful but fun.  

The ability to reliably connect learner/player preferences in games to learning objectives in classes across the full spectrum of tabletop and video games would, in turn, transform game-based learning from the pedagogical technique du jour to a lasting  and important part of the educational landscape.

Who Will Fund The Game Genome Project And Why Is This Question So Important?

If I am right about the importance of the Game Genome Project to the future of game-based learning, then who will fund it?

The first possible source is, of course, private investment.  A Pandora-like game recommendation engine makes about as much business sense as Pandora itself.  Pandora, however, let's you listen to music it thinks you will like and then makes money when you buy it (and ads, of course, but that would be true for any website).  

Since most games take longer than 3 minutes to play (or even to download...), it is unclear to me if this business model would work as well (or at all) for games.  More importantly, private investors are unlikely to want to invest in the hard work of tying learning objectives from all of the various curricula to the games.  It is something that only someone with deep pockets and a financial incentive (like an educational publisher?) might be able to attempt.

Government could do this, of course.  It looks like a good NSF or Dept. of Education grant, perhaps.  The military or intelligence community could certainly do it but would be highly likely to focus almost exclusively on a narrow range of skills and games.

Whoever will do it, it will have to be done. Until we are able to connect game to learning objective and learner to game, game-based learning is likely to remain a niche teaching technique, full of unrealized potential.

Monday, June 15, 2015

Why The Most Important Question In Game-based Learning Is "Who Will Fund The Game Genome Project?" (Part 1 of 2)

Way back in 2000, two researchers, Will Glaser and Tim Westergren, began what was then called The Music Genome Project.  It was designed to categorize music by more than 400 different "genes" or characteristics of the music.  The goal was to build a better music recommendation engine.

Today, this project is better known as Pandora.

Glaser and Westergren's fundamental insight was that breaking music down into broad general categories such as Rock or Pop or Country wasn't very useful when it came to making recommendations.  Some people liked music with male vocalists or heavy beats or a fast tempo and no one liked all of country music or everything produced that was labelled "rock".  

In fact most people liked a little bit of everything.  Sure, they had genre preferences, but that didn't keep the Jethro Tull fanatic from liking (and buying) the occasional Mike Oldfield album (ahem...not that I know anyone who would do such a thing...).

Thus the Music Genome Project was born.  By analyzing the genetic makeup of each song, the Project wasn't just able to better dissect individual pieces of music.  It was actually able to make reliable cross genre recommendations.  Oh, you like this driving, 120 beats per minute, sung by a female vocalist with lots of guitar distortion rock anthem?  Then you might also like this hip-hop track with many of the same musical genes!

What Does This Have To Do With Game-based Learning?

This isn't going to sound that earthshaking but it was to me the first time I realized it:  All games teach.  You can design a game that will explicitly (or implicitly) teach something like math or grammar but you don't have to.  With all of the good games, both video and tabletop, that are out there, it is not difficult to find a game that can be used to teach almost any K-12 and many university level subjects.  

How many classrooms routinely use Monopoly, for example, to help teach basic addition and subtraction or units of currency?  Monopoly certainly wasn't designed with this purpose in mind but it serves that purpose nonetheless.  

While I might be bold in my assertion that every subject is covered, I would argue that, if I am wrong, I am not wrong by much.  This is the golden age of gaming.  There are more games being produced (and more good games) than at any other time in human history.  The selection is already immense and growing.  In fact, it might be more accurate for me to say that, while I might be wrong, I won't be for much longer.

So, to put it more formally, you can connect all games to one or more learning objectives (See image to the left).  I am using the term "learning objective" loosely here.  Your learning objectives may come from a formal document, such as the common core, or from a less formal desire "to teach these darn kids something about X".  

Given the prevalence of formal standards in modern education, however, it is pretty easy to imagine (though infinitely less easy to actually do...) professional educators and gamers sitting down together and dissecting every game for the learning objectives that each game addresses (i.e. the things each game teaches).

Eventually - and, of course, you would start with the most popular games and the most important learning objectives - you would have a database that could answer the question, "What game teaches this?"  Almost certainly, multiple games will cover the same learning objectives and some games will cover more relevant learning objectives than others.  It is conceivable that a teacher would be able to query this database and find a single game (See image below) that adequately addressed all of the learning objectives for a particular block of instruction.

Next:  What's Missing From These Pictures?