Hey, Admiral Blair! Did You Get Tetlock's Memo? (NationalInterest.org)

Philip Tetlock's Expert Political Judgment is required reading here at Mercyhurst. While some analysts take issue with Tetlock's findings, I have yet to see anyone create as compelling a study as his regarding the essential qualities of a good forecaster.

So, I pay attention whenever Tetlock writes something, even a book review. Thanks to my colleague, Steve Marrin (who knows of my interest in all things Tetlockian...), I recently had a chance to read Tetlock's review of three new books by people who claim to have an inside track when it comes to predicting the future.

I will let you read Tetlock's review of the books on your own. Suffice it to say that his critique of all three works is based on his research and, as a result, is skeptical (in varying degrees) of the claims of accuracy in the three books.

What really fascinated me about this review is buried at the end of it, though. Tetlock worries about how to improve forecasting, about how we can know which forecasters are worth listening to and which are modern day snake oil salesmen. Then, he makes a point that I agree with in whole:

• "There is one potential savior on the horizon: a big institutional purchaser of forecasting services that has the financial clout and technical-support staff ready to run forecasting tournaments that would shed light on the relative performance of competing approaches—a big player that also has powerful incentives to discover superior analytical strategies, for even small improvements in its prediction accuracy can translate into billions of dollars and millions of lives saved. And that player is the Office of the Director of National Intelligence."

Right on, Professor T! The DNI has every reason to want to improve forecasting and has the purchasing power to insist upon it. Specifically, according to Tetlock:

• "Players high up in the political system—who really do want the best-possible forecasts—could decide that it is worth investing a nontrivial share of their intelligence agencies’ budgets into a series of long-term forecasting tournaments designed to distinguish the more from the less promising forecasting approaches across policy problems."

The DNI dipped its toes in this water back in 2008 when it sponsored the Open Source Challenge. Since then, of course, we have not had another Challenge or another Open Source Conference. This does not bode well for Tetlock's very reasonable suggestion.

Still, it is a great idea. If we are ever to break out of the "two-thirds right" trap we are in, we need a robust research program aimed at validating intelligence methods and the DNI will have to be the one to sponsor it.

ONI's New "Hoo-ahh!" Video, Deconstructing Analysis Techniques, The Geography Of Job Loss And The Future Of Shopping (Link List)

Lot's of interesting stuff crossing my desk this week:

• The Office of Naval Intelligence has a new promotional (i.e. "hoo-ahh) video out. It gives a brief overview of the ONI's new organizational structure and mission. Many people don't think about ONI as an intel career option but they actually do some pretty cool stuff. The video is certainly worth 5 minutes of your time (Note: It takes a few minutes to get started (I don't understand why these guys don't just upload these videos to YouTube...). Also, if you are interested, see it quickly as Matchbox Twenty's lawyers may slap a take-down notice on the ONI for unauthorized use of copyrighted material (not even a music credit, ONI? That was cold...).

• My favorite online design magazine, Johnny Holland, continues its excellent (if slow-motion) series of posts on deconstructing analysis techniques. The current entry focuses on transformation or "the act of taking a set of values from a dataset, processing them in some way (depending on the aims of the research) and arriving at a new set of values with the goal of revealing some aspect of the data from a new perspective." The entire series is very good and well worth the read.

• Visualizing information is a powerful way to communicate analysis. A good example of this is Tip Strategies infographic showing job loss and gain in the US from 2004-2009. It is both stunning and depressing but clearly shows the value of a good visual (Sorry, no embed. You will have to go to the site to see it).

• One of the trends I have been tracking now for a number of years is the growth and influence of virtual worlds and augmented reality. Cisco has recently produced an interesting video (via Kzero) that explores the future of shopping (you can see the video below). It sounds mundane but the underlying technical implications are pretty significant to intelligence and become doubly so when you realize that this world is just around the corner.

Intel Legend Art Hulnick On The Future Of OSINT (ISN)

The ISN, on one of its recent podcasts, scored a very good (if too short) interview with Professor Arthur Hulnick (see picture at right). Art currently teaches intelligence related courses at Boston University and has for a number of years. Before that he was at the CIA for several decades and has contributed significantly to the open body of literature on intelligence through his books (including Keeping Us Safe: Secret Intelligence and Homeland Security (2004) and Fixing the Spy Machine: Preparing American Intelligence for the 21st Century (1999)) and many articles.

Art is one of those guys who has been around, as we used to say in the army, "since Christ was a corporal" and is always worth listening to. Unfortunately, the ISN did not make an embeddable version of the podcast but you can get it on iTunes, download the MP3 or just go to the ISN site to listen to it.

Surreal Saturday: Saturn's Hexagons... (DailyMotion Via Neatorama)

...is not the name of a rock band. Nor is it the name of a science fiction novel. It is even weirder than that.

It is a real phenomena on Saturn that the Cassini probe confirms has lasted for at least 20 years.

Very cool video on DailyMotion (via Neatorama) below along with very interesting possible explanation on Softpedia (also via Neatorama)...

The Mystery Hexagon on SATURN
by the-pho3nix

Reblog this post [with Zemanta]">

Building A Better Analyst With Technology (10/GUI via FlowingData)

One of the ways to make analysts better is to make them faster. It has always been surprising to me that the various intelligence communities don't do more research into speed reading, for example, in order to figure out how to make analysts be able to access information faster on the job.

Another fairly archaic approach is the QWERTY keyboard and mouse as the primary input devices for computers. There have been lots of attempts to improve on this model (voice recognition, for example, is pretty much there already) but none of them have been able to knock King QWERTY and his trusty sidekick Mouse from the hill.

The most recent entries have been touchscreens but there are lots of problems with them beyond mobile and occasional big screen use (see CNN at just about any time of the day or night...).

The latest entry into this competition is 10/GUI. There seems to be some pretty interesting human-computer interface oriented thought that has gone into the tool. Not sure if it going to sweep the nation but it is worth reviewing and thinking about. See the video below...

10/GUI from C. Miller on Vimeo.

I thought it was pretty cool but I have two concerns. The first is learning the new workflow. I am not too worried about that as it looked like something I would pick up fairly quickly (I figure you have to put in some time on any new piece of equipment and this doesn't look like it would take much more than normal).

I think I would also like to see the left hand functions and the right hand functions on two separate pads, one to the left and one to the right (or at least have this as an option). It looks like it is going to be cramped (particularly for people with big hands).

How would you improve the interface? Leave a comment!