Monday, February 3, 2014

Keeping the Human Touch in Big Data

When I was growing up, people believed that there could never be a computer smart enough to beat a human being at chess.  Now, we know there may never be a human being smart enough to beat a computer.

What nobody anticipated, however, is that the very best chess "player" happens to be a combination of man and machine, or what the chess world calls Freestyle.  Man + computer beats computer.  There are several reasons for this, one being that humans can see where different computers disagree--not unlike a weather forecaster looking at three or four storm models--and work within the variations to achieve a better result.

Of course, if chess is ever fully "solved," like checkers is today, than the machine is fine all by itself.  But then, fully-solved problems have never been things human beings spend a lot of time worrying about.

Another example where human touch applied to data improves the solution is genealogy.  In the old days before the Web, it took a Herculean effort to build a credible, well sourced family tree.  Research dead-ends could be easily encountered just three and four generations back.  Data was hard earned, sometimes involving plane trips and convincing priests and town administrators to grant access to their precious archives.

Today, Big Genealogy is moving toward a single world tree, with all seven billion of us connected globally.  Seach your name and birth date and, voila, there's your tree.  It's not hard to envision the day when the act of building a tree will be among the least important tasks a genealogist undertakes.  (Even as I write this, a 23andMe email is chirping that it's found me a "4th cousin, with likely range to 3rd or Distant cousin.)

If you're lucky enough to have a family tree, though, you know that far too many of the names and dates are just that--bits of data.  These real people lived long, full, interesting lives but are now reduced to a few dates and places.