A Programmer’s Guide to Data Mining: The Ancient Art of the Numerati

Cover-Programmer-Guide-DataMining (Book)Before you work through this book you might think that systems like Pandora, Amazon’s recommendations, and automatic data mining for terrorists, must be very complex and the math behind the algorithms must be extremely complex requiring a PhD to understand. You might think the people who work on developing these systems are like rocket scientists. One goal I have for this book is to pull back this curtain of complexity and show some of the rudimentary methods involved. download-buttons-75 Granted there are super-smart people at Google, the National Security Agency and elsewhere developing amazingly complex algorithms, but for the most part data mining relies on easy-to-understand principles. Before you start the book you might think data mining is pretty amazing stuff. By the end of the book, I hope you will be able to say nothing special.

The Japanese characters above, Shoshin, represent the concept of Beginner’s Mind—the idea of having an open mind that is eager to explore possibilities. Most of us have heard some version of the following story (possibly from Bruce Lee’s Enter the Dragon). A professor is seeking enlightenment and goes to a wise monk for spiritual direction. The professor dominates the discussion outlining everything he has learned in his life and summarizing papers he has written. The monk asks tea? and begins to pour tea into the professor’s cup.

And continues to pour, and continues to pour, until the tea over pours the teacup, the table, and spills onto the floor. What are you doing? the professor shouts. Pouring tea the monk says and continues: Your mind is like this teacup. It is so filled with ideas that nothing else will go in. You must empty your mind before we can begin.

To me, the best programmers are empty cups, who constantly explore new technology (noSQL, node-js, whatever) with open minds. Mediocre programmers have surrounded their minds with cities of delusion—C++ is good, Java is bad, PHP is the only way to do web programming, MySQL is the only database to consider. My hope is that you will find some of the ideas in this book valuable and I ask that you keep a beginner’s mind when reading it.