Can You Live Without a Data Scientist?

There are some great articles about big data and analytics in the current issue of HBR, and I am happy to have co-authored one of them with DJ Patil, one of the world's first practicing data scientists. In fact, data scientists are what our article is about. I would argue that they are the most important resource for capitalizing on big data. While there is a lot of Hadoopalooza in the technology press about the tools for managing big data, and they are wonderful, it's also true that they are a) widely available, and b) mostly free. Neither can be said of data scientists. The other necessary resource, massive quantities of data, can also be found on every virtual corner these days. If your customers have internet access, for example, you've got big data.

Simply put, you can't do much with big data without data scientists. They are the magicians who transform an inchoate mass of bits into a fit subject for analysis. God may have been the first to produce order out of chaos, but data scientists do it too, admittedly on a smaller scale. They can suck data out of a server log, a telecom billing file, or the alternator on a locomotive, and figure out what the heck is going on with it. They create new products and services for customers. They can also interface with carbon-based lifeforms — senior executives, product managers, CTOs, and CIOs. You need them.

But here's the burning question: do you need them now and on your payroll? Data scientists are definitely in short supply at the moment; ask any company that's trying to hire them. And trained data scientists seem to be aware of their scarcity, judging from the compensation packages they are pulling down. One admitted to me in an interview that, "We're a pain in the ass. We're constantly telling managers that their data isn't good enough or interesting enough. We're never satisfied with how the company is being run. We ask for big options packages, and then we leave after six months."

I suspect that over the next couple of years, it will become a lot easier to hire data scientists. A wide variety of universities — including Harvard, Berkeley, Stanford, Columbia, NC State, and others — are offering data science courses and starting degree programs. (Berkeley is reputed to be starting three different programs, which I'm not sure represents progress.) In the meantime, there are a number of consulting organizations that can offer data science services. The big analytics providers, including Accenture, Deloitte, and IBM, are all gearing up on the topic, and there are a variety of boutique firms that have formed. The large offshore providers, of which Mu Sigma is the largest, are also getting in on the big data game.

If you want to stake out a leadership position in big data for your company — and I would encourage you to do so — you don't really have an alternative to joining the data scientist recruiting scrum. But if you want to dip a toe into the water, you might think about starting with a consultant. In either case, in the meantime, start sending people to school on the topic.


More >>

This entry was posted in Leadership. Bookmark the permalink.

Comments are closed.