One of the questions that I get repeatedly nowadays from a lot of beginners in the data science space is around the longevity potential of this profession. The questions invariably drift towards platforms like “Auto ML” and how they would make the jobs of data scientists irrelevant and redundant.
I was watching a talk by Cassie Kozyrkov – Chief Data Scientist @Google on Youtube the other day and she put some arguments very succinctly which I want to discuss here. In any discipline as in data science, doing things well is hard – but then all “hard” is not the same.
Some of the hard is due to
a) The stuff being intellectually hard and requiring a lot of thought and research
b) Some other stuff is hard just because of bad tooling and lack of interoperability between various systems etc.
Now the former is genuinely interesting and lies at the very core of our discipline. The latter is just because of bad engineering and honestly not very interesting – It is just a chore that we had to put up with all these years.
A lot of fun and value in our discipline comes from our ability to perform some of the so-called softer functions, things like deciding whether ML is actually the right solution here, actually interpreting the results, and ensuring that the right decisions are getting automated. This is the stuff that cannot be automated and will always depend on “human intelligence”.
In fact, we should stop calling them “soft skills” as I believe that these are the skills that will become more and more important in the coming months and years and as long as we are building muscle here, there is absolutely nothing to worry in terms of our jobs getting automated.
A lot of the tooling part that is getting automated should actually make us happy. It is the first time (at least that I am aware of) that the tooling community is considering us (data scientists and ml engineers) important enough to create tools for making our chores less painful and our working lives that much better.