The graphic above is rather…involved…but it’s quite good / useful. It’s actually the superset of all modern technology architecture diagrams that exist today, the superset of all possible “how we built our data stack” posts. The folks at A16Z have been working on this post for ~6 months at this point and the hard work has paid off.
What I particularly find valuable is the separation of the total stack into separate reference architectures for BI and AI/ML, while recognizing that it’s also possible to combine those two into a single cohesive platform. Having such a clear depiction of how these pieces fit together is incredibly useful.
I find this type of article annoying and maybe you do too. Someone did a survey of executives at big companies and found that most of them aren’t seeing much value from AI yet. Saved you a click.
So why link to it? I think there’s specifically a problem with this type of survey, and it points to something bigger. The types of companies who benefit most from AI are often technology-first companies. FAANG, Spotify, Stitch Fix, Airbnb, etc. While certainly a part of this is about talent—and those companies have a lions’ share of it—I think it’s more importantly about business model. I don’t believe you just take a fundamentally industrial-age company and just slap AI on top of it and—voila!—profit. I think the companies benefiting from it most have architected how value is provided from the beginning to the end of their operations with AI at the heart. Netflix’s entire content strategy (and thus corporate strategy) is centered around their recommendation algorithm. Etc…you get the point.
If AI is to have the huge impact that I believe it will, most of that impact will not come through “transformation” efforts at existing large companies. It will come through the creation of brand new companies built from the ground up with AI at their core. And so, asking a bunch of large existing businesses whether AI is having a positive impact has an inherent sampling bias—the impact it’s going to have on many/most of them is to threaten their market share. If you want to get a more interesting answer, ask Gong.io how AI is impacting its business.
IMHO, one of the questions that a data scientist should be asking before taking a job is: “Is AI at the heart of this company’s value to its customer or am I just being bolted on?”
Data scientist, data engineer, and now AI specialist are all in the top 15. The AI Specialist role in particular is experiencing pretty stunning growth—74% annual growth over the past 4 years. Not revolutionary, but a quick read.
A few weeks ago, I got curious about how organizations can intentionally retrain analysts for data science roles. This post is the result of a few conversations with data leaders who have been there, done that, and my own research on the topic.
Really interesting topic. One of the most interesting things in data right now is (IMO) figuring out the right career path for a data analyst. One of dbt’s main goals is to create another career path—the analytics engineer—that can be a source of long-term satisfaction and high salaries. But I 100% agree that data science is a legit pathway for someone with analytics skills who begins to ask where their career is going.
If you run a team and want to enable 1:N analysts to go on this path, the right answer isn’t just “stick them in some Coursera classes and start having them work in Python.” Read this post first.
OK, you almost definitely have heard about this. It’s easy to criticize the folks at PHE for building a terrible system with Excel (and the deprecated-in-the-90’s .xls file format) at its core. But if you believe this “expert”, you’re just as detached from reality as he is:
“Excel was always meant for people mucking around with a bunch of data for their small company to see what it looked like. And then when you need to do something more serious, you build something bespoke that works - there’s dozens of other things you could do. But you wouldn’t use XLS. Nobody would start with that.”
Reality check—Excel is the most-used data tool in existence. If you’re not taking this into account in your thinking, you should be. How are you helping to enable Excel users at your org?