Fewer than one in four companies—only 22%—consider their AI deployment as strategic. That’s according to the respondents of our State of Data Science and AI survey. An unclear or absent strategy can limit how productive artificial intelligence (AI) initiatives become.
However, we’re seeing progress toward those strategic approaches. In particular, there’s a varied mix of AI implementation and how organizations can measure its value. And that’s encouraging as companies look to build on their AI foundations.
For the eighth consecutive year, we conducted our State of Data Science and AI survey to dive further into how data scientists, developers, engineers, and other roles are innovating with AI and open source, and where roadblocks might remain.
ROI Is Taking On Many Forms
Productivity improvements (58%) and cost savings (47%) are the most common ways companies are measuring ROI. Fewer (25%) are looking at go/no-go decisions, error reduction, or risk mitigation.
Other areas that may be worth exploring include reducing time-to-value, selling more of a product, or broadening the user base, and creating a happier, more engaged workforce. There’s no one-size-fits-all solution to measuring the value of AI. It’s a matter of looking at what makes it valuable to your company, your customers, and your people.
About 26% of respondents said difficulty demonstrating ROI is a top concern about AI risk. And 13.5% reported they don’t measure ROI at all, or are unsure how their company tracks it.
Is that necessarily a bad thing? It’s all too easy to get caught up in metrics. Yes, it’s important to measure your AI usage and the outcomes tools are producing. But when hitting a certain number or benchmark becomes the main goal, that’s where we’ve lost sight of what really matters. The key is understanding where AI improves your company. Where does it make you better? How does it solve real business challenges, and not just fabricated ones that look good in a report?
Getting a better grasp of those results lays the foundation for AI initiatives. And it sets up companies to scale at a more rapid pace.
How Can AI Scale Faster?
Ignoring structural issues can stall AI projects before they get off the ground. We asked respondents about challenges when they’re moving data science or AI models to a production environment. Here are the most common answers:
45%: Data quality and pipeline consistency issues
40%: Security, privacy, and regulatory compliance challenges
39%: Scaling inference and managing compute costs
33%: Cross-team collaboration and communication gaps
For as long as we’ve been doing the State of Data Science survey, we’ve seen these issues persist in the data science and machine learning world. It’s interesting to note that they continue to affect projects in the relatively new field of enterprise AI. Several of these challenges seem to suggest that organizations may not be fully aligned when they’re taking on AI initiatives.
For instance, a lack of alignment between tech teams leads to challenges for deployment, scaling, and cross-team collaboration. The classic one that arises in Python is choice of tooling and versions for various packages between the data/ML/AI teams, and the various ops teams that need to operationalize their work.
The power of Python’s open source ecosystem comes from its great diversity of software libraries, but this is a double-edged sword. Without a priori alignment on choices for deployment platform and technologies, teams will invariably waste a lot of time reconciling and re-coding work after the fact. Of course, the entire ops team doesn’t need to be experts on the inner workings of an AI model, but they should be sensitive to the particular hardware and resource needs of these new kinds of data-intensive workloads.
The rapidly-evolving field of AI offers additional organizational challenges. One part of a business may be receiving extremely strong mandates to deliver AI features and show ROI for AI tech investments, while a different part is working on compliance and security. Unless there is clarity and agreement at the highest executive levels about the trade-offs and implications, the senior leadership across product development, legal, and security may waste tremendous time going in circles.
Deeply aligning on goals, and being realistic about trade-offs, is a crucial requirement for AI success. Both employees and customers need to have confidence and trust in the decision-making process. More than half of our respondents (53.3%) said they have no AI governance policies or frameworks in place, or those policies are still being developed. A start is better than nothing at all, and putting some guardrails in place keeps people on the right path, even if they’re experimenting along the way—which is absolutely encouraged.
Open Source Will Guide the Path Ahead
The good news is that in the last couple of years, the explosion in open models has given birth to a thriving open innovation ecosystem. This has allowed researchers to develop evaluations and tools that help practitioners understand what models (both open and closed) are really capable of. In the meantime, the Python community has been making strides in classic challenges such as scalability and ease of deployment.
Open source software has historically enabled collaborations which have produced tremendous innovation, and made those innovations broadly available to everyone. With AI it should be no different, and I’m encouraged by the growing enthusiasm around open source and AI. About three in four respondents (76%) say their organization has more priority on open source this year compared to their previous 12 months. And 92% of respondents use open-source AI tools and models already.
When we’re intentional about learning how to use AI and how to secure, deploy, and monitor it, that’s when we’re at our best. Anaconda is here to empower faster time-to-value and innovation, and we’re looking forward to seeing more creative implementations of open-source AI tools.
Download the full 8th Annual Data Science and AI Report: How Companies Are Moving Ahead—Or Not—in the AI Race.