Selecting an Enterprise Platform for Python and Open Source: A Checklist for Buyers

Introduction

Marc Andreessen famously opened an August 2011 blog article with this provocative sentence: “Software is eating the world.” His prediction was that software development would disrupt traditional industries. Indeed, companies like Airbnb, Netflix, and Uber emerged as just a few of many winners in the “on-demand” economy that disrupted industries like travel, entertainment, and shopping in significant and lasting ways.

About a year later, in October 2012, Harvard Business Review reported that data scientist was the “sexiest job of the 21st century,” promising professionals who could “coax treasure out of unstructured data.” And the race to structured data began, with organizations taking a closer look at their messy data and finding ways to make it more consumable by machines.

Fast-forward seven years to October 2019, and McKinsey Global Institute offered exciting and cautionary words about the “coming of AI spring.” Their research showed hundreds of business cases that, combined, had the potential to create between $3.5 trillion and $5.8 trillion in value annually. As organizations applied artificial intelligence, they found it could yield outsized business value. Data science capabilities emerged as a prerequisite for high-performing AI, so organizations increased their investments in technologies, data science teams, and techniques like machine learning and deep learning.

In August 2022, Stable Diffusion rocked the visual arts world with its text-to-image model built with Python and deep learning that could generate detailed images based on text prompts. It stoked the world’s fascination with AI and unleashed its next wave: generative AI.

Finally, three months later, in November 2022, OpenAI released another generative model: ChatGPT, a large language model (LLM) that uses training on OpenAI’s GPT-3 and GPT-4 LLMs to generate text based on prompts from the user. In a short time, LLMs have taken many industries by storm, with new products and capabilities that make it possible for programmers to write and debug code alongside a machine and for writers to work with AI to produce content, just to name a couple of a growing number of use cases.

Finding an Enterprise Platform

With all of this progress, it would seem that every organization should somehow incorporate these new technologies into their research, products, and operations. However, applying these techniques to their fullest potential requires a set of fully featured tools, clean and structured data, expert teams, and the power of open-source software, backed by an engaged community of makers and maintainers.

Leveraging the power of open-source software across an enterprise organization requires capabilities for building and deploying secure Python solutions. There are a burgeoning number of options available for stitching together tools that can enable teams to collaborate and build powerful applications with data science and machine learning. But bolting together tools to deploy predictive models into production is not the best way to approach creating a platform that your organization can rely on to deliver excellent outcomes. And building your own can be expensive and complex, because you’ll need to maintain the platform you create.

Finding an enterprise platform that can provide the open-source packages you need, the managed environments that allow you to reproduce and scale models in production, and the security tools to protect your organization from bad code and bad actors can be a tough challenge. That’s what this guide is all about—exploring what to consider when you are selecting an enterprise platform to use with Python and open-source software to achieve your organization’s development goals. 

What makes a platform?

One popular description of a platform comes from Microsoft CEO Bill Gates, as paraphrased by Charmath Palapithiya: “A platform is when the economic value of everybody that uses it, exceeds the value of the company that creates it.” As you evaluate platforms, consider these basic characteristics that will help you leverage the innovation of the community as your teams develop and deploy applications using open source with Python:

• Number of individual users: The more users, the more opportunities there are to discover new techniques shared by others, identify and address security risks faster, and benefit from a rich community of software makers and maintainers.
• Number of enterprise users: The more enterprise users, the more the platform has been tested at scale. Number of users may be expressed as a percentage of a total group of organizations or businesses, such as the Fortune 500.
• Years of experience: The longer an organization has been working to develop their platform, the more expertise their team is likely to have across tools, techniques, and use cases.

Cross-industry customers: The more industries in which a platform has been applied, the more integrations, use cases, and data types the platform and supporting team have likely encountered.

Python, Open Source, Data Science, and the Enterprise

Data science has revolutionized the way businesses operate. Today, it seems that everyone is working with data in some capacity, whether it’s analyzing customer behavior, building predictive models, or creating generative models. As the demand for data-driven insights continues to grow, Python has emerged as the go-to language for data science work.

In fact, Python has long been the gold standard for data science work, thanks in large part to its simplicity and versatility. Unlike other languages, Python allows users to easily manipulate and analyze data, making it an ideal choice for everything from data visualization to machine learning. Additionally, the availability of numerous open-source libraries and frameworks ensures that Python remains a popular choice for data scientists.

Open-source software provides developers access to a global network of contributors who are constantly updating and improving code, making it possible for companies to create applications much faster and more efficiently than ever before. The vast majority (96%) of code bases contain open-source software, according to the Synopys 2023 OSS Risk Analysis Report.

The widespread application of OSS makes sense; open source not only helps companies save on licensing costs, but also allows them to leverage the collective knowledge of the open-source community to create customized solutions to meet their specific business needs. As a result, open source has become an essential strategic tool for organizations looking to stay ahead in the fast-paced world of technology.

However, managing Python development in enterprise organizations has become more complex and difficult over the past few years. This is due in part to the rapid pace of development within the Python community, which has led to the release of new tools and technologies on a regular basis. Some of these tools are proprietary, and some are open source. While this is ultimately a positive development for teams that work with data, it can make it challenging to keep up with the newest techniques and best practices.

Despite these challenges, Python remains one of the most powerful and versatile tools available for data science work. As the industry continues to evolve, Python will remain a critical component of any successful data science team’s toolbox.