How to build a GenAI platform with cloud-based machine learning ops

Since the emergence of popular generative AI (GenAI) platforms such as ChatGPT, Claude, and LLama2, business owners who understand the potential of AI have transitioned from merely investigating its capabilities to harnessing it at scale. As a result, many companies now develop in-house AI and machine learning models for customer service, data analysis, and product ideation.

With this surge in AI integration, creating best practices becomes very important, and that’s where machine learning operations (MLOps) come in. Think of it as the behind-the-scenes machinery that makes GenAI development as smooth and efficient as possible.

Let’s take a look at what MLOps does to help companies integrate generative AI:

MLOps function as a set of rules and tools that help simplify the development process and reduce the required resources. It’s a core function of machine learning engineering and focuses on streamlining the process of taking machine learning models to production and then maintaining and monitoring them. MLOps also run as a collaborative function, requiring data scientists, DevOps engineers, and IT to work together to make processes more efficient.

Here’s why that’s important: GenAI requires massive amounts of quality data, but also development, deployment, and monitoring capabilities at a level that was unheard of before. MLOps particularly benefits companies that don’t have unlimited computing resources and data to develop and fine-tune machine learning models. MLOps starts with pre-packaged models called foundation models.

Examples of some of these foundation models are the GPT 3.5 and GPT-4 models, upon which ChatGPT has been based; BERT, which serves as the basis for Google’s Bard; and Llama 2 Large Language Model (LLM) from Meta. Once a company has access to a foundation model, the organization can use cloud services, like Amazon Web Services (AWS) to offer the data processing capabilities and computing power that the business may not have available in-house.

Companies can train these foundation models on highly-specific proprietary data so that the resulting generative AI tools are exactly tailored to the organization’s needs while requiring minimal resources. And therein lies the value of MLOps when it comes to generative AI: producing a completely customized software that’s ideal for each company in a streamlined, simplified process.

Build GenAI the right way

Now that we understand MLOps let’s talk about the practicality of how it works in an organization. Clarify the company’s goals so the team can choose the ideal foundation model. Here’s where the team has to decide whether it should choose a proprietary foundation model like ChatGPT or if it makes more sense to use an open-source model like Llama 2.

Teams benefit from an open-source model because they can fine-tune it based on their own proprietary data without disclosing confidential info to language model operators or external companies. A provider can change its service agreements or suffer a data leak at any time. If that happens, the company’s in-house GenAI tool – for example, a customer service chatbot that consistently handles customer data – can become more of a liability than an advantage. While some tools, including ChatGPT, have enterprise versions that offer companies more control over their data security, it’s expensive. Not to mention, these large models may have more than 150 billion parameters, meaning the computing and energy costs are far above what many companies actually need.

Once the team selects a foundation model, prepare and clean the data through a cloud service provider like Azure or AWS. The team will also need computing hardware that contains tensor processing units (TPUs) and graphic processing units (GPUs). TPUs are integrated circuits that Google developed to accelerate machine learning development.

Next, the developer team will build MLOps frameworks to fine-tune the model. This involves training, testing, and validating code. Typically, MLOps also includes automating as much as possible in terms of debugging, scaling, and deployment. Afterward, deploy the software for both internal and – if the company plans to sell it commercially or if it’s a customer-facing platform – external access through mobile apps or web pages.

Building a generative AI platform through MLOps is an ongoing process that requires monitoring and retraining the model. When the model doesn’t adapt to external factors like changes in input data or user behavior correctly, it results in what we call model drift, which can result in biases and other inaccuracies, and potential liabilities. To avoid that problem, constantly retrain and monitor the model as the team scales.

Generative AI has become increasingly accessible for businesses. If there's quality data, access to data processing software such as AWS or Azure, and enough quality hardware, the team can now develop a fine-tuned GenAI model that everyone in the company can benefit from.

Priyank Kapadia, product and technology partner, Accolite