Turing Post
Posts
Token 1.9: Open- vs Closed-Source AI Models: Which is the Better Choice for Your Business?

Token 1.9: Open- vs Closed-Source AI Models: Which is the Better Choice for Your Business?

A Deep Dive into the Pros and Cons of Each Approach with a Few Useful Tips

Valeriia Kuka
November 15, 2023

Introduction

In Token 1.2, we explored how to choose between traditional machine learning models and when to apply Foundation Models (FMs)/Large Language Models (LLMs). You've determined that for your specific task, an FM/LLM is the right fit. It may seem like the choice is made, but there's more to it – now you need to decide: will you opt for an open-source model or the API of a closed model?

This pivotal decision is not just about selecting a tool; it's about choosing a path that aligns with your strategic objectives and operational philosophy. In this Token, we analyze this critical choice and its implications for your AI strategy.

In today’s Token:

Understanding Open- and Closed-source Models
Factors to Consider
Key Differences of Open- and Closed-source Models, including:
- Customization and control
- Cost implications:
  - typical costs in closed-source models
  - hidden costs in open-source models
- Scalability and performance
- Support and community
- Data privacy and security
- Long-term viability
Conclusion

Understanding Open- and Closed-source Models (Let’s Define Them!)

While at first glance, open- and closed-source models might seem to differ only in their source code accessibility, a deeper dive reveals a more complex landscape.

Closed-source models are straightforward in one aspect: their source code, which details the model's construction and training, remains private. To use these models, you need to pay the owning company, referred to as the provider. However, even after the payment, the source code of the model will not be available to you. Instead, it allows the use of the model's capabilities within boundaries set by the provider. About the specifics that – in the following section.

Contrary to initial impressions, open-source models are not that simple. The term 'open-source' indeed implies publicly accessible code, but this is just the beginning. The intricacies lie in several key areas:

Diverse open-source licenses: Open-source models come with a variety of licenses, each with its own set of regulations. Some may restrict commercial use, confining the model's application to research purposes. For example: there are still debates if Llama 2 is open-sourced. Meta AI says it is, but Open Source Initiative (OSI) says, it isn’t.
The question of model weights: Open-source models universally share their source code, but there's variability in whether they include model weights*. The absence of these weights, critical for the model's operation, means users would need to undertake the resource-intensive task of training the model themselves – a feat often only feasible for major tech companies. The ML community debates: is a model truly open-source if it doesn't include its weights?
Open-source datasets: This aspect extends the philosophy of open-source models and may be perceived as a bonus addition to the open-source model. Unlike their closed-source counterparts, which typically don't reveal their training data, open-source models might provide an added advantage by sharing the datasets used for training. These datasets are crucial not only for training and fine-tuning foundation models but also for fostering open research and innovation.

*Model weights are numerical values assigned to neuron connections in a neural network, indicating the strength and direction of influence between neurons. Adjusted during training, they minimize the output error by influencing the signal as it passes through the network.

Let’s review the main factors you need to take into account while choosing the model and choosing between open- and closed-source alternatives.

Factors to Consider

Choosing the right foundation model, and deciding between open- and closed-source options, involves a multifaceted analysis. What to consider:

Project requirements and objectives: What are the specific tasks and goals of your project? What do you need the model to achieve?
Cost implications: What are the visible and hidden costs of each option, including initial expenses, maintenance, and possible future costs?
Data privacy and security: How does each model handle sensitive data? Is it secure for projects involving confidential or personal information?
Customization and control: What level of adaptability do you need? Do you you require extensive customization capabilities, like fine-tuning and modifying model parameters?
Support and community: The level of available support and the robustness of the community can be vital, how does this align with your team's expertise and resources?
Scalability and performance: What is the model's ability to handle growing volumes of data and increasing task complexity, both currently and in the future?
Legal and ethical considerations: What ethical implications, such as potential biases in the model, and legal aspects, including data usage rights and commercial application restrictions, should be considered?
Availability of skills and resources: Does your team have the necessary skills to implement and maintain an open-source model, or are a closed-source model's ready-made solutions more suitable?
Long-term viability: How sustainable is the model in terms of ongoing support and development? This ensures the model's long-term usefulness.
Integration with existing systems: How well does the model integrate with your current infrastructure and workflows, especially in complex or established operational environments?

Balancing these factors against your project’s unique requirements will guide you in making an informed decision between open- and closed-source foundation models.

In the next section, we'll detail the main differences between open and closed-source models to help you choose the right one for your project/business. It’s filled with useful information! Upgrade to →

Thank you! Turing Post is made possible by readers like you. It’s invaluable.

Tip: An important aspect of costs: model size*

As highlighted in "A Survey of Large Language Models," the size of an LLM can vary immensely, ranging from several billion to, in some cases, several trillion parameters. While the initial trend favored building larger models, research and industry leaders are now pivoting towards smaller, more efficient models. The same can be said about other types of foundation models. This shift has spurred new research in model compression for LLMs and Vision-Transformers (ViTs).

*Model size refers to the total number of parameters in a model. In the context of neural networks, parameters include both weights and biases*. This number is often used as a measure of the model's complexity and capacity for learning.

*Biases in neural networks are parameters added to the sum of weighted inputs in each neuron, allowing it to adjust its output independently of its inputs, thereby aiding in accurately modeling complex data patterns.

The size of a model directly affects its processing power requirements, inference speed, and memory usage. This is especially crucial for open-source models, where computational resources are user-supplied, in contrast to closed-source models where computation is generally handled by the provider.

Tip: Fine-tuning

Our discussions with practitioners about implementing LLMs highlighted the need to decide between using existing models or fine-tuning them. While using an existing model can save time and resources, fine-tuning open-source models demands significant computational resources and expertise. The same is true for vision models and other types of FMs.

Some tips for fine-tuning:

Employ parameter-efficient techniques like Low-Rank Adaptation of Large Language Models (LoRA) for LLMs and LCM-LoRA for Stable-Diffusion models
Experiment with LLM configurations (e.g., max seq length, temperature) and understand their impacts
Consider retrieval-augmented generation (RAG) as an alternative to fine-tuning.

Previously in the FM/LLM series:

Please give us feedback

Thank you for reading, please feel free to share with your friends and colleagues. In the next couple of weeks, we are announcing our referral program 🤍

Join the conversation

or to participate.