Token 1.1: From Task-Specific to Task-Centric ML: A Paradigm Shift

Welcome to the beginning of our series on Foundation Model and Large Language Model Operations (FMOp/LLMOps). You might think if you've mastered MLOps, you can easily take on these new algorithms. Well, yes and no – it's complicated.

MLOps and Beyond: FMOps/LLMOps

As we’ve mentioned before, Eduardo Ordax, a leading MLOps Business Developer at AWS, places LLMOps under the broader umbrella of FMOps. According to him, while FMOps builds on MLOps, it also introduces new roles and life cycles. We're talking providers, fine-tuners, and consumers, each with their unique life cycle.

Before writing this series about FMOps, I also talked to Chip Huyen, co-founder at Claypot.ai and author of the brilliant book 'Designing Machine Learning Systems.' When thinking about foundation models, she advises to focus on one critical aspect: the use case. As ML models become increasingly efficient and accessible, companies face a shifting cost-benefit landscape. This necessitates a clear understanding of potential use cases to perform accurate and timely cost-benefit analyses.

The Shift to Task-Centric ML

This chat with Chip got me thinking: what if we are transitioning from a model-centric vs. data-centric, from task-specific to a completely different universe: use-case-centric or task-centric ML?

Some might argue that ML has always been task-focused. “Originally, says Chip Huyen, models were developed to solve tasks independently, and the promise of foundation models is that they can be used for a wide range of tasks.” My argument for task-centric ML is that before ML was limited by the amount of tasks it could be useful to. It was more task-specific. Before we had that amount of data we knew: we could build a recommendation system based on this data! With limited data and computing resources, this was often the most feasible approach. Now, we have a foundation model that we can fine-tune to our specific use case while letting it also multitask simultaneously. But it all comes from the use case (that might consist of a lot of tasks).

So, in a sense, we've moved from a world of “narrow AI,” where each model was fine-tuned for a specific problem, to a more “general AI” framework, where foundation models can serve as a base layer for various applications.

It’s a different mindset that opens up a lot of possibilities.

Yes, a foundation models once trained, might remain fixed and unaware of data generated post-training. This static nature and the training on broad, non-specialized datasets can limit model’s efficacy in domain-specific tasks. But here comes Retrieval Augmented Generation (RAG). This technique, or pattern, is a connecter between FM and your own data. It allows you to pull in external, specialized data to supplement the model's base knowledge, enhancing its performance in tasks requiring up-to-date or niche expertise. We will talk about it more in the continuation of our series.

Core Principles of Task-Centric ML

Task-centric ML focuses on using foundation models to perform wide-range of tasks efficiently and with fewer training examples. You're no longer burdened by the never-ending thirst for new data. This eliminates the 'data bottleneck,' allowing you to iterate faster and meet evolving business needs.

Thoroughly searching the internet to see if anyone had previously discussed the task-centric approach, we found that it had indeed been mentioned – but only once. The Allen Institute of AI (AI2) Incubator addressed it in their December newsletter, writing the following:

“This learning efficiency is a game changer. This enables us to take on a much larger number of tasks and to effectively deal with changing task requirements. We call this emerging paradigm task-centric ML, characterized by a large number of tasks with few training examples per task, in contrast to the currently prevailing data-centric ML where there’s a small number of tasks with many training examples per task. Foundation models may over time become the foundation (pun unintended) of a company’s ML stack, reducing the need to have fragmented and bespoke ML solutions that are expensive and difficult to maintain.”

Why It Matters

Task-centric ML isn’t just a technological shift; it's about redefining the economics of machine learning. It’s cheaper, quicker, and more adaptable, opening up new avenues for innovation and investment.

You might be wondering where FMOps in this conversation. We are getting to them, but first we need to consider advantages and concerns of using foundation models in business. That gives us a better understanding why it makes sense to branch out FMOps/LLMOps from MLOps.

The suggested list of advantages and concerns is not exhaustive. As we continue to explore this topic and learn more about evolving foundation models, these aspects are subject to change.

Advantages of Using Foundation Models in Business

• Data Efficiency: Because they are pre-trained, foundation models require less data for fine-tuning, making them highly efficient in data-scarce environments. That also leads to->

• Dynamic Adaptation: Task-centric models can be adapted on the fly to meet changing business requirements.

• Scalability and Generalization: Foundation models can adapt to a wide variety of tasks by undergoing fine-tuning, making them highly flexible and reusable across different business applications.

• Rapid Prototyping & Ease of Use: Models can be quickly fine-tuned without extensive new model designs, facilitating rapid development cycles and quicker time-to-market.

• High Performance: Foundation models often deliver state-of-the-art results in various applications, such as natural language processing, image recognition, and data analysis.

• Rich Representations & Knowledge Transfer: These models can apply learnings from one domain to another, thereby improving efficiency and potentially revealing new insights from data.

• Open-source FM and Community Support: The open-source movement around foundation models ensures a robust ecosystem of tools, tutorials, and forums, providing businesses with ample support resources. Also encouraging businesses to share their developments as well.

Concerns of Using Foundation Models in Business

• Rapid Evolution of Models: Foundation models are evolving at breakneck speed. Every new release can be a paradigm shift, requiring engineers to start from scratch to understand how to implement it effectively. This rapid innovation cycle could be seen as a double-edged sword. On the one hand, the continual advancements mean better performance and capabilities. On the other, this can create a knowledge gap even among seasoned engineers who might struggle to keep up with the ever-changing landscape. Companies must weigh this when jumping into the foundation model arena.

• Risk of Misuse & Safety: Safety is a growing concern, especially with the ease of creating misleading or harmful content using these models. It's essential to ensure that safety measures are integrated into the models to mitigate misuse. Which leads to ->

• Ethical and Societal Impact: These models can perpetuate biases, stereotypes, and unfairness present in the data, leading to ethical and PR concerns.

• Data Leakage and Security: Sensitive business information could be inadvertently encoded in the model, posing a security risk.

• Lack of Interpretability & Transparency: Foundation models are often difficult to interpret, making it challenging to understand their decision-making processes. This can be a significant issue for regulatory compliance.

• Overfitting and Brittleness: These models can be sensitive to outliers and may produce unreliable or unpredictable results when exposed to out-of-distribution data.

• Legal Ambiguity: Questions surrounding attribution, liability, and intellectual property rights can complicate the legal landscape for businesses using foundation models.

FMOps: The New Ecosystem

Considering all the above, the updated infrastructure to efficiently operationalize foundation models is needed. Here's where FMOps comes in. It's the infrastructure that enables and nurtures the task-centric ML universe.

While MLOps deals with individual, customized ML models, FMOps tackle generalized foundation models, including their tuning, updates, and security.

Some might argue that terms are just piling up. We still review MLOps structure every week, now we have to deal with FMOps. But our short investigation and discussions with many practitioners showed that there is no consensus. Instead, there is a lot of misunderstanding of the foundation models and their implementation in business.

We think that foundation models are the next step from traditional ML, it’s a mindset shift to task-centric ML, and it does require refined operational techniques.

Conclusion

We think that the shift to task-centric ML might redefine economics, scalability, and potential applications of machine learning. While the landscape is teeming with innovation, it also presents its own set of challenges and considerations, from governance to rapid model evolution.

In future episodes of this series, we'll dissect all these layers even further to a) arm practitioners with the nuanced knowledge and practical skills essential for navigating this new landscape, and b) provide business leaders with a comprehensive understanding of what to expect from foundation models, thereby enabling informed, strategic decision-making. As foundation models continue to evolve, so must our approaches and understandings. Stay tuned.

This is the first episode that is available for each of our readers and beyond. Please feel free to share it. Starting next week, the continuation of the series will be available only to our paid subscribers.

It’s an exciting times we live in. Thank you for your support and for your feedback, we truly appreciate each and one of you 🤍

Please feel free to share with your friends and colleagues