• Turing Post
  • Posts
  • A List of Time Series Foundation Models

A List of Time Series Foundation Models

The latest developments: TimesFM, Moirai, Chronos and more

Using large language models (LLMs) in time-series forecasting isn't entirely new but, for some reason, it's been picking up steam since early 2024. We explored this topic in a recent interview with TimeGPT creators.

In this article, we list some of the models and the latest surveys covering this topic extensively.

Models

  • TimeGPT (Nixtla): A pioneering foundation model for time series forecasting, offering accurate zero-shot predictions across varied datasets. TimeGPT uses a transformer-based architecture trained on the largest publicly available time series collection. It outperforms traditional methods in efficiency, accuracy, and simplicity without retraining. This model democratizes precise forecasting and cuts computational complexity, marking a significant advancement in time series analysis. Paper, GitHub.

  • TimesFM (Google Research): A foundation model optimized for time-series forecasting, featuring a decoder-only attention mechanism trained on both real-world and synthetic data. It offers high accuracy and adaptability across various forecasting contexts without needing retraining. Paper, GitHub

  • Moirai (Salesforce AI Research, Singapore Management University): Advances the Transformer architecture to handle multivariate series and cross-frequency learning, trained on over 27 billion observations. It shows superior zero-shot forecasting ability. Paper, GitHub (with code, data, and model weights)

  • Lag-Llama (Morgan Stanley, ServiceNow Research, others): Focuses on univariate probabilistic forecasting with a decoder-only transformer architecture. Notable for its exceptional generalization in zero-shot and few-shot scenarios. Paper, GitHub

  • CARD (Alibaba Group): Improves forecasting by using a channel-aligned attention structure and a robust loss function, showing substantial accuracy improvements over existing methods. Paper, GitHub

  • Chronos (AWS AI Labs): Adapts language model techniques for time series forecasting, demonstrating strong generalization and simplifying forecasting processes. Paper, GitHub

  • Pathformer (East China Normal University, Alibaba Group): A multi-scale Transformer that dynamically adjusts to input data, achieving excellent performance across various datasets. Paper, GitHub

  • GAFormer (Georgia Institute of Technology): Introduces group embeddings to capture complex dynamics, enhancing performance in time series tasks. Paper

  • TMDM (Xidian University, University of Texas at Austin): Combines transformers with diffusion models to improve distribution forecasting and uncertainty estimation in multivariate series. Paper

  • iTransformer (Tsinghua University, Ant Group): Inverts traditional Transformer architecture to improve forecasting accuracy and generalization across different scenarios. Paper, GitHub

  • HTV-Trans (Xidian University): Addresses non-stationarity in multivariate series with a hierarchical probabilistic model, enhancing forecasting accuracy. Paper

  • TIME-LLM (Monash University, others): Repurposes LLMs for forecasting by using text prototypes and Prompt-as-Prefix techniques, surpassing traditional models in various learning scenarios. Paper, GitHub

  • MOMENT (Carnegie Mellon University, University of Pennsylvania): A family of models pre-trained on a diverse dataset, excelling in tasks like forecasting and anomaly detection with minimal fine-tuning. Paper, GitHub

Surveys

  • Transformers in Time Series: A Survey. Paper, GitHub

  • Large Language Models for Time Series: A Survey. Paper, GitHub

  • Foundation Models for Time Series Analysis: A Tutorial and Survey. Paper

  • Time Series Forecasting with LLMs: Understanding and Enhancing Model Capabilities. Paper

Thanks for reading!

Subscribe to keep reading

This content is free, but you must be subscribed to Turing Post to continue reading.

Already a subscriber?Sign In.Not now

Join the conversation

or to participate.