Using large language models (LLMs) in time-series forecasting isn't entirely new but, for some reason, it's been picking up steam since early 2024. We explored this topic in a recent interview with TimeGPT creators.
In this article, we list some of the models and the latest surveys covering this topic extensively.
Models
TimeGPT (Nixtla): A pioneering foundation model for time series forecasting, offering accurate zero-shot predictions across varied datasets. TimeGPT uses a transformer-based architecture trained on the largest publicly available time series collection. It outperforms traditional methods in efficiency, accuracy, and simplicity without retraining. This model democratizes precise forecasting and cuts computational complexity, marking a significant advancement in time series analysis. Paper, GitHub.
TimesFM (Google Research): A foundation model optimized for time-series forecasting, featuring a decoder-only attention mechanism trained on both real-world and synthetic data. It offers high accuracy and adaptability across various forecasting contexts without needing retraining. Paper, GitHub
Moirai (Salesforce AI Research, Singapore Management University): Advances the Transformer architecture to handle multivariate series and cross-frequency learning, trained on over 27 billion observations. It shows superior zero-shot forecasting ability. Paper, GitHub (with code, data, and model weights)
Lag-Llama (Morgan Stanley, ServiceNow Research, others): Focuses on univariate probabilistic forecasting with a decoder-only transformer architecture. Notable for its exceptional generalization in zero-shot and few-shot scenarios. Paper, GitHub
CARD (Alibaba Group): Improves forecasting by using a channel-aligned attention structure and a robust loss function, showing substantial accuracy improvements over existing methods. Paper, GitHub
Chronos (AWS AI Labs): Adapts language model techniques for time series forecasting, demonstrating strong generalization and simplifying forecasting processes. Paper, GitHub
Pathformer (East China Normal University, Alibaba Group): A multi-scale Transformer that dynamically adjusts to input data, achieving excellent performance across various datasets. Paper, GitHub
GAFormer (Georgia Institute of Technology): Introduces group embeddings to capture complex dynamics, enhancing performance in time series tasks. Paper
TMDM (Xidian University, University of Texas at Austin): Combines transformers with diffusion models to improve distribution forecasting and uncertainty estimation in multivariate series. Paper
iTransformer (Tsinghua University, Ant Group): Inverts traditional Transformer architecture to improve forecasting accuracy and generalization across different scenarios. Paper, GitHub
HTV-Trans (Xidian University): Addresses non-stationarity in multivariate series with a hierarchical probabilistic model, enhancing forecasting accuracy. Paper
TIME-LLM (Monash University, others): Repurposes LLMs for forecasting by using text prototypes and Prompt-as-Prefix techniques, surpassing traditional models in various learning scenarios. Paper, GitHub
MOMENT (Carnegie Mellon University, University of Pennsylvania): A family of models pre-trained on a diverse dataset, excelling in tasks like forecasting and anomaly detection with minimal fine-tuning. Paper, GitHub
Surveys
Large Language Models for Time Series: A Survey. Paper, GitHub
Foundation Models for Time Series Analysis: A Tutorial and Survey. Paper
Time Series Forecasting with LLMs: Understanding and Enhancing Model Capabilities. Paper