Time series forecast plays a vital role in crucial decision-making processes in various industries such as retail, finance, manufacturing and healthcare. However, compared to areas like natural language processing And image recognition, the integration of advanced artificial intelligence (AI) techniques into time series forecasting has been relatively slow. Although Fundamental AI has made significant advances in areas such as natural language processing and image recognition, its impact on time series forecasting has been limited until recently. However, today there is growing momentum in the development of fundamental models specifically suited to time series forecasting. This article aims to delve deeper into the evolving landscape of fundamental AI for time series forecasting, exploring recent advances in this area. However, before discussing these advances, let us briefly introduce time series forecasting and its applications in various industries.
Time Series Forecasting and Applications
Time series data refers to a sequence of data points collected or recorded at regular time intervals. This type of data is widespread in various fields, such as economics, weather, health, etc. Each data point in a time series is timestamped, and the sequence is often used to analyze trends, patterns, and seasonal variations over time.
Time series forecasting involves using historical data to predict future values of the series. It is an essential method in statistics and machine learning that helps make informed decisions based on past patterns. Forecasting can be as simple as projecting the same growth rate into the future or as complex as using AI models to predict future trends based on complex models and external factors.
Some applications of time series forecasting are:
- Financial markets: In finance, time series forecasting is used to predict stock prices, exchange rates, and market trends. Investors and analysts use historical data to predict future movements and make business decisions.
- Weather forecast: Weather services use time series data to predict weather conditions. By analyzing past weather data, they predict future weather trends, helping in planning and decision-making in agriculture, travel and disaster management.
- Sales and Marketing: Businesses use time series forecasting to predict future sales, demand, and consumer behavior. This helps with inventory management, setting sales targets, and developing marketing strategies.
- Energy sector: Energy companies forecast demand and supply to optimize production and distribution. Time series forecasting helps predict energy consumption patterns, enabling effective energy management and planning.
- Health care: In the healthcare industry, time series forecasting is used to predict disease outbreaks, patient admissions, and medical inventory requirements. This helps in healthcare planning, resource allocation and policy development.
Basic time series models
Fundamental AI models are extensive, pre-trained models that form the basis of various artificial intelligence applications. They are trained on large and diverse data sets, allowing them to discern patterns, connections, and structures within the data. The term “fundamental” refers to their ability to be refined or modified for tasks or areas with minimal additional training. In the context of time series forecasting, these models are constructed in the same way as major language models (LLM), using transformer architectures. Like LLMs, they are trained to predict the next or missing element in a sequence of data. However, unlike LLMs, which process text as subwords through transformation layers, fundamental time series models treat sequences of continuous time points as tokens, allowing them to sequentially process time series data.
Recently, various fundamental models have been developed for time series data. By understanding better and choosing the appropriate fundamental model, we can harness their capabilities more effectively and efficiently. In the following sections, we will explore the different fundamental models available for analyzing time series data.
- TimesFM: Developed by Google Research, TimesFM is a basic set-top box only model with 200 million parameters. The model is trained on a dataset of 100 billion real time points, encompassing both synthetic and real data from varied sources such as Google Trends and Wikipedia page views. TimesFM is capable of performing zero-shot forecasting across multiple industries, including retail, finance, manufacturing, healthcare, and natural sciences, across varying time granularities. Google intends to release TimesFM on its Google Cloud Vertex AI platform, providing its sophisticated forecasting capabilities to external customers.
- Lag-Lama: Created by researchers at the University of Montreal, the Mila-Québec AI Institute and McGill University, Lag-Llama is a fundamental model designed for forecasting univariate probabilistic time series. Built on top of Llama, the model uses a decoder-only transformer architecture that uses lead times and temporal resolutions of varying sizes for prediction. The model is trained on various time series datasets from multiple sources from six different groups including energy, transportation, economy, nature, air quality, and cloud operations. The model is easily accessible via the Huggingface Bookshelf.
- Moirai: Developed by Salesforce AI Research, Moirai is a fundamental time series model designed for world forecasting. Moirai is trained on the Large-scale Open Time Series Archive (LOTSA) dataset, which contains 27 billion observations from nine distinct domains, making it the largest collection of open time series datasets . This diverse dataset allows Moirai to learn from a wide range of time series data, enabling it to handle different forecasting tasks. Moirai uses multiple patch size projection layers to capture temporal patterns across different frequencies. An important aspect of Moirai is to use an attention-to-any-variable mechanism, allowing predictions on any number of variables. The code, model weights, and data associated with Moirai are available in the GitHub repository called “units2ts“
- Chronos: Developed by Amazon, Chronos is a collection of pre-trained probabilistic models for time series forecasting. Built on the T5 Transformer architecture, the models use a vocabulary of 4,096 tokens and have variable parameters, ranging from 8 million to 710 million. Chronos is pre-trained on a wide range of public and synthetic data generated from Gaussian processes. Chronos differs from TimesFM in that it is an encoder-decoder model, which allows the extraction of encoder embeddings from time series data. Chronos can be easily integrated into a Python environment and accessed via its API.
- Moment: Developed collaboratively by Carnegie Mellon University and the University of Pennsylvania, Moment is a family of open source fundamental time series models. It uses variants of T5 architectures, including small, basic and large versions, with the base model incorporating around 125 million parameters. The model undergoes pre-training on the vast “time series stack,” a diverse collection of public time series data spanning various domains. Unlike many other fundamental models, MOMENT is pre-trained on a wide range of tasks, improving its effectiveness in applications such as forecasting, classification, anomaly detection, and imputation. The Complete Python Repository and Jupyter Notebook coded are publicly available to use the model.
The essential
Time series forecasting is a crucial tool in various fields, from finance to healthcare, enabling informed decision-making based on historical patterns. Advanced fundamental models such as TimesFM, Chronos, Moment, Lag-Llama and Moirai offer sophisticated features, leveraging transformer architectures and diverse training datasets for accurate forecasting and analysis. These models provide insight into the future of time series analysis, providing businesses and researchers with powerful tools to effectively navigate complex data landscapes.
