Update README.md
Browse files
README.md
CHANGED
@@ -10,7 +10,7 @@ tags:
|
|
10 |
- time-series
|
11 |
---
|
12 |
|
13 |
-
# TinyTimeMixer (TTM) Model Card
|
14 |
|
15 |
<p align="center" width="100%">
|
16 |
<img src="ttm_image.webp" width="600">
|
@@ -21,7 +21,7 @@ TinyTimeMixers (TTMs) are compact pre-trained models for Multivariate Time-Serie
|
|
21 |
|
22 |
TTM outperforms several popular benchmarks demanding billions of parameters in zero-shot and few-shot forecasting. TTMs are lightweight
|
23 |
forecasters, pre-trained on publicly available time series data with various augmentations. TTM provides state-of-the-art zero-shot forecasts and can easily be
|
24 |
-
fine-tuned for multi-variate forecasts with just 5% of the training data to be competitive.
|
25 |
|
26 |
|
27 |
**The current open-source version supports point forecasting use-cases specifically ranging from minutely to hourly resolutions
|
@@ -36,49 +36,33 @@ fine-tuned for multi-variate forecasts with just 5% of the training data to be c
|
|
36 |
|
37 |
- **512-96:** Given the last 512 time-points (i.e. context length), this model can forecast up to next 96 time-points (i.e. forecast length)
|
38 |
in future. This model is targeted towards a forecasting setting of context length 512 and forecast length 96 and
|
39 |
-
recommended for hourly and minutely resolutions (Ex. 10 min, 15 min, 1 hour, etc). (branch name: main)
|
40 |
|
41 |
- **1024-96:** Given the last 1024 time-points (i.e. context length), this model can forecast up to next 96 time-points (i.e. forecast length)
|
42 |
in future. This model is targeted towards a long forecasting setting of context length 1024 and forecast length 96 and
|
43 |
-
recommended for hourly and minutely resolutions (Ex. 10 min, 15 min, 1 hour, etc). (branch name: 1024-96-v1)
|
44 |
|
45 |
- **New Releases (trained on larger pretraining datasets, released on October 2024)**:
|
46 |
|
47 |
- **512-96-r2**: Given the last 512 time-points (i.e. context length), this model can forecast up to next 96 time-points (i.e. forecast length)
|
48 |
in future. This model is pre-trained with a larger pretraining dataset for improved accuracy. Recommended for hourly and minutely
|
49 |
-
resolutions (Ex. 10 min, 15 min, 1 hour, etc). (branch name: 512-96-r2)
|
50 |
|
51 |
-
- **1024-96-r2**: Given the last 1024 time-points (i.e. context length), this model can forecast up to next 96 time-points (i.e. forecast length)
|
52 |
-
in future. This model is pre-trained with a larger pretraining dataset for improved accuracy. Recommended for hourly and minutely
|
53 |
-
resolutions (Ex. 10 min, 15 min, 1 hour, etc). (branch name: 1024-96-r2)
|
54 |
-
|
55 |
|
56 |
-
- **1536-96-r2**: Given the last 1536 time-points (i.e. context length), this model can forecast up to next 96 time-points (i.e. forecast length)
|
57 |
-
in future. This model is pre-trained with a larger pretraining dataset for improved accuracy. Recommended for hourly and minutely
|
58 |
-
resolutions (Ex. 10 min, 15 min, 1 hour, etc). (branch name: 1536-96-r2)
|
59 |
|
60 |
|
61 |
## Model Capabilities with example scripts
|
62 |
-
-
|
|
|
63 |
- Finetuned Multivariate Forecasting:
|
64 |
-
- Channel-Independent Finetuning
|
65 |
-
- Channel-Mix Finetuning
|
66 |
- **New Releases (extended features released on October 2024)**
|
67 |
-
- Finetuning and Forecasting with Exogenous/Control Variables
|
68 |
-
- Finetuning and Forecasting with static categorical features
|
69 |
-
- Rolling Forecasts - Extend forecast lengths beyond 96 via rolling capability
|
70 |
-
-
|
71 |
|
72 |
-
|
73 |
-
|
74 |
-
|
75 |
-
## How to Get Started with the Model
|
76 |
-
|
77 |
-
Please refer to the below scrips for **zero-shot** and **finetuning** support:
|
78 |
-
- [colab](https://colab.research.google.com/github/IBM/tsfm/blob/main/notebooks/tutorial/ttm_tutorial.ipynb)
|
79 |
-
- [512-96 Benchmarks](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_benchmarking_512_96.ipynb)
|
80 |
-
- [1024-96 Benchmarks](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_benchmarking_1024_96.ipynb)
|
81 |
-
- Script for Exogenous support - to be added soon
|
82 |
|
83 |
|
84 |
## Recommended Use
|
@@ -87,22 +71,12 @@ Please refer to the below scrips for **zero-shot** and **finetuning** support:
|
|
87 |
3. Enabling any upsampling or prepending zeros to virtually increase the context length for shorter-length datasets is not recommended and will
|
88 |
impact the model performance.
|
89 |
|
90 |
-
## Benchmark
|
91 |
-
|
92 |
-
- TTM (with less than 1 Million parameters) outperforms the following popular Pre-trained SOTAs demanding several hundred Million to Billions of parameters [paper](https://arxiv.org/pdf/2401.03955v5.pdf):
|
93 |
-
- *GPT4TS (NeurIPS 23) by 7-12% in few-shot forecasting*
|
94 |
-
- *LLMTime (NeurIPS 23) by 24% in zero-shot forecasting*.
|
95 |
-
- *SimMTM (NeurIPS 23) by 17% in few-shot forecasting*.
|
96 |
-
- *Time-LLM (ICLR 24) by 2-8% in few-shot forecasting*
|
97 |
-
- *UniTime (WWW 24) by 27% in zero-shot forecasting.*
|
98 |
-
- Zero-shot results of TTM surpass the *few-shot results of many popular SOTA approaches* including
|
99 |
-
PatchTST (ICLR 23), PatchTSMixer (KDD 23), TimesNet (ICLR 23), DLinear (AAAI 23) and FEDFormer (ICML 22).
|
100 |
- TTM (1024-96, released in this model card with 1M parameters) outperforms pre-trained MOIRAI-Small (14M parameters) by 10%, MOIRAI-Base (91M parameters) by 2% and
|
101 |
MOIRAI-Large (311M parameters) by 3% on zero-shot forecasting (horizon = 96). [[notebook]](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_benchmarking_1024_96.ipynb)
|
102 |
- TTM quick fine-tuning also outperforms the competitive statistical baselines (Statistical ensemble and S-Naive) in
|
103 |
M4-hourly dataset which existing pretrained TS models are finding difficult to outperform. [[notebook]](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_m4_hourly.ipynb)
|
104 |
-
|
105 |
-
opposed to long timing-requirements and heavy computing infra needs of other existing pre-trained models.
|
106 |
|
107 |
|
108 |
|
|
|
10 |
- time-series
|
11 |
---
|
12 |
|
13 |
+
# TinyTimeMixer (TTM) 1M Model Card
|
14 |
|
15 |
<p align="center" width="100%">
|
16 |
<img src="ttm_image.webp" width="600">
|
|
|
21 |
|
22 |
TTM outperforms several popular benchmarks demanding billions of parameters in zero-shot and few-shot forecasting. TTMs are lightweight
|
23 |
forecasters, pre-trained on publicly available time series data with various augmentations. TTM provides state-of-the-art zero-shot forecasts and can easily be
|
24 |
+
fine-tuned for multi-variate forecasts with just 5% of the training data to be competitive. Refer to our [paper](https://arxiv.org/pdf/2401.03955.pdf) for more details.
|
25 |
|
26 |
|
27 |
**The current open-source version supports point forecasting use-cases specifically ranging from minutely to hourly resolutions
|
|
|
36 |
|
37 |
- **512-96:** Given the last 512 time-points (i.e. context length), this model can forecast up to next 96 time-points (i.e. forecast length)
|
38 |
in future. This model is targeted towards a forecasting setting of context length 512 and forecast length 96 and
|
39 |
+
recommended for hourly and minutely resolutions (Ex. 10 min, 15 min, 1 hour, etc). (branch name: main) [Benchmarks](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_benchmarking_512_96.ipynb)
|
40 |
|
41 |
- **1024-96:** Given the last 1024 time-points (i.e. context length), this model can forecast up to next 96 time-points (i.e. forecast length)
|
42 |
in future. This model is targeted towards a long forecasting setting of context length 1024 and forecast length 96 and
|
43 |
+
recommended for hourly and minutely resolutions (Ex. 10 min, 15 min, 1 hour, etc). (branch name: 1024-96-v1) [Benchmarks](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_benchmarking_1024_96.ipynb)
|
44 |
|
45 |
- **New Releases (trained on larger pretraining datasets, released on October 2024)**:
|
46 |
|
47 |
- **512-96-r2**: Given the last 512 time-points (i.e. context length), this model can forecast up to next 96 time-points (i.e. forecast length)
|
48 |
in future. This model is pre-trained with a larger pretraining dataset for improved accuracy. Recommended for hourly and minutely
|
49 |
+
resolutions (Ex. 10 min, 15 min, 1 hour, etc). (branch name: 512-96-r2) [Benchmarks](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/hfdemo/tinytimemixer/ttm_v2_benchmarking_512_96.ipynb)
|
50 |
|
|
|
|
|
|
|
|
|
51 |
|
|
|
|
|
|
|
52 |
|
53 |
|
54 |
## Model Capabilities with example scripts
|
55 |
+
- Getting Started [colab](https://colab.research.google.com/github/IBM/tsfm/blob/main/notebooks/tutorial/ttm_tutorial.ipynb)
|
56 |
+
- Zeroshot Multivariate Forecasting [Example](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/hfdemo/ttm_getting_started.ipynb)
|
57 |
- Finetuned Multivariate Forecasting:
|
58 |
+
- Channel-Independent Finetuning [Example](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/hfdemo/ttm_getting_started.ipynb)
|
59 |
+
- Channel-Mix Finetuning [Example](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/tutorial/ttm_channel_mix_finetuning.ipynb)
|
60 |
- **New Releases (extended features released on October 2024)**
|
61 |
+
- Finetuning and Forecasting with Exogenous/Control Variables [Example](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/tutorial/ttm_with_exog_tutorial.ipynb)
|
62 |
+
- Finetuning and Forecasting with static categorical features [Example: To be added soon]
|
63 |
+
- Rolling Forecasts - Extend forecast lengths beyond 96 via rolling capability [Example](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/hfdemo/ttm_rolling_prediction_getting_started.ipynb)
|
64 |
+
- Helper scripts for optimal Learning Rate suggestions for Finetuning [Example](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/tutorial/ttm_with_exog_tutorial.ipynb)
|
65 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
66 |
|
67 |
|
68 |
## Recommended Use
|
|
|
71 |
3. Enabling any upsampling or prepending zeros to virtually increase the context length for shorter-length datasets is not recommended and will
|
72 |
impact the model performance.
|
73 |
|
74 |
+
## Other Benchmark Scripts:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
75 |
- TTM (1024-96, released in this model card with 1M parameters) outperforms pre-trained MOIRAI-Small (14M parameters) by 10%, MOIRAI-Base (91M parameters) by 2% and
|
76 |
MOIRAI-Large (311M parameters) by 3% on zero-shot forecasting (horizon = 96). [[notebook]](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_benchmarking_1024_96.ipynb)
|
77 |
- TTM quick fine-tuning also outperforms the competitive statistical baselines (Statistical ensemble and S-Naive) in
|
78 |
M4-hourly dataset which existing pretrained TS models are finding difficult to outperform. [[notebook]](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_m4_hourly.ipynb)
|
79 |
+
|
|
|
80 |
|
81 |
|
82 |
|