add-readme

README.md

@@ -1,104 +1,83 @@
 ---
 license: apache-2.0
-pipeline_tag: time-series-forecasting
 tags:
-  - time series
-  - forecasting
-  - pretrained models
-  - foundation models
-  - time series foundation models
-  - time-series
+- time series
+- forecasting
+- pretrained models
+- foundation models
+- time series foundation models
+- time-series
 ---
 
-# Chronos-T5 (Mini)
+# Chronos-T5 Mini
 
-Chronos is a family of **pretrained time series forecasting models** based on language model architectures. A time series is transformed into a sequence of tokens via scaling and quantization, and a language model is trained on these tokens using the cross-entropy loss. Once trained, probabilistic forecasts are obtained by sampling multiple future trajectories given the historical context. Chronos models have been trained on a large corpus of publicly available time series data, as well as synthetic data generated using Gaussian processes.
+Chronos models are pre-trained **time series forecasting models** based on language model architectures.
+A time series is transformed into a sequence of tokens via scaling and quantization, and forecasts are obtained by sampling multiple sequences of future observations given historical context.
+Chronos models are trained on a large corpus of publicly available time series data, as well as synthetic data.
 
-For details on Chronos models, training data and procedures, and experimental results, please refer to the paper [Chronos: Learning the Language of Time Series](https://arxiv.org/abs/2403.07815).
-
-<p align="center">
-  <img src="figures/main-figure.png" width="100%">
-  <br />
-  <span>
-    Fig. 1: High-level depiction of Chronos. (<b>Left</b>) The input time series is scaled and quantized to obtain a sequence of tokens. (<b>Center</b>) The tokens are fed into a language model which may either be an encoder-decoder or a decoder-only model. The model is trained using the cross-entropy loss. (<b>Right</b>) During inference, we autoregressively sample tokens from the model and map them back to numerical values. Multiple trajectories are sampled to obtain a predictive distribution.
-  </span>
-</p>
-
----
+For details on Chronos models, training data and procedures, and experimental results, refer to the paper [Chronos: Learning the Language of Time Series](https://www.example.com/).
 
 ## Architecture
 
-The models in this repository are based on the [T5 architecture](https://arxiv.org/abs/1910.10683). The only difference is in the vocabulary size: Chronos-T5 models use 4096 different tokens, compared to 32128 of the original T5 models, resulting in fewer parameters.
+The model in this repository is based on the [T5 architecture](https://arxiv.org/abs/1910.10683).
+The only difference is in the vocabulary size:
+Chronos-T5 uses 4096 different tokens, compared to 32128 of the original T5 models, resulting in a smaller number of total parameters.
 
-| Model                                                                  | Parameters | Based on                                                               |
-| ---------------------------------------------------------------------- | ---------- | ---------------------------------------------------------------------- |
-| [**chronos-t5-tiny**](https://huggingface.co/amazon/chronos-t5-tiny)   | 8M         | [t5-efficient-tiny](https://huggingface.co/google/t5-efficient-tiny)   |
-| [**chronos-t5-mini**](https://huggingface.co/amazon/chronos-t5-mini)   | 20M        | [t5-efficient-mini](https://huggingface.co/google/t5-efficient-mini)   |
-| [**chronos-t5-small**](https://huggingface.co/amazon/chronos-t5-small) | 46M        | [t5-efficient-small](https://huggingface.co/google/t5-efficient-small) |
-| [**chronos-t5-base**](https://huggingface.co/amazon/chronos-t5-base)   | 200M       | [t5-efficient-base](https://huggingface.co/google/t5-efficient-base)   |
-| [**chronos-t5-large**](https://huggingface.co/amazon/chronos-t5-large) | 710M       | [t5-efficient-large](https://huggingface.co/google/t5-efficient-large) |
+ Model          | Parameters        | Based on
+----------------|-------------------|----------------------
+[chronos-t5-mini](https://huggingface.co/amazon/chronos-t5-mini) | 20M | [t5-efficient-mini](https://huggingface.co/google/t5-efficient-mini)
+[chronos-t5-small](https://huggingface.co/amazon/chronos-t5-small) | 46M | [t5-efficient-small](https://huggingface.co/google/t5-efficient-small)
+[chronos-t5-base](https://huggingface.co/amazon/chronos-t5-base) | 200M | [t5-efficient-base](https://huggingface.co/google/t5-efficient-base)
+[chronos-t5-large](https://huggingface.co/amazon/chronos-t5-large) | 710M | [t5-efficient-large](https://huggingface.co/google/t5-efficient-large)
 
 ## Usage
 
-To perform inference with Chronos models, install the package in the GitHub [companion repo](https://github.com/amazon-science/chronos-forecasting) by running:
+To do inference with Chronos models, you will need to install the code from the [companion GitHub repo](https://www.example.com/).
 
-```
+```bash
 pip install git+https://github.com/amazon-science/chronos-forecasting.git
 ```
 
-A minimal example showing how to perform inference using Chronos models:
+A minimal example:
 
 ```python
-import matplotlib.pyplot as plt
 import numpy as np
 import pandas as pd
+import matplotlib.pyplot as plt
 import torch
 from chronos import ChronosPipeline
 
-pipeline = ChronosPipeline.from_pretrained(
-  "amazon/chronos-t5-mini",
-  device_map="cuda",
-  torch_dtype=torch.bfloat16,
-)
+pipeline = ChronosPipeline.from_pretrained("amazon/chronos-t5-base")
 
-df = pd.read_csv("https://raw.githubusercontent.com/AileenNielsen/TimeSeriesAnalysisWithPython/master/data/AirPassengers.csv")
+df = pd.read_csv(
+    "https://raw.githubusercontent.com/AileenNielsen/"
+    "TimeSeriesAnalysisWithPython/master/data/AirPassengers.csv",
+    index_col=0,
+    parse_dates=True,
+)
 
-# context must be either a 1D tensor, a list of 1D tensors,
-# or a left-padded 2D tensor with batch as the first dimension
-context = torch.tensor(df["#Passengers"])
-prediction_length = 12
-forecast = pipeline.predict(context, prediction_length)  # shape [num_series, num_samples, prediction_length]
+context = torch.Tensor(df["#Passengers"].values)
+forecast = pipeline.predict(context, prediction_length=12)
 
-# visualize the forecast
-forecast_index = range(len(df), len(df) + prediction_length)
-low, median, high = np.quantile(forecast[0].numpy(), [0.1, 0.5, 0.9], axis=0)
+forecast_steps = range(len(df), len(df) + 12)
+forecast_np = forecast.numpy()[0].T
+low = np.quantile(forecast_np, 0.1, axis=1)
+median = np.quantile(forecast_np, 0.5, axis=1)
+high = np.quantile(forecast_np, 0.9, axis=1)
 
-plt.figure(figsize=(8, 4))
-plt.plot(df["#Passengers"], color="royalblue", label="historical data")
-plt.plot(forecast_index, median, color="tomato", label="median forecast")
-plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% prediction interval")
+plt.plot(range(len(df)), df["#Passengers"], color="royalblue", label="historical data")
+plt.plot(forecast_steps, forecast_np, color="grey", alpha=0.1)
+plt.fill_between(forecast_steps, low, high, color="tomato", alpha=0.4, label="80% interval")
+plt.plot(forecast_steps, median, color="tomato", label="median")
 plt.legend()
 plt.grid()
 plt.show()
 ```
 
-## Citation
+## References
 
-If you find Chronos models useful for your research, please consider citing the associated [paper](https://arxiv.org/abs/2403.07815):
+If you find Chronos models useful for your research, please consider citing the associated [paper](https://www.example.com/):
 
 ```
-@article{ansari2024chronos,
-  author  = {Ansari, Abdul Fatir and Stella, Lorenzo and Turkmen, Caner and Zhang, Xiyuan, and Mercado, Pedro and Shen, Huibin and Shchur, Oleksandr and Rangapuram, Syama Syndar and Pineda Arango, Sebastian and Kapoor, Shubham and Zschiegner, Jasper and Maddix, Danielle C. and Mahoney, Michael W. and Torkkola, Kari and Gordon Wilson, Andrew and Bohlke-Schneider, Michael and Wang, Yuyang},
-  title   = {Chronos: Learning the Language of Time Series},
-  journal = {arXiv preprint arXiv:2403.07815},
-  year    = {2024}
-}
+paper citation
 ```
-
-## Security
-
-See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
-
-## License
-
-This project is licensed under the Apache-2.0 License.

chronos_config.json

@@ -0,0 +1,16 @@
+{
+  "low_limit": -15.0,
+  "high_limit": 15.0,
+  "n_tokens": 4096,
+  "n_special_tokens": 2,
+  "pad_token_id": 0,
+  "eos_token_id": 1,
+  "use_eos_token": true,
+  "model_type": "seq2seq",
+  "context_length": 512,
+  "prediction_length": 64,
+  "num_samples": 20,
+  "temperature": 1.0,
+  "top_k": 50,
+  "top_p": 1.0
+}

config.json

@@ -1,5 +1,7 @@
 {
-  "architectures": ["T5ForConditionalGeneration"],
+  "architectures": [
+    "T5ForConditionalGeneration"
+  ],
   "d_ff": 1536,
   "d_kv": 64,
   "d_model": 384,
@@ -11,7 +13,7 @@
   "initializer_factor": 0.05,
   "is_encoder_decoder": true,
   "is_gated_act": false,
-  "layer_norm_epsilon": 1e-6,
+  "layer_norm_epsilon": 1e-06,
   "model_type": "t5",
   "n_positions": 512,
   "num_decoder_layers": 4,
@@ -20,27 +22,8 @@
   "pad_token_id": 0,
   "relative_attention_max_distance": 128,
   "relative_attention_num_buckets": 32,
-  "torch_dtype": "float32",
+  "torch_dtype": "bfloat16",
   "transformers_version": "4.31.0",
   "use_cache": true,
-  "vocab_size": 4096,
-  "chronos_config": {
-    "tokenizer_class": "MeanScaleUniformBins",
-    "tokenizer_kwargs": {
-      "low_limit": -15.0,
-      "high_limit": 15.0
-    },
-    "n_tokens": 4096,
-    "n_special_tokens": 2,
-    "pad_token_id": 0,
-    "eos_token_id": 1,
-    "use_eos_token": true,
-    "model_type": "seq2seq",
-    "context_length": 512,
-    "prediction_length": 64,
-    "num_samples": 20,
-    "temperature": 1.0,
-    "top_k": 50,
-    "top_p": 1.0
-  }
+  "vocab_size": 4096
 }

model.safetensors → pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5cf49253460171e8dfc3a75cfd5e3796091352ec48df16baeb8a1bfeacef769c
-size 81835272
+oid sha256:a009ea2d6d916da48e620eefda4d11c1b072550f92be248b4c5abdde34cb2060
+size 40943036