metadata

tags:
  - image-classification
  - timm
  - chart
  - charts
  - fintwit
  - stocks
  - crypto
  - finance
  - financial
  - financial charts
  - graphs
  - financial graphs
  - plot
  - plots
  - financial plots
  - cryptocurrency
  - image-recognition
  - recognition
library_name: timm
license: mit
datasets:
  - StephanAkkerman/crypto-charts
  - StephanAkkerman/stock-charts
  - StephanAkkerman/fintwit-images
language:
  - en
metrics:
  - accuracy
  - f1
  - precision
  - recall
model-index:
  - name: chart-recognizer
    results:
      - task:
          type: image-classification
        dataset:
          name: Test Set
          type: images
        metrics:
          - type: accuracy
            value: 0.9782
          - type: f1
            value: 0.9685
pipeline_tag: image-classification
base_model: timm/efficientnet_b0.ra_in1k

Chart Recognizer

chart-recognizer is a finetuned model for classifying images. It uses efficientnet as its base model, making it a fast and small model. This model is trained on my own dataset of financial charts posted on Twitter, which can be found here StephanAkkerman/fintwit-charts.

Intended Uses

chart-recognizer is intended for classifying images, mainly images posted on social media.

Dataset

chart-recognizer has been trained on my own dataset. So far I have not been able to find another image dataset about financial charts.

StephanAkkerman/crypto-charts: 4,880 images.
StephanAkkerman/stock-charts: 5,203 images.
StephanAkkerman/fintwit-images: 4,579 images.

Example Images

The following images are not part of the training set and can be used for testing purposes.

Chart

Non-Chart

This can be any image that does not represent a (financial) chart.

More Information

For a comprehensive overview, including the training setup and analysis of the model, visit the chart-recognizer GitHub repository.

Usage

Using HuggingFace's transformers library the model can be converted into a pipeline for image classification.

import timm
import torch
from PIL import Image
from timm.data import resolve_data_config, create_transform

# Load and set model to eval mode
model = timm.create_model("hf_hub:StephanAkkerman/chart-recognizer", pretrained=True)
model.eval()

# Create transform and get labels
transform = create_transform(**resolve_data_config(model.pretrained_cfg, model=model))
labels = model.pretrained_cfg["label_names"]

# Load and preprocess image
image = Image.open("img/examples/tweet_example.png").convert("RGB")
x = transform(image).unsqueeze(0)

# Get model output and apply softmax
probabilities = torch.nn.functional.softmax(model(x)[0], dim=0)

# Map probabilities to labels
output = {label: prob.item() for label, prob in zip(labels, probabilities)}

# Print the predicted probabilities
print(output)

Citing & Authors

If you use chart-recognizer in your research, please cite me as follows:

@misc{chart-recognizer,
  author = {Stephan Akkerman},
  title = {chart-recognizer: A Specialized Image Model for Financial Charts},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/StephanAkkerman/chart-recognizer}}
}

License

This project is licensed under the MIT License. See the LICENSE file for details.