metadata

language:
  - pl
tags:
  - text
  - sentiment
  - politics
metrics:
  - accuracy
  - f1
pipeline_tag: text-classification
widget:
  - text: Przykro patrzeć, a słuchać się nie da.
    example_title: example 1
  - text: Oczywiście ze Pan Prezydent to nasza duma narodowa!!
    example_title: example 2
base_model: dkleczek/bert-base-polish-cased-v1
model-index:
  - name: PaReS-sentimenTw-political-PL
    results:
      - task:
          type: sentiment-classification
          name: Text Classification
        dataset:
          name: tweets_2020_electionsPL
          type: tweets
        metrics:
          - type: f1
            value: 94.4

PaReS-sentimenTw-political-PL

This model is a fine-tuned version of dkleczek/bert-base-polish-cased-v1 to predict 3-categorical sentiment. Fine-tuned on 1k sample of manually annotated Twitter data.

Model developed as a part of ComPathos project: https://www.ncn.gov.pl/sites/default/files/listy-rankingowe/2020-09-30apsv2/streszczenia/497124-en.pdf

from transformers import pipeline

model_path = "eevvgg/PaReS-sentimenTw-political-PL"
sentiment_task = pipeline(task = "sentiment-analysis", model = model_path, tokenizer = model_path)

sequence = ["Cała ta śmieszna debata była próbą ukrycia problemów gospodarczych jakie są i nadejdą, pytania w większości o mało istotnych sprawach", 
            "Brawo panie ministrze!"]
            
result = sentiment_task(sequence)
labels = [i['label'] for i in result] # ['Negative', 'Positive']

Intended uses & limitations

Sentiment detection in Polish data (fine-tuned on tweets from political domain).

Training and evaluation data

Trained for 3 epochs, mini-batch size of 8.
Training results: loss: 0.1358926964368792

It achieves the following results on the test set (10%):

No. examples = 100
mini batch size = 8
accuracy = 0.950

macro f1 = 0.944

        precision    recall  f1-score   support

     0      0.960     0.980     0.970        49
     1      0.958     0.885     0.920        26
     2      0.923     0.960     0.941        25