---
license: apache-2.0

language:
- ru
tags:
- PyTorch
- Transformers
---

# BERT base model for pair ranking (reward model for RLHF) in Russian language.
For training i use the next [pair-ranking-loss](https://pytorch.org/docs/stable/generated/torch.nn.MarginRankingLoss.html)

Datasets have been translated with google-translate-api for reward training:

- [Anthropic/hh-rlhf](https://huggingface.co/datasets/Anthropic/hh-rlhf)
- [Dahoas/synthetic-instruct-gptj-pairwise](https://huggingface.co/datasets/Dahoas/synthetic-instruct-gptj-pairwise)
- [openai/webgpt_comparisons](https://huggingface.co/datasets/openai/webgpt_comparisons)

For better quality, use mean token embeddings.
## Usage (HuggingFace Models Repository)
You can use the model directly from the model repository to compute score:
```python

from transformers import AutoModelForSequenceClassification, AutoTokenizer

#Create model object and inits pretrain weights:
reward_name = "Andrilko/ruBert-base-reward"
rank_model =  AutoModelForSequenceClassification.from_pretrained(reward_name)
tokenizer = AutoTokenizer.from_pretrained(reward_name)


#Sentences that we want to score:
sentences =  ['Человек: Что такое QR-код?','Ассистент: QR-код - это тип матричного штрих-кода.']

#Compute token embeddings
inputs = tokenizer(sentences[0], sentences[1], return_tensors='pt')
score = rank_model(**inputs).logits[0].cpu().detach()
print(score)
```

# Authors
+ Aleksandr Abramov: [Github](https://github.com/Ab1992ao), [Kaggle Competitions Master](https://www.kaggle.com/andrilko);