metadata
license: apache-2.0
tags:
- protein language model
- generated_from_trainer
datasets:
- train
metrics:
- spearmanr
model-index:
- name: tape-fluorescence-prediction-RITA_s
results:
- task:
name: Text Classification
type: text-classification
dataset:
name: cradle-bio/tape-fluorescence
type: train
metrics:
- name: Spearmanr
type: spearmanr
value: 0.32362885705173594
tape-fluorescence-prediction-RITA_s
This model is a fine-tuned version of lightonai/RITA_s on the cradle-bio/tape-fluorescence dataset. It achieves the following results on the evaluation set:
- Loss: 0.7029
- Spearmanr: 0.3236
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 17
- gradient_accumulation_steps: 128
- total_train_batch_size: 4096
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Spearmanr |
|---|---|---|---|---|
| 7.0854 | 0.85 | 4 | 0.8055 | 0.0686 |
| 0.8791 | 1.85 | 8 | 0.7455 | 0.1288 |
| 0.822 | 2.85 | 12 | 0.7385 | 0.2069 |
| 0.8155 | 3.85 | 16 | 0.7346 | 0.2680 |
| 0.8139 | 4.85 | 20 | 0.7345 | 0.3102 |
| 0.81 | 5.85 | 24 | 0.7375 | 0.3349 |
| 0.805 | 6.85 | 28 | 0.7211 | 0.3298 |
| 0.799 | 7.85 | 32 | 0.7114 | 0.3282 |
| 0.784 | 8.85 | 36 | 0.7132 | 0.3220 |
| 0.6627 | 9.85 | 40 | 0.7029 | 0.3236 |
Framework versions
- Transformers 4.18.0
- Pytorch 1.11.0
- Datasets 2.1.0
- Tokenizers 0.12.1