File size: 1,984 Bytes
e821e69 6a83889 e821e69 6a83889 e821e69 6a83889 e821e69 1035829 6a83889 e821e69 6a83889 e821e69 6a83889 e821e69 6a83889 e821e69 6a83889 e821e69 6a83889 e821e69 6a83889 e821e69 6a83889 e821e69 6a83889 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
---
library_name: transformers
tags: []
---
## Model Details
### Model Description
This model was part of the Evolutionary Scale BioML Hackathon.
## Uses
Used for ddG prediction for single mutation.
## How to Get Started with the Model
```python
# Make sure `esm` is installed, if not use: `pip install esm`
from transformers import AutoModel
from esm.tokenization.sequence_tokenizer import EsmSequenceTokenizer
import torch
model = AutoModel.from_pretrained("hazemessam/esm3_ddg_v2", trust_remote_code=True)
tokenizer = EsmSequenceTokenizer()
model.eval()
with torch.no_grad():
output = model(tokenized_seq1, tokenized_seq2, positions=mutation_position)
```
## Training Details
### Training Data
Training Data: https://huggingface.co/datasets/hazemessam/ddg/blob/main/S2648.csv
### Training Procedure
The results listed below are the best results for each evaluation dataset, but this checkpoint is the best checkpoint based on `Ssym` evaluation dataset
#### Training Hyperparameters
* Scheduler: Cosine
* Warmup steps: 400
* Seed: 7
* Gradient accumulation steps: 16
* Batch size: 1
* DoRA rank: 16
* DoRA alpha: 32
* Updated Layers: ["layernorm_qkv.1", "ffn.1", "ffn.3"]
* DoRA bias: "none"
[More Information Needed]
## Evaluation
### Testing Data, Factors & Metrics
#### Testing Data
The model was evaluated on the following:
* Ssym: https://huggingface.co/datasets/hazemessam/ddg/blob/main/ssym.csv
* Ssym_r: https://huggingface.co/datasets/hazemessam/ddg/blob/main/ssym_r.csv
* P53: https://huggingface.co/datasets/hazemessam/ddg/blob/main/p53.csv
* Myoglobin: https://huggingface.co/datasets/hazemessam/ddg/blob/main/myoglobin.csv
* Myoglobin_r: https://huggingface.co/datasets/hazemessam/ddg/blob/main/myoglobin_r.csv
### Results
Ssym pearson correlation: 0.85
Ssym RMSE: 0.83
Ssym_r pearson correlation: 0.85
Ssym_r RMSE: 0.83
Myoglobin pearson correlation: 0.65
Myoglobin RMSE: 0.83
Myoglobin_r pearson correlation: 0.65
Myoglobin_r RMSE: 0.84
|