metadata
library_name: transformers
tags: []
Model Details
Model Description
This model was part of the Evolutionary Scale BioML Hackathon.
Uses
Used for ddG prediction for single mutation.
How to Get Started with the Model
# Make sure `esm` is installed, if not use: `pip install esm`
from transformers import AutoModel
from esm.tokenization.sequence_tokenizer import EsmSequenceTokenizer
import torch
model = AutoModel.from_pretrained("hazemessam/esm3_ddg_v2", trust_remote_code=True)
tokenizer = EsmSequenceTokenizer()
model.eval()
with torch.no_grad():
output = model(tokenized_seq1, tokenized_seq2, positions=mutation_position)
Training Details
Training Data
Training Data: https://huggingface.co/datasets/hazemessam/ddg/blob/main/S2648.csv
Training Procedure
The results listed below are the best results for each evaluation dataset, but this checkpoint is the best checkpoint based on Ssym evaluation dataset
Training Hyperparameters
- Scheduler: Cosine
- Warmup steps: 400
- Seed: 7
- Gradient accumulation steps: 16
- Batch size: 1
- DoRA rank: 16
- DoRA alpha: 32
- Updated Layers: ["layernorm_qkv.1", "ffn.1", "ffn.3"]
- DoRA bias: "none"
[More Information Needed]
Evaluation
Testing Data, Factors & Metrics
Testing Data
The model was evaluated on the following:
- Ssym: https://huggingface.co/datasets/hazemessam/ddg/blob/main/ssym.csv
- Ssym_r: https://huggingface.co/datasets/hazemessam/ddg/blob/main/ssym_r.csv
- P53: https://huggingface.co/datasets/hazemessam/ddg/blob/main/p53.csv
- Myoglobin: https://huggingface.co/datasets/hazemessam/ddg/blob/main/myoglobin.csv
- Myoglobin_r: https://huggingface.co/datasets/hazemessam/ddg/blob/main/myoglobin_r.csv
Results
Ssym pearson correlation: 0.85 Ssym RMSE: 0.83
Ssym_r pearson correlation: 0.85 Ssym_r RMSE: 0.83
Myoglobin pearson correlation: 0.65 Myoglobin RMSE: 0.83
Myoglobin_r pearson correlation: 0.65 Myoglobin_r RMSE: 0.84