|
--- |
|
library_name: transformers |
|
tags: |
|
- Compiler |
|
- LLVM |
|
- Intermediate Representation |
|
- IR |
|
- Path |
|
- Hot Path |
|
datasets: |
|
- zhaojer/compiler_hot_paths |
|
language: |
|
- en |
|
base_model: |
|
- google-bert/bert-base-uncased |
|
--- |
|
|
|
# Model Card for BERT Hot Path Predictor |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
This BERT model performs hot path prediction: Given a path (i.e. a sequence of LLVM IR instructions), predict whether it is "hot" (1) or "cold" (0). |
|
It was fine-tuned on the [hot paths dataset](https://huggingface.co/datasets/zhaojer/compiler_hot_paths) for 3 epochs with standard learning hyperparameters. |
|
|
|
- **Model type:** Binary Sequence Classification |
|
- **Language(s) (NLP):** English, Compiler/LLVM |
|
- **Finetuned from model:** google-bert/bert-base-uncased |
|
- **Dataset used:** zhaojer/compiler_hot_paths |
|
|
|
## Uses |
|
|
|
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. --> |
|
|
|
The model can be used to predict whether a path is hot or cold, which is important information for compiler optimizations. Here is an instance of the prediction pipeline: |
|
1. Given a program (written in C, C++, Fortran, or other languages supported by LLVM), compile it into LLVM IR (e.g., `clang -S -emit-llvm program.c -o program.ll`) |
|
2. Select a sequence of instructions (in the unit of basic blocks) from the IR file; use this as the input to the model. |
|
3. Load the present model and feed it the selected input, the model will then output either 0 (cold path) or 1 (hot path). |
|
|
|
The model can be further fine-tuned using additional data. Please see zhaojer/compiler_hot_paths dataset card for more information on the expected data used for fine-tuning. |
|
|
|
## How to Get Started with the Model |
|
|
|
Use the code below to get started with the model. |
|
|
|
``` |
|
from transformers import BertForSequenceClassification, BertTokenizer, pipeline |
|
|
|
# Load saved model |
|
saved_model = BertForSequenceClassification.from_pretrained("zhaojer/bert-hot-path-predictor") |
|
saved_tokenizer = BertTokenizer.from_pretrained("zhaojer/bert-hot-path-predictor") |
|
|
|
# Pipeline for predictions |
|
classifier = pipeline("text-classification", model=saved_model, tokenizer=saved_tokenizer) |
|
|
|
# Example prediction |
|
new_path = "%26 = load i32, ptr %21, align 4\n%27 = load i32, ptr %11, align\n%28 = icmp slt i32 %26, %27\nbr i1 %28, label %29, label %59\n\nstore i32 0, ptr %22, align 4\nbr label %30" |
|
prediction = classifier(new_path) |
|
print(prediction) |
|
``` |
|
|
|
## Training Details |
|
|
|
### Training Data |
|
|
|
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. --> |
|
The model was fine-tuned on the hot paths dataset: [zhaojer/compiler_hot_paths](https://huggingface.co/datasets/zhaojer/compiler_hot_paths) |
|
|
|
The dataset is already split into train, validation, test sets with necessary columns/data needed for training/fine-tuning. No further preprocessing was performed for the data. |
|
|
|
The data (in the `path` column) were tokenized using the standard `BertTokenizer` for the `bert-base-uncased` model. |
|
|
|
### Training Procedure |
|
|
|
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. --> |
|
|
|
We defined accuracy and AUROC as evaluation metrics for the model. |
|
|
|
The model was fine-tuned for 3 epochs with standard hyperparameters, which took about 10 minutes to complete using NVIDIA T4 GPU. |
|
|
|
#### Detailed Training Hyperparameters |
|
|
|
- `evaluation_strategy="epoch"` |
|
- `logging_strategy="epoch"` |
|
- `save_strategy="epoch"` |
|
- `num_train_epochs=3` |
|
- `per_device_train_batch_size=16` |
|
- `per_device_eval_batch_size=16` |
|
- `learning_rate=5e-5` |
|
- `load_best_model_at_end=True` |
|
- `metric_for_best_model="accuracy"` |
|
|
|
Note: Anything not explicitly stated used default value. |
|
|
|
## Evaluation |
|
|
|
<!-- This section describes the evaluation protocols and provides the results. --> |
|
|
|
### Testing Data |
|
|
|
<!-- This should link to a Dataset Card if possible. --> |
|
The testing data consist of 68 hot paths and 92 cold paths generated from 4 distinct C programs. |
|
They are also from [zhaojer/compiler_hot_paths](https://huggingface.co/datasets/zhaojer/compiler_hot_paths); please see its dataset card for how the testing data were created. |
|
The model had never seen these testing data previously. |
|
|
|
### Metrics |
|
|
|
<!-- These are the evaluation metrics being used, ideally with a description of why. --> |
|
We evaluated the model on the testing data using the following metrics: |
|
- Loss (available by default) |
|
- Accuracy |
|
- AUROC |
|
- Precision, Recall, F1 score |
|
- Confusion matrix |
|
|
|
### Results |
|
|
|
| Loss | Accuracy | AUROC | Precision | Recall | F1 | |
|
| ---- | -------- | ----- | --------- | ------ | ---- | |
|
| 0.0620 | 0.9875 | 0.9952| 1.0000 | 0.9706 | 0.99 | |
|
|
|
| | Actually Hot | Actually Cold | |
|
| ------------- | ----------- | ------------ | |
|
| Predicted Hot | 66 | 0 | |
|
| Predicted Cold| 2 | 92 | |