javicorvi/pretoxtm-ner

README.md

@@ -2,6 +2,11 @@
 base_model: dmis-lab/biobert-v1.1
 tags:
 - generated_from_trainer
+metrics:
+- precision
+- recall
+- f1
+- accuracy
 model-index:
 - name: pretoxtm-ner
   results: []
@@ -14,15 +19,19 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [dmis-lab/biobert-v1.1](https://huggingface.co/dmis-lab/biobert-v1.1) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.1810
-- Study Test: {'precision': 0.8215384615384616, 'recall': 0.8841059602649006, 'f1': 0.8516746411483254, 'number': 302}
-- Manifestation: {'precision': 0.8041958041958042, 'recall': 0.905511811023622, 'f1': 0.8518518518518519, 'number': 127}
-- Finding: {'precision': 0.6886657101865137, 'recall': 0.7570977917981072, 'f1': 0.7212622088655146, 'number': 634}
-- Specimen: {'precision': 0.7944162436548223, 'recall': 0.8236842105263158, 'f1': 0.8087855297157622, 'number': 380}
-- Dose: {'precision': 0.8647540983606558, 'recall': 0.9461883408071748, 'f1': 0.9036402569593148, 'number': 223}
-- Dose Qualification: {'precision': 0.65, 'recall': 0.8125, 'f1': 0.7222222222222223, 'number': 32}
-- Sex: {'precision': 0.9285714285714286, 'recall': 0.9285714285714286, 'f1': 0.9285714285714286, 'number': 84}
-- Group: {'precision': 0.5666666666666667, 'recall': 0.6938775510204082, 'f1': 0.6238532110091742, 'number': 49}
+- Loss: 0.2722
+- Study Test: {'precision': 0.8222222222222222, 'recall': 0.8763157894736842, 'f1': 0.8484076433121018, 'number': 380}
+- Manifestation: {'precision': 0.841025641025641, 'recall': 0.9265536723163842, 'f1': 0.8817204301075269, 'number': 177}
+- Finding: {'precision': 0.7870485678704857, 'recall': 0.8154838709677419, 'f1': 0.8010139416983523, 'number': 775}
+- Specimen: {'precision': 0.7793427230046949, 'recall': 0.8469387755102041, 'f1': 0.8117359413202934, 'number': 392}
+- Dose: {'precision': 0.9595959595959596, 'recall': 0.9726962457337884, 'f1': 0.9661016949152542, 'number': 293}
+- Dose Qualification: {'precision': 0.8787878787878788, 'recall': 0.8787878787878788, 'f1': 0.8787878787878788, 'number': 33}
+- Sex: {'precision': 0.9279279279279279, 'recall': 0.9809523809523809, 'f1': 0.9537037037037037, 'number': 105}
+- Group: {'precision': 0.8913043478260869, 'recall': 0.8817204301075269, 'f1': 0.8864864864864864, 'number': 93}
+- Precision: 0.8298
+- Recall: 0.8719
+- F1: 0.8503
+- Accuracy: 0.9530
 
 ## Model description
 
@@ -41,26 +50,30 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 5e-05
-- train_batch_size: 8
+- learning_rate: 5.760003080365119e-05
+- train_batch_size: 4
 - eval_batch_size: 8
-- seed: 42
+- seed: 4
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 3
+- num_epochs: 7
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | Study Test                                                                                               | Manifestation                                                                                            | Finding                                                                                                  | Specimen                                                                                                 | Dose                                                                                                     | Dose Qualification                                                                          | Sex                                                                                                     | Group                                                                                                   |
-|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------:|
-| No log        | 1.0   | 257  | 0.2005          | {'precision': 0.6658227848101266, 'recall': 0.8708609271523179, 'f1': 0.7546628407460545, 'number': 302} | {'precision': 0.7647058823529411, 'recall': 0.9212598425196851, 'f1': 0.8357142857142856, 'number': 127} | {'precision': 0.6425339366515838, 'recall': 0.6719242902208202, 'f1': 0.6569005397070162, 'number': 634} | {'precision': 0.7099767981438515, 'recall': 0.8052631578947368, 'f1': 0.75462392108508, 'number': 380}   | {'precision': 0.8969957081545065, 'recall': 0.9372197309417041, 'f1': 0.9166666666666667, 'number': 223} | {'precision': 0.6764705882352942, 'recall': 0.71875, 'f1': 0.696969696969697, 'number': 32} | {'precision': 0.7448979591836735, 'recall': 0.8690476190476191, 'f1': 0.8021978021978022, 'number': 84} | {'precision': 0.3880597014925373, 'recall': 0.5306122448979592, 'f1': 0.4482758620689655, 'number': 49} |
-| 0.2932        | 2.0   | 514  | 0.1689          | {'precision': 0.8170347003154574, 'recall': 0.8576158940397351, 'f1': 0.8368336025848143, 'number': 302} | {'precision': 0.8226950354609929, 'recall': 0.9133858267716536, 'f1': 0.8656716417910448, 'number': 127} | {'precision': 0.6904400606980273, 'recall': 0.7176656151419558, 'f1': 0.7037896365042536, 'number': 634} | {'precision': 0.7746478873239436, 'recall': 0.868421052631579, 'f1': 0.8188585607940446, 'number': 380}  | {'precision': 0.8870292887029289, 'recall': 0.9506726457399103, 'f1': 0.9177489177489178, 'number': 223} | {'precision': 0.7567567567567568, 'recall': 0.875, 'f1': 0.8115942028985507, 'number': 32}  | {'precision': 0.8695652173913043, 'recall': 0.9523809523809523, 'f1': 0.909090909090909, 'number': 84}  | {'precision': 0.6, 'recall': 0.673469387755102, 'f1': 0.6346153846153846, 'number': 49}                 |
-| 0.2932        | 3.0   | 771  | 0.1810          | {'precision': 0.8215384615384616, 'recall': 0.8841059602649006, 'f1': 0.8516746411483254, 'number': 302} | {'precision': 0.8041958041958042, 'recall': 0.905511811023622, 'f1': 0.8518518518518519, 'number': 127}  | {'precision': 0.6886657101865137, 'recall': 0.7570977917981072, 'f1': 0.7212622088655146, 'number': 634} | {'precision': 0.7944162436548223, 'recall': 0.8236842105263158, 'f1': 0.8087855297157622, 'number': 380} | {'precision': 0.8647540983606558, 'recall': 0.9461883408071748, 'f1': 0.9036402569593148, 'number': 223} | {'precision': 0.65, 'recall': 0.8125, 'f1': 0.7222222222222223, 'number': 32}               | {'precision': 0.9285714285714286, 'recall': 0.9285714285714286, 'f1': 0.9285714285714286, 'number': 84} | {'precision': 0.5666666666666667, 'recall': 0.6938775510204082, 'f1': 0.6238532110091742, 'number': 49} |
+| Training Loss | Epoch | Step | Validation Loss | Study Test                                                                                               | Manifestation                                                                                            | Finding                                                                                                  | Specimen                                                                                                 | Dose                                                                                                     | Dose Qualification                                                                                      | Sex                                                                                                      | Group                                                                                                   | Precision | Recall | F1     | Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------:|:---------:|:------:|:------:|:--------:|
+| 0.3365        | 1.0   | 514  | 0.2023          | {'precision': 0.740909090909091, 'recall': 0.8578947368421053, 'f1': 0.7951219512195122, 'number': 380}  | {'precision': 0.7874396135265701, 'recall': 0.9209039548022598, 'f1': 0.8489583333333334, 'number': 177} | {'precision': 0.7055214723926381, 'recall': 0.7419354838709677, 'f1': 0.7232704402515724, 'number': 775} | {'precision': 0.7312072892938497, 'recall': 0.8188775510204082, 'f1': 0.772563176895307, 'number': 392}  | {'precision': 0.9243986254295533, 'recall': 0.9180887372013652, 'f1': 0.9212328767123289, 'number': 293} | {'precision': 0.8, 'recall': 0.8484848484848485, 'f1': 0.823529411764706, 'number': 33}                 | {'precision': 0.9074074074074074, 'recall': 0.9333333333333333, 'f1': 0.9201877934272301, 'number': 105} | {'precision': 0.7441860465116279, 'recall': 0.6881720430107527, 'f1': 0.7150837988826816, 'number': 93} | 0.7617    | 0.8203 | 0.7899 | 0.9397   |
+| 0.1325        | 2.0   | 1028 | 0.2012          | {'precision': 0.7920792079207921, 'recall': 0.8421052631578947, 'f1': 0.8163265306122449, 'number': 380} | {'precision': 0.8488372093023255, 'recall': 0.8248587570621468, 'f1': 0.836676217765043, 'number': 177}  | {'precision': 0.7163289630512515, 'recall': 0.775483870967742, 'f1': 0.7447335811648079, 'number': 775}  | {'precision': 0.723175965665236, 'recall': 0.8596938775510204, 'f1': 0.7855477855477856, 'number': 392}  | {'precision': 0.9013157894736842, 'recall': 0.9351535836177475, 'f1': 0.9179229480737018, 'number': 293} | {'precision': 0.8181818181818182, 'recall': 0.8181818181818182, 'f1': 0.8181818181818182, 'number': 33} | {'precision': 0.911504424778761, 'recall': 0.9809523809523809, 'f1': 0.944954128440367, 'number': 105}   | {'precision': 0.8645833333333334, 'recall': 0.8924731182795699, 'f1': 0.8783068783068784, 'number': 93} | 0.7792    | 0.8412 | 0.8090 | 0.9450   |
+| 0.0733        | 3.0   | 1542 | 0.2101          | {'precision': 0.7451403887688985, 'recall': 0.9078947368421053, 'f1': 0.8185053380782918, 'number': 380} | {'precision': 0.7922705314009661, 'recall': 0.9265536723163842, 'f1': 0.8541666666666666, 'number': 177} | {'precision': 0.7481840193704601, 'recall': 0.7974193548387096, 'f1': 0.7720174890693317, 'number': 775} | {'precision': 0.7676056338028169, 'recall': 0.8341836734693877, 'f1': 0.7995110024449877, 'number': 392} | {'precision': 0.9276315789473685, 'recall': 0.962457337883959, 'f1': 0.9447236180904522, 'number': 293}  | {'precision': 0.7941176470588235, 'recall': 0.8181818181818182, 'f1': 0.8059701492537314, 'number': 33} | {'precision': 0.9292035398230089, 'recall': 1.0, 'f1': 0.9633027522935781, 'number': 105}                | {'precision': 0.8695652173913043, 'recall': 0.8602150537634409, 'f1': 0.8648648648648649, 'number': 93} | 0.7903    | 0.8665 | 0.8266 | 0.9484   |
+| 0.0431        | 4.0   | 2056 | 0.2260          | {'precision': 0.8477157360406091, 'recall': 0.8789473684210526, 'f1': 0.8630490956072351, 'number': 380} | {'precision': 0.8190954773869347, 'recall': 0.9209039548022598, 'f1': 0.8670212765957446, 'number': 177} | {'precision': 0.7653562653562653, 'recall': 0.8038709677419354, 'f1': 0.7841409691629955, 'number': 775} | {'precision': 0.7897196261682243, 'recall': 0.8622448979591837, 'f1': 0.824390243902439, 'number': 392}  | {'precision': 0.9459459459459459, 'recall': 0.9556313993174061, 'f1': 0.9507640067911713, 'number': 293} | {'precision': 0.8055555555555556, 'recall': 0.8787878787878788, 'f1': 0.8405797101449276, 'number': 33} | {'precision': 0.9203539823008849, 'recall': 0.9904761904761905, 'f1': 0.9541284403669724, 'number': 105} | {'precision': 0.8842105263157894, 'recall': 0.9032258064516129, 'f1': 0.8936170212765957, 'number': 93} | 0.8232    | 0.8697 | 0.8458 | 0.9515   |
+| 0.0282        | 5.0   | 2570 | 0.2492          | {'precision': 0.835820895522388, 'recall': 0.8842105263157894, 'f1': 0.8593350383631714, 'number': 380}  | {'precision': 0.8333333333333334, 'recall': 0.9322033898305084, 'f1': 0.8800000000000001, 'number': 177} | {'precision': 0.7820197044334976, 'recall': 0.8193548387096774, 'f1': 0.800252047889099, 'number': 775}  | {'precision': 0.785377358490566, 'recall': 0.8494897959183674, 'f1': 0.8161764705882352, 'number': 392}  | {'precision': 0.9627118644067797, 'recall': 0.9692832764505119, 'f1': 0.9659863945578231, 'number': 293} | {'precision': 0.8235294117647058, 'recall': 0.8484848484848485, 'f1': 0.8358208955223881, 'number': 33} | {'precision': 0.9285714285714286, 'recall': 0.9904761904761905, 'f1': 0.9585253456221199, 'number': 105} | {'precision': 0.9120879120879121, 'recall': 0.8924731182795699, 'f1': 0.9021739130434783, 'number': 93} | 0.8311    | 0.8754 | 0.8527 | 0.9528   |
+| 0.0125        | 6.0   | 3084 | 0.2668          | {'precision': 0.830423940149626, 'recall': 0.8763157894736842, 'f1': 0.852752880921895, 'number': 380}   | {'precision': 0.839572192513369, 'recall': 0.8870056497175142, 'f1': 0.8626373626373628, 'number': 177}  | {'precision': 0.7724477244772447, 'recall': 0.8103225806451613, 'f1': 0.7909319899244331, 'number': 775} | {'precision': 0.7617977528089888, 'recall': 0.8647959183673469, 'f1': 0.8100358422939069, 'number': 392} | {'precision': 0.9726962457337884, 'recall': 0.9726962457337884, 'f1': 0.9726962457337884, 'number': 293} | {'precision': 0.875, 'recall': 0.8484848484848485, 'f1': 0.8615384615384615, 'number': 33}              | {'precision': 0.9279279279279279, 'recall': 0.9809523809523809, 'f1': 0.9537037037037037, 'number': 105} | {'precision': 0.8913043478260869, 'recall': 0.8817204301075269, 'f1': 0.8864864864864864, 'number': 93} | 0.8235    | 0.8697 | 0.8460 | 0.9529   |
+| 0.006         | 7.0   | 3598 | 0.2722          | {'precision': 0.8222222222222222, 'recall': 0.8763157894736842, 'f1': 0.8484076433121018, 'number': 380} | {'precision': 0.841025641025641, 'recall': 0.9265536723163842, 'f1': 0.8817204301075269, 'number': 177}  | {'precision': 0.7870485678704857, 'recall': 0.8154838709677419, 'f1': 0.8010139416983523, 'number': 775} | {'precision': 0.7793427230046949, 'recall': 0.8469387755102041, 'f1': 0.8117359413202934, 'number': 392} | {'precision': 0.9595959595959596, 'recall': 0.9726962457337884, 'f1': 0.9661016949152542, 'number': 293} | {'precision': 0.8787878787878788, 'recall': 0.8787878787878788, 'f1': 0.8787878787878788, 'number': 33} | {'precision': 0.9279279279279279, 'recall': 0.9809523809523809, 'f1': 0.9537037037037037, 'number': 105} | {'precision': 0.8913043478260869, 'recall': 0.8817204301075269, 'f1': 0.8864864864864864, 'number': 93} | 0.8298    | 0.8719 | 0.8503 | 0.9530   |
 
 
 ### Framework versions
 
-- Transformers 4.38.2
+- Transformers 4.39.3
 - Pytorch 2.2.1+cu121
 - Datasets 2.18.0
 - Tokenizers 0.15.2

config.json

@@ -57,7 +57,7 @@
   "pad_token_id": 0,
   "position_embedding_type": "absolute",
   "torch_dtype": "float32",
-  "transformers_version": "4.38.2",
+  "transformers_version": "4.39.3",
   "type_vocab_size": 2,
   "use_cache": true,
   "vocab_size": 28996

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3310998b3007e3241d50263db893b2075c1a6b9d2163cd275f90a5c2197a1552
+oid sha256:4c1555df63fbfc9a68284f288245e1bad452a0afc01f48ceb1b85d1c40ed2dcb
 size 430954348

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:80484f580dbe9de0dc0cb2a29c6199bae154c9e7ece1db865994094b9eb0da9c
+oid sha256:4f6a66be23cbad9274df4135c95e8fa448a45fc9866588bbc7a8ec3dafa7fbdb
 size 4856