|
--- |
|
base_model: microsoft/layoutlm-base-uncased |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- funsd |
|
model-index: |
|
- name: layoutlm-funsd |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# layoutlm-funsd |
|
|
|
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.6801 |
|
- Answer: {'precision': 0.6998916576381365, 'recall': 0.7985166872682324, 'f1': 0.745958429561201, 'number': 809} |
|
- Header: {'precision': 0.30158730158730157, 'recall': 0.31932773109243695, 'f1': 0.310204081632653, 'number': 119} |
|
- Question: {'precision': 0.7674216027874564, 'recall': 0.8272300469483568, 'f1': 0.7962042476276546, 'number': 1065} |
|
- Overall Precision: 0.7123 |
|
- Overall Recall: 0.7852 |
|
- Overall F1: 0.7470 |
|
- Overall Accuracy: 0.8102 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 3e-05 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 15 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| |
|
| 1.8269 | 1.0 | 10 | 1.5983 | {'precision': 0.0274949083503055, 'recall': 0.03337453646477132, 'f1': 0.03015075376884422, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.10761154855643044, 'recall': 0.07699530516431925, 'f1': 0.08976464148877941, 'number': 1065} | 0.0625 | 0.0547 | 0.0583 | 0.3800 | |
|
| 1.4969 | 2.0 | 20 | 1.3148 | {'precision': 0.14439140811455847, 'recall': 0.14956736711990112, 'f1': 0.14693381906496664, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.3501683501683502, 'recall': 0.39061032863849765, 'f1': 0.36928539724811366, 'number': 1065} | 0.2651 | 0.2694 | 0.2672 | 0.5643 | |
|
| 1.1839 | 3.0 | 30 | 1.0447 | {'precision': 0.40492957746478875, 'recall': 0.4264524103831891, 'f1': 0.41541240216736913, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5461672473867596, 'recall': 0.5887323943661972, 'f1': 0.5666516041572526, 'number': 1065} | 0.486 | 0.4877 | 0.4869 | 0.6655 | |
|
| 0.9261 | 4.0 | 40 | 0.8452 | {'precision': 0.5813715455475946, 'recall': 0.7021013597033374, 'f1': 0.6360582306830906, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.6362068965517241, 'recall': 0.6929577464788732, 'f1': 0.6633707865168539, 'number': 1065} | 0.6035 | 0.6553 | 0.6283 | 0.7335 | |
|
| 0.7296 | 5.0 | 50 | 0.7494 | {'precision': 0.6223849372384938, 'recall': 0.7354758961681088, 'f1': 0.6742209631728047, 'number': 809} | {'precision': 0.16666666666666666, 'recall': 0.06722689075630252, 'f1': 0.09580838323353293, 'number': 119} | {'precision': 0.6773356401384083, 'recall': 0.7352112676056338, 'f1': 0.7050877982890591, 'number': 1065} | 0.6417 | 0.6954 | 0.6675 | 0.7676 | |
|
| 0.6079 | 6.0 | 60 | 0.6998 | {'precision': 0.629399585921325, 'recall': 0.7515451174289246, 'f1': 0.6850704225352113, 'number': 809} | {'precision': 0.12222222222222222, 'recall': 0.09243697478991597, 'f1': 0.10526315789473684, 'number': 119} | {'precision': 0.6630686198920586, 'recall': 0.8075117370892019, 'f1': 0.7281964436917865, 'number': 1065} | 0.6286 | 0.7421 | 0.6806 | 0.7881 | |
|
| 0.5267 | 7.0 | 70 | 0.6732 | {'precision': 0.6366427840327533, 'recall': 0.7688504326328801, 'f1': 0.696528555431131, 'number': 809} | {'precision': 0.20652173913043478, 'recall': 0.15966386554621848, 'f1': 0.1800947867298578, 'number': 119} | {'precision': 0.7107296137339055, 'recall': 0.7774647887323943, 'f1': 0.7426008968609865, 'number': 1065} | 0.6576 | 0.7371 | 0.6951 | 0.7864 | |
|
| 0.4762 | 8.0 | 80 | 0.6606 | {'precision': 0.6577319587628866, 'recall': 0.788627935723115, 'f1': 0.71725688589095, 'number': 809} | {'precision': 0.25663716814159293, 'recall': 0.24369747899159663, 'f1': 0.25, 'number': 119} | {'precision': 0.731665228645384, 'recall': 0.7962441314553991, 'f1': 0.762589928057554, 'number': 1065} | 0.6757 | 0.7602 | 0.7155 | 0.7936 | |
|
| 0.4175 | 9.0 | 90 | 0.6566 | {'precision': 0.6815761448349308, 'recall': 0.7911001236093943, 'f1': 0.7322654462242563, 'number': 809} | {'precision': 0.25892857142857145, 'recall': 0.24369747899159663, 'f1': 0.2510822510822511, 'number': 119} | {'precision': 0.7513089005235603, 'recall': 0.8084507042253521, 'f1': 0.7788331071913162, 'number': 1065} | 0.6964 | 0.7677 | 0.7303 | 0.8021 | |
|
| 0.374 | 10.0 | 100 | 0.6592 | {'precision': 0.6956521739130435, 'recall': 0.7911001236093943, 'f1': 0.7403123192596877, 'number': 809} | {'precision': 0.319672131147541, 'recall': 0.3277310924369748, 'f1': 0.32365145228215775, 'number': 119} | {'precision': 0.7637931034482759, 'recall': 0.831924882629108, 'f1': 0.7964044943820225, 'number': 1065} | 0.7107 | 0.7852 | 0.7461 | 0.8070 | |
|
| 0.3406 | 11.0 | 110 | 0.6666 | {'precision': 0.7, 'recall': 0.796044499381953, 'f1': 0.7449392712550607, 'number': 809} | {'precision': 0.3305084745762712, 'recall': 0.3277310924369748, 'f1': 0.32911392405063294, 'number': 119} | {'precision': 0.7656387665198238, 'recall': 0.815962441314554, 'f1': 0.79, 'number': 1065} | 0.7142 | 0.7787 | 0.7451 | 0.8071 | |
|
| 0.332 | 12.0 | 120 | 0.6704 | {'precision': 0.6941798941798942, 'recall': 0.8108776266996292, 'f1': 0.7480045610034207, 'number': 809} | {'precision': 0.3220338983050847, 'recall': 0.31932773109243695, 'f1': 0.32067510548523204, 'number': 119} | {'precision': 0.7660311958405546, 'recall': 0.8300469483568075, 'f1': 0.7967552951780081, 'number': 1065} | 0.7118 | 0.7918 | 0.7496 | 0.8078 | |
|
| 0.3061 | 13.0 | 130 | 0.6787 | {'precision': 0.6908108108108109, 'recall': 0.7898640296662547, 'f1': 0.7370242214532873, 'number': 809} | {'precision': 0.3253968253968254, 'recall': 0.3445378151260504, 'f1': 0.33469387755102037, 'number': 119} | {'precision': 0.7669305189094108, 'recall': 0.8187793427230047, 'f1': 0.7920072661217076, 'number': 1065} | 0.7093 | 0.7787 | 0.7424 | 0.8091 | |
|
| 0.2879 | 14.0 | 140 | 0.6781 | {'precision': 0.6951871657754011, 'recall': 0.8034610630407911, 'f1': 0.7454128440366973, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.33613445378151263, 'f1': 0.33472803347280333, 'number': 119} | {'precision': 0.7746478873239436, 'recall': 0.8262910798122066, 'f1': 0.7996365288505224, 'number': 1065} | 0.7166 | 0.7878 | 0.7505 | 0.8099 | |
|
| 0.2831 | 15.0 | 150 | 0.6801 | {'precision': 0.6998916576381365, 'recall': 0.7985166872682324, 'f1': 0.745958429561201, 'number': 809} | {'precision': 0.30158730158730157, 'recall': 0.31932773109243695, 'f1': 0.310204081632653, 'number': 119} | {'precision': 0.7674216027874564, 'recall': 0.8272300469483568, 'f1': 0.7962042476276546, 'number': 1065} | 0.7123 | 0.7852 | 0.7470 | 0.8102 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.31.0 |
|
- Pytorch 2.0.1+cu118 |
|
- Datasets 2.14.2 |
|
- Tokenizers 0.13.3 |
|
|
