End of training

README.md

@@ -0,0 +1,80 @@
+---
+license: mit
+base_model: microsoft/layoutlm-base-uncased
+tags:
+- generated_from_trainer
+datasets:
+- funsd
+model-index:
+- name: layoutlm-funsd
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# layoutlm-funsd
+
+This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.1516
+- Answer: {'precision': 0.38400702987697716, 'recall': 0.5401730531520396, 'f1': 0.44889573703133023, 'number': 809}
+- Header: {'precision': 0.3218390804597701, 'recall': 0.23529411764705882, 'f1': 0.27184466019417475, 'number': 119}
+- Question: {'precision': 0.5132192846034215, 'recall': 0.6197183098591549, 'f1': 0.5614632071458954, 'number': 1065}
+- Overall Precision: 0.4480
+- Overall Recall: 0.5645
+- Overall F1: 0.4996
+- Overall Accuracy: 0.6209
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 3e-05
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 15
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Answer                                                                                                      | Header                                                                                                      | Question                                                                                                    | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
+| 1.7219        | 1.0   | 10   | 1.5555          | {'precision': 0.04431137724550898, 'recall': 0.04573547589616811, 'f1': 0.04501216545012165, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                 | {'precision': 0.26320501342882724, 'recall': 0.27605633802816903, 'f1': 0.2694775435380385, 'number': 1065} | 0.1696            | 0.1661         | 0.1678     | 0.3589           |
+| 1.4917        | 2.0   | 20   | 1.3323          | {'precision': 0.17622377622377622, 'recall': 0.311495673671199, 'f1': 0.2251004912907548, 'number': 809}    | {'precision': 0.1, 'recall': 0.008403361344537815, 'f1': 0.015503875968992248, 'number': 119}               | {'precision': 0.29382407985028075, 'recall': 0.4422535211267606, 'f1': 0.3530734632683658, 'number': 1065}  | 0.2379            | 0.3633         | 0.2875     | 0.4413           |
+| 1.2799        | 3.0   | 30   | 1.2482          | {'precision': 0.24236517218973358, 'recall': 0.4610630407911001, 'f1': 0.317717206132879, 'number': 809}    | {'precision': 0.273972602739726, 'recall': 0.16806722689075632, 'f1': 0.20833333333333331, 'number': 119}   | {'precision': 0.35655737704918034, 'recall': 0.4084507042253521, 'f1': 0.38074398249452956, 'number': 1065} | 0.2924            | 0.4155         | 0.3432     | 0.4580           |
+| 1.1477        | 4.0   | 40   | 1.1758          | {'precision': 0.2900516795865633, 'recall': 0.5550061804697157, 'f1': 0.38099278744166315, 'number': 809}   | {'precision': 0.3559322033898305, 'recall': 0.17647058823529413, 'f1': 0.2359550561797753, 'number': 119}   | {'precision': 0.4393939393939394, 'recall': 0.49014084507042255, 'f1': 0.46338215712383485, 'number': 1065} | 0.3549            | 0.4977         | 0.4144     | 0.5219           |
+| 1.0484        | 5.0   | 50   | 1.0885          | {'precision': 0.3271441202475685, 'recall': 0.4573547589616811, 'f1': 0.3814432989690722, 'number': 809}    | {'precision': 0.2826086956521739, 'recall': 0.2184873949579832, 'f1': 0.24644549763033172, 'number': 119}   | {'precision': 0.4808, 'recall': 0.564319248826291, 'f1': 0.5192224622030237, 'number': 1065}                | 0.4032            | 0.5003         | 0.4465     | 0.5827           |
+| 0.9672        | 6.0   | 60   | 1.0745          | {'precision': 0.30431309904153353, 'recall': 0.47095179233621753, 'f1': 0.36972343522561857, 'number': 809} | {'precision': 0.34782608695652173, 'recall': 0.20168067226890757, 'f1': 0.25531914893617025, 'number': 119} | {'precision': 0.43936243936243935, 'recall': 0.5953051643192488, 'f1': 0.5055821371610846, 'number': 1065}  | 0.3759            | 0.5213         | 0.4368     | 0.5916           |
+| 0.8787        | 7.0   | 70   | 1.1863          | {'precision': 0.3697033898305085, 'recall': 0.43139678615574784, 'f1': 0.3981745579007416, 'number': 809}   | {'precision': 0.25, 'recall': 0.2184873949579832, 'f1': 0.23318385650224216, 'number': 119}                 | {'precision': 0.4801556420233463, 'recall': 0.5793427230046948, 'f1': 0.5251063829787234, 'number': 1065}   | 0.4252            | 0.4977         | 0.4586     | 0.5870           |
+| 0.8501        | 8.0   | 80   | 1.1043          | {'precision': 0.31553860819828405, 'recall': 0.40914709517923364, 'f1': 0.3562970936490851, 'number': 809}  | {'precision': 0.3484848484848485, 'recall': 0.19327731092436976, 'f1': 0.24864864864864866, 'number': 119}  | {'precision': 0.41997593261131166, 'recall': 0.6553990610328638, 'f1': 0.5119178584525119, 'number': 1065}  | 0.3788            | 0.5278         | 0.4411     | 0.5878           |
+| 0.805         | 9.0   | 90   | 1.0872          | {'precision': 0.3356828193832599, 'recall': 0.47095179233621753, 'f1': 0.39197530864197533, 'number': 809}  | {'precision': 0.32894736842105265, 'recall': 0.21008403361344538, 'f1': 0.25641025641025644, 'number': 119} | {'precision': 0.45454545454545453, 'recall': 0.6197183098591549, 'f1': 0.5244338498212157, 'number': 1065}  | 0.4003            | 0.5349         | 0.4579     | 0.6053           |
+| 0.7686        | 10.0  | 100  | 1.1006          | {'precision': 0.35418427726120033, 'recall': 0.5179233621755254, 'f1': 0.42068273092369474, 'number': 809}  | {'precision': 0.3333333333333333, 'recall': 0.2184873949579832, 'f1': 0.2639593908629441, 'number': 119}    | {'precision': 0.49634443541835904, 'recall': 0.5737089201877934, 'f1': 0.5322299651567944, 'number': 1065}  | 0.4238            | 0.5299         | 0.4709     | 0.6028           |
+| 0.7078        | 11.0  | 110  | 1.1631          | {'precision': 0.38475665748393023, 'recall': 0.5179233621755254, 'f1': 0.4415173867228662, 'number': 809}   | {'precision': 0.28846153846153844, 'recall': 0.25210084033613445, 'f1': 0.26905829596412556, 'number': 119} | {'precision': 0.520764119601329, 'recall': 0.5887323943661972, 'f1': 0.5526663728514765, 'number': 1065}    | 0.4489            | 0.5399         | 0.4902     | 0.6064           |
+| 0.7162        | 12.0  | 120  | 1.1517          | {'precision': 0.36400817995910023, 'recall': 0.4400494437577256, 'f1': 0.3984331281477337, 'number': 809}   | {'precision': 0.28421052631578947, 'recall': 0.226890756302521, 'f1': 0.25233644859813087, 'number': 119}   | {'precision': 0.4661458333333333, 'recall': 0.672300469483568, 'f1': 0.550557477893118, 'number': 1065}     | 0.4212            | 0.5514         | 0.4776     | 0.6014           |
+| 0.6912        | 13.0  | 130  | 1.2013          | {'precision': 0.3880718954248366, 'recall': 0.5871446229913473, 'f1': 0.4672897196261682, 'number': 809}    | {'precision': 0.3888888888888889, 'recall': 0.23529411764705882, 'f1': 0.2931937172774869, 'number': 119}   | {'precision': 0.5526552655265526, 'recall': 0.5765258215962441, 'f1': 0.5643382352941176, 'number': 1065}   | 0.4641            | 0.5605         | 0.5077     | 0.6082           |
+| 0.664         | 14.0  | 140  | 1.1337          | {'precision': 0.37344028520499106, 'recall': 0.5179233621755254, 'f1': 0.4339720352149145, 'number': 809}   | {'precision': 0.3218390804597701, 'recall': 0.23529411764705882, 'f1': 0.27184466019417475, 'number': 119}  | {'precision': 0.5037650602409639, 'recall': 0.6281690140845071, 'f1': 0.5591307981613038, 'number': 1065}   | 0.4399            | 0.5600         | 0.4927     | 0.6142           |
+| 0.6496        | 15.0  | 150  | 1.1516          | {'precision': 0.38400702987697716, 'recall': 0.5401730531520396, 'f1': 0.44889573703133023, 'number': 809}  | {'precision': 0.3218390804597701, 'recall': 0.23529411764705882, 'f1': 0.27184466019417475, 'number': 119}  | {'precision': 0.5132192846034215, 'recall': 0.6197183098591549, 'f1': 0.5614632071458954, 'number': 1065}   | 0.4480            | 0.5645         | 0.4996     | 0.6209           |
+
+
+### Framework versions
+
+- Transformers 4.37.2
+- Pytorch 2.1.0+cu121
+- Datasets 2.17.1
+- Tokenizers 0.15.2

logs/events.out.tfevents.1708926945.0c0388cb45f4.9156.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0bc64054ade4ad7ab6f34cca2fdf02df0b2686d5f9be38343e13e5e8830ac497
-size 13715
+oid sha256:e6a012274f4d6a9fdfb56b6996e442c0067c5e55e979d78908df549929a1b32e
+size 14730

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6de008b4d69a3a51fad35a4a5f7924e1374386751157db654b5a98e1096d2c83
+oid sha256:80f253c24281ffb92e748d74aa5c5063ed783298e1034d14f2b6200eb9880d48
 size 450558212

preprocessor_config.json

@@ -0,0 +1,13 @@
+{
+  "apply_ocr": true,
+  "do_resize": true,
+  "image_processor_type": "LayoutLMv2ImageProcessor",
+  "ocr_lang": null,
+  "processor_class": "LayoutLMv2Processor",
+  "resample": 2,
+  "size": {
+    "height": 224,
+    "width": 224
+  },
+  "tesseract_config": ""
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,80 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "apply_ocr": false,
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "cls_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "only_label_first_subword": true,
+  "pad_token": "[PAD]",
+  "pad_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "pad_token_label": -100,
+  "processor_class": "LayoutLMv2Processor",
+  "sep_token": "[SEP]",
+  "sep_token_box": [
+    1000,
+    1000,
+    1000,
+    1000
+  ],
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "LayoutLMv2Tokenizer",
+  "unk_token": "[UNK]"
+}