layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7279
• Answer: {'precision': 0.706858407079646, 'recall': 0.7898640296662547, 'f1': 0.7460595446584939, 'number': 809}
• Header: {'precision': 0.312, 'recall': 0.3277310924369748, 'f1': 0.31967213114754095, 'number': 119}
• Question: {'precision': 0.7741935483870968, 'recall': 0.8338028169014085, 'f1': 0.8028933092224232, 'number': 1065}
• Overall Precision: 0.7197
• Overall Recall: 0.7858
• Overall F1: 0.7513
• Overall Accuracy: 0.8034

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7555	1.0	10	1.5320	{'precision': 0.02317596566523605, 'recall': 0.03337453646477132, 'f1': 0.02735562310030395, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.11706783369803063, 'recall': 0.10046948356807511, 'f1': 0.10813542193026779, 'number': 1065}	0.0645	0.0672	0.0658	0.3975
1.3972	2.0	20	1.1941	{'precision': 0.2606060606060606, 'recall': 0.2657601977750309, 'f1': 0.2631578947368421, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4398805078416729, 'recall': 0.5530516431924882, 'f1': 0.49001663893510816, 'number': 1065}	0.3715	0.4034	0.3868	0.6107
1.0502	3.0	30	0.9315	{'precision': 0.5343347639484979, 'recall': 0.61557478368356, 'f1': 0.5720850086157381, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5706304868316041, 'recall': 0.6713615023474179, 'f1': 0.6169111302847283, 'number': 1065}	0.5484	0.6086	0.5769	0.7259
0.8107	4.0	40	0.7975	{'precision': 0.6146288209606987, 'recall': 0.695920889987639, 'f1': 0.6527536231884058, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.6344605475040258, 'recall': 0.739906103286385, 'f1': 0.683138274815778, 'number': 1065}	0.6124	0.6779	0.6435	0.7593
0.6553	5.0	50	0.7487	{'precision': 0.6507760532150776, 'recall': 0.7255871446229913, 'f1': 0.6861484511981296, 'number': 809}	{'precision': 0.12, 'recall': 0.07563025210084033, 'f1': 0.09278350515463916, 'number': 119}	{'precision': 0.6690085870413739, 'recall': 0.8046948356807512, 'f1': 0.7306052855924979, 'number': 1065}	0.6435	0.7291	0.6836	0.7719
0.5642	6.0	60	0.7147	{'precision': 0.6557203389830508, 'recall': 0.765142150803461, 'f1': 0.7062179121505989, 'number': 809}	{'precision': 0.20833333333333334, 'recall': 0.12605042016806722, 'f1': 0.15706806282722513, 'number': 119}	{'precision': 0.7058333333333333, 'recall': 0.7953051643192488, 'f1': 0.7479028697571745, 'number': 1065}	0.6683	0.7431	0.7037	0.7847
0.4833	7.0	70	0.6895	{'precision': 0.6919691969196919, 'recall': 0.7775030902348579, 'f1': 0.7322467986030267, 'number': 809}	{'precision': 0.25742574257425743, 'recall': 0.2184873949579832, 'f1': 0.23636363636363636, 'number': 119}	{'precision': 0.7268041237113402, 'recall': 0.7943661971830986, 'f1': 0.7590847913862719, 'number': 1065}	0.6904	0.7531	0.7204	0.7920
0.4356	8.0	80	0.6896	{'precision': 0.6869109947643979, 'recall': 0.8108776266996292, 'f1': 0.7437641723356009, 'number': 809}	{'precision': 0.27884615384615385, 'recall': 0.24369747899159663, 'f1': 0.2600896860986547, 'number': 119}	{'precision': 0.730185497470489, 'recall': 0.8131455399061033, 'f1': 0.7694358063083073, 'number': 1065}	0.6909	0.7782	0.7319	0.7919
0.3874	9.0	90	0.7000	{'precision': 0.7158836689038032, 'recall': 0.7911001236093943, 'f1': 0.7516147974163241, 'number': 809}	{'precision': 0.2818181818181818, 'recall': 0.2605042016806723, 'f1': 0.27074235807860264, 'number': 119}	{'precision': 0.7395388556789069, 'recall': 0.8131455399061033, 'f1': 0.7745974955277279, 'number': 1065}	0.7067	0.7712	0.7375	0.7957
0.3771	10.0	100	0.7200	{'precision': 0.7097142857142857, 'recall': 0.7676143386897404, 'f1': 0.7375296912114014, 'number': 809}	{'precision': 0.272, 'recall': 0.2857142857142857, 'f1': 0.27868852459016397, 'number': 119}	{'precision': 0.7438715131022823, 'recall': 0.8262910798122066, 'f1': 0.7829181494661922, 'number': 1065}	0.7032	0.7702	0.7352	0.7886
0.3231	11.0	110	0.7101	{'precision': 0.7115177610333692, 'recall': 0.8170580964153276, 'f1': 0.760644418872267, 'number': 809}	{'precision': 0.29133858267716534, 'recall': 0.31092436974789917, 'f1': 0.3008130081300813, 'number': 119}	{'precision': 0.7672188317349607, 'recall': 0.8262910798122066, 'f1': 0.7956600361663653, 'number': 1065}	0.7163	0.7918	0.7521	0.8000
0.3026	12.0	120	0.7186	{'precision': 0.7138009049773756, 'recall': 0.7799752781211372, 'f1': 0.7454223272297696, 'number': 809}	{'precision': 0.325, 'recall': 0.3277310924369748, 'f1': 0.3263598326359833, 'number': 119}	{'precision': 0.7658833768494343, 'recall': 0.8262910798122066, 'f1': 0.7949412827461607, 'number': 1065}	0.7199	0.7777	0.7477	0.7984
0.2876	13.0	130	0.7213	{'precision': 0.7156862745098039, 'recall': 0.8121137206427689, 'f1': 0.7608569774174871, 'number': 809}	{'precision': 0.325, 'recall': 0.3277310924369748, 'f1': 0.3263598326359833, 'number': 119}	{'precision': 0.7737676056338029, 'recall': 0.8253521126760563, 'f1': 0.7987278509768287, 'number': 1065}	0.7245	0.7903	0.7559	0.8025
0.2709	14.0	140	0.7269	{'precision': 0.7147613762486127, 'recall': 0.796044499381953, 'f1': 0.7532163742690058, 'number': 809}	{'precision': 0.30158730158730157, 'recall': 0.31932773109243695, 'f1': 0.310204081632653, 'number': 119}	{'precision': 0.7709059233449478, 'recall': 0.8309859154929577, 'f1': 0.7998192498870312, 'number': 1065}	0.7205	0.7863	0.7519	0.8033
0.2745	15.0	150	0.7279	{'precision': 0.706858407079646, 'recall': 0.7898640296662547, 'f1': 0.7460595446584939, 'number': 809}	{'precision': 0.312, 'recall': 0.3277310924369748, 'f1': 0.31967213114754095, 'number': 119}	{'precision': 0.7741935483870968, 'recall': 0.8338028169014085, 'f1': 0.8028933092224232, 'number': 1065}	0.7197	0.7858	0.7513	0.8034

Framework versions

• Transformers 4.42.4
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1