BERT-NER-Classifier

The BERT-NER-Classifier is a sophisticated model based on the bert-base-uncased architecture. It has been fine-tuned specifically for Named Entity Recognition (NER) using the CoNLL-2003 dataset, aiming to accurately identify entities such as persons, organizations, locations, and miscellaneous entities in text.

• Developed by: phanerozoic
• Model type: BertForTokenClassification
• Source model: bert-base-uncased
• License: cc-by-nc-4.0
• Languages: English

Model Details

The BERT-NER-Classifier uses a self-attention mechanism that differentiates the importance of each word in the context of others, tailored for NER tasks.

Configuration

• Attention probs dropout prob: 0.1
• Hidden act: gelu
• Hidden size: 768
• Number of attention heads: 12
• Number of hidden layers: 12

Training and Evaluation Data

The model utilizes the CoNLL-2003 dataset, which consists of texts annotated with named entities. This dataset is a standard benchmark for NER models.

Training Procedure

The model training was guided by an automated script designed to explore and identify the best hyperparameters for optimal performance. The script conducted extensive experimentation across the hyperparameter space, iteratively training and evaluating the model to pinpoint the most effective settings.

• Initial exploratory training: Using various combinations of epochs, batch sizes, and learning rates.
• Refinement and focused training: Upon identifying the best performing hyperparameters, the model underwent further training three additional times to ensure stability and consistency in performance.

Optimal Hyperparameters Identified

• Epochs: 1
• Batch size: 16
• Learning rate: 3e-5

Performance

The refined training approach resulted in a model with robust predictive capabilities:

• Validation Precision: 0.9358
• Validation Recall: 0.9271
• Validation F1 Score: 0.9311

Usage

This model is highly effective for identifying named entities in English texts, particularly in contexts similar to the CoNLL-2003 dataset upon which the model was trained.

Limitations

While the model excels in contexts similar to its training data (CoNLL-2003), its performance might vary on text from other domains or other languages. Future enhancements could involve expanding the training data to include more diverse text sources.

Acknowledgments

Thanks to the developers of the BERT architecture and the Hugging Face team. The tools and frameworks provided were instrumental in the development of this model.