---
base_model: BAAI/bge-base-en-v1.5
datasets: []
language: []
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:6300
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: As of January 31, 2023, the weighted average remaining lease term
for operating leases was 7 years and for finance leases was 3 years.
sentences:
- What was the Company's net deferred tax assets as of December 30, 2023, and December
31, 2022?
- What were the weighted average remaining lease terms for operating and finance
leases as of January 31, 2023?
- How much did the net investment income change from 2021 to 2023?
- source_sentence: The 4.500% notes due in August 2034 have an interest rate of 4.55%.
sentences:
- What types of insurance coverage does the company provide to its employees at
no premium cost, as part of their general employee benefits package?
- What is the interest rate for the 4.500% notes due in August 2034?
- How much did the company's revenues decrease in 2023 compared to 2022?
- source_sentence: In 2023, other income (expense), net included $376 million of interest
income, partially offset by $167 million of net unrealized losses on equity investments.
Other income (expense), net in 2022 included $657 million of net unrealized losses
on equity investments, partially offset by $106 million of interest income.
sentences:
- What contributed to the net other income (expense) in 2023?
- What types of products does the Canada operation offer?
- What was the net change in cash and cash equivalents in 2022?
- source_sentence: We believe the claims in these cases are without merit and are
vigorously defending these lawsuits.
sentences:
- Where in the Annual Report can one find a description of certain legal matters
and their impact on the company?
- What is the goal of the company regarding its global corporate operations by 2030?
- What is the stance of the defending airlines on the claims made against them in
the capacity antitrust litigation?
- source_sentence: North America's total net revenues for the fiscal year ended October
1, 2023, were $26,569.6 million.
sentences:
- What was the total net revenue for North America in fiscal 2023?
- What are the consequences of impermissible use or disclosure of PHI according
to the HITECH Act?
- What does the index in a financial report indicate?
model-index:
- name: SentenceTransformer based on BAAI/bge-base-en-v1.5
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 768
type: dim_768
metrics:
- type: cosine_accuracy@1
value: 0.6171428571428571
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.7457142857142857
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.8114285714285714
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.8585714285714285
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.6171428571428571
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.24857142857142858
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.16228571428571428
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.08585714285714285
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.6171428571428571
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.7457142857142857
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.8114285714285714
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.8585714285714285
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.7357204832416036
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.6965260770975052
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.7015509951793545
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 512
type: dim_512
metrics:
- type: cosine_accuracy@1
value: 0.6214285714285714
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.74
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.8
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.8642857142857143
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.6214285714285714
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.24666666666666665
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.15999999999999998
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.08642857142857142
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.6214285714285714
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.74
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.8
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.8642857142857143
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.738181682287809
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.6983236961451246
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.7027820040111107
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 256
type: dim_256
metrics:
- type: cosine_accuracy@1
value: 0.6
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.7271428571428571
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.7928571428571428
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.8442857142857143
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.6
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.24238095238095236
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.15857142857142856
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.08442857142857142
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.6
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.7271428571428571
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.7928571428571428
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.8442857142857143
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.7182448637999702
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.6782879818594099
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.683606591058064
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 128
type: dim_128
metrics:
- type: cosine_accuracy@1
value: 0.5728571428571428
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.7014285714285714
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.7557142857142857
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.8157142857142857
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.5728571428571428
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.2338095238095238
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.1511428571428571
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.08157142857142856
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.5728571428571428
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.7014285714285714
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.7557142857142857
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.8157142857142857
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.6915163160852085
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.6521536281179136
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.6580414471513885
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 64
type: dim_64
metrics:
- type: cosine_accuracy@1
value: 0.5142857142857142
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.6371428571428571
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.6728571428571428
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.7357142857142858
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.5142857142857142
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.21238095238095234
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.13457142857142856
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.07357142857142857
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.5142857142857142
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.6371428571428571
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.6728571428571428
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.7357142857142858
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.6197107516374883
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.5832369614512468
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.5907376271746598
name: Cosine Map@100
---
# SentenceTransformer based on BAAI/bge-base-en-v1.5
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5)
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("ethan-ky/bge-base-financial-matryoshka")
# Run inference
sentences = [
"North America's total net revenues for the fiscal year ended October 1, 2023, were $26,569.6 million.",
'What was the total net revenue for North America in fiscal 2023?',
'What are the consequences of impermissible use or disclosure of PHI according to the HITECH Act?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
## Evaluation
### Metrics
#### Information Retrieval
* Dataset: `dim_768`
* Evaluated with [InformationRetrievalEvaluator
](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.6171 |
| cosine_accuracy@3 | 0.7457 |
| cosine_accuracy@5 | 0.8114 |
| cosine_accuracy@10 | 0.8586 |
| cosine_precision@1 | 0.6171 |
| cosine_precision@3 | 0.2486 |
| cosine_precision@5 | 0.1623 |
| cosine_precision@10 | 0.0859 |
| cosine_recall@1 | 0.6171 |
| cosine_recall@3 | 0.7457 |
| cosine_recall@5 | 0.8114 |
| cosine_recall@10 | 0.8586 |
| cosine_ndcg@10 | 0.7357 |
| cosine_mrr@10 | 0.6965 |
| **cosine_map@100** | **0.7016** |
#### Information Retrieval
* Dataset: `dim_512`
* Evaluated with [InformationRetrievalEvaluator
](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.6214 |
| cosine_accuracy@3 | 0.74 |
| cosine_accuracy@5 | 0.8 |
| cosine_accuracy@10 | 0.8643 |
| cosine_precision@1 | 0.6214 |
| cosine_precision@3 | 0.2467 |
| cosine_precision@5 | 0.16 |
| cosine_precision@10 | 0.0864 |
| cosine_recall@1 | 0.6214 |
| cosine_recall@3 | 0.74 |
| cosine_recall@5 | 0.8 |
| cosine_recall@10 | 0.8643 |
| cosine_ndcg@10 | 0.7382 |
| cosine_mrr@10 | 0.6983 |
| **cosine_map@100** | **0.7028** |
#### Information Retrieval
* Dataset: `dim_256`
* Evaluated with [InformationRetrievalEvaluator
](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.6 |
| cosine_accuracy@3 | 0.7271 |
| cosine_accuracy@5 | 0.7929 |
| cosine_accuracy@10 | 0.8443 |
| cosine_precision@1 | 0.6 |
| cosine_precision@3 | 0.2424 |
| cosine_precision@5 | 0.1586 |
| cosine_precision@10 | 0.0844 |
| cosine_recall@1 | 0.6 |
| cosine_recall@3 | 0.7271 |
| cosine_recall@5 | 0.7929 |
| cosine_recall@10 | 0.8443 |
| cosine_ndcg@10 | 0.7182 |
| cosine_mrr@10 | 0.6783 |
| **cosine_map@100** | **0.6836** |
#### Information Retrieval
* Dataset: `dim_128`
* Evaluated with [InformationRetrievalEvaluator
](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:----------|
| cosine_accuracy@1 | 0.5729 |
| cosine_accuracy@3 | 0.7014 |
| cosine_accuracy@5 | 0.7557 |
| cosine_accuracy@10 | 0.8157 |
| cosine_precision@1 | 0.5729 |
| cosine_precision@3 | 0.2338 |
| cosine_precision@5 | 0.1511 |
| cosine_precision@10 | 0.0816 |
| cosine_recall@1 | 0.5729 |
| cosine_recall@3 | 0.7014 |
| cosine_recall@5 | 0.7557 |
| cosine_recall@10 | 0.8157 |
| cosine_ndcg@10 | 0.6915 |
| cosine_mrr@10 | 0.6522 |
| **cosine_map@100** | **0.658** |
#### Information Retrieval
* Dataset: `dim_64`
* Evaluated with [InformationRetrievalEvaluator
](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.5143 |
| cosine_accuracy@3 | 0.6371 |
| cosine_accuracy@5 | 0.6729 |
| cosine_accuracy@10 | 0.7357 |
| cosine_precision@1 | 0.5143 |
| cosine_precision@3 | 0.2124 |
| cosine_precision@5 | 0.1346 |
| cosine_precision@10 | 0.0736 |
| cosine_recall@1 | 0.5143 |
| cosine_recall@3 | 0.6371 |
| cosine_recall@5 | 0.6729 |
| cosine_recall@10 | 0.7357 |
| cosine_ndcg@10 | 0.6197 |
| cosine_mrr@10 | 0.5832 |
| **cosine_map@100** | **0.5907** |
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 6,300 training samples
* Columns: positive
and anchor
* Approximate statistics based on the first 1000 samples:
| | positive | anchor |
|:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
| type | string | string |
| details |
Our ability to develop and operate units at the right locations and to deliver a customer-centric omni-channel experience largely determines our competitive position within the retail industry. We believe price leadership is a critical part of our business model and we continue to focus on moving our markets towards an EDLP approach. Additionally, our ability to operate food departments effectively has a significant impact on our competitive position in the markets where we operate.
| What factors contribute to Walmart International's competitive position?
|
| tax annual aggregate losses incurred in any year from U.S. hurricane events could be in excess of $3,827 million (or 6.4 percent of total Chubb shareholders’ equity at December 31, 2023).
| What is the expected maximum potential loss from hurricane events for Chubb as of the end of 2023?
|
| The 'Glossary of Terms and Acronyms’ is included on pages 315-321.
| What is included on pages 315 to 321 of the document?
|
* Loss: [MatryoshkaLoss
](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
```json
{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
768,
512,
256,
128,
64
],
"matryoshka_weights": [
1,
1,
1,
1,
1
],
"n_dims_per_step": -1
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: epoch
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 4
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `bf16`: True
- `tf32`: True
- `load_best_model_at_end`: True
- `optim`: adamw_torch_fused
- `batch_sampler`: no_duplicates
#### All Hyperparameters