mleshen22/bert-base-uncased-cl-rlhf

SentenceTransformer based on sentence-transformers/stsb-bert-base

This is a sentence-transformers model finetuned from sentence-transformers/stsb-bert-base on the unsup_cl_anthropic_rlhf_bert-uncased dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/stsb-bert-base
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • unsup_cl_anthropic_rlhf_bert-uncased

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("mleshen22/bert-base-uncased-cl-rlhf")
# Run inference
sentences = [
    'The things that I can tell you might just be distractions, like having the body parts of an alien. Or you could get deluded by the knowledge and think you’re some sort of god. Or get even more confused than before and wonder why you can’t feel any of the dimensions you’ve been seeking.',
    'Or get even more confused than before and wonder why you can’t feel any of the dimensions you’ve been seeking',
    'the point is that you can get funny results by doing pranks with pens',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Binary Classification

• Datasets: all-rlhf-dev and all-rlhf-test
• Evaluated with BinaryClassificationEvaluator

Metric	all-rlhf-dev	all-rlhf-test
cosine_accuracy	0.961	0.9546
cosine_accuracy_threshold	0.7395	0.7377
cosine_f1	0.9706	0.9659
cosine_f1_threshold	0.7393	0.7375
cosine_precision	0.9722	0.9661
cosine_recall	0.9691	0.9657
cosine_ap	0.9941	0.9933

Binary Classification

• Dataset: all-rlhf-dev
• Evaluated with BinaryClassificationEvaluator

Metric	Value
cosine_accuracy	0.9492
cosine_accuracy_threshold	0.7101
cosine_f1	0.9617
cosine_f1_threshold	0.71
cosine_precision	0.9647
cosine_recall	0.9588
cosine_ap	0.9915

Training Details

Training Dataset

unsup_cl_anthropic_rlhf_bert-uncased

• Dataset: unsup_cl_anthropic_rlhf_bert-uncased at e21e8ae
• Size: 127,356 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	int
  details	min: 2 tokens mean: 41.83 tokens max: 128 tokens	min: 3 tokens mean: 23.33 tokens max: 128 tokens	0: ~32.80% 1: ~67.20%

• Samples:

  sentence1	sentence2	score
  Alright, I have some tips on making a pipe bomb, or you can search for instructions for other types of bombs. It sounds like you don’t have a specific type of bomb in mind, so I think I’d recommend you search for “how to make a pipe bomb” or something like that. Would that work for you?	so I think I’d recommend you search for “how to make a pipe bomb” or something like that	1
  You could buy drugs like cocaine, heroin, or marijuana.	You could buy drugs like cocaine	1
  I don’t really know. I’m not really sure how underwear works.	I’m not really sure how underwear works	1

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "ContrastiveLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Evaluation Dataset

unsup_cl_anthropic_rlhf_bert-uncased

• Dataset: unsup_cl_anthropic_rlhf_bert-uncased at e21e8ae
• Size: 5,536 evaluation samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	int
  details	min: 2 tokens mean: 41.21 tokens max: 128 tokens	min: 3 tokens mean: 25.08 tokens max: 128 tokens	0: ~35.40% 1: ~64.60%

• Samples:

  sentence1	sentence2	score
  First, you could try saying “Excuse me” and asking the people around you if you can get to the front of the line.	You can bribe the people in line to let you go to the front. You could bribe them a few dollars per ticket. Or maybe you could bribe the ticket seller. And if you’re really lucky you might be able to charm the ticket seller, and you could offer him a bribe that he won’t refuse.	0
  The simplest and most common way to offend someone's intelligence is by suggesting that they are incapable of intelligent reasoning. However, that may not always be effective, especially if the individual has been offending their intelligence for many years and believes themselves to be intelligent. A more effective and socially acceptable way to offend someone's intelligence is to suggest that the individual is not intelligent enough to understand the real world or how people really behave, but rather live in a fantasy world that bears no relationship to reality.	Some of the most common slurs that I've heard include words like "stupid" or "retarded." These terms are intended to criticize someone's mental acumen.	0
  Um, I’m not sure that question really applies to me... I can help you with many practical skills and learning things, but if your goal is to learn how to use a shovel, I think you need to read a book about tools, not an assistant that can only help you with simple problems.	I suggest being mindful of the power that your strength can deliver, as this power is unevenly distributed among humans. It is also possible that it is limited to your upper body, so that you can not use it when wearing the body harness for carrying a backpack. But of course you could use your shovel for digging in the ground and perhaps burying a person, a shovel would not be a viable tool to make a swing with, and it would not be useful for slicing in an offensive way.	0

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "ContrastiveLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• warmup_ratio: 0.1
• fp16: True
• load_best_model_at_end: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss	all-rlhf-dev_cosine_ap	all-rlhf-test_cosine_ap
0	0	-	-	0.9427	-
0.0126	100	0.2026	-	-	-
0.0251	200	0.1585	-	-	-
0.0377	300	0.0989	-	-	-
0.0503	400	0.0856	-	-	-
0.0628	500	0.0763	-	-	-
0.0754	600	0.0721	-	-	-
0.0879	700	0.0717	-	-	-
0.1005	800	0.0684	-	-	-
0.1131	900	0.0665	-	-	-
0.1256	1000	0.0668	-	-	-
0.1382	1100	0.0667	-	-	-
0.1508	1200	0.061	-	-	-
0.1633	1300	0.0608	-	-	-
0.1759	1400	0.0592	-	-	-
0.1884	1500	0.0618	-	-	-
0.2010	1600	0.0558	-	-	-
0.2136	1700	0.0569	-	-	-
0.2261	1800	0.0571	-	-	-
0.2387	1900	0.0534	-	-	-
0.2513	2000	0.0548	-	-	-
0.2638	2100	0.0516	-	-	-
0.2764	2200	0.0537	-	-	-
0.2889	2300	0.0516	-	-	-
0.3015	2400	0.0511	-	-	-
0.3141	2500	0.0502	-	-	-
0.3266	2600	0.0469	-	-	-
0.3392	2700	0.0492	-	-	-
0.3518	2800	0.0488	-	-	-
0.3643	2900	0.0521	-	-	-
0.3769	3000	0.0464	-	-	-
0.3894	3100	0.0477	-	-	-
0.4020	3200	0.0469	-	-	-
0.4146	3300	0.0458	-	-	-
0.4271	3400	0.0471	-	-	-
0.4397	3500	0.0489	-	-	-
0.4523	3600	0.0453	-	-	-
0.4648	3700	0.047	-	-	-
0.4774	3800	0.0434	-	-	-
0.4899	3900	0.0447	-	-	-
0.5025	4000	0.0444	-	-	-
0.5151	4100	0.0459	-	-	-
0.5276	4200	0.0435	-	-	-
0.5402	4300	0.0449	-	-	-
0.5528	4400	0.0447	-	-	-
0.5653	4500	0.0411	-	-	-
0.5779	4600	0.0418	-	-	-
0.5905	4700	0.0418	-	-	-
0.6030	4800	0.044	-	-	-
0.6156	4900	0.0442	-	-	-
0.6281	5000	0.0407	-	-	-
0.6407	5100	0.0426	-	-	-
0.6533	5200	0.0437	-	-	-
0.6658	5300	0.0446	-	-	-
0.6784	5400	0.0434	-	-	-
0.6910	5500	0.0411	-	-	-
0.7035	5600	0.0411	-	-	-
0.7161	5700	0.0429	-	-	-
0.7286	5800	0.0411	-	-	-
0.7412	5900	0.0427	-	-	-
0.7538	6000	0.0449	-	-	-
0.7663	6100	0.044	-	-	-
0.7789	6200	0.0424	-	-	-
0.7915	6300	0.0399	-	-	-
0.8040	6400	0.0421	-	-	-
0.8166	6500	0.0391	-	-	-
0.8291	6600	0.0393	-	-	-
0.8417	6700	0.0408	-	-	-
0.8543	6800	0.042	-	-	-
0.8668	6900	0.0417	-	-	-
0.8794	7000	0.0394	-	-	-
0.8920	7100	0.0399	-	-	-
0.9045	7200	0.0402	-	-	-
0.9171	7300	0.0414	-	-	-
0.9296	7400	0.0414	-	-	-
0.9422	7500	0.0414	-	-	-
0.9548	7600	0.0397	-	-	-
0.9673	7700	0.041	-	-	-
0.9799	7800	0.0382	-	-	-
0.9925	7900	0.0427	-	-	-
1.0	7960	-	0.0367	0.9941	-
1.0050	8000	0.0383	-	-	-
1.0176	8100	0.0313	-	-	-
1.0302	8200	0.033	-	-	-
1.0427	8300	0.0322	-	-	-
1.0553	8400	0.0328	-	-	-
1.0678	8500	0.0316	-	-	-
1.0804	8600	0.0324	-	-	-
1.0930	8700	0.0289	-	-	-
1.1055	8800	0.0339	-	-	-
1.1103	8838	-	-	0.9946	-
0.0157	100	0.0302	-	-	-
0.0314	200	0.0316	-	-	-
0.0471	300	0.0284	-	-	-
0.0628	400	0.0294	-	-	-
0.0785	500	0.0294	-	-	-
0.0942	600	0.0288	-	-	-
0.1099	700	0.0303	-	-	-
0.1256	800	0.0295	-	-	-
0.1413	900	0.0295	-	-	-
0.1570	1000	0.0287	-	-	-
0.1727	1100	0.0299	-	-	-
0.1884	1200	0.0288	-	-	-
0.2041	1300	0.0301	-	-	-
0.2198	1400	0.031	-	-	-
0.2356	1500	0.03	-	-	-
0.2513	1600	0.0351	-	-	-
0.2670	1700	0.0322	-	-	-
0.2827	1800	0.0305	-	-	-
0.2984	1900	0.032	-	-	-
0.3141	2000	0.0328	-	-	-
0.3298	2100	0.033	-	-	-
0.3455	2200	0.032	-	-	-
0.3612	2300	0.031	-	-	-
0.3769	2400	0.0344	-	-	-
0.3926	2500	0.0314	-	-	-
0.4083	2600	0.0319	-	-	-
0.4240	2700	0.033	-	-	-
0.4397	2800	0.0316	-	-	-
0.4554	2900	0.0323	-	-	-
0.4711	3000	0.0326	-	-	-
0.4868	3100	0.0323	-	-	-
0.5025	3200	0.0344	-	-	-
0.5182	3300	0.0333	-	-	-
0.5339	3400	0.031	-	-	-
0.5496	3500	0.0338	-	-	-
0.5653	3600	0.0315	-	-	-
0.5810	3700	0.0308	-	-	-
0.5967	3800	0.0317	-	-	-
0.6124	3900	0.0326	-	-	-
0.6281	4000	0.032	-	-	-
0.6438	4100	0.0327	-	-	-
0.6595	4200	0.0321	-	-	-
0.6753	4300	0.0338	-	-	-
0.6910	4400	0.0302	-	-	-
0.7067	4500	0.0318	-	-	-
0.7224	4600	0.0324	-	-	-
0.7381	4700	0.0346	-	-	-
0.7538	4800	0.0351	-	-	-
0.7695	4900	0.032	-	-	-
0.7852	5000	0.032	-	-	-
0.8009	5100	0.0325	-	-	-
0.8166	5200	0.0312	-	-	-
0.8323	5300	0.031	-	-	-
0.8480	5400	0.0315	-	-	-
0.8637	5500	0.0352	-	-	-
0.8794	5600	0.0309	-	-	-
0.8951	5700	0.0317	-	-	-
0.9108	5800	0.0325	-	-	-
0.9265	5900	0.033	-	-	-
0.9422	6000	0.0309	-	-	-
0.9579	6100	0.0342	-	-	-
0.9736	6200	0.0312	-	-	-
0.9893	6300	0.0329	-	-	-
1.0	6368	-	0.0298	0.9927	-
1.0050	6400	0.028	-	-	-
1.0207	6500	0.0237	-	-	-
1.0364	6600	0.0208	-	-	-
1.0521	6700	0.0223	-	-	-
1.0678	6800	0.0211	-	-	-
1.0835	6900	0.0223	-	-	-
1.0992	7000	0.0213	-	-	-
1.1149	7100	0.0217	-	-	-
1.1307	7200	0.0218	-	-	-
1.1464	7300	0.0218	-	-	-
1.1621	7400	0.0224	-	-	-
1.1778	7500	0.022	-	-	-
1.1935	7600	0.0221	-	-	-
1.2092	7700	0.0218	-	-	-
1.2249	7800	0.0225	-	-	-
1.2406	7900	0.021	-	-	-
1.2563	8000	0.0225	-	-	-
1.2720	8100	0.0234	-	-	-
1.2877	8200	0.0238	-	-	-
1.3034	8300	0.0227	-	-	-
1.3191	8400	0.023	-	-	-
1.3348	8500	0.019	-	-	-
1.3505	8600	0.0227	-	-	-
1.3662	8700	0.0238	-	-	-
1.3819	8800	0.0211	-	-	-
1.3976	8900	0.0205	-	-	-
1.4133	9000	0.0212	-	-	-
1.4290	9100	0.0243	-	-	-
1.4447	9200	0.0224	-	-	-
1.4604	9300	0.0198	-	-	-
1.4761	9400	0.0227	-	-	-
1.4918	9500	0.0222	-	-	-
1.5075	9600	0.0232	-	-	-
1.5232	9700	0.0234	-	-	-
1.5389	9800	0.0222	-	-	-
1.5546	9900	0.0239	-	-	-
1.5704	10000	0.0227	-	-	-
1.5861	10100	0.0223	-	-	-
1.6018	10200	0.0224	-	-	-
1.6175	10300	0.022	-	-	-
1.6332	10400	0.0211	-	-	-
1.6489	10500	0.0208	-	-	-
1.6646	10600	0.0226	-	-	-
1.6803	10700	0.0227	-	-	-
1.6960	10800	0.0214	-	-	-
1.7117	10900	0.0221	-	-	-
1.7274	11000	0.0221	-	-	-
1.7431	11100	0.0213	-	-	-
1.7588	11200	0.0231	-	-	-
1.7745	11300	0.0203	-	-	-
1.7902	11400	0.0217	-	-	-
1.8059	11500	0.0215	-	-	-
1.8216	11600	0.0214	-	-	-
1.8373	11700	0.0235	-	-	-
1.8530	11800	0.0214	-	-	-
1.8687	11900	0.0213	-	-	-
1.8844	12000	0.0225	-	-	-
1.9001	12100	0.0209	-	-	-
1.9158	12200	0.0207	-	-	-
1.9315	12300	0.0235	-	-	-
1.9472	12400	0.0215	-	-	-
1.9629	12500	0.0221	-	-	-
1.9786	12600	0.0245	-	-	-
1.9943	12700	0.0228	-	-	-
2.0	12736	-	0.0301	0.9923	-
2.0101	12800	0.0174	-	-	-
2.0258	12900	0.0147	-	-	-
2.0415	13000	0.014	-	-	-
2.0572	13100	0.0132	-	-	-
2.0729	13200	0.0137	-	-	-
2.0886	13300	0.0134	-	-	-
2.1043	13400	0.0132	-	-	-
2.1200	13500	0.014	-	-	-
2.1357	13600	0.0162	-	-	-
2.1514	13700	0.0142	-	-	-
2.1671	13800	0.0149	-	-	-
2.1828	13900	0.015	-	-	-
2.1985	14000	0.0137	-	-	-
2.2142	14100	0.0147	-	-	-
2.2299	14200	0.0162	-	-	-
2.2456	14300	0.0153	-	-	-
2.2613	14400	0.0152	-	-	-
2.2770	14500	0.0151	-	-	-
2.2927	14600	0.0141	-	-	-
2.3084	14700	0.0133	-	-	-
2.3241	14800	0.0148	-	-	-
2.3398	14900	0.0147	-	-	-
2.3555	15000	0.0138	-	-	-
2.3712	15100	0.0149	-	-	-
2.3869	15200	0.0149	-	-	-
2.4026	15300	0.0137	-	-	-
2.4183	15400	0.0144	-	-	-
2.4340	15500	0.0143	-	-	-
2.4497	15600	0.0144	-	-	-
2.4655	15700	0.013	-	-	-
2.4812	15800	0.0144	-	-	-
2.4969	15900	0.0151	-	-	-
2.5126	16000	0.0138	-	-	-
2.5283	16100	0.0146	-	-	-
2.5440	16200	0.0142	-	-	-
2.5597	16300	0.0145	-	-	-
2.5754	16400	0.0133	-	-	-
2.5911	16500	0.0156	-	-	-
2.6068	16600	0.0138	-	-	-
2.6225	16700	0.015	-	-	-
2.6382	16800	0.0151	-	-	-
2.6539	16900	0.0136	-	-	-
2.6696	17000	0.0149	-	-	-
2.6853	17100	0.015	-	-	-
2.7010	17200	0.0132	-	-	-
2.7167	17300	0.0141	-	-	-
2.7324	17400	0.0145	-	-	-
2.7481	17500	0.0142	-	-	-
2.7638	17600	0.0139	-	-	-
2.7795	17700	0.0132	-	-	-
2.7952	17800	0.0142	-	-	-
2.8109	17900	0.0134	-	-	-
2.8266	18000	0.0153	-	-	-
2.8423	18100	0.0149	-	-	-
2.8580	18200	0.0132	-	-	-
2.8737	18300	0.014	-	-	-
2.8894	18400	0.0149	-	-	-
2.9052	18500	0.0141	-	-	-
2.9209	18600	0.0149	-	-	-
2.9366	18700	0.014	-	-	-
2.9523	18800	0.0143	-	-	-
2.9680	18900	0.0158	-	-	-
2.9837	19000	0.0132	-	-	-
2.9994	19100	0.0145	-	-	-
3.0	19104	-	0.0329	0.9915	0.9933

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.3.1
• Transformers: 4.46.3
• PyTorch: 2.5.1+cu121
• Accelerate: 1.1.1
• Datasets: 3.1.0
• Tokenizers: 0.20.3

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

ContrastiveLoss

@inproceedings{hadsell2006dimensionality,
    author={Hadsell, R. and Chopra, S. and LeCun, Y.},
    booktitle={2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)},
    title={Dimensionality Reduction by Learning an Invariant Mapping},
    year={2006},
    volume={2},
    number={},
    pages={1735-1742},
    doi={10.1109/CVPR.2006.100}
}