AssistiveRecGraniteSmall

This is a sentence-transformers model for assistive technology (AT) recommendations finetuned from ibm-granite/granite-embedding-125m-english. It maps sentences & paragraphs describing AT products and user goals/challenges to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. The document format is intended to be a natural language product description. Queries are formed per this template "Goals: [comma-seperated list of goals], Challenges: [comma-seperated list of challenges]". See examples.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: ibm-granite/granite-embedding-125m-english
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Language: en
• License: Apache 2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: RobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("hbriegel/AssistiveRecGraniteSmall")
# Run inference
sentences = [
    'The Oticon Connectline TV Adapter 2.0 is a device designed to enhance the audio experience for individuals using Oticon hearing aids. This adapter allows users to stream sound directly from their television to their hearing aids, providing high-quality sound and the convenience of wireless connectivity. The hearing aids function as wireless headsets, while the Streamer Pro acts as a volume control, ensuring that the user can adjust the sound to their comfort and preference. This device is particularly beneficial for those who struggle with hearing in noisy environments or who prefer a more personalized audio experience.',
    "Goals: ['To improve overall audio experience and clarity while watching television.', 'To enhance communication and social interactions by ensuring clear sound transmission.', 'To maintain independence and comfort in managing personal audio preferences.'], Challenges: ['Hearing impairments that require the use of hearing aids for clear audio reception.', 'Difficulty in understanding dialogue or sound in noisy environments.', 'Need for personalized audio settings to accommodate individual hearing needs.']",
    "Goals: ['To improve mobility and independence for individuals with temporary or permanent mobility challenges.', 'To reduce discomfort and strain associated with the use of crutches.', 'To enhance the overall quality of life by providing a comfortable and supportive mobility aid.'], Challenges: ['Temporary mobility issues due to injury or surgery.', 'Chronic conditions affecting mobility, such as arthritis or multiple sclerosis.', 'Post-surgical recovery requiring the use of crutches.', 'Conditions that require the use of assistive devices for mobility.']",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Contains data from Assistive Technology Data Denmark provided by the National Board of Social Services.

Multimodal assistive technology dataset

• Size: 4,623 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 68 tokens mean: 115.92 tokens max: 211 tokens	min: 50 tokens mean: 91.54 tokens max: 142 tokens

• Samples:

  anchor	positive
  The image depicts a hairbrush designed with an ergonomic handle for easy grip and control. The brush features white bristles with blue tips, which are gentle on the scalp and effective in detangling hair. The handle is predominantly blue with a white section near the bristles, providing a comfortable and secure grip. This hairbrush is suitable for daily use and helps maintain healthy hair by distributing natural oils and removing dirt and debris.	Goals: ['Maintain personal hygiene and grooming habits to feel confident and well-presented.', 'Incorporate self-care routines to improve mental well-being and reduce stress.', 'Ensure comfort and ease of use in daily activities to promote independence.'], Challenges: ['Mobility impairments that affect the ability to hold and use traditional hairbrushes.', 'Arthritis or other conditions that limit grip strength and dexterity.', 'Visual impairments that require larger, more easily identifiable brushes.']
  The Arjo Maxi Slide pull straps are an essential accessory designed to facilitate safe and ergonomic lateral transfers for patients. These straps are specifically engineered to be used with Arjo's Maxi Slide products, which are commonly found in healthcare settings. The set includes eight straps that can be easily mounted onto the Maxi Slide system, allowing healthcare professionals to assist patients in moving sideways from one surface to another, such as from a bed to a wheelchair or vice versa. This equipment is crucial for minimizing the risk of injury to both patients and caregivers by promoting proper body mechanics during transfers.	Goals: ['To ensure safe and comfortable patient transfers', 'To reduce the risk of injury for both patients and caregivers', 'To maintain independence and mobility', 'To enhance the quality of care in healthcare settings'], Challenges: ['Mobility impairments', 'Risk of falls and injuries during transfers', 'Need for assistance with lateral movements', 'Conditions requiring frequent transfers between different surfaces']
  The Vilhelm Hertz crutch is a beautifully designed assistive device crafted from high-quality materials including oak wood, aluminum, steel, and carbon fiber. Each crutch is custom-made to fit the individual user, ensuring both comfort and functionality. The crutches feature an innovative elastic element that enhances user comfort during use. The non-height adjustable design ensures a perfect fit for the user, making it a stylish and practical choice for those needing mobility assistance.	Goals: ['Maintain independence in daily activities', 'Enhance mobility and reduce reliance on others', 'Improve overall quality of life through comfortable and stylish assistive devices', 'Participate fully in social and community activities'], Challenges: ['Lower limb injuries', 'Post-surgical recovery', 'Mobility impairments', 'Chronic conditions affecting lower limb function']

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Evaluation Dataset

Contains data from Assistive Technology Data Denmark provided by the National Board of Social Services.

Multimodal assistive technology dataset

• Size: 514 evaluation samples
• Columns: anchor and positive
• Approximate statistics based on the first 514 samples:

  	anchor	positive
  type	string	string
  details	min: 58 tokens mean: 114.87 tokens max: 181 tokens	min: 58 tokens mean: 91.31 tokens max: 162 tokens

• Samples:

  anchor	positive
  The Amoena Lara SBP Bh Lined w / Pockets is a specialized t-shirt bra designed to provide comfort and support for women who have undergone mastectomies. The bra features molded cups that offer a smooth and uniform appearance under clothing. The straps are fully adjustable, ensuring a customized fit for maximum comfort. Internal pockets are included to securely hold breast prostheses in place, providing a natural silhouette. This bra is available in sizes ranging from AA to E cup and comes in colors such as off-white, black, and nude.	Goals: ['To regain confidence in personal appearance after a mastectomy', 'To achieve a natural and balanced body silhouette', 'To maintain comfort and support during daily activities', 'To feel secure and supported during physical activities'], Challenges: ['Post-mastectomy recovery and adjustment', 'Breast asymmetry', 'Maintaining a natural appearance with breast prostheses', 'Managing discomfort and irritation from surgical sites']
  The Bathing Chair with Swivel Seat is a versatile and practical assistive device designed to enhance safety and comfort during bathing or showering. This chair features a swivel seat that allows for easy entry and exit from the shower area, minimizing the risk of falls and injuries. The seat is equipped with a non-slip surface to ensure stability and comfort. The chair's height can be adjusted between 40 to 50 cm to accommodate different user needs and preferences. Constructed with durable aluminum legs, the chair is lightweight yet sturdy, capable of supporting up to 150 kg. The legs are equipped with non-slip feet to provide additional stability on wet surfaces. This chair is ideal for individuals who require assistance with bathing and showering, helping them maintain independence and safety in the bathroom.	Goals: ['To maintain independence and self-sufficiency in daily personal care activities.', 'To enhance safety and reduce the risk of falls and injuries during bathing.', 'To improve comfort and accessibility in the bathroom.'], Challenges: ['Mobility impairments that affect balance and stability.', 'Age-related declines in strength and flexibility.', 'Conditions that require assistance with personal hygiene, such as post-surgery recovery or chronic illness.']
  The item depicted is an iPad holder designed to securely mount an iPad in various settings such as tables, walls, or floors. The holder is constructed with a stable and attractive design, available in black or polished/chrome finishes. It can be customized to match specific color preferences for an additional fee. The floor model weighs 2.0 kg, while the table and wall-mounted units weigh 1.5 kg. An optional step feature is available for an additional cost of 625 kr.	Goals: ['Enhance accessibility to digital content and communication tools for improved daily life management.', 'Facilitate independent use of technology to maintain personal and professional connections.', 'Create a user-friendly environment that supports both work and leisure activities at home or in the office.'], Challenges: ['Assistive needs for individuals with mobility impairments who require stable mounting solutions for tablets.', 'Support for users with visual impairments by providing a secure and accessible position for screen readers or magnification software.', 'Assistance for individuals with fine motor impairments who need a stable mount to interact with touchscreen devices.']

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 6
• warmup_ratio: 0.1
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 6
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss
0.3460	100	0.9193	0.2646
0.6920	200	0.2099	0.1084
1.0381	300	0.1354	0.1004
1.3841	400	0.097	0.0833
1.7301	500	0.0622	0.0817
2.0761	600	0.0433	0.0768
2.4221	700	0.0313	0.0705
2.7682	800	0.0244	0.0776
3.1142	900	0.018	0.0863
3.4602	1000	0.0124	0.0821
3.8062	1100	0.0125	0.0821
4.1522	1200	0.0078	0.0751
4.4983	1300	0.0069	0.0769
4.8443	1400	0.0046	0.0804
5.1903	1500	0.005	0.0772
5.5363	1600	0.0051	0.0784
5.8824	1700	0.003	0.0758

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.3.1
• Transformers: 4.47.1
• PyTorch: 2.5.1+cu121
• Accelerate: 1.2.1
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}