metadata

tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:19979
  - loss:CoSENTLoss
base_model: avsolatorio/GIST-small-Embedding-v0
widget:
  - source_sentence: why oval face shape is attractive?
    sentences:
      - >-
        Both are also designed to add depth to your face, but in different ways.
        Bronzing primarily warms up your face, adding color in places where the
        sun would naturally hit. ... On the other hand, contouring is a makeup
        artist-approved technique that works to add structure and shadow to your
        face.
      - >-
        'Ahjussi' literally means uncle. You can use it for people who are very
        old than you. Like someone who is the age of your father, or someone
        with an age gap of 20 years or above.
      - >-
        Most major banks will be open on Christmas Eve 2018, even though it's a
        federal holiday and generally recognized as a bank holiday.
  - source_sentence: is ceo same as owner?
    sentences:
      - >-
        The CEO reports to the Chairman (acting on behalf of the Board) and to
        the Board directly. The Chairman is not responsible for executive
        matters regarding the Company's business. Other than the CEO and the
        Company Secretary, no executive reports to the Chairman, other than
        through the Board.
      - >-
        Understanding Deregulation In response to the country's greatest
        financial crisis in its history, Franklin D. Roosevelt's administration
        enacted many forms of financial regulation, including the Securities
        Exchange Acts of 1933 and 1934 and the U.S. Banking Act of 1933,
        otherwise known as the Glass-Steagall Act.
      - >-
        Gdzie kupić wodorosty? Na naszym rynku są głównie algi suszone. Bez
        problemu kupimy je w supermarketach, sklepach ze zdrową żywnością,
        bywają w rybnych - największy wybór jest oczywiście w sklepach
        internetowych. Za 10 arkuszy glonów nori zapłacimy ok.
  - source_sentence: is gern stock a good buy?
    sentences:
      - >-
        The majority of these pads are made from one of two absorptive
        materials: Silica gel (a purified sand) or cellulose (a purified plant
        fiber), which are then coated in a non-toxic plastic wrapping that's
        perforated, allowing the liquid to seep in and stay there.
      - >-
        ['The Vanguard Total Stock Market ETF (NYSEMKT:VTI)', 'The Vanguard
        Total International Stock ETF (NASDAQ:VXUS)', 'Amazon.com
        (NASDAQ:AMZN)', 'Alphabet (NASDAQ:GOOG)(NASDAQ:GOOGL)', 'Facebook
        (NASDAQ:FB)', 'Intuitive Surgical (NASDAQ:ISRG)']
      - >-
        SCD is a disease that worsens over time. Treatments are available that
        can prevent complications and lengthen the lives of those who have this
        condition.
  - source_sentence: where are sulfhydryl groups found?
    sentences:
      - >-
        Sulfhydryl groups can be found in the amino acid cysteine. When two
        cysteine residues are in close proximity to each other, they can form a
        disulfide bridge also called cystine.
      - >-
        ["On your Android phone or tablet, open your device's Settings app .",
        'Tap Google. Manage your Google Account.', 'At the top, tap Personal
        info.', 'Under "Profile," tap Name Edit. . You may be asked to sign
        in.', 'Enter your name, then tap Done.']
      - >-
        ['Difficulty digesting fat. It may take your body time to adjust to its
        new method of digesting fat. ... ', 'Diarrhea and flatulence.
        Indigestion can cause diarrhea or flatulence, often made worse by excess
        fat or too little fiber in the diet. ... ', 'Constipation. ... ',
        'Intestinal injury. ... ', 'Jaundice or fever.']
  - source_sentence: do assets in an irrevocable trust get a step up in basis?
    sentences:
      - >-
        An irrevocable grantor trust can own S corporation stock if it meets IRS
        regulations. ... If the trust owner designation is not made or is
        unclear, the trust will not qualify under IRS regulations. An
        irrevocable grantor trust does not have to make an election to be an S
        corporation shareholder.
      - >-
        Pineapple juice also contains bromelain, a group of enzymes linked to
        health benefits, such as reduced inflammation, improved digestion, and
        stronger immunity ( 9 ). Pineapple juice is rich in antioxidants, which
        help protect your body from damage and disease.
      - >-
        Ideally, fuel up two hours before you exercise by: Hydrating with water.
        Eating healthy carbohydrates such as whole-grain cereals (with low-fat
        or skim milk), whole-wheat toast, low-fat or fat-free yogurt, whole
        grain pasta, brown rice, fruits and vegetables.
pipeline_tag: sentence-similarity
library_name: sentence-transformers

SentenceTransformer based on avsolatorio/GIST-small-Embedding-v0

This is a sentence-transformers model finetuned from avsolatorio/GIST-small-Embedding-v0. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: avsolatorio/GIST-small-Embedding-v0
Maximum Sequence Length: 512 tokens
Output Dimensionality: 384 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("moshew/gist_small_ft_gooaq_v4")
# Run inference
sentences = [
    'do assets in an irrevocable trust get a step up in basis?',
    'An irrevocable grantor trust can own S corporation stock if it meets IRS regulations. ... If the trust owner designation is not made or is unclear, the trust will not qualify under IRS regulations. An irrevocable grantor trust does not have to make an election to be an S corporation shareholder.',
    'Ideally, fuel up two hours before you exercise by: Hydrating with water. Eating healthy carbohydrates such as whole-grain cereals (with low-fat or skim milk), whole-wheat toast, low-fat or fat-free yogurt, whole grain pasta, brown rice, fruits and vegetables.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

Size: 19,979 training samples
Columns: sentence1, sentence2, and label
Approximate statistics based on the first 1000 samples:
sentence1 sentence2 label
type string string float
details
min: 8 tokens
mean: 11.87 tokens
max: 24 tokens

min: 15 tokens
mean: 59.82 tokens
max: 139 tokens

min: 0.0
mean: 0.5
max: 1.0

	sentence1	sentence2	label
type	string	string	float
details	min: 8 tokens mean: 11.87 tokens max: 24 tokens	min: 15 tokens mean: 59.82 tokens max: 139 tokens	min: 0.0 mean: 0.5 max: 1.0

Samples:

sentence1	sentence2	label
`if someone blocked you on iphone can you send a text?`	`If someone has blocked you on their device, you won't get an alert when it happens. You can still use iMessage to text your former contact, but they'll never receive the message or any notification of a text received in their Messages app.`	`1.0`
`if someone blocked you on iphone can you send a text?`	`If someone has blocked you on their device, you won't get an alert when it happens. You can still use iMessage to text your former contact, but they'll never receive the message or any notification of a text received in their Messages app. There is one clue that you've been blocked, though.`	`0.0`
`can you have a relationship without expectations?`	`Loving without expectations means being able to love someone even when they are letting you down. It means loving even when it feels awful. Even when you're crying so hard you can't see straight or say clear sentences. Loving someone without expectations means knowing they aren't perfect, but neither are you.`	`1.0`

Loss: CoSENTLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "pairwise_cos_sim"
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 16
per_device_eval_batch_size: 16
num_train_epochs: 1
warmup_ratio: 0.1
seed: 12
bf16: True
dataloader_num_workers: 4

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 16
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 12
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 4
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
tp_size: 0
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss
0.0008	1	3.6013
0.8006	1000	3.4341

Framework Versions

Python: 3.11.12
Sentence Transformers: 4.1.0
Transformers: 4.51.3
PyTorch: 2.6.0+cu124
Accelerate: 1.5.2
Datasets: 3.5.0
Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CoSENTLoss

@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}