Add new SentenceTransformer model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 384,
+  "pooling_mode_cls_token": true,
+  "pooling_mode_mean_tokens": false,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,398 @@
+---
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:19979
+- loss:CoSENTLoss
+base_model: avsolatorio/GIST-small-Embedding-v0
+widget:
+- source_sentence: why oval face shape is attractive?
+  sentences:
+  - Both are also designed to add depth to your face, but in different ways. Bronzing
+    primarily warms up your face, adding color in places where the sun would naturally
+    hit. ... On the other hand, contouring is a makeup artist-approved technique that
+    works to add structure and shadow to your face.
+  - '''Ahjussi'' literally means uncle. You can use it for people who are very old
+    than you. Like someone who is the age of your father, or someone with an age gap
+    of 20 years or above.'
+  - Most major banks will be open on Christmas Eve 2018, even though it's a federal
+    holiday and generally recognized as a bank holiday.
+- source_sentence: is ceo same as owner?
+  sentences:
+  - The CEO reports to the Chairman (acting on behalf of the Board) and to the Board
+    directly. The Chairman is not responsible for executive matters regarding the
+    Company's business. Other than the CEO and the Company Secretary, no executive
+    reports to the Chairman, other than through the Board.
+  - Understanding Deregulation In response to the country's greatest financial crisis
+    in its history, Franklin D. Roosevelt's administration enacted many forms of financial
+    regulation, including the Securities Exchange Acts of 1933 and 1934 and the U.S.
+    Banking Act of 1933, otherwise known as the Glass-Steagall Act.
+  - Gdzie kupić wodorosty? Na naszym rynku są głównie algi suszone. Bez problemu kupimy
+    je w supermarketach, sklepach ze zdrową żywnością, bywają w rybnych - największy
+    wybór jest oczywiście w sklepach internetowych. Za 10 arkuszy glonów nori zapłacimy
+    ok.
+- source_sentence: is gern stock a good buy?
+  sentences:
+  - 'The majority of these pads are made from one of two absorptive materials: Silica
+    gel (a purified sand) or cellulose (a purified plant fiber), which are then coated
+    in a non-toxic plastic wrapping that''s perforated, allowing the liquid to seep
+    in and stay there.'
+  - '[''The Vanguard Total Stock Market ETF (NYSEMKT:VTI)'', ''The Vanguard Total
+    International Stock ETF (NASDAQ:VXUS)'', ''Amazon.com (NASDAQ:AMZN)'', ''Alphabet
+    (NASDAQ:GOOG)(NASDAQ:GOOGL)'', ''Facebook (NASDAQ:FB)'', ''Intuitive Surgical
+    (NASDAQ:ISRG)'']'
+  - SCD is a disease that worsens over time. Treatments are available that can prevent
+    complications and lengthen the lives of those who have this condition.
+- source_sentence: where are sulfhydryl groups found?
+  sentences:
+  - Sulfhydryl groups can be found in the amino acid cysteine. When two cysteine residues
+    are in close proximity to each other, they can form a disulfide bridge also called
+    cystine.
+  - '["On your Android phone or tablet, open your device''s Settings app .", ''Tap
+    Google. Manage your Google Account.'', ''At the top, tap Personal info.'', ''Under
+    "Profile," tap Name Edit. . You may be asked to sign in.'', ''Enter your name,
+    then tap Done.'']'
+  - '[''Difficulty digesting fat. It may take your body time to adjust to its new
+    method of digesting fat. ... '', ''Diarrhea and flatulence. Indigestion can cause
+    diarrhea or flatulence, often made worse by excess fat or too little fiber in
+    the diet. ... '', ''Constipation. ... '', ''Intestinal injury. ... '', ''Jaundice
+    or fever.'']'
+- source_sentence: do assets in an irrevocable trust get a step up in basis?
+  sentences:
+  - An irrevocable grantor trust can own S corporation stock if it meets IRS regulations.
+    ... If the trust owner designation is not made or is unclear, the trust will not
+    qualify under IRS regulations. An irrevocable grantor trust does not have to make
+    an election to be an S corporation shareholder.
+  - Pineapple juice also contains bromelain, a group of enzymes linked to health benefits,
+    such as reduced inflammation, improved digestion, and stronger immunity ( 9 ).
+    Pineapple juice is rich in antioxidants, which help protect your body from damage
+    and disease.
+  - 'Ideally, fuel up two hours before you exercise by: Hydrating with water. Eating
+    healthy carbohydrates such as whole-grain cereals (with low-fat or skim milk),
+    whole-wheat toast, low-fat or fat-free yogurt, whole grain pasta, brown rice,
+    fruits and vegetables.'
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+---
+
+# SentenceTransformer based on avsolatorio/GIST-small-Embedding-v0
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [avsolatorio/GIST-small-Embedding-v0](https://huggingface.co/avsolatorio/GIST-small-Embedding-v0). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [avsolatorio/GIST-small-Embedding-v0](https://huggingface.co/avsolatorio/GIST-small-Embedding-v0) <!-- at revision 75e62fd210b9fde790430e0b2f040b0b00a021b1 -->
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 384 dimensions
+- **Similarity Function:** Cosine Similarity
+<!-- - **Training Dataset:** Unknown -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
+  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Normalize()
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("moshew/gist_small_ft_gooaq_v4")
+# Run inference
+sentences = [
+    'do assets in an irrevocable trust get a step up in basis?',
+    'An irrevocable grantor trust can own S corporation stock if it meets IRS regulations. ... If the trust owner designation is not made or is unclear, the trust will not qualify under IRS regulations. An irrevocable grantor trust does not have to make an election to be an S corporation shareholder.',
+    'Ideally, fuel up two hours before you exercise by: Hydrating with water. Eating healthy carbohydrates such as whole-grain cereals (with low-fat or skim milk), whole-wheat toast, low-fat or fat-free yogurt, whole grain pasta, brown rice, fruits and vegetables.',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 384]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### Unnamed Dataset
+
+* Size: 19,979 training samples
+* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | sentence1                                                                         | sentence2                                                                           | label                                                         |
+  |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:--------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                              | float                                                         |
+  | details | <ul><li>min: 8 tokens</li><li>mean: 11.87 tokens</li><li>max: 24 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 59.82 tokens</li><li>max: 139 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.5</li><li>max: 1.0</li></ul> |
+* Samples:
+  | sentence1                                                          | sentence2                                                                                                                                                                                                                                                                                                                           | label            |
+  |:-------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
+  | <code>if someone blocked you on iphone can you send a text?</code> | <code>If someone has blocked you on their device, you won't get an alert when it happens. You can still use iMessage to text your former contact, but they'll never receive the message or any notification of a text received in their Messages app.</code>                                                                        | <code>1.0</code> |
+  | <code>if someone blocked you on iphone can you send a text?</code> | <code>If someone has blocked you on their device, you won't get an alert when it happens. You can still use iMessage to text your former contact, but they'll never receive the message or any notification of a text received in their Messages app. There is one clue that you've been blocked, though.</code>                    | <code>0.0</code> |
+  | <code>can you have a relationship without expectations?</code>     | <code>Loving without expectations means being able to love someone even when they are letting you down. It means loving even when it feels awful. Even when you're crying so hard you can't see straight or say clear sentences. Loving someone without expectations means knowing they aren't perfect, but neither are you.</code> | <code>1.0</code> |
+* Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "pairwise_cos_sim"
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `num_train_epochs`: 1
+- `warmup_ratio`: 0.1
+- `seed`: 12
+- `bf16`: True
+- `dataloader_num_workers`: 4
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: no
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 5e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 12
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: True
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 4
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `tp_size`: 0
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+| Epoch  | Step | Training Loss |
+|:------:|:----:|:-------------:|
+| 0.0008 | 1    | 3.6013        |
+| 0.8006 | 1000 | 3.4341        |
+
+
+### Framework Versions
+- Python: 3.11.12
+- Sentence Transformers: 4.1.0
+- Transformers: 4.51.3
+- PyTorch: 2.6.0+cu124
+- Accelerate: 1.5.2
+- Datasets: 3.5.0
+- Tokenizers: 0.21.1
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### CoSENTLoss
+```bibtex
+@online{kexuefm-8847,
+    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
+    author={Su Jianlin},
+    year={2022},
+    month={Jan},
+    url={https://kexue.fm/archives/8847},
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,30 @@
+{
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.51.3",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "4.1.0",
+    "transformers": "4.51.3",
+    "pytorch": "2.6.0+cu124"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:791abe7f5a88953363d55678d78f28905fef3b3bdc20acdf4e5cbe133ac5df25
+size 133462128

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": true
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}