Upload folder using huggingface_hub

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,62 @@
+
+---
+library_name: sentence-transformers
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- autotrain
+base_model: nomic-ai/nomic-embed-text-v1.5
+widget:
+- source_sentence: 'search_query: i love autotrain'
+  sentences:
+  - 'search_query: huggingface auto train'
+  - 'search_query: hugging face auto train'
+  - 'search_query: i love autotrain'
+pipeline_tag: sentence-similarity
+---
+
+# Model Trained Using AutoTrain
+
+- Problem type: Sentence Transformers
+
+## Validation Metrics
+loss: 0.0011504614958539605
+
+runtime: 12.6072
+
+samples_per_second: 112.396
+
+steps_per_second: 7.059
+
+: 3.0
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the Hugging Face Hub
+model = SentenceTransformer("sentence_transformers_model_id")
+# Run inference
+sentences = [
+    'search_query: autotrain',
+    'search_query: auto train',
+    'search_query: i love autotrain',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+```

checkpoint-10119/1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

checkpoint-10119/README.md

@@ -0,0 +1,1076 @@
+---
+base_model: nomic-ai/nomic-embed-text-v1.5
+datasets: []
+language: []
+library_name: sentence-transformers
+pipeline_tag: sentence-similarity
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:26984
+- loss:MultipleNegativesRankingLoss
+widget:
+- source_sentence: 'search_query: Regulations for taking hair samples in Northern
+    Ireland'
+  sentences:
+  - "search_document: 39) In Article 63 of the Police and Criminal Evidence (Northern\
+    \ Ireland) Order 1989 (regulation of taking of non-intimate samples), at the end,\
+    \ there shall be inserted the following paragraph—\n\tWhere a sample of hair other\
+    \ than pubic hair is to be taken the sample may be taken either by cutting hairs\
+    \ or by plucking hairs with their roots so long as no more are plucked than the\
+    \ person taking the sample reasonably considers to be necessary (in point of quantity\
+    \ or quality) for the purpose of enabling information to be produced by means\
+    \ of analysis used or to be used in relation to the sample."
+  - "search_document: ### Section 253\n### Declaration of general improvement area.\n\
+    \t1) \n\t\tWhere a report with respect to a predominantly residential area within\
+    \ their district is submitted to the local housing authority by a person appearing\
+    \ to the authority to be suitably qualified (who may be an officer of the authority),\
+    \ and it appears to the authority, upon consideration of the report and of any\
+    \ other information in their possession—\n\t\tthe authority may cause the area\
+    \ to be defined on a map and by resolution declare it to be a general improvement\
+    \ area.\n\t\t\ta) that living conditions in the area can most appropriately be\
+    \ improved by the improvement of the amenities of the area or of dwellings in\
+    \ the area, or both, and\n\t\t\tb) that such an improvement may be effected or\
+    \ assisted by the exercise of their powers under the provisions of this Part relating\
+    \ to general improvement areas,\n\t2) A general improvement area may not be defined\
+    \ so as to include, but may be defined so as to surround, land which is comprised\
+    \ in a housing action area.\n\t3) \n\t\tA general improvement area may not (unless\
+    \ the land has been cleared of buildings) be so defined as to include, but may\
+    \ be so defined as to surround—\n\t\tand where the Secretary of State on confirming\
+    \ a compulsory purchase order under Schedule 22 (acquisition of land for clearance)\
+    \ modifies the order by excluding from a clearance area land adjoining a general\
+    \ improvement area, the land shall, unless the Secretary of State otherwise directs,\
+    \ be taken to be included in the general improvement area.\n\t\t\ta) land comprised\
+    \ in a clearance area,\n\t\t\tb) land purchased by the local housing authority\
+    \ under section 290(2) (land surrounded by or adjoining clearance area), or\n\t\
+    \t\tc) land included in a clearance area under section 293(1) (local housing authority’s\
+    \ own property);"
+  - "search_document: 2) Regulations made by virtue of section 88(2)(e) of this Act\
+    \ may make provision—\n\ta) for all matters relevant to the maintenance of a register\
+    \ of submersible apparatus,\n\tb) without prejudice to sub-paragraph (a) above,\
+    \ for the period for which any registration or exemption is to remain effective\
+    \ without renewal, the alteration or cancellation in any prescribed circumstances\
+    \ of registration or exemption or of any conditions attached thereto, the person\
+    \ by whom and manner in which applications in connection with any registration\
+    \ or exemption are to be made, and information and evidence to be furnished in\
+    \ connection with any such application,\n\tc) for the marking or other means of\
+    \ identification of any submersible apparatus,\n\td) for the issue of certificates\
+    \ of registration or exemption, and the custody, surrender, production or display\
+    \ of the certificates or copies of them,\n\te) for matters arising out of the\
+    \ termination of any registration or exemption, or any conditions attached thereto."
+- source_sentence: 'search_query: Regulations regarding the designation of special
+    parking areas in London.'
+  sentences:
+  - 'search_document: 4) The enforcement authority may, if they consider that the
+    penalty charge notice ought not to have been given, give the recipient a notice
+    withdrawing the penalty charge notice.'
+  - "search_document: ### Section 268\n### Reckoning of time spent pending appeal.\n\
+    \t1) Subject to subsection (2) below, where None is admitted to bail under section\
+    \ 238 of this Act the period beginning with the date of his admission to bail\
+    \ and ending on the date of his readmission to prison in consequence of the determination\
+    \ or abandonment of his appeal None None None shall not be reckoned as part of\
+    \ any term of imprisonment under None sentence.\n\t2) The time (including any\
+    \ period consequent on the recall of bail) during which a convicted person is\
+    \ in custody pending the determination of his appeal, or as the case may be of\
+    \ any None\n\t3) Subject to any direction which the High Court may give to the\
+    \ contrary, imprisonment of an appellant None —\n\t\ta) who is in custody in consequence\
+    \ of the conviction or sentence appealed against shall be deemed to run as from\
+    \ the date on which the sentence was passed;\n\t\tb) who is in custody other than\
+    \ in consequence of such conviction or sentence shall be deemed to run or to be\
+    \ resumed as from the date on which his appeal was determined or abandoned;\n\t\
+    \tc) who is not in custody shall be deemed to run or to be resumed as from the\
+    \ date on which he is received into prison under the sentence.\n\t4) young offenders\
+    \ institution or, as respects a child sentenced to be detained under section 206\
+    \ of this Act, the place directed by the Secretary of State or, as respects such\
+    \ a child, place directed by the Secretary of State"
+  - "search_document: ### Section 76\n### Special parking areas.\n\t1) Where a London\
+    \ authority apply to the Secretary of State for an order to be made under this\
+    \ section, the Secretary of State may make an order designating the whole, or\
+    \ any part, of that authority’s area as a special parking area.\n\tAn application\
+    \ for an order under subsection (1) above may only be made—\n\t\tby Transport\
+    \ for London, to the extent that the special parking area is to consist of GLA\
+    \ roads or trunk roads; or\n\t\tby a London local authority, to the extent that\
+    \ the special parking area is to consist of roads other than GLA roads and trunk\
+    \ roads.\n\t2) Before making an order under this section, the Secretary of State\
+    \ shall consult the relevant Commissioner or, if appropriate, both Commissioners.\n\
+    \t3) While an order under this section is in force, the following provisions shall\
+    \ cease to apply in relation to the special parking area designated by the order—\n\
+    \t\ta) section 8 of the Road Traffic Regulation Act 1984 (contravention of, or\
+    \ failure to comply with, an order under section 6 of that Act to be an offence),\
+    \ so far as it relates to the contravention of, or failure to comply with, any\
+    \ provision of such an order—\n\t\t\ti) prohibiting or restricting the waiting\
+    \ of vehicles on any road; or\n\t\t\tii) relating to any of the matters mentioned\
+    \ in paragraph 7 or 8 of Schedule 1 to that Act (conditions for loading or unloading,\
+    \ or delivery or collecting);\n\t\tb) section 11 of the Act of 1984 (contravention\
+    \ of, or failure to comply with, an experimental traffic order under section 9\
+    \ of that Act to be an offence), so far as it relates to any contravention of,\
+    \ or failure to comply with, any provision of such an experimental traffic order—\n\
+    \t\t\ti) prohibiting or restricting the waiting of vehicles on any road; or\n\t\
+    \t\tii) relating to any of the matters mentioned in paragraph 7 or 8 of Schedule\
+    \ 1 to that Act (conditions for loading or unloading, or delivery or collecting);\n\
+    \t\tc) section 16(1) of the Act of 1984  so far as it relates to the contravention\
+    \ of any provision of an order or notice under section 14 of that Act—\n\t\t\t\
+    i) prohibiting or restricting the waiting of vehicles on any road; or\n\t\t\t\
+    ii) relating to any of the matters mentioned in paragraph 7 or 8 of Schedule 1\
+    \ to that Act;\n\t\tsection 35A(1) (contravention of parking place orders) of\
+    \ the Act of 1984 so far as it applies in relation to stationary vehicles;\n\t\
+    \tsection 61(5) (prohibition of vehicles in loading areas) of the Act of 1984\
+    \ so far as it applies in relation to stationary vehicles;.\n\t\td) section 15\
+    \ of the Greater London Council (General Powers) Act 1974 (parking of vehicles\
+    \ on verges, central reservations and footpaths etc. to be an offence);\n\t\t\
+    e) section 19 of the Road Traffic Act 1988 (parking of heavy vehicles on verges,\
+    \ central reservations and footpaths etc. to be an offence);\n\t\tf) section 21\
+    \ of the Act of 1988 (prohibition of driving or parking on cycle tracks), so far\
+    \ as it makes it an offence to park a motor vehicle wholly or partly on a cycle\
+    \ track;\n\t\tsection 36(1) of the Act of 1988 (failure to comply with traffic\
+    \ signs), so far as it makes it an offence to fail to comply with an indication\
+    \ given by a traffic sign of a prohibition on causing a vehicle to stop on part\
+    \ of a road in London demarcated by that sign as a stopping area for a bus.\n\t\
+    4) The Secretary of State may by order amend subsection (3) above by adding further\
+    \ provisions (but only in so far as they apply in relation to stationary vehicles).\n\
+    \t5) Before making an order under subsection (4) above, the Secretary of State\
+    \ shall consult—\n\t\ta) the two Commissioners; and\n\t\tb) such associations\
+    \ of London authorities (if any) as he thinks appropriate."
+- source_sentence: 'search_query: Financial adjustments between local authorities
+    for adult care and support needs'
+  sentences:
+  - "search_document: ### Section 84\n### Determination of questions arising out of\
+    \ section 83.\n\t1) \n\t\tA question as to whether—\n\t\tshall be decided—\n\t\
+    \t\ti) if the parties so agree, by a single arbiter appointed by them; or\n\t\t\
+    \tii) in default of such agreement by the sheriff.\n\t2) In determining any such\
+    \ question as is mentioned in paragraph (a) of subsection (1) above, the arbiter\
+    \ or sheriff shall have power to order that the requirement or restriction shall\
+    \ have effect subject to such modifications, if any, as he may direct."
+  - "search_document: ### Section 41\n### Financial adjustments between local authorities\n\
+    \t1) This section applies where—\n\t\ta) a local authority has been meeting an\
+    \ adult's needs for care and support, but\n\t\tb) it transpires (whether following\
+    \ the determination of a dispute under section 40 or otherwise) that the adult\
+    \ was, for some or all of the time that the authority has been meeting the adult's\
+    \ needs, ordinarily resident in the area of another local authority.\n\t2) This\
+    \ section also applies where—\n\t\ta) a local authority has been meeting a carer's\
+    \ needs for support, but\n\t\tb) it transpires (whether following the determination\
+    \ of a dispute under section 40 or otherwise) that the adult needing care was,\
+    \ for some or all of the time that the authority has been meeting the carer's\
+    \ needs, ordinarily resident in the area of another local authority.\n\t3) The\
+    \ local authority concerned may recover from the other local authority the amount\
+    \ of any payments it made towards meeting the needs in question at a time when\
+    \ the other local authority was instead liable to meet them under section 18 or\
+    \ 20(1) (as the case may be).\n\t4) Subsection (3) does not apply to payments\
+    \ which are the subject of a deferred payment agreement entered into by the local\
+    \ authority in question, unless it agrees with the other local authority to assign\
+    \ its rights and obligations under the deferred payment agreement to that other\
+    \ authority.\n\t5) Any period during which a local authority was meeting the needs\
+    \ in question under section 19 or 20(6) is to be disregarded for the purposes\
+    \ of this section."
+  - "search_document: ### Section 64\n### Decision on applications for operators’\
+    \ licences.\n\t1) Subject to section 69E of this ActOn an application for an operator’s\
+    \ licence, the licensing authority shall in every case consider whether the requirements\
+    \ mentioned in paragraphs (a) to (d) of subsection (2) of this section, and, if\
+    \ the licensing authority in any case thinks fit, paragraph (e) of that subsection,\
+    \ are satisfied, and in doing so shall have regard to any objection duly made\
+    \ under section 63 of this Act.\n\t2) The said requirements are as follows—\n\t\
+    \ta) that the applicant is a fit person to hold an operator’s licence, having\
+    \ regard to the matters of which particulars may be required to be given under\
+    \ section 62(4)(d) and (e) of this Act and to any conviction required to be notified\
+    \ in accordance with section 62(4A) thereof;\n\t\tb) . . . . . . . . . . . . .\
+    \ . . . . . . . . . . . . . . . . . . . \n\t\tc) that there will be satisfactory\
+    \ arrangements for securing that Part VI of this Act (or, so long as those sections\
+    \ remain in force, sections 73 and 186 of the Act of 1960) will be complied with\
+    \ in the case of the authorised vehicles, and for securing that those vehicles\
+    \ are not overloaded;\n\t\td) that there will be satisfactory facilities and arrangements\
+    \ for maintaining the authorised vehicles in a fit and serviceable condition and\
+    \ that the place which is to be the operating centre for those vehicles is suitable\
+    \ for that purpose;\n\t\te) that the provision of such facilities and arrangements\
+    \ as are mentioned in paragraph (d) of this subsection and of a suitable operating\
+    \ centre will not be prejudiced by reason of the applicant’s having insufficient\
+    \ financial resources for that purpose.\n\t3) If the licensing authority determines\
+    \ that any requirement which he has taken into consideration in accordance with\
+    \ subsection (1) of this section is not satisfied, he shall refuse the application\
+    \ but, in any other case, he shall, subject to subsection (4) of this section\
+    \ and section 69B of this Act, grant the application.\n\t4) In any case in which\
+    \ the licensing authority grants an application for an operator’s licence, the\
+    \ licensing authority may issue that licence in the terms applied for or, if the\
+    \ authority thinks fit, subject to either or both of the following modifications\
+    \ or limitations, that is to say—\n\t\ta) so that the licence is in respect of\
+    \ motor vehicles other than those of which particulars were contained in the application,\
+    \ or in respect of motor vehicles or trailers greater or less in number than,\
+    \ or differing in type from, those for the use of which authorisation was applied\
+    \ for;\n\t\tb) so that the licence does not permit the addition of authorised\
+    \ vehicles under section 61(1)(c) of this Act.\n\t5) In exercising his functions\
+    \ under this section in relation to the requirement mentioned in subsection (2)(e)\
+    \ thereof, a licensing authority may be assisted by an assessor drawn from a panel\
+    \ of persons appointed by the Minister for that purpose; and there shall be paid\
+    \ by the licensing authority to any such assessor in respect of his services remuneration\
+    \ on a scale prescribed by the Minister with the approval of the Treasury."
+- source_sentence: 'search_query: Legal powers to establish a constitution for Zimbabwe'
+  sentences:
+  - "search_document: ### Section 72\n### Section 71: supplementary provisions.\n\t\
+    1) \n\t\tSubject to subsection (2) below, where the members of the existing governing\
+    \ body of a school to which section 71 of this Act applies include a person—\n\
+    \t\tthe governing body may by notice in writing to that person terminate his term\
+    \ of office on a date specified in the notice.\n\t\t\ta) who holds office as a\
+    \ governor of an elected category, and\n\t\t\tb) whose term of office is due to\
+    \ come to an end before the date of implementation of the proposals or at any\
+    \ time within the period of six months beginning with that date,\n\t2) The governing\
+    \ body may only terminate a person’s term of office under subsection (1) above\
+    \ if—\n\t\ta) his term of office is due to come to an end after the proposed date\
+    \ of publication of the proposals, or\n\t\tb) it would not in their view be reasonably\
+    \ practicable, in the time available between the date on which his term of office\
+    \ is due to come to an end and the proposed date of publication of the proposals,\
+    \ to fill the vacancy by the procedure applicable under the Education (No. 2)\
+    \ Act 1986.\n\t3) Without prejudice to section 8(2) of that Act (instrument of\
+    \ government for county, controlled or maintained special school to provide for\
+    \ four year term of office for governors other than ex officio governors), the\
+    \ term of office of a person elected or appointed in accordance with the requirements\
+    \ of that Act and any requirements of the instrument of government of the school\
+    \ to fill a vacancy arising by virtue of subsection (1) above shall be four years.\n\
+    \t4) Where any such election or appointment as is referred to in section 71(2)\
+    \ of this Act is held or made on or after the date of publication of the proposals,\
+    \ the existing governing body shall publish at such time and in such manner as\
+    \ may be prescribed notice of the election or appointment."
+  - "search_document: ### Section 90\n### Compensation for statements in listing particulars\
+    \ or prospectus\n\t1) Any person responsible for listing particulars is liable\
+    \ to pay compensation to a person who has—\n\t\ta) acquired securities to which\
+    \ the particulars apply; and\n\t\tb) suffered loss in respect of them as a result\
+    \ of—\n\t\t\ti) any untrue or misleading statement in the particulars; or\n\t\t\
+    \tii) the omission from the particulars of any matter required to be included\
+    \ by section 80 or 81.\n\t2) Subsection (1) is subject to exemptions provided\
+    \ by Schedule 10.\n\t3) If listing particulars are required to include information\
+    \ about the absence of a particular matter, the omission from the particulars\
+    \ of that information is to be treated as a statement in the listing particulars\
+    \ that there is no such matter.\n\t4) Any person who fails to comply with section\
+    \ 81 is liable to pay compensation to any person who has—\n\t\ta) acquired securities\
+    \ of the kind in question; and\n\t\tb) suffered loss in respect of them as a result\
+    \ of the failure.\n\t5) Subsection (4) is subject to exemptions provided by Schedule\
+    \ 10.\n\t6) This section does not affect any liability which may be incurred apart\
+    \ from this section.\n\t7) References in this section to the acquisition by a\
+    \ person of securities include references to his contracting to acquire them or\
+    \ any interest in them.\n\t8) No person shall, by reason of being a promoter of\
+    \ a company or otherwise, incur any liability for failing to disclose information\
+    \ which he would not be required to disclose in listing particulars in respect\
+    \ of a company’s securities—\n\t\ta) if he were responsible for those particulars;\
+    \ or\n\t\tb) if he is responsible for them, which he is entitled to omit by virtue\
+    \ of section 82.\n\t9) The reference in subsection (8) to a person incurring liability\
+    \ includes a reference to any other person being entitled as against that person\
+    \ to be granted any civil remedy or to rescind or repudiate an agreement.\n\t\
+    10) “Listing particulars”, in subsection (1) and Schedule 10, includes supplementary\
+    \ listing particulars.\n\t11) This section applies in relation to a prospectus\
+    \ as it applies to listing particulars, with the following modifications—\n\t\t\
+    a) references in this section or in Schedule 10 to listing particulars, supplementary\
+    \ listing particulars or sections 80, 81 or 82 are to be read, respectively, as\
+    \ references to a prospectus, supplementary prospectus and None ;\n\t\tb) references\
+    \ in Schedule 10 to admission to the official list are to be read as references\
+    \ to admission to trading on a regulated market;\n\t\tc) in relation to a prospectus,\
+    \  “ ” means  “transferable securities”.\n\tNone\n\t12) \n\t\tNone\n\t\tNone\n\
+    \t\t\ta) None\n\t\t\tb) None None None"
+  - "search_document: ### Section 1\n### Power to provide constitution for Zimbabwe.\n\
+    \t1) Her Majesty may by Order in Council provide a constitution for Zimbabwe to\
+    \ come into effect on the day (in this Act referred to as “the appointed day”)\
+    \ on which, in accordance with such provision in that behalf as may after the\
+    \ passing of this Act be made by Act of Parliament, Southern Rhodesia becomes\
+    \ independent as a Republic under the name of Zimbabwe.\n\t2) Her Majesty may\
+    \ by Order in Council revoke the Constitution of Southern Rhodesia 1961, and may\
+    \ make such transitional provision as appears to Her Majesty to be necessary or\
+    \ expedient in connection with the coming into effect of the new constitution\
+    \ or the revocation of the said Constitution of 1961.\n\t3) Any Order in Council\
+    \ under this section shall be laid before Parliament after being made.\n\t4) Subsection\
+    \ (1) is without prejudice to any power conferred on Her Majesty by section 2."
+- source_sentence: 'search_query: How is a poinding enforced on premises with existing
+    poinding for the same debt?'
+  sentences:
+  - "search_document: ### Section 15\n### Orders and directions.\n\t1) Any order under\
+    \ this Act shall be made by statutory instrument, and may be varied or revoked\
+    \ by a subsequent order so made.\n\t2) A statutory instrument containing an order\
+    \ made under section 1(1) or 6(3) above shall be subject to annulment in pursuance\
+    \ of a resolution of either House of Parliament.\n\t3) It is hereby declared that\
+    \ any direction given under this Act may be varied or revoked by a subsequent\
+    \ direction so given."
+  - "search_document: ### Section 70\n### Exemptions from section 69.\n\t1) Section\
+    \ 69(1) of this Act shall not apply in relation to a vehicle if—\n\t\ta) a current\
+    \ disabled person’s badge is displayed on the vehicle;\n\t\ta current recognised\
+    \ badge (within the meaning given by section 21A of the Chronically Sick and Disabled\
+    \ Persons Act 1970) is displayed on the vehicle;\n\t\tb) not more than 15 minutes\
+    \ have elapsed since the end of any period for which the appropriate charge was\
+    \ duly paid at the time of parking; or\n\t\tc) not more than 15 minutes have elapsed\
+    \ since the end of any unexpired time (in respect of another vehicle) which is\
+    \ available at the relevant parking meter at the time of parking.\n\t2) \n\t\t\
+    In any case in which section 69(1) of this Act would apply to a vehicle but for\
+    \ subsection (1)(a) above and the vehicle was not, at the time at which it was\
+    \ parked, being used—\n\t\t\ta) in accordance with regulations under section 21\
+    \ of the Chronically Sick and Disabled Persons Act 1970; and\n\t\t\tb) in circumstances\
+    \ falling within section 117(1)(b) of theRoad Traffic Regulation Act 1984 (use\
+    \ where a disabled person’s concession would be available),\n\t\tthe person in\
+    \ charge of the vehicle at that time shall be guilty of an offence and liable\
+    \ on summary conviction to a fine not exceeding level 3 on the standard scale.\n\
+    \t\n\t\tIn any case in which section 69(1) of this Act would apply to a vehicle\
+    \ but for subsection (1)(aa) above and the vehicle was not, at the time at which\
+    \ it was parked, being used—\n\t\t\tin accordance with regulations under section\
+    \ 21A of the Chronically Sick and Disabled Persons Act 1970, and\n\t\t\tin circumstances\
+    \ falling within section 117(1A)(b) of the Road Traffic Regulation Act 1984 (use\
+    \ where a disabled person’s concession would be available by virtue of displaying\
+    \ a non-GB badge),\n\t\tthe person in charge of the vehicle at that time shall\
+    \ be guilty of an offence and liable on summary conviction to a fine not exceeding\
+    \ level 3 on the standard scale.\n\t3) In this section “disabled person’s badge”\
+    \ has the same meaning as in section 142(1) of the Road Traffic Regulation Act\
+    \ 1984, and “parking meter” has the same meaning as in section 46(2)(a) of that\
+    \ Act."
+  - 'search_document: 9) Subject to paragraph 7(2) above and paragraphs 12(2) and
+    (6), 13(2), 21(4) and 22(5) below, where articles are poinded in any premises
+    (whether or not the poinding is valid), another poinding in those premises to
+    enforce the same debt shall not be competent except in relation to articles which
+    have been brought on to the premises since the execution of the first poinding.'
+---
+
+# SentenceTransformer based on nomic-ai/nomic-embed-text-v1.5
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [nomic-ai/nomic-embed-text-v1.5](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [nomic-ai/nomic-embed-text-v1.5](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5) <!-- at revision 679199c2575b5bfe93b06161d06cd7c16ebe4124 -->
+- **Maximum Sequence Length:** 8192 tokens
+- **Output Dimensionality:** 768 tokens
+- **Similarity Function:** Cosine Similarity
+<!-- - **Training Dataset:** Unknown -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertModel 
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("sentence_transformers_model_id")
+# Run inference
+sentences = [
+    'search_query: How is a poinding enforced on premises with existing poinding for the same debt?',
+    'search_document: 9) Subject to paragraph 7(2) above and paragraphs 12(2) and (6), 13(2), 21(4) and 22(5) below, where articles are poinded in any premises (whether or not the poinding is valid), another poinding in those premises to enforce the same debt shall not be competent except in relation to articles which have been brought on to the premises since the execution of the first poinding.',
+    'search_document: ### Section 15\n### Orders and directions.\n\t1) Any order under this Act shall be made by statutory instrument, and may be varied or revoked by a subsequent order so made.\n\t2) A statutory instrument containing an order made under section 1(1) or 6(3) above shall be subject to annulment in pursuance of a resolution of either House of Parliament.\n\t3) It is hereby declared that any direction given under this Act may be varied or revoked by a subsequent direction so given.',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 768]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### Unnamed Dataset
+
+
+* Size: 26,984 training samples
+* Columns: <code>anchor</code> and <code>positive</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | anchor                                                                            | positive                                                                             |
+  |:--------|:----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                               |
+  | details | <ul><li>min: 11 tokens</li><li>mean: 19.2 tokens</li><li>max: 40 tokens</li></ul> | <ul><li>min: 26 tokens</li><li>mean: 308.96 tokens</li><li>max: 994 tokens</li></ul> |
+* Samples:
+  | anchor                                                                                                                                 | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
+  |:---------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>search_query: What powers do universities and colleges have regarding their land ownership?</code>                               | <code>search_document: ### Section 24<br>### Amendments of the Universities and College Estates Act 1925<br>	1) The Universities and College Estates Act 1925 is amended in accordance with subsections  to (2)(6).<br>	2) After section 1 insert—<br>		General power over land<br>			1A) General power over land<br>				1) A university or college has in relation to land belonging to the university or college all the powers of an absolute owner.<br>				2) The power conferred by subsection  is subject to(1)—<br>					a) any restriction, condition or limitation imposed by, or arising under, any enactment,<br>					b) any rule of law or equity, or<br>					c) the statutes regulating the university or college.<br>	3) Omit sections 2 to 38 and Schedule 1 (provisions relating to land and the application of capital money).<br>	4) In section 40 (power to transfer to university or college), omit “with the consent of the Minister”.<br>	5) In section 42 (saving of existing powers), omit from “: Provided that” to the end.<br>	6) In section 43 (definitions)—<br>		a) in the opening words, omit from “unless” to “say”;<br>		b) omit sub-paragraphs (i), (ii), (viii) and (x).<br>	7) Schedule 1 (which contains consequential amendments) has effect.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+  | <code>search_query: What particulars must be registered for corporate directors and firms in a company's register of directors?</code> | <code>search_document: ### Section 164<br>### Particulars of directors to be registered: corporate directors and firms<br>A company's register of directors must contain the following particulars in the case of a body corporate, or a firm that is a legal person under the law by which it is governed—<br>	a) corporate or firm name;<br>	b) registered or principal office;<br>	in the case of a limited company that is a UK-registered company, the registered number;<br>	d) in any other case, particulars of—<br>		i) the legal form of the company or firm and the law by which it is governed, and<br>		ii) if applicable, the register in which it is entered (including details of the state) and its registration number in that register.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
+  | <code>search_query: Details on the continuation and effect of a criminal care order.</code>                                            | <code>search_document: 36) <br>	1) This paragraph applies where, immediately before the commencement of section 90(2) there was in force an order (“a criminal care order") made—<br>		a) under section 7(7)(a) of the Children and Young Persons Act 1969 (alteration in treatment of young offenders etc.); or<br>		b) under section 15(1) of that Act, on discharging a supervision order made under section 7(7)(b) of that Act.<br>	2) The criminal care order shall continue to have effect until the end of the period of six months beginning with the day on which section 90(2) comes into force unless it is brought to an end earlier in accordance with—<br>		a) the provisions of the Act of 1969 preserved by sub-paragraph (3)(a); or<br>		b) this paragraph.<br>	3) <br>		While the criminal care order remains in force, any relevant provisions—<br>			a) of the Act of 1969; and<br>			b) of the Child Care Act 1980,<br>		shall continue to have effect with respect to it.<br>	4) <br>		While the criminal care order remains in force, a court may, on the application of the appropriate person, make—<br>			a) a residence order;<br>			b) a care order or a supervision order under section 31;<br>			c) an education supervision order under section 36 (regardless of subsection (6) of that section); or<br>			d) an order falling within sub-paragraph (5),<br>		and shall, on making any of those orders, discharge the criminal care order.<br>	5) The order mentioned in sub-paragraph (4)(d) is an order having effect as if it were a supervision order of a kind mentioned in section 12AA of the Act of 1969 (as inserted by paragraph 23 of Schedule 12), that is to say, a supervision order—<br>		a) imposing a requirement that the child shall live for a specified period in local authority accommodation; but<br>		b) in relation to which the conditions mentioned in subsection (6) of section 12AA are not required to be satisfied.<br>	6) The maximum period which may be specified in an order made under sub-paragraph (4)(d) is six months and such an order may stipulate that the child shall not live with a named person.<br>	7) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .<br>	8) In sub-paragraph (4) “appropriate person” means—<br>		a) in the case of an application for a residence order, any person (other than a local authority) who has the leave of the court;<br>		b) in the case of an application for an education supervision order, a local education authority; and<br>		c) in any other case, the local authority to whose care the child was committed by the order.</code> |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim"
+  }
+  ```
+
+### Evaluation Dataset
+
+#### Unnamed Dataset
+
+
+* Size: 1,417 evaluation samples
+* Columns: <code>anchor</code> and <code>positive</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | anchor                                                                             | positive                                                                             |
+  |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
+  | type    | string                                                                             | string                                                                               |
+  | details | <ul><li>min: 12 tokens</li><li>mean: 19.41 tokens</li><li>max: 36 tokens</li></ul> | <ul><li>min: 23 tokens</li><li>mean: 295.19 tokens</li><li>max: 993 tokens</li></ul> |
+* Samples:
+  | anchor                                                                                                                                     | positive                                                                                                                                                                                                                                                                                                                               |
+  |:-------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>search_query: Can an order made under this legislation be questioned in any legal proceedings?</code>                                | <code>search_document: 7) Subject to the provisions of paragraph 6 above, an order under section 1 of this Act shall not, either before or after it has been made, be questioned in any legal proceedings whatsoever, and shall become operative on the date on which notice is first published as mentioned in that paragraph.</code> |
+  | <code>search_query: When did the Badgers Act 1991 come into force?</code>                                                                  | <code>search_document: ### Section 6<br>### Citation and commencement.<br>	1) This Act may be cited as the Badgers Act 1991.<br>	2) This Act shall come into force at the end of the period of three months beginning with the day on which it is passed.</code>                                                                       |
+  | <code>search_query: Under what circumstances do intangible fixed assets not apply to legislation regarding non-commercial purposes?</code> | <code>search_document: ### Section 803<br>### Non-commercial purposes etc<br>This Part does not apply to an intangible fixed asset so far as it is held—<br>	a) for a purpose that is not a business or other commercial purpose of the company, or<br>	b) , otherwise than as a result of Chapter 3A of Part 2</code>                 |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim"
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: epoch
+- `per_device_eval_batch_size`: 16
+- `learning_rate`: 3e-05
+- `warmup_ratio`: 0.1
+- `fp16`: True
+- `load_best_model_at_end`: True
+- `ddp_find_unused_parameters`: False
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: epoch
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 8
+- `per_device_eval_batch_size`: 16
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 3e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 3
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: True
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: True
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: False
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: False
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `eval_use_gather_object`: False
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+<details><summary>Click to expand</summary>
+
+| Epoch  | Step  | Training Loss | loss   |
+|:------:|:-----:|:-------------:|:------:|
+| 0.0074 | 25    | 0.07          | -      |
+| 0.0148 | 50    | 0.0304        | -      |
+| 0.0222 | 75    | 0.0184        | -      |
+| 0.0296 | 100   | 0.0019        | -      |
+| 0.0371 | 125   | 0.0103        | -      |
+| 0.0445 | 150   | 0.002         | -      |
+| 0.0519 | 175   | 0.0017        | -      |
+| 0.0593 | 200   | 0.0136        | -      |
+| 0.0667 | 225   | 0.0008        | -      |
+| 0.0741 | 250   | 0.0038        | -      |
+| 0.0815 | 275   | 0.0005        | -      |
+| 0.0889 | 300   | 0.0002        | -      |
+| 0.0964 | 325   | 0.0014        | -      |
+| 0.1038 | 350   | 0.0004        | -      |
+| 0.1112 | 375   | 0.0002        | -      |
+| 0.1186 | 400   | 0.0004        | -      |
+| 0.1260 | 425   | 0.0012        | -      |
+| 0.1334 | 450   | 0.0017        | -      |
+| 0.1408 | 475   | 0.0002        | -      |
+| 0.1482 | 500   | 0.0006        | -      |
+| 0.1556 | 525   | 0.0003        | -      |
+| 0.1631 | 550   | 0.0075        | -      |
+| 0.1705 | 575   | 0.0006        | -      |
+| 0.1779 | 600   | 0.0002        | -      |
+| 0.1853 | 625   | 0.0019        | -      |
+| 0.1927 | 650   | 0.0088        | -      |
+| 0.2001 | 675   | 0.0023        | -      |
+| 0.2075 | 700   | 0.0005        | -      |
+| 0.2149 | 725   | 0.0005        | -      |
+| 0.2224 | 750   | 0.0002        | -      |
+| 0.2298 | 775   | 0.003         | -      |
+| 0.2372 | 800   | 0.0006        | -      |
+| 0.2446 | 825   | 0.001         | -      |
+| 0.2520 | 850   | 0.0012        | -      |
+| 0.2594 | 875   | 0.001         | -      |
+| 0.2668 | 900   | 0.0004        | -      |
+| 0.2742 | 925   | 0.0016        | -      |
+| 0.2816 | 950   | 0.0002        | -      |
+| 0.2891 | 975   | 0.0008        | -      |
+| 0.2965 | 1000  | 0.0008        | -      |
+| 0.3039 | 1025  | 0.0016        | -      |
+| 0.3113 | 1050  | 0.0007        | -      |
+| 0.3187 | 1075  | 0.0206        | -      |
+| 0.3261 | 1100  | 0.0009        | -      |
+| 0.3335 | 1125  | 0.0007        | -      |
+| 0.3409 | 1150  | 0.0057        | -      |
+| 0.3484 | 1175  | 0.0004        | -      |
+| 0.3558 | 1200  | 0.0003        | -      |
+| 0.3632 | 1225  | 0.0022        | -      |
+| 0.3706 | 1250  | 0.0022        | -      |
+| 0.3780 | 1275  | 0.0014        | -      |
+| 0.3854 | 1300  | 0.0017        | -      |
+| 0.3928 | 1325  | 0.0048        | -      |
+| 0.4002 | 1350  | 0.0068        | -      |
+| 0.4076 | 1375  | 0.0012        | -      |
+| 0.4151 | 1400  | 0.0095        | -      |
+| 0.4225 | 1425  | 0.0014        | -      |
+| 0.4299 | 1450  | 0.0008        | -      |
+| 0.4373 | 1475  | 0.0091        | -      |
+| 0.4447 | 1500  | 0.002         | -      |
+| 0.4521 | 1525  | 0.0033        | -      |
+| 0.4595 | 1550  | 0.0027        | -      |
+| 0.4669 | 1575  | 0.0016        | -      |
+| 0.4744 | 1600  | 0.0003        | -      |
+| 0.4818 | 1625  | 0.0004        | -      |
+| 0.4892 | 1650  | 0.0045        | -      |
+| 0.4966 | 1675  | 0.0008        | -      |
+| 0.5040 | 1700  | 0.0009        | -      |
+| 0.5114 | 1725  | 0.0015        | -      |
+| 0.5188 | 1750  | 0.0149        | -      |
+| 0.5262 | 1775  | 0.0154        | -      |
+| 0.5336 | 1800  | 0.0036        | -      |
+| 0.5411 | 1825  | 0.0028        | -      |
+| 0.5485 | 1850  | 0.0033        | -      |
+| 0.5559 | 1875  | 0.0086        | -      |
+| 0.5633 | 1900  | 0.0124        | -      |
+| 0.5707 | 1925  | 0.0005        | -      |
+| 0.5781 | 1950  | 0.0037        | -      |
+| 0.5855 | 1975  | 0.0052        | -      |
+| 0.5929 | 2000  | 0.004         | -      |
+| 0.6004 | 2025  | 0.0041        | -      |
+| 0.6078 | 2050  | 0.0006        | -      |
+| 0.6152 | 2075  | 0.0035        | -      |
+| 0.6226 | 2100  | 0.0079        | -      |
+| 0.6300 | 2125  | 0.0012        | -      |
+| 0.6374 | 2150  | 0.0012        | -      |
+| 0.6448 | 2175  | 0.0035        | -      |
+| 0.6522 | 2200  | 0.0027        | -      |
+| 0.6597 | 2225  | 0.0015        | -      |
+| 0.6671 | 2250  | 0.0004        | -      |
+| 0.6745 | 2275  | 0.001         | -      |
+| 0.6819 | 2300  | 0.0002        | -      |
+| 0.6893 | 2325  | 0.0009        | -      |
+| 0.6967 | 2350  | 0.0019        | -      |
+| 0.7041 | 2375  | 0.0005        | -      |
+| 0.7115 | 2400  | 0.0008        | -      |
+| 0.7189 | 2425  | 0.0012        | -      |
+| 0.7264 | 2450  | 0.0036        | -      |
+| 0.7338 | 2475  | 0.0007        | -      |
+| 0.7412 | 2500  | 0.0036        | -      |
+| 0.7486 | 2525  | 0.0007        | -      |
+| 0.7560 | 2550  | 0.0008        | -      |
+| 0.7634 | 2575  | 0.0008        | -      |
+| 0.7708 | 2600  | 0.0016        | -      |
+| 0.7782 | 2625  | 0.008         | -      |
+| 0.7857 | 2650  | 0.0058        | -      |
+| 0.7931 | 2675  | 0.0003        | -      |
+| 0.8005 | 2700  | 0.0005        | -      |
+| 0.8079 | 2725  | 0.0101        | -      |
+| 0.8153 | 2750  | 0.0025        | -      |
+| 0.8227 | 2775  | 0.0041        | -      |
+| 0.8301 | 2800  | 0.0113        | -      |
+| 0.8375 | 2825  | 0.0219        | -      |
+| 0.8449 | 2850  | 0.0004        | -      |
+| 0.8524 | 2875  | 0.0062        | -      |
+| 0.8598 | 2900  | 0.0097        | -      |
+| 0.8672 | 2925  | 0.002         | -      |
+| 0.8746 | 2950  | 0.0093        | -      |
+| 0.8820 | 2975  | 0.0046        | -      |
+| 0.8894 | 3000  | 0.0009        | -      |
+| 0.8968 | 3025  | 0.0014        | -      |
+| 0.9042 | 3050  | 0.0021        | -      |
+| 0.9117 | 3075  | 0.0009        | -      |
+| 0.9191 | 3100  | 0.0082        | -      |
+| 0.9265 | 3125  | 0.0016        | -      |
+| 0.9339 | 3150  | 0.0023        | -      |
+| 0.9413 | 3175  | 0.0019        | -      |
+| 0.9487 | 3200  | 0.002         | -      |
+| 0.9561 | 3225  | 0.0009        | -      |
+| 0.9635 | 3250  | 0.0111        | -      |
+| 0.9709 | 3275  | 0.0137        | -      |
+| 0.9784 | 3300  | 0.0038        | -      |
+| 0.9858 | 3325  | 0.0061        | -      |
+| 0.9932 | 3350  | 0.0045        | -      |
+| 1.0    | 3373  | -             | 0.0038 |
+| 1.0006 | 3375  | 0.0004        | -      |
+| 1.0080 | 3400  | 0.0104        | -      |
+| 1.0154 | 3425  | 0.0033        | -      |
+| 1.0228 | 3450  | 0.0004        | -      |
+| 1.0302 | 3475  | 0.0065        | -      |
+| 1.0377 | 3500  | 0.004         | -      |
+| 1.0451 | 3525  | 0.0003        | -      |
+| 1.0525 | 3550  | 0.0004        | -      |
+| 1.0599 | 3575  | 0.0006        | -      |
+| 1.0673 | 3600  | 0.0018        | -      |
+| 1.0747 | 3625  | 0.0246        | -      |
+| 1.0821 | 3650  | 0.007         | -      |
+| 1.0895 | 3675  | 0.0002        | -      |
+| 1.0969 | 3700  | 0.0005        | -      |
+| 1.1044 | 3725  | 0.0097        | -      |
+| 1.1118 | 3750  | 0.0011        | -      |
+| 1.1192 | 3775  | 0.0005        | -      |
+| 1.1266 | 3800  | 0.0015        | -      |
+| 1.1340 | 3825  | 0.0009        | -      |
+| 1.1414 | 3850  | 0.0002        | -      |
+| 1.1488 | 3875  | 0.0002        | -      |
+| 1.1562 | 3900  | 0.0021        | -      |
+| 1.1637 | 3925  | 0.0003        | -      |
+| 1.1711 | 3950  | 0.0006        | -      |
+| 1.1785 | 3975  | 0.0019        | -      |
+| 1.1859 | 4000  | 0.0013        | -      |
+| 1.1933 | 4025  | 0.002         | -      |
+| 1.2007 | 4050  | 0.0005        | -      |
+| 1.2081 | 4075  | 0.0037        | -      |
+| 1.2155 | 4100  | 0.0013        | -      |
+| 1.2229 | 4125  | 0.0006        | -      |
+| 1.2304 | 4150  | 0.0037        | -      |
+| 1.2378 | 4175  | 0.0018        | -      |
+| 1.2452 | 4200  | 0.0005        | -      |
+| 1.2526 | 4225  | 0.0003        | -      |
+| 1.2600 | 4250  | 0.0003        | -      |
+| 1.2674 | 4275  | 0.0003        | -      |
+| 1.2748 | 4300  | 0.0004        | -      |
+| 1.2822 | 4325  | 0.0029        | -      |
+| 1.2897 | 4350  | 0.0021        | -      |
+| 1.2971 | 4375  | 0.0004        | -      |
+| 1.3045 | 4400  | 0.0008        | -      |
+| 1.3119 | 4425  | 0.0002        | -      |
+| 1.3193 | 4450  | 0.0031        | -      |
+| 1.3267 | 4475  | 0.0005        | -      |
+| 1.3341 | 4500  | 0.0006        | -      |
+| 1.3415 | 4525  | 0.0005        | -      |
+| 1.3489 | 4550  | 0.0012        | -      |
+| 1.3564 | 4575  | 0.0009        | -      |
+| 1.3638 | 4600  | 0.0006        | -      |
+| 1.3712 | 4625  | 0.0237        | -      |
+| 1.3786 | 4650  | 0.0004        | -      |
+| 1.3860 | 4675  | 0.0001        | -      |
+| 1.3934 | 4700  | 0.0005        | -      |
+| 1.4008 | 4725  | 0.0003        | -      |
+| 1.4082 | 4750  | 0.0059        | -      |
+| 1.4157 | 4775  | 0.0005        | -      |
+| 1.4231 | 4800  | 0.0014        | -      |
+| 1.4305 | 4825  | 0.0001        | -      |
+| 1.4379 | 4850  | 0.0003        | -      |
+| 1.4453 | 4875  | 0.0003        | -      |
+| 1.4527 | 4900  | 0.0062        | -      |
+| 1.4601 | 4925  | 0.0002        | -      |
+| 1.4675 | 4950  | 0.0004        | -      |
+| 1.4749 | 4975  | 0.0001        | -      |
+| 1.4824 | 5000  | 0.0003        | -      |
+| 1.4898 | 5025  | 0.0003        | -      |
+| 1.4972 | 5050  | 0.0004        | -      |
+| 1.5046 | 5075  | 0.0003        | -      |
+| 1.5120 | 5100  | 0.0001        | -      |
+| 1.5194 | 5125  | 0.0031        | -      |
+| 1.5268 | 5150  | 0.0002        | -      |
+| 1.5342 | 5175  | 0.0006        | -      |
+| 1.5417 | 5200  | 0.0001        | -      |
+| 1.5491 | 5225  | 0.0017        | -      |
+| 1.5565 | 5250  | 0.0163        | -      |
+| 1.5639 | 5275  | 0.0049        | -      |
+| 1.5713 | 5300  | 0.0029        | -      |
+| 1.5787 | 5325  | 0.0035        | -      |
+| 1.5861 | 5350  | 0.007         | -      |
+| 1.5935 | 5375  | 0.0006        | -      |
+| 1.6009 | 5400  | 0.0003        | -      |
+| 1.6084 | 5425  | 0.0004        | -      |
+| 1.6158 | 5450  | 0.0067        | -      |
+| 1.6232 | 5475  | 0.0112        | -      |
+| 1.6306 | 5500  | 0.0002        | -      |
+| 1.6380 | 5525  | 0.0005        | -      |
+| 1.6454 | 5550  | 0.0029        | -      |
+| 1.6528 | 5575  | 0.0006        | -      |
+| 1.6602 | 5600  | 0.0012        | -      |
+| 1.6677 | 5625  | 0.0002        | -      |
+| 1.6751 | 5650  | 0.0005        | -      |
+| 1.6825 | 5675  | 0.0002        | -      |
+| 1.6899 | 5700  | 0.0002        | -      |
+| 1.6973 | 5725  | 0.0003        | -      |
+| 1.7047 | 5750  | 0.0018        | -      |
+| 1.7121 | 5775  | 0.0027        | -      |
+| 1.7195 | 5800  | 0.0002        | -      |
+| 1.7269 | 5825  | 0.0001        | -      |
+| 1.7344 | 5850  | 0.0007        | -      |
+| 1.7418 | 5875  | 0.0047        | -      |
+| 1.7492 | 5900  | 0.0001        | -      |
+| 1.7566 | 5925  | 0.0002        | -      |
+| 1.7640 | 5950  | 0.0007        | -      |
+| 1.7714 | 5975  | 0.0039        | -      |
+| 1.7788 | 6000  | 0.0001        | -      |
+| 1.7862 | 6025  | 0.0001        | -      |
+| 1.7937 | 6050  | 0.0002        | -      |
+| 1.8011 | 6075  | 0.0001        | -      |
+| 1.8085 | 6100  | 0.0016        | -      |
+| 1.8159 | 6125  | 0.0004        | -      |
+| 1.8233 | 6150  | 0.0003        | -      |
+| 1.8307 | 6175  | 0.0005        | -      |
+| 1.8381 | 6200  | 0.0005        | -      |
+| 1.8455 | 6225  | 0.0004        | -      |
+| 1.8529 | 6250  | 0.0036        | -      |
+| 1.8604 | 6275  | 0.0002        | -      |
+| 1.8678 | 6300  | 0.0006        | -      |
+| 1.8752 | 6325  | 0.0001        | -      |
+| 1.8826 | 6350  | 0.0022        | -      |
+| 1.8900 | 6375  | 0.0001        | -      |
+| 1.8974 | 6400  | 0.0003        | -      |
+| 1.9048 | 6425  | 0.0001        | -      |
+| 1.9122 | 6450  | 0.001         | -      |
+| 1.9197 | 6475  | 0.0005        | -      |
+| 1.9271 | 6500  | 0.0002        | -      |
+| 1.9345 | 6525  | 0.0049        | -      |
+| 1.9419 | 6550  | 0.0002        | -      |
+| 1.9493 | 6575  | 0.0009        | -      |
+| 1.9567 | 6600  | 0.0145        | -      |
+| 1.9641 | 6625  | 0.0004        | -      |
+| 1.9715 | 6650  | 0.0009        | -      |
+| 1.9790 | 6675  | 0.0003        | -      |
+| 1.9864 | 6700  | 0.0053        | -      |
+| 1.9938 | 6725  | 0.0002        | -      |
+| 2.0    | 6746  | -             | 0.0012 |
+| 2.0012 | 6750  | 0.0002        | -      |
+| 2.0086 | 6775  | 0.0018        | -      |
+| 2.0160 | 6800  | 0.0001        | -      |
+| 2.0234 | 6825  | 0.0084        | -      |
+| 2.0308 | 6850  | 0.0004        | -      |
+| 2.0382 | 6875  | 0.0001        | -      |
+| 2.0457 | 6900  | 0.0001        | -      |
+| 2.0531 | 6925  | 0.0023        | -      |
+| 2.0605 | 6950  | 0.0006        | -      |
+| 2.0679 | 6975  | 0.0002        | -      |
+| 2.0753 | 7000  | 0.0025        | -      |
+| 2.0827 | 7025  | 0.0001        | -      |
+| 2.0901 | 7050  | 0.0003        | -      |
+| 2.0975 | 7075  | 0.0003        | -      |
+| 2.1050 | 7100  | 0.0004        | -      |
+| 2.1124 | 7125  | 0.0002        | -      |
+| 2.1198 | 7150  | 0.0137        | -      |
+| 2.1272 | 7175  | 0.0002        | -      |
+| 2.1346 | 7200  | 0.0001        | -      |
+| 2.1420 | 7225  | 0.0002        | -      |
+| 2.1494 | 7250  | 0.0002        | -      |
+| 2.1568 | 7275  | 0.0002        | -      |
+| 2.1642 | 7300  | 0.0002        | -      |
+| 2.1717 | 7325  | 0.0001        | -      |
+| 2.1791 | 7350  | 0.0003        | -      |
+| 2.1865 | 7375  | 0.0002        | -      |
+| 2.1939 | 7400  | 0.0002        | -      |
+| 2.2013 | 7425  | 0.0003        | -      |
+| 2.2087 | 7450  | 0.0109        | -      |
+| 2.2161 | 7475  | 0.0003        | -      |
+| 2.2235 | 7500  | 0.0006        | -      |
+| 2.2310 | 7525  | 0.0009        | -      |
+| 2.2384 | 7550  | 0.0004        | -      |
+| 2.2458 | 7575  | 0.0004        | -      |
+| 2.2532 | 7600  | 0.0002        | -      |
+| 2.2606 | 7625  | 0.0003        | -      |
+| 2.2680 | 7650  | 0.0001        | -      |
+| 2.2754 | 7675  | 0.0002        | -      |
+| 2.2828 | 7700  | 0.0           | -      |
+| 2.2902 | 7725  | 0.0006        | -      |
+| 2.2977 | 7750  | 0.0001        | -      |
+| 2.3051 | 7775  | 0.0001        | -      |
+| 2.3125 | 7800  | 0.0105        | -      |
+| 2.3199 | 7825  | 0.0001        | -      |
+| 2.3273 | 7850  | 0.0001        | -      |
+| 2.3347 | 7875  | 0.0001        | -      |
+| 2.3421 | 7900  | 0.0005        | -      |
+| 2.3495 | 7925  | 0.0023        | -      |
+| 2.3570 | 7950  | 0.0           | -      |
+| 2.3644 | 7975  | 0.0002        | -      |
+| 2.3718 | 8000  | 0.0003        | -      |
+| 2.3792 | 8025  | 0.0001        | -      |
+| 2.3866 | 8050  | 0.0001        | -      |
+| 2.3940 | 8075  | 0.0001        | -      |
+| 2.4014 | 8100  | 0.0001        | -      |
+| 2.4088 | 8125  | 0.0           | -      |
+| 2.4162 | 8150  | 0.0002        | -      |
+| 2.4237 | 8175  | 0.0001        | -      |
+| 2.4311 | 8200  | 0.0001        | -      |
+| 2.4385 | 8225  | 0.0001        | -      |
+| 2.4459 | 8250  | 0.0001        | -      |
+| 2.4533 | 8275  | 0.0001        | -      |
+| 2.4607 | 8300  | 0.001         | -      |
+| 2.4681 | 8325  | 0.0003        | -      |
+| 2.4755 | 8350  | 0.0003        | -      |
+| 2.4830 | 8375  | 0.0001        | -      |
+| 2.4904 | 8400  | 0.0009        | -      |
+| 2.4978 | 8425  | 0.0011        | -      |
+| 2.5052 | 8450  | 0.0005        | -      |
+| 2.5126 | 8475  | 0.0024        | -      |
+| 2.5200 | 8500  | 0.0002        | -      |
+| 2.5274 | 8525  | 0.0006        | -      |
+| 2.5348 | 8550  | 0.0001        | -      |
+| 2.5422 | 8575  | 0.0001        | -      |
+| 2.5497 | 8600  | 0.0003        | -      |
+| 2.5571 | 8625  | 0.0007        | -      |
+| 2.5645 | 8650  | 0.0009        | -      |
+| 2.5719 | 8675  | 0.0002        | -      |
+| 2.5793 | 8700  | 0.0001        | -      |
+| 2.5867 | 8725  | 0.0006        | -      |
+| 2.5941 | 8750  | 0.0           | -      |
+| 2.6015 | 8775  | 0.0002        | -      |
+| 2.6090 | 8800  | 0.0           | -      |
+| 2.6164 | 8825  | 0.0004        | -      |
+| 2.6238 | 8850  | 0.0001        | -      |
+| 2.6312 | 8875  | 0.0005        | -      |
+| 2.6386 | 8900  | 0.0002        | -      |
+| 2.6460 | 8925  | 0.0001        | -      |
+| 2.6534 | 8950  | 0.0001        | -      |
+| 2.6608 | 8975  | 0.0001        | -      |
+| 2.6682 | 9000  | 0.0           | -      |
+| 2.6757 | 9025  | 0.0004        | -      |
+| 2.6831 | 9050  | 0.0002        | -      |
+| 2.6905 | 9075  | 0.0001        | -      |
+| 2.6979 | 9100  | 0.0001        | -      |
+| 2.7053 | 9125  | 0.0           | -      |
+| 2.7127 | 9150  | 0.0002        | -      |
+| 2.7201 | 9175  | 0.0014        | -      |
+| 2.7275 | 9200  | 0.0003        | -      |
+| 2.7350 | 9225  | 0.0006        | -      |
+| 2.7424 | 9250  | 0.0002        | -      |
+| 2.7498 | 9275  | 0.001         | -      |
+| 2.7572 | 9300  | 0.0002        | -      |
+| 2.7646 | 9325  | 0.0002        | -      |
+| 2.7720 | 9350  | 0.0021        | -      |
+| 2.7794 | 9375  | 0.0001        | -      |
+| 2.7868 | 9400  | 0.0033        | -      |
+| 2.7942 | 9425  | 0.0           | -      |
+| 2.8017 | 9450  | 0.0005        | -      |
+| 2.8091 | 9475  | 0.0002        | -      |
+| 2.8165 | 9500  | 0.0001        | -      |
+| 2.8239 | 9525  | 0.0002        | -      |
+| 2.8313 | 9550  | 0.0006        | -      |
+| 2.8387 | 9575  | 0.0002        | -      |
+| 2.8461 | 9600  | 0.0001        | -      |
+| 2.8535 | 9625  | 0.0           | -      |
+| 2.8610 | 9650  | 0.0001        | -      |
+| 2.8684 | 9675  | 0.0003        | -      |
+| 2.8758 | 9700  | 0.0011        | -      |
+| 2.8832 | 9725  | 0.0024        | -      |
+| 2.8906 | 9750  | 0.0002        | -      |
+| 2.8980 | 9775  | 0.0001        | -      |
+| 2.9054 | 9800  | 0.0001        | -      |
+| 2.9128 | 9825  | 0.0002        | -      |
+| 2.9202 | 9850  | 0.0001        | -      |
+| 2.9277 | 9875  | 0.0013        | -      |
+| 2.9351 | 9900  | 0.0           | -      |
+| 2.9425 | 9925  | 0.0002        | -      |
+| 2.9499 | 9950  | 0.0           | -      |
+| 2.9573 | 9975  | 0.0           | -      |
+| 2.9647 | 10000 | 0.0002        | -      |
+| 2.9721 | 10025 | 0.0001        | -      |
+| 2.9795 | 10050 | 0.0001        | -      |
+| 2.9870 | 10075 | 0.0           | -      |
+| 2.9944 | 10100 | 0.0002        | -      |
+| 3.0    | 10119 | -             | 0.0012 |
+
+</details>
+
+### Framework Versions
+- Python: 3.10.14
+- Sentence Transformers: 3.0.1
+- Transformers: 4.44.1
+- PyTorch: 2.3.0
+- Accelerate: 0.33.0
+- Datasets: 2.19.1
+- Tokenizers: 0.19.1
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### MultipleNegativesRankingLoss
+```bibtex
+@misc{henderson2017efficient,
+    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
+    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
+    year={2017},
+    eprint={1705.00652},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

checkpoint-10119/config.json

@@ -0,0 +1,58 @@
+{
+  "_name_or_path": "nomic-ai/nomic-embed-text-v1.5",
+  "activation_function": "swiglu",
+  "architectures": [
+    "NomicBertModel"
+  ],
+  "attn_pdrop": 0.0,
+  "auto_map": {
+    "AutoConfig": "nomic-ai/nomic-bert-2048--configuration_hf_nomic_bert.NomicBertConfig",
+    "AutoModel": "nomic-ai/nomic-bert-2048--modeling_hf_nomic_bert.NomicBertModel",
+    "AutoModelForMaskedLM": "nomic-ai/nomic-bert-2048--modeling_hf_nomic_bert.NomicBertForPreTraining"
+  },
+  "bos_token_id": null,
+  "causal": false,
+  "dense_seq_output": true,
+  "embd_pdrop": 0.0,
+  "eos_token_id": null,
+  "fused_bias_fc": true,
+  "fused_dropout_add_ln": true,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-12,
+  "max_trained_positions": 2048,
+  "mlp_fc1_bias": false,
+  "mlp_fc2_bias": false,
+  "model_type": "nomic_bert",
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": 3072,
+  "n_layer": 12,
+  "n_positions": 8192,
+  "pad_vocab_size_multiple": 64,
+  "parallel_block": false,
+  "parallel_block_tied_norm": false,
+  "prenorm": false,
+  "qkv_proj_bias": false,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.0,
+  "rotary_emb_base": 1000,
+  "rotary_emb_fraction": 1.0,
+  "rotary_emb_interleaved": false,
+  "rotary_emb_scale_base": null,
+  "rotary_scaling_factor": null,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.0,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.44.1",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "use_flash_attn": true,
+  "use_rms_norm": false,
+  "use_xentropy": true,
+  "vocab_size": 30528
+}

checkpoint-10119/config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.0.1",
+    "transformers": "4.44.1",
+    "pytorch": "2.3.0"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": null
+}

checkpoint-10119/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:759cbec716931ff76cdbd4a7871383f735a315ed6d81bcd06b39a1b086f4ffaf
+size 546938168

checkpoint-10119/modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

checkpoint-10119/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63589318d8cdc24ec044c6e074813cd7098f9fe4265eef0156ef85184886bbf4
+size 1093947386

checkpoint-10119/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:efe90f99c39bdf981bce1b64e4354d2e88ce3ca786c2d8bbcc0d1dfd0ab8057f
+size 14244

checkpoint-10119/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f1acdf701a5c9298adf7b2791b4bb6332200dc7cd3808ee627e187b1eae0b9b
+size 1064

checkpoint-10119/sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 8192,
+  "do_lower_case": false
+}

checkpoint-10119/special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-10119/tokenizer_config.json

@@ -0,0 +1,55 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 8192,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

checkpoint-10119/trainer_state.json

@@ -0,0 +1,2894 @@
+{
+  "best_metric": 0.0011504614958539605,
+  "best_model_checkpoint": "autotrain-7flvh-khn72/checkpoint-10119",
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 10119,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.007411799584939223,
+      "grad_norm": 7.738826751708984,
+      "learning_rate": 7.411067193675889e-07,
+      "loss": 0.07,
+      "step": 25
+    },
+    {
+      "epoch": 0.014823599169878446,
+      "grad_norm": 1.3767492771148682,
+      "learning_rate": 1.4822134387351778e-06,
+      "loss": 0.0304,
+      "step": 50
+    },
+    {
+      "epoch": 0.02223539875481767,
+      "grad_norm": 0.9382590055465698,
+      "learning_rate": 2.193675889328063e-06,
+      "loss": 0.0184,
+      "step": 75
+    },
+    {
+      "epoch": 0.029647198339756892,
+      "grad_norm": 0.22179915010929108,
+      "learning_rate": 2.9347826086956523e-06,
+      "loss": 0.0019,
+      "step": 100
+    },
+    {
+      "epoch": 0.037058997924696115,
+      "grad_norm": 0.022122476249933243,
+      "learning_rate": 3.6758893280632412e-06,
+      "loss": 0.0103,
+      "step": 125
+    },
+    {
+      "epoch": 0.04447079750963534,
+      "grad_norm": 0.049805257469415665,
+      "learning_rate": 4.416996047430831e-06,
+      "loss": 0.002,
+      "step": 150
+    },
+    {
+      "epoch": 0.051882597094574566,
+      "grad_norm": 0.30325883626937866,
+      "learning_rate": 5.158102766798419e-06,
+      "loss": 0.0017,
+      "step": 175
+    },
+    {
+      "epoch": 0.059294396679513785,
+      "grad_norm": 0.013520199805498123,
+      "learning_rate": 5.899209486166008e-06,
+      "loss": 0.0136,
+      "step": 200
+    },
+    {
+      "epoch": 0.06670619626445301,
+      "grad_norm": 0.08144009858369827,
+      "learning_rate": 6.640316205533597e-06,
+      "loss": 0.0008,
+      "step": 225
+    },
+    {
+      "epoch": 0.07411799584939223,
+      "grad_norm": 0.03251965716481209,
+      "learning_rate": 7.381422924901186e-06,
+      "loss": 0.0038,
+      "step": 250
+    },
+    {
+      "epoch": 0.08152979543433146,
+      "grad_norm": 0.009104475378990173,
+      "learning_rate": 8.122529644268774e-06,
+      "loss": 0.0005,
+      "step": 275
+    },
+    {
+      "epoch": 0.08894159501927068,
+      "grad_norm": 0.004219604656100273,
+      "learning_rate": 8.863636363636365e-06,
+      "loss": 0.0002,
+      "step": 300
+    },
+    {
+      "epoch": 0.0963533946042099,
+      "grad_norm": 0.006504200864583254,
+      "learning_rate": 9.604743083003952e-06,
+      "loss": 0.0014,
+      "step": 325
+    },
+    {
+      "epoch": 0.10376519418914913,
+      "grad_norm": 0.004516596905887127,
+      "learning_rate": 1.0345849802371542e-05,
+      "loss": 0.0004,
+      "step": 350
+    },
+    {
+      "epoch": 0.11117699377408835,
+      "grad_norm": 0.0006104411440901458,
+      "learning_rate": 1.108695652173913e-05,
+      "loss": 0.0002,
+      "step": 375
+    },
+    {
+      "epoch": 0.11858879335902757,
+      "grad_norm": 0.035294629633426666,
+      "learning_rate": 1.182806324110672e-05,
+      "loss": 0.0004,
+      "step": 400
+    },
+    {
+      "epoch": 0.1260005929439668,
+      "grad_norm": 0.019285837188363075,
+      "learning_rate": 1.256916996047431e-05,
+      "loss": 0.0012,
+      "step": 425
+    },
+    {
+      "epoch": 0.13341239252890602,
+      "grad_norm": 0.3558584451675415,
+      "learning_rate": 1.3310276679841897e-05,
+      "loss": 0.0017,
+      "step": 450
+    },
+    {
+      "epoch": 0.14082419211384525,
+      "grad_norm": 0.059573035687208176,
+      "learning_rate": 1.4051383399209487e-05,
+      "loss": 0.0002,
+      "step": 475
+    },
+    {
+      "epoch": 0.14823599169878446,
+      "grad_norm": 0.18277816474437714,
+      "learning_rate": 1.4792490118577074e-05,
+      "loss": 0.0006,
+      "step": 500
+    },
+    {
+      "epoch": 0.1556477912837237,
+      "grad_norm": 0.059150487184524536,
+      "learning_rate": 1.5533596837944665e-05,
+      "loss": 0.0003,
+      "step": 525
+    },
+    {
+      "epoch": 0.16305959086866292,
+      "grad_norm": 0.0057366532273590565,
+      "learning_rate": 1.6274703557312253e-05,
+      "loss": 0.0075,
+      "step": 550
+    },
+    {
+      "epoch": 0.17047139045360213,
+      "grad_norm": 0.00453445827588439,
+      "learning_rate": 1.701581027667984e-05,
+      "loss": 0.0006,
+      "step": 575
+    },
+    {
+      "epoch": 0.17788319003854136,
+      "grad_norm": 0.0051128678023815155,
+      "learning_rate": 1.7756916996047432e-05,
+      "loss": 0.0002,
+      "step": 600
+    },
+    {
+      "epoch": 0.1852949896234806,
+      "grad_norm": 0.04630223289132118,
+      "learning_rate": 1.849802371541502e-05,
+      "loss": 0.0019,
+      "step": 625
+    },
+    {
+      "epoch": 0.1927067892084198,
+      "grad_norm": 0.7785825729370117,
+      "learning_rate": 1.9239130434782607e-05,
+      "loss": 0.0088,
+      "step": 650
+    },
+    {
+      "epoch": 0.20011858879335903,
+      "grad_norm": 0.48438361287117004,
+      "learning_rate": 1.9980237154150198e-05,
+      "loss": 0.0023,
+      "step": 675
+    },
+    {
+      "epoch": 0.20753038837829826,
+      "grad_norm": 0.01803724654018879,
+      "learning_rate": 2.0721343873517786e-05,
+      "loss": 0.0005,
+      "step": 700
+    },
+    {
+      "epoch": 0.21494218796323747,
+      "grad_norm": 0.009904771111905575,
+      "learning_rate": 2.1462450592885377e-05,
+      "loss": 0.0005,
+      "step": 725
+    },
+    {
+      "epoch": 0.2223539875481767,
+      "grad_norm": 0.02594936452805996,
+      "learning_rate": 2.2203557312252964e-05,
+      "loss": 0.0002,
+      "step": 750
+    },
+    {
+      "epoch": 0.22976578713311593,
+      "grad_norm": 0.0479484386742115,
+      "learning_rate": 2.2944664031620555e-05,
+      "loss": 0.003,
+      "step": 775
+    },
+    {
+      "epoch": 0.23717758671805514,
+      "grad_norm": 0.07394908368587494,
+      "learning_rate": 2.3685770750988143e-05,
+      "loss": 0.0006,
+      "step": 800
+    },
+    {
+      "epoch": 0.24458938630299437,
+      "grad_norm": 0.00584772601723671,
+      "learning_rate": 2.442687747035573e-05,
+      "loss": 0.001,
+      "step": 825
+    },
+    {
+      "epoch": 0.2520011858879336,
+      "grad_norm": 0.5621177554130554,
+      "learning_rate": 2.516798418972332e-05,
+      "loss": 0.0012,
+      "step": 850
+    },
+    {
+      "epoch": 0.25941298547287284,
+      "grad_norm": 0.03085828199982643,
+      "learning_rate": 2.590909090909091e-05,
+      "loss": 0.001,
+      "step": 875
+    },
+    {
+      "epoch": 0.26682478505781204,
+      "grad_norm": 0.004033543635159731,
+      "learning_rate": 2.6650197628458497e-05,
+      "loss": 0.0004,
+      "step": 900
+    },
+    {
+      "epoch": 0.27423658464275125,
+      "grad_norm": 0.015690704807639122,
+      "learning_rate": 2.7391304347826085e-05,
+      "loss": 0.0016,
+      "step": 925
+    },
+    {
+      "epoch": 0.2816483842276905,
+      "grad_norm": 0.026539193466305733,
+      "learning_rate": 2.813241106719368e-05,
+      "loss": 0.0002,
+      "step": 950
+    },
+    {
+      "epoch": 0.2890601838126297,
+      "grad_norm": 0.0341588519513607,
+      "learning_rate": 2.8873517786561267e-05,
+      "loss": 0.0008,
+      "step": 975
+    },
+    {
+      "epoch": 0.2964719833975689,
+      "grad_norm": 0.0481799840927124,
+      "learning_rate": 2.9614624505928854e-05,
+      "loss": 0.0008,
+      "step": 1000
+    },
+    {
+      "epoch": 0.3038837829825082,
+      "grad_norm": 0.028982171788811684,
+      "learning_rate": 2.9960469968156362e-05,
+      "loss": 0.0016,
+      "step": 1025
+    },
+    {
+      "epoch": 0.3112955825674474,
+      "grad_norm": 0.007785373833030462,
+      "learning_rate": 2.9878115735148788e-05,
+      "loss": 0.0007,
+      "step": 1050
+    },
+    {
+      "epoch": 0.3187073821523866,
+      "grad_norm": 0.16795194149017334,
+      "learning_rate": 2.9799055671461514e-05,
+      "loss": 0.0206,
+      "step": 1075
+    },
+    {
+      "epoch": 0.32611918173732585,
+      "grad_norm": 0.006599959917366505,
+      "learning_rate": 2.9716701438453936e-05,
+      "loss": 0.0009,
+      "step": 1100
+    },
+    {
+      "epoch": 0.33353098132226505,
+      "grad_norm": 0.005065989214926958,
+      "learning_rate": 2.963434720544636e-05,
+      "loss": 0.0007,
+      "step": 1125
+    },
+    {
+      "epoch": 0.34094278090720426,
+      "grad_norm": 0.06960754096508026,
+      "learning_rate": 2.9551992972438783e-05,
+      "loss": 0.0057,
+      "step": 1150
+    },
+    {
+      "epoch": 0.3483545804921435,
+      "grad_norm": 0.0384642519056797,
+      "learning_rate": 2.9469638739431205e-05,
+      "loss": 0.0004,
+      "step": 1175
+    },
+    {
+      "epoch": 0.3557663800770827,
+      "grad_norm": 0.005713630933314562,
+      "learning_rate": 2.938728450642363e-05,
+      "loss": 0.0003,
+      "step": 1200
+    },
+    {
+      "epoch": 0.3631781796620219,
+      "grad_norm": 0.011937534436583519,
+      "learning_rate": 2.9304930273416055e-05,
+      "loss": 0.0022,
+      "step": 1225
+    },
+    {
+      "epoch": 0.3705899792469612,
+      "grad_norm": 0.007326376158744097,
+      "learning_rate": 2.9222576040408477e-05,
+      "loss": 0.0022,
+      "step": 1250
+    },
+    {
+      "epoch": 0.3780017788319004,
+      "grad_norm": 0.016074607148766518,
+      "learning_rate": 2.9140221807400903e-05,
+      "loss": 0.0014,
+      "step": 1275
+    },
+    {
+      "epoch": 0.3854135784168396,
+      "grad_norm": 0.010498239658772945,
+      "learning_rate": 2.9057867574393324e-05,
+      "loss": 0.0017,
+      "step": 1300
+    },
+    {
+      "epoch": 0.39282537800177886,
+      "grad_norm": 0.14429619908332825,
+      "learning_rate": 2.8975513341385746e-05,
+      "loss": 0.0048,
+      "step": 1325
+    },
+    {
+      "epoch": 0.40023717758671806,
+      "grad_norm": 0.0035154372453689575,
+      "learning_rate": 2.889315910837817e-05,
+      "loss": 0.0068,
+      "step": 1350
+    },
+    {
+      "epoch": 0.40764897717165727,
+      "grad_norm": 0.05379508063197136,
+      "learning_rate": 2.8810804875370593e-05,
+      "loss": 0.0012,
+      "step": 1375
+    },
+    {
+      "epoch": 0.41506077675659653,
+      "grad_norm": 0.002669960493221879,
+      "learning_rate": 2.8728450642363015e-05,
+      "loss": 0.0095,
+      "step": 1400
+    },
+    {
+      "epoch": 0.42247257634153573,
+      "grad_norm": 0.021320341154932976,
+      "learning_rate": 2.8646096409355444e-05,
+      "loss": 0.0014,
+      "step": 1425
+    },
+    {
+      "epoch": 0.42988437592647494,
+      "grad_norm": 0.7060187458992004,
+      "learning_rate": 2.8563742176347866e-05,
+      "loss": 0.0008,
+      "step": 1450
+    },
+    {
+      "epoch": 0.4372961755114142,
+      "grad_norm": 0.9327678680419922,
+      "learning_rate": 2.8481387943340288e-05,
+      "loss": 0.0091,
+      "step": 1475
+    },
+    {
+      "epoch": 0.4447079750963534,
+      "grad_norm": 0.027771558612585068,
+      "learning_rate": 2.8399033710332713e-05,
+      "loss": 0.002,
+      "step": 1500
+    },
+    {
+      "epoch": 0.4521197746812926,
+      "grad_norm": 0.004578313324600458,
+      "learning_rate": 2.8316679477325135e-05,
+      "loss": 0.0033,
+      "step": 1525
+    },
+    {
+      "epoch": 0.45953157426623187,
+      "grad_norm": 2.601940393447876,
+      "learning_rate": 2.8234325244317557e-05,
+      "loss": 0.0027,
+      "step": 1550
+    },
+    {
+      "epoch": 0.4669433738511711,
+      "grad_norm": 0.020152784883975983,
+      "learning_rate": 2.8151971011309982e-05,
+      "loss": 0.0016,
+      "step": 1575
+    },
+    {
+      "epoch": 0.4743551734361103,
+      "grad_norm": 0.006129037588834763,
+      "learning_rate": 2.8069616778302404e-05,
+      "loss": 0.0003,
+      "step": 1600
+    },
+    {
+      "epoch": 0.48176697302104954,
+      "grad_norm": 0.0011216738494113088,
+      "learning_rate": 2.798726254529483e-05,
+      "loss": 0.0004,
+      "step": 1625
+    },
+    {
+      "epoch": 0.48917877260598874,
+      "grad_norm": 0.18000362813472748,
+      "learning_rate": 2.7904908312287254e-05,
+      "loss": 0.0045,
+      "step": 1650
+    },
+    {
+      "epoch": 0.49659057219092795,
+      "grad_norm": 0.2856181561946869,
+      "learning_rate": 2.7822554079279676e-05,
+      "loss": 0.0008,
+      "step": 1675
+    },
+    {
+      "epoch": 0.5040023717758672,
+      "grad_norm": 0.20144785940647125,
+      "learning_rate": 2.77401998462721e-05,
+      "loss": 0.0009,
+      "step": 1700
+    },
+    {
+      "epoch": 0.5114141713608064,
+      "grad_norm": 0.07046128064393997,
+      "learning_rate": 2.7657845613264523e-05,
+      "loss": 0.0015,
+      "step": 1725
+    },
+    {
+      "epoch": 0.5188259709457457,
+      "grad_norm": 0.23730403184890747,
+      "learning_rate": 2.7575491380256945e-05,
+      "loss": 0.0149,
+      "step": 1750
+    },
+    {
+      "epoch": 0.5262377705306849,
+      "grad_norm": 0.006619285326451063,
+      "learning_rate": 2.749313714724937e-05,
+      "loss": 0.0154,
+      "step": 1775
+    },
+    {
+      "epoch": 0.5336495701156241,
+      "grad_norm": 3.927973747253418,
+      "learning_rate": 2.7410782914241792e-05,
+      "loss": 0.0036,
+      "step": 1800
+    },
+    {
+      "epoch": 0.5410613697005633,
+      "grad_norm": 0.2023760825395584,
+      "learning_rate": 2.7328428681234214e-05,
+      "loss": 0.0028,
+      "step": 1825
+    },
+    {
+      "epoch": 0.5484731692855025,
+      "grad_norm": 0.008169805631041527,
+      "learning_rate": 2.7246074448226643e-05,
+      "loss": 0.0033,
+      "step": 1850
+    },
+    {
+      "epoch": 0.5558849688704417,
+      "grad_norm": 0.03589046001434326,
+      "learning_rate": 2.7163720215219065e-05,
+      "loss": 0.0086,
+      "step": 1875
+    },
+    {
+      "epoch": 0.563296768455381,
+      "grad_norm": 0.0460566021502018,
+      "learning_rate": 2.7081365982211486e-05,
+      "loss": 0.0124,
+      "step": 1900
+    },
+    {
+      "epoch": 0.5707085680403202,
+      "grad_norm": 0.038972873240709305,
+      "learning_rate": 2.699901174920391e-05,
+      "loss": 0.0005,
+      "step": 1925
+    },
+    {
+      "epoch": 0.5781203676252594,
+      "grad_norm": 0.023740865290164948,
+      "learning_rate": 2.6916657516196334e-05,
+      "loss": 0.0037,
+      "step": 1950
+    },
+    {
+      "epoch": 0.5855321672101986,
+      "grad_norm": 5.729846477508545,
+      "learning_rate": 2.6834303283188755e-05,
+      "loss": 0.0052,
+      "step": 1975
+    },
+    {
+      "epoch": 0.5929439667951378,
+      "grad_norm": 0.0025414160918444395,
+      "learning_rate": 2.675194905018118e-05,
+      "loss": 0.004,
+      "step": 2000
+    },
+    {
+      "epoch": 0.600355766380077,
+      "grad_norm": 0.006911627948284149,
+      "learning_rate": 2.6669594817173603e-05,
+      "loss": 0.0041,
+      "step": 2025
+    },
+    {
+      "epoch": 0.6077675659650164,
+      "grad_norm": 0.046909186989068985,
+      "learning_rate": 2.6587240584166024e-05,
+      "loss": 0.0006,
+      "step": 2050
+    },
+    {
+      "epoch": 0.6151793655499556,
+      "grad_norm": 1.3354122638702393,
+      "learning_rate": 2.6504886351158453e-05,
+      "loss": 0.0035,
+      "step": 2075
+    },
+    {
+      "epoch": 0.6225911651348948,
+      "grad_norm": 0.12669125199317932,
+      "learning_rate": 2.6422532118150875e-05,
+      "loss": 0.0079,
+      "step": 2100
+    },
+    {
+      "epoch": 0.630002964719834,
+      "grad_norm": 0.001392390113323927,
+      "learning_rate": 2.6340177885143297e-05,
+      "loss": 0.0012,
+      "step": 2125
+    },
+    {
+      "epoch": 0.6374147643047732,
+      "grad_norm": 0.0009501671884208918,
+      "learning_rate": 2.6257823652135722e-05,
+      "loss": 0.0012,
+      "step": 2150
+    },
+    {
+      "epoch": 0.6448265638897124,
+      "grad_norm": 9.893109321594238,
+      "learning_rate": 2.6175469419128144e-05,
+      "loss": 0.0035,
+      "step": 2175
+    },
+    {
+      "epoch": 0.6522383634746517,
+      "grad_norm": 0.00362469838000834,
+      "learning_rate": 2.6093115186120566e-05,
+      "loss": 0.0027,
+      "step": 2200
+    },
+    {
+      "epoch": 0.6596501630595909,
+      "grad_norm": 0.01450972817838192,
+      "learning_rate": 2.601076095311299e-05,
+      "loss": 0.0015,
+      "step": 2225
+    },
+    {
+      "epoch": 0.6670619626445301,
+      "grad_norm": 0.010469946078956127,
+      "learning_rate": 2.5928406720105413e-05,
+      "loss": 0.0004,
+      "step": 2250
+    },
+    {
+      "epoch": 0.6744737622294693,
+      "grad_norm": 0.2838996350765228,
+      "learning_rate": 2.5846052487097838e-05,
+      "loss": 0.001,
+      "step": 2275
+    },
+    {
+      "epoch": 0.6818855618144085,
+      "grad_norm": 0.0013039965415373445,
+      "learning_rate": 2.5763698254090263e-05,
+      "loss": 0.0002,
+      "step": 2300
+    },
+    {
+      "epoch": 0.6892973613993477,
+      "grad_norm": 0.0675489753484726,
+      "learning_rate": 2.5681344021082685e-05,
+      "loss": 0.0009,
+      "step": 2325
+    },
+    {
+      "epoch": 0.696709160984287,
+      "grad_norm": 0.3720860481262207,
+      "learning_rate": 2.5598989788075107e-05,
+      "loss": 0.0019,
+      "step": 2350
+    },
+    {
+      "epoch": 0.7041209605692262,
+      "grad_norm": 0.005105954594910145,
+      "learning_rate": 2.5516635555067532e-05,
+      "loss": 0.0005,
+      "step": 2375
+    },
+    {
+      "epoch": 0.7115327601541654,
+      "grad_norm": 0.46072980761528015,
+      "learning_rate": 2.5434281322059954e-05,
+      "loss": 0.0008,
+      "step": 2400
+    },
+    {
+      "epoch": 0.7189445597391046,
+      "grad_norm": 0.030280854552984238,
+      "learning_rate": 2.5351927089052376e-05,
+      "loss": 0.0012,
+      "step": 2425
+    },
+    {
+      "epoch": 0.7263563593240439,
+      "grad_norm": 0.007249627728015184,
+      "learning_rate": 2.52695728560448e-05,
+      "loss": 0.0036,
+      "step": 2450
+    },
+    {
+      "epoch": 0.7337681589089831,
+      "grad_norm": 0.007724974304437637,
+      "learning_rate": 2.5187218623037223e-05,
+      "loss": 0.0007,
+      "step": 2475
+    },
+    {
+      "epoch": 0.7411799584939224,
+      "grad_norm": 0.014301043003797531,
+      "learning_rate": 2.510486439002965e-05,
+      "loss": 0.0036,
+      "step": 2500
+    },
+    {
+      "epoch": 0.7485917580788616,
+      "grad_norm": 0.003261040663346648,
+      "learning_rate": 2.5022510157022074e-05,
+      "loss": 0.0007,
+      "step": 2525
+    },
+    {
+      "epoch": 0.7560035576638008,
+      "grad_norm": 0.01370958425104618,
+      "learning_rate": 2.4940155924014496e-05,
+      "loss": 0.0008,
+      "step": 2550
+    },
+    {
+      "epoch": 0.76341535724874,
+      "grad_norm": 0.000538276566658169,
+      "learning_rate": 2.4857801691006917e-05,
+      "loss": 0.0008,
+      "step": 2575
+    },
+    {
+      "epoch": 0.7708271568336792,
+      "grad_norm": 1.4628574848175049,
+      "learning_rate": 2.4775447457999343e-05,
+      "loss": 0.0016,
+      "step": 2600
+    },
+    {
+      "epoch": 0.7782389564186184,
+      "grad_norm": 0.569579541683197,
+      "learning_rate": 2.4693093224991765e-05,
+      "loss": 0.008,
+      "step": 2625
+    },
+    {
+      "epoch": 0.7856507560035577,
+      "grad_norm": 0.16939352452754974,
+      "learning_rate": 2.4610738991984186e-05,
+      "loss": 0.0058,
+      "step": 2650
+    },
+    {
+      "epoch": 0.7930625555884969,
+      "grad_norm": 0.11091727018356323,
+      "learning_rate": 2.452838475897661e-05,
+      "loss": 0.0003,
+      "step": 2675
+    },
+    {
+      "epoch": 0.8004743551734361,
+      "grad_norm": 0.007662549149245024,
+      "learning_rate": 2.4446030525969037e-05,
+      "loss": 0.0005,
+      "step": 2700
+    },
+    {
+      "epoch": 0.8078861547583753,
+      "grad_norm": 0.003822131548076868,
+      "learning_rate": 2.436367629296146e-05,
+      "loss": 0.0101,
+      "step": 2725
+    },
+    {
+      "epoch": 0.8152979543433145,
+      "grad_norm": 0.004805045202374458,
+      "learning_rate": 2.4281322059953884e-05,
+      "loss": 0.0025,
+      "step": 2750
+    },
+    {
+      "epoch": 0.8227097539282537,
+      "grad_norm": 0.004129350651055574,
+      "learning_rate": 2.4198967826946306e-05,
+      "loss": 0.0041,
+      "step": 2775
+    },
+    {
+      "epoch": 0.8301215535131931,
+      "grad_norm": 0.0160355307161808,
+      "learning_rate": 2.4116613593938728e-05,
+      "loss": 0.0113,
+      "step": 2800
+    },
+    {
+      "epoch": 0.8375333530981323,
+      "grad_norm": 0.017653746530413628,
+      "learning_rate": 2.4034259360931153e-05,
+      "loss": 0.0219,
+      "step": 2825
+    },
+    {
+      "epoch": 0.8449451526830715,
+      "grad_norm": 0.004908108152449131,
+      "learning_rate": 2.3951905127923575e-05,
+      "loss": 0.0004,
+      "step": 2850
+    },
+    {
+      "epoch": 0.8523569522680107,
+      "grad_norm": 0.0010231022024527192,
+      "learning_rate": 2.3869550894915997e-05,
+      "loss": 0.0062,
+      "step": 2875
+    },
+    {
+      "epoch": 0.8597687518529499,
+      "grad_norm": 1.0080921649932861,
+      "learning_rate": 2.3787196661908422e-05,
+      "loss": 0.0097,
+      "step": 2900
+    },
+    {
+      "epoch": 0.8671805514378891,
+      "grad_norm": 0.014476609416306019,
+      "learning_rate": 2.3704842428900847e-05,
+      "loss": 0.002,
+      "step": 2925
+    },
+    {
+      "epoch": 0.8745923510228284,
+      "grad_norm": 12.20775032043457,
+      "learning_rate": 2.362248819589327e-05,
+      "loss": 0.0093,
+      "step": 2950
+    },
+    {
+      "epoch": 0.8820041506077676,
+      "grad_norm": 0.04047137871384621,
+      "learning_rate": 2.3540133962885694e-05,
+      "loss": 0.0046,
+      "step": 2975
+    },
+    {
+      "epoch": 0.8894159501927068,
+      "grad_norm": 1.3424041271209717,
+      "learning_rate": 2.3457779729878116e-05,
+      "loss": 0.0009,
+      "step": 3000
+    },
+    {
+      "epoch": 0.896827749777646,
+      "grad_norm": 1.0432305335998535,
+      "learning_rate": 2.3375425496870538e-05,
+      "loss": 0.0014,
+      "step": 3025
+    },
+    {
+      "epoch": 0.9042395493625852,
+      "grad_norm": 0.013758724555373192,
+      "learning_rate": 2.3293071263862963e-05,
+      "loss": 0.0021,
+      "step": 3050
+    },
+    {
+      "epoch": 0.9116513489475244,
+      "grad_norm": 0.022794967517256737,
+      "learning_rate": 2.3210717030855385e-05,
+      "loss": 0.0009,
+      "step": 3075
+    },
+    {
+      "epoch": 0.9190631485324637,
+      "grad_norm": 0.00779375247657299,
+      "learning_rate": 2.3128362797847807e-05,
+      "loss": 0.0082,
+      "step": 3100
+    },
+    {
+      "epoch": 0.9264749481174029,
+      "grad_norm": 0.0006471537053585052,
+      "learning_rate": 2.3046008564840236e-05,
+      "loss": 0.0016,
+      "step": 3125
+    },
+    {
+      "epoch": 0.9338867477023421,
+      "grad_norm": 0.010517094284296036,
+      "learning_rate": 2.2963654331832658e-05,
+      "loss": 0.0023,
+      "step": 3150
+    },
+    {
+      "epoch": 0.9412985472872814,
+      "grad_norm": 0.0007998093497008085,
+      "learning_rate": 2.288130009882508e-05,
+      "loss": 0.0019,
+      "step": 3175
+    },
+    {
+      "epoch": 0.9487103468722206,
+      "grad_norm": 0.03586767986416817,
+      "learning_rate": 2.2798945865817505e-05,
+      "loss": 0.002,
+      "step": 3200
+    },
+    {
+      "epoch": 0.9561221464571598,
+      "grad_norm": 0.11343257874250412,
+      "learning_rate": 2.2716591632809927e-05,
+      "loss": 0.0009,
+      "step": 3225
+    },
+    {
+      "epoch": 0.9635339460420991,
+      "grad_norm": 0.05241592228412628,
+      "learning_rate": 2.2637531569122653e-05,
+      "loss": 0.0111,
+      "step": 3250
+    },
+    {
+      "epoch": 0.9709457456270383,
+      "grad_norm": 0.0006894416292198002,
+      "learning_rate": 2.255517733611508e-05,
+      "loss": 0.0137,
+      "step": 3275
+    },
+    {
+      "epoch": 0.9783575452119775,
+      "grad_norm": 0.001717191538773477,
+      "learning_rate": 2.24728231031075e-05,
+      "loss": 0.0038,
+      "step": 3300
+    },
+    {
+      "epoch": 0.9857693447969167,
+      "grad_norm": 0.020298583433032036,
+      "learning_rate": 2.2390468870099925e-05,
+      "loss": 0.0061,
+      "step": 3325
+    },
+    {
+      "epoch": 0.9931811443818559,
+      "grad_norm": 0.02918991446495056,
+      "learning_rate": 2.2308114637092347e-05,
+      "loss": 0.0045,
+      "step": 3350
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 0.0037797815166413784,
+      "eval_runtime": 12.579,
+      "eval_samples_per_second": 112.648,
+      "eval_steps_per_second": 7.075,
+      "step": 3373
+    },
+    {
+      "epoch": 1.000592943966795,
+      "grad_norm": 0.0031215930357575417,
+      "learning_rate": 2.222576040408477e-05,
+      "loss": 0.0004,
+      "step": 3375
+    },
+    {
+      "epoch": 1.0080047435517343,
+      "grad_norm": 0.15596337616443634,
+      "learning_rate": 2.2143406171077194e-05,
+      "loss": 0.0104,
+      "step": 3400
+    },
+    {
+      "epoch": 1.0154165431366735,
+      "grad_norm": 0.024601919576525688,
+      "learning_rate": 2.2061051938069616e-05,
+      "loss": 0.0033,
+      "step": 3425
+    },
+    {
+      "epoch": 1.0228283427216127,
+      "grad_norm": 0.05934903770685196,
+      "learning_rate": 2.1978697705062038e-05,
+      "loss": 0.0004,
+      "step": 3450
+    },
+    {
+      "epoch": 1.030240142306552,
+      "grad_norm": 0.0020625083707273006,
+      "learning_rate": 2.1896343472054467e-05,
+      "loss": 0.0065,
+      "step": 3475
+    },
+    {
+      "epoch": 1.0376519418914913,
+      "grad_norm": 0.0041097910143435,
+      "learning_rate": 2.181398923904689e-05,
+      "loss": 0.004,
+      "step": 3500
+    },
+    {
+      "epoch": 1.0450637414764306,
+      "grad_norm": 0.019944315776228905,
+      "learning_rate": 2.173163500603931e-05,
+      "loss": 0.0003,
+      "step": 3525
+    },
+    {
+      "epoch": 1.0524755410613698,
+      "grad_norm": 0.021289722993969917,
+      "learning_rate": 2.1649280773031736e-05,
+      "loss": 0.0004,
+      "step": 3550
+    },
+    {
+      "epoch": 1.059887340646309,
+      "grad_norm": 0.0058084093034267426,
+      "learning_rate": 2.1566926540024158e-05,
+      "loss": 0.0006,
+      "step": 3575
+    },
+    {
+      "epoch": 1.0672991402312482,
+      "grad_norm": 0.0045490204356610775,
+      "learning_rate": 2.148457230701658e-05,
+      "loss": 0.0018,
+      "step": 3600
+    },
+    {
+      "epoch": 1.0747109398161874,
+      "grad_norm": 0.23320598900318146,
+      "learning_rate": 2.140551224332931e-05,
+      "loss": 0.0246,
+      "step": 3625
+    },
+    {
+      "epoch": 1.0821227394011266,
+      "grad_norm": 0.4190017879009247,
+      "learning_rate": 2.132315801032173e-05,
+      "loss": 0.007,
+      "step": 3650
+    },
+    {
+      "epoch": 1.0895345389860658,
+      "grad_norm": 0.012420530430972576,
+      "learning_rate": 2.1240803777314157e-05,
+      "loss": 0.0002,
+      "step": 3675
+    },
+    {
+      "epoch": 1.096946338571005,
+      "grad_norm": 0.0024064735043793917,
+      "learning_rate": 2.115844954430658e-05,
+      "loss": 0.0005,
+      "step": 3700
+    },
+    {
+      "epoch": 1.1043581381559442,
+      "grad_norm": 0.025788817554712296,
+      "learning_rate": 2.1076095311299e-05,
+      "loss": 0.0097,
+      "step": 3725
+    },
+    {
+      "epoch": 1.1117699377408834,
+      "grad_norm": 0.007708790246397257,
+      "learning_rate": 2.0993741078291425e-05,
+      "loss": 0.0011,
+      "step": 3750
+    },
+    {
+      "epoch": 1.1191817373258228,
+      "grad_norm": 0.006970885209739208,
+      "learning_rate": 2.0911386845283847e-05,
+      "loss": 0.0005,
+      "step": 3775
+    },
+    {
+      "epoch": 1.126593536910762,
+      "grad_norm": 0.27546170353889465,
+      "learning_rate": 2.082903261227627e-05,
+      "loss": 0.0015,
+      "step": 3800
+    },
+    {
+      "epoch": 1.1340053364957012,
+      "grad_norm": 0.047784604132175446,
+      "learning_rate": 2.0746678379268698e-05,
+      "loss": 0.0009,
+      "step": 3825
+    },
+    {
+      "epoch": 1.1414171360806404,
+      "grad_norm": 0.21240082383155823,
+      "learning_rate": 2.066432414626112e-05,
+      "loss": 0.0002,
+      "step": 3850
+    },
+    {
+      "epoch": 1.1488289356655796,
+      "grad_norm": 0.010284190997481346,
+      "learning_rate": 2.058196991325354e-05,
+      "loss": 0.0002,
+      "step": 3875
+    },
+    {
+      "epoch": 1.1562407352505188,
+      "grad_norm": 0.009727857075631618,
+      "learning_rate": 2.0499615680245967e-05,
+      "loss": 0.0021,
+      "step": 3900
+    },
+    {
+      "epoch": 1.163652534835458,
+      "grad_norm": 0.003159493440762162,
+      "learning_rate": 2.041726144723839e-05,
+      "loss": 0.0003,
+      "step": 3925
+    },
+    {
+      "epoch": 1.1710643344203973,
+      "grad_norm": 0.003643095726147294,
+      "learning_rate": 2.033490721423081e-05,
+      "loss": 0.0006,
+      "step": 3950
+    },
+    {
+      "epoch": 1.1784761340053365,
+      "grad_norm": 0.007551380433142185,
+      "learning_rate": 2.0252552981223236e-05,
+      "loss": 0.0019,
+      "step": 3975
+    },
+    {
+      "epoch": 1.1858879335902757,
+      "grad_norm": 2.013258218765259,
+      "learning_rate": 2.0170198748215658e-05,
+      "loss": 0.0013,
+      "step": 4000
+    },
+    {
+      "epoch": 1.1932997331752149,
+      "grad_norm": 0.847743570804596,
+      "learning_rate": 2.0087844515208083e-05,
+      "loss": 0.002,
+      "step": 4025
+    },
+    {
+      "epoch": 1.200711532760154,
+      "grad_norm": 0.015584784559905529,
+      "learning_rate": 2.0005490282200508e-05,
+      "loss": 0.0005,
+      "step": 4050
+    },
+    {
+      "epoch": 1.2081233323450933,
+      "grad_norm": 0.002146722748875618,
+      "learning_rate": 1.992313604919293e-05,
+      "loss": 0.0037,
+      "step": 4075
+    },
+    {
+      "epoch": 1.2155351319300327,
+      "grad_norm": 0.2943512797355652,
+      "learning_rate": 1.9840781816185352e-05,
+      "loss": 0.0013,
+      "step": 4100
+    },
+    {
+      "epoch": 1.222946931514972,
+      "grad_norm": 0.01032156590372324,
+      "learning_rate": 1.9758427583177777e-05,
+      "loss": 0.0006,
+      "step": 4125
+    },
+    {
+      "epoch": 1.2303587310999111,
+      "grad_norm": 0.0028528086841106415,
+      "learning_rate": 1.96760733501702e-05,
+      "loss": 0.0037,
+      "step": 4150
+    },
+    {
+      "epoch": 1.2377705306848503,
+      "grad_norm": 0.01456816028803587,
+      "learning_rate": 1.959371911716262e-05,
+      "loss": 0.0018,
+      "step": 4175
+    },
+    {
+      "epoch": 1.2451823302697895,
+      "grad_norm": 0.06787148863077164,
+      "learning_rate": 1.9511364884155046e-05,
+      "loss": 0.0005,
+      "step": 4200
+    },
+    {
+      "epoch": 1.2525941298547287,
+      "grad_norm": 0.03550760820508003,
+      "learning_rate": 1.9429010651147468e-05,
+      "loss": 0.0003,
+      "step": 4225
+    },
+    {
+      "epoch": 1.260005929439668,
+      "grad_norm": 0.04113534837961197,
+      "learning_rate": 1.9346656418139893e-05,
+      "loss": 0.0003,
+      "step": 4250
+    },
+    {
+      "epoch": 1.2674177290246071,
+      "grad_norm": 0.030371995642781258,
+      "learning_rate": 1.926430218513232e-05,
+      "loss": 0.0003,
+      "step": 4275
+    },
+    {
+      "epoch": 1.2748295286095463,
+      "grad_norm": 0.14188840985298157,
+      "learning_rate": 1.918194795212474e-05,
+      "loss": 0.0004,
+      "step": 4300
+    },
+    {
+      "epoch": 1.2822413281944856,
+      "grad_norm": 0.02768622897565365,
+      "learning_rate": 1.9099593719117162e-05,
+      "loss": 0.0029,
+      "step": 4325
+    },
+    {
+      "epoch": 1.2896531277794248,
+      "grad_norm": 0.0008035032660700381,
+      "learning_rate": 1.9017239486109588e-05,
+      "loss": 0.0021,
+      "step": 4350
+    },
+    {
+      "epoch": 1.2970649273643642,
+      "grad_norm": 0.0008042781846597791,
+      "learning_rate": 1.893488525310201e-05,
+      "loss": 0.0004,
+      "step": 4375
+    },
+    {
+      "epoch": 1.3044767269493032,
+      "grad_norm": 0.0014630877412855625,
+      "learning_rate": 1.885253102009443e-05,
+      "loss": 0.0008,
+      "step": 4400
+    },
+    {
+      "epoch": 1.3118885265342426,
+      "grad_norm": 0.3650088608264923,
+      "learning_rate": 1.8770176787086857e-05,
+      "loss": 0.0002,
+      "step": 4425
+    },
+    {
+      "epoch": 1.3193003261191818,
+      "grad_norm": 0.0012581262271851301,
+      "learning_rate": 1.8687822554079282e-05,
+      "loss": 0.0031,
+      "step": 4450
+    },
+    {
+      "epoch": 1.326712125704121,
+      "grad_norm": 0.009501802735030651,
+      "learning_rate": 1.8605468321071704e-05,
+      "loss": 0.0005,
+      "step": 4475
+    },
+    {
+      "epoch": 1.3341239252890602,
+      "grad_norm": 0.023450246080756187,
+      "learning_rate": 1.852311408806413e-05,
+      "loss": 0.0006,
+      "step": 4500
+    },
+    {
+      "epoch": 1.3415357248739994,
+      "grad_norm": 0.041543640196323395,
+      "learning_rate": 1.844075985505655e-05,
+      "loss": 0.0005,
+      "step": 4525
+    },
+    {
+      "epoch": 1.3489475244589386,
+      "grad_norm": 0.030355799943208694,
+      "learning_rate": 1.8358405622048973e-05,
+      "loss": 0.0012,
+      "step": 4550
+    },
+    {
+      "epoch": 1.3563593240438778,
+      "grad_norm": 0.002064308850094676,
+      "learning_rate": 1.8276051389041398e-05,
+      "loss": 0.0009,
+      "step": 4575
+    },
+    {
+      "epoch": 1.363771123628817,
+      "grad_norm": 0.04516091197729111,
+      "learning_rate": 1.819369715603382e-05,
+      "loss": 0.0006,
+      "step": 4600
+    },
+    {
+      "epoch": 1.3711829232137562,
+      "grad_norm": 0.0014140811981633306,
+      "learning_rate": 1.811134292302624e-05,
+      "loss": 0.0237,
+      "step": 4625
+    },
+    {
+      "epoch": 1.3785947227986957,
+      "grad_norm": 0.01035268697887659,
+      "learning_rate": 1.8028988690018667e-05,
+      "loss": 0.0004,
+      "step": 4650
+    },
+    {
+      "epoch": 1.3860065223836346,
+      "grad_norm": 0.0017914492636919022,
+      "learning_rate": 1.7946634457011092e-05,
+      "loss": 0.0001,
+      "step": 4675
+    },
+    {
+      "epoch": 1.393418321968574,
+      "grad_norm": 1.8334236145019531,
+      "learning_rate": 1.7864280224003514e-05,
+      "loss": 0.0005,
+      "step": 4700
+    },
+    {
+      "epoch": 1.4008301215535133,
+      "grad_norm": 0.093447744846344,
+      "learning_rate": 1.778192599099594e-05,
+      "loss": 0.0003,
+      "step": 4725
+    },
+    {
+      "epoch": 1.4082419211384525,
+      "grad_norm": 0.004699540790170431,
+      "learning_rate": 1.769957175798836e-05,
+      "loss": 0.0059,
+      "step": 4750
+    },
+    {
+      "epoch": 1.4156537207233917,
+      "grad_norm": 0.01479920744895935,
+      "learning_rate": 1.7617217524980783e-05,
+      "loss": 0.0005,
+      "step": 4775
+    },
+    {
+      "epoch": 1.423065520308331,
+      "grad_norm": 0.007109279744327068,
+      "learning_rate": 1.7534863291973208e-05,
+      "loss": 0.0014,
+      "step": 4800
+    },
+    {
+      "epoch": 1.43047731989327,
+      "grad_norm": 0.007401443552225828,
+      "learning_rate": 1.745250905896563e-05,
+      "loss": 0.0001,
+      "step": 4825
+    },
+    {
+      "epoch": 1.4378891194782093,
+      "grad_norm": 0.5947569012641907,
+      "learning_rate": 1.7370154825958052e-05,
+      "loss": 0.0003,
+      "step": 4850
+    },
+    {
+      "epoch": 1.4453009190631485,
+      "grad_norm": 0.001491679809987545,
+      "learning_rate": 1.728780059295048e-05,
+      "loss": 0.0003,
+      "step": 4875
+    },
+    {
+      "epoch": 1.4527127186480877,
+      "grad_norm": 0.009767352603375912,
+      "learning_rate": 1.7205446359942902e-05,
+      "loss": 0.0062,
+      "step": 4900
+    },
+    {
+      "epoch": 1.460124518233027,
+      "grad_norm": 0.0028043785132467747,
+      "learning_rate": 1.7123092126935324e-05,
+      "loss": 0.0002,
+      "step": 4925
+    },
+    {
+      "epoch": 1.4675363178179661,
+      "grad_norm": 0.037593334913253784,
+      "learning_rate": 1.704073789392775e-05,
+      "loss": 0.0004,
+      "step": 4950
+    },
+    {
+      "epoch": 1.4749481174029055,
+      "grad_norm": 0.046085864305496216,
+      "learning_rate": 1.695838366092017e-05,
+      "loss": 0.0001,
+      "step": 4975
+    },
+    {
+      "epoch": 1.4823599169878445,
+      "grad_norm": 0.0233808271586895,
+      "learning_rate": 1.6876029427912593e-05,
+      "loss": 0.0003,
+      "step": 5000
+    },
+    {
+      "epoch": 1.489771716572784,
+      "grad_norm": 0.06796186417341232,
+      "learning_rate": 1.679367519490502e-05,
+      "loss": 0.0003,
+      "step": 5025
+    },
+    {
+      "epoch": 1.4971835161577232,
+      "grad_norm": 0.017352351918816566,
+      "learning_rate": 1.671132096189744e-05,
+      "loss": 0.0004,
+      "step": 5050
+    },
+    {
+      "epoch": 1.5045953157426624,
+      "grad_norm": 0.003020123578608036,
+      "learning_rate": 1.6628966728889862e-05,
+      "loss": 0.0003,
+      "step": 5075
+    },
+    {
+      "epoch": 1.5120071153276016,
+      "grad_norm": 0.004012423101812601,
+      "learning_rate": 1.654661249588229e-05,
+      "loss": 0.0001,
+      "step": 5100
+    },
+    {
+      "epoch": 1.5194189149125408,
+      "grad_norm": 0.001218698569573462,
+      "learning_rate": 1.6464258262874713e-05,
+      "loss": 0.0031,
+      "step": 5125
+    },
+    {
+      "epoch": 1.52683071449748,
+      "grad_norm": 0.003939245827496052,
+      "learning_rate": 1.6381904029867135e-05,
+      "loss": 0.0002,
+      "step": 5150
+    },
+    {
+      "epoch": 1.5342425140824192,
+      "grad_norm": 0.00894363597035408,
+      "learning_rate": 1.629954979685956e-05,
+      "loss": 0.0006,
+      "step": 5175
+    },
+    {
+      "epoch": 1.5416543136673584,
+      "grad_norm": 0.0632522702217102,
+      "learning_rate": 1.6217195563851982e-05,
+      "loss": 0.0001,
+      "step": 5200
+    },
+    {
+      "epoch": 1.5490661132522976,
+      "grad_norm": 0.02053726837038994,
+      "learning_rate": 1.6134841330844404e-05,
+      "loss": 0.0017,
+      "step": 5225
+    },
+    {
+      "epoch": 1.556477912837237,
+      "grad_norm": 0.0023088334128260612,
+      "learning_rate": 1.605248709783683e-05,
+      "loss": 0.0163,
+      "step": 5250
+    },
+    {
+      "epoch": 1.563889712422176,
+      "grad_norm": 0.007689852733165026,
+      "learning_rate": 1.597013286482925e-05,
+      "loss": 0.0049,
+      "step": 5275
+    },
+    {
+      "epoch": 1.5713015120071154,
+      "grad_norm": 0.02146066538989544,
+      "learning_rate": 1.5887778631821676e-05,
+      "loss": 0.0029,
+      "step": 5300
+    },
+    {
+      "epoch": 1.5787133115920544,
+      "grad_norm": 0.007000596262514591,
+      "learning_rate": 1.58054243988141e-05,
+      "loss": 0.0035,
+      "step": 5325
+    },
+    {
+      "epoch": 1.5861251111769938,
+      "grad_norm": 0.040092576295137405,
+      "learning_rate": 1.5723070165806523e-05,
+      "loss": 0.007,
+      "step": 5350
+    },
+    {
+      "epoch": 1.593536910761933,
+      "grad_norm": 0.008139602839946747,
+      "learning_rate": 1.5640715932798945e-05,
+      "loss": 0.0006,
+      "step": 5375
+    },
+    {
+      "epoch": 1.6009487103468723,
+      "grad_norm": 0.0004769627994392067,
+      "learning_rate": 1.555836169979137e-05,
+      "loss": 0.0003,
+      "step": 5400
+    },
+    {
+      "epoch": 1.6083605099318115,
+      "grad_norm": 0.0034301020205020905,
+      "learning_rate": 1.5476007466783792e-05,
+      "loss": 0.0004,
+      "step": 5425
+    },
+    {
+      "epoch": 1.6157723095167507,
+      "grad_norm": 0.002083411207422614,
+      "learning_rate": 1.5393653233776217e-05,
+      "loss": 0.0067,
+      "step": 5450
+    },
+    {
+      "epoch": 1.6231841091016899,
+      "grad_norm": 9.624741554260254,
+      "learning_rate": 1.531129900076864e-05,
+      "loss": 0.0112,
+      "step": 5475
+    },
+    {
+      "epoch": 1.630595908686629,
+      "grad_norm": 0.0528758242726326,
+      "learning_rate": 1.5228944767761063e-05,
+      "loss": 0.0002,
+      "step": 5500
+    },
+    {
+      "epoch": 1.6380077082715685,
+      "grad_norm": 0.0011297637829557061,
+      "learning_rate": 1.5146590534753488e-05,
+      "loss": 0.0005,
+      "step": 5525
+    },
+    {
+      "epoch": 1.6454195078565075,
+      "grad_norm": 0.03151678293943405,
+      "learning_rate": 1.506423630174591e-05,
+      "loss": 0.0029,
+      "step": 5550
+    },
+    {
+      "epoch": 1.652831307441447,
+      "grad_norm": 0.01291624829173088,
+      "learning_rate": 1.4981882068738333e-05,
+      "loss": 0.0006,
+      "step": 5575
+    },
+    {
+      "epoch": 1.660243107026386,
+      "grad_norm": 0.0060423207469284534,
+      "learning_rate": 1.4899527835730757e-05,
+      "loss": 0.0012,
+      "step": 5600
+    },
+    {
+      "epoch": 1.6676549066113253,
+      "grad_norm": 0.025346050038933754,
+      "learning_rate": 1.481717360272318e-05,
+      "loss": 0.0002,
+      "step": 5625
+    },
+    {
+      "epoch": 1.6750667061962643,
+      "grad_norm": 0.0028023323975503445,
+      "learning_rate": 1.4734819369715602e-05,
+      "loss": 0.0005,
+      "step": 5650
+    },
+    {
+      "epoch": 1.6824785057812037,
+      "grad_norm": 0.0084083816036582,
+      "learning_rate": 1.4652465136708028e-05,
+      "loss": 0.0002,
+      "step": 5675
+    },
+    {
+      "epoch": 1.689890305366143,
+      "grad_norm": 0.00158981594722718,
+      "learning_rate": 1.4570110903700451e-05,
+      "loss": 0.0002,
+      "step": 5700
+    },
+    {
+      "epoch": 1.6973021049510821,
+      "grad_norm": 0.004132249392569065,
+      "learning_rate": 1.4487756670692873e-05,
+      "loss": 0.0003,
+      "step": 5725
+    },
+    {
+      "epoch": 1.7047139045360213,
+      "grad_norm": 0.0054726507514715195,
+      "learning_rate": 1.4405402437685297e-05,
+      "loss": 0.0018,
+      "step": 5750
+    },
+    {
+      "epoch": 1.7121257041209605,
+      "grad_norm": 0.006865357980132103,
+      "learning_rate": 1.4323048204677722e-05,
+      "loss": 0.0027,
+      "step": 5775
+    },
+    {
+      "epoch": 1.7195375037058998,
+      "grad_norm": 0.07013662904500961,
+      "learning_rate": 1.4240693971670144e-05,
+      "loss": 0.0002,
+      "step": 5800
+    },
+    {
+      "epoch": 1.726949303290839,
+      "grad_norm": 0.00439479760825634,
+      "learning_rate": 1.4158339738662567e-05,
+      "loss": 0.0001,
+      "step": 5825
+    },
+    {
+      "epoch": 1.7343611028757784,
+      "grad_norm": 0.024386949837207794,
+      "learning_rate": 1.4075985505654991e-05,
+      "loss": 0.0007,
+      "step": 5850
+    },
+    {
+      "epoch": 1.7417729024607174,
+      "grad_norm": 0.005310242995619774,
+      "learning_rate": 1.3993631272647414e-05,
+      "loss": 0.0047,
+      "step": 5875
+    },
+    {
+      "epoch": 1.7491847020456568,
+      "grad_norm": 0.008037442341446877,
+      "learning_rate": 1.3911277039639838e-05,
+      "loss": 0.0001,
+      "step": 5900
+    },
+    {
+      "epoch": 1.7565965016305958,
+      "grad_norm": 0.0005814226460643113,
+      "learning_rate": 1.3828922806632262e-05,
+      "loss": 0.0002,
+      "step": 5925
+    },
+    {
+      "epoch": 1.7640083012155352,
+      "grad_norm": 0.0108102485537529,
+      "learning_rate": 1.3746568573624685e-05,
+      "loss": 0.0007,
+      "step": 5950
+    },
+    {
+      "epoch": 1.7714201008004744,
+      "grad_norm": 0.03512908145785332,
+      "learning_rate": 1.3664214340617107e-05,
+      "loss": 0.0039,
+      "step": 5975
+    },
+    {
+      "epoch": 1.7788319003854136,
+      "grad_norm": 0.001055323169566691,
+      "learning_rate": 1.3581860107609532e-05,
+      "loss": 0.0001,
+      "step": 6000
+    },
+    {
+      "epoch": 1.7862436999703528,
+      "grad_norm": 0.0011812145821750164,
+      "learning_rate": 1.3499505874601956e-05,
+      "loss": 0.0001,
+      "step": 6025
+    },
+    {
+      "epoch": 1.793655499555292,
+      "grad_norm": 0.004607312846928835,
+      "learning_rate": 1.3417151641594378e-05,
+      "loss": 0.0002,
+      "step": 6050
+    },
+    {
+      "epoch": 1.8010672991402312,
+      "grad_norm": 0.01913565769791603,
+      "learning_rate": 1.3334797408586801e-05,
+      "loss": 0.0001,
+      "step": 6075
+    },
+    {
+      "epoch": 1.8084790987251704,
+      "grad_norm": 4.993204116821289,
+      "learning_rate": 1.3252443175579227e-05,
+      "loss": 0.0016,
+      "step": 6100
+    },
+    {
+      "epoch": 1.8158908983101099,
+      "grad_norm": 0.004902560729533434,
+      "learning_rate": 1.3170088942571648e-05,
+      "loss": 0.0004,
+      "step": 6125
+    },
+    {
+      "epoch": 1.8233026978950488,
+      "grad_norm": 0.02545163407921791,
+      "learning_rate": 1.3087734709564072e-05,
+      "loss": 0.0003,
+      "step": 6150
+    },
+    {
+      "epoch": 1.8307144974799883,
+      "grad_norm": 0.0022334237582981586,
+      "learning_rate": 1.3005380476556495e-05,
+      "loss": 0.0005,
+      "step": 6175
+    },
+    {
+      "epoch": 1.8381262970649273,
+      "grad_norm": 0.020511144772171974,
+      "learning_rate": 1.2923026243548919e-05,
+      "loss": 0.0005,
+      "step": 6200
+    },
+    {
+      "epoch": 1.8455380966498667,
+      "grad_norm": 0.00042026047594845295,
+      "learning_rate": 1.2840672010541343e-05,
+      "loss": 0.0004,
+      "step": 6225
+    },
+    {
+      "epoch": 1.8529498962348057,
+      "grad_norm": 0.01851501315832138,
+      "learning_rate": 1.2758317777533766e-05,
+      "loss": 0.0036,
+      "step": 6250
+    },
+    {
+      "epoch": 1.860361695819745,
+      "grad_norm": 0.0019784029573202133,
+      "learning_rate": 1.2675963544526188e-05,
+      "loss": 0.0002,
+      "step": 6275
+    },
+    {
+      "epoch": 1.8677734954046843,
+      "grad_norm": 0.005374327767640352,
+      "learning_rate": 1.2593609311518612e-05,
+      "loss": 0.0006,
+      "step": 6300
+    },
+    {
+      "epoch": 1.8751852949896235,
+      "grad_norm": 0.009900149889290333,
+      "learning_rate": 1.2511255078511037e-05,
+      "loss": 0.0001,
+      "step": 6325
+    },
+    {
+      "epoch": 1.8825970945745627,
+      "grad_norm": 0.4626489579677582,
+      "learning_rate": 1.2428900845503459e-05,
+      "loss": 0.0022,
+      "step": 6350
+    },
+    {
+      "epoch": 1.890008894159502,
+      "grad_norm": 0.012091380544006824,
+      "learning_rate": 1.2346546612495882e-05,
+      "loss": 0.0001,
+      "step": 6375
+    },
+    {
+      "epoch": 1.8974206937444411,
+      "grad_norm": 0.01360028050839901,
+      "learning_rate": 1.2264192379488306e-05,
+      "loss": 0.0003,
+      "step": 6400
+    },
+    {
+      "epoch": 1.9048324933293803,
+      "grad_norm": 0.010124334134161472,
+      "learning_rate": 1.218183814648073e-05,
+      "loss": 0.0001,
+      "step": 6425
+    },
+    {
+      "epoch": 1.9122442929143197,
+      "grad_norm": 0.0006189885316416621,
+      "learning_rate": 1.2099483913473153e-05,
+      "loss": 0.001,
+      "step": 6450
+    },
+    {
+      "epoch": 1.9196560924992587,
+      "grad_norm": 0.012544695287942886,
+      "learning_rate": 1.2017129680465577e-05,
+      "loss": 0.0005,
+      "step": 6475
+    },
+    {
+      "epoch": 1.9270678920841982,
+      "grad_norm": 0.03128047287464142,
+      "learning_rate": 1.1934775447457998e-05,
+      "loss": 0.0002,
+      "step": 6500
+    },
+    {
+      "epoch": 1.9344796916691371,
+      "grad_norm": 0.0022450904361903667,
+      "learning_rate": 1.1852421214450424e-05,
+      "loss": 0.0049,
+      "step": 6525
+    },
+    {
+      "epoch": 1.9418914912540766,
+      "grad_norm": 0.012787646614015102,
+      "learning_rate": 1.1770066981442847e-05,
+      "loss": 0.0002,
+      "step": 6550
+    },
+    {
+      "epoch": 1.9493032908390158,
+      "grad_norm": 0.0022699732799082994,
+      "learning_rate": 1.1687712748435269e-05,
+      "loss": 0.0009,
+      "step": 6575
+    },
+    {
+      "epoch": 1.956715090423955,
+      "grad_norm": 0.001058919820934534,
+      "learning_rate": 1.1605358515427693e-05,
+      "loss": 0.0145,
+      "step": 6600
+    },
+    {
+      "epoch": 1.9641268900088942,
+      "grad_norm": 0.002509496873244643,
+      "learning_rate": 1.1523004282420118e-05,
+      "loss": 0.0004,
+      "step": 6625
+    },
+    {
+      "epoch": 1.9715386895938334,
+      "grad_norm": 0.0002950621419586241,
+      "learning_rate": 1.144065004941254e-05,
+      "loss": 0.0009,
+      "step": 6650
+    },
+    {
+      "epoch": 1.9789504891787726,
+      "grad_norm": 0.007797342259436846,
+      "learning_rate": 1.1358295816404963e-05,
+      "loss": 0.0003,
+      "step": 6675
+    },
+    {
+      "epoch": 1.9863622887637118,
+      "grad_norm": 17.98861312866211,
+      "learning_rate": 1.1275941583397387e-05,
+      "loss": 0.0053,
+      "step": 6700
+    },
+    {
+      "epoch": 1.9937740883486512,
+      "grad_norm": 0.00034863411565311253,
+      "learning_rate": 1.1193587350389809e-05,
+      "loss": 0.0002,
+      "step": 6725
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.00123180216178298,
+      "eval_runtime": 12.6622,
+      "eval_samples_per_second": 111.908,
+      "eval_steps_per_second": 7.029,
+      "step": 6746
+    },
+    {
+      "epoch": 2.00118588793359,
+      "grad_norm": 0.00869747158139944,
+      "learning_rate": 1.1111233117382234e-05,
+      "loss": 0.0002,
+      "step": 6750
+    },
+    {
+      "epoch": 2.0085976875185296,
+      "grad_norm": 0.021044636145234108,
+      "learning_rate": 1.1028878884374658e-05,
+      "loss": 0.0018,
+      "step": 6775
+    },
+    {
+      "epoch": 2.0160094871034686,
+      "grad_norm": 0.12446173280477524,
+      "learning_rate": 1.0946524651367081e-05,
+      "loss": 0.0001,
+      "step": 6800
+    },
+    {
+      "epoch": 2.023421286688408,
+      "grad_norm": 0.6716703772544861,
+      "learning_rate": 1.0864170418359503e-05,
+      "loss": 0.0084,
+      "step": 6825
+    },
+    {
+      "epoch": 2.030833086273347,
+      "grad_norm": 0.022197945043444633,
+      "learning_rate": 1.0781816185351928e-05,
+      "loss": 0.0004,
+      "step": 6850
+    },
+    {
+      "epoch": 2.0382448858582864,
+      "grad_norm": 0.020370731130242348,
+      "learning_rate": 1.0699461952344352e-05,
+      "loss": 0.0001,
+      "step": 6875
+    },
+    {
+      "epoch": 2.0456566854432254,
+      "grad_norm": 0.0005291994893923402,
+      "learning_rate": 1.0617107719336774e-05,
+      "loss": 0.0001,
+      "step": 6900
+    },
+    {
+      "epoch": 2.053068485028165,
+      "grad_norm": 0.8127829432487488,
+      "learning_rate": 1.0534753486329197e-05,
+      "loss": 0.0023,
+      "step": 6925
+    },
+    {
+      "epoch": 2.060480284613104,
+      "grad_norm": 0.008397881872951984,
+      "learning_rate": 1.0452399253321622e-05,
+      "loss": 0.0006,
+      "step": 6950
+    },
+    {
+      "epoch": 2.0678920841980433,
+      "grad_norm": 0.001353152678348124,
+      "learning_rate": 1.0370045020314044e-05,
+      "loss": 0.0002,
+      "step": 6975
+    },
+    {
+      "epoch": 2.0753038837829827,
+      "grad_norm": 2.089665174484253,
+      "learning_rate": 1.0287690787306468e-05,
+      "loss": 0.0025,
+      "step": 7000
+    },
+    {
+      "epoch": 2.0827156833679217,
+      "grad_norm": 0.004494812805205584,
+      "learning_rate": 1.0205336554298891e-05,
+      "loss": 0.0001,
+      "step": 7025
+    },
+    {
+      "epoch": 2.090127482952861,
+      "grad_norm": 0.0007270874921232462,
+      "learning_rate": 1.0122982321291315e-05,
+      "loss": 0.0003,
+      "step": 7050
+    },
+    {
+      "epoch": 2.0975392825378,
+      "grad_norm": 0.955007016658783,
+      "learning_rate": 1.0040628088283739e-05,
+      "loss": 0.0003,
+      "step": 7075
+    },
+    {
+      "epoch": 2.1049510821227395,
+      "grad_norm": 0.002524587558582425,
+      "learning_rate": 9.958273855276162e-06,
+      "loss": 0.0004,
+      "step": 7100
+    },
+    {
+      "epoch": 2.1123628817076785,
+      "grad_norm": 0.002211947925388813,
+      "learning_rate": 9.875919622268584e-06,
+      "loss": 0.0002,
+      "step": 7125
+    },
+    {
+      "epoch": 2.119774681292618,
+      "grad_norm": 0.12918436527252197,
+      "learning_rate": 9.793565389261008e-06,
+      "loss": 0.0137,
+      "step": 7150
+    },
+    {
+      "epoch": 2.127186480877557,
+      "grad_norm": 0.06649267673492432,
+      "learning_rate": 9.711211156253433e-06,
+      "loss": 0.0002,
+      "step": 7175
+    },
+    {
+      "epoch": 2.1345982804624963,
+      "grad_norm": 0.007204731460660696,
+      "learning_rate": 9.628856923245855e-06,
+      "loss": 0.0001,
+      "step": 7200
+    },
+    {
+      "epoch": 2.1420100800474353,
+      "grad_norm": 0.020881352946162224,
+      "learning_rate": 9.546502690238278e-06,
+      "loss": 0.0002,
+      "step": 7225
+    },
+    {
+      "epoch": 2.1494218796323747,
+      "grad_norm": 0.0012391919735819101,
+      "learning_rate": 9.464148457230702e-06,
+      "loss": 0.0002,
+      "step": 7250
+    },
+    {
+      "epoch": 2.156833679217314,
+      "grad_norm": 0.07225095480680466,
+      "learning_rate": 9.381794224223125e-06,
+      "loss": 0.0002,
+      "step": 7275
+    },
+    {
+      "epoch": 2.164245478802253,
+      "grad_norm": 0.0023537089582532644,
+      "learning_rate": 9.299439991215549e-06,
+      "loss": 0.0002,
+      "step": 7300
+    },
+    {
+      "epoch": 2.1716572783871926,
+      "grad_norm": 0.0023852228187024593,
+      "learning_rate": 9.217085758207972e-06,
+      "loss": 0.0001,
+      "step": 7325
+    },
+    {
+      "epoch": 2.1790690779721316,
+      "grad_norm": 0.00434435810893774,
+      "learning_rate": 9.134731525200394e-06,
+      "loss": 0.0003,
+      "step": 7350
+    },
+    {
+      "epoch": 2.186480877557071,
+      "grad_norm": 0.0017129664774984121,
+      "learning_rate": 9.05237729219282e-06,
+      "loss": 0.0002,
+      "step": 7375
+    },
+    {
+      "epoch": 2.19389267714201,
+      "grad_norm": 0.031847961246967316,
+      "learning_rate": 8.970023059185243e-06,
+      "loss": 0.0002,
+      "step": 7400
+    },
+    {
+      "epoch": 2.2013044767269494,
+      "grad_norm": 0.0012447929475456476,
+      "learning_rate": 8.887668826177665e-06,
+      "loss": 0.0003,
+      "step": 7425
+    },
+    {
+      "epoch": 2.2087162763118884,
+      "grad_norm": 0.0008703101193532348,
+      "learning_rate": 8.805314593170089e-06,
+      "loss": 0.0109,
+      "step": 7450
+    },
+    {
+      "epoch": 2.216128075896828,
+      "grad_norm": 0.0011485127033665776,
+      "learning_rate": 8.722960360162514e-06,
+      "loss": 0.0003,
+      "step": 7475
+    },
+    {
+      "epoch": 2.223539875481767,
+      "grad_norm": 0.0031809755600988865,
+      "learning_rate": 8.640606127154936e-06,
+      "loss": 0.0006,
+      "step": 7500
+    },
+    {
+      "epoch": 2.230951675066706,
+      "grad_norm": 0.025839926674962044,
+      "learning_rate": 8.55825189414736e-06,
+      "loss": 0.0009,
+      "step": 7525
+    },
+    {
+      "epoch": 2.2383634746516456,
+      "grad_norm": 0.0020416686311364174,
+      "learning_rate": 8.475897661139783e-06,
+      "loss": 0.0004,
+      "step": 7550
+    },
+    {
+      "epoch": 2.2457752742365846,
+      "grad_norm": 0.0024621151387691498,
+      "learning_rate": 8.393543428132205e-06,
+      "loss": 0.0004,
+      "step": 7575
+    },
+    {
+      "epoch": 2.253187073821524,
+      "grad_norm": 0.00846477784216404,
+      "learning_rate": 8.31118919512463e-06,
+      "loss": 0.0002,
+      "step": 7600
+    },
+    {
+      "epoch": 2.260598873406463,
+      "grad_norm": 0.0007214430370368063,
+      "learning_rate": 8.228834962117053e-06,
+      "loss": 0.0003,
+      "step": 7625
+    },
+    {
+      "epoch": 2.2680106729914025,
+      "grad_norm": 0.0028224673587828875,
+      "learning_rate": 8.146480729109475e-06,
+      "loss": 0.0001,
+      "step": 7650
+    },
+    {
+      "epoch": 2.2754224725763414,
+      "grad_norm": 0.0036359152290970087,
+      "learning_rate": 8.064126496101899e-06,
+      "loss": 0.0002,
+      "step": 7675
+    },
+    {
+      "epoch": 2.282834272161281,
+      "grad_norm": 0.01586153358221054,
+      "learning_rate": 7.981772263094324e-06,
+      "loss": 0.0,
+      "step": 7700
+    },
+    {
+      "epoch": 2.29024607174622,
+      "grad_norm": 0.0017704274505376816,
+      "learning_rate": 7.899418030086748e-06,
+      "loss": 0.0006,
+      "step": 7725
+    },
+    {
+      "epoch": 2.2976578713311593,
+      "grad_norm": 0.0014685478527098894,
+      "learning_rate": 7.81706379707917e-06,
+      "loss": 0.0001,
+      "step": 7750
+    },
+    {
+      "epoch": 2.3050696709160983,
+      "grad_norm": 0.0018168434035032988,
+      "learning_rate": 7.734709564071593e-06,
+      "loss": 0.0001,
+      "step": 7775
+    },
+    {
+      "epoch": 2.3124814705010377,
+      "grad_norm": 0.0007384305936284363,
+      "learning_rate": 7.652355331064018e-06,
+      "loss": 0.0105,
+      "step": 7800
+    },
+    {
+      "epoch": 2.319893270085977,
+      "grad_norm": 0.023864462971687317,
+      "learning_rate": 7.57000109805644e-06,
+      "loss": 0.0001,
+      "step": 7825
+    },
+    {
+      "epoch": 2.327305069670916,
+      "grad_norm": 0.0014232024550437927,
+      "learning_rate": 7.487646865048864e-06,
+      "loss": 0.0001,
+      "step": 7850
+    },
+    {
+      "epoch": 2.334716869255855,
+      "grad_norm": 0.009727663360536098,
+      "learning_rate": 7.405292632041287e-06,
+      "loss": 0.0001,
+      "step": 7875
+    },
+    {
+      "epoch": 2.3421286688407945,
+      "grad_norm": 0.002969397697597742,
+      "learning_rate": 7.322938399033711e-06,
+      "loss": 0.0005,
+      "step": 7900
+    },
+    {
+      "epoch": 2.349540468425734,
+      "grad_norm": 0.006611447781324387,
+      "learning_rate": 7.240584166026134e-06,
+      "loss": 0.0023,
+      "step": 7925
+    },
+    {
+      "epoch": 2.356952268010673,
+      "grad_norm": 0.0021192070562392473,
+      "learning_rate": 7.158229933018557e-06,
+      "loss": 0.0,
+      "step": 7950
+    },
+    {
+      "epoch": 2.3643640675956124,
+      "grad_norm": 0.010217389091849327,
+      "learning_rate": 7.075875700010981e-06,
+      "loss": 0.0002,
+      "step": 7975
+    },
+    {
+      "epoch": 2.3717758671805513,
+      "grad_norm": 0.0004124463885091245,
+      "learning_rate": 6.993521467003404e-06,
+      "loss": 0.0003,
+      "step": 8000
+    },
+    {
+      "epoch": 2.3791876667654908,
+      "grad_norm": 0.007629746571183205,
+      "learning_rate": 6.911167233995827e-06,
+      "loss": 0.0001,
+      "step": 8025
+    },
+    {
+      "epoch": 2.3865994663504297,
+      "grad_norm": 0.060671769082546234,
+      "learning_rate": 6.8288130009882514e-06,
+      "loss": 0.0001,
+      "step": 8050
+    },
+    {
+      "epoch": 2.394011265935369,
+      "grad_norm": 0.0004924974055029452,
+      "learning_rate": 6.746458767980674e-06,
+      "loss": 0.0001,
+      "step": 8075
+    },
+    {
+      "epoch": 2.401423065520308,
+      "grad_norm": 0.0014034019550308585,
+      "learning_rate": 6.664104534973098e-06,
+      "loss": 0.0001,
+      "step": 8100
+    },
+    {
+      "epoch": 2.4088348651052476,
+      "grad_norm": 0.006600458174943924,
+      "learning_rate": 6.581750301965521e-06,
+      "loss": 0.0,
+      "step": 8125
+    },
+    {
+      "epoch": 2.4162466646901866,
+      "grad_norm": 0.001500199898146093,
+      "learning_rate": 6.499396068957945e-06,
+      "loss": 0.0002,
+      "step": 8150
+    },
+    {
+      "epoch": 2.423658464275126,
+      "grad_norm": 0.07915302366018295,
+      "learning_rate": 6.4170418359503675e-06,
+      "loss": 0.0001,
+      "step": 8175
+    },
+    {
+      "epoch": 2.4310702638600654,
+      "grad_norm": 0.0008610423537902534,
+      "learning_rate": 6.334687602942792e-06,
+      "loss": 0.0001,
+      "step": 8200
+    },
+    {
+      "epoch": 2.4384820634450044,
+      "grad_norm": 0.005987235344946384,
+      "learning_rate": 6.252333369935215e-06,
+      "loss": 0.0001,
+      "step": 8225
+    },
+    {
+      "epoch": 2.445893863029944,
+      "grad_norm": 0.0015195367159321904,
+      "learning_rate": 6.169979136927638e-06,
+      "loss": 0.0001,
+      "step": 8250
+    },
+    {
+      "epoch": 2.453305662614883,
+      "grad_norm": 0.00758790411055088,
+      "learning_rate": 6.087624903920062e-06,
+      "loss": 0.0001,
+      "step": 8275
+    },
+    {
+      "epoch": 2.4607174621998222,
+      "grad_norm": 0.000942397688049823,
+      "learning_rate": 6.005270670912485e-06,
+      "loss": 0.001,
+      "step": 8300
+    },
+    {
+      "epoch": 2.468129261784761,
+      "grad_norm": 0.01316822413355112,
+      "learning_rate": 5.922916437904908e-06,
+      "loss": 0.0003,
+      "step": 8325
+    },
+    {
+      "epoch": 2.4755410613697006,
+      "grad_norm": 0.004891947843134403,
+      "learning_rate": 5.840562204897332e-06,
+      "loss": 0.0003,
+      "step": 8350
+    },
+    {
+      "epoch": 2.4829528609546396,
+      "grad_norm": 0.0005889868480153382,
+      "learning_rate": 5.758207971889755e-06,
+      "loss": 0.0001,
+      "step": 8375
+    },
+    {
+      "epoch": 2.490364660539579,
+      "grad_norm": 0.004644739907234907,
+      "learning_rate": 5.675853738882179e-06,
+      "loss": 0.0009,
+      "step": 8400
+    },
+    {
+      "epoch": 2.497776460124518,
+      "grad_norm": 0.009528938680887222,
+      "learning_rate": 5.593499505874602e-06,
+      "loss": 0.0011,
+      "step": 8425
+    },
+    {
+      "epoch": 2.5051882597094575,
+      "grad_norm": 0.003150611650198698,
+      "learning_rate": 5.511145272867025e-06,
+      "loss": 0.0005,
+      "step": 8450
+    },
+    {
+      "epoch": 2.512600059294397,
+      "grad_norm": 0.0655895322561264,
+      "learning_rate": 5.428791039859449e-06,
+      "loss": 0.0024,
+      "step": 8475
+    },
+    {
+      "epoch": 2.520011858879336,
+      "grad_norm": 0.012722983956336975,
+      "learning_rate": 5.346436806851872e-06,
+      "loss": 0.0002,
+      "step": 8500
+    },
+    {
+      "epoch": 2.527423658464275,
+      "grad_norm": 0.0908973291516304,
+      "learning_rate": 5.264082573844296e-06,
+      "loss": 0.0006,
+      "step": 8525
+    },
+    {
+      "epoch": 2.5348354580492143,
+      "grad_norm": 0.0024270841386169195,
+      "learning_rate": 5.181728340836719e-06,
+      "loss": 0.0001,
+      "step": 8550
+    },
+    {
+      "epoch": 2.5422472576341537,
+      "grad_norm": 0.015870608389377594,
+      "learning_rate": 5.099374107829143e-06,
+      "loss": 0.0001,
+      "step": 8575
+    },
+    {
+      "epoch": 2.5496590572190927,
+      "grad_norm": 0.00794550683349371,
+      "learning_rate": 5.0170198748215655e-06,
+      "loss": 0.0003,
+      "step": 8600
+    },
+    {
+      "epoch": 2.557070856804032,
+      "grad_norm": 0.0217802282422781,
+      "learning_rate": 4.93466564181399e-06,
+      "loss": 0.0007,
+      "step": 8625
+    },
+    {
+      "epoch": 2.564482656388971,
+      "grad_norm": 0.00040505212382413447,
+      "learning_rate": 4.852311408806413e-06,
+      "loss": 0.0009,
+      "step": 8650
+    },
+    {
+      "epoch": 2.5718944559739105,
+      "grad_norm": 0.0009371866472065449,
+      "learning_rate": 4.769957175798836e-06,
+      "loss": 0.0002,
+      "step": 8675
+    },
+    {
+      "epoch": 2.5793062555588495,
+      "grad_norm": 0.0234972070902586,
+      "learning_rate": 4.68760294279126e-06,
+      "loss": 0.0001,
+      "step": 8700
+    },
+    {
+      "epoch": 2.586718055143789,
+      "grad_norm": 0.0047541470266878605,
+      "learning_rate": 4.605248709783683e-06,
+      "loss": 0.0006,
+      "step": 8725
+    },
+    {
+      "epoch": 2.5941298547287284,
+      "grad_norm": 0.0020951908081769943,
+      "learning_rate": 4.522894476776106e-06,
+      "loss": 0.0,
+      "step": 8750
+    },
+    {
+      "epoch": 2.6015416543136674,
+      "grad_norm": 0.005374910309910774,
+      "learning_rate": 4.4405402437685296e-06,
+      "loss": 0.0002,
+      "step": 8775
+    },
+    {
+      "epoch": 2.6089534538986063,
+      "grad_norm": 0.0005201067542657256,
+      "learning_rate": 4.358186010760953e-06,
+      "loss": 0.0,
+      "step": 8800
+    },
+    {
+      "epoch": 2.6163652534835458,
+      "grad_norm": 0.01255794707685709,
+      "learning_rate": 4.275831777753377e-06,
+      "loss": 0.0004,
+      "step": 8825
+    },
+    {
+      "epoch": 2.623777053068485,
+      "grad_norm": 0.020597679540514946,
+      "learning_rate": 4.1934775447458e-06,
+      "loss": 0.0001,
+      "step": 8850
+    },
+    {
+      "epoch": 2.631188852653424,
+      "grad_norm": 0.0005912929191254079,
+      "learning_rate": 4.111123311738223e-06,
+      "loss": 0.0005,
+      "step": 8875
+    },
+    {
+      "epoch": 2.6386006522383636,
+      "grad_norm": 0.005026647355407476,
+      "learning_rate": 4.028769078730647e-06,
+      "loss": 0.0002,
+      "step": 8900
+    },
+    {
+      "epoch": 2.6460124518233026,
+      "grad_norm": 0.0007542280363850296,
+      "learning_rate": 3.94641484572307e-06,
+      "loss": 0.0001,
+      "step": 8925
+    },
+    {
+      "epoch": 2.653424251408242,
+      "grad_norm": 0.0026307408697903156,
+      "learning_rate": 3.864060612715494e-06,
+      "loss": 0.0001,
+      "step": 8950
+    },
+    {
+      "epoch": 2.660836050993181,
+      "grad_norm": 0.0027355581987649202,
+      "learning_rate": 3.781706379707917e-06,
+      "loss": 0.0001,
+      "step": 8975
+    },
+    {
+      "epoch": 2.6682478505781204,
+      "grad_norm": 0.0014947542222216725,
+      "learning_rate": 3.6993521467003403e-06,
+      "loss": 0.0,
+      "step": 9000
+    },
+    {
+      "epoch": 2.67565965016306,
+      "grad_norm": 0.004945519380271435,
+      "learning_rate": 3.616997913692764e-06,
+      "loss": 0.0004,
+      "step": 9025
+    },
+    {
+      "epoch": 2.683071449747999,
+      "grad_norm": 0.0699424147605896,
+      "learning_rate": 3.534643680685187e-06,
+      "loss": 0.0002,
+      "step": 9050
+    },
+    {
+      "epoch": 2.690483249332938,
+      "grad_norm": 0.02918228507041931,
+      "learning_rate": 3.4522894476776106e-06,
+      "loss": 0.0001,
+      "step": 9075
+    },
+    {
+      "epoch": 2.6978950489178772,
+      "grad_norm": 0.0005474461358971894,
+      "learning_rate": 3.369935214670034e-06,
+      "loss": 0.0001,
+      "step": 9100
+    },
+    {
+      "epoch": 2.7053068485028167,
+      "grad_norm": 0.005172941833734512,
+      "learning_rate": 3.2875809816624577e-06,
+      "loss": 0.0,
+      "step": 9125
+    },
+    {
+      "epoch": 2.7127186480877556,
+      "grad_norm": 0.0016668111784383655,
+      "learning_rate": 3.205226748654881e-06,
+      "loss": 0.0002,
+      "step": 9150
+    },
+    {
+      "epoch": 2.720130447672695,
+      "grad_norm": 0.009217623621225357,
+      "learning_rate": 3.1228725156473044e-06,
+      "loss": 0.0014,
+      "step": 9175
+    },
+    {
+      "epoch": 2.727542247257634,
+      "grad_norm": 0.020931560546159744,
+      "learning_rate": 3.040518282639728e-06,
+      "loss": 0.0003,
+      "step": 9200
+    },
+    {
+      "epoch": 2.7349540468425735,
+      "grad_norm": 0.0015090492088347673,
+      "learning_rate": 2.958164049632151e-06,
+      "loss": 0.0006,
+      "step": 9225
+    },
+    {
+      "epoch": 2.7423658464275125,
+      "grad_norm": 0.0008311509736813605,
+      "learning_rate": 2.8758098166245747e-06,
+      "loss": 0.0002,
+      "step": 9250
+    },
+    {
+      "epoch": 2.749777646012452,
+      "grad_norm": 0.002105722902342677,
+      "learning_rate": 2.7934555836169982e-06,
+      "loss": 0.001,
+      "step": 9275
+    },
+    {
+      "epoch": 2.7571894455973913,
+      "grad_norm": 0.002964325714856386,
+      "learning_rate": 2.7111013506094214e-06,
+      "loss": 0.0002,
+      "step": 9300
+    },
+    {
+      "epoch": 2.7646012451823303,
+      "grad_norm": 0.03895282372832298,
+      "learning_rate": 2.628747117601845e-06,
+      "loss": 0.0002,
+      "step": 9325
+    },
+    {
+      "epoch": 2.7720130447672693,
+      "grad_norm": 0.0005878718220628798,
+      "learning_rate": 2.5463928845942685e-06,
+      "loss": 0.0021,
+      "step": 9350
+    },
+    {
+      "epoch": 2.7794248443522087,
+      "grad_norm": 0.0044658612459897995,
+      "learning_rate": 2.4640386515866916e-06,
+      "loss": 0.0001,
+      "step": 9375
+    },
+    {
+      "epoch": 2.786836643937148,
+      "grad_norm": 0.002798914909362793,
+      "learning_rate": 2.3816844185791147e-06,
+      "loss": 0.0033,
+      "step": 9400
+    },
+    {
+      "epoch": 2.794248443522087,
+      "grad_norm": 0.008348217234015465,
+      "learning_rate": 2.2993301855715383e-06,
+      "loss": 0.0,
+      "step": 9425
+    },
+    {
+      "epoch": 2.8016602431070265,
+      "grad_norm": 1.2814829349517822,
+      "learning_rate": 2.216975952563962e-06,
+      "loss": 0.0005,
+      "step": 9450
+    },
+    {
+      "epoch": 2.8090720426919655,
+      "grad_norm": 0.0006835550884716213,
+      "learning_rate": 2.134621719556385e-06,
+      "loss": 0.0002,
+      "step": 9475
+    },
+    {
+      "epoch": 2.816483842276905,
+      "grad_norm": 0.0062784478068351746,
+      "learning_rate": 2.0522674865488086e-06,
+      "loss": 0.0001,
+      "step": 9500
+    },
+    {
+      "epoch": 2.823895641861844,
+      "grad_norm": 0.0010312970262020826,
+      "learning_rate": 1.969913253541232e-06,
+      "loss": 0.0002,
+      "step": 9525
+    },
+    {
+      "epoch": 2.8313074414467834,
+      "grad_norm": 0.0003074634587392211,
+      "learning_rate": 1.8875590205336553e-06,
+      "loss": 0.0006,
+      "step": 9550
+    },
+    {
+      "epoch": 2.8387192410317224,
+      "grad_norm": 0.012108029797673225,
+      "learning_rate": 1.8052047875260788e-06,
+      "loss": 0.0002,
+      "step": 9575
+    },
+    {
+      "epoch": 2.846131040616662,
+      "grad_norm": 0.001735036144964397,
+      "learning_rate": 1.7228505545185024e-06,
+      "loss": 0.0001,
+      "step": 9600
+    },
+    {
+      "epoch": 2.8535428402016008,
+      "grad_norm": 0.00708107789978385,
+      "learning_rate": 1.6404963215109257e-06,
+      "loss": 0.0,
+      "step": 9625
+    },
+    {
+      "epoch": 2.86095463978654,
+      "grad_norm": 0.12342353910207748,
+      "learning_rate": 1.5581420885033493e-06,
+      "loss": 0.0001,
+      "step": 9650
+    },
+    {
+      "epoch": 2.8683664393714796,
+      "grad_norm": 0.013508424162864685,
+      "learning_rate": 1.4757878554957726e-06,
+      "loss": 0.0003,
+      "step": 9675
+    },
+    {
+      "epoch": 2.8757782389564186,
+      "grad_norm": 0.010019199922680855,
+      "learning_rate": 1.3934336224881958e-06,
+      "loss": 0.0011,
+      "step": 9700
+    },
+    {
+      "epoch": 2.8831900385413576,
+      "grad_norm": 0.0018716283375397325,
+      "learning_rate": 1.3110793894806193e-06,
+      "loss": 0.0024,
+      "step": 9725
+    },
+    {
+      "epoch": 2.890601838126297,
+      "grad_norm": 0.002642314415425062,
+      "learning_rate": 1.2287251564730427e-06,
+      "loss": 0.0002,
+      "step": 9750
+    },
+    {
+      "epoch": 2.8980136377112364,
+      "grad_norm": 0.17313824594020844,
+      "learning_rate": 1.1463709234654662e-06,
+      "loss": 0.0001,
+      "step": 9775
+    },
+    {
+      "epoch": 2.9054254372961754,
+      "grad_norm": 0.0017125840531662107,
+      "learning_rate": 1.0640166904578896e-06,
+      "loss": 0.0001,
+      "step": 9800
+    },
+    {
+      "epoch": 2.912837236881115,
+      "grad_norm": 0.018445823341608047,
+      "learning_rate": 9.81662457450313e-07,
+      "loss": 0.0002,
+      "step": 9825
+    },
+    {
+      "epoch": 2.920249036466054,
+      "grad_norm": 0.0010156502248719335,
+      "learning_rate": 8.993082244427364e-07,
+      "loss": 0.0001,
+      "step": 9850
+    },
+    {
+      "epoch": 2.9276608360509933,
+      "grad_norm": 0.014282972551882267,
+      "learning_rate": 8.169539914351597e-07,
+      "loss": 0.0013,
+      "step": 9875
+    },
+    {
+      "epoch": 2.9350726356359322,
+      "grad_norm": 0.0004315798287279904,
+      "learning_rate": 7.345997584275832e-07,
+      "loss": 0.0,
+      "step": 9900
+    },
+    {
+      "epoch": 2.9424844352208717,
+      "grad_norm": 0.002567910123616457,
+      "learning_rate": 6.522455254200066e-07,
+      "loss": 0.0002,
+      "step": 9925
+    },
+    {
+      "epoch": 2.949896234805811,
+      "grad_norm": 0.003978910855948925,
+      "learning_rate": 5.6989129241243e-07,
+      "loss": 0.0,
+      "step": 9950
+    },
+    {
+      "epoch": 2.95730803439075,
+      "grad_norm": 0.009646277874708176,
+      "learning_rate": 4.875370594048534e-07,
+      "loss": 0.0,
+      "step": 9975
+    },
+    {
+      "epoch": 2.964719833975689,
+      "grad_norm": 0.000439764786278829,
+      "learning_rate": 4.0518282639727684e-07,
+      "loss": 0.0002,
+      "step": 10000
+    },
+    {
+      "epoch": 2.9721316335606285,
+      "grad_norm": 0.020473280921578407,
+      "learning_rate": 3.2282859338970024e-07,
+      "loss": 0.0001,
+      "step": 10025
+    },
+    {
+      "epoch": 2.979543433145568,
+      "grad_norm": 0.0054059335961937904,
+      "learning_rate": 2.4047436038212364e-07,
+      "loss": 0.0001,
+      "step": 10050
+    },
+    {
+      "epoch": 2.986955232730507,
+      "grad_norm": 0.0017266918439418077,
+      "learning_rate": 1.5812012737454706e-07,
+      "loss": 0.0,
+      "step": 10075
+    },
+    {
+      "epoch": 2.9943670323154463,
+      "grad_norm": 0.026864001527428627,
+      "learning_rate": 7.576589436697046e-08,
+      "loss": 0.0002,
+      "step": 10100
+    },
+    {
+      "epoch": 3.0,
+      "eval_loss": 0.0011504614958539605,
+      "eval_runtime": 12.7367,
+      "eval_samples_per_second": 111.254,
+      "eval_steps_per_second": 6.988,
+      "step": 10119
+    }
+  ],
+  "logging_steps": 25,
+  "max_steps": 10119,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 5,
+        "early_stopping_threshold": 0.01
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 0
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-10119/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f97648ab2ccbb7cbb3c9b98d0d6657cf08efa67a7046b1c06f57fd80342c95a1
+size 5496

config.json

@@ -0,0 +1,58 @@
+{
+  "_name_or_path": "nomic-ai/nomic-embed-text-v1.5",
+  "activation_function": "swiglu",
+  "architectures": [
+    "NomicBertModel"
+  ],
+  "attn_pdrop": 0.0,
+  "auto_map": {
+    "AutoConfig": "nomic-ai/nomic-bert-2048--configuration_hf_nomic_bert.NomicBertConfig",
+    "AutoModel": "nomic-ai/nomic-bert-2048--modeling_hf_nomic_bert.NomicBertModel",
+    "AutoModelForMaskedLM": "nomic-ai/nomic-bert-2048--modeling_hf_nomic_bert.NomicBertForPreTraining"
+  },
+  "bos_token_id": null,
+  "causal": false,
+  "dense_seq_output": true,
+  "embd_pdrop": 0.0,
+  "eos_token_id": null,
+  "fused_bias_fc": true,
+  "fused_dropout_add_ln": true,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-12,
+  "max_trained_positions": 2048,
+  "mlp_fc1_bias": false,
+  "mlp_fc2_bias": false,
+  "model_type": "nomic_bert",
+  "n_embd": 768,
+  "n_head": 12,
+  "n_inner": 3072,
+  "n_layer": 12,
+  "n_positions": 8192,
+  "pad_vocab_size_multiple": 64,
+  "parallel_block": false,
+  "parallel_block_tied_norm": false,
+  "prenorm": false,
+  "qkv_proj_bias": false,
+  "reorder_and_upcast_attn": false,
+  "resid_pdrop": 0.0,
+  "rotary_emb_base": 1000,
+  "rotary_emb_fraction": 1.0,
+  "rotary_emb_interleaved": false,
+  "rotary_emb_scale_base": null,
+  "rotary_scaling_factor": null,
+  "scale_attn_by_inverse_layer_idx": false,
+  "scale_attn_weights": true,
+  "summary_activation": null,
+  "summary_first_dropout": 0.0,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.44.1",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "use_flash_attn": true,
+  "use_rms_norm": false,
+  "use_xentropy": true,
+  "vocab_size": 30528
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.0.1",
+    "transformers": "4.44.1",
+    "pytorch": "2.3.0"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": null
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:759cbec716931ff76cdbd4a7871383f735a315ed6d81bcd06b39a1b086f4ffaf
+size 546938168

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

runs/Sep03_17-43-39_r-olivernormand-playground-c5ew0tu8-cbc63-h8ipc/events.out.tfevents.1725385422.r-olivernormand-playground-c5ew0tu8-cbc63-h8ipc.91.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:65c2c9fba42b706a711426cfb73630f20d1f67f15828e085d1dd91378437710e
-size 80790
+oid sha256:1d819e0eebd57384fd3a64934131684193f2456209109239fbe0cea4195876d8
+size 90699

runs/Sep03_17-43-39_r-olivernormand-playground-c5ew0tu8-cbc63-h8ipc/events.out.tfevents.1725388175.r-olivernormand-playground-c5ew0tu8-cbc63-h8ipc.91.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2df92524d9adf5173926ab0a09123736f738fdf0127fcf1d73fff1dca6169626
+size 359

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 8192,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,55 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 8192,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f97648ab2ccbb7cbb3c9b98d0d6657cf08efa67a7046b1c06f57fd80342c95a1
+size 5496

training_params.json

@@ -0,0 +1,33 @@
+{
+    "data_path": "autotrain-7flvh-khn72/autotrain-data",
+    "model": "nomic-ai/nomic-embed-text-v1.5",
+    "lr": 3e-05,
+    "epochs": 3,
+    "max_seq_length": 128,
+    "batch_size": 8,
+    "warmup_ratio": 0.1,
+    "gradient_accumulation": 1,
+    "optimizer": "adamw_torch",
+    "scheduler": "linear",
+    "weight_decay": 0.0,
+    "max_grad_norm": 1.0,
+    "seed": 42,
+    "train_split": "train",
+    "valid_split": "validation",
+    "logging_steps": -1,
+    "project_name": "autotrain-7flvh-khn72",
+    "auto_find_batch_size": false,
+    "mixed_precision": "fp16",
+    "save_total_limit": 1,
+    "push_to_hub": true,
+    "eval_strategy": "epoch",
+    "username": "olivernormand",
+    "log": "tensorboard",
+    "early_stopping_patience": 5,
+    "early_stopping_threshold": 0.01,
+    "trainer": "pair",
+    "sentence1_column": "autotrain_sentence1",
+    "sentence2_column": "autotrain_sentence2",
+    "sentence3_column": "autotrain_sentence3",
+    "target_column": "autotrain_target"
+}