Hugging Face's logo

jinaai
/
jina-bert-flash-implementation

Transformers
bert
custom_code
Inference Endpoints
🇪🇺 Region: EU
Model card Files Community
18
jina-bert-flash-implementation / modeling_bert.py

Commit History

feat: added cleaved_layers property
4441ce6

Markus28 commited on

feat: added functionality to cleave off layers from BERT encoder
86b0438

Markus28 commited on

fix BertForMaskedLM
c0b46cc

Markus28 commited on

feat: added separate BertForMaskedLM class
3cb3930

Markus28 commited on

feat: added return_dict
59c0808

Markus28 commited on

feat: choose flash attention heuristically if not set explicitly
2e2b8d0

Markus28 commited on

fix: assert is None for other kwargs too
3f5615c

Markus28 commited on

feat: added head_mask
599c64e

Markus28 commited on

fix: cast mask to bool
ca5f516

Markus28 commited on

fix: move flash components into top-level
5944ec8

Markus28 commited on

feat: try to fix import error
4c4562b

Markus28 commited on

feat: moved flash attention code into this repository
46df05d

Markus28 commited on

feat: added encode method
32458be

Markus28 commited on

fix: try to skip initialization of task type embeddings
3b35eab

Markus28 commited on

fix: try to skip initialization of task type embeddings
95ca1a8

Markus28 commited on

feat: added option for QK normalization
463061d

Markus28 commited on

feat: implement task type embeddings (#1)
8adf551
verified

Markus28 commited on

feat: added back option not to use flash attention
d4d5621

Markus28 commited on

feat: support gradient checkpointing
75d7a16

Markus28 commited on

removed unused imports
5e7b835

Markus28 commited on

removed __init__ from BertPretrainedModel
44fd417

Markus28 commited on

added config_class and base_model_prefix
45b2292

Markus28 commited on

Fixed typo
80472cb

Markus28 commited on

Fixed typo
6fb6577

Markus28 commited on

Try to subclass PretrainedModel
e209593

Markus28 commited on

Try to subclass PretrainedModel
2b23340

Markus28 commited on

strict=True for debugging
a0c289c

Markus28 commited on

try to simplify checkpointing
4c68a4c

Markus28 commited on

removed debugging
c2d8dc3

Markus28 commited on

debugging
c4185ce

Markus28 commited on

debugging
a1e1eff

Markus28 commited on

debugging assertion
4d2995d

Markus28 commited on

fix: fixed get_input_embeddings method
7e06371

Markus28 commited on

feat: added get_input_embeddings method to BertForPreTraining
bb281f0

Markus28 commited on

feat: fixed _from_config
18eed80

Markus28 commited on

removed from_config
0ce78aa

Markus28 commited on

fix: try to get from_config to work
871fd36

Markus28 commited on

feat: added from_config, also pass additional kwargs from config to model
4164fd6

Markus28 commited on

feat: updated modeling_bert.py to allow MLM-only training
0f43653

Markus28 commited on

feat: reverted monkey patch
3160695

Markus28 commited on

feat: try to monkey-patch index_first_axis
ed92835

Markus28 commited on

feat: try to fix compilation
03d8e7c

Markus28 commited on

feat: added debug print
63832b9

Markus28 commited on

feat: updated .to() override to handle kwargs
e86d612

Markus28 commited on

Revert "feat updated debug pring"
a62c2ab

Markus28 commited on

feat updated debug pring
d21ee1b

Markus28 commited on

fix: try to fix .to(torch.float16) with ALiBi
adf376f

Markus28 commited on

Revert "feat: added back option to disable flash attention"
b7ee9c4

Markus28 commited on

feat: added back option to disable flash attention
a2c07ba

Markus28 commited on

fix: always use flash attention
bfc0b2d

Markus28 commited on