chunked/content_aware_chunking/_model_summary/chunk_37.txt

Natural language processing

Encoder[[nlp-encoder]]
BERT is an encoder-only Transformer that randomly masks certain tokens in the input to avoid seeing other tokens, which would allow it to "cheat".