BERT handles rare or infrequent words by using a technique called subword tokenization. In subword tokenization, the input text is divided into smaller subword units called tokens, rather than treating every word as a single unit. This approach allows BERT to effectively handle rare or infrequent words.

  BERT uses the WordPiece tokenizer, which starts by breaking down words into smaller subwords. It first applies a set of pre-defined common subwords, called WordPiece vocabulary, to split common words into subwords. For example, the word "unhappiness" might be split into "un", "##hap", and "##piness".

  During the training process, BERT also learns to generate additional subwords for rare or infrequent words that are not in the pre-defined vocabulary. This process helps BERT capture the semantic meaning of rare words even if they appear infrequently in the training data.

  When tokenizing a sentence, BERT maps each input word into its corresponding subword tokens. Rare words that are out of the vocabulary will be further divided into subwords using the WordPiece tokenizer. This subword tokenization allows BERT to retain the meaning of rare words and effectively handle them during training and inference.

  By using subword tokenization, BERT ensures that rare or infrequent words are not completely ignored, allowing the model to better understand their context and capture their overall meaning. This approach improves the performance of BERT on tasks that involve rare words or out-of-vocabulary terms.