FastText: Definition and Examples

FastText is a natural language processing (NLP) library created by the Facebook AI Research team in 2016. Unlike traditional approaches like Word2Vec that represent each word as a single vector, FastText decomposes words into character substrings (n-grams) before vectorizing them. This approach captures word morphology and generates representations even for words never seen during training.

FastText operates on two main architectures: CBOW (Continuous Bag of Words) and Skip-gram, inherited from Word2Vec, but enriched by subword processing. For example, the word 'apprentissage' is broken into fragments like 'app', 'ppr', 'pre', etc. The final word vector is the sum of all its n-gram vectors, giving the model remarkable robustness against spelling mistakes, neologisms, and morphologically rich languages such as French or German.

In addition to generating embeddings, FastText includes an extremely fast supervised text classifier. This classifier can be trained on millions of examples in seconds while achieving performance comparable to much more complex deep neural networks. It is this combination of speed and accuracy that has made FastText a reference tool in the industry.

Today, FastText offers pre-trained models for 157 languages, trained on Wikipedia and Common Crawl. Although large language models (LLMs) like GPT or Claude have far surpassed FastText in contextual understanding, FastText remains relevant for tasks requiring speed, lightness, and deployment on limited resources.