Word2vec: Definition and Examples

Word2vec is a natural language processing (NLP) technique introduced in 2013 by Tomas Mikolov and his team at Google. It relies on shallow neural networks trained on vast text corpora to produce vector representations of words, known as 'word embeddings'. The fundamental idea is that words appearing in similar contexts have similar meanings, which translates to proximity in the vector space.

The model offers two main architectures: CBOW (Continuous Bag of Words), which predicts a word from its context, and Skip-gram, which predicts the context from a given word. Skip-gram works particularly well with rare words, while CBOW is faster to train and performs better on frequent words. These two approaches learn dense and compact representations, typically with 100 to 300 dimensions.

One of the most remarkable properties of Word2vec is its ability to capture analogical relationships. The most famous example is the vector equation: king - man + woman ≈ queen. This word arithmetic shows that the model encodes not only semantic similarity but also complex relational structures like gender, country capitals, or verb conjugations.

Although Word2vec has been largely supplanted by contextual models like BERT and GPT, it remains a cornerstone of modern NLP understanding. The principles it popularized — distributed representation, context-based learning, semantic vector space — are at the core of all major current language models, including those used in prompt engineering.