Natural Language Processing (NLP)

A model can only manipulate numbers. Text, on the other hand, is a sequence of characters whose meaning depends on order, context, implicit information, and sometimes on what is not written at all. A bridge has to be built between these two worlds, and how that bridge is built determines almost everything the system will be able to do, and everything it will get wrong.

This explains some baffling behaviors. A language model capable of writing a nuanced essay can miscount the letters in a word. It can assert, with perfect confidence, a bibliographic reference that never existed. These failures seem random; they are not. Each one follows from a specific design choice, located somewhere on the bridge between characters and numbers.

This lesson walks across that bridge in order: split the text up, turn the pieces into vectors, and make each piece take the others into account. That third step is attention, the idea that tipped the field in 2017 and on which everything now called a “large language model” rests. We will code it in five lines of NumPy.

We will end with the mechanism behind hallucination. Not to lament it, but to show that it is the direct and predictable consequence of the objective these models are given.

Splitting Text: Tokenization

First decision: what is the basic unit? The obvious answer, the word, separated by spaces, quickly reveals its limits.

texte = "Le chat dort. Les chats dorment, le chaton dort aussi !"

mots = texte.lower().replace(".", " ").replace(",", " ").replace("!", " ").split()
print(mots)
# ['le', 'chat', 'dort', 'les', 'chats', 'dorment', 'le', 'chaton', 'dort', 'aussi']
print(len(mots), len(set(mots)))   # 10 8

To the machine, chat, chats, and chaton are three symbols with no relationship to each other, as foreign to one another as chat and aussi. The same goes for dort and dorment. Each inflected form has to be learned separately, which multiplies the number of examples needed.

The second problem is out-of-vocabulary words. A word absent from the training vocabulary (a proper noun, a neologism, a typo) has no representation and ends up replaced by a generic unknown symbol.

That is why current models split text into subwords. The most widespread algorithm, Byte Pair Encoding, starts from characters and iteratively merges the most frequent pairs in the corpus. The result is a compromise: common words stay in one piece, and rare words break down into known fragments.

Word-level split         : ["anticonstitutionnellement"]  → unknown if absent
Subword-level split      : ["anti", "constitution", "nelle", "ment"]  → everything is known

This choice has a direct and often misunderstood consequence: a language model does not see letters. It sees fragment identifiers. Asking it how many times the letter “r” appears in a word amounts to asking for information it does not have in front of it, which explains the well-documented fragility of these models on tasks like character counting, anagrams, or rhymes.

For classical approaches, each document is then represented as a vector of counts:

from sklearn.feature_extraction.text import CountVectorizer

corpus = [
    "le chat dort sur le canape",
    "le chien dort sur le tapis",
    "le chat mange une souris",
    "le chien mange une croquette",
    "la souris mange du fromage",
]

v = CountVectorizer()
M = v.fit_transform(corpus)
print(M.shape)                    # (5, 14)
print(v.get_feature_names_out())
# ['canape' 'chat' 'chien' 'croquette' 'dort' 'du' 'fromage' 'la' 'le'
#  'mange' 'souris' 'sur' 'tapis' 'une']

Five very short sentences already produce 14 columns. On a real corpus, this vector has tens of thousands of dimensions, almost all of which are zero for any given document. And above all, this representation completely ignores order: “le chat mange la souris” (the cat eats the mouse) and “la souris mange le chat” (the mouse eats the cat) are identical in it.

From Words to Vectors: Word Embeddings

The idea that unlocks the situation is linguistic before it is mathematical: words that appear in the same contexts have similar meanings. This is the distributional hypothesis, and it can be verified by computation.

import numpy as np

vocab = v.get_feature_names_out().tolist()

# Co-occurrence matrix: how many times two words appear
# in the same sentence.
C = np.zeros((len(vocab), len(vocab)))
for phrase in corpus:
    presents = [vocab.index(m) for m in phrase.split() if m in vocab]
    for i in presents:
        for j in presents:
            if i != j:
                C[i, j] += 1

def cosinus(u, w):
    return float(u @ w / (np.linalg.norm(u) * np.linalg.norm(w)))

i_chat = vocab.index("chat")
i_chien = vocab.index("chien")
i_fromage = vocab.index("fromage")

print(round(cosinus(C[i_chat], C[i_chien]), 3))     # 0.867
print(round(cosinus(C[i_chat], C[i_fromage]), 3))   # 0.258

On just five sentences, chat (cat) and chien (dog) get a similarity of 0.867, versus 0.258 between chat and fromage (cheese). The computation received no information about animals: it simply observed that chat and chien are surrounded by the same words (le, dort, sur, mange), whereas fromage keeps different company.

Word embeddings industrialize this idea. Rather than sparse vectors the size of the vocabulary, each word gets a dense vector of a few hundred dimensions, optimized to predict its context. This is the principle behind word2vec (Mikolov et al., 2013) and GloVe (Pennington et al., 2014).

The similarity measure itself stays the same:

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

T = TfidfVectorizer().fit_transform(corpus)
print(cosine_similarity(T[0:1], T).round(2))
# [[1.   0.61 0.38 0.18 0.  ]]

Sentence 0 (“le chat dort sur le canapé”) is close to sentence 1 (“le chien dort sur le tapis”) at 0.61, and completely unrelated to sentence 4 (“la souris mange du fromage”) at 0.00. TF-IDF weighting also corrects a flaw of raw counting: it lowers the weight of words that appear everywhere, like le, which carry no discriminating information.

One limitation remains, and it is a significant one: a classical embedding assigns a single vector per word. The word “avocat” therefore has the same representation in “un avocat au barreau” (a lawyer at the bar) and “un avocat bien mûr” (a ripe avocado). Solving this requires making the representation depend on context, which is exactly what the next mechanism contributes.

Attention: Not Looking at Every Word the Same Way

Attention answers a simple question: to represent a given word, which other words in the sentence should be taken into account, and in what proportion?

The computation uses three objects per word: a query (what this word is looking for), a key (what each word offers), and a value (the information to retrieve). The query is compared to all the keys, the scores are turned into proportions that sum to 1, and a weighted average of the values is computed from them.

import numpy as np
np.set_printoptions(suppress=True)

mots = ["chat", "souris", "dort"]

# Illustrative keys and values, chosen to keep the computation readable.
K = np.array([[4.0, 0.0, 0.0, 0.0],     # chat
              [3.0, 2.0, 0.0, 0.0],     # souris
              [0.0, 0.0, 4.0, 0.0]])    # dort
V = np.array([[10.0], [6.0], [1.0]])

Q = np.array([[4.0, 0.0, 0.0, 0.0]])    # the query looks for an animate subject

d = K.shape[1]
scores = Q @ K.T / np.sqrt(d)           # division: avoids overly spread-out scores
poids = np.exp(scores - scores.max())
poids = poids / poids.sum()             # softmax

print(scores.round(2))       # [[8. 6. 0.]]
print(poids.round(3))        # [[0.881 0.119 0.   ]]
print((poids @ V).round(2))  # [[9.52]]

The reading is straightforward: 88.1% of the attention goes to chat, 11.9% to souris, and practically nothing to dort. The resulting representation, 9.52, is almost that of chat (10.0), pulled slightly toward that of souris (6.0).

Three properties explain the success of this mechanism:

• The weights are computed, not fixed. The same word will get a different distribution in another sentence. The representation becomes contextual, which solves the “avocat” problem.
• Distance costs nothing. A word at position 1 can give its full attention to a word at position 400 as easily as to its immediate neighbor. The recurrent architectures that came before, which processed text word by word, gradually lost distant information.
• Everything is parallelizable. The scores for all words are computed in a single matrix product, and therefore on a GPU. This property, as much as the quality of the results, is what made scaling up possible.

Transformers and Large Language Models

The architecture that stacks attention layers is called the transformer, introduced in 2017 in the paper “Attention Is All You Need” by Vaswani et al. Since attention alone ignores positions, a positional encoding is added, along with ordinary dense layers and normalizations.

A large language model is a transformer trained on a task of disarming simplicity: given the preceding tokens, predict the next one. No human annotation is required; the text is its own label, which makes it possible to train on gigantic corpora.

A few documented orders of magnitude, useful for getting a sense of scale:

These figures come from the respective publication papers. Recent commercial models no longer systematically publish this information: without a source, it is better not to claim anything.

The remarkable fact, which was not anticipated, is that this narrow objective of predicting the next word produces capabilities that were never programmed: translating, summarizing, rephrasing, following an instruction. We need to be precise, however, about what is established. What is established is performance measured on evaluation tasks. What remains debated is the interpretation: whether these capabilities reflect a form of generalization or a very fine-grained interpolation from an immense corpus is an open question, and any categorical claim in either direction goes beyond what the data can establish.

Why a Language Model Hallucinates

The mechanism can be demonstrated on a tiny model. A bigram model retains only one thing: for each word, the list of words that followed it in the corpus. It then generates text by drawing at random from that list.

import random
from collections import defaultdict

texte = ("le chat dort sur le canape le chien dort sur le tapis "
         "le chat mange une souris le chien mange une croquette")
mots = texte.split()

suivants = defaultdict(list)
for a, b in zip(mots, mots[1:]):
    suivants[a].append(b)

random.seed(3)
mot = "le"
phrase = [mot]
for _ in range(11):
    if mot not in suivants:
        break
    mot = random.choice(suivants[mot])
    phrase.append(mot)

print(" ".join(phrase))
# le canape le chien mange une souris le chien dort sur le

The result is almost French. The local transitions are impeccable: “le chien mange une souris” (the dog eats a mouse) is a perfectly formed sentence. It is also false with respect to the corpus, where the dog eats a treat (croquette) and it is the cat that eats the mouse. The model did not lie and did not make a mistake in the sense of a bug: it produced a statistically plausible continuation, which is exactly, and only, what it was asked to do.

That is hallucination, at the scale of a twenty-word vocabulary. A model with billions of parameters does the same thing with incomparably greater fluency, which makes the error much harder to spot. The difference in size changes the quality of the surface, not the nature of the objective.

Four practical consequences follow, all verifiable:

• Fluency is not an indicator of reliability. An invented reference is stated with the same confidence as an accurate one, because both are produced by the same process.
• There is no source. The model does not copy from a searchable knowledge base; it produces a sequence of tokens. Asking “where does this information come from?” gets a generated answer, not a trace.
• The context window is finite. Beyond a certain length, older tokens fall out of the context being considered.
• Fixes mitigate without eliminating. Providing documents in context and requiring verifiable citations, known as retrieval-augmented generation, strongly reduces the phenomenon by anchoring the answer in real texts. To date, no published technique eliminates it completely.

The right engineering stance follows: treat the output of a language model as a proposal to be verified, not as a result. Wherever accuracy is contractual, verification must be external to the model.

Key Takeaways

• Tokenization is the first design choice and it has lasting effects. Subword splitting solves the problem of unknown words, but means a model never sees letters, hence its fragility on character counting.
• The distributional hypothesis can be verified by computation: on five sentences, chat and chien reach 0.867 similarity versus 0.258 between chat and fromage, without any zoological knowledge having been provided.
• Attention computes, for each word, the proportion of attention to give to the others. It makes the representation context-dependent, removes the cost of distance, and is computed in parallel.
• A large language model is a transformer trained to predict the next token. This single objective produces unprogrammed capabilities; their theoretical interpretation remains debated.
• Hallucination is not a malfunction but a consequence of the objective: the model optimizes for plausibility, not truth. A twenty-word bigram model already reproduces it. Any output intended for serious use must be verified outside the model.