The Mathematical Symbols of AI & ML

The notation is what stalls most people, not the ideas. This is a symbol-by-symbol reference for the math of AI, machine learning, deep learning, and LLMs. Every symbol gets a plain-language explanation, the formula you meet it in, and a micro-diagram of what it actually does. It runs from the everyday operators and Greek letters through classic ML notation to the deep-learning and LLM symbols. Built to be skimmed and bookmarked.

A symbol-by-symbol reference for the math notation of AI, ML, deep learning, and LLMs. Each symbol gets a plain-English meaning, the formula you actually see it in, the confusion it causes, and a micro-diagram of what it does. It starts with the everyday operators (Σ, ∏, ∫, ∂) and Greek letters (α, λ, σ, π), moves through classic ML notation (x, X, y and ŷ), and ends with the deep-learning and LLM symbols like scaled dot-product attention and the top-p knob.

A lot of machine learning feels difficult only because the symbols are never explained first. You see something like [latex]\theta \leftarrow \theta - \eta\,\nabla_\theta J(\theta)[/latex] and you're expected to already know what every symbol means. This glossary breaks them down one by one. For each symbol, you'll learn what it means in plain English, where you'll see it, why people get confused by it, and what it's actually doing.

This isn't a book to read from start to finish. It's something you open whenever you get stuck on a symbol. It starts with basic math symbols, then Greek letters, common ML notation, deep learning symbols, and finally the symbols used in LLMs. Just jump to the section you need.

A fast way to read any unfamiliar symbol is to ask what it turns into what. Softmax turns scores into probabilities. Argmax turns scores into a position. Almost every symbol here is a small, concrete transformation.

Operators and notation

Start here with the basic math symbols you'll see in almost every formula. Learn these first, and the rest of the AI/ML notation becomes much easier to understand. Instead of looking like a different language, the formulas will start to make sense.

• Σ (sigma, sum over): Think of Σ as the add everything together symbol. It's like a calculator adding up a shopping bill [latex]\sum_{i=1}^{n} x_i = x_1 + x_2 + \dots + x_n[/latex]. The i = 1 at the bottom tells you where to start. The n at the top tells you where to stop. In AI and ML, you'll see Σ everywhere because many formulas need to add up lots of values, like losses or averages.
• ∏ (capital pi, product over): Think of ∏ as the multiply everything together symbol. Instead of adding a list of values like Σ, it multiplies them [latex]\prod_{i=1}^{n} x_i[/latex]. In AI and ML, it's often used when you want the probability of several independent events all happening together. Since multiplying lots of small probabilities quickly gives an extremely tiny number, we usually take the log, which turns the multiplication [latex]\prod[/latex] into addition [latex]\sum[/latex]. This makes the math much easier to work with. Do not confuse capital ∏ (multiply) with lowercase π (pi).
• ∫ (integral, area under a curve): Think of ∫ as the area under a curve symbol. Where Σ adds up a few separate bars, ∫ adds up a smooth curve. Slice the curve into infinitely many thin strips and sum them, written [latex]\int_a^b f(x)\,dx[/latex] between the points a and b. In AI and ML, you will see it when you need an average over a continuous range of values instead of a fixed list. The long S shape is actually an old-style letter for sum.
• ∂ (partial derivative): A partial derivative measures how steep a function is when you change only one input and hold the rest still, written [latex]\frac{\partial J}{\partial \theta_i}[/latex]. Imagine standing on a hill and asking how steep it is if you step only east, ignoring north and south. Line up these one-at-a-time slopes for every input and together they form the gradient ([latex]\nabla[/latex]). The curly ∂ is the hint that the other variables are being kept fixed, unlike the straight d of an ordinary derivative.
• Δ (capital delta, change in): Δ simply means how much something changed, the new value minus the old one, [latex]\Delta x = x_{new} - x_{old}[/latex]. For example, if a price goes from 8 to 10, then Δ price = 2. Shrink that change until it is tiny and it turns into the dx of calculus. Watch the look-alikes. Capital Δ is a change you can measure, lowercase δ usually means a small error, and ∂ is the partial derivative.
• ∝ (proportional to): ∝ means two things grow together at a steady rate, without stating the exact number. [latex]y \propto x[/latex] is short for [latex]y = k x[/latex] for some constant k. It captures the idea that if you double x, then y doubles too, while leaving the exact multiplier for later. It is handy when the pattern matters more than the precise value.
• ≈ and ≠ (approximately, not equal): These are the close enough and not equal signs. ≈ means near enough to treat as the same, like [latex]\pi \approx 3.14[/latex]; ≠ means clearly different. You will also run into ≤ and ≥ (at most and at least), plus the tricky ∼, which in statistics does not mean about equal but is drawn from.
• ∈ (element of): ∈ means belongs to. [latex]x \in S[/latex] reads as x is in the set S, and the crossed-out [latex]x \notin S[/latex] means it is not. In AI and ML you will usually see it just stating what kind of thing a variable is, as in [latex]x \in \mathbb{R}^n[/latex], which reads x is a list of n real numbers.
• ∀ and ∃ (for all, there exists): These say how many cases a statement covers. ∀ means for every single one, and ∃ means at least one exists. Saying every student passed is a much stronger claim than saying some student passed, and these two symbols make that difference exact. They are the building blocks of any careful definition.
• |x| (absolute value): Absolute value asks only how far a number is from zero, ignoring the sign, so [latex]|-3| = |3| = 3[/latex]. It keeps the size and drops the plus or minus. Keep the single bars |x| (the size of one number) separate from the double bars ‖x‖, which measure the length of a whole vector.
• ᵀ (transpose): Transpose flips a matrix onto its side. [latex]A^{\top}[/latex] swaps the rows and columns, turning an m by n table into an n by m one. It may sound minor, but it is exactly what makes two matrices line up so they can be multiplied, which is why attention is written [latex]QK^{\top}[/latex].

Greek letters in ML

The same handful of Greek letters show up again and again in ML, each with a usual job (and a few that moonlight as something completely different). Learn the default meaning and you are most of the way there.

• α (alpha, the learning rate): α is the learning rate, the size of each step the model takes as it learns, [latex]\theta \leftarrow \theta - \alpha \nabla J[/latex]. If α is too small, training is very slow and crawls along. If α is too big, it overshoots the target and can bounce around or blow up, so finding a good middle value matters. The letter η (eta) is used for exactly the same thing.
• β (beta, coefficients or momentum): β has two common jobs. In regression, the β are the coefficients, telling you how much each feature counts toward the answer. In optimisers, β is the momentum setting, how much of your previous step you carry into the next. You build up a velocity [latex]v \leftarrow \beta v + \nabla J[/latex] and then step against it, [latex]\theta \leftarrow \theta - \eta v[/latex], which keeps the updates smooth instead of jumpy. Adam uses two of them, [latex]\beta_1[/latex] and [latex]\beta_2[/latex].
• γ (gamma, the discount factor): γ is the discount factor in reinforcement learning. It decides how much a future reward is worth right now. The total reward is [latex]r_0 + \gamma r_1 + \gamma^2 r_2 + \dots[/latex] with [latex]0 \le \gamma < 1[/latex]. Rewards further away count for less, just as money is worth less the longer you wait for it. Near 1 the agent plans far ahead; near 0 it grabs the nearest reward.
• ε (epsilon, a tiny amount): ε stands for a very small number. As a safety guard it stops division by zero, as in [latex]\frac{x}{\sqrt{v} + \epsilon}[/latex], where adding a tiny ε keeps the bottom from ever being exactly 0. In reinforcement learning, the ε in epsilon-greedy is the small chance the agent tries a random action instead of its best-known one, so it keeps exploring.
• η (eta, the learning rate): η is the learning rate, the size of each training step in [latex]\theta \leftarrow \theta - \eta \nabla J[/latex]. This is the same job as α; different papers just happen to pick different letters. You can safely read η and α as the same idea, how big a step to take.
• λ (lambda, regularisation or eigenvalue): λ is usually the regularisation strength, a dial for how much the model is punished for being too complex, [latex]J + \lambda \cdot \text{penalty}[/latex]. A larger λ forces the weights smaller and keeps the model simple; a smaller λ lets it fit the training data more tightly, which can lead to overfitting. Separately, in linear algebra λ means an eigenvalue, a completely different use of the same letter.
• μ (mu, the mean): μ is the mean, the average or centre of a set of numbers, as in the [latex]\mathcal{N}(\mu, \sigma^2)[/latex] of a bell curve. Now and then it stands for a momentum term instead, but reading μ as the centre is right almost every time.
• σ (sigma, sigmoid or standard deviation): σ is the most reused letter in ML, with two jobs. As the sigmoid, it is the S-shaped curve that squashes any number into the range 0 to 1, [latex]\sigma(x) = \frac{1}{1+e^{-x}}[/latex]. As the standard deviation, it measures how spread out data is around the mean. (And capital Σ is the summation sign.) Context tells you which one is meant.
• π (pi, the policy in RL): In reinforcement learning, π is not the number 3.14, it is the agent's policy, its strategy for acting. [latex]\pi(a \mid s)[/latex] gives the chance of taking action a while in state s. That is literally how the agent decides its next move. Only in geometry does π go back to being the familiar constant.
• φ (phi, a feature map): φ is a feature map, a function that rewrites your raw input into a more useful form, [latex]\phi(x)[/latex]. It often lifts the data into a higher-dimensional space where a problem that looked tangled can suddenly be separated by a simple straight line. This is the idea behind kernels and basis functions.

Classic ML notation

Finally, the notation for the data itself and for what a model predicts. This is the everyday vocabulary of nearly every textbook and paper, so a little fluency here pays off constantly.

• x (a feature vector): x is one example, written as a list of its features, [latex]x = [x_1, x_2, \dots, x_d][/latex], such as a person's age, income, and number of clicks. Picture it as a single row of a spreadsheet. Lowercase bold x is that one example, and each slot inside it is one feature.
• X (the design matrix): X is the whole dataset laid out as a grid, one row per example and one column per feature, written [latex]X \in \mathbb{R}^{n \times d}[/latex] for n examples and d features. Capital X is the full spreadsheet, while lowercase x is a single row taken out of it.
• y and ŷ (truth versus prediction): y is the true answer (the correct label), and ŷ (y-hat) is the model's prediction. Training is all about making ŷ as close to y as possible, which is exactly what a loss like [latex]-\sum_i y_i \log \hat{y}_i[/latex] measures. The little hat always means this is an estimate, not the real value.
• x⁽ⁱ⁾ and xⱼ (which example, which feature): These are two kinds of label on the data. A superscript in parentheses points to an example, a row. [latex]x^{(i)}[/latex] is the i-th training example. A subscript points to a feature, a column. [latex]x_j[/latex] is the j-th feature. Put them together and [latex]x^{(i)}_j[/latex] is a single cell, feature j of example i.
• w and b (weights and bias): w and b are the two dials of a straight line. w (the weight) is the slope, how strongly the input affects the output; b (the bias) is a fixed shift up or down, giving [latex]\hat{y} = wx + b[/latex]. Every neuron in a network is just this wx + b followed by a nonlinearity.
• ∼ (tilde, distributed as): The squiggle reads is drawn from. [latex]x \sim \mathcal{N}(0, 1)[/latex] means x is a random sample taken from a standard bell curve. It tells you the source a random value comes from. Do not confuse it with ≈ (approximately equal); ∼ is about where randomness comes from, not about two numbers being close.
• P(A | B) (conditional probability): The bar means given. [latex]P(A \mid B)[/latex] is the chance of A once you already know B is true, like the chance of rain given that the sky is grey. Formally [latex]P(A \mid B) = \frac{P(A \cap B)}{P(B)}[/latex]. Knowing B narrows the world down to the cases where B holds, then you ask how much of that is also A. This is the engine behind Bayes theorem.
• 𝒩(μ, σ²) (the normal, or Gaussian): This is the famous bell curve, the default choice for random noise and for a network's starting weights. It is fully described by just two numbers, the mean μ (where the peak sits) and the variance σ² (how wide it spreads out). As a rule of thumb, about 68% of the values fall within one σ of the mean.
• ℝⁿ (real vector space): ℝⁿ is a formal way to say a list of n plain numbers, treated as a single point. So [latex]x \in \mathbb{R}^n[/latex] means x is a point in n-dimensional space, and an embedding of size 768 simply lives in [latex]\mathbb{R}^{768}[/latex]. The bold ℝ means real numbers, and the small superscript is just how many of them there are, that is, how many dimensions.
• L or J (loss, or cost): This is the scorecard that training tries to make as small as possible, a single number for how wrong the model currently is. Conventions differ, L is often the loss on one example and J the cost averaged over the whole dataset, [latex]J = \frac{1}{n}\sum_i L(\hat{y}_i, y_i)[/latex]. Pushing this number down is, quite literally, the whole goal of training.

Vectors and layers (ML and DL)

These operators act on the vectors that stream through a network, inside embeddings and between layers. They turn up in almost every architecture, not just transformers, so they are worth a moment.

• ⟨·,·⟩ (dot product / cosine similarity): The dot product is a similarity score for two vectors. Multiply them slot by slot, add it all up, and you get one number for how much they point the same way. Divide by their lengths and it becomes cosine similarity, [latex]\cos\theta = \dfrac{\langle a,b\rangle}{\lVert a\rVert\,\lVert b\rVert}[/latex]. A 1 means the same direction, 0 means at right angles, -1 means opposite. This, not straight-line distance, is the score behind most vector search.
• ‖·‖ (norm, the length of a vector): The norm is simply how long a vector is. The common L2 norm is the straight-line length, [latex]\lVert v\rVert_2 = \sqrt{\sum_i v_i^2}[/latex], straight from Pythagoras. If you divide a vector by its norm, you keep its direction but set its length to 1, which is how embeddings get compared purely by angle. Note the double bars, which tell it apart from the single bars of absolute value.
• ⊙ (Hadamard, elementwise product): This multiplies two vectors slot by slot, so the result keeps the same shape, [latex]h = \sigma(g)\odot c[/latex]. It works like a row of switches. A 0 in one vector turns that slot off, and a 1 lets it through. It is not normal matrix multiplication, which mixes everything together and changes the shape; the circle-dot means each position stays in its own lane.
• ⊕ (residual / skip connection): A residual adds the input back onto a layer output, [latex]y = x + F(x)[/latex], so the layer only has to learn the change instead of rebuilding everything. The big payoff is in training. That plus gives the gradient a direct path straight back through the network, which is what lets very deep models train at all. When you see a plus feeding into a box in a diagram, it is almost always this.

Scores and answers (ML and DL)

Two little functions do the job of turning a pile of raw scores into an actual answer. Both are everywhere in classification, and LLM decoding is built right on top of them.

• softmax (turns scores into probabilities): Softmax turns a set of raw scores into probabilities. It makes every score positive and forces them to add up to 100%, by exponentiating each one and dividing by the total, [latex]\sigma(z)_i = \dfrac{e^{z_i}}{\sum_j e^{z_j}}[/latex]. You will meet it at the end of a classifier and inside attention. It deliberately exaggerates the gaps, so the top score gets more than its plain share. Do not confuse it with the sigmoid, which squashes a single number, not a whole list.
• argmax (the position of the maximum): Argmax answers which one won, not by how much. It returns the position of the biggest value, not the value itself, [latex]\arg\max_i z_i[/latex]. Greedy decoding is just calling argmax at every step to grab the top token. The classic mix-up is with max. Max gives the height of the tallest bar, while argmax gives which bar it is.

Training and loss (ML and DL)

These symbols are about what the model learns from and how that learning gets scored. They are shared across classical ML and deep learning alike.

• θ (theta, the parameters): θ is shorthand for every number the model learns, the whole set of weights and biases, often millions or billions of them bundled into one letter. Training is just the search for a good [latex]\theta[/latex]. It is a naming convention, so depending on context it can mean a single weight or the entire model. Watch the collision. θ also means an angle in geometry.
• ∇θ J (the gradient step): The gradient is an arrow pointing uphill on the loss, showing the direction that makes the error grow fastest. It collects the slope for every parameter, [latex]\nabla_\theta J = \left[\dfrac{\partial J}{\partial \theta_1}, \dots\right][/latex]. So training simply walks the opposite way, downhill, [latex]\theta \leftarrow \theta - \eta\,\nabla_\theta J[/latex]. The upside-down triangle is called nabla, and that minus sign is the whole reason learning works.
• 𝔼[·] (expectation, the weighted average): Expectation is an average that weights each outcome by how likely it is, [latex]\mathbb{E}[X] = \sum_i p_i x_i[/latex]. A rare big value counts for little, while a common value counts for a lot. Almost every loss is really an expectation, the average error across your data. It is not the middle value or the most common one, each outcome pulls the average toward itself by its probability.
• -Σ y log ŷ (cross-entropy loss): Cross-entropy is the standard loss for classifiers. It stays low when you are confidently right and shoots up when you are confidently wrong. The formula is [latex]H = -\sum_i y_i \log \hat{y}_i[/latex], and since the true label is all zeros except a single 1, it boils down to [latex]-\log \hat{y}_{\text{true}}[/latex]. In plain words, your penalty is just how little probability you gave the correct answer.
• D(P ‖ Q) (KL divergence): KL divergence measures how far your approximate distribution Q is from the true one P, a kind of one-way distance, [latex]D_{KL}(P\,\Vert\,Q) = \sum_i P_i \log\dfrac{P_i}{Q_i}[/latex]. It is 0 only when the two match exactly and grows as they drift apart. The catch is that it is not symmetric, [latex]D_{KL}(P\,\Vert\,Q) \ne D_{KL}(Q\,\Vert\,P)[/latex], so the order you write P and Q in changes the answer.

Transformers and decoding (LLM)

Now the LLM-specific notation. One symbol is the beating heart of the transformer; the other three are the knobs that decide which word actually comes out.

• QKᵀ / √d (scaled dot-product attention): Attention is the mechanism that lets each word look around and decide which other words matter to it. Every query word is scored against every key word with a dot product. That score is divided by [latex]\sqrt{d_k}[/latex] to keep the numbers from blowing up, and softmax turns the scores into weights. All of it is captured by [latex]\text{softmax}\!\left(\dfrac{QK^{\top}}{\sqrt{d_k}}\right)V[/latex]. The division by [latex]\sqrt{d_k}[/latex] keeps the attention scores under control. Without it, the model may focus almost entirely on one word. With it, the model can spread its attention across several important words, leading to better understanding of the context.
• T (sampling temperature): Temperature controls how predictable or creative a language model is when choosing the next token. A low temperature (close to 0) makes the model almost always pick the most likely token, giving more consistent but often repetitive answers. A high temperature gives less likely tokens a better chance, making the output more varied and creative but also more likely to make mistakes. Temperature doesn't change what the model knows, it only changes how it chooses the next token.
• top-k (keep the k most likely): Top-k keeps only the k most likely tokens and ignores the rest. The model then picks the next token from this smaller set. If k is small, the output is more predictable. If k is larger, the model has more choices and can be more creative. The downside is that k is always fixed. Sometimes the model only needs a few choices, while other times there are many good options. Top-k can't adapt, which is why top-p is often preferred.
• top-p (nucleus, keep a share of the mass): Top-p (nucleus sampling) picks the smallest set of most likely tokens whose total probability reaches a chosen value, such as 0.9 [latex]\sum_{i \in \text{kept}} p_i \ge p[/latex]. If the model is very confident, only a few tokens are kept. If it's less confident, more tokens are included. This makes top-p adapt to the situation, which is why it's usually preferred over top-k, where the number of tokens is always fixed.

That is the core set

That covers the symbols you meet in almost every AI, ML, deep-learning, and LLM formula, from the everyday operators and Greek letters through the classic ML notation to attention and the decoding knobs. Read a formula slowly, name each symbol, and it turns from a wall of symbols into a sentence.