Tag
A vector norm measures its length or magnitude. The L1 norm adds the absolute values of the components, while the L2 norm applies the Pythagorean theorem: the square root of the sum of squares. Both define different distances between points and underpin the regularisation that prevents overfitting in neural networks.
A scalar is a single number, a vector an ordered list of numbers, a matrix a two-dimensional table and a tensor the generalisation to any number of dimensions. In a neural network the data enters as vectors and the weights form matrices, so every layer computes z = Wx + b by combining the two.
A vector embedding is a list of real numbers that represents the semantic meaning of a piece of text, an image, or any other data. Two sentences with the same meaning produce vectors that are close together; two unrelated ones produce vectors that are far apart. Semantic search, RAG, and recommendation systems are all built on this principle.
