Tag
DINOv2 is Meta AI's computer vision model, trained via self-supervision on 142 million images with no human labels. With a simple linear layer on the frozen encoder, it matches or beats supervised models on ImageNet classification, semantic segmentation and monocular depth estimation.
The hyperbolic tangent (tanh) is an activation function that maps any real value to the interval (-1, 1) with zero-centred output, which makes it more stable than sigmoid in hidden layers. It is the standard in LSTM and GRU memory cells, though it shares with sigmoid the vanishing-gradient problem at extreme inputs.
The sigmoid function compresses any real value into the range (0, 1), making it the natural activation function for modelling probabilities in neural networks. It is differentiable everywhere, enabling training via backpropagation, though it suffers from saturation and vanishing gradients in deep layers, where ReLU and tanh have taken over.
The Softmax function converts a vector of logits (arbitrary values) into a probability distribution where every value is positive and the values sum to exactly 1. It is the standard output-layer activation for multi-class classification, and the final operation language models use to predict the next token.
Leaky ReLU is a variant of the ReLU function that replaces zero for negative values with a small slope, keeping neurons from ever fully shutting down. This solves the dying neuron problem and improves training stability in deep neural networks, CNNs, and GAN discriminators.
ReLU (rectified linear unit) is the most widely used activation function in deep neural networks: it returns the input value if positive, and zero if negative, defined as f(x) = max(0, x). Its low computational cost and resistance to the vanishing gradient problem that hampers sigmoid made it the de facto standard since AlexNet in 2012.
The step function, or Heaviside function, is the simplest activation function in neural networks: it converts any numeric input into a binary output, 0 or 1, depending on whether it crosses a fixed threshold. It was the central mechanism of Rosenblatt's 1958 perceptron, but because it is not differentiable, it cannot be used in modern backpropagation training.
The linear function, f(x) = ax + b, is the simplest activation a neural network can use: its output is directly proportional to the input, with no non-linear transformation. It is the standard choice for the output layer in regression problems, but in hidden layers it collapses the entire network into a single linear model, so it should never be used there.
A fully connected neural network, also called a dense network, is the fundamental architecture of deep learning: every neuron in a layer connects to all neurons in the previous and next layer. This total connectivity lets it approximate any continuous function, though its computational cost grows quadratically with the number of neurons.
In a neural network, the input is represented as a column vector x in R^n that the hidden layer transforms through a weight matrix W, a bias vector b, and a non-linear activation function such as ReLU, sigmoid, or tanh. Training adjusts W and b by minimising the loss function via gradient descent and backpropagation.
A multilayer neural network consists of an input layer, one or more hidden layers, and an output layer, where each neuron weights its inputs and applies a non-linear activation function before passing the result to the next layer. Through forward propagation and backpropagation, the network adjusts millions of weights to learn hierarchical representations capable of classifying images, translating text, or generating language.
Transfer learning lets you reuse a model already trained on a massive dataset, such as ImageNet or a large text corpus, to solve a new task with far less proprietary data and compute time. It works through fine-tuning, feature extraction, or prompting, and it performs best when the source and target domains are similar to each other.
Adversarial machine learning studies deliberate attacks on AI systems (evasion, poisoning and model extraction) and the defenses used to resist them, chiefly adversarial training, robustness certification and monitoring the distribution of production input data.
Federated learning trains AI models collaboratively across many devices or organisations without moving the original data: each participant trains locally and sends only gradients to the central server. Formalised by Google in 2016, it does not guarantee privacy on its own: it needs differential privacy or secure aggregation to prevent leaks from those gradients.
Reinforcement learning is the AI technique in which an agent learns to make optimal decisions through trial and error, without labelled data: it acts in an environment, receives a reward or penalty based on the outcome, and adjusts its strategy to maximise long-term cumulative reward.
Computer vision is the branch of artificial intelligence that lets machines interpret digital images: detecting objects, segmenting regions and recognising patterns through convolutional neural networks. Since 2012, when AlexNet cut ImageNet classification error to 15.3%, it has spread into manufacturing, medicine, transport and precision agriculture.
Deep neural networks are today the foundation of almost every artificial intelligence application: from facial recognition to machine translation. Built on architectures like CNNs, RNNs, and Transformers, deep learning has transformed computer vision, speech recognition, and natural language processing over the last decade.
Modern artificial intelligence rests on three pillars: machine learning, deep neural networks, and natural language processing. These techniques have pushed image recognition and machine translation past human-level precision on specific tasks, though the overall system still depends on quality data and constant human oversight.
