Kernel trick
Technique that computes inner products in a high-dimensional feature space implicitly, enabling linear algorithms to fit nonlinear decision boundaries efficiently.
The kernel trick is a mathematical shortcut that lets a machine learning algorithm work in a rich feature space while operating only on pairwise similarities between raw inputs. A kernel function computes the inner product that would result if each example were mapped into a higher-dimensional representation, but without ever constructing that mapping explicitly. Linear methods such as support vector machines (SVMs) can therefore fit curved or complex decision boundaries while keeping the optimisation problem tractable.
The idea traces to Aizerman, Braverman, and Rozonoer (1964). Bernhard Boser, Isabelle Guyon, and Vladimir Vapnik combined it with maximum-margin classifiers in 1992, laying the basis for nonlinear SVMs. Corinna Cortes and Vapnik published the widely used 1995 formulation. Common kernels include polynomial and radial-basis functions; choosing the right kernel was often as important as tuning regularisation in pipelines before the era of deep learning. See the support vector machines article and timeline entry.