ReLU (rectified linear unit)
Activation f(x) = max(0, x) widely used in deep nets; popularized mid-2010s as a fast alternative to sigmoid/tanh.
ReLU (rectified linear unit) is an activation function f(x) = max(0, x): positive inputs pass through unchanged, negative inputs become zero. It is a default nonlinearity in many modern neural networks.
Popularized in the mid-2010s (AlexNet era), ReLU largely replaced sigmoid and tanh in deep learning: it is simple, fast, and helps mitigate vanishing gradients during machine learning training. A known caveat is dying ReLU, units that stay at zero and stop learning.