leaky rectified linear unit

English

Diagram of the leaky rectified linear unit. The alpha parameter is 0.1.

Etymology

Named after the rectified linear unit, where “leaky” denotes how the negative half of the function becomes slightly negative, being “leaky” in comparison to the default ReLU, which is zero.

Noun

leaky rectified linear unit (plural leaky rectified linear units)

  1. (artificial intelligence, neural computing) A variation of the rectified linear unit, defined as f(x)=x when x≥0 and f(x)=αx for some small α, e.g. 0.01, when x<0.

Derived terms

This page is issued from Wiktionary. The text is available under Creative Commons Attribution-Share Alike 4.0 unless otherwise noted. Additional terms may apply for the media files.