Softmax function: An activation function for classification

Exponential function

  • value is always non-negative
  • if value is negative (sic), it returns a 0
  • if value is 0, it returns a 1
  • otherwise, value increases exponentially (which is why its results must be tempered by Softmax)