Unit step function

The unit step function (also called the Heaviside function) $u(t)$ is a piecewise function that is off ($u=0$) until a certain time $t=a$ and on ($u=1$) after that time.

$$u(t)=\{\begin{array}{ll}0& 0\le t<a\\ 1& t\ge a\end{array}$$

The motivation is that in electrical engineering we frequently encounter things that are either off or on, particularly starting at a certain time. Multiplying a signal by the step creates a signal that is right-sided. We can also construct square waves with the unit step:

$$x(t)=\sum _{k=-\infty }^{\infty }(-1{)}^{k}u(t-k)-u(t-1-k)$$

Important applications of the step function include the second translation theorem and the step response of a system.

Curiously:

$$\delta (t)=\frac{d}{dt}u(t)$$ $$u(t)={\int }_{-\infty }^t\delta (\tau )d\tau$$

where $\delta (t)$ is the impulse function.

Deep learning

The binary unit step function, as well as the sign function are primitive types of activation functions. They have the ability to divide the input space into two sides of a hyperplane.

However! Both aren’t differentiable, smooth, or continuous, which makes backprop difficult. These flaws motivate the use of sigmoid activation functions, which are differentiable and continuous.

In programming

In MATLAB, we can define the Heaviside function:

syms t, a
f = heaviside(t - a)