Convolution

For two functions $f$ and $g$ that are piecewise continuous, the continuous-time convolution of the functions is defined as the integral:

$$f\ast g={\int }_{-\infty }^{\infty }f(\tau )g(t-\tau )d\tau ={\int }_{-\infty }^{\infty }f(t-\tau )g(\tau )d\tau$$

If we’re working with the Laplace transform, the lower bound becomes 0 and the upper bound becomes $t$ itself.

The discrete-time convolution is given by:

$$f\ast g=\sum _{k=-\infty }^{\infty }f[n-k]x[k]$$

Applications

Convolutions find significant uses in broad (and often disparate) fields of the pure and applied sciences (incl. engineering). This includes:

• In deep learning:
  • Convolutional neural network (CNN)
  • Convolutional autoencoder
  • Graph convolutional network (GCN)
• Within image processing, due to some key feature extraction properties.
• In optics, to explain the physical phenomena of bokeh.
• In other forms of signal processing, including telecommunications and music.

Properties

Convolutions are commutative and distributive, similar to multiplication (in CT/DT). It also follows superposition, and is time-invariant.

A critical property is the convolution theorem: multiplication in either time/frequency domain results in a convolution in the opposite domain.

$$\mathcal{F}\{f\ast g\}=F(j\omega )G(j\omega )$$ $$\mathcal{L}\{f\ast g\}=F(s)G(s)$$

Differentiation (with respect to time or otherwise) of the convolution yields:

$$\frac{\partial }{\partial t}(v\ast w)=\frac{\partial }{\partial t}v\ast w=v\ast \frac{\partial }{\partial t}w$$

Usefully, an implication of the convolution theorem is that:

$$\mathcal{L}\{{\int }_0^{t}f(\tau )d\tau \}=\frac{F(s)}{s}$$

When we take the convolution with an impulse, i.e., the sifting property:

$$f(t)\ast \delta (t-t_0)=f(t-t_0)$$ $$f[n]\ast \delta [n-n_0]=f[n-n_0]$$

If we take the convolution of two time-shifted impulses: $\delta (t-t_0)$ and $\delta (t-t_1)$, the result is:

$$\delta (t-(t_0-t_1))=\delta (t-t_0)\ast \delta (t-t_1)$$

i.e., the result is shifted by the difference of their time shifts.

Existence

The convolution does not always exist for two input signals. The following statements will hold, where $\mathsf{L}$ refers to the Lebesgue space:

• If $f,g$ are right-sided, then $f\ast g$ exists and is right-sided.
• If $f$ or $g$ has finite duration, then $f\ast g$ exists. If both have finite duration, then $f\ast g$ also has finite duration.
• If $f,g\in {\mathsf{L}}_1$, then $f\ast g\in {\mathsf{L}}_1$ also exists. Also: $\Vert f\ast g{\Vert }_1\le \Vert f{\Vert }_1\Vert g{\Vert }_1$.
• If $f\in {\mathsf{L}}_1$ and $g\in {\mathsf{L}}_2$, then $f\ast g\in {\mathsf{L}}_2$ exists. Also: $\Vert f\ast g{\Vert }_2\le \Vert f{\Vert }_1\Vert g{\Vert }_2$.
• If $f\in {\mathsf{L}}_1$ and $g\in {\mathsf{L}}_{\infty }$, then $f\ast g\in {\mathsf{L}}_{\infty }$ exists. Also: $\Vert f\ast g\Vert \le \Vert f{\Vert }_1\Vert g{\Vert }_{\infty }$

In the first three cases, the convolution is associative as well.

Convolution theorem proof

The proof of the convolution theorem is as follows:

$$Y(s)=H(s)X(s)=H(s){\int }_0^{\infty }x(\tau )e^{-s\tau }d\tau$$

where the integral is the Laplace transform of $X$. So far we’ve made no manipulations. Since $H(s)$ is independent of $\tau$, we can move it into the integral:

$$Y(s)={\int }_0^{\infty }H(s)e^{-s\tau }x(\tau )d\tau$$ $$H(s)e^{-s\tau }=\mathcal{L}\{h(t-\tau )u(t-\tau )\}$$

With a substitution, we can:

$$Y(s)={\int }_0^{\infty }{\int }_0^{\infty }h(t-\tau )u(t-\tau )e^{-st}dtx(\tau )d\tau$$ $$Y(s)={\int }_0^{\infty }({\int }_0^{t}h(t-\tau )x(\tau )d\tau )e^{-st}dt$$

The outer integral is the Laplace transform, i.e.,:

$$y(t)={\int }_0^{t}h(t-\tau )x(\tau )d\tau$$

which is the convolution integral.

Computations

The convolution tends to be a difficult integral to evaluate. The steps we need to do are:

• Time-reverse $f$.
• Shift the result by $t$ seconds (such that we get $f(t-\tau )$).
• Multiply the result pointwise ($\forall \tau$) by $g$.
• Integrate over all $\tau$.

Some observations:

• Since the variable of integration is $\tau$, any $t$ term is functionally constant here. In more explicit terms, $\tau$ is the independent variable here, not $t$.
• Take note of any cases where the unit step function is in the integral. We can change the integral’s bounds depending on the step function’s offset. In the Laplace case, we can ignore any $u(t)$ since the lower-bound is 0.

If either function is piecewise-linear or discontinuous (like a step function), the convolution is generally difficult to compute by hand. If this is the case, we use the flip and slide procedure.

What’s going on?

Wait. So what’s the convolution and why does it matter?¹

if this section is empty i didn’t get to this yet

Footnotes

1. Much of this is from a great article at BetterExplained. ↩