Operators & Orthogonal Bases

Operators

In mathematics, an operator is something that acts on a function, expression, or object and produces a new one. In this sense, an operator is like a machine: we put in a function, the operator performs a particular mathematical action, and another function or expression comes out.

For example, the derivative operator takes a function and returns its rate of change:

$$ \frac{d}{dx}\left( \, f(x) \, \right) = f'(x) $$

Operators appear everywhere in differential equations. Some common examples are:

The ordinary derivative operator $ \displaystyle \frac{d}{dx} $
The partial derivative operator $ \displaystyle \partial_x = \frac{\partial}{\partial x} $
The gradient operator $ \vec{\nabla} $
The divergence operator $ \vec{\nabla} \cdot $
The curl operator $ \vec{\nabla} \times $
The Laplacian operator $ \nabla^2 $

In partial differential equations, operators give us a compact way of describing how a function changes across space and time. Instead of writing out every derivative separately, we can use operators to express the structure of an equation more clearly.

Inner Products

An inner product is a way of measuring angles, lengths, and orthogonality in a vector space. It generalizes the familiar dot product from ordinary Euclidean geometry to more abstract spaces such as spaces of functions.

In ordinary three-dimensional space, the dot product of two vectors is

$$ \vec{v}_1 \cdot \vec{v}_2 = |\vec{v}_1||\vec{v}_2|\cos\theta $$

where $ \theta $ is the angle between the vectors. In particular, two vectors are orthogonal whenever their dot product vanishes:

$$ \vec{v}_1 \cdot \vec{v}_2 = 0 $$

In more general vector spaces, we replace the dot product with an inner product, usually written

$$ \langle f, g \rangle $$

For example, in a space of real functions on an interval $ [a,b] $, a common inner product is

$$ \langle f, g \rangle = \int_a^b f(x)g(x)\,dx $$

while for complex functions we typically include a complex conjugate:

$$ \langle f, g \rangle = \int_a^b f^*(x)g(x)\,dx $$

This allows us to extend geometric ideas such as orthogonality and projection to spaces of functions. For example, two functions are orthogonal whenever

$$ \langle f, g \rangle = 0 $$

Inner products are fundamental in differential equations because many important solution sets form orthogonal bases. Fourier series, eigenfunction expansions, quantum mechanics, and separation of variables all rely heavily on this structure.

Symmetric & Hermitian Operators

A linear operator $ A $ on a real inner product space is symmetric if and only if

$$ \langle Av_1, v_2 \rangle = \langle v_1, Av_2 \rangle $$

for any two vectors $ v_1 $ and $ v_2 $.

Consider two eigenvectors of the same operator with different eigenvalues:

$$ Av_1 = \lambda_1 v_1, \quad Av_2 = \lambda_2 v_2, \quad \lambda_1 \neq \lambda_2 $$

Then

$$ \langle Av_1, v_2 \rangle = \langle \lambda_1 v_1, v_2 \rangle = \lambda_1 \langle v_1, v_2 \rangle $$ $$ \langle v_1, Av_2 \rangle = \langle v_1, \lambda_2 v_2 \rangle = \lambda_2 \langle v_1, v_2 \rangle $$

Therefore,

$$ (\lambda_1 - \lambda_2)\langle v_1, v_2 \rangle = \langle Av_1, v_2 \rangle - \langle v_1, Av_2 \rangle $$

If $ A $ is symmetric, the right-hand side is zero, so

$$ (\lambda_1 - \lambda_2)\langle v_1, v_2 \rangle = 0 $$

Since $ \lambda_1 \neq \lambda_2 $, it follows that

$$ \langle v_1, v_2 \rangle = 0 $$

So eigenvectors of a symmetric operator with distinct eigenvalues are orthogonal.

A linear operator $ A $ on a complex inner product space is hermitian if and only if

$$ \langle Av_1, v_2 \rangle = \langle v_1, Av_2 \rangle $$

for any two vectors $ v_1 $ and $ v_2 $.

Consider, in a complex vector space, two eigenvectors of the same operator with different eigenvalues:

$$ Av_1 = \lambda_1 v_1, \quad Av_2 = \lambda_2 v_2, \quad \lambda_1 \neq \lambda_2 $$

Then

$$ \langle Av_1, v_2 \rangle = \langle \lambda_1 v_1, v_2 \rangle = \lambda_1^* \langle v_1, v_2 \rangle $$ $$ \langle v_1, Av_2 \rangle = \langle v_1, \lambda_2 v_2 \rangle = \lambda_2 \langle v_1, v_2 \rangle $$

If $ A $ is Hermitian, these are equal, so

$$ (\lambda_1^* - \lambda_2)\langle v_1, v_2 \rangle = 0 $$

and therefore

$$ \langle v_1, v_2 \rangle = 0 $$

So on a complex vector space, eigenvectors of a hermitian operator with distinct eigenvalues are orthogonal.

eg. Consider the operator $ \color{#00f0ff} \partial_x^2 $ acting on real functions on the interval $ \color{#00f0ff} [0,L] $, whose first derivatives vanish at the ends of the interval. Show that this operator is symmetric.

$$ f'(0) = f'(L) = g'(0) = g'(L) = 0 $$

We compute

$$ \langle f, \partial_x^2 g \rangle = \int_{0}^{L} f(x) \left( \partial_x^2 g(x) \right) \,dx $$

Integrate by parts

$$ \int uv' = uv - \int vu' $$ $$ u = f, \quad u' = f' $$ $$ v = g', \quad v' = g'' $$ $$ \int_{0}^{L} f(x)g''(x)\,dx = \left[f(x)g'(x)\right]_{0}^{L} - \int_{0}^{L} f'(x)g'(x)\,dx $$

The boundary term vanishes, so

$$ \int_{0}^{L} f(x)g''(x)\,dx = - \int_{0}^{L} f'(x)g'(x)\,dx $$

Now integrate by parts again

$$ u = f', \quad u' = f'' $$ $$ v = g, \quad v' = g' $$ $$ \int_{0}^{L} f'(x)g'(x)\,dx = \left[f'(x)g(x)\right]_{0}^{L} - \int_{0}^{L} f''(x)g(x)\,dx $$

Again the boundary term vanishes, so

$$ \int_{0}^{L} f'(x)g'(x)\,dx = - \int_{0}^{L} f''(x)g(x)\,dx $$

Therefore

$$ \int_{0}^{L} f(x) \left( \, \partial_x^2 g(x) \, \right) \,dx = \int_{0}^{L} \left( \, \partial_x^2 f(x) \, \right) g(x)\,dx $$

That is,

$$ \langle f, \partial_x^2 g \rangle = \langle \partial_x^2 f, g \rangle $$

Hence $ \partial_x^2 $ is symmetric on this space of functions.

Eigenvalues of a Hermitian Operator are Real

Suppose $ v $ is an eigenvector of a Hermitian operator $ A $, with

$$ Av = \lambda v $$

Then

$$ \langle Av, v \rangle = \langle v, Av \rangle $$

$$ \langle \lambda v, v \rangle = \langle v, \lambda v \rangle $$ $$ \lambda^* \langle v, v \rangle = \lambda \langle v, v \rangle $$

Since $ \langle v, v \rangle \neq 0 $, it follows that

$$ \lambda^* = \lambda $$

Therefore $ \lambda $ is real.

Orthogonal Bases and Fourier Coefficients

Let $ \{g_n\} $ be an orthogonal basis of some space $ V $, so that

$$ \langle g_n, g_m \rangle = \delta_{nm}\|g_n\|^2 $$

where

$$ \delta_{nm} = \begin{cases} 1 & n = m \\ 0 & n \neq m \end{cases} $$

Then for any vector $ v \in V $, it must be possible to find coefficients $ c_n $ such that

$$ v = \sum_n c_n g_n $$ $$ c_n = \frac{\langle g_n, v \rangle}{\langle g_n, g_n \rangle} = \frac{\langle g_n, v \rangle}{\|g_n\|^2} $$

Proof via Fourier's Trick:

$$ v = \sum_k c_k g_k $$

Take the inner product with $ g_n $:

$$ \langle g_n, v \rangle = \left\langle g_n, \, \sum_k c_k g_k \right\rangle $$ $$ = \sum_k c_k \, \langle g_n, g_k \rangle $$ $$ = \sum_k c_k \, \delta_{nk} \, \|g_n\|^2 $$ $$ = c_n \, \|g_n\|^2 $$

Therefore,

$$ c_n = \frac{\langle g_n, v \rangle}{ \; \|g_n\|^2} $$

To see Fourier's Trick explicitely in integrals click here.

eg. Consider the space of complex functions on the interval $ \color{#00f0ff} [0,L] $. Show that if

$$ \color{#00f0ff} g(x) = \sum_n c_n f_n(x) $$

where the functions $ \color{#00f0ff} f_n $ are orthogonal and $ \color{#00f0ff} \|f_n\| = 1 $, then

$$ \color{#00f0ff} \int_{0}^{L} |g(x)|^2\,dx = \sum_n |c_n|^2 $$

We compute

$$ \|g\|^2 = \langle g, g \rangle $$ $$ = \left\langle \sum_n c_n f_n, \sum_m c_m f_m \right\rangle $$ $$ = \sum_{n,m} c_n^* c_m \langle f_n, f_m \rangle $$ $$ = \sum_{n,m} c_n^* c_m \delta_{nm}\|f_n\|^2 $$ $$ = \sum_n |c_n|^2 $$

since $ \|f_n\| = 1 $. Therefore,

$$ \|g\|^2 = \int_{0}^{L} |g(x)|^2\,dx = \sum_n |c_n|^2 $$

In this question, we consider the vector space of complex periodic functions $ \color{#00f0ff} f(\theta) = f(\theta + 2\pi). $ Consider the inner product

$$ \color{#00f0ff} \langle f, g \rangle = \int_{0}^{2\pi} f^*(\theta)g(\theta)\,d\theta $$

a) Prove that $ \color{#00f0ff} f_n(\theta) = e^{in\theta} $ are orthogonal functions by showing that:

$ \quad \quad \quad $ i) They are eigenvectors of the operator $ \color{#00f0ff} \displaystyle -i\frac{\partial}{\partial \theta} $

$ \quad \quad \quad $ ii) The operator $ \color{#00f0ff} \displaystyle -i\frac{\partial}{\partial \theta} $ is Hermitian under the inner product above

where the functions $ \color{#00f0ff} f_n $ are orthogonal and $ \color{#00f0ff} \|f_n\| = 1 $, then

b) Using the inner product and orthogonality of $ \color{#00f0ff} f_n $, find $ \color{#00f0ff} c_n $ such that

$$ \color{#00f0ff} \sum_n c_n f_n(\theta) = \sin\left(\tfrac{\theta}{2}\right) $$

c) Write a solution to $ \color{#00f0ff} \nabla^2 u(r,\theta) = 0 $ on a unit disk subject to the boundary condition

$$ \color{#00f0ff} u(1,\theta) = \sin\left(\tfrac{\theta}{2}\right) $$

a) i) Let $ f_n(\theta) = e^{in\theta} $ and $ \displaystyle A = -i\frac{\partial}{\partial \theta} $

$$ Af_n = -i\frac{\partial}{\partial \theta}e^{in\theta} = n e^{in\theta} $$ $$ A f_n = n f_n $$

Therefore $ f_n $ are eigenvectors of $ \displaystyle -i\frac{\partial}{\partial \theta} $ with eigenvalue $ n $.

a) ii) Consider

$$ \langle Af, g \rangle = \int_{0}^{2\pi} (Af)^* g\,d\theta $$ $$ = i\int_{0}^{2\pi} f^{*\,\prime} g\,d\theta $$

Use integration by parts

$$ \int uv' = uv - \int vu' $$ $$ u = g, \quad u' = g' $$ $$ v = f^*, \quad v' = f^{*\,\prime} $$ $$ \int_{0}^{2\pi} f^{*\,\prime} g\,d\theta = \left[f^* g\right]_{0}^{2\pi} - \int_{0}^{2\pi} f^* g'\,d\theta $$

The boundary term vanishes since $ f $ and $ g $ are periodic:

$$ f^*(2\pi)g(2\pi) = f^*(0)g(0) $$

Therefore,

$$ \langle Af, g \rangle = -i\int_{0}^{2\pi} f^* g'\,d\theta $$ $$ = \int_{0}^{2\pi} f^* \left( -i\frac{\partial g}{\partial \theta} \right) d\theta $$ $$ = \langle f, Ag \rangle $$

Therefore $ \displaystyle -i\frac{\partial}{\partial \theta} $ is a Hermitian operator, and $ f_n(\theta) = e^{in\theta} $ are orthogonal eigenfunctions, $ \langle f_n, f_m \rangle = \delta_{nm}\|f_n\|^2 $

b) We want

$$ \sum_n c_n f_n(\theta) = \sin\left(\tfrac{\theta}{2}\right) $$ $$ c_n = \frac{\left\langle f_n, \sin\left(\tfrac{\theta}{2}\right)\right\rangle}{\langle f_n, f_n \rangle} $$

First compute the numerator:

$$ \left\langle f_n, \, \sin\left(\tfrac{\theta}{2}\right)\right\rangle = \int_{0}^{2\pi} e^{-in\theta} \left( \frac{e^{i\theta/2} - e^{-i\theta/2}}{2i} \right) \,d\theta $$ $$ = \frac{1}{2i} \int_{0}^{2\pi} e^{i\theta\left(\frac{1}{2}-n\right)} - e^{-i\theta\left(n+\frac{1}{2}\right)} \; d\theta $$

Integrating,

$$ = \frac{1}{2i} \left[ \frac{e^{i\theta\left(\frac{1}{2}-n\right)}}{i\left(\frac{1}{2}-n\right)} + \frac{e^{-i\theta\left(n+\frac{1}{2}\right)}}{i\left(n+\frac{1}{2}\right)} \right]_{0}^{2\pi} $$

This simplifies to

$$ = -\frac{1}{2} \left( \frac{e^{i\pi(1-2n)} - 1}{\frac{1}{2}-n} + \frac{e^{-i\pi(2n+1)} - 1}{n+\frac{1}{2}} \right) $$

$ e^{i\pi(\pm 1-2n)} = -1^{\pm 1-2n} = - (-1)^{-2n} = -1 $ since $ \, 2n \, $is always even. Therefore

$$ \left\langle f_n, \sin\left(\tfrac{\theta}{2}\right)\right\rangle = -\frac{1}{2} \left( \frac{-2}{\frac{1}{2}-n} + \frac{-2}{n+\frac{1}{2}} \right) $$ $$ = \frac{1}{\frac{1}{2}-n} + \frac{1}{n+\frac{1}{2}} $$ $$ = \frac{1}{\left(\frac{1}{2}+n\right)\left(\frac{1}{2}-n\right)} $$ $$ = \frac{1}{\frac{1}{4}-n^2} $$ $$ = \frac{4}{1 - 4n^2} $$

Now compute the denominator:

$$ \langle f_n, f_n \rangle = \int_{0}^{2\pi} e^{-in\theta} e^{in\theta} \,d\theta = 2\pi $$

Therefore

$$ c_n = \frac{2}{\pi}\frac{1}{1 - 4n^2} $$

c) We know that the general solution to $ \nabla^2 u(r,\theta) = 0 $ can be written in the form (see derivation)

$$ u(r,\theta) = a_0 + b_0\ln r + \sum_{\substack{n=-\infty \\ n\neq 0}}^{\infty} e^{in\theta} \left( a_n r^n + b_n r^{-n} \right) $$

Dropping terms that blow up at $ r = 0 $, we get

$$ u(r,\theta) = \sum_{n=-\infty}^{\infty} c_n r^{|n|} e^{in\theta} $$

The boundary condition is

$$ u(1,\theta) = \sum_{n=-\infty}^{\infty} c_n e^{in\theta} = \sin\left(\tfrac{\theta}{2}\right) $$

Therefore,

$$ u(r,\theta) = \sum_{n=-\infty}^{\infty} \frac{2}{\pi}\frac{1}{1 - 4n^2} r^{|n|} e^{in\theta} $$