Orthogonal Functions
What does it mean for functions to be orthogonal?
Today in class, we discussed orthogonal functions. I find it beautiful that the geometry of vectors extends so naturally to functions, viewed as infinite vectors. I wanted to work through a simple example to make the connection concrete.
Orthogonal vectors have a dot product of zero. For real-valued functions, we can replace the sum in a dot product with an integral:
$$\langle f, g \rangle = \int_a^b f(x)g(x)\,dx$$
If this is zero, $f$ and $g$ are orthogonal on $[a,b]$.
A few modes from the Fourier basis
Let's plot $f_k(x) = \sin(kx) + \cos(kx)$ for a few integer frequencies on $[0,2\pi]$.
grid = 200
domain = [0, 2*π]
dx = (domain[1]-domain[0])/grid
grid = np.linspace(*domain, grid)
def fourier(k, x): return sin(k*x)+cos(k*x)
n = 5
basis = pd.DataFrame({k: fourier(k, grid) for k in range(1,n)}, index=grid)
ax = basis.plot.line(lw=0.4, xlim=domain)
ax.axhline(0, c='black', lw='0.3')
Now multiply two different modes point by point and integrate their product:
from scipy import integrate
def compare_two(i, j):
product = pd.Series(basis[i]*basis[j], name='product')
product = pd.DataFrame([basis[i], basis[j], product]).T
ax = product.plot.line(lw=0.5, color=['red', 'blue', 'purple'])
ax.fill_between(grid, product['product'], alpha=0.1)
return integrate.trapz(product['product'], x=product.index)
print('integral =', np.round(compare_two(3,4), 4))
The positive and negative areas cancel. The numerical result above is zero, up to rounding: these two functions are orthogonal on $[0,2\pi]$.
"Fourier modes are eigenfunctions of the derivative operator"
I overheard someone say this sentence today, and the connection caught my attention. What does it mean for a function to be an eigenvector of differentiation?
An eigenfunction is a function an operator simply multiplies by a constant. Write a Fourier mode as $\phi_k(x) = e^{ikx} = \cos(kx) + i\sin(kx)$. Then:
$$\frac{d}{dx}\phi_k(x) = ik\phi_k(x)$$
Differentiation gives back the same function, multiplied by $ik$! This is the function version of $Av = \lambda v$.
Sines and cosines individually swap under differentiation; both are eigenfunctions of the second derivative, with eigenvalue $-k^2$. In the complex Fourier basis, differentiating means multiplying each mode's coefficient by $ik$.