Euler and Runge-Kutta Methods

Most differential equations cannot be solved analytically. Instead, we approximate their solutions numerically. Suppose we are given an initial value problem

$$ \frac{dy}{dt} = f(t, y), \quad y(t_0) = y_0 $$

Our goal is to determine the value of \(y(t)\) at later times. The key idea behind numerical methods is to replace the continuous problem with a discrete one. We introduce a step size \(h\), and define a sequence of points

$$ t_n = t_0 + nh $$

We then attempt to approximate the solution at these points: \(y_n \approx y(t_n)\). The sequence \(\{y_n\}\) is intended to approximate the exact solution only at the discrete grid points \(t_n\).

To understand how numerical methods work, it is useful to rewrite the differential equation in integral form. Integrating both sides from \(t_n\) to \(t_{n+1}\), we obtain

$$ y(t_{n+1}) = y(t_n) + \int_{t_n}^{t_{n+1}} f(t, y(t)) \, dt $$

The entire problem now reduces to approximating this integral. Different numerical methods correspond to different ways of approximating the integral on the right-hand side. The quality of the numerical method therefore depends on how accurately this integral is estimated.

Euler's Method

The simplest possible approximation is to assume that the function \(f(t, y)\) does not change over the interval \([t_n, t_{n+1}]\). In other words, we approximate the integrand by its value at the left endpoint:

$$ \int_{t_n}^{t_{n+1}} f(t, y(t)) \, dt \approx h f(t_n, y_n) $$

Substituting into the integral equation gives

$$ y_{n+1} = y_n + h f(t_n, y_n) $$

This is Euler's method. Geometrically, it means that we follow the tangent line at each point to estimate the next value. The solution is therefore approximated by a sequence of straight-line segments.

If the slope changes significantly across a step, following a single tangent line becomes inaccurate. This is especially important when the solution has noticeable curvature, since the tangent line only captures the local behavior at one point.

To understand the accuracy of this method, we compare it to the Taylor expansion of the exact solution:

$$ y(t_n + h) = y(t_n) + h y'(t_n) + \frac{h^2}{2} y''(t_n) \, \; + \, \; ... $$

Using \(y' = f(t, y)\), we obtain

$$ y(t_n + h) = y_n + h f(t_n, y_n) + O(h^2) $$

Euler's method neglects all terms of order \(h^2\) and higher. The local truncation error is therefore \(O(h^2)\), while the global error scales like \(O(h)\). Euler's method is therefore referred to as a first-order method. Halving the step size approximately halves the global error, so achieving high accuracy often requires a large number of steps.

Although Euler's method is simple and intuitive, it is often not very accurate unless the step size is extremely small. This motivates the development of more accurate methods.

Improving the Approximation

The limitation of Euler's method comes from its crude approximation of the integral. In reality, the function \(f(t, y(t))\) changes over the interval, so using a single value is not sufficient. A natural idea is to sample the function at additional points within the interval and use this information to construct a better approximation. This leads to the family of Runge-Kutta methods. Runge-Kutta methods address Euler's main weakness. Instead of relying on a single slope, they sample multiple slopes within the interval and combine them in a systematic way, capturing some of the changing curvature of the solution.

Runge-Kutta Methods

Runge-Kutta methods approximate the integral by evaluating the function \(f(t, y)\) multiple times within each step and combining these values in a systematic way.

The simplest improvement is the midpoint method (a second-order Runge-Kutta method). We first estimate the value of the solution halfway through the interval using Euler's method:

$$ y_{mid} = y_n + \frac{h}{2} f(t_n, y_n) $$

We then evaluate the slope at the midpoint and use this as our estimate for the entire step:

$$ y_{n+1} = y_n + h f\left(t_n + \frac{h}{2}, y_{mid}\right) $$

By sampling the slope at the midpoint, the method captures some of the changing curvature of the solution. The resulting method has global error \(O(h^2)\), making it substantially more accurate than Euler's method.

Fourth-Order Runge-Kutta (RK4)

A more accurate and widely used method is the fourth-order Runge-Kutta method. Instead of using a single slope or even a single midpoint, RK4 samples the slope at four different points within the interval.

$$ k_1 = f(t_n, y_n) $$ $$ k_2 = f\left(t_n + \frac{h}{2}, y_n + \frac{h}{2}k_1\right) $$ $$ k_3 = f\left(t_n + \frac{h}{2}, y_n + \frac{h}{2}k_2\right) $$ $$ k_4 = f(t_n + h, y_n + hk_3) $$

These represent estimates of the slope at the beginning, two midpoints, and the end of the interval. RK4 therefore estimates how the slope evolves across the interval rather than relying on a single local value.

The next value is computed as a weighted average of these slopes:

$$ y_{n+1} = y_n + \frac{h}{6}(k_1 + 2k_2 + 2k_3 + k_4) $$

The choice of weights is not arbitrary. They are chosen so that the local truncation error is \(O(h^5)\). As a result, the accumulated global error scales like \(O(h^4)\), which is why RK4 is referred to as a fourth-order method.

Conceptually, RK4 can be viewed as constructing a much better approximation to the integral by sampling the slope across the interval and averaging in a way that accounts for curvature.

Comparison of Methods

All of these methods approximate the same underlying equation

$$ y(t_{n+1}) = y(t_n) + \int_{t_n}^{t_{n+1}} f(t, y(t)) \, dt $$

but differ in how accurately they approximate the integral.

Euler's method uses a single value and assumes the slope is constant. Runge-Kutta methods use multiple evaluations to approximate how the slope changes within the interval.

This difference has important consequences:

In practice, modern numerical solvers (such as those in computational libraries) use adaptive Runge-Kutta methods, which automatically adjust the step size to balance accuracy and efficiency. Regions where the solution changes rapidly require smaller steps, while smoother regions allow larger ones.