Experiment I

Elliptic Perimeters

approximating the elliptic integral - how fast and accurate can we get?

Here is a brief reminder of the anatomy of an ellipse.

What are we doing?

We fix the semi-major axis at π‘Ž=1 and vary the semi-minor axis 𝑏 between 0.0025 and 1, for a total of 400 shapes. Of course, f64 and single-threaded. For each of 400 shapes we compute the perimeter in several different ways and compare each result against a reference good to Β 32 digits.

methodworkworst errorns a call
Power mean, fast cbrt2 sqrt, 1 div7.28Γ—10βˆ’33.51
Ramanujan's first1 sqrt, 3 mul3.95Γ—10βˆ’32.17
Ramanujan's second1 div, 1 sqrt3.47Γ—10βˆ’44.48
Ramanujan's second + correctionr2 + 5 mul3.3Γ—10βˆ’44.79
Series in e^2loop, 4 flops a term4.39Γ—10βˆ’51492.28
Simpson, 64 panels65 sin, cos, sqrt1.62Γ—10βˆ’5696.88
Gauss, 16 nodes16 sin, cos, sqrt4.78Γ—10βˆ’6178.78
Series in hloop, 3 flops a term5.55Γ—10βˆ’1559.80
Remez [5/5]10 mul-add, 1 div, 1 sqrt4.28Γ—10βˆ’127.83
Remez [5/5] in cbrt10 mul-add, 1 div, 1 sqrt, 1 cbrt1.49Γ—10βˆ’1315.88
AGM1 sqrt a turn1.11Γ—10βˆ’1516.95
Carlson1 sqrt a turn, 1 div6.66Γ—10βˆ’1618.13
TableΒ 1. Worst relative error over the sweep, and the average cost of a call.
Fig.Β 2. Relative error against 𝑏/π‘Ž; the jagged line along the bottom is due to f64 rounding.
Fig.Β 3. Time per call for the same 12 methods.
methodπ‘π‘Ž=10.50.10.01
Series in e^2110623935000
Series in h11566501
AGM1566
TableΒ 2. Terms needed to reach the last bit.

The power mean

If I told you right now to guess the perimeter of an ellipse with a semimajor of 5 units and a semiminor of 1, I suspect a good number of you might think to average the two to get 3 and try considering the perimeter of a circle with radius 3. It’s not a terrible strategy, and I think it is worth considering. Though, there are lots of averages, so we’ll consider the most flexible which is the power mean, as we can optimise it a lot as you’ll soon see.

𝑀𝑝=(π‘Žπ‘+𝑏𝑝2)1/𝑝

The arithmetic mean formula is at 𝑝=1, the root mean square at 𝑝=2, and the geometric mean as 𝑝→0. They all don’t do too well though, and in fact the theoretical best 𝑝 via a bit of algebra turns out to be ln(2)ln(πœ‹2) which is just over 1.53. The usual choice for this is 𝑝=3/2 - it’s close enough, but I was curious how much 𝑝 really matters.

Fig.Β 4. Worst error against the exponent 𝑝.

The practical minimum is at 𝑝=1.527, where the worst error is 2.3Γ—10βˆ’3, 4.5x better than at 3/2. All things considered, given 𝑝=3/2 differs by less than one order of magnitude, and dramatically simplifies our calculation, it is definitely worth using. With 𝑠=(π‘Žπ‘Ž+𝑏𝑏)/2,

𝑀3/2=((π‘Ž3/2+𝑏3/2)/2)2/3=(𝑠2)1/3

so it is two square roots and a cube root. The cube root is, as you might expect, the expensive part: with the library root the call costs 17.34 ns, more than the whole AGM. Some bit-hacking and Newton-Raphson a la Quake:

const MAGIC: u64 = 682 << 52;

fn cbrt(x: f64) -> f64 {
    let mut y = f64::from_bits(x.to_bits() / 3 + MAGIC);
    y = y * (2.0 / 3.0) + x / (3.0 * y * y);
    y
}

Dividing the floating-point bit representation by three gives a surprisingly good initial estimate of the cube root. This works because the exponent encodes the logarithmic scale of the number, so dividing the representation by three approximately divides its logarithm by three. An iteration of Newton-Raphson afterwards brings our error from 6% to just 0.3%, and altogether brings the call from 17.34 ns to just 3.75 ns.

Where does 682 come from?
A double keeps its exponent with a bias of 1023 in IEEE-754, so dividing the bits by three divides 𝑒+1023 rather than 𝑒. Adding 2/3 of the bias back undoes that; 2/3Γ—1023=682.

In my research I learnt of another iterative method, called Halley’s method. Basically, the main differences are that Halley uses cubic convergence (NR is quadratic) and Halley also needs the second derivative (NR needs only the first).

cube rootroot errorperimeter errorns a call
1 Newton3.15Γ—10βˆ’37.28Γ—10βˆ’33.75
1 Halley1.21Γ—10βˆ’41.03Γ—10βˆ’23.86
2 Halley1.17Γ—10βˆ’121.04Γ—10βˆ’25.65
3 Halley3.18Γ—10βˆ’161.04Γ—10βˆ’28.17
libm cbrt1.76Γ—10βˆ’161.04Γ—10βˆ’217.34
TableΒ 3. The cube root, five ways.

As you can see, Halley is pretty much a free win, 1.21Γ—10βˆ’4 against the 3.15Γ—10βˆ’3 of Newton for 5% more time, since both are one division and a few multiplies. The perimeter column looks like it disagrees, but it is not measuring the root: at 𝑝=3/2 the power mean itself is only good to about 10βˆ’2 at the flat end and it comes out low there, so any root better than that lands on the same number. Newton only looks better because one step from the seed always lands above the true root, by about 0.3% here, and that bias cancels part of the formula’s error, pulling the worst case to 7.28Γ—10βˆ’3 while Halley sits at the formula’s own 1.03Γ—10βˆ’2.

Ramanujan’s first

Ramanujan published this formula (and another one) in 1914, of course without derivation.

πΏβ‰ˆπœ‹(3(π‘Ž+𝑏)βˆ’(3π‘Ž+𝑏)(π‘Ž+3𝑏))
p = pi * (3*(a + b) - sqrt((3*a + b)*(a + 3*b)))

It needs one square root and a few muls. At π‘π‘Ž=0.5 the error is 2.8Γ—10βˆ’6, at 0.1 it is 8.4Γ—10βˆ’4, and its worst case is 3.95Γ—10βˆ’3 at the flatter end of the graph near 𝑒=0. That’s still 2.5 orders better than the geometric mean for the same square root, and 0.5x the error of the power mean at 0.6x the cost. A call takes 2.17 ns.

Ramanujan’s second

The second formula moves the square root inside the correction and adds a division.

πΏβ‰ˆπœ‹(π‘Ž+𝑏)(1+3β„Ž10+4βˆ’3β„Ž)
t = (a - b) / (a + b)
h = t*t
p = pi * (a + b) * (1 + 3*h / (10 + sqrt(4 - 3*h)))

It’s not too bad! Up to 9 digits for a division and a square root, though it is 7-8 digits over most of the range and degrades at the flat end, where the correction saturates as β„Žβ†’1: the worst error is 3.47Γ—10βˆ’4.

A call takes 4.48 ns, 2x the first formula, for 3 or 4 more digits in the middle of the range. It is a good place to stop if a few digits are enough; though Ramanujan left one more correction in his notes.

Ramanujan’s second with his correction

The last formula in the notebooks is the second one with a term added inside the bracket, 3β„Ž5/217. This is a correction found by considering the difference between the exact perimeter series and his approximation.

πΏβ‰ˆπœ‹(π‘Ž+𝑏)(1+3β„Ž10+4βˆ’3β„Ž+3β„Ž5131072)
t = (a - b) / (a + b)
h = t*t
h5 = h*h*h*h*h
p = pi * (a + b) * (1 + 3*h / (10 + sqrt(4 - 3*h)) + 3*h5 / 131072)

It costs five more multiplications but no new square root. At 2:1 it takes the error from 4.56Γ—10βˆ’10 to 7.89Γ—10βˆ’11, which is 6x better and 10 digits, but at the flat end it is more or less the same: 3.3Γ—10βˆ’4 vs 3.47Γ—10βˆ’4.

So regrettably the gain isn’t too helpful: 4.79 ns against 4.48 ns is Β 6% more time for Β 5% less error. But I would be remiss to not at least consider it.

The obvious series

So far everything we’ve looked at is a basically an elaborate guess. The alternative is to expand the integral itself. We can expand for eccentricity as follows:

𝐿=2πœ‹π‘Ž(1βˆ’βˆ‘π‘›=1∞((2𝑛)!22𝑛(𝑛!)2)2𝑒2𝑛2π‘›βˆ’1)
m = 1 - (b/a)*(b/a)
g = 1
mp = 1
s = 0
for n in 1..=max_terms:
    g *= (2*n - 1) / (2*n)
    mp *= m
    term = g*g * mp / (2*n - 1)
    s += term
    if term < 1e-18: break
p = 2 * pi * a * (1 - s)

As expected, though the expansion is exact in the limit, it’s impractical at the flat end. Its terms go like 𝑒2𝑛/𝑛2, so the sum converges quickly only while 𝑒 is small, which fails as the ellipse flattens. At π‘π‘Ž=0.5 it needs quite a ridiculous 106 terms to reach the last bit; at 0.1, 2393, and at 0.01, Β 400,000; and at the flat end of the sweep, Β 6 million. The bench gives it 5000 and it is still wrong by only the 5th decimal there. So all in all, in its current state, it’s basically useless. But the principle itself can be improved!

Brute force

Thousands of terms is not particularly appealing. Another option is numerical integration.

𝐿=4∫0πœ‹2π‘Ž2cos2πœƒ+𝑏2sin2πœƒdπœƒ
end = pi / 2
h = end / 64
total = f(0) + f(end)
for i in 1..64:
    y = f(i * h)
    total += (i odd ? 4 : 2) * y
p = 4 * total * h / 3

With 64 panels, Simpson evaluates a sine, a cosine and a square root 65 times, so it is again quite expensive. It is the most accurate method here where the ellipse is nice and round, 1.83Γ—10βˆ’16 at 2:1. Better than any of the formulas above, but the error climbs unforunately quite rapidly at a sharp corner on the graph at about 𝑒=0.08: by 𝑏/π‘Ž=0.01 it is 1.48Γ—10βˆ’5.

Panels are somewhat crude. We can do better using Gauss-Legendre, which to put it simply, places them where they do the most good. A node costs about what a panel costs (11 ns of sine, cosine and square root). Error-wise, 4 nodes against 4 panels is 8.82Γ—10βˆ’4 against 4.26Γ—10βˆ’3, 16 against 16 is 4.78Γ—10βˆ’6 against 2.63Γ—10βˆ’4, and 32 against 32 is 2.35Γ—10βˆ’7 against 6.57Γ—10βˆ’5.

A final method is Carlson’s symmetric integrals. Instead of sampling the integrand, they write the perimeter as a symmetric integral of three arguments, 𝐿=8π‘Žπ‘…πΊ(0,1βˆ’π‘˜2,1), and it uses the same duplication loop logic as AGM. It is the most accurate method here at 6.66Γ—10βˆ’16 for 18.13 ns, about the same as the AGM.

A better variable

Let’s go back to the converging series. The series in 𝑒2 converges slowly not because the integral is hard or something but because 𝑒 is a poor variable. Expanding in β„Ž=((π‘Žβˆ’π‘)/(π‘Ž+𝑏))2 instead gives a series with positive coefficients:

𝐿=πœ‹(π‘Ž+𝑏)βˆ‘π‘›=0βˆžπ‘π‘›β„Žπ‘›,𝑐0=1,𝑐𝑛=π‘π‘›βˆ’1(π‘›βˆ’32𝑛)2
t = (a - b) / (a + b)
h = t*t
c = 1
hp = 1
s = 1
for n in 1..=max_terms:
    c *= (n - 1.5)*(n - 1.5) / (n*n)
    hp *= h
    term = c * hp
    s += term
    if term < 1e-18 * s: break
p = pi * (a + b) * s

β„Ž is bounded by 1 and is usually far smaller than 𝑒: at π‘π‘Ž=0.5 it is only 19. Table 2 compares the two. At π‘π‘Ž=0.5 the β„Ž series needs 15 terms where the series in 𝑒2 needs 106, and at 0.1 it is 66 against 2393. It is the same integral and the same loop with one substitution, and at the flat end of the sweep the two differ by 10 orders of magnitude on the same term budget.

Though it is exact, it is still not cheap; 3-4x the AGM for the same digits.

Remez

Another method we can use is the Chebyshev minimax, in particular the Remez exchange algorithm; this basically tries to fit a polynomial to minimise our maximum error. For what it’s worth, I think it is a very beautiful way to approximate a whole host of functions. If you’re interested, check out the Sollya project for more.

We fit the ratio 𝑃/(πœ‹(π‘Ž+𝑏)) that Ramanujan’s formulas use, not 𝑃 itself, and the criterion is relative error in that ratio: the algorithm moves (𝑃fitβˆ’π‘ƒ)/𝑃 to the same size at every extremum, alternating in sign. Since πœ‹(π‘Ž+𝑏) is exact for a given shape, that is relative error in 𝑃 too, which is the number the tables report, so nothing needs weighting by 1/𝑃. [5/5] in β„Ž:

num = p0 + h*(p1 + h*(p2 + h*(p3 + h*(p4 + h*p5))))
den = 1 + h*(q1 + h*(q2 + h*(q3 + h*(q4 + h*q5))))
p = pi * (a + b) * num / den

The fit is one-off, over 801 evenly spaced points in whichever variable is being fitted, β„Ž, 𝑠 or 𝑒, across the whole sweep (𝑏/π‘Ž from 0.0025 to 1).

With 10 multiply-adds and a division, from [3/3] to [5/5] we have 1Γ—10βˆ’6, 9.49Γ—10βˆ’8, 1.01Γ—10βˆ’8. We can improve this by making the fit be the square root. In fact, Ramanujan’s second formula is a rational function of 4βˆ’3β„Ž rather than of β„Ž, which is why it holds up at the flat end. In

𝑠=1βˆ’β„Ž=2π‘Žπ‘π‘Ž+𝑏

the singularity (β„Ž=1, i.e. the degenerate ellipse 𝑏=0) moves from 1% outside the interval to 10% outside it. We see way more bang for our buck this way.

fitted in[3/3][4/4][5/5]
β„Ž1Γ—10βˆ’69.49Γ—10βˆ’81.01Γ—10βˆ’8
1βˆ’β„Ž3.64Γ—10βˆ’88.86Γ—10βˆ’104.28Γ—10βˆ’12
1βˆ’β„Ž31.11Γ—10βˆ’76.63Γ—10βˆ’101.49Γ—10βˆ’13
TableΒ 4. Worst error for the same fit in three variables.

You’ll notice I also tested cube root for this, so that 𝑒=(1βˆ’β„Ž)1/3=𝑠2/3, which reduces our error again. [5/5] reaches 1.49Γ—10βˆ’13 against 4.28Γ—10βˆ’12. Unfortunately, it’s unsurprisingly slower than sqrt: 15.88 ns against 7.83 ns. The AGM at 16.95 ns is cheaper and 3 orders more accurate.

The arithmetic geometric mean

I mentioned this one a couple times in passing in the previous methods. I suspect that this is the most widely used approximation. It’s pretty simple, and very accurate, but not too fast. It essentially replaces a pair of numbers by their arithmetic and geometric means and repeats; it rapidly converges too.

π‘Ž0=1,𝑏0=1βˆ’π‘˜2,π‘Žπ‘›+1=π‘Žπ‘›+𝑏𝑛2,𝑏𝑛+1=π‘Žπ‘›π‘π‘›πΎ(π‘˜)=πœ‹2π‘Žβˆž,𝐸(π‘˜)=𝐾(π‘˜)(1βˆ’βˆ‘π‘›=0∞2π‘›βˆ’1𝑐𝑛2)

where 𝑐0=π‘˜ and 𝑐𝑛=π‘Žπ‘›βˆ’1βˆ’π‘π‘›βˆ’12. Then 𝐿=4π‘ŽπΈ(𝑒), as before.

r = b / a
k = sqrt(1 - r*r)
x = 1
y = r
c = k
s = k*k / 2
w = 0.5
for turn in 1..=8:
    xn = (x + y) / 2
    yn = sqrt(x*y)
    c = (x - y) / 2
    x = xn
    y = yn
    w *= 2
    s += w * c*c
    if w * c*c < 1e-18 * s: break
p = 4 * a * (pi / (2*x)) * (1 - s)

Incredibly, each turn roughly doubles the number of correct digits. 5 turns suffice at 2:1 and 6 for anything flatter in the sweep, and the worst error anywhere is 1.11Γ—10βˆ’15, which we can attribute to f64 rounding.

16.95 ns, 4x Ramanujan’s second but still less than either series or Simpson.

Which one to use

If speed is the priority, Ramanujan’s first is the best of the cheapest ones; use the second if you realise you want a bit more accuracy. If accuracy is the priority, AGM and Carlson (which is what Boost C++ uses) will both serve you well. Otherwise, for a best of both worlds, I would suggest Remez (sqrt).

Sources