Experiment I
Elliptic Perimeters
approximating the elliptic integral - how fast and accurate can we get?
Here is a brief reminder of the anatomy of an ellipse.
An ellipse is the set of points whose distances and to two fixed points, the foci, add up to a constant.
Fig.Β 1. For every on the curve, we have . - The longest chord is the major axis, and half of its length is denoted by .
- The chord through the centre at right angles to it is the minor axis, and half of its length is denoted by .
The distance from center to either foci is denoted by .
Why is the constant ?
Choose at the right-hand end of the major axis; and are the distances to the foci, which add to .Where does come from?
Choose at the top, at . The distances to either focus are equal by symmetry and still add to , hence each is . Itβs hopefully clear that due to Pythagoras!- A circle is the case , so its eccentricity is ; otherwise , and as the ellipse flattens towards its major axis.
- We say a function / expression is elementary if it can be expressed as the sum, product, quotient or composition of the trigonometric, inverse-trigonometric, exponential, logarithmic, or n-th root functions.
- An algebraic function is a function that satisfies a finite polynomial in and . For example, , , , etc.
- Liouvilleβs theorem tells us that if a function has an elementary integral, the integrand (the expression youβre integrating) must be expressed as the derivative of an algebraic function, plus the sum of a bunch of terms in the form (called a logarithmic derivative), where each is an algebraic function.
- A circleβs area, , and perimeter, , are elementary.
An ellipseβs area, (note the symmetry with a circle, where ), is elementary. However, its perimeter is not elementary. It is given by where is the βcomplete elliptic integral of the second kindβ.
Why is the perimeterβs integral not elementary?
The integral
Letβs parametrise to get and . Using the arc-length formula on quadrant one:
Since
we have the following for the total perimeter:
And as we will soon show, this integral has no elementary antiderivative, so we give it a name, . The same was done for . So we have:
A brief introduction to complex analysis
This proof uses some complex analysis. In a nutshell: consider a function like . We call this the Laurent series expansion form; it is the sum of a bunch of terms in the form . This function shoots off to as . At , we have a pole. We have different ways of classifying poles - one such way is via numbers called residues. The example above has residue 4, and we calculate it as the coefficient of the term. Another measure is the order, which is simply the highest negative power. The above example has order 2. There are some useful theorems / definitions concerning these:
- Differentiation makes poles one order stronger. This is somewhat intuitive if we consider an example like - the order increased from 1 to 2.
- A holomorphic function is one that has a complex derivative at every point in a region. In laypersonβs terms, when you zoom in close enough near any point, they can be expressed as a convergent power series (an infinite polynomial). Note that their differentiability means they canβt have poles.
- Derivatives of the functions weβre considering (called meromorphic functions; these are holomorphic everywhere in a region except at their poles) always have residue 0. This should hopefully be clear: thereβs no way for the derivative of a function containing finitely many powers of to have a term.
The residues of logarithmic derivatives are always integers. An example:
which has residue .
Thatβs not quite the whole story, and there is a slight non-triviality. We are considering the integral of a differential . When we change coordinates as we often do in calculus, both function and differential must be transformed too. This can cause poles to disappear. Consider for example , and substituting , so we obtain which trivially has no pole.
Rewriting the integral in a more useful form
Now letβs make the substitution , so that :
We want to show that itβs not possible to represent it as a Liouville decomposition. First we multiply by the numerator over itself to get:
Let , so our differential is
Finding the poles of our differential
We first want to find all the poles of this differential.
At , we have a pole. However, we need to look at the entire differential, not just its coefficient. So, near , choose , and hence . Then:
and
Substituting everything into our differential gives
Notice that the factors of have cancelled. Setting is totally fine so at we have no pole. Same logic applies for the other value, .
At , the differential is also regular. To see why, let be for example. Since has a simple zero at , we can write:
where is a nonzero constant. We can do this legitimately because weβre near a zero, and a polynomial zoomed in looks like a line; in our case, .
Rearranging and differentiating gives
Note that is nonzero; this is because if , and , then . Our differential contains a factor of from its numerator, and another from . So when we reconstruct weβll get something like:
And as , this of course tends to 0 instead of becoming a pole. So I hope youβre convinced there are no poles at any finite point.
But weβre not quite done! We still have to consider infinity. Letβs switch to , so that instead of considering , we can consider 0 instead. To find the behaviour of our differential, first calculate its coefficient:
We can expand the expression inside the square root using the geometric series expansion:
Taking the square root gives
Only even powers of appear! And since:
then:
There is a pole of order 2, but notice how thereβs no term involving . Therefore, its residue is zero, so we have now shown that every residue of is zero.
So far so good. Now the proof gets maybe a little bit more complicatedβ¦
What if an elementary antiderivative existed?
FTSOC assume that our integral has an elementary primitive. By Liouvilleβs theorem, its differential must satisfy the decomposition described earlier:
Recall that derivatives always have residue zero, whereas logarithmic derivatives have integer residues.
Let
be the sum of the logarithmic terms. Since both and have residue zero everywhere, must also have residue zero everywhere.
Each function can only have finitely many zeros and poles. Let these points be . At each point , the residue of is
where is the order of the zero or pole of at . These numbers are integers.
Why do the logarithmic terms disappear?
We now have to use a little bit of linear algebra for the sake of rigor. Itβd be all too easy if we could just conclude all , as itβs completely possible for some of them to cancel each other out.
The equations above form a system of linear equations with integer coefficients. Consider something like which has as a solution. But note that any scalar multiple of this solution is also valid, and this generalises for bigger linear systems too. So its solutions can be written as linear combinations of vectors with rational values.
More rigorously, there are rational vectors and constants such that the vector of coefficients is a linear combination of these rational vectors. For each rational vector, we can multiply by a suitable integer to make all its entries integers, by virtue of the definition of a rational number. Call the resulting integer vectors .
The idea from here is to use the fact that if we multiply two functions together, their orders add, and if we raise a function to a power, its order is multiplied.
Each of these integer vectors gives us a product
whose zeros and poles cancel at every point. Thus has no zeros or poles anywhere on our curve. But a nonconstant algebraic function on our curve must have zeros and poles.
Why?
On the compact version (it doesnβt have any missing end points or places where it stretches to infinity) of our curve, suppose we have a holomorphic function . Since the curve is compact, must have a maximum somewhere. We want to show that a holomorphic function can only have a maximum of its absolute value if it is constant.
Near any point , a holomorphic function can be written as a power series. If it isnβt locally constant, let be the first nonconstant term. If isnβt zero, we can choose the direction of so that this term points in the same direction as . For sufficiently small , the higher order terms are smaller, so the modulus increases. This contradicts having a maximum. If , a maximum of modulus zero means that is zero in a neighbourhood anyway. Thus, a maximum of the modulus forces the function to be locally constant, and the Identity Theorem tells us it is constant everywhere on our connected curve.
Therefore, if a meromorphic function has no poles, it is holomorphic everywhere and must be constant. The same argument applies to its reciprocal: if a function has no zeroes, its reciprocal has no poles, and must be constant. So a nonconstant meromorphic function must have both zeroes and poles. In fact, the number of poles and the number of zeroes is equal, counting multiplicity!
Therefore, using the contrapositive, each is constant, meaning
Since the original logarithmic sum is a linear combination of these expressions, it must also be zero. Hence, all the logarithmic terms are gone, leaving
Characterising A
Since , and has poles of order 2 at infinity and nowhere else, can only have simple poles at infinity and no poles anywhere else. This is because of the property we established earlier, that differentiation increases pole order.
Now letβs find all algebraic functions with this property. Since is a polynomial in , we can express any function on our curve, which are functions of that involve:
where and are functions of .
Because has no poles at finite points, neither nor can have a finite pole. To see why, consider the two points above any ordinary finite value of : one has and the other has by symmetry. Adding and subtracting the two values of shows that both and must be finite.
At the branch points, the same conclusion follows by using : the terms involving contain even powers of , while those involving contain odd powers, so their poles cannot cancel.
Therefore, and must be polynomials.
Why does this lead to a contradiction?
Now consider what happens at infinity. Since
the leading term of is either or , depending on which point at infinity we approach.
If is nonzero, then has a pole of order at least 2 at infinity. The polynomial cannot cancel this at both points at infinity, because the leading term of has opposite signs at the two points. This contradicts the fact that can have at most simple poles at infinity. Therefore, , leaving
Since can have at most simple poles at infinity, can have degree no more than 1. Hence:
Differentiating gives
But this would mean
Squaring and cancelling:
Comparing coefficients gives
Hence , which is impossible for an ellipse with . Therefore, the elliptic integral has no elementary antiderivative.
- Although it has no elementary form, we can approximate it numerically.
What are we doing?
We fix the semi-major axis at and vary the semi-minor axis between and , for a total of 400 shapes. Of course, f64 and single-threaded. For each of 400 shapes we compute the perimeter in several different ways and compare each result against a reference good to Β 32 digits.
| method | work | worst error | ns a call |
|---|---|---|---|
| Power mean, fast cbrt | 2 sqrt, 1 div | 3.51 | |
| Ramanujan's first | 1 sqrt, 3 mul | 2.17 | |
| Ramanujan's second | 1 div, 1 sqrt | 4.48 | |
| Ramanujan's second + correction | r2 + 5 mul | 4.79 | |
| Series in e^2 | loop, 4 flops a term | 1492.28 | |
| Simpson, 64 panels | 65 sin, cos, sqrt | 696.88 | |
| Gauss, 16 nodes | 16 sin, cos, sqrt | 178.78 | |
| Series in h | loop, 3 flops a term | 59.80 | |
| Remez [5/5] | 10 mul-add, 1 div, 1 sqrt | 7.83 | |
| Remez [5/5] in cbrt | 10 mul-add, 1 div, 1 sqrt, 1 cbrt | 15.88 | |
| AGM | 1 sqrt a turn | 16.95 | |
| Carlson | 1 sqrt a turn, 1 div | 18.13 |
| method | ||||
|---|---|---|---|---|
| Series in e^2 | 1 | 106 | 2393 | 5000 |
| Series in h | 1 | 15 | 66 | 501 |
| AGM | 1 | 5 | 6 | 6 |
The power mean
If I told you right now to guess the perimeter of an ellipse with a semimajor of 5 units and a semiminor of 1, I suspect a good number of you might think to average the two to get 3 and try considering the perimeter of a circle with radius 3. Itβs not a terrible strategy, and I think it is worth considering. Though, there are lots of averages, so weβll consider the most flexible which is the power mean, as we can optimise it a lot as youβll soon see.
The arithmetic mean formula is at , the root mean square at , and the geometric mean as . They all donβt do too well though, and in fact the theoretical best via a bit of algebra turns out to be which is just over 1.53. The usual choice for this is - itβs close enough, but I was curious how much really matters.
The practical minimum is at , where the worst error is , 4.5x better than at . All things considered, given differs by less than one order of magnitude, and dramatically simplifies our calculation, it is definitely worth using. With ,
so it is two square roots and a cube root. The cube root is, as you might expect, the expensive part: with the library root the call costs 17.34 ns, more than the whole AGM. Some bit-hacking and Newton-Raphson a la Quake:
const MAGIC: u64 = 682 << 52;
fn cbrt(x: f64) -> f64 {
let mut y = f64::from_bits(x.to_bits() / 3 + MAGIC);
y = y * (2.0 / 3.0) + x / (3.0 * y * y);
y
}Dividing the floating-point bit representation by three gives a surprisingly good initial estimate of the cube root. This works because the exponent encodes the logarithmic scale of the number, so dividing the representation by three approximately divides its logarithm by three. An iteration of Newton-Raphson afterwards brings our error from 6% to just 0.3%, and altogether brings the call from 17.34 ns to just 3.75 ns.
Where does 682 come from?
In my research I learnt of another iterative method, called Halleyβs method. Basically, the main differences are that Halley uses cubic convergence (NR is quadratic) and Halley also needs the second derivative (NR needs only the first).
| cube root | root error | perimeter error | ns a call |
|---|---|---|---|
| 1 Newton | 3.75 | ||
| 1 Halley | 3.86 | ||
| 2 Halley | 5.65 | ||
| 3 Halley | 8.17 | ||
| libm cbrt | 17.34 |
As you can see, Halley is pretty much a free win, against the of Newton for 5% more time, since both are one division and a few multiplies. The perimeter column looks like it disagrees, but it is not measuring the root: at the power mean itself is only good to about at the flat end and it comes out low there, so any root better than that lands on the same number. Newton only looks better because one step from the seed always lands above the true root, by about 0.3% here, and that bias cancels part of the formulaβs error, pulling the worst case to while Halley sits at the formulaβs own .
Ramanujanβs first
Ramanujan published this formula (and another one) in 1914, of course without derivation.
p = pi * (3*(a + b) - sqrt((3*a + b)*(a + 3*b)))It needs one square root and a few muls. At the error is , at it is , and its worst case is at the flatter end of the graph near . Thatβs still 2.5 orders better than the geometric mean for the same square root, and 0.5x the error of the power mean at 0.6x the cost. A call takes 2.17 ns.
Ramanujanβs second
The second formula moves the square root inside the correction and adds a division.
t = (a - b) / (a + b)
h = t*t
p = pi * (a + b) * (1 + 3*h / (10 + sqrt(4 - 3*h)))Itβs not too bad! Up to 9 digits for a division and a square root, though it is 7-8 digits over most of the range and degrades at the flat end, where the correction saturates as : the worst error is .
A call takes 4.48 ns, 2x the first formula, for 3 or 4 more digits in the middle of the range. It is a good place to stop if a few digits are enough; though Ramanujan left one more correction in his notes.
Ramanujanβs second with his correction
The last formula in the notebooks is the second one with a term added inside the bracket, . This is a correction found by considering the difference between the exact perimeter series and his approximation.
t = (a - b) / (a + b)
h = t*t
h5 = h*h*h*h*h
p = pi * (a + b) * (1 + 3*h / (10 + sqrt(4 - 3*h)) + 3*h5 / 131072)It costs five more multiplications but no new square root. At 2:1 it takes the error from to , which is 6x better and 10 digits, but at the flat end it is more or less the same: vs .
So regrettably the gain isnβt too helpful: 4.79 ns against 4.48 ns is Β 6% more time for Β 5% less error. But I would be remiss to not at least consider it.
The obvious series
So far everything weβve looked at is a basically an elaborate guess. The alternative is to expand the integral itself. We can expand for eccentricity as follows:
m = 1 - (b/a)*(b/a)
g = 1
mp = 1
s = 0
for n in 1..=max_terms:
g *= (2*n - 1) / (2*n)
mp *= m
term = g*g * mp / (2*n - 1)
s += term
if term < 1e-18: break
p = 2 * pi * a * (1 - s)As expected, though the expansion is exact in the limit, itβs impractical at the flat end. Its terms go like , so the sum converges quickly only while is small, which fails as the ellipse flattens. At it needs quite a ridiculous 106 terms to reach the last bit; at , 2393, and at , Β 400,000; and at the flat end of the sweep, Β 6 million. The bench gives it 5000 and it is still wrong by only the 5th decimal there. So all in all, in its current state, itβs basically useless. But the principle itself can be improved!
Brute force
Thousands of terms is not particularly appealing. Another option is numerical integration.
end = pi / 2
h = end / 64
total = f(0) + f(end)
for i in 1..64:
y = f(i * h)
total += (i odd ? 4 : 2) * y
p = 4 * total * h / 3With 64 panels, Simpson evaluates a sine, a cosine and a square root 65 times, so it is again quite expensive. It is the most accurate method here where the ellipse is nice and round, at 2:1. Better than any of the formulas above, but the error climbs unforunately quite rapidly at a sharp corner on the graph at about : by it is .
Panels are somewhat crude. We can do better using Gauss-Legendre, which to put it simply, places them where they do the most good. A node costs about what a panel costs (11 ns of sine, cosine and square root). Error-wise, 4 nodes against 4 panels is against , 16 against 16 is against , and 32 against 32 is against .
A final method is Carlsonβs symmetric integrals. Instead of sampling the integrand, they write the perimeter as a symmetric integral of three arguments, , and it uses the same duplication loop logic as AGM. It is the most accurate method here at for 18.13 ns, about the same as the AGM.
A better variable
Letβs go back to the converging series. The series in converges slowly not because the integral is hard or something but because is a poor variable. Expanding in instead gives a series with positive coefficients:
t = (a - b) / (a + b)
h = t*t
c = 1
hp = 1
s = 1
for n in 1..=max_terms:
c *= (n - 1.5)*(n - 1.5) / (n*n)
hp *= h
term = c * hp
s += term
if term < 1e-18 * s: break
p = pi * (a + b) * sis bounded by 1 and is usually far smaller than : at it is only . Table 2 compares the two. At the series needs 15 terms where the series in needs 106, and at it is 66 against 2393. It is the same integral and the same loop with one substitution, and at the flat end of the sweep the two differ by 10 orders of magnitude on the same term budget.
Though it is exact, it is still not cheap; 3-4x the AGM for the same digits.
Remez
Another method we can use is the Chebyshev minimax, in particular the Remez exchange algorithm; this basically tries to fit a polynomial to minimise our maximum error. For what itβs worth, I think it is a very beautiful way to approximate a whole host of functions. If youβre interested, check out the Sollya project for more.
We fit the ratio that Ramanujanβs formulas use, not itself, and the criterion is relative error in that ratio: the algorithm moves to the same size at every extremum, alternating in sign. Since is exact for a given shape, that is relative error in too, which is the number the tables report, so nothing needs weighting by . in :
num = p0 + h*(p1 + h*(p2 + h*(p3 + h*(p4 + h*p5))))
den = 1 + h*(q1 + h*(q2 + h*(q3 + h*(q4 + h*q5))))
p = pi * (a + b) * num / denThe fit is one-off, over 801 evenly spaced points in whichever variable is being fitted, , or , across the whole sweep ( from 0.0025 to 1).
With 10 multiply-adds and a division, from to we have , , . We can improve this by making the fit be the square root. In fact, Ramanujanβs second formula is a rational function of rather than of , which is why it holds up at the flat end. In
the singularity (, i.e. the degenerate ellipse ) moves from 1% outside the interval to 10% outside it. We see way more bang for our buck this way.
| fitted in | |||
|---|---|---|---|
Youβll notice I also tested cube root for this, so that , which reduces our error again. reaches against . Unfortunately, itβs unsurprisingly slower than sqrt: 15.88 ns against 7.83 ns. The AGM at 16.95 ns is cheaper and 3 orders more accurate.
The arithmetic geometric mean
I mentioned this one a couple times in passing in the previous methods. I suspect that this is the most widely used approximation. Itβs pretty simple, and very accurate, but not too fast. It essentially replaces a pair of numbers by their arithmetic and geometric means and repeats; it rapidly converges too.
where and . Then , as before.
r = b / a
k = sqrt(1 - r*r)
x = 1
y = r
c = k
s = k*k / 2
w = 0.5
for turn in 1..=8:
xn = (x + y) / 2
yn = sqrt(x*y)
c = (x - y) / 2
x = xn
y = yn
w *= 2
s += w * c*c
if w * c*c < 1e-18 * s: break
p = 4 * a * (pi / (2*x)) * (1 - s)Incredibly, each turn roughly doubles the number of correct digits. 5 turns suffice at 2:1 and 6 for anything flatter in the sweep, and the worst error anywhere is , which we can attribute to f64 rounding.
16.95 ns, 4x Ramanujanβs second but still less than either series or Simpson.
Which one to use
If speed is the priority, Ramanujanβs first is the best of the cheapest ones; use the second if you realise you want a bit more accuracy. If accuracy is the priority, AGM and Carlson (which is what Boost C++ uses) will both serve you well. Otherwise, for a best of both worlds, I would suggest Remez (sqrt).
Sources
- B. Conrad, Impossibility theorems for elementary integration, 2005. Example 4.3 gives the general idea of the argument used above.
- S. Ramanujan, Modular equations and approximations to , Quarterly Journal of Mathematics 45 (1914), 350-372.
- J. M. Borwein and P. B. Borwein, Pi and the AGM, Wiley, 1987. The arithmetic geometric mean and the elliptic integrals.
- E. Ya. Remez, Sur la determination des polynomes dβapproximation de degre donnee, Communications de la Societe Mathematique de Kharkov 10 (1934), 41-63.
- B. C. Carlson, Numerical computation of real or complex elliptic integrals, Numerical Algorithms 10 (1995), 13-26. The symmetric integrals and the duplication behind them.
- NIST Digital Library of Mathematical Functions, chapter 19, dlmf.nist.gov/19. The relations between the symmetric and Legendre forms.
- Boost.Math, special_functions, for the branch and the stored polynomials described above.