Floating-Point Numbers

What is a floating-point number?

A floating-point number is a number written in binary scientific notation with a fixed number of digits: a sign, a significand (the digits) and an exponent that says where the point goes. In decimal, scientific notation writes -6.02×10^23 or 1.6×10^-19: a few significant digits and a power of ten. A computer does the same in base 2: x=(-1)^s×(1.f_1f_2f_3… f_m)_2×2^e. The point "floats" because the exponent e moves it, so the same format holds the size of the universe and the size of an atom. The price is that the number of digits m+1 is fixed, so every number has only a limited number of significant digits, however large or small it is.

Why not fixed point? A fixed-point format keeps the point in one place, for example always 16 binary digits after it. Then it cannot hold both 10^30 and 10^-30; floating point gives up some precision to cover that whole range.

The standard. Almost all hardware follows IEEE 754. It defines a few binary formats, which differ in how many bits go to the exponent and to the fraction:

The number of decimal digits is the number of significant bits times \log_102≈0.301. In Python a float is a double, in JavaScript every Number is a double, and in C and C++ double is a double and float is a single. This article is about what these formats do, in the way that matters when you write code; real numbers explains the short version, and binary numbers explains why 0.1 has no finite binary expansion.

How are the bits of a double laid out?

A double has three fields: 1 sign bit, 11 exponent bits and 52 fraction bits, and the value follows from the field of the exponent.

• Normal numbers (exponent field 1 to 2046): the value is (-1)^s×\left(1+(F)/(2^52)\right)×2^E-1023, where E is the exponent field read as an integer, F is the fraction field, and 1023 is the bias. The leading 1 is implied, which is why 52 stored bits give 53 bits of precision. The bias lets the exponent be stored without its own sign.
• Exponent field 0: the value ±0 if the fraction is 0, and otherwise a subnormal number (-1)^s×(F)/(2^52)×2^-1022 with no implied 1.
• Exponent field 2047 (all ones): ±∞ if the fraction is 0, and otherwise NaN.

Two examples. The number -2.5=-1.25×2^1 has sign 1, exponent field 1+1023=1024=10000000000_2 and fraction 0100…0 (the .25 of 1.25), so its pattern in hexadecimal is C004000000000000. The number 0.1 is 1.6000000000000000888…×2^-4: sign 0, exponent field 1019, and a fraction that repeats 1001 1001… and is rounded up at the end, giving 3FB999999999999A. That last A is the rounding, and it is why the stored value is 0.1000000000000000055511151231257827021181583404541015625, slightly more than 0.1. For a single, the numbers are the same except that the bias is 127 and there are 8 and 23 bits.

Type a number below and choose a format. The widget shows the three fields, the formula they stand for, the exact value that is stored, the error of the rounding and the neighbors above and below.

Why are floating-point numbers not evenly spaced?

Between two consecutive powers of two, 2^k and 2^k+1, a format always has the same number of values (2^52 for a double), so the gap between neighbors is 2^k-52 and doubles at each power of two. Near 1 the gap is 2^-52≈2.2×10^-16; near 1000 it is 1.1×10^-13; near 10^16 it is 2. The relative spacing, the gap divided by the number, stays between 2^-53 and 2^-52 everywhere, and that is what "about 16 digits" means. The gap between a number and the next one is called an ulp, the unit in the last place.

A format with very few bits shows it plainly. Take 1 sign bit, 3 exponent bits (bias 3) and 2 fraction bits. Its non-negative numbers are 0, \tfrac116, \tfrac18, \tfrac316, \tfrac14, \tfrac516, \tfrac38, \tfrac716, \tfrac12, \tfrac58, \tfrac34, \tfrac78, 1, \tfrac54, \tfrac32, \tfrac74, 2, …, 14. Between \frac14 and \frac12 there are four numbers, a quarter of the gap apart, as there are between 1 and 2, but there the gap is four times larger. The numbers below \frac14, the subnormal numbers, share one gap, \frac116, which is the gap of the first binade: that is what lets the format reach 0 gradually instead of jumping from \frac14 to 0.

Two consequences matter in code. First, integers are exact only up to 2^53 in a double (9 007 199 254 740 992) and up to 2^24=16 777 216 in a single: above that the gap is 2, so 2^53+1 rounds back to 2^53. Second, a large number absorbs a small one: 1e16 + 1 == 1e16 is true.

Choose the number of bits below and slide through the numbers. The table gives the gap of each power of two.

How does a computer round, and what is machine epsilon?

IEEE 754 requires that + − × ÷ and the square root give the correctly rounded result: the exact answer rounded to the nearest number of the format, and in a tie to the one whose last fraction bit is even. Rounding to the nearest even avoids a bias: with ties always rounded up, long calculations would drift upward. The other modes (toward 0, toward +∞, toward -∞) exist for interval arithmetic and special cases.

The standard model. Provided nothing overflows or underflows, \mathrmfl(a\circ b)=(a\circ b)(1+\delta), |\delta|\le u, where \circ is any of the operations and u=2^-53≈1.11×10^-16 for a double is the unit roundoff. Each operation is as accurate as the format allows, and the whole story of numerical error is how these small relative errors combine.

Machine epsilon. The gap between 1 and the next double is 2^-52≈2.22×10^-16, called the machine epsilon (in Python sys.float_info.epsilon, in JavaScript Number.EPSILON); it is twice the unit roundoff. In a single it is 2^-23≈1.19×10^-7 and in a half 2^-10≈9.8×10^-4.

What is exact. The result of an operation is exact when it fits: integers up to 2^53, fractions with a power of two in the denominator such as 0.5 and 0.375, and anything times a power of two (unless it overflows or underflows). That is why 0.5+0.25=0.75 is true in floating point while 0.1+0.2=0.3 is not: the sum 0.1+0.2 is 0.30000000000000004, one ulp above the double that is nearest to 0.3.

What are infinity, NaN, negative zero and subnormal numbers?

IEEE 754 reserves some patterns for values that are not ordinary numbers, so that every operation has an answer and a long calculation does not stop at the first problem.

• Infinity. An exponent field of all ones with fraction 0 is ±∞. It is the result of an overflow, such as Number.MAX_VALUE * 10, and of dividing a non-zero number by zero in JavaScript or C. Python raises ZeroDivisionError for 1 / 0.0, but float('inf') exists, and NumPy returns inf with a warning.
• NaN, "not a number". An exponent field of all ones with a non-zero fraction. It is the result of 0/0, ∞-∞ and √(-1) (where these do not raise errors). NaN is unordered: it is not equal to anything, not even itself, so x != x is the test for NaN (or isnan, Number.isNaN, Object.is), and every comparison with NaN is false. It spreads: any operation with a NaN gives NaN.
• Negative zero. Sign 1 with everything else 0. It is equal to 0 in comparisons, but \frac1+0=+∞ and \frac1-0=-∞, which keeps the sign of a limit that approaches 0 from below.
• Subnormal numbers. Without them the smallest positive double would be the smallest normal number 2^-1022≈2.2×10^-308, with a gap to 0 much larger than the gaps near it. The subnormal numbers (exponent field 0) fill that gap evenly down to 2^-1074≈4.9×10^-324, so underflow is gradual: x-y=0 happens only when x=y.

Overflow and underflow. A result above the largest double, 1.7976931348623157×10^308, rounds to ∞; a result below half of 2^-1074 rounds to 0. In practice these appear in products and in exponentials, and the cure is to work with logarithms, or to scale: the length of a vector √(x^2+y^2) overflows for x=10^200 although the answer does not, and hypot(x, y) avoids it.

Why do rounding errors add up, and how do you limit them?

Each addition rounds, and the rounding errors of a long sum do not cancel: they add up, and the error of a naive sum grows roughly in proportion to the number of terms. Add 0.1 ten times in a double, one at a time, and the result is 0.9999999999999999; in a single it is 1.0000001192092896. Add it a million times and a double gives 100000.00000133288, off by 1.3×10^-6, about 90 000 ulps. The bound for n terms is about n· u times the sum of the absolute values.

A large partial sum also absorbs small terms: once the total is 2^53 the gap is 2, so adding 1 changes nothing, however many times you do it.

Three remedies. 1. Kahan summation keeps a second number c that holds the part that was rounded away in the last addition, and puts it back in the next one: y=x_i-c, t=s+y, c=(t-s)-y, s=t. The error then no longer grows with n: it stays within an ulp or two. A compiler that rearranges floating-point arithmetic freely can remove the compensation, so the option for strict arithmetic matters. 2. Pairwise summation adds in a tree, so that the terms have a similar size and the error grows like \log n instead of n. NumPy does this. 3. Exact summation. Python's math.fsum returns the correctly rounded sum of the exact values by keeping partial sums that do not lose anything. It is the reference to use in tests.

Since Python 3.12 the built-in sum of floats also uses a compensated algorithm, so sum([0.1] * 10) gives 1.0 there, whereas a plain loop still gives 0.9999999999999999.

Try it below: pick a value, a count and a precision.

What is catastrophic cancellation?

When two nearly equal numbers are subtracted, the leading digits cancel and what is left is mostly the old rounding errors: the result is exact, but the information in it is gone. Take two computed numbers a≈1.0000000000000002 and b≈1. Each carries a relative error of up to 10^-16, and their difference 2.2×10^-16 is computed without any new error, but it is of the same size as the errors that a and b already had, so the difference may have no correct digit. The relative error of a subtraction is magnified by about (|a|+|b|)/(|a-b|).

Examples. - (1-\cos x)/(x^2) for x=10^-8: \cos x=1-5×10^-17 rounds to 1, so the naive formula gives 0, and the true value is \frac12. Using 1-\cos x=2\sin^2\frac x2 gives 0.5. - (e^x-1)/(x) for x=10^-15: e^x rounds to 1.000000000000001, and the naive formula gives 1.1102230246251565 instead of 1. The library function expm1(x) computes e^x-1 directly, and gives 1.0000000000000004 (the last digit is rounding). - √(x^2+1)-x for large x: for x=10^8 the naive formula gives 0; the stable form \dfrac1√(x^2+1)+x gives 5×10^-9. - The small root of x^2-bx+1=0 for b=10^8: (b-√(b^2-4))/(2) gives 7.45×10^-9, while the stable form (2)/(b+√(b^2-4)) gives the correct 10^-8.

The cure is almost never more precision: it is to avoid the subtraction by algebra (a conjugate, an identity), by a series for small x, or by a library function made for it (expm1, log1p, hypot). Cancellation also explains why the difference quotient with a tiny h in numerical differentiation gets worse, not better, below h≈10^-8.

Move x in each of the four cases below and read the number of correct digits.

How should you compare two floating-point numbers?

Compare computed floats with a tolerance that has a relative part and an absolute part: |a-b|\le\max\left(rel·\max(|a|,|b|), abs\right). This is what Python's math.isclose(a, b, rel_tol=1e-09, abs_tol=0.0) does, and what you write by hand in JavaScript and C++.

• A purely absolute tolerance such as 10^-9 is wrong for large numbers: near 10^16 two neighbors are 2 apart, so no two different numbers are within 10^-9, and near 10^-12 everything is "equal".
• A purely relative tolerance is wrong near zero: 10^-300 and 0 are not close in relative terms, although both are "zero" for the problem. So give an absolute part, as small as the scale of the data justifies, when the true value can be zero.
• The relative part should be a few times the unit roundoff times the number of operations: 10^-9 is a common choice for doubles, and about 10^-5 for singles.
• ulps. In tests of numerical code the distance in ulps, the number of floats between the two numbers, is a measure that scales with the size: 0.1 + 0.2 and 0.3 are 1 ulp apart.
• Never loop on equality. while x != 1.0: x += 0.1 never ends, because x steps over 1.0 and goes on forever. Count with an integer, or loop while x < 1.0 - 1e-9.
• NaN is never close to anything, itself included; test for it separately.

Money. Prices are not real numbers: they are integers of the smallest unit. Store cents, or use a decimal type (Python decimal.Decimal, Java BigDecimal, C# decimal), where 0.1+0.2=0.3 holds exactly because the base is 10.

Which type should you use: float16, float32, float64 or a decimal?

Use a double unless you have a reason not to; use less precision when memory and speed matter more than digits, and an exact type when the answer must be exact.

• float64 (double): the default for science and engineering and for most programs. About 16 digits and a range up to 10^308.
• float32 (single): graphics, games and GPUs, where 7 digits are enough and halving the memory doubles the throughput. Be careful with sums of many terms and with integers above 2^24 (it counts from 16 777 216 upward in steps of 2).
• float16 and bfloat16: machine learning. Half precision has 11 bits of precision and a maximum of 65504; bfloat16 keeps the range of a single (8 exponent bits) with only 8 bits of precision, which suits the large dynamic range of neural-network training. Both are used with care, often accumulating in float32.
• Decimal, integers, fractions: money, counts and exact ratios. Python's fractions.Fraction and decimal.Decimal are exact for the numbers they can hold but slower.
• Arbitrary precision (mpmath, BigDecimal, GMP): when a calculation needs more digits than a double has, at a cost in speed.

Two famous failures. In February 1991 a Patriot missile battery at Dhahran failed to intercept a Scud. Its clock counted tenths of a second in a 24-bit register, and 0.1 is not exact in binary: the error was about 0.000000095 s per tick, which after about 100 hours of operation was about 0.34 s, enough for a Scud, which travels more than 1.6 km each second, to be half a kilometer away from where the system looked for it. In June 1996 the first Ariane 5 rocket was lost about 37 seconds after launch when a 64-bit floating-point value was converted to a 16-bit signed integer that was too small for it: an overflow, not a rounding error, but in the same family, a conversion that was never checked.

How do you inspect floating-point numbers in code?

Every language lets you look at the bits and the spacing: Python with struct, math.ulp and decimal.Decimal, JavaScript with a DataView, and C++ with a loop that finds the machine epsilon. The programs below show the bit patterns of 0.1 and -2.5, the exact value of 0.1, the epsilon and the ulp, the neighbor of 0.1 + 0.2, the three kinds of sum, absorption, tolerance comparison, cancellation and NaN.

import math
import struct
import sys
from decimal import Decimal


def bits(x):
    """The 64 bits of a double as sign, exponent and fraction."""
    b = format(struct.unpack('>Q', struct.pack('>d', x))[0], '064b')
    return b[0], b[1:12], b[12:]


def naive(xs):
    total = 0.0
    for x in xs:
        total += x
    return total


def kahan(xs):
    total = comp = 0.0
    for x in xs:
        y = x - comp
        t = total + y
        comp = (t - total) - y
        total = t
    return total


print(bits(0.1))
print(bits(-2.5))
print(Decimal(0.1))
print(sys.float_info.epsilon, math.ulp(1.0), math.ulp(2.0 ** 53))
print(math.nextafter(0.1 + 0.2, 0.0) == 0.3)
print(naive([0.1] * 10), kahan([0.1] * 10), math.fsum([0.1] * 10))
print(naive([0.1] * 1000000), kahan([0.1] * 1000000), math.fsum([0.1] * 1000000))
print(1e16 + 1 == 1e16, 2.0 ** 53 + 1 == 2.0 ** 53)
print(math.isclose(0.1 + 0.2, 0.3), 0.1 + 0.2 == 0.3)
x = 1e-8
print((1 - math.cos(x)) / x ** 2, 2 * math.sin(x / 2) ** 2 / x ** 2)
print((math.exp(1e-15) - 1) / 1e-15, math.expm1(1e-15) / 1e-15)
print(float('nan') == float('nan'), math.isnan(float('nan')), float('inf') - float('inf'))
print(Decimal('0.1') + Decimal('0.2') == Decimal('0.3'))
# ('0', '01111111011', '1001100110011001100110011001100110011001100110011010')
# ('1', '10000000000', '0100000000000000000000000000000000000000000000000000')
# 0.1000000000000000055511151231257827021181583404541015625
# 2.220446049250313e-16 2.220446049250313e-16 2.0
# True
# 0.9999999999999999 1.0 1.0
# 100000.00000133288 100000.0 100000.0
# True True
# True False
# 0.0 0.5
# 1.1102230246251565 1.0000000000000004
# False True nan
# True

const bits = (x) => {
  const v = new DataView(new ArrayBuffer(8))
  v.setFloat64(0, x)
  return v.getBigUint64(0).toString(2).padStart(64, '0')
}
bits(1)                                  // '0011111111110000000000000000000000000000000000000000000000000000'
bits(0.1)                                // '0011111110111001100110011001100110011001100110011001100110011010'
Number.EPSILON                           // 2.220446049250313e-16
;[Math.fround(0.1), 0.1]                 // [0.10000000149011612, 0.1]: the nearest single to 0.1
;[2 ** 53 + 1 === 2 ** 53, 2 ** 53 + 2]  // [true, 9007199254740994]
;[0 / 0, 1 / 0, 1 / -0]                  // [NaN, Infinity, -Infinity]
;[NaN === NaN, Object.is(NaN, NaN)]      // [false, true]
;[Object.is(0, -0), 0 === -0]            // [false, true]
;[Number.MAX_VALUE * 10, 5e-324 / 2]     // [Infinity, 0]: overflow, and underflow past the smallest subnormal
;[0.1 + 0.2, 0.1 + 0.2 === 0.3]          // [0.30000000000000004, false]
Math.abs(0.1 + 0.2 - 0.3) < Number.EPSILON // true: a tolerance of one machine epsilon is fine near 1
;[Math.fround(16777217), 16777217]       // [16777216, 16777217]: a single cannot hold 2^24 + 1
let s32 = 0
for (let i = 0; i < 10; i++) s32 = Math.fround(s32 + Math.fround(0.1))
s32                                      // 1.0000001192092896: ten tenths in single precision

#include <iostream>
#include <cstdio>
#include <cmath>
using namespace std;

int main() {
  double eps = 1;
  int count = 0;
  while (1 + eps / 2 > 1) {                              // halve until 1 + eps/2 is rounded back to 1
    eps = eps / 2;
    count = count + 1;
  }
  printf("%d %.17f\n", count, eps);                      // 52 halvings: eps = 2^-52
  double s = 0;
  for (int i = 0; i < 10; i++) s = s + 0.1;
  printf("%.17f %d\n", s, s == 1.0);                     // ten times 0.1 is not 1
  double big = 9007199254740992.0;                      // 2^53
  printf("%d %d\n", big + 1 == big, big + 2 == big);     // 1 0: the 1 is absorbed, the 2 is a whole gap
  return 0;
}

/* output:
52 0.00000000000000022
0.99999999999999989 0
1 0
*/

Where does floating point come from?

Floating point is older than the electronic computer, and for decades each machine had its own format until a standard made programs portable.

• 1914. Leonardo Torres Quevedo proposed floating-point arithmetic for a mechanical calculator in his Essays on Automatics.
• 1930s–1941. Konrad Zuse's machines, the Z1 and the Z3, used binary floating point, with a sign, an exponent and a fraction.
• 1954. The IBM 704 had floating-point arithmetic in hardware, and through the 1960s and 1970s every maker chose its own base, size and rounding, so that the same program gave different answers on different computers.
• 1980s. William Kahan, who had worked on Intel's 8087 coprocessor, led the committee that wrote IEEE 754, published in 1985: formats, correct rounding, infinities, NaN and gradual underflow. Kahan received the Turing Award in 1989.
• 1994. The Pentium FDIV bug, a few wrong entries in a table used by the divider, gave results that were wrong in the fourth or fifth significant digit or beyond, and showed that floating-point hardware had to be tested as carefully as software; Intel recalled the chips.
• 2008 and 2019. The standard was revised: fused multiply-add (computing a× b+c with one rounding), decimal formats, half precision as an exchange format, and clearer rules for NaN and for operations such as the minimum and the maximum.
• 2010s. bfloat16, with the exponent range of a single and only 8 bits of precision, was designed for machine-learning hardware, and half precision became common on GPUs.

What mistakes do people make with floating-point numbers?

The most common mistakes with floating-point numbers are the nine below, each with the correct statement.

Practice: test your understanding

These questions mix everything above. A wrong answer costs nothing here: read the hint and try again.

Summary: floating-point numbers at a glance

• Format: (-1)^s×(1.f)_2×2^e; a double is 1+11+52 bits, bias 1023, 53 bits of precision, about 16 digits.
• Spacing: the gap doubles at every power of two; integers are exact up to 2^53 (a single: 2^24).
• Rounding: correctly rounded, ties to even, relative error at most u=2^-53; machine epsilon 2^-52.
• Special values: ±0, ±∞, NaN (not equal to itself), subnormal numbers for gradual underflow.
• Errors: they accumulate in sums, are absorbed by large numbers, and cancellation destroys digits in subtractions of nearly equal numbers.
• Remedies: Kahan or pairwise or exact sums, algebraic rewriting, expm1, log1p, hypot.
• Compare: with a relative and an absolute tolerance, or in ulps; never loop on ==.
• Choose: float64 by default, float32 or lower for memory and speed, integers or decimals for money.
• Code: struct, math.ulp, math.fsum, math.isclose, decimal in Python; DataView and Number.EPSILON in JavaScript.

Glossary: key terms

Floating-point number

A number stored as a sign, an exponent and a fixed number of significant bits, so that the point floats with the exponent.

IEEE 754

The standard that defines the binary floating-point formats, correct rounding, infinities, NaN and gradual underflow used by almost all hardware.

Machine epsilon

The gap between 1 and the next number of a format: 2 to the power minus 52 for a double.

Ulp

The unit in the last place: the gap between a floating-point number and the next one.

Subnormal number

A number below the smallest normal one, stored with exponent field zero and no implied leading 1, which makes underflow gradual.

NaN

Not a number: the result of operations such as zero over zero, not equal even to itself.

Absorption

The loss of a small number when it is added to a much larger one, because the sum rounds back to the larger number.

Catastrophic cancellation

The loss of most correct digits when two nearly equal numbers are subtracted, exposing the rounding errors they already had.

Frequently asked questions

References

1. IEEE (2019). IEEE Standard for Floating-Point Arithmetic (IEEE 754-2019). Institute of Electrical and Electronics Engineers.
2. David Goldberg (1991). What Every Computer Scientist Should Know About Floating-Point Arithmetic. ACM Computing Surveys 23(1).
3. Nicholas J. Higham (2002). Accuracy and Stability of Numerical Algorithms (2nd ed.). SIAM.
4. William Kahan (1965). Further remarks on reducing truncation errors. Communications of the ACM 8(1).

Related articles

• Binary Numbers: How Computers Count, Convert and Calculate: Binary numbers explained: bits and place value, converting to decimal and hex, binary arithmetic, two’s complement, bitwise operators and binary fractions.
• Real Numbers: Definition, Types, Examples and Properties: Real numbers explained: definition, types (rational and irrational), examples, properties and the number line, with interactive practice, a proof and FAQ.
• Newton’s Method: Newton’s method for solving f(x) = 0: the formula from tangent lines, examples, quadratic convergence, when it fails, square roots by hand, and code.
• Linearization and Differentials: Linearization and differentials: the tangent-line approximation L(x) = f(a) + f′(a)(x − a), df = f′(x) dx, estimates, errors, percentage change and code.