Floating point precision

[Mathias_Westerdahl]

Every now and then, you’ll see issues that can be described as “choppy” animations, or perhaps your code doesn’t work when comparing various variables.

Sometimes, it could be related to something called “floating point precision”.
It stems from the fact that on a computer, real numbers have to be approximated in some way.
And as you might imagine, the approximation result might not always be what you intended. (See table at the end)

TL;DR:

Floating point values…

• … does not have infinite range
• … does not have infinite precision
• … exist in most modern engines.
• … have better precision when closer to 0 than FLT_MAX (3.40282e+38)
• … should (most often) not be compared using the == or != operators
• … can be tricky to use, even with operators < and > due to imprecision

In (only slightly) more detail:

You use floating point representations in code (Lua, C++, shaders etc) and data (textures, heightmaps etc). That’s why it’s very good to know about the general workings of them, in order to anticipate any issues that might arise.

What we’re using in our C++ engine are 32 bit floating point numbers. Lua itself uses 64bit floating point numbers.

In OpenGL ES, you can set lowp / mediump / highp precision on your variables.
The OpenGL ES standard sets a minumum requirement of the precision so it might still differ between graphics drivers.

No infinite precision

If you think about it, a 32 bit integer can represent the values -2147483647 to 2147483647, that’s it. Then all 32 bits are used. But, if you would also like decimals, how could you fit all those numbers, as well as decimals, in only 32 bits?

The answer is, of course, you cannot. You have to make sacrifices.

In the single floating point representation (usually refers to the standard IEEE 754) case, this means that we sacrifice precision the larger the number is.

Since the precision varies depending how large your number is, the operators might not work as you’d expect.

Accumulation

Another thing to consider, is how the precision “error” of calculations accumulate.
It’s very easy to calculate something that has less precision than you’d like, and use that value in your next calculation, in yet another manner that might introduce even more precision errors.

Small Example

How to deal with it?

A common thing is to keep your data and computations closer to 0 (or Origin) than in world space.

• E.g. Moving camera and objects towards Origin (0,0,0). Used for open world games, infinite runners etc.

Another way is to increase the number of bits to represent the value

• E.g 64 bit precision on CPU, or highp in shaders

Each way will come with a drawback, you have to decide which one is the best for you.

Addendum

To give you an idea of how much the precision changes, the larger your numbers get:

Table contains: integer, the next 32 bit floating point value after the integer, the difference between the two previous values

f: 0.00  next: 1.4012984643248170709e-45  diff: 1.4012984643248170709237e-45
f: 1.00  next: 1.0000001192092895508  diff: 1.1920928955078125e-07
f: 2.00  next: 2.0000002384185791016  diff: 2.384185791015625e-07
f: 3.00  next: 3.0000002384185791016  diff: 2.384185791015625e-07
f: 4.00  next: 4.0000004768371582031  diff: 4.76837158203125e-07
f: 5.00  next: 5.0000004768371582031  diff: 4.76837158203125e-07
f: 6.00  next: 6.0000004768371582031  diff: 4.76837158203125e-07
f: 7.00  next: 7.0000004768371582031  diff: 4.76837158203125e-07
f: 8.00  next: 8.0000009536743164062  diff: 9.5367431640625e-07
f: 9.00  next: 9.0000009536743164062  diff: 9.5367431640625e-07
f: 10.00  next: 10.000000953674316406  diff: 9.5367431640625e-07
f: 20.00  next: 20.000001907348632812  diff: 1.9073486328125e-06
f: 30.00  next: 30.000001907348632812  diff: 1.9073486328125e-06
f: 40.00  next: 40.000003814697265625  diff: 3.814697265625e-06
f: 50.00  next: 50.000003814697265625  diff: 3.814697265625e-06
f: 60.00  next: 60.000003814697265625  diff: 3.814697265625e-06
f: 70.00  next: 70.00000762939453125  diff: 7.62939453125e-06
f: 80.00  next: 80.00000762939453125  diff: 7.62939453125e-06
f: 90.00  next: 90.00000762939453125  diff: 7.62939453125e-06
f: 100.00  next: 100.00000762939453125  diff: 7.62939453125e-06
f: 200.00  next: 200.0000152587890625  diff: 1.52587890625e-05
f: 300.00  next: 300.000030517578125  diff: 3.0517578125e-05
f: 400.00  next: 400.000030517578125  diff: 3.0517578125e-05
f: 500.00  next: 500.000030517578125  diff: 3.0517578125e-05
f: 600.00  next: 600.00006103515625  diff: 6.103515625e-05
f: 700.00  next: 700.00006103515625  diff: 6.103515625e-05
f: 800.00  next: 800.00006103515625  diff: 6.103515625e-05
f: 900.00  next: 900.00006103515625  diff: 6.103515625e-05
f: 1000.00  next: 1000.00006103515625  diff: 6.103515625e-05
f: 2000.00  next: 2000.0001220703125  diff: 0.0001220703125
f: 3000.00  next: 3000.000244140625  diff: 0.000244140625
f: 4000.00  next: 4000.000244140625  diff: 0.000244140625
f: 5000.00  next: 5000.00048828125  diff: 0.00048828125
f: 6000.00  next: 6000.00048828125  diff: 0.00048828125
f: 7000.00  next: 7000.00048828125  diff: 0.00048828125
f: 8000.00  next: 8000.00048828125  diff: 0.00048828125
f: 9000.00  next: 9000.0009765625  diff: 0.0009765625
f: 10000  next: 10000.0009765625  diff: 0.000976562
f: 20000  next: 20000.001953125  diff: 0.00195312
f: 30000  next: 30000.001953125  diff: 0.00195312
f: 40000  next: 40000.00390625  diff: 0.00390625
f: 50000  next: 50000.00390625  diff: 0.00390625
f: 60000  next: 60000.00390625  diff: 0.00390625
f: 70000  next: 70000.0078125  diff: 0.0078125
f: 80000  next: 80000.0078125  diff: 0.0078125
f: 90000  next: 90000.0078125  diff: 0.0078125
f: 100000  next: 100000.0078125  diff: 0.0078125
f: 200000  next: 200000.015625  diff: 0.015625
f: 300000  next: 300000.03125  diff: 0.03125
f: 400000  next: 400000.03125  diff: 0.03125
f: 500000  next: 500000.03125  diff: 0.03125
f: 600000  next: 600000.0625  diff: 0.0625
f: 700000  next: 700000.0625  diff: 0.0625
f: 800000  next: 800000.0625  diff: 0.0625
f: 900000  next: 900000.0625  diff: 0.0625
f: 1e+06  next: 1000000.0625  diff: 0.0625
f: 2e+06  next: 2000000.125  diff: 0.125
f: 3e+06  next: 3000000.25  diff: 0.25
f: 4e+06  next: 4000000.25  diff: 0.25
f: 5e+06  next: 5000000.5  diff: 0.5
f: 6e+06  next: 6000000.5  diff: 0.5
f: 7e+06  next: 7000000.5  diff: 0.5
f: 8e+06  next: 8000000.5  diff: 0.5
f: 9e+06  next: 9000001  diff: 1
f: 1e+07  next: 10000001  diff: 1
f: 2e+07  next: 20000002  diff: 2
f: 3e+07  next: 30000002  diff: 2
f: 4e+07  next: 40000004  diff: 4
f: 5e+07  next: 50000004  diff: 4
f: 6e+07  next: 60000004  diff: 4
f: 7e+07  next: 70000008  diff: 8
f: 8e+07  next: 80000008  diff: 8
f: 9e+07  next: 90000008  diff: 8
f: 1e+08  next: 100000008  diff: 8
f: 2e+08  next: 200000016  diff: 16
f: 3e+08  next: 300000032  diff: 32
f: 4e+08  next: 400000032  diff: 32
f: 5e+08  next: 500000032  diff: 32
f: 6e+08  next: 600000064  diff: 64
f: 7e+08  next: 700000064  diff: 64
f: 8e+08  next: 800000064  diff: 64
f: 9e+08  next: 900000064  diff: 64
f: 1e+09  next: 1000000064  diff: 64
f: 2e+09  next: 2000000128  diff: 128
f: 3e+09  next: 3000000256  diff: 256
f: 4e+09  next: 4000000256  diff: 256
f: 5e+09  next: 5000000512  diff: 512
f: 6e+09  next: 6000000512  diff: 512
f: 7e+09  next: 7000000512  diff: 512
f: 8e+09  next: 8000000512  diff: 512
f: 9e+09  next: 9000000512  diff: 1024
f: 1e+10  next: 10000001024  diff: 1024
f: 2e+10  next: 20000002048  diff: 2048
f: 3e+10  next: 30000003072  diff: 2048
f: 4e+10  next: 40000004096  diff: 4096
f: 5e+10  next: 50000003072  diff: 4096
f: 6e+10  next: 60000006144  diff: 4096
f: 7e+10  next: 70000009216  diff: 8192
f: 8e+10  next: 80000008192  diff: 8192
f: 9e+10  next: 90000007168  diff: 8192
f: 1e+11  next: 100000006144  diff: 8192
f: 2e+11  next: 200000012288  diff: 16384
f: 3e+11  next: 300000018432  diff: 32768
f: 4e+11  next: 400000024576  diff: 32768
f: 5e+11  next: 500000030720  diff: 32768
f: 6e+11  next: 600000036864  diff: 65536
f: 7e+11  next: 700000043008  diff: 65536
f: 8e+11  next: 800000049152  diff: 65536
f: 9e+11  next: 900000055296  diff: 65536
f: 1e+12  next: 1000000061440  diff: 65536
f: 2e+12  next: 2000000122880  diff: 131072
f: 3e+12  next: 3000000315392  diff: 262144
f: 4e+12  next: 4000000245760  diff: 262144
f: 5e+12  next: 5000000438272  diff: 524288
f: 6e+12  next: 6000000630784  diff: 524288
f: 7e+12  next: 7000000299008  diff: 524288
f: 8e+12  next: 8000000491520  diff: 524288
f: 9e+12  next: 9000001208320  diff: 1.04858e+06
f: 1e+13  next: 10000000876544  diff: 1.04858e+06
f: 2e+13  next: 20000001753088  diff: 2.09715e+06
f: 3e+13  next: 30000002629632  diff: 2.09715e+06
f: 4e+13  next: 40000003506176  diff: 4.1943e+06
f: 5e+13  next: 50000004382720  diff: 4.1943e+06
f: 6e+13  next: 60000005259264  diff: 4.1943e+06
f: 7e+13  next: 70000006135808  diff: 4.1943e+06
f: 8e+13  next: 80000007012352  diff: 8.38861e+06
f: 9e+13  next: 90000007888896  diff: 8.38861e+06
f: 1e+14  next: 100000008765440  diff: 8.38861e+06
f: 2e+14  next: 200000017530880  diff: 1.67772e+07
f: 3e+14  next: 300000043073536  diff: 3.35544e+07
f: 4e+14  next: 400000035061760  diff: 3.35544e+07
f: 5e+14  next: 500000027049984  diff: 3.35544e+07
f: 6e+14  next: 600000086147072  diff: 6.71089e+07
f: 7e+14  next: 700000078135296  diff: 6.71089e+07
f: 8e+14  next: 800000070123520  diff: 6.71089e+07
f: 9e+14  next: 900000062111744  diff: 6.71089e+07
f: 1e+15  next: 1000000054099968  diff: 6.71089e+07
f: 2e+15  next: 2000000108199936  diff: 1.34218e+08
f: 3e+15  next: 3000000296517632  diff: 2.68435e+08
f: 4e+15  next: 4000000216399872  diff: 2.68435e+08
f: 5e+15  next: 5000000673153024  diff: 5.36871e+08
f: 6e+15  next: 6000000593035264  diff: 5.36871e+08
f: 7e+15  next: 7000000512917504  diff: 5.36871e+08
f: 8e+15  next: 8000000432799744  diff: 5.36871e+08
f: 9e+15  next: 9000000352681984  diff: 5.36871e+08
f: 1e+16  next: 10000001346306048  diff: 1.07374e+09
f: 2e+16  next: 20000002692612096  diff: 2.14748e+09
f: 3e+16  next: 30000002965176320  diff: 2.14748e+09
f: 4e+16  next: 40000005385224192  diff: 4.29497e+09
f: 5e+16  next: 50000003510304768  diff: 4.29497e+09
f: 6e+16  next: 60000005930352640  diff: 4.29497e+09
f: 7e+16  next: 70000004055433216  diff: 4.29497e+09
f: 8e+16  next: 80000010770448384  diff: 8.58993e+09
f: 9e+16  next: 90000008895528960  diff: 8.58993e+09
f: 1e+17  next: 100000007020609536  diff: 8.58993e+09
f: 2e+17  next: 200000014041219072  diff: 1.71799e+10
f: 3e+17  next: 300000029651763200  diff: 3.43597e+10
f: 4e+17  next: 400000028082438144  diff: 3.43597e+10
f: 5e+17  next: 500000026513113088  diff: 3.43597e+10
f: 6e+17  next: 600000059303526400  diff: 6.87195e+10
f: 7e+17  next: 700000092093939712  diff: 6.87195e+10
f: 8e+17  next: 800000056164876288  diff: 6.87195e+10
f: 9e+17  next: 900000088955289600  diff: 6.87195e+10
f: 1e+18  next: 1000000053026226176  diff: 6.87195e+10
f: 2e+18  next: 2000000106052452352  diff: 1.37439e+11
f: 3e+18  next: 3000000159078678528  diff: 2.74878e+11
f: 4e+18  next: 4000000212104904704  diff: 2.74878e+11
f: 5e+18  next: 5000000540009037824  diff: 5.49756e+11
f: 6e+18  next: 6000000318157357056  diff: 5.49756e+11
f: 7e+18  next: 7000000646061490176  diff: 5.49756e+11
f: 8e+18  next: 8000000424209809408  diff: 5.49756e+11
f: 9e+18  next: 9000000752113942528  diff: 5.49756e+11
f: 1e+19  next: 10000001080018075648  diff: 1.09951e+12
f: 2e+19  next: 20000002160036151296  diff: 2.19902e+12
f: 3e+19  next: 30000003240054226944  diff: 2.19902e+12
f: 4e+19  next: 40000004320072302592  diff: 4.39805e+12
f: 5e+19  next: 50000005400090378240  diff: 4.39805e+12
f: 6e+19  next: 60000006480108453888  diff: 4.39805e+12
f: 7e+19  next: 70000003162080018432  diff: 4.39805e+12
f: 8e+19  next: 80000008640144605184  diff: 8.79609e+12
f: 9e+19  next: 90000005322116169728  diff: 8.79609e+12
f: 1e+20  next: 1.0000001080018075648e+20  diff: 8.79609e+12
f: 2e+20  next: 2.0000002160036151296e+20  diff: 1.75922e+13
f: 3e+20  next: 3.0000004119663529165e+20  diff: 3.51844e+13
f: 4e+20  next: 4.0000004320072302592e+20  diff: 3.51844e+13
f: 5e+20  next: 5.0000004520481076019e+20  diff: 3.51844e+13
f: 6e+20  next: 6.000000823932705833e+20  diff: 7.03687e+13
f: 7e+20  next: 7.0000004921298622874e+20  diff: 7.03687e+13
f: 8e+20  next: 8.0000008640144605184e+20  diff: 7.03687e+13
f: 9e+20  next: 9.0000005322116169728e+20  diff: 7.03687e+13
f: 1e+21  next: 1.0000000904096215204e+21  diff: 7.03687e+13
f: 2e+21  next: 2.0000001808192430408e+21  diff: 1.40737e+14
f: 3e+21  next: 3.0000002712288645612e+21  diff: 2.81475e+14
f: 4e+21  next: 4.0000003616384860815e+21  diff: 2.81475e+14
f: 5e+21  next: 5.0000004520481076019e+21  diff: 5.6295e+14
f: 6e+21  next: 6.0000005424577291223e+21  diff: 5.6295e+14
f: 7e+21  next: 7.0000006328673506427e+21  diff: 5.6295e+14
f: 8e+21  next: 8.0000007232769721631e+21  diff: 5.6295e+14
f: 9e+21  next: 9.0000008136865936835e+21  diff: 5.6295e+14
f: 1e+22  next: 1.0000000904096215204e+22  diff: 1.1259e+15
f: 2e+22  next: 2.0000001808192430408e+22  diff: 2.2518e+15
f: 3e+22  next: 3.0000001586388738769e+22  diff: 2.2518e+15
f: 4e+22  next: 4.0000003616384860815e+22  diff: 4.5036e+15
f: 5e+22  next: 5.0000003394581169177e+22  diff: 4.5036e+15
f: 6e+22  next: 6.0000003172777477538e+22  diff: 4.5036e+15
f: 7e+22  next: 7.0000002950973785899e+22  diff: 4.5036e+15
f: 8e+22  next: 8.0000007232769721631e+22  diff: 9.0072e+15
f: 9e+22  next: 9.0000007010966029992e+22  diff: 9.0072e+15
f: 1e+23  next: 1.0000000678916233835e+23  diff: 9.0072e+15
f: 1.1e+23  next: 1.1000000656735864671e+23  diff: 9.0072e+15
f: 2.1e+23  next: 2.1000002236372023981e+23  diff: 1.80144e+16
f: 3.1e+23  next: 3.1000002014568332342e+23  diff: 3.60288e+16
f: 4.1e+23  next: 4.1000003594204491651e+23  diff: 3.60288e+16
f: 5.1e+23  next: 5.1000005173840650961e+23  diff: 3.60288e+16
f: 6.1e+23  next: 6.1000010356356512167e+23  diff: 7.20576e+16
f: 7.1e+23  next: 7.1000004730233267683e+23  diff: 7.20576e+16
f: 8.1e+23  next: 8.1000006309869426993e+23  diff: 7.20576e+16
f: 9.1e+23  next: 9.1000007889505586302e+23  diff: 7.20576e+16
f: 1.01e+24  next: 1.0100000946914174561e+24  diff: 7.20576e+16
f: 2.01e+24  next: 2.0100001805974393491e+24  diff: 1.44115e+17
f: 3.01e+24  next: 3.0100001944458672042e+24  diff: 2.8823e+17
f: 4.01e+24  next: 4.0100002082942950593e+24  diff: 2.8823e+17
f: 5.01e+24  next: 5.0100007986034752178e+24  diff: 5.76461e+17
f: 6.01e+24  next: 6.0100005242215269212e+24  diff: 5.76461e+17
f: 7.01e+24  next: 7.010000826300330928e+24  diff: 5.76461e+17
f: 8.01e+24  next: 8.0100005519183826314e+24  diff: 5.76461e+17
f: 9.01e+24  next: 9.0100008539971866382e+24  diff: 5.76461e+17
f: 1.001e+25  next: 1.0010001156075990645e+25  diff: 1.15292e+18
f: 2.001e+25  next: 2.0010001871021021499e+25  diff: 2.30584e+18
f: 3.001e+25  next: 3.0010002585966052354e+25  diff: 2.30584e+18
f: 4.001e+25  next: 4.0010005606754092421e+25  diff: 4.61169e+18
f: 5.001e+25  next: 5.0010004015856114062e+25  diff: 4.61169e+18
f: 6.001e+25  next: 6.0010002424958135703e+25  diff: 4.61169e+18
f: 7.001e+25  next: 7.0010005445746175771e+25  diff: 4.61169e+18
f: 8.001e+25  next: 8.0010013078220234266e+25  diff: 9.22337e+18
f: 9.001e+25  next: 9.0010011487322255907e+25  diff: 9.22337e+18
f: 1.0001e+26  next: 1.0001000989642427755e+26  diff: 9.22337e+18
f: 2.0001e+26  next: 2.0001002165756060452e+26  diff: 1.84467e+19
f: 3.0001e+26  next: 3.0001002419532489463e+26  diff: 1.84467e+19
f: 4.0001e+26  next: 4.0001004517983325846e+26  diff: 3.68935e+19
f: 5.0001e+26  next: 5.0001002927085347486e+26  diff: 3.68935e+19
f: 6.0001e+26  next: 6.0001005025536183869e+26  diff: 3.68935e+19
f: 7.0001e+26  next: 7.0001010813335834993e+26  diff: 7.3787e+19
f: 8.0001e+26  next: 8.0001005533089041892e+26  diff: 7.3787e+19
f: 9.0001e+26  next: 9.0001007631539878274e+26  diff: 7.3787e+19
f: 1.00001e+27  next: 1.0000100972999071466e+27  diff: 7.3787e+19
f: 2.00001e+27  next: 2.0000101595710381951e+27  diff: 1.47574e+20
f: 3.00001e+27  next: 3.0000102218421692437e+27  diff: 2.95148e+20
f: 4.00001e+27  next: 4.0000102841133002923e+27  diff: 2.95148e+20
f: 5.00001e+27  next: 5.0000103463844313409e+27  diff: 5.90296e+20
f: 6.00001e+27  next: 6.0000104086555623894e+27  diff: 5.90296e+20
f: 7.00001e+27  next: 7.000010470926693438e+27  diff: 5.90296e+20
f: 8.00001e+27  next: 8.0000105331978244866e+27  diff: 5.90296e+20
f: 9.00001e+27  next: 9.0000105954689555352e+27  diff: 5.90296e+20
f: 1e+28  next: 1.0000011248035896942e+28  diff: 1.18059e+21
f: 2e+28  next: 2.0000013051338828146e+28  diff: 2.36118e+21
f: 3e+28  next: 3.0000011312866897196e+28  diff: 2.36118e+21
f: 4e+28  next: 4.0000016657944690552e+28  diff: 4.72237e+21
f: 5e+28  next: 5.0000014919472759603e+28  diff: 4.72237e+21
f: 6e+28  next: 6.0000013181000828654e+28  diff: 4.72237e+21
f: 7e+28  next: 7.0000016164895380574e+28  diff: 4.72237e+21
f: 8e+28  next: 8.0000023871156415364e+28  diff: 9.44473e+21
f: 9e+28  next: 9.0000022132684484415e+28  diff: 9.44473e+21
f: 1e+29  next: 1.0000002039421255347e+29  diff: 9.44473e+21
f: 2e+29  next: 2.0000002189895917545e+29  diff: 1.88895e+22
f: 3e+29  next: 3.0000002340370579744e+29  diff: 1.88895e+22
f: 4e+29  next: 4.0000004379791835091e+29  diff: 3.77789e+22
f: 5e+29  next: 5.000000453026649729e+29  diff: 3.77789e+22
f: 6e+29  next: 6.0000004680741159488e+29  diff: 3.77789e+22
f: 7e+29  next: 7.0000008609109007983e+29  diff: 7.55579e+22
f: 8e+29  next: 8.0000008759583670182e+29  diff: 7.55579e+22
f: 9e+29  next: 9.000000891005833238e+29  diff: 7.55579e+22
f: 1e+30  next: 1.0000000906053299458e+30  diff: 7.55579e+22
f: 2e+30  next: 2.0000001812106598916e+30  diff: 1.51116e+23
f: 3e+30  next: 3.0000001962581261115e+30  diff: 3.02231e+23
f: 4e+30  next: 4.0000003624213197832e+30  diff: 3.02231e+23
f: 5e+30  next: 5.0000002263530585512e+30  diff: 3.02231e+23
f: 6e+30  next: 6.0000003925162522229e+30  diff: 6.04463e+23
f: 7e+30  next: 7.0000008609109007983e+30  diff: 6.04463e+23
f: 8e+30  next: 8.0000007248426395663e+30  diff: 6.04463e+23
f: 9e+30  next: 9.0000005887743783344e+30  diff: 6.04463e+23
f: 1e+31  next: 1.0000000452706117102e+31  diff: 6.04463e+23
f: 2e+31  next: 2.0000000905412234205e+31  diff: 1.20893e+24
f: 3e+31  next: 3.0000001962581261115e+31  diff: 2.41785e+24
f: 4e+31  next: 4.000000181082446841e+31  diff: 2.41785e+24
f: 5e+31  next: 5.0000006494770954163e+31  diff: 4.8357e+24
f: 6e+31  next: 6.0000003925162522229e+31  diff: 4.8357e+24
f: 7e+31  next: 7.0000006191257368754e+31  diff: 4.8357e+24
f: 8e+31  next: 8.0000003621648936819e+31  diff: 4.8357e+24
f: 9e+31  next: 9.0000010723447061802e+31  diff: 9.67141e+24
f: 1e+32  next: 1.0000001298954190833e+32  diff: 9.67141e+24
f: 2e+32  next: 2.0000002597908381665e+32  diff: 1.93428e+25
f: 3e+32  next: 3.0000000995440605423e+32  diff: 1.93428e+25
f: 4e+32  next: 4.0000005195816763331e+32  diff: 3.86856e+25
f: 5e+32  next: 5.0000003593348987088e+32  diff: 3.86856e+25
f: 6e+32  next: 6.0000001990881210846e+32  diff: 3.86856e+25
f: 7e+32  next: 7.0000008125538680137e+32  diff: 7.73713e+25
f: 8e+32  next: 8.0000010391633526661e+32  diff: 7.73713e+25
f: 9e+32  next: 9.0000004920603127652e+32  diff: 7.73713e+25
f: 1e+33  next: 1.0000000718669797418e+33  diff: 7.73713e+25
f: 2e+33  next: 2.0000001437339594835e+33  diff: 1.54743e+26
f: 3e+33  next: 3.000000370343444136e+33  diff: 3.09485e+26
f: 4e+33  next: 4.0000002874679189671e+33  diff: 3.09485e+26
f: 5e+33  next: 5.0000002045923937981e+33  diff: 3.09485e+26
f: 6e+33  next: 6.0000007406868882719e+33  diff: 6.1897e+26
f: 7e+33  next: 7.000000657811363103e+33  diff: 6.1897e+26
f: 8e+33  next: 8.0000005749358379341e+33  diff: 6.1897e+26
f: 9e+33  next: 9.0000004920603127652e+33  diff: 6.1897e+26
f: 1e+34  next: 1.0000000409184787596e+34  diff: 6.1897e+26
f: 2e+34  next: 2.0000000818369575193e+34  diff: 1.23794e+27
f: 3e+34  next: 3.000000370343444136e+34  diff: 2.47588e+27
f: 4e+34  next: 4.0000001636739150385e+34  diff: 2.47588e+27
f: 5e+34  next: 5.0000006997684095123e+34  diff: 4.95176e+27
f: 6e+34  next: 6.0000007406868882719e+34  diff: 4.95176e+27
f: 7e+34  next: 7.0000002864293513174e+34  diff: 4.95176e+27
f: 8e+34  next: 8.000000327347830077e+34  diff: 4.95176e+27
f: 9e+34  next: 9.000001358618340265e+34  diff: 9.90352e+27
f: 1e+35  next: 1.0000001399536819025e+35  diff: 9.90352e+27
f: 2e+35  next: 2.0000002799073638049e+35  diff: 1.9807e+28
f: 3e+35  next: 3.0000001227554362789e+35  diff: 1.9807e+28
f: 4e+35  next: 4.0000005598147276098e+35  diff: 3.96141e+28
f: 5e+35  next: 5.0000002045923937981e+35  diff: 3.96141e+28
f: 6e+35  next: 6.0000002455108725578e+35  diff: 3.96141e+28
f: 7e+35  next: 7.0000006825701638887e+35  diff: 7.92282e+28
f: 8e+35  next: 8.0000011196294552197e+35  diff: 7.92282e+28
f: 9e+35  next: 9.000000764407121408e+35  diff: 7.92282e+28
f: 1e+36  next: 1.0000000409184787596e+36  diff: 7.92282e+28
f: 2e+36  next: 2.0000000818369575193e+36  diff: 1.58456e+29
f: 3e+36  next: 3.0000003604399238217e+36  diff: 3.16913e+29
f: 4e+36  next: 4.0000001636739150385e+36  diff: 3.16913e+29
f: 5e+36  next: 5.0000002838205563124e+36  diff: 3.16913e+29
f: 6e+36  next: 6.0000007208798476434e+36  diff: 6.33825e+29
f: 7e+36  next: 7.0000008410264889173e+36  diff: 6.33825e+29
f: 8e+36  next: 8.000000327347830077e+36  diff: 6.33825e+29
f: 9e+36  next: 9.0000004474944713509e+36  diff: 6.33825e+29
f: 1e+37  next: 1.0000000567641112625e+37  diff: 6.33825e+29
f: 2e+37  next: 2.000000113528222525e+37  diff: 1.26765e+30
f: 3e+37  next: 3.0000003604399238217e+37  diff: 2.5353e+30
f: 4e+37  next: 4.0000002270564450499e+37  diff: 2.5353e+30
f: 5e+37  next: 5.0000003472030863238e+37  diff: 5.0706e+30
f: 6e+37  next: 6.0000007208798476434e+37  diff: 5.0706e+30
f: 7e+37  next: 7.0000005874963688716e+37  diff: 5.0706e+30
f: 8e+37  next: 8.0000004541128900999e+37  diff: 5.0706e+30
f: 9e+37  next: 9.0000008277896514194e+37  diff: 1.01412e+31
f: 1e+38  next: 1.0000000694406172648e+38  diff: 1.01412e+31
f: 2e+38  next: 2.0000001388812345295e+38  diff: 2.02824e+31
f: 3e+38  next: 3.0000002083218517943e+38  diff: 2.02824e+31

Conclusion

I hope I at least shed some light on the topic, and you at least know that there is such a thing as “floating point precision errors”.
And, now that you’re aware of the topic, you know what to google for

Here are some reading materials:

• http://gamedevs.org/uploads/numerical-robustness-geometric-calculations.pdf
• https://en.wikipedia.org/wiki/Single-precision_floating-point_format
• http://fabiensanglard.net/floating_point_visually_explained (Added 1/10/2017)

Please let me know if this topic contains any errors, and I’ll try to correct them asap!

[britzl]

Brilliant write up Mathias! This will be an excellent post to refer to the next time it comes up.

[nicloay]

Another thing to consider, is how the precision “error” of calculations accumulate.

It’s very easy to calculate something that has

Thanks. It’s really useful info.
I face this issue quite often. And what i found in another page is that you don’t need to worry about this problem if your action happens within 10 km. or 10 000 points. in this case you still have 3 numbers after the point (actually you can see that in the table above). If it’s more you will decently need to solve this problem.

Below in the video I have 300x1500 tile map, and you can see that i place my character to 70k Y position. but I don’t use built in physics. so everything is still smooth for me. otherwise I think i’ll have some problem in ground check calculations and so on.

[Mathias_Westerdahl]

Added link to this great explanation/visualization of floating points: http://fabiensanglard.net/floating_point_visually_explained/

@nicolay: Your case illustrates quite nicely the problems of accuracy. And it’s very similar to terrain in open world games (Just Cause etc). Instead of having one big terrain map, you have many “patches”. This way you can cull them easily, and LOD them, but also (as discussed here) render them at higher precision by moving the camera to each patch. I.e. each patch is defined in the space (let’s say) [0.0, 512.0].

For instance, in your case, you could have several tilemaps, and move them around and refill them when they are reused.

As for where you “don’t have to worry”, it’s very much application specific. In the beginning of Just Cause, the characters were rendered in world space, i.e. [-16384.0, +16384.0], but that really messed up the animations of the characters (they became jerky and ugly). Instead, we had to render the character closer to origin (0,0,0).

[sicher]

Here’s another visualization: https://twitter.com/D_M_Gregory/status/1044008750162604032

[Mathias_Westerdahl]

Rest assured, this a topic still tripping up even professional developers, so I can really recommend reading up on it until you feel you get the gist of it.
Here’s another good write up

[Mathias_Westerdahl]

And here’s a little web tool to play around with different values:
https://float.exposed/0x38fffff9

[eklmv]

Sometimes you just say “hello” to float overflow. Mine problem was false human logic, 9 or 20 not so big numbers. Then you can agree that 1/9 is something infinite 0.11111111 and definetly here is a problem, but what with 1/20 it’s round and nice 0.05, here computers hit you in the face, always remember about floats.
Little example which i run into right now:

local step = 1/20
for i = 0, 1, step do
	pprint(i)
end

[eklmv]

Ok, sorry for spamming that much, i found those workarounds:

local steps = 20
local step = 1/steps
local res = 0
for i = 0, steps do
	pprint(res)
	res = res + step
end

Or you need to compare numbers with some precision:

function equal(a, b)
	local precision = 0.00000001
	return math.abs(a - b) < precision
end

local step = 1/20
local i = 0
while i < 1 or equal(i, 1) do
	pprint(i)
	i = i + step
end

Or more cooler coroutine iterator which start with 0 and ends with actual 1 (1 == 1):

local function zero_to_one(float_step)
	local precision = 0.0000001 
	local before_last = 1 - float_step
	return coroutine.wrap(function()
			local i = 0
			while i < before_last or math.abs(i - before_last) < precision do 
				coroutine.yield(i)
				i = i + float_step
			end
			coroutine.yield(1)
		end)
end

for i in zero_to_one(1/20) do
	pprint(i)
end

[NeZvers]

Table contains: integer, the next 32 bit floating point value after the integer

What is that “next 32 bit floating point value”, where you got it from int?

[Mathias_Westerdahl]

f: 1.00 next: 1.0000001192092895508 diff: 1.1920928955078125e-07

It means that it cannot represent any number in between
E.g. “1.0000000000000001” will become either 1 or 1.0000001192092895508.

And when numbers get larger, the precision loss gets bigger:

f: 20000 next: 20000.001953125 diff: 0.00195312

Here, the difference between the two floating point numbers is ~0.002 (or 2 millimeters if you use meters as units). You cannot represent a 32 bit IEEE number in between those to numbers.

It might not sound much but it can easily manifest itself as animation jittering, or that geometry edges doesn’t align properly etc. And these “precision errors” will also easily propagate when you do multiplications/divisions/additions.

[sicher]

Stefan (schteppe)

[britzl]

And a follow-up:

Stefan (schteppe)

[TheKing0x9]

I wonder then, how do big open world games like GTAs have such a big open worlds loaded at one time, both precision and memory wise?
Is it some kind of smart loading that loads a new part of the world when the player gets near it, and unloads the older collection, meanwhile resetting position to origin?

[britzl]

big open worlds loaded at one time

The world can be big but I’m sure not all of it is loaded and rendered. You can still load and unload sections of the world and shift the world around without the player noticing.

[Mathias_Westerdahl]

Exactly. In Just Cause they have 32x32km map, and it would produce e.g. Animation artifacts if using positions around 16000meters.

So instead, the world is divided into 512meter patches, and when rendering/animating objects, they’re moved closer to origin (0,0,0) to be able to have better precision for all calculations.

Also, the patches are unloaded if they are outside of the camera frustum in order ti save memory.

Of course, there are several layers of detail for terrain and models (and animations), so the lowest detail is always loaded in memory in order to keep long drawing distances.

[Mathias_Westerdahl]

@TheKing0x9 Thanks to some old colleagues, I found this presentation about some more detailed insight from another former colleague (Emil “Humus” Persson). It explains some pitfalls we encountered during the development of Just Cause 2 with regards to a huge game world and floating point precision issues:

http://www.humus.name/Articles/Persson_CreatingVastGameWorlds.pdf

[TheKing0x9]

Thanks a lot @Mathias_Westerdahl.