Floating Point/Rounding

From testwiki
Revision as of 02:57, 15 June 2017 by imported>PokestarFan (Rounding: {{BookCat}}/possible general fixes using AWB)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Template:Floating Point Page

Rounding

IEEE 754 specified four distinct rounding modes. We will talk about those modes, and their effects in this chapter. Also, rounding can play an important role in the accuracy of a floating-point computation, so we will discuss those effects here as well.

Rounding Modes

Rounding Errors

Relative Error

Relative error is one measure of error commonly used with floating point numbers. If x~ is an approximation to x, then the relative error, r, is defined as

r=|xx~|x