Fixed-point Arithmetic

Analyzer uses fixed-point arithmetic for its numeric operations, with the exception of financial functions, for two reasons:

Whenever two or more values are involved in an arithmetic operation, the result takes on a fixed number of decimals based on the operand with the largest number of decimals, often resulting in rounding. Analyzer always takes the largest number of decimals from the operands.

This rule is applied at each level in any calculation, starting at the lowest level and repeated until the expression is evaluated. Rounding is the most common cause of numeric errors in Analyzer.

Analyzer uses floating-point arithmetic for financial functions. See Financial Functions for details.

Rounding in Division

Rounding occurs more commonly in division than in multiplication. Analyzer rounds to the largest number of decimals from the operands.

For example:

7/3 = 2

The correct answer is 2.333333333…, but Analyzer rounds this to zero decimals since neither operand has any decimals.

Further examples are:

7/3.00 = 2.33

7.0000/3.000 = 2.3333

Rounding in Multiplication

Analyzer rounds to the largest number of decimals from the operands.

For example:

1.1*1.1 = 1.2

Whereas

1.10*1.1 = 1.21

Adding Decimals to Control Rounding

To control rounding, add the number of decimal places you require to a calculation by multiplying by one followed by a decimal point followed by a number of zeroes equal to the number of decimal places you require.

In the following example, Analyzer rounds the answer to two decimals:

7.21*2.33 = 16.80

To control the number of decimals to achieve a more accurate result, do the following:

1.0000*7.21*2.33 = 16.7993

Order of Operations

Be careful when using parentheses to override the normal order of operations. Placement of parentheses plays an important role in determining the level of precision obtained in the calculation.

An example of how Analyzer processes an expression that includes division, multiplication and parentheses involves calculating one day’s interest on $100,000 at 12%:

100000*(.12/365)

Analyzer first divides.12 by 365 (giving 0.00) and then multiplies this result by 100000 to yield 0.00. The correct answer is 32.876712…, but since the example uses parentheses to force Analyzer to evaluate the interest rate first, the result is rounded to zero.

If the parentheses are removed, the answer is correct to the penny:

100000*.12/365 = 32.88

Overflow Errors

Analyzer has a limit of 22 digits, including decimal digits, for mathematical operations. This limit applies to intermediate calculations as well. Calculations that exceed this number of digits result in overflow errors. Analyzer indicates a numeric overflow in the View by displaying the message ###ERR### in the affected fields.

By default, a numeric overflow will stop Analyzer from processing. You can change this behavior to allow continued processing. For more information, see Stop on Numeric Overflow.

Oversized numbers generally result from calculations involving many decimal places, such as financial calculations, percentages and present values. Although unlikely, multiplication can cause problems with oversized numbers.