You can specify MINIMUM SCALE in the IP for numeric results by returning a value for the ipGetInfo option IP_INFO_MINIMUM_NUMERIC_SCALE. The value can range from 0 to 127. An error is reported if the value is greater than 127 or less than 0.
If the value for MINIMUM SCALE is not specified in IP, a default value of 3 is used.
The result precision of a numeric expression has an absolute maximum of 127. If the result precision is greater than 127, precision is set to 127 and the corresponding scale is reduced to prevent the integral part of the result from being truncated. However, if the reduced scale is less than MINIMUM SCALE, the scale resets to MINIMUM SCALE and MANTISSA is (M = 127 - Scale, that is M = P - S).
This MINIMUM SCALE configuration in IP ensures that:
• The operations AVG, STDDEV, DIVISION will have a scale >= MINIMUM SCALE.
• SCALAR FUNCTIONS have a scale specified by IP. However, for the SQRT function, scale >= MINIMUM SCALE.
• In case of ADDITION, SUBTRACTION, MULTIPLICATION, and VARIANCE operations, if the computed precision exceeds 127, the scale is set to MIN(MINIMUM SCALE, computed scale).
Note: If IP does not specify MINIMUM SCALE, for AVG, STDDEV, SQRT, and DIVISION operations, the default MINIMUM SCALE is 6 and for other operations it is 3.
