Menu: Found in the DATA menu
The Evaluate command determines the impact of detected sampling errors on a population for a specified confidence level and sets out a “worst case scenario”, returning an upper error limit frequency for record sampling and an upper error limit for monetary sampling.
Note: Before using Evaluate, you must have determined the sample size using Analyzer’s Size command and you must have drawn the sample using Analyzer’s Sample command.
Use Evaluate to determine the effect of errors detected in record or monetary samples.
| • | In record sampling, the upper error limit frequency is expressed as a rate of error because results are based on the number, not the monetary value, of errors. For this reason, the upper error limit frequency appears in the Command Log as a percentage. |
| • | In monetary sampling, the upper error limit (UEL) is expressed as a monetary amount. The most likely error provides the amount of error we expect in the population. The upper error limit provides the “worst case” amount of error, based on the required confidence. |
Note: The theory behind statistical sampling is complex. If you are not familiar with the critical judgements required to perform statistical sampling, we recommend that you consult a statistical sampling specialist before using the Analyzer’s Sample, Size and Evaluate commands.
The Evaluate command uses the original sampling parameters and any errors found in the sample to calculate the upper error limit for the population.
Note: In monetary unit sampling, you can use Evaluate for the fixed interval sampling method only. In record sampling, you can use it for any method.
For record samples, the UEL is the rate of error (expressed as a percentage) that you are confident is not exceeded. For example, you can be 90% confident that the total error rate does not exceed 6.5%.
For monetary samples, the errors recorded are processed and reported in detail. The report includes the effects of each error and shows the most likely amount of total error and the UEL expressed as a monetary amount. This is the amount you are confident that total errors do not exceed. For example, you can estimate that the most likely errors are 50,000, but you can also be 95% confident that the total errors do not exceed 288,000.
In evaluating sampling errors, Analyzer uses the upper error limit cumulative factors of the Poisson distribution.
Note: By default, Evaluate dialog displays options for monetary unit sampling. To display the options for record sampling, click the Record radio button.
Analyzer uses the following formula to evaluate record errors:
Upper Error Limit Frequency = (UEL Cumulative / Sample Size)
The sample size is supplied and the UEL cumulative factor is determined from the Poisson distribution based on the supplied number of errors and the confidence level.
Analyzer uses a slightly more complex formula to evaluate monetary errors, but the formula is also based on the UEL cumulative factors for the Poisson distribution:
| • | The basic precision is the amount of error we are confident of not exceeding if no errors are reported for the sample. It is determined by multiplying the sampling interval by the Poisson UEL factor for the specified confidence (assuming no errors). |
| • | For each error entered, the percentage of tainting is determined by dividing the error amount by the recorded item amount. |
| • | For each error entered, an estimate of the most likely error in the population is determined. |
| • | For items smaller than the selection interval, the most likely error is the tainting percentage multiplied by the interval used for selection. This calculation is based on the fact that the particular item selected was not certain to be selected and therefore is representative of other errors in the population. |
| • | For items equal to or greater than the interval (top stratum items), the most likely error is the amount of the error. The previous formula does not apply because all top stratum items are selected and therefore the error is not representative of others in the population. |
| • | On completing the error entry, the errors are sorted in decreasing size of most likely error amount, with top stratum and understatement items listed at the end. |
| • | A precision adjustment factor is calculated for each error. |
| • | For items smaller than the sampling interval, the precision adjustment factor is the most likely error multiplied by the UEL cumulative factor for that error number in the Poisson tables. This reordering of the errors matches the largest errors with the largest adjustment factors, ensuring the most conservative (i.e., highest) estimate of the upper error limit. |
| • | For top stratum items, the precision adjustment factor is the amount of the error. Since all top stratum items are selected, all items in this population (and presumably all errors) are detected. |
| • | For understatement errors, the precision adjustment factor is zero. This means that the estimate of the upper error limit is not reduced when understatements are detected because this type of error is not directly tested for with a monetary sample. |
Note: Various sample evaluation methodologies use adjustment values for understatement factors ranging from zero (as in Analyzer) to the amount of the most likely error. If you prefer to use a different assumption regarding the treatment of understatement errors, you can easily adjust the detail supplied to reflect your reduction in the upper error limit. This does not affect the estimate of the most likely error, which is the same regardless of your assumptions about understatements.
| • | Finally, the most likely errors are accumulated to produce the total most likely errors for the sample errors noted. As well, the basic precision is summed, together with all the precision adjustment factors for the errors noted, to produce the upper error limit for the sample within the required confidence. |
In addition to the command parameters described below, the Evaluate command has the following command parameters: Screen and To. For a description of these parameters, see Command Parameters. For a description of supported field modifiers see Field Modifiers.
Specifies monetary sampling type. In the command dialog, click the MUS radio button.
Specifies record sampling type. In the command dialog, click the RECORD radio button
Specifies the confidence level, entered as a percentage for either MUS or RECORD sampling. In the command dialog, enter the confidence in the “Confidence” text box.
Specifies the sampling interval used for the MUS sample drawn. See Fixed-interval Sampling. In the command dialog, enter the sampling interval in the “Interval” text box.
Specifies the recorded amount of the sample item and the amount of the error associated with that item in an MUS sample. In the command dialog, enter the sample item and amount of error in the “Errors” text box as “item amount, error amount”.
For example, if the recorded amount is 700, but should have been 900, the item amount is 700 and the error amount is 200.
When entering, separate the two amounts with a comma. For monetary unit sampling, enter positive amounts for overstated errors and negative amounts for understated errors. For example:
| • | If the sampled amount is 100, but investigation shows it should have been 75, the item is overstated by 25. Enter 100,25 in the Errors text box. |
| • | If the sampled amount is 100, but should have been 125, this item is understated. Enter 100, –25 in the Errors text box. |
| • | You record understatement as negative because it is the opposite of overstatement. |
Specifies the number of records that were tested in the record sample. In the command dialog, enter the size in the “Sample Size” text box.
Specifies the number of errors encountered in the record sample you are evaluating. In the command dialog, enter the number in the “Number of Errors” text box.
EVALUATE RECORD CONFIDENCE confidence-level SIZE sample-size ERROR number-of-errors¿
EVALUATE MONETARY CONFIDENCE confidence-level ERRORLIMIT item-amount,error INTERVAL monetary-interval¿
You can use the Evaluate command to evaluate the upper error limit for a record sample with 95% confidence given a sample size of 120 and 3 errors found in the sample.
| • | The result is displayed in the Command Log showing that you can be 95% confident that the actual error rate in the population does not exceed 6.47%. |
You can use the Evaluate command to evaluate the upper error limit for a monetary sample with 90% confidence, given a sampling interval of 300,000 and three item amounts and their errors (700,250), (25000,1050) and (500000,12000).
| • | The result is displayed in the Command Log showing that the basic precision is 693,000. In your evaluation you specified a monetary interval of 300,000 and a confidence level of 90%. The basic precision lets you make the best statement you can: that you are 90% confident that the total errors do not exceed 693,000 if no errors are reported in the sample. |
| • | Given that three errors were reported, the most likely estimate of the total errors in the population is 131,742.86 and you can be 90% confident that the total errors do not exceed 892,429.72. |
| • | In the results, the 12,000 error in the 500,000 item is carried over unchanged into the Most Likely Error and Upper Error Limit columns because it is a top stratum item. A top stratum item is an item whose amount is greater than the specified sampling interval. Because all top stratum items are automatically selected, presumably all errors in such items are also reported and so there is no need for statistical projection. |
| • | The 250 error in the 700 item results in the largest adjustment to the errors reported. This is because the 250 error has the highest tainting (the error as a percentage of the item) at 36% and therefore results in the largest adjustment when statistically projected. |