This document outlines sampling issues to be considered by auditors and taxpayers in designing a sales and use tax audit. These issues were identified from my own review of the literature and helpful comments from sources in universities, government agencies, industrial firms, consulting firms, and professional organizations. All views and errors in this document are my own. I welcome your comments suggestions for improving this document. Please e-mail any comments or suggestions to will@willyancey.com
 

Written by

Will Yancey, PhD, CPA
6848 Midcrest Drive
Dallas, Texas  75240
Phone  972.387.8558
Fax      972.934.2813
Email: will@willyancey.com
Web:  http://www.willyancey.com/

Revised July 29, 2000.

Web address of this document: http://www.willyancey.com/sampling-design.htm
 
 

Contents

1.0 Audit Objectives

2.0 Sampling Methods

3.0 Planning a Statistical Sample

4.0 Executing Sampling Plan

5.0 Evaluating Sample Results

6.0 Projecting Results

Bibliography at http://www.willyancey.com/sampling.htm

1.0 Audit Objectives
 
 

1.1 Legal Constraints

• Statute and regulations
• Case law
• Agency policy and memoranda
• Annotated bibliography on the Web at http://www.willyancey.com/sampling.htm

1.2 Scope Alternatives

• Correct return - change either direction
• Deficiency assessment
• Refund claim
• Estimate effective use tax rate for a prospective compliance agreement

1. 3 Period and Entities To Be Examined

• Records available
• Statute of limitations
• Problem areas identified on past audits

• Cost of planning, data collection, evaluation, appeals
• Time available

• Risk that auditors’ estimate of total dollars of incorrect tax based on the auditors’ sample is different than the true amount in the population.
• Attempt to design and execute sampling plan to minimize sampling risk.

• Cost of data collection and analysis versus
• Cost of sampling risk ("cost of incorrect projection")

2.0 Sampling Methods

• Method choice depends on knowledge of available methods
• Acceptability to government and taxpayers

2.1 Sampling Method Categories

• Detail Census (section 2.2)
• Judgment and Block Sampling (section 2.3)
• Statistical Sampling (section 2.4)

• Examine each invoice in a population or a sub-population.
• No sampling risk.
• High cost of data collection.

2.3  Judgment and Block Sampling
 

2.3.1 Characteristics of Judgment Sampling

• Relies on auditors’ judgment about representativeness of sample.
• Unlike statistical sampling, quantification of sampling risk is not possible with judgment sampling.
• Block sampling is a form of judgment sampling
• Blocks often determined by how records are stored (day, week, month, invoice number sequence, etc.)

• Dr. Edward Bryant was a consulting statistician to Pennsylvania Department of Revenue and many non-tax-related government agencies.
• Block sampling provides no assurance that a month which has an average volume of business will have an average number of errors.
• I am reminded of the case in which two things were being compared:
• Each had four legs. Each was 20 inches high. Each was brown. Each weighed 14 pounds. Neither had a medical record.
• But one was a dog; and the other, a table.
• It is my impression that the courts are requiring a measure of precision; and block sampling cannot provide such a measure.

2.4 Statistical Sampling

• Plan: specifies probability that items are selected for the sample.
• Execution: requires a random number process to select sample items.
• Evaluation: sampling risk is quantified based on statistical theory.

• Population is not divided into stratum.
• Each item in population has equal probability of being selected.
• Reasonable method if expect all items have equal probability of error.

• Population is divided into strata according to known attributes.
• Goal is to design strata so that error rate is homogeneous (similar) within a stratum and heterogeneous (different) between strata.
• Texas Comptroller Audit Division uses stratified random sampling on large computerized audits, but does not perform the analysis of sampling risk required for true statistical sampling.

3.0  Planning a Statistical Sample
 

3.1  Defining Population

• Invoices with taxes accrued (possibly overpaid)
• Invoices with no taxes accrued (possibly underpaid)
• Selected divisions and time periods.

3.2 Sampling Unit

• Invoice or line items
• Credit invoices included in audit of purchases
• Refunds included in audit of sales
• Voided transactions
• Date of invoice or delivery

3.3 Testing Sampling Frame

• Sampling frame is the computer file of items that could be sampled, such as accounts payable ledger or sales register.
• Test that sampling frame is complete and accurate by comparing master list and supporting records.
• Estimate cost and time to retrieve supporting documentation.

3.4 Tolerable Error

• Specify the maximum difference in tax dollars between the sample projection and the true population value that would be tolerated (acceptable).
• Fixed example: within $5,000 of population mean
• Relative example: within 5% of population mean
• Decreasing tolerable error increases sample size (and data costs)
• Could depend on government policy, auditor judgment, and negotiation with taxpayer.

3.5 Acceptable Sampling Risk

• Specify the maximum acceptable probability that the sample projection differs by more than the tolerable error from true population value
• Example: 5% risk that projection and true value differ by more than $5,000 (also known as 95% confidence that projection is within $5,000 of true amount in the population).
• Decreasing acceptable risk increases sample size.
• By stratum or population as a whole?

3.6 Stratification Attributes

• What attributes are associated with changes in probability of error?
• Invoice size is most common stratification attribute, but often no reliable evidence that probability of tax errors systematically changes with invoice size.
• Consider detail census on strata with unusually high error rate or large dollar invoices.

• Invoice size category
• Operating division
• Slow versus busy months
• Experience of clerks
• Product line
• Customer
• Vendor
• Delivery location
• Areas effected by law change
• etc.

3.7 Sample Leverage

• Sample leverage is the amount the projected total error will change as the result of an error in one sample item.
• Sample leverage increases as:
• number of items in stratum increases
• average invoice amount increases

• Excel spreadsheet
• http://www.willyancey.com/leverage.xls

3.8 Initial Sample Size (N)

• Sample size increases as
• tolerable error decreases
• confidence requirement increases (or acceptable risk decreases)
• population size increases
• expected variance increases
• Allocate sample size among strata
• Consider expected variance and cost of data collection

• Ignore and select replacement.
• Treat all sample items with missing documentation as errors and include in projection.
• Estimate error rate on missing invoices at the average error rate on complete invoices in the sample.
• Detail missing invoices (assess tax only on sampled invoices with missing documentation).
• Compare missing invoice sample's observable characteristics (invoice size, invoice date, operating department, etc.) to the characteristics of the complete invoice sample.

3.10 Reviewing the Plan

• Government's statistical experts
• Taxpayer and advisers
• Attempt to reach consensus on procedures before data collection.
• Document areas where disagree on procedures.

4.1 Random Selection

• Random numbers from printed tables or computer generated random number function.
• Sampling plan may specify higher probability of selection for some strata than others.

4.2 Correctly Evaluate Each Sample Item

• Validity of taxable or non-taxable determination.
• Accuracy of tax rate by use, jurisdiction, and time period.
• Review questionable items.

4.3 Follow Sampling Plan

• Follow sampling plan's procedures for missing documentation, voids, credits, etc.
• Note issues that should be addressed in next audit plan for this taxpayer.

5.0 Evaluating Sample Results
 
 

5.1 Compare Results to Sampling Plan

• Is there a reasonably small probability that the total amount projected from the sample is not significantly different from the true amount in the population?
• Should the sampling plan be revised?
• What additional data, if any, should be collected?

5.2 Example of Tolerable Error Goal

• Goal: 95% confidence that point estimate of incorrect tax dollars is within $5,000 of the true amount of incorrect tax dollars in the population.
• Assuming a standard normal distribution ("bell curve"), 95% confidence interval around mean = estimated mean +/- 1.96 standard errors.
• Standard error around estimated mean equals standard deviation divided by square root of sample size.
• 1.96 times estimated standard error < $5,000
• Implies estimated standard error < $2,551

5.2.1 Example of Tolerable Error Achieved

• Estimated mean = $100,000
• Estimated standard deviation = square root of the sum of the squared differences from the sample mean.
• Estimated standard deviation = $20,000
• Sample size (N) = 100
• Square root of N = 10
• Standard error = $20,000 / 10 = $2,000
• (1.96) x $2,000 = $3,920
• Estimated 95% C. I. = $100,000 +/- (1.96)($2,000)
• Estimated 95% C. I. = ($96,080 , $103,920 )
• Compare to tolerable error: $3,920 < $5,000
• ACCEPT.  The $100,000 estimate is within a 95% confidence interval of the true mean.

5.2.2 Example of Tolerable Error Failure

• Estimated mean = $100,000
• Estimated standard deviation = square root of the sum of the squared differences from the sample mean.
• Estimated standard deviation = $40,000
• Sample size (N) = 100
• Square root of N = 10
• Standard error = $40,000 / 10 = $4,000
• (1.96) x $4,000 = $7,840
• Estimated 95% C. I. = $100,000 +/- (1.96)($4,000)
• Estimated 95% C. I. = ( $92,160 , $107,840 )
• Compare to tolerable error: $7,840 > $5,000
• FAILED.  The $100,000 estimate is not within a 95% confidence interval of the true mean.

5.3. Why Tolerable Error Not Achieved?

• Sample size is too small.
• Outliers (a few unusual items influencing estimates of mean and standard deviation).
• Data distribution is so skewed it is not close to the standard normal distribution.
• Achieving the tolerable error goal does not eliminate non-sampling risk.  For example, auditors may have incorrectly determined the taxability of a particular invoice in the sample.

• Collect more data.
• Remove outliers.
• Change strata boundaries.
• Revise tolerable error goal.
• Project to lower bound of confidence interval
• Other valid statistical procedures

5.5 Involvement of Taxpayer

• Taxpayer's representatives and consultants should be involved in the sample evaluation phase and discussions with examiner.
• Collecting additional data and delaying final determination is costly to the taxpayer.

6.0 Projecting Results
 
 
 
 

6.1 Minimum Number of Errors Required Before Projecting a Stratum

• When very few errors exist in a stratum, the estimated variance and mean have a higher risk they do not represent the true variance and mean in the population.
• California State Board of Equalization (SBE) Audit Manual, chapter 13, "Statistical Sampling",  section 1308.10 requires at least 3 errors in a strata before projecting error from that sample.

6.2 Mean Projection

• Also known as mean-per-unit estimator
• Mean error in sample = ($ error in sample) / (number items in sample)
• Projected error in population = (mean error in sample) x (number items in population)
• Could be sensitive to outliers, such as one extremely large error.

6.3 Ratio Projection

• Sample error rate = (error $ in sample) / (total $ in sample)
• Projected total error $ = (sample error rate) x (total $ in population)
• Ratio projection tends to overstate total error when sample size is small.
• Ratio projection is biased when data distribution is skewed, such as lots of small invoices and relatively large invoices.

6.4 Point and Interval Estimates

• Statistical practice requires both a point estimate (total $ of error) and a confidence interval estimate (quantified estimate of the range around the point estimate).
• Wider confidence intervals indicate greater sampling risk.

6.5 Lower Bound of Confidence Interval

• If achieved confidence intervals are wider than planned, consider adjusting the estimate of total error, such as the lower bound of 95% confidence interval.
• Example:
• Predetermined tolerable error = $5,000
• Estimated mean = $100,000
• 95% CI = ( $92,160 , $107,840 )
• Lower bound assessment at $92,160

• Taxpayer and government consult statistical experts while preparing appeals.
• Watch deadlines for filing appeals
• Educating judges and hearing officers about the reliability of statistical evidence.
• Precedents and revised policies

6.7 Use Results for Planning Future Audits

• Opportunities to improve compliance
• Estimate error rates across invoice type
• Identify factors associated with errors
• Identify possible stratification variables
• Opportunities to improve audit data collection process
• Opportunities to improve tax compliance function

Bibliography at http://www.willyancey.com/sampling.htm

I welcome your comments suggestions for improving this document. Please e-mail any comments or suggestions to info@willyancey.com