Strategy

In order to document the test battery’s validity and to assist in the setting of an appropriate cut score, a concurrent, criterion-referenced validation study was conducted. The concurrent, criterion-referenced validation method requires that the test(s) be administered to current employees. Performance data is then gathered on those employees. If the test(s) were a valid predictor of performance, one would expect a statistically significant correlation between test scores and the performance data collected. In other words, those employees who score high on the test are the same employees who demonstrate high levels of performance. Those employees who do poorly on the test would likely be those who demonstrate poor performance.

The four Scales identified as relevant to the job (i.e., Helping Disposition, Reliability, Rules Compliance and Responsibility) were administered to 108 grocery store personnel. Study participants included cashiers, stock clerks, deli personnel and butchers. The supervisors of these employees were then asked to rate them on reliability, attendance and punctuality, helping disposition, policy compliance and trustworthiness.

Factor Analysis
In order to determine if we could sum all of the performance ratings for each study participant to create an overall measure of job performance, the performance ratings were factor analyzed creating an overall performance measure. This “overall performance measure” was then used as the performance criteria for this study.

Correlation Analysis
To determine the extent to which test scores were related to performance, correlation analyses were performed. The first set of correlation analyses were performed to determine the degree to which the test battery chosen was predictive of the overall job performance rating. In addition, the correlations for the individual scales were also calculated.

Cut Score Analysis
One of the most frequently used measures of the utility of a selection procedure involves evaluating how well the procedure screens out those applicants who will become below satisfactory performers once they are hired. To the extent that these false positives are reduced, the costs associated with them (such as the cost of errors on the job, the costs associated with any additional training, costs of needed overtime to compensate for poor performance and the cost of terminating them and hiring a replacement) are also reduced. While the ideal selection procedure would screen out 100% of these individuals, this is very rarely (if ever) the case. A selection procedure is useful if it reduces the proportion of below satisfactory prospects at all (over random selection or the existing selection procedures). The actual usefulness (or utility) of the selection procedure depends on a number of factors, including the value of a satisfactory or above satisfactory employee and the cost of selecting an applicant who becomes a below satisfactory employee.

In order to evaluate how the test battery score could best be used, the effect of different “cut scores” on the number of unsatisfactory performers was examined, using the study sample. Job performance level was defined as follows: if the employee’s overall job performance rating was less than 4.0 (on a 1-7 performance rating scale), the employee was considered unsatisfactory. Using this definition for unsatisfactory performers, the percentages of the “selected” group considered unsatisfactory at various cut scores were calculated.

Conclusion

From the results of the statistical analyses presented here, it can be concluded that the HRTL Scales evaluated in this validation study were highly predictive of overall job performance for the employees of a large U.S. grocery store.

Aside from significantly increasing the standardization, objectivity and validity of the chain’s selection process, the CEO reported a significant return on investment, documenting the company saved $750,000 in turnover costs in the first year alone of implementing the solution.