Investments in human capital development – that is, the development of the knowledge, skills and qualifications of the workforce – are today seen as just as important for economic growth as spending on the economy. physical capital (Gu W. and Wong A., 2010). The main human capital development measure available to organizations remains the continuous training of their employees. However, Canadian and Quebec companies are still reluctant to devote the necessary resources to it, due to the difficulty of assessing the impact of this investment on organizational results (Bouteiller D. and Cossette M., 2007; Bailey A., 2007 ).
In this article, we present a performance evaluation approach to nature corporate training to reduce their reticence: the analysis of the utility (Utility Analysis). This approach – applied relatively recently to the field of training – is unique in that it centers the evaluation of training performance on direct observation of the development of skills and their mobilization in action.
What are the advantages of evaluating the performance of the training based on the skills developed by the participants?
The utility analysis has the following advantages, among others:
It makes it possible to determine the profitability of a training course directly on the basis of the skills developed by the participants who followed it.
It identifies the factors that influence profitability and on which we can act to improve training.
It makes it possible to evaluate all types of technical, commercial and managerial training.
In the remainder of this article, we compare utility analysis with two alternative assessment approaches and illustrate its application through recent academic studies.
What types of training results can we assess?
The evaluation model most cited in the academic literature is that of D. Kirkpatrick (Kirkpatrick D. and Kirkpatrick J., 2006; Kirkpatrick, 1959): This model describes four and then five levels of results according to which a training can be evaluated. (figure 1). The first level, that of reactions, is based on the evaluation of participants’ satisfaction with the training activity. The second level is that of the evaluation of learning, where we evaluate what the participants have learned during the training. However, a positive result at this level does not guarantee that what has been learned will be applied at work (Bailey A., 2007). The third level is that of behaviors, where we assess the application by the participants of what they have learned during the training. The fourth level of evaluation is that of the effects of training on the performance of the organization. Finally, J. Phillips (Phillips J. and Schirmer F., 2008) adds a fifth level, that of the economic return to training.
Figure 1: The 5 levels of evaluation of a training course (adapted from Le Louarn and Wils, 2001)
Evaluating the economic return of training: three approaches
There are several approaches to determining the results of training at the last level of assessment of D. Kirkpatrick’s model. Drawing inspiration from the typology proposed by Ann Bartel (2000), we present three of these approaches:
Analysis of the overall return on investment in training.
J. Phillips’ Return on Investment (ROI) assessment process, which involves evaluating the performance of a business training program based on performance indicators.
Utility analysis, which consists of evaluating the performance of a company’s training program on the basis of the skills developed in training and transferred to professional activities.
The presentation of each of these approaches will be accompanied by an example of application as well as an analysis of the advantages and limitations.
The first approach: analyzing the return on the overall training investment
In this first approach, the objective is to assess the impact of the overall investment in training on the performance of organizations. There are two variants: studies carried out on a large sample of companies or those carried out on a limited number of companies.
The first variant is the most widespread. The size of the sample makes it possible to calculate an average return on training which is representative for a particular economic sector or territory. To do this, the data is collected by means of surveys by mail or by telephone. The information collected mainly relates to organizational performance and the value of its human capital. Different statistical and econometric models are then applied to these data in order to demonstrate a relationship between the investment made in training and the performance of the company (Dostie B. and Pelletier M.-P., 2007).
The Representative of the sample is the main strength of this variant. Economists and policy makers can then rely on statistically significant results to make decisions about public training policy (Bailey A., 2007).
Several difficulties are nevertheless inherent in this first variant. The information collected is sometimes drawn from surveys with multiple purposes. This is why some important information can sometimes be missing (Bartel A., 2000). For example, researchers cannot always calculate a level of return on investment due to a lack of reliable data on training costs. Then the choice of variables and equations is always delicate, because they must more or less suit all the organizational contexts encountered. Finally, it is sometimes difficult to clearly determine the direction of the relationship between the results obtained. Indeed, how to distinguish between a company that becomes more efficient because it trains more and a company that trains more because, as it is more efficient,
This approach is sometimes applied to a smaller sample of companies (less than ten) in order to be able to collect more data in each company and to adapt the analysis method to each organizational context. In this second variant, scientists will seek information directly inside the company through interviews with managers or by consulting internal company documents. The information collected is thus richer. In terms of impact analysis, the statistical and econometric methods are quite similar to those described previously. However, they can be more easily modified to better suit the context of each company (its production processes or the role played by training, for example).