Abstract

This paper highlights the promise and scope of deep learning in the context of addressing one of the most burning issues in human resource management – predicting employee attrition. The study reviews key research papers in this area and tries to justify how much the practitioner community needs to update themselves to catch up with the latest developments in the field of machine and deep learning.

Introduction

Employee attrition is one of the major headaches for all organizations (Quinn et al., 2002). Attrition accounts for billions of dollars in terms of direct and indirect costs in the form of new hiring, lost productivity and opportunity costs (Sexton et al., 2004). In fact, the indirect or hidden costs of attrition account for 75-80 % of the total cost attributed due to attrition (Viswesvaran, 2003). Low turnover rates can help in ensuring over 50% more profit margins.  As per the United States Department of Labor, hiring costs for new employees may be as high as one-third of new hire salary. This statistic is more alarming for middle and high-level vacancy filling (Sexton et al., 2004).

It has so far been a holy grail for HR managers to successfully identify the employees who are the potential leavers vis. a vis. those who are the likely stayers. However, thanks to rising in awareness in data analytics, HR practitioners have slowly begun developing a data-driven culture and have started adopting powerful predictive algorithms to predict attrition of potential churners (Alao and Adeyemo, 2013; Tamizharasi and UmaRani, 2014). HR Analytics has become a major game-changer in making HR decision-making more evidence-based (Lepak and Shaw, 2008). Davenport and colleagues (2010) have further added fuel to the debate that Analytics tools and techniques can help firms leverage several pressing HR issues such as talent retention and management.

When it comes to the application of machine learning in HR functions, a number of studies are also similarly limited in methods. Studies documenting such practices like application of decision trees (Alao and Adeyemo, 2013) and logistic regression (Somers, 1999; Quinn et al., 2002) to help managers predict important HR processes and outcomes including employee selection (Chien and Chen, 2008), employee performance (Al-Radaideh and Al Nagi, 2012) and attrition (Alao and Adeyemo, 2013) have not provided enough practitioner-focused materials so that these applications can become more mainstream in adoption.

From a practitioner’s perspective, some of the leading organizations in the world such as Google, IBM and Hewlett-Packard have successfully implemented predictive algorithms to identify potential churners and have taken preventive measures to restrict voluntary turnover of high-potential candidates (Rasmussen and Ulrich, 2015; Elkjaer and Filmer, 2015).

Antecedents of Employee Turnover: A Theoretical Perspective

Why do employees voluntarily quit organizations? Researchers have identified several macros and micro-organizational factors behind the phenomenon. Price (1977) defines turnover as ‘the ratio of a total number of organizational members who have left the firm during a particular period divided by an average number of people working at the firm during the same period. Past research has shed light on several antecedent factors behind employee attrition. For instance, Firth and colleagues (2004) have highlighted job-related stress, organizational commitment, and job dissatisfaction as major antecedents behind employees’ decision to quit.  On the other hand, several researchers have argued that employee leaves their organizations primarily for monetary reasons (Griffeth et al., 2000). Toxic work environment (Abassi and Hollman, 2000) and organizational policies such as supervisory practices and communication transparency (Griffeth et al., 2012) have also been identified as major antecedents of voluntary attrition.

Researchers have developed several key theoretical frameworks for explaining the voluntary employee turnover process (Mobley et al., 1979). One early theory which explains why employees quit voluntarily is the ‘image theory’ (Beach, 1990), which posits that employees leave their organization after selectively analyzing certainly available decision heuristics. As per this theory, employees rarely have all the cues needed to decide whether to continue with their current employment. Even if most work-related factors are all right, individuals may decide to quit by concentrating on only a few negative aspects. Mobley (1977) developed one of the most widely cited and validated models describing the process of employee turnover. As per their model, employees first evaluate their level of satisfaction with their existing job and may either feel satisfied or dissatisfied with the same. If the latter is the case, then it triggers thoughts of quitting in the minds of the employees. Employees then start searching for alternatives and if they assess that the cost of quitting is not more than accepting the new opportunity, then they decide to quit. Later, Mobley, Griffeth, Hand and Meglino (1979) have expanded on their original model to include more distal antecedents such as adjustment issues of a spouse, organizational commitment, leadership perception and economic conditions of the labour market. The Mobley et al. model has received validation from several follow up studies and is considered as a benchmark work for conducting research on employee attrition (Lee et al., 2017). In this study, the author has referred to the extended model of Mobley and colleagues (1979) for identifying the variables/features to be captured/recorded in the historical dataset for predicting attrition. 

HR Analytics: mainstream adoption scenario

The application of analytics in HR functions has been received with divided opinions from HR practitioners. The pioneering efforts of Jac Fitz-enz and Thomas Davenport to raise awareness about measuring the impact of HR processes on firm performance have gradually gained enthusiasm among the HR fraternity (Bassi, 2011; Davenport and Mittal, 2020).

However, there is also a serious concern among most HR practitioners that HR Analytics is only a passing fad. Falleta (2014), for instance, has conducted a survey to determine the use of HR Analytics across Fortune 1000 firms. His survey of 220 such firms suggests that only 15% of study respondents claimed HR Analytics played a central role in determining or implementing HR strategy. Lawler and Boudreau (2015) also report the results of a survey of over 100 Fortune 500 companies suggesting only about 30 per cent of these companies measure the relationship between HRM processes and their impact on financial and business outcomes. This was puzzling since over 70% of respondents of this survey reported using HR metrics to establish how efficient their HR processes are. Marler and Boudreau (2016) have provided a critical review of the state of adoption of HR Analytics. Their review identified 32 research papers published on the theme of HR Analytics in either peer-reviewed or practitioner-focused journals. The researchers excluded the practitioner-focused papers due to a lack of empirical rigour which reduced the number of relevant papers published on HR Analytics to 14. Out of these 14 papers, the empirical rigour was restricted to mostly illustrative descriptions or descriptive surveys with the only exception of Aral et al (2012) who conducted some predictive analytics testing the impact of HR Analytics on financial performance. Marler and Boudreau (2016) have cited a lack of studies demonstrating the actual application of analytics in HR.

Rasmussen and Ulrich (2015) posit a balanced argument to highlight why HR Analytics as a discipline has its sceptics and how it can manage to steer clear of such negative perceptions. They argue that the main hindrance in adopting HR Analytics rests at the top of the organization. If the management is on board, then the implementation of the applications gets more serious focus. This posits a very interesting context – can HR Analytics be actually effective in solving critical HR issues such as attrition and employee performance? Can machine learning be the key to unravelling these unsolved mysteries? In the next section, the author tries to build a case for this argument.

Machine Learning Algorithms in predicting voluntary attrition

Data analysts have argued on the supremacy of one predictive model over another (West et al., 1997; Olson et al., 2012). Predictive modelling techniques help in forecasting future actions of individuals through a supervised learning approach. Such techniques have been widely used in several spheres of management research including credit risk assessment (Olson et al., 2012), consumer buying behaviour (West et al., 1997) and off late in the area of human resource analytics (Sexton et al., 2004; Alao and Adeyemo, 2013; Tamizharasi and UmaRani, 2014).

Predictive analytics models can be clubbed under two broad categories – white-box models (eg: classification trees, linear models etc.) and black-box models (eg: artificial neural networks, random forests, XG Boost etc). For example, in one study, Saradhi and Palshikar (2011) developed predictive employee churn models through a case study where they compare different models such as Naïve Bayes, Support Vector Machines (SVM) and Random Forests to predict churn of current employees based on past employee data.

In terms of complexity, there are essentially two broad types of ML algorithms – white-box and black-box. White box models have the advantage of easy interpretation. For example, decision trees have been found to produce easily interpretable tree diagrams and they can function seamlessly even with missing values in data, while black-box models such as neural networks and random forests require the data to be complete in all aspects and researchers need to be wary of missing value issues (Zhang, 2007). Black-box models are extremely complex and interpretation from such models is not an easy task given what goes on in the hidden layer can never be observed (West et al., 1997; Zhang, 2007). The biggest advantage of black-box type machine learning algorithms lies in their error correction mechanism known as backpropagation (Garson, 1998), which helps the models to find the best possible prediction. Black-box algorithms also operate well with data distributed in a non-linear manner. In other words, black-box models are not restricted by the basic statistical assumptions of linearity and normal distribution, making these more robust for capturing complex volatile non-linear relationships (Somers, 1999).

Neural Networks and Deep Learning

Neural networks can compute multiple iterative solutions by splitting the overall sample into training, holdout and testing data and then choose the outcome which best describes the relationship between the independent variables and the outcome variable(s). A simple Neural Network comprises three basic components – an input layer, a hidden layer for data processing and an output layer (refer to figure 1). 

The Multi-Layer Perceptron (MLP) is one of the most popular neural network architectures (Swingler, 1996; Somers, 1999). The MLP operates using the backpropagation technique of error minimization (Ripley, 2007). In backpropagation, the difference between predicted and actual outcomes is minimized iteratively by computing the observed error and then rectifying the weights of the input layer accordingly. Therefore, with each iterative learning cycle, individual cases are fed forward and the error is calculated and fed back to adjust the weights of the input layer in the model. This way, the final iteration renders a prediction that has higher accuracy compared to other similar statistical techniques such as logistic regression (Ripley, 2007).