no usage, no value

We talk a lot about the power of artificial intelligence (AI) algorithms. Many interesting personalized insights and recommendations can be provided to your end users. My first real experience with implementing AI at Randstad Group was the Talent Recommender algorithm. It is used to match candidates to vacancies. Recruiters are presented with promising candidates from our database for their vacancy.

It is easy to get blinded by the power of such an algorithm and to close our eyes for the practicality of actually getting it in production. Only when an algorithm is used to improve the life of our end users, we know the real value of it. From an engineering point of view, we realized that we needed more than just data science to make the Talent Recommender a success.

This is why we started with a multidisciplinary team. Two data scientists are paired with three software engineers, a scrum master and product owner. This way we had the knowledge in the team to properly build and deploy the algorithm and provide it as a web service to end user facing applications.

We were partly successful. We were autonomous in building such a service. Working with Amazon Web Services, we could build, deploy and monitor the solution. We could publish the web service and track its usage. All in all, we felt pretty good about it.

However, it turned out we still had a blind spot that obscured a very important aspect. The problem with building a web service is that it is not automatically integrated in a user-facing application. It looked somewhat like getting a beautiful clean silent sustainable electric car, while there is no power infrastructure yet to actually drive longer distances. To some extent we could use results from the algorithm with a small pilot group of recruiters, but we could not reach a larger population.

It takes a long time to put the necessary UI work on the backlog of the team responsible for the frontend. They have their own priorities, which did not necessarily match with ours. Therefore it was hard to gather feedback on the algorithm performance in real life. Also, changes to the algorithm or UI need to go the same route, making it hard to experiment.

Currently we have a dedicated team working on a new recruitment application, called Spotter. The two teams are working closely together. Recruiters can now use search and the Talent Recommender results to find and match candidates. In Spotter we also collect the necessary metrics to validate and improve our algorithms.

This collaboration quickly ensured a much broader adoption of the Talent Recommender among the recruiters of Randstad and Tempo-Team. Around 1000 vacancies are pulled through the algorithm every week. This allows us validate the algorithm much better and see where it needs improvement.

Regardless of the hype or coolness of a technology, it cannot perform on its own. The need to keep focussed on delivering value to the end users requires a broader perspective of data science. It does not matter how advanced your tech solution is, if it is not or cannot be used, it is just worthless. It is not about the algorithm, it is about the user.

tags blog