New assessment methodology. Fads or here to stay?

At the SIOP Annual Psychology Conference 2019 in Washington DC, there were many fascinating discussions about the new ways to assess individuals and whether they are valid and minimise adverse impact. Most of the new methods are as a result of the advances in IT technology and the ability to analyse large amounts of data.

Big Data is nothing new

Big Data is talked about a lot but as one presenter said “It is like teenage sex: a lot of talking but very few doing it”.  Big Data is nothing new, it has been used by organisations for many years. Supermarkets, for example, have analysed shopping habits to look for patterns in products bought together. This helps determine where to place products in stores  When coupled with demographic and contact details held in a reward or club card database, shopping habits can be linked to demographic data. So analysing data is nothing new. What however has changed is the speed at which huge amounts of data can be processed. Other factors that have changed is the question of who owns your data (including data on social media profiles) and what organisations can legally do with it.  New GDPR legislation states that personal data can only be processed with the individual’s consent. Cambridge Analytica hit the global news when they analysed millions of Facebook profiles to target individuals with political advertising, without their permission.

Predicting behaviours

In the field of psychology, researchers are looking to see whether personal data and the words people use can help predict behaviours and personality traits and how the findings correlate with the findings obtained from the traditional and established ways to measure behaviour: self-reported personality questionnaires.

There were several presentations and discussions about how Natural Language Processing (NLP), video interviewing, gamification and even the wearing of virtual technology headsets are being trialed and used by some organisations. The analysis of an individual’s scores in these new assessments when compared with findings from well-researched Big 5 personality instruments showed mixed and varying levels of correlations. As more data is analysed, however, and algorithms refined, then this is likely to improve. Many of these assessments are built by IT engineers and not psychologists who should be consulted in their design and validation.

The interesting point from the meetings I attended, was that whilst clients are trialing new methodology, the areas they are measuring stay the same: skills (learned knowledge), cognitive ability, reasoning skills, personality, interests and motivation.  The HOW may be changing, but not the WHAT. The general consensus was that many of the new methods are not going to go away, but probably they need refining.  Until such time, traditional and well researched assessment methods will still play an important role.


Image credit: Photo by bruce mars from Pexels.