The extensive norm update now available for the Watson Glaser Critical Thinking Appraisal provides more choice and accuracy for employers when determining candidates’ and existing employees’ ability.
In this week’s post, with 26 new norm groups available for the Watson Glaser, we highlight the importance of choosing a relevant group of people to compare test takers’ scores to.
When candidates sit an ability test, employers are attempting to discover their capability to perform a particular task, relevant to role they are applying for.
Ability tests are often used to help employers sift through a group of candidates, progress those who meet the required level and remove those from the recruitment process who do not.
In order to provide context, scores are compared to other people who have previously taken the test.
What does answering 20 out of 40 ability test questions correctly tell us about someone’s aptitude? Is this an average score? Or is it above average?
The way we make interpretations about how someone performs on a test is by comparing the test-takers raw score (i.e. 20 out of 40) against the raw scores of a group of people who have previously taken the test.
This process is referred to as ‘Norm referencing’ and the group we are comparing a test taker to, is commonly referred to as a ‘norm group’.
It is important for us to compare our test-takers to norm groups relevant to the role:
For selection, choose a norm group that closely reflects employees in a similar job role to the one you are selecting for. For example, Law firms might want to compare applicants to the Legal Professionals or Law Graduates norm. Similar comparisons are now available across many sectors, such as Accountancy, Professional Services, Consultancy roles and so on.
Comparisons against level of education achieved or seniority level have also been updated and expanded, enabling clients to compare results more accurately then ever before.
For development purposes, and to compare a test taker to others similar to themselves, select a norm closely matching their position or job role, e.g. Comparing a senior manager to a ‘Senior Managers’ norm group.
This will enable accurate judgements about how a test taker has performed in comparison to their peers. Inaccurate judgements will be made if a non-relevant norm group is chosen.
For example, a Senior Manager’s test performance of 20 out of 40 shows them to be ‘Well Above Average’ in comparison to a norm group containing school leavers. However, when the Senior Manager’s score is compared to a Senior Managers norm group, their score of 20 out of 40 is considered ‘Below Average’.
It is crucial to use norm groups that are closely representative of the group of people they are aiming to represent. A ‘Graduates’ norm group only containing 5 females with Engineering PhD’s, would not be an accurate representation of all graduates across the country.
Using this norm group would therefore generate less accurate interpretations about how a test taker has performed in relation to their peers.
Therefore when selecting a norm group you must ensure it 1) contains 200 people or more (as advised by the BPS), and, 2) contains the demographics of the group the norm group is presenting.
When developing the new Watson Glaser norms, we wanted to create a wide range of norm groups that would be able to fit many purposes. You will see we’ve created 28 different norms which feature varying employment groups, levels of education and career levels.
All of our norms contain at least 200 people or more, and are representative, e.g. the ‘UK Population’ norm group is demographically representative of the current UK population for gender, age and ethnicity.
The net effect of the update is more accuracy and choice.
For more information about the norm update and for advice about selecting the most suitable norm group for your test takers, please feel free to get in touch!
Title Image: Monika Chas, USA
Body Image: Marcella Robles Villela, Mexico
Request a free demo of the Watson Glaser
If you want to keep up to date with all the latest posts from TalentLens, sign up below and we'll let you know when new blog articles are posted.