Blog – So, you think you could be a rater…Maybe not!

My name is Emma Law and I have been working in various capacities in clinical trials over the past 35 years, including being a trial coordinator, a rater, and a participant! In this blog I will outline what being a ‘rater’ actually means in the context of Clinical Trials and talk about one of the issues encountered.

In the landscape of clinical trials, we must be consistent across the world with how we measure various aspects of a person’s cognition, abilities and understanding. Therefore, we need to have specialist roles within the clinical trial team of people who can perform to a very high standard, these ratings. This is the job of the ‘rater.’

An example of something that a rater would do is to perform a Mini Mental State Examination (MMSE), as a crude measure of the persons current cognitive state, to the specifics outlined by the sponsor i.e. they might specify that the words Apple, Table, Penny are used as the three words to be memorised and recalled or it could be Lemon, Ball, Tree. Whichever it is – this must be done exactly as the sponsor specifies. The ratings must be standardised and performed in exactly the same way, no matter which country the trial is based in or which centre within each country, to accurately measure what changes are taking place.

So, you think you are an experienced rater? It’s all down to your background and qualifications! I’m going to outline some of the anomalies in being a rater in clinical dementia research- you probably won’t believe some of what I’m about to outline- but I can guarantee this is how it happens.

One of our Scottish sites were chosen to be a site for a commercial dementia study. The site had gone through all the necessary checks and balances of why it could deliver this study: they did have access to a minus 80 degree freezer -tick; they did have experienced staff to do the study-tick, they had a principle investigator willing to dedicate their meagre hard fought for research time, away from their clinical role – tick.

So, what’s the rub…the team had to all give their previous experience in rating scales. And this study was using the clinical global impression of change (CGIC), a scale measuring many global aspects of a person’s wellbeing and abilities including memory, orientation, judgement, and problem solving, community affairs, home life and hobbies and their ability perform personal care as one of their primary outcome measures of the efficacy of the trial. Enter the two people who were to be measured by another company (vendor) if they were capable of carrying out this assessment.

Candidate 1– a registered mental health nurse with a Masters degree, of 25 years experience in dementia care. Fifteen of those years working in research. Study coordinator to countless dementia studies and an experienced rater particularly in global scales. Much recent experience in all of these aspects of rating and a specialist in global scales such as the CGIC and Clinical Dementia Rating (CDR).

Candidate 2 – a registered mental health nurse with 35 years experience in dementia care with a doctorate. 13 years experience of working in research in a managerial role. Limited recent experience in scales used in dementia and some experience in global scales, but little over the past year.

Who was accepted as a rater and who was rejected with no form of appeal- yes candidate two accepted as a rater, while candidate 1 was not.

The rater experience- which is measured by tediously asking the prospective rater, how many times in the last year they have performed scales such as the MMSE, ADAS-COG, CGIC and CDR.

Who keeps a count of this? Therefore, it is sometime a finger in the wind, guessing game.

Then one is asked about years of experience and qualifications. And it is judged by the faceless vendor if you are capable of performing scales.

I was candidate two and I answered the questions truthfully- i.e. how many times in the last year have you performed an MMSE -zero, an ADAS-COG- zero, a CGIC – zero, a CDR – great, one I can answer other than zero- three times!

The biggest difference is that I have a doctorate and my fellow colleague does not. They have bags more practical experience in rating scales than I do, as I managed teams rather than delivered the research at a site level.

My main exposure over the years was to perform blinded CDRs which was especially useful to me as a manager, as I could keep my practical and clinical skills polished, still be involved in patient/participant care within research studies and visit sites actively delivering research.

The outcome was that the site had to withdraw from this study. As the CGIC was to be performed every 2 weeks and the site is a round trip of two hundred miles, and I only work 2 days per week, I could not take on this role. I was supposed to be the back up rater to cover holidays. Another potential study, which the people with dementia in that area of Scotland would not get access to.

There is a fundamental issue around the vendors who assess raters’ suitability.

I am active within the NIHR rater programme team dedicated to addressing issues such as this- we have a way to go yet!

Dr Emma Law

Author

Dr Emma Law is Strategic Manager for the The Neuroprogressive and Dementia Network in Scotland. Emma has 13 years experience as a Clinical Trails Network Manager and over 35 years experience as a Nurse, many of which were spent in the delivery of Clinical Research Trials. Emma completed her PhD and is passionate about giving people living with dementia and their carers access to participate in research.