Research Article - Journal of Aging and Geriatric Psychiatry (2017) Journal of Aging and Geriatric Psychiatry (Special Issue 1-2017)

Validation of the Competency Assessment Tool (CAT).

¹Master's student in health law and policy, Faculté de droit, Université de Sherbrooke, Centre d'excellence sur le vieillissement, CHU de Québec- Université Laval

²Research Professional , Centre d’excellence sur le vieillissement, CHU de Québec-Université Laval, Centre hospitalier St-Sacrement, 1050 chemin Ste-Foy, Québec, Québec G1S 4L8, Canada

³Département de réadaptation, Faculté de médecine, pavillon Vandry, Université Laval, Québec, Canada, Centre d’excellence sur le vieillissement, CHU de Québec-Université Laval

*Corresponding Author:

Dominique Giroux
Département de réadaptation
CHU de Québec-Université Laval
Canada
Tel: 418-656-2131
E-mail: dominique.giroux@fmed.ulaval.ca

Citation: Roy V, Giroux D, Carignan M. Validation of the competency assessment tool (CAT). Arch Dig Disord. 2017;1(1):9-15

Abstract

With the population ageing, the number of seniors with cognitive impairments is growing, as is the number of requests for competency assessments. The Competency Assessment Tool (CAT) was designed to assist health and social service professionals to clinically assess competency. A qualitative study was held to validate de CAT. Five focus group (including elders, caregivers and professionals from organizations protecting elders’ rights) were held, along with ten telephone interviews (with legal practitioners). All participants were asked for their overall opinion of the CAT and its relevance in improving assessments and protecting people’s rights. According to the results, the CAT is an innovative, comprehensive and relevant tool that provides a fair and nuanced assessment of competency. Implementing this tool will help to protect the rights and freedoms of the individuals evaluated by producing an assessment that will make it easier to identify the most appropriate protective measures.

Keywords

Cognitive impairments, Quebec, Alzheimer’s, Competency Assessment Tool (CAT), Validation study, Assessment process

Introduction

The number of people with aging-related cognitive impairments and disabilities is on the rise in Canada. The number of Canadians aged 40 and over living with Alzheimer’s disease and other dementias was estimated to be 340,200 in 2011 and this number is expected to double in the next 20 years [1]. Cognitive disorders can have a major impact on how people function in their environment [1]. For example, when people reach the point at which they are no longer able to manage their property or take care of themselves, the healthcare system has resources that can be put in place to support them and their families and compensate for their disabilities [2]. In Quebec, as in most jurisdictions, there are legal measures designed to protect people who cannot take care of themselves, such as guardianships and “Mandates in Case of Incapacity” (protection mandates) [2]. Since recourse to legal protection is based on assessments done by health and social service professionals, to protect people who have become vulnerable because of cognitive impairments, the legal and healthcare systems must work together efficiently. To date, however, this cooperation does not seem optimal [2,3].

In Quebec, assessing people’s ability to manage their property and take care of themselves with a view to instituting legal protection requires a medical examination and a psychosocial assessment done by a social worker. A judge analyzes the situation based on the assessments received and ultimately makes the decision regarding whether the person needs protection. The information required to do an appropriate and comprehensive assessment of competency is complex and varied. Some authors have lamented the lack of a competency assessment tool that is comprehensive and valid and integrates the medical, psychosocial and legal aspects of a person’s situation [3-5]. In 2013, a survey of the tools available to assist with competency assessments found that there was no common tool used by all the professionals involved and no systematic procedure for doing a comprehensive assessment of competency [6]. Since the quality of the assessment and the fairness of the conclusions reached regarding competency are directly related to the rigor of the process and the completeness of the information collected, it is vital to use assessment tools that integrate the different disciplines involved with vulnerable patients [2]. The lack of a valid assessment tool consistent with the professional practices of all the stakeholders concerned reduces the quality of the assessments and could lead to a violation of the rights of vulnerable people, for example, by not focusing enough on their residual autonomy, what they are still capable of doing and the resources that could be put in place to compensate for certain disabilities [2].

The Competency Assessment Process (CAP) and resulting Competency Assessment Tool (CAT) were developed to fill this gap [7]. The value and originality of this tool are based not only on the rigor of the approach used but also on its ethical approach to decision-making. The French version of the CAT has undergone several validation studies, including a study of content validity with focus groups and a Delphi study with health and social service professionals [6,7]. A summary of the steps in the Competency Assessment Process (CAP) that led to the CAT and the aspects covered are presented in Table 1.

Table 1.A summary of the steps in the Competency Assessment Process (CAP) that led to the CAT and the aspects covered are presented in the table below.

The objective of this study was to continue to examine the content validity of the CAT by triangulating the data with other stakeholders such as elders, caregivers and advocacy organizations, and to explore its content validity with legal practitioners. The study was approved by the CIUSSS de la Capitale-Nationale Research Ethics Committee (2015-2016- R22). The experiment was conducted with the consent of the participants.

Method

First, data triangulation was applied through focus group meetings in two areas of Quebec (Quebec City and Montreal). These focus groups comprised three types of stakeholders concerned about protecting rights in the competency assessment process, namely: 1) elders without any cognitive impairment (elders), 2) caregivers of elders with cognitive impairments (CG), and 3) professionals working in organizations that protect the rights of elders and caregivers (Org). The plan was to hold six 2h focus group meetings (three in Quebec City and three in Montreal) facilitated by the principal investigator.

To recruit participants, emails were sent to: 1) directors of relevant community organizations and support groups to ask for help in publicizing the study and recruiting elders and caregivers, and 2) directors of relevant advocacy groups to recruit professionals working in these organizations. Recruitment posters were also put up. People who expressed an interest were contacted by phone to explain the research objective. Before the first meeting, they were sent a PDF copy of the CAT so they could study it. At the beginning of the focus group meetings, participants were given the opportunity to read the consent form and ask questions about the study, after which they all signed the consent form.

Each focus group meeting consisted of three parts. First, the members of the group were asked to identify the challenges and difficulties encountered by elders and their caregivers when they had to consider (or may eventually have to consider) the need to protect an elder with cognitive impairments. Next, the group members were asked to share their expectations regarding which aspects should be taken into account in assessments done by health and social service professionals trained to make clinical judgements concerning competency. These first two parts were described in a recent article [8]. The third phase, which is discussed in the present article, consisted of getting feedback from the groups about the proposed tool (CAT), and especially its relevance, contribution to protecting elders’ rights, and completeness.

Subsequently, a content validity study was done with 10 legal practitioners who practise in Quebec and are often involved in determining competency. Participants were recruited by snowball sampling. Ten telephone interviews were conducted lasting about 30 min each. A PDF copy of the CAT was sent to the legal practitioners who agreed to participate in the study. They were asked to study the tool before the interview. The telephone interview covered three aspects: 1) relevance of the CAT, 2) irrelevant items in the CAT and items that should be added, and 3) clarity of the statements.

Everything said in the discussions and telephone interviews was recorded and transcribed. A thematic analysis of the content of the transcripts was done to extract comments and suggestions [9,10]. The principal investigator and a research professional not involved in the study reached a consensus regarding emerging themes. N’Vivo was used to code the data. Data saturation was reached during the analysis.

Results

Five groups instead of six were formed because recruitment problems prevented the formation of one group of caregivers in the Montreal area.

The results are discussed in two parts, first, the results of the five focus group meetings involving three types of stakeholders (elders, caregivers and advocacy organization professionals), followed by the results of the 10 telephone interviews with legal practitioners.

Focus groups

The participants commented on their overall opinion of the tool and its relevance in helping to improve assessments and protect people’s rights. They also made a few suggestions for improving the CAT. The results of these discussions are presented below. Table 2 summarizes the characteristics of the 39 people who participated in the focus groups.

Table 2 It summarizes the characteristics of the 39 people who participated in the focus groups.

Overall opinion and relevance

First, the CAT was considered “very relevant” (F2 – Elders – QC) in guiding competency assessments. According to the participants, it structures a complex process: “The tool, (…) it helps to reach, a reminder, it structures your thoughts” (H2 – Org – QC). This structure ensures a more rigorous and objective process. This was mentioned by participants in two groups (elders and organization professionals). They thought this objectivity was optimized by giving “examples of questions [that] are really relevant” (F2 – Elders – QC) in supporting assessments that collect facts: “It takes facts. (…) That’s why a tool can be interesting” (H2 – Org – QC). This rigor and objectivity are particularly important in a context where decisions have a major impact on people’s rights and freedoms. In addition, the comprehensiveness of the aspects covered was considered a major asset: “(…) you have tracking grids, when you collect (…) the tools (…) used to gather [information]. Sometimes, you‘ve forgotten something, you’ll check. That’s partly what it’s useful for” (H2 – Org – QC). One elder added:

“That’s where the tool you are developing, in my opinion, will be extremely valuable. Because who will judge the loss of competency of someone who doesn’t realize it themself? That’s where I think your tool could be fantastic. (…) an objective tool that says: “Look here, according to certain criteria, you are no longer able to look after [your] affairs.” (H2 – Elders – QC).

Another aspect raised by participants was that the CAT is personcentered and takes into account the person’s opinion, wishes and perceived need for protection throughout the assessment. Particularly during the ethical reflection step, this aspect was considered “fundamental” (H3 – Org – QC) because it “adds value to all the rest” (H3 – Org – QC). This unique aspect was identified as strength of the CAT:

“It’s what will make the difference, I think, with this new document (…) if it starts with the person, if it’s really personcentered (…). We start from another perspective (…), it’s an aspect that would be unique like a new tool to rely on.” (F2 – Org – Mtl)

According to an advocacy organization professional, this characteristic “humanizes the Mandate in Case of Incapacity” (H3 – Org – QC).

Another important aspect mentioned was that the CAT considers the family’s views: “What makes me really happy is when I see: ‘questionnaire for the person and family member’, I think that’s very important” (H3 – CG – QC). Involving caregivers in the assessment process is appreciated. According to participants, family members often have a more accurate view of the real situation since they are the ones who spend time with the vulnerable person on a daily basis. An organization professional said something similar: “Person/environment assessment [Step 3 in the CAT], that’s a relief (...). It’s important.” (F1 – Org – QC)

Suggestions for improving the tool

During the focus group meetings, there were not many comments about additions to make to the CAT. According to the participants, the CAT is “very comprehensive” (F4 – Elders – Mtl, H3 – Elders – QC), “complete” (H3 – CG – QC), “seems complete” (F3 – Elders – QC) or “seems to cover nearly all aspects” (F7 – Elders – Mtl). One participant said:

“It’s a very good tool, a tool that can be very effective for people in authority who will have to analyze, diagnose the older person’s condition. Personally, I’d add absolutely nothing to it; it’s really complete in my opinion.” (H3 – Elders – QC)

A few suggestions for improvements were made, for example, considering the person’s condition at the time of the assessment: “Has she just lost her spouse […] and is very emotional and upset? That will falsify the data for her assessment because she’ll be distracted. The emotional stress” (F4 – Org – Mtl). Also, some organization professionals suggested that the person who manages the property should be identified: “[…] because I know some people who are completely functional who don’t manage that aspect of their life at all.” (H3 – Org – QC) Another aspect mentioned by a professional was to allow people to express their opinion about their network and evaluate their ability to judge the quality of the help received. Some suggested rephrasing some questions. Finally, organization professionals in both Quebec City and Montreal suggested sharing the responses of the person and family member in order to clearly illustrate if there is a consensus between them regarding the perceived need for protection.

Interviews with legal practitioners

All of the legal practitioners interviewed by phone practise in the province of Quebec. Table 3 shows the professions of these participants.

Table 3 It shows the professions of these participants.

As with the focus groups, the legal practitioners who participated in the study gave their overall opinion of the tool and its relevance in helping to improve assessments and protect people’s rights. They also made some suggestions for improving the CAT.

Overall opinion and relevance

The legal practitioners consulted found the CAT relevant in ensuring a fair and exhaustive assessment. It helps to “(…) standardize, provide benchmarks, target the main elements, document some specific aspects rigorously; without doubt, it could be a very useful tool (…)” (J1). They also mentioned that “it’s a good way to think about the issue” (J2). Some indicated that the CAT is “very detailed” (J2, J7), an aspect that improves assessments because the tool is “much more detailed than what we usually do (…). It’s the kind of information I’d like to have” (J2). Like the focus groups, the legal practitioners found the CAT “complete” (J4, J8, J9, J10) or “exhaustive” (J7): “It is complete in the sense that it covers everything one would want to cover” (J3) and makes it possible to “really analyze all aspects of the person’s life” (J7). They said the CAT provides “a good overview of the person’s situation” (J8). One jurist mentioned the importance of ensuring a detailed assessment of competency: “the more detailed it is, the easier it is for the court” (J4). The decision regarding the need to institute legal protection is made by the court based on the information from health and social service professionals.

In addition, in the opinion of the legal practitioners consulted, the CAT does not contain any superfluous items: “each of these aspects helps to assess the person’s competency” (J8) and “everything is relevant” (J3). “When I read it, nothing seemed superfluous” (J1). Like the elders consulted, the legal practitioners noted that the examples of questions made the CAT more relevant: “(…) for a court that has difficulty figuring things out, with all the examples you give (…), it’s helpful” (J4).

Like the focus group participants, the legal practitioners stressed that the CAT is person-centered. In the future application of the tool, one said that he “really saw the multi- or (…) interdisciplinary team that meets with the family. It’s beneficial. It’s really helpful” (J6). Another jurist said that the CAT helps “to analyze the person’s needs” (J8). Finally, one legal practitioner said that the CAT contained the items he considered most important in the competency assessment, namely:

“The interests and values of the person and family members. I think it’s something that’s fundamental (…). To respect people, to ensure that the protection they get respects their values and the wishes they would have been able to express before.” (J8).

The person’s wishes and values are included in Step 3 of the CAT (Systemic assessment of the person and environment) and Step 5 (Ethical reflection and decision-making).

According to the participants, the CAT helps to protect the rights and freedoms of the individuals evaluated. It ensures a fair assessment and helps to avoid situations where “the finding regarding competency is made too quickly” (J5). On the contrary, “it forces [health professionals] to go through a long process covering different aspects, which points up how complex the process is” (J5). One legal practitioner noted that “knowing that such a tool is used is also reassuring for legal practitioners” (J7).

One legal practitioner thought the CAT was relevant because it “enables nuances to be made in the person’s need for protection” (J8). The aspects documented by the CAT help to “really understand [the person’s] situation, needs, abilities” (J8) in order to “be able to make nuances when appraising the person’s competency and how much protection is needed” (J8). One jurist indicated why it is essential to know how to bring out the nuances: “In law, we have a terrible shortcoming. We like labeling things. Is the person competent or incompetent? But no, it’s not like that in real life (...) we have a shortcoming in law: categorizing people too quickly” (J6). The CAT helps to evaluate “if we should institute a curatorship or guardianship” (J8). By fostering a more meticulous and nuanced assessment, it ultimately ensures that the person’s rights and freedoms are protected by enabling the jurist to determine more accurately the real need for protection and leave the person with some autonomy. In situations where “it’s never black or white” (J9), the CAT, and especially the examples of questions in the CAT, make “it easier (…) to determine the degree of competency” (J9) by making it “easier to quantify” (J9). One jurist added: “(…) competency is full of grey areas (…). (…) it’s a complex question that we must ask ourselves for almost every aspect of being human. And your tool makes this possible” (J3).

Finally, the importance of the ethical reflection and decisionmaking step (Step 5 of the CAT) was raised. One legal practitioner said it was his “first pleasant surprise, the ethical dilemma” (J6). He added that it would need “many more of these [competency] assessments with [the CAT]” (J6) because “ethical reflection is extremely important” (J6). This same jurist noted that the CAT was relevant because it included this ethical reflection step: “Finally, we are enlightening the professionals involved about an ethical education” (J6) and “the dynamic of the protection-autonomy conflict” (J6). The importance of this step was also raised by some organization professionals, who described it as “fundamental” (H3 – Org – QC) since it humanized the process and “added value to all the rest” (H3 – Org – QC) of the assessment.

Suggestions for improving the CAT

The legal practitioners made very few comments about additions to make to the CAT. As mentioned above, many of them said the CAT was a very complete tool covering all relevant aspects. A few modifications were suggested. For example, like some organization professionals in the focus groups, one legal practitioner suggested considering the person’s situation and state of health during the assessment (J8). It was also proposed that the person’s remaining capacities be highlighted: “to ensure that any residual autonomy is given greater prominence in the report’s conclusions” (J8).

Discussion

This content validity study collected information from a wide range of stakeholders. First, elders who may eventually have this type of assessment and caregivers who spend time with vulnerable elders on a daily basis (and are often asked to provide health and social service professionals with crucial information) gave their opinion of the CAT. This study also targeted people working for organizations promoting recognition and respect for elders’ and caregivers’ rights. The unique perspective of these organization professionals enriched the CAT. Finally, legal practitioners directly involved in the process of instituting legal protection and ratifying mandates were also consulted. This ensured that the information collected using the CAT is relevant and useful for making the final decision. Validation with legal practitioners addressed the research approach proposed by Kapp [11] and Moye and Marson [4], who recommended including legal practitioners when validating a competency assessment tool to ensure that assessments meet the needs of the legal system. Moye [12] and Moye and Marson [4] stressed that future work should focus on the numerous interrelationships between the law and clinical practice related to competency assessments. These authors maintained that research in this area must take into account legal norms and procedures with respect to competency and legal protection since declaring someone incompetent is ultimately a legal decision. Thus validating the CAT with legal practitioners improves consistency between health care and social services on the one hand and the legal system on the other.

Over the years, many authors noted the lack of comprehensive tools to assess competency [2,4,13-15], the assessment tools generally used are not precise enough to adequately support assessments of the ability to manage property and look after oneself, and they are often affected by the evaluator’s subjectivity. The CAT is the first comprehensive, valid tool assisting health and social service professionals to do clinical assessments of people’s ability to manage their property and take care of themselves. The discussions also indicated that assessments based on the CAT are objective and rigorous. They “(…) will be based much more on the facts. Facts are necessary (...) That’s why a tool may be interesting” (H2 – Org – QC). This also ensures a fair and nuanced assessment of the need for protection because, with all of the aspects evaluated, it is possible to precisely and rigorously analyze the degree of protection needed based on the type and extent of the difficulties: “(...) all the aspects that you highlight are important in order to see the nuances and gradation you can make” (J8). Gauthier and Pauzé [15] stressed the importance of being able to make nuances when assessing competency because this enables the professionals to better construct and explain their decision-making process and address criticisms, if any. One aspect of the CAT that results in this fair and nuanced assessment is the examples of questions used to gather the facts supporting decision-making. This important contribution was mentioned by all the participants. Implementing the CAT will, therefore, lead to the recommendation of protective measures adapted to the real needs of the person evaluated, thus avoiding an unwarranted loss of rights on the one hand or insufficient protection on the other.

According to Canuel et al. [13], competency certification requires a meticulous, multifactorial assessment. They maintain that, although competency requires a medical diagnosis, the person’s social and living environment must also be taken into account. In practice, these aspects determine whether or not a person who is incompetent needs to be legally protected since not everyone with similar disabilities is put under legal protection. According to Moye and Marson [4], by their very nature competency assessments are a complex, interdisciplinary process encompassing medical knowledge, clinical assessments, ethics and the law. They stress that one of the main objectives of competency assessments is to assist the legal practitioners who will have to confirm the type of legal protection required. This must be tailored to the person’s needs and clearly identify the areas in which the person needs protection, as well as the areas in which the person is still competent. This approach ensures the person’s rights are protected [4]. Because it is comprehensive, i.e., gives “a good overview of the person’s situation” (J8) and identifies relevant nuances when assessing the need for protection, the CAT is consistent with the current trend in law, described by Moye and Marson [4], to move away from the practice of creating a protection plan initially covering all areas of disability and to use less restrictive measures. Thus, with its “really relevant” examples of questions (F2 – Elders – QC), the CAT helps to establish differing needs for protection fairly. In addition, these “really relevant examples of questions” (F2 – Elders – QC) to ask the person being assessed and family members make it easier to understand the aspects evaluated and the comments made and shed light on facts relevant to the competency assessment, which makes it possible to establish the necessary nuances [16,17]. Some of the legal practitioners made similar comments:

“ (...) I found that there were lots of examples (...) and therefore for a court that has difficulty figuring things out, with all the examples you provide (...), it’s helpful” (J4);

“The examples of questions really help us to see if the person is capable of managing property” (J8).

Another important and innovative aspect of the CAT is that it prompts health and social service professionals to question the person assessed, with or without cognitive problems, and their caregiver, over and over again. The examples of questions throughout the CAT are addressed directly to the person being assessed or their family member. Also, the ethical reflection and decision-making step (Step 5 of the CAT) includes thinking about the person’s and family’s wishes the values that are most meaningful to them and the impact of the interdisciplinary team’s recommendations on both. Thus the CAT encourages the elder’s and caregiver’s participation in the reflection, as many of the participants noted. Having elders participate and putting them at the center of the intervention is a practice that has been encouraged. For example, Feinberg and Whitlatch [18] showed that people with mild to moderate cognitive impairments are able to consistently answer questions about their preferences and choices and their involvement in decisions about daily life.

According to some researchers, competency is graded on a spectrum that is difficult to quantify and some criteria that may determine a cut-off between competency and incompetency may include the degree of risk for the person, risk to others and indirect impacts on society. These same authors maintained that the lack of objective measures of competency and being unable to quantify the degree of risk are factors that complicate the assessment and make it necessary to shed light on contextual differences in each assessment. Step 4 of the CAT (Analysis of the situation and risk identification) meets this need by systematically identifying the risks that justify protective measures and grading these risks (imminent, probable, possible).

In addition, the CAT helps decision-making by including an ethical reflection step (Step 5 of the CAT) that guides the interdisciplinary team’s analysis of the possible alternatives taking into account some important aspects before considering whether or not to recommend legal protection or ratify a mandate. Aubé [16] stressed the importance of this aspect, saying that although there are standardized tools that can be used to try to measure competency as objectively as possible, competency assessments must remain a matter of professional judgement because it is a professional activity involving clinical reasoning [15]. This step also encourages the interdisciplinary team to discuss an ethical dilemma often encountered in practice, namely “the dynamic of the protection-autonomy conflict” (J6), described as the dilemma between the values of autonomy and beneficence by some authors [14,15]. Frank [14] notes that protecting vulnerable people’s independence requires asking their opinion and considering their history, values, beliefs and priorities, which is a more holistic approach. According to Frank, it is essential to look at the whole person and recognize people’s existential dimension and the meaning they give to their own life and to events and decisions that affect them.

Step 5 of the CAT addresses this expectation by integrating medical knowledge, clinical assessments, ethics and the law, as recommended by Moye and Marson [4]. The systematic inclusion of ethical reflection in the CAT’s approach also helps to develop health and social service professionals’ ethical awareness, as noted by one of the legal practitioners. An organization professional also mentioned the relevance of “trying to take the broadest possible view. Conflicting values, it’s interesting (...) to see if there are conflicts of values if one can identify them” (H3 – Org – QC). The CAT’s ethical reflection and decision-making framework helps to pinpoint conflicts of values, resulting in a better balance between selfdetermination and protecting the individual in the competency assessment process. A comprehensive assessment tool like the CAT strikes this necessary balance between society’s duty to protect vulnerable people and the fundamental need to respect people’s independence and dignity [14]. One legal practitioner noted that the CAT shows respect for the person’s autonomy while the person is still capable of making decisions: “I think that, through that, we can talk about the person’s remaining autonomy. Through the questions that are asked, the assessment that we do, we will see if the person still has some autonomy or not” (J8). “It [the CAT] helps to analyze the person’s needs (…), to say ‘she is capable of doing this but not that’” (J8). Competency is “a complex question that we must ask ourselves for every aspect of being human. And your tool makes this possible” (J3).