Research Article - Journal of Public Health Policy and Planning (2021) Volume 5, Issue 8

Designing for scale and taking scale to account: Lessons from a community score card project in Uganda.

Christine Aanyu^1*, Elizabeth Ekirapa Kiracho¹, Rebecca Racheal Apolot¹, Sara Bennett², Suzanne N Kiwanuka¹, Ligia Paina²

¹Department of Health Policy Planning and Management, Makerere University, Kampala, Uganda

²Department of International Health, Johns Hopkins Bloomberg, Baltimore, United States of America

Corresponding Author:

Dr. Christine Aanyu
Department of Health Policy Planning and
Management
Makerere University
Kampala Uganda
E-mail: caanyu@musph.ac.ug

Accepted date: August 08, 2021

Citation: Aanyu C, Kiracho EE, Apolot RR, Bennett S, N Kiwanuka S, et al. Designing for scale and taking scale to account: Lessons from a community score card project in Uganda. J Public Health Policy Plann May 2021;5(8):75-80.

Abstract

Background: Planning for the implementation of community scorecards is an important, though seldom documented process. Makerere University School of Public Health (MakSPH) and Future Health Systems Consortium set out to develop and test a sustainable and scalable model. This paper documents the process of planning and adapting the design of the incorporating key domains of the scalable model such as embeddedness, legitimacy, feasibility and ownership, challenges encountered in this process and how they were mitigated.

Keywords

Community score cards, Scale up, Accountability, Theory of change

Introduction

The purpose of this paper is to document the iterative design and planning undertaken by the MakSPH team to support the implementation of a community score card pilot in five sub counties and one town council in Kibuku district, located in the Eastern region of Uganda with a population of 202,033 people [1].

Methods and Materials

The intervention was conducted for five quarterly rounds of the intervention consisted of eight main stages; as illustrated in Figure 1, below. More details about the intervention can be obtained from the paper.

Figure 1: The community score card process (adapted from thecare CSC).

Theory of change for CSC implementation

The planning phase was guided by a theory of change (see Figure 2), whose development was guided by previously published scaling up frameworks in particular the Expand Net framework the FHS project institutionalization framework as well as project wide discussions (Figure 2). According to our theory of change, four main factors were central to ensuring that the Community Score Card process (CSC) designed was scalable and sustainable. These included embeddedness (entrenchment into already existing systems or processes or policies at the local or national level), legitimacy (working with persons/structures that are mandated to carry out specific activities), feasibility (low cost, simplicity of tools, acceptability, less human resource intensive) and ownership (high level stakeholder participation and acceptance of the CSC). We believed that the inclusion of these components would facilitate the ability of the CSC to stimulate collective action by the community, health providers, health facility managers, Sub County and the district leaders [2]. These actionswould then act through the six pathways proposed about the desired changes at various levels. The six pathways include strengthening citizens’ demand, increased resourcing, improving information flows, greater top down performance pressure, collective action on the side of citizens and collective action encompassing demand and supply. If these actions addressed the needs of the community and the leaders then we believed that the chances of institutionalizing, sustaining and taking scale into account of the CSC would be increased.

Figure 2: The community score card process (adapted from thecare CSC).

Our theory of change had five main assumptions; firstly, if the CSC was embedded in existing structures, it had a greater chance of institutionalization and sustainability. Working at community, health facility, Sub County, district and national levels would increase buy in and influence decision making processes to favor the needs of the community members. Secondly, CSCs that use legitimate stakeholders would trigger collective action from communities, providers and district officials. Thirdly, if there was ownership by various levels of stakeholders, it would be easy to sustain, institutionalize and scale-up the CSC. Fourthly, if the CSC was feasible then it would most likely be sustained, institutionalized and scaled-up and lastly if we took scale into account, then the CSC could be sustained beyond the life of the project.

Data collection methods

We conducted ten Focus Group Discussions (FGDs); five female and five male and seven Key Informant Interviews (KIIs) as well as one reflection meeting with the project team. Data was also abstracted from quarterly project and stakeholder meeting reports.

All FGDs and four KIIs were conducted while three KIIs were conducted to KIIs were conducted with purposively selected technical and political leaders involved in the implementation of the CSC.

The 10 FGDs were randomly selected from the 20 FGDs involved in the first three rounds of scoring. Each FGD had 10-12 participants representing different categories of interest groups (women and men of reproductive age, disabled persons, people with different socio economic status, elderly) and the villages in that particular sub county [3].

The FGD and KII guides contained questions aimed at gathering information related to changes observed, facilitators, challenges/ barriers, feasibility, sustainability, institutionalization and scaling up of CSC. FGDs and KIIs were conducted by trained research assistants fluent in both.

The reflection meeting was conducted once at the end of the fifth round of scoring with researchers from MakSPH. It was guided by a tool adapted from the ExpandNet 20 questions for developing a case study for scaling up.

Data was also abstracted from notes from the stakeholder and project meeting reports. These meetings were held with stakeholders from the district and sub county, implementers of the project as well as the research team from MakSPH every quarter throughout the 18 months’ period of the project. Highlevel district political and technical leaders facilitated the stakeholders and project meetings [4].

Data management and analysis

All FGDs and KIIs were transcribed verbatim. During the stakeholder meetings, notes were taken and then later typed. All transcripts were read several times to allow familiarization with the data. We then developed an analytical framework based on key themes; embeddedness, legitimacy, feasibility and ownership guided by the ExpandNet framework as highlighted in the project theory of change. Codes were then developed and applied according to the analytical framework. Any new emerging codes related to the study objectives were also included [5].

Results

We present the actions that were taken to design and set up a scalable CSC model that takes scale into account by putting in place features that enhance ownership, embeddedness, legitimacy and feasibility of the CSC. We also present the challenges that were encountered and how they were mitigated.

Ownership of CSC

To promote ownership of the CSC process two main actions were taken by the MakSPH team, firstly a wide range of leaders from different levels at the district were engaged throughout the planning, design and implementation process. Secondly a participatory implementation design was used where modifications were made based on feedback from the facilitators of CSCs. The inclusion of political and technical leaders in the community, sub county and district levels, as well as fostering spaces for joint dialogue across these groups was important for securing buy-in and enhancing inclusion of locally appropriate plans based on their needs [6]. Some of these leaders also participated as facilitators of the CSC meetings. This not only promoted buy-in, involvement and ownership of the intervention in all the sub counties, but also enhanced the implementation of the project.

The participatory implementation design further promoted stakeholder buy-in, ownership and involvement of the local stakeholders. Local leaders at various levels with the mandate to mobilize and call for community meetings were involved to secure community buy-in. Feedback from facilitators was sought after every scoring to help identify what worked well and what did not during the implementation. This helped the MakSPH team modify the CSC process so as to make it more acceptable and enhance its chances for institutionalization and scale-up. For example, in the initial rounds of scoring, the process was reported to be extremely labor intensive and so in the last two rounds of scoring (fourth and fifth rounds), the MakSPH team combined the FGD and interface meetings into one community meeting held at parish level to reduce the human resource obligations which had previously made CSC labor intensive [7]. Taking community feedback into account when re-designing the CSC intervention also helped to promote ownership as echoed below by one of the MakSPH team members.

Embeddedness of the CSC

Embeddedness into the local structures and systems was promoted by working with political and technical officers from Kibuku district local government (user organization) and alignment with existing structures and policies. In the selection of user organizations, a choice had to be made between using locally based Non-Government Organizations (NGOs) working in a related accountability area and using other locally existing structures. However, in Kibuku district there was only one active NGO doing accountability related work, with minimal staff. Moreover, conducting CSC meetings was not one of the major activities in their work plan. During the design phase, stakeholders and the research team therefore decided to use multiple existing technical and political structures such as Senior Assistant Secretary (SAS), Health Unit Management Committees (HUMCs), Local Council (LC) leaders and Village Health Team (VHT) members among others. These personnel existed in adequate numbers, and could carry out CSC activities as part of their daily activities since this was not outside their job description [8].

Legitimacy of the CSC

As noted above legitimacy was ensured by aligning CSC implementation within existing systems, policies and processes including; working with personnel who had the mandate to perform different tasks within the CSC process. This was considered important because such structures could potentially continue performing the expected services even after the project exits or continue with minimal additional pay since them (the local personnel) would be performing duties that are within their mandate. These leaders felt that the CSC was enabling them fulfil their mandate and were therefore supportive of the programme and its continuity. In addition, they command the respect that is required from the community, as acknowledged.

Feasibility of CSC

The research team aimed at designing a simple low-cost intervention to enhance the feasibility for scale-up, sustainability, and institutionalization of the CSC. During the implementation of this intervention, several actions were undertaken to lower the associated costs. These included use of locally existing personnel who could be paid government allowance rates which are lower than rates often paid to NGOs, removal of refreshments for the community meetings and allowances for the community and health workers. These low cost implementation approaches were however not always welcomed by stakeholders who were used to receiving allowances from other projects and political leaders as noted in the quotations.

Discussion

Whereas the scale up literature often puts emphasis on the ability to implement an intervention on a large geographical scale, taking scale into account for social accountability interventions emphasizes the importance of putting in place deliberate actions that encourage strategic partnerships that can enhance accountability by leveraging the influence of more powerful parties/stakeholders. In the discussion we reflect on the extent to which we were able to achieve both these aims by using a model that aimed at enhancing embeddedness, feasibility, ownership and legitimacy.

We found that by far the most important domains for enabling wide scale implementation within our framework were feasibility and ownership. To make the CSC feasible and scalable, attention should be paid to its design, technical capacity of implementers and the cost of implementation. The design should be simple without overly complicated processes and tools to allow stakeholders with limited capacity to use them. This calls for flexibility during implementation to allow modification of the model and its implementation. The complexity of the CSC process with regard to the number of meetings held and the time commitments for both the community members who attend the meetings as well as the facilitators of these meetings also affected the feasibility of implementing the intervention on a wide scale. Reducing the number and length of meetings therefore redeemed time and simplified the CSC process making it more feasible to the implementing team and other stakeholders.

One of the barriers to implementation of CSCs identified in earlier projects in Uganda, was the human resource intensity of the CSC process. In our CSC process, we made changes by reducing the number of meetings hence time commitments for both the implementers and community members. Additionally, the facilitators of CSCs should also have the technical capacity required to facilitate the CSC if it is to be implemented sustainably using existing structures [9]. This was achieved by the training that was offered to the district stakeholders who acted as facilitators and coordinators during the CSC scoring. Selection criteria of the facilitators by the implementers should therefore ensure that their capacity to carry out the required tasks is included. If the local facilitators lack this capacity, a team external to the district should provide support with the aim of enabling the district to strengthen its own capacity to support CSC activities. Furthermore, existing teams need to be available in adequate numbers to carry out scoring as required. Whereas we did the scoring quarterly, implementers should consider bi annual scoring if it is to be done as a routine activity.

High costs were also noted as factors that constrained scale up of interventions including social accountability interventions. To keep costs low it is important to minimize the inclusion of inputs that may attract high costs. Further details about the cost of implementing CSCs can be obtained [10].

Information access and citizen voice are often not enough to deliver accountability. They need to be accompanied by the support of powerful leaders and building of relationships. Local ownership and legitimacy were therefore particularly pivotal for taking scale into account. Working with legitimate persons enabled us to involve leaders who had the authority and mandate to take the required actions at community and sub national levels. Leaders at different levels can play a critical role in influencing the scale of impact of the CSCs. While legitimate community level leaders can play an important role in ensuring that the CSC’s are locally accepted and implemented successfully, they may not have much leverage in influencing upstream factors but can build coalitions with powerful stakeholders at higher levels. It is therefore important to plan for early and continuous meaningful engagement of leaders at community, district and national level. In Uganda community score cards are not routinely implemented under existing public sector processes. There are ongoing discussions with the national leadership to identify appropriate entry points for carrying out community score cards routinely and linking them with existing decision making platforms. This requires that the CSC processes are conducted by legitimate persons and embedded into the routine public sector processes aimed at enhancing accountability. Legitimacy and embeddedness are particularly important for scale up if the implementation model is relying on the use of existing public sector processes and systems [11]. Leaders also need to appreciate the benefits of their participation in the CSC to secure their buy-in and active participation in holding duty bearers accountable. It is therefore important to ensure that the CSC design allows the CSC to identify and contribute to meeting the local needs. From our findings, the key technical and political stakeholders and leaders interviewed reported that the CSC provides a useful method of assessing their performance giving them an opportunity to identify and solve problems affecting their communities. Hence their desire and enthusiasm to see the CSC implementation continue on a wider scale. These kinds of interactions can also lead to a scale shift where you find a large scale change in accountability as a result of influence from specific key leaders. Such changes can then be embedded into local systems by their inclusion in work plans, budgets and job descriptions.

One of the shortcomings of using legitimate persons may be political and elite capture. If this is detected, steps should be taken to overcome it by leveraging the support of the pro accountability actors. Another challenge was frequent changes in leadership. For example, during the eighteen months’ period of this pilot, two of the top technical and political leaders in the district were changed. Other authors have sighted this as a barrier to scaling up interventions.

Heavy reliance on interviews done among community members, leaders and the research team who were involved in the implementation of the project is one of the limitations of the study. This may have biased the responses. However, these interviews were triangulated by considering responses from all the different groups of stakeholders involved. Furthermore, we reported both positive and negative findings [12]. Another limitation was the short implementation period which was inadequate for observing scale up. Furthermore, this design did not allow us to assess the extent to which the community voice was truly realized. We recommend this as an area for further research.

Conclusion

Embeddedness, legitimacy and ownership were mainly encouraged and promoted through alignment with existing processes and systems as well as meaningful and strategic involvement of the stakeholders and local leaders at local and sub national level. The key factors that enhanced feasibility included use of a simple low cost design that was implemented by locally existing stakeholders. The use of a participatory implementation design with mechanisms for continuous support during implementation and availability of minimal funding for supporting key activities were also central to the success of the implementation process. Governments seeking to scale up CSCs and to take scale to account should keenly adapt existing models to the local implementation context with the strategic and meaningful involvement of legitimate key local and sub national leaders in decision making during the design and implementation process. Social accountability practitioners should document their planning and adaptive design efforts in order to share good practices and lessons learned. Ideally, enhancing local capacity to implement CSCs should be ensured through the use of existing local structures and the provision of technical support by the implementing partners until the local partners are competent enough to conduct CSC activities including facilitation, negotiation, mediation and community mobilization.

Consent for Publication

Consent to publish the findings of the study was obtained at the point of seeking consent to participate. The participants’ confidentiality and anonymity while reporting was assured with only investigators and the research team having access to the transcripts.

Availability of Data and Materials

The data used to undertake this study can be availed on request from the corresponding author.

Competing Interests

All the authors had no competing interests.

Funding

The project was supported by DFID through the Future Health Systems (FHS) Consortium program under the Award Number HRPC09 Delivering Effective Health Services. The content of this manuscript is solely the responsibility of the authors and does not necessarily represent the official views of the DFID.

We also acknowledge the Kibuku district administration, the research assistants and our study participants.

Authors’ Contributions

All authors contributed towards the conceptualization of the paper. EKK and CA did the data coding, conducted the analysis for the study and led the drafting of the manuscript. All authors reviewed the drafts, provided significant intellectual input and approved the final manuscript.

Acknowledgements

Special thanks to DFID who provided funding for this work through the Future Health Systems consortium. We also appreciate Kibuku Local government and the study participants for taking part in this study.

References

1. Agarwal S, Heltberg R, Diachok M. Scaling-up social accountability in World Bank operations. The World Bank. 2009;1?12.
2. Blake C, Annorbah-Sarpei NA, Bailey C, et al. Scorecards and social accountability for improved maternal and newborn health services: A pilot in the Ashanti and Volta regions of Ghana. Int J Gynaecol Obstet. 2016;135:372?79.
3. Mutale W, Stringer J, Chintu N, et al. Application of Balanced Scorecard in the Evaluation of a Complex Health
4. System Intervention: 12 Months Post Intervention Findings from the BHOMA Intervention: A Cluster Randomised Trial in Zambia. PLOS ONE. 2014;9:93977.
5. Weir E, d?Entremont N, Stalker S, et al. Applying the balanced scorecard to local public health performance measurement: deliberations and decisions. BMC Pub Heal. 2009;9:127.
6. Edward A, Osei-Bonsu K, Branchini C, et al. Enhancing governance and health system accountability for people centered healthcare: an exploratory study of community
7. scorecards in Afghanistan. BMC Heal Serv Res. 2015;15:299.
8. Milat AJ, King L, Bauman AE, et al. The concept of scalability: increasing the scale and potential adoption of health promotion interventions into policy and practice. Heal Promot Int. 2013;28:285?98.
9. Gullo S, Galavotti C, Altman L, et al. A review of CARE?s Community Score Card experience and evidence. Health Policy Plan. 2016;31:1467?78.
10. Hanson K, Ranson MK, Oliveira-Cruz V, et al. Expanding access to priority health interventions: a framework for understanding the constraints to scaling-up. J Int Dev. 2003;15:1-14.
11. Keenan KF, Van Teijlingen E, Pitchforth E, et al. Theanalysis of qualitative research data in family planning and reproductive health care. BMJ. 2005;31:40?3.
12. Greenhalgh T, Robert G, Macfarlane F, et al. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82:581?629.