The Role of Coherence in Strengthening Community Accountability for Remote Schools in Indonesia

Incoherence in accountability relationships can hamper the quality of education. Such incoherence can be a particular challenge in resource-constrained, remote villages where teachers tend to have higher educational capital and social status than the parents and communities that they serve. We analyze quantitative and qualitative data from a randomized controlled trial of a social accountability mechanism (SAM) for schools in remote Indonesian villages. The intervention had three treatment arms, all of which included the SAM, which engaged village-level stakeholders in a consensus-building process that led to joint service agreements for supporting the learning process. Prior analyses have found that all three treatment arms significantly improved student learning, but the treatment arm combining the SAM with performance pay based on camera-monitored teacher attendance led to much larger gains than the SAM-only treatment or the treatment arm combining the SAM with teacher performance pay based on a community-evaluated scorecard. Drawing on a range of quantitative data sources across all treatment schools (process monitoring, survey, and service agreement indicators) and qualitative data from nine case study schools (interviews and focus group discussions), we show firstly that the student learning gains across all three treatment arms were accompanied by increases in the coherence of the accountability relationships between village-level stakeholders, and in the degree to which these relationships were oriented toward the purpose of cultivating learning. We further show that the treatment combining SAM with camera-monitored teacher performance pay led to greater improvements in the coherence of accountability relationships than the other treatment arms, because the cameras improved both the technical capacity and the social legitimacy of community members to hold teachers accountable. This coherence-focused, relational explanation for the relative effectiveness of the treatment arms has more explanatory power than alternative explanations that focus narrowly on information quality or incentive structure. Our analysis reinforces arguments for ensuring that accountability structures are coherent with the local context, including local social structures and power dynamics.

x partially partially ✓ The cameras didn't just improve information, but also support (by empowering user committees) The most accurate are those using cameras, because teacher attendance is really evidenced with those cameras.
They are more scared of the cameras than of the user committees. Especially when it is connected to their allowance … -Project facilitator, for user committees in 6 schools " " KIAT Guru also had some negative impact because, actually, UCs don't have the right to evaluate teachers. As far as we know, only inspectors can evaluate. … In my view, teachers might just say to themselves: you farmers, on what basis could you evaluate me? They might just feel, in their hearts, that the community isn't eligible to do evaluations.
-Parent, SDK Konang (SAM+Score) " " The importance of power dynamics (and, hence, of support) is evident in descriptions of accountability processes Once, I didn't want to be photographed, because I considered it an oppression of teachers.
-Teacher who is ineligible for the allowance (and is unaffected by camera-based deductions), SDS Usaba Sepotong (SAM+Cam) " " Because of their lack of knowledge, we are evaluated by people who have a lower education level than us. That's what the teachers objected to, having former students evaluate us. … Because it's a government program, it must be done, and whatever happens it's important that we submit.

Coherent for student learning?
x partially partially ✓ But we think this probably wasn't the main driver, because information quality wasn't necessarily better … The other day, I didn't want to use the camera because I actually did go home early.
A user committee member said I could just go home first and take a photo later, but I didn't want to tell a lie.

Average teacher attendance within KIAT Guru treatment and control groups at baseline and endline, with two-sample t-tests for between-group differences (%)
... and SAM+Cam teacher quality also improved in areas beyond attendance 29 There has been a change in the attendance of teachers and more careful with their service agreement There are many examples of good communication between teachers and the community … Teachers have fulfilled their promise to use a variety of teaching and learning methods, and the children feel happier.
-User committee members, SDK Kondok " " Conclusion It's not just about the quality of information, but the coherence of information with other elements of accountability relationships.

It's not just about which indicators are monitored,
but whether the monitoring process strengthens the accountability relationship as a whole.

26613
KIAT Guru was implemented in 203 remote primary schools in 5 low-performing districts

KIAT Guru started involving community members in administering a low-stake learning assessment
Tes Cepat is a community-led, low-stake, and adaptive student learning assessment … with a teacher Service Agreement indicators example from SDK Kondok (a case study school)

Indicator Weight
Teacher arrives and leaves school on time. Monday -Thursday : 7.30am-12.20pm. Friday -Saturday: 7.30am-10.55am 20 Teacher disciplines students gently with positive discipline. Teachers are not to use harsh words and/or physical punishment when disciplining students.

20
Teacher gives homeworks to students, and makes sure parents are aware and signs students' completed homeworks. 10 Teacher motivates students using positive encouragements and advices. 10 Teacher informs parents of students who are facing challenges in school by conducting a visit to their home and writing a formal letter to the parents 10 Teacher creates problem sets for student learning groups to work on, and provides instructions to the problem sets during classroom learning 10 Teacher uses varieties of teaching methods, including story telling, singing, role playing, and question-answer with students, as well as teaching aids 10 Teacher supervises student learning groups by conducting regular visits to all groups at least once a month 10

Indicator Weight
Teacher arrives and leaves school on time. Monday -Thursday : 7.30am-12.20pm. Friday -Saturday: 7.30am-10.55am 20 Teacher disciplines students gently with positive discipline. Teachers are not to use harsh words and/or physical punishment when disciplining students.

10
Teacher gives homeworks to students, and makes sure parents are aware and signs students' completed homeworks. 10

Grade 1-3 Teachers ask students to rehearse reading letters and numbers daily before classroom lessons begin 10
Grade 3 Teachers to ask students to read short readings during classroom lessons 10 Teachers provides remedial assistance for students who are behind by providing additional lesson 15 minutes before class begins.

20
Grade 1-3 Teachers use letter and number cards as teaching aids for students to be able to read and count 10 Teacher supervises student learning groups by conducting regular visits to all groups at least once a month 10 Coherence between relationships: SAM schools Coherence between relationships: SAM+Cam schools Coherence between relationships: SAM+Score schools Improvements in teachers' service indicator scores were not associated with improvements in student outcomes Teachers in SAM+Score schools tried to exert more power over user committees