66526 Strengt henn ig d E uao c n Quai t ly n i a EstAsa i A S BERSs ytmA e s ses smentandBenh cmak ringforEduao c nReu sls t H T EWORL DBANK H T EWORD L A B NK Strengthening Education Quality in East Asia A SBR E:S yse tmAs ses smen tad n Be ncm h r aki ngo frE duao c nRe sut s l a EstAs iaPl o it t Srn e g thenn igEduao c nQuai t ly inEas tAsia d E t iedby Har ryAnh tonyPt arn ios 2012 TableofContents o Freword.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..x i Acn kow l edgm ent s.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..x . i h T eCont ribuo trs. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..xii i AboutSABER. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. . xv S um mary. .. ... .. .. .. .. .. .. ... .. .. .. .. .. ... .. .. .. .. .. .. ...xvii CHAPTR E 1. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..1 t Srug gln igw t ihQualt iy CHAPTR E 2. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..9 e Larnn igfo rmH g ihPerformingSyse tm s CHAPTR E 3. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..17 n Ifom r aon Ss ytems Ths ivolumeis ap r oductofthestaffof CHAPTE R 4. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..27 h teWo l rdB a n kwithext ernalconr tb iu- on. s Thefindings,n iter preao t ns,and t SudentAs sessments o cnclus ionsexprese sdinthisvou lme do notnecesa sri lyreflectthevie w softhe CHAPTE R 5. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..45 Word l Bank,t isBoardofExc euv eD r ie- c o trs,orh te gover nme nt stheyrepe rsen. t e TacherPoi c lies h T eWo l rdB ankdoesnotguarane te h te c acuracyo fthed ataincu ldedint his CHAPTE R 6. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..61 work.Theb oundari es,col ors,denomi- c ShoolAuo tnom yand Account abii ly t nao n, s ando therino frma ons hown onanyma pinths iw orkdonoti mp y l any u jdgme ntonthepa r toftheWo rd l Bank CHAPTE R 7. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..73 c oncerni ngt hel egalstat uso fa ny n E a ggi ng h tePrivt ae S ector e trri toy o r r the e ndors eme nt o r c aceptanceofsuchboundare is. CHAPTR E 8. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..87 Thes e c haptesw r ere p repar ed a s Voao c nalTa rckn ig bacg kroundp apersfort heE astA sa i Rego inalConfee rnceo nS yt semA sses- s ment andB enh cma rkn igforE ducao n CHAPTE R 9. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .101 Resut lshedi l nB ai l,I ndonesia,J une 6-8, 2011.Ths ireports ino tafoma r lpubi la c- n Ifom r aon andCom municao nsTechnologe is ono ftheWo l rdB ankG roup.I tis cr icul atedtoe no c ua rget houghta nd CHAPTE R 10. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..109 ds icusi son,a nditsu sea n dc t iao n h so ud ltakethi sintoaccount . Asaresut l e Tra y r Eduao c n ofh t econsut lao nprocesswit h goven r- mentsa ttheB aliconfee rn e c,u ne lss oh terw i sestae td, al ldatac ona tinedin CHAPTE R 11. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..123 h tisreportweresubsequentl y vl ai dated e Squencn igand Ta rde-offs h trougho fficialchannelsandn ao nal expet rs,andthema insourceof daai t s h te reu sltofths ipio lt po rgram; daa t o fr CHAPTER12. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..133 Chn iaa ndJ apana r en oto ffic a il .A ll Capac iy t Dee vlopment er rorsareourown. Foreword E ducation is vital to social and economic development. It empowers individuals and generates the knowledge and skills that fuel growth, promote peace, and reduce poverty. In the dynamic region of East Asia, education is a key element of investment strategies. International donors stand ready to respond to countries’ demands to link education to the demands of the international labor market and focus on breaking the cycle of inter-generational poverty by extending educational opportunities to more young people. While access and equity in education have improved in the region, serious challenges remain for many countries in terms of quality, the importance of which cannot be underestimated. The quality of teaching and learning will determine the pace and sustainability of economic growth in the region. It spurs higher productivity, higher earnings, and improved health outcomes. Importantly, it also builds resilience for households and countries, which is crucial for living in an increasingly interconnected and uncertain world. But to realize the benefits of education, schooling must be of good quality. Above all, to deliver high-quality education, education systems must be geared toward results for all: through strong stewardship, better alignment of policies, and enhanced service delivery. And yet international and domestic assessments show that learning and skill levels in many countries are alarmingly low, especially among poor and disadvantaged groups. Learning for all is the overarching message of the new World Bank Education Strategy and has been an important component of the education agenda of the United Nations Educational, Scientific, and Cultural Organization (UNESCO) since the release of the Delors Report, Learning: The Treasure Within, in 1996. The East Asia region consists of a diverse set of countries at different stages of educational development. While a few East Asian countries are looked on as educational models, the region also includes countries that need major education reforms, such as:  Improving outcomes for poor and minority groups  Ensuring that skills learned in school enable students to be productive and responsible members of society  Equipping young people with problem-solving and critical thinking skills and with communication skills To address these challenges, we need to take concrete steps to improve learning outcomes. First, a system-wide approach to educational reform is needed. Second, decisions must be based on what we know works. SABER (System Assessment and Benchmarking for Education Results) is a tool that policy makers can use to achieve these goals. It is a multi-year program we commit to in order to provide some guidance to improving the quality of education policies in a number of domains. It also enables comparisons between countries and promotes knowledge-sharing. The World Bank and UNESCO hold a shared commitment to quality Education for All (EFA). Our work will continue to be motivated by the knowledge that equal opportunity to a quality education is not simply a question of fairness; it is vital for generating sustainable economic development. Through this initiative and our ongoing collaborations with member states, we hope to realize that vision. Gwang-Jo Kim, Director, UNESCO Bangkok Elizabeth King, Director, Education, World Bank Ackno e wl dgements T On h i i s L s u t s k u - d S y t o h v a e s r , b H e a n n e a fit e o Y dg h si r ea mo t l t o y , f a o r n mt dJ i h a ei nG n u pu oZ t h o u f . S T h h ai s et aB t ea a k mw sh a s , u gidedandsuper vie sd byEi la zbeh t Kn igandEmma nueJ lime n z e,s a wel lasRobn iHorn,Albet roRodi rguez ,andEduad roVele. z h T est udy ge rwo utofan acve par tnerh sipw t ihtheUnitedNao ns d E uao c nal,Si cenfi c,andCulu tralOrgani zao n(UNESCO)Bangkok A (sa i andPacfi ic Rego inalBue r auforEduao c n, )headedbyG wang- oK J im, andb ytheg overnmento ftheR epubli co fKorea;the Kn igdomo fN eh t e rlands ; h t eU . K.D epar tment f orInternao nal Deveo lpme n; ttheKoreanMi nit sryofEducao n,c Siene c,and Tc eh- noo lgy;theEuropeanUnion;Russa iEducao n Ad i f orD ee vlopment R (EAD ) ;andtheG l obalParn tershp i f orEducao n. h T estudywas s upportedby e tchnia clteams r fomt hegovernments ofCambodia,Chn ia,n Ido nei sa,a Jpan,theRepublc iofKorea,theLao Peope l’sDemo cr ac Republ ic,Malay sia,Mo ngoi a l,theP hl ip ipn i s e, i Sngapore,h T a ia lnd,and Ve itnam,as wells a HongKong SAR,Chn ia, n a dShanghai,Chn ia.Wea s loa pprei catetheh epo l fGw ang- Chol ChangatUN EC S OBangkokand theteams h tatfacl i itt aed h te SABER S (yst emA s sessment n a dBenchma r kingforEducao nR eu slt s)East n a dSoutheastAsap i il ot:Aurei leA cocaandS i mo nE l i lsfromt he UNESCOInst uteofSta scs andYoungsupC hoi ,KateGlazebr ook, HuongLeT hu,Mo l l y Lee,MikiNozawa,SatokoYanofromU N S ECO Bangko. k Wea s lothank u Jng-HwanC hoi,TuyaDorg jotov,Ma narEi lrq isous, i Veo rnicaG rg ier, a Chi rsn eH oa rnsky,andJessc iaLeefo ranal yss i, o cding,webst iesa ,ndb log, sasw el lasVy TN . guyenforcont rb iun g h te phot o . sWe thankthenao nae lxpertswho hep led col lec t daa t: ThaoLeAnh,Hendarman Anwar,Si thaChhinh,Kwangho Ki m,H aena e Le,AriunaaLh kagvasue rn,Jiei xnLu,HonestoNuq ui ,Kei ichiOgawa, o Jng-HyoP r ak,SopheaPheng,Khamp haoP honee ko ,HoThanhMy Phuong,Die rkP on rsima,C hay iuthP una ysavatu st,G anu skhS ukh- bat aar,SoonSengThah,Eh t elAgnesVae lnzuea l,Ron gWa ng, Yendri Wid r , aVise ineX ay iasens ou , kBan HarYeap,and Ahmad Nurua lzam Md Zan i. Wet hankparc p iantsat e sveralcone frencesinWa shi ngton,D.C; . So e ul ,Ko e ra; Bangk ok, Thai land;andH ongK ongS AR,China.In parcula, r h teWo rdB l ankisgrateu fltot heMinisr tyo fNao nal Eduao c nof n Ido nei safor hosn gtheS ABERE at s AsaR i egi onal Cone frence;H.E.Mo hamma dN uh,Minis terof Nao nalEducao n ofn Idone i saandF asi lJaa ll,Vc ieMi ni sterofN ao nalEduao c no f n Idonesa i,f orrepresenn gtheg overnmen tatt hee ve n ;a t nd HubertGiz jena ndMe eYoungC hoifo rmU NESCOJ aa krtaandt he n Idonesa inN ao nalC ommi s sionfo rU NESCO,f orh ep lingt o og raniz et heminise tra ilforum.Thecon e frencew asas losupported bytheWo rdB l ank Eat s Ai saandP acfi ic e tama ndt heWo rl dB ank n Idonesac i ountryteam: Ma eChuC hang, MeghaK apoor,Shel ia Town,andYvonne Te rthewey,aswellasAymiLt ian, iIgaSai r,andthe t saffo fPat co ConvexLd t. TheContr ibutors Gusa tvoAri ca Cons ula tnt ,Wo l rdBank mi E y l Br eal rey o Sca ilPr otco e nSpec iai s ltWo , r ldBank Marguei rt e Ca lre k Sn e o irEduao c n Spec iai s ltWo , r ldBank h S nJ a g aannah tan d E uc aon p Sec iai lt s,As ianDee vlop n me tBank u S nt iaKoa srj au d E uao c nSpei cal i stWo , l rdBank a Lua r L ewis e LadAdvs io, rCf BTEduao c nTu rst u Jla i L ibema r n d E uao c nSpei cal i stWo , l rdBank Kei vnMacdonad l c Eono s mit ,Wo rd l Bank u S ngmi n Pak r e SniorEduao c nSpec iai s ltWo , r ldBank Har ryAn h tonyPat rn io s e Lad d E uao c nEo cn omit s,Wor ldBank mi E i loPot ra e Sno irEduao c nSpec iai s ltWo , r ldBank Mara i-o JseRamie rz Eduao c nSpei cal i stWo , l rdBank a J l miSami l e TraryEduao c nCoor dinao tr, Wol rdBank Michael Trucano Sn e o irC I TandEduc aonPol i cySpec iai s ltWo , l rdBank mi E i lanaVegs a e LadEconomi stWo , l rdBank AboutSABER OPENN IGT H EBA L CKBOXOFEDUCATI O Y NSSE TMS I mpo rvn igeduao c nqual i tyrma e n isa j maorchal l ene g o frgov ern n me s tac ros s h tewol rd.Asa e rsul t,h teWo l rdBank, inco lsecol laboao r nwih t h teUnie t dNao nsEducaona, lSce infi c,and Cut lua rlOra gnz iao n(UNEC SO )n a dwih t u fndi ngfo rmt hegovr enme ntoft heRepubl i c ofKoe ra, e stupa mulyearpror gamtop o rvidegud ianc e op t oi c ly makes r onhowtoimprovequai ly t by benh cmarkn igthei reducaonsystms e . h T s iprogramisknown asSABER– y SstemA se sssmentand Benh cma ri kngforEduao c n Reu sls t –andispartofh teWo rd lBank’sEduc ao nSeco trSta rteg, yLearnn igforAl: ln Ivesn gi n Peope ls ’ Knowld e e g andSki l lstoPromoteDevelopment.SABERenabe lspoli cyma e krstol ook n isd ietheir d e uao c nsys tems,andbee runderst and h te pol iy c doman isthatma e k upthewhole.Thefirst A S BERpil ot wascondut cedinE a stA i saw her elow -andmiddlei -no cmec ounti resareco lseto c ahievn ig unv ier salpr imaryenrome l n. t Thechal lengenowistoimprovethequal iy t ofl ean ring. SABE s Ri aninia veth t a hep lscount re is ss ytma c e l aye l xamineands r tengh tent hepero fr- manceofther i d e uao c ns yt sems oa t chiv eelearnn igforal. lTheWo rldBankisw or kingw t ih pat rnes raroundthew or ldtodevel opda ignosc toos lthatbenh cma k r d e uao c n poi li cesaco crd- n igtoevd ience- b s aed g lobals tandad rsandbestpa rcc e.Byl everagn igglobak lnow l edge,SABER fil lsagapi n h te avai l abii ly t ofpoi c lydata ,ino frma on,and n kow e ldgeon whatma e s r mostto mp i o rvethequal ityofeducao n. d E uc ao n y sst ems ae r l cas sifiedasbeingmoreorle ssadvancedn i eah c ofthee s goal s.h T efour e lvel sofcl assfi icaon ae r r fomleasttomostdeve loped.Foreah c ofh te ouc tome , seachcounty r s iassg inedal v eelofdeveo lpme ntgaugn igthee xe tnttow hi chithas suc cessu flyr l egulae td h tissa cl, ew hc ihrn a e gsfromlatentt o emeg ringtoesa tbl ih s d e toadvane cd. a Ltent E meg rin s g Eta b i s lhed Advn a e cd ○○○ ○○ ○ Reflec tsl imie td Reflet cssome Reflet csgood Reflec tsine tr- bet spracces o god prcc a e pa rcc e,wt ih nao nab lest o sm elm i i taons pa rcc e Summary Benh c r maki ngEduao c nPoi ly c n i E astAsa i m E l y i Br eal rey T hen idivi dualbenefis tofine vsn gineducao narea lrgeand wel lknown:h terei s asto rng i n lk bew t e e nwhatsomeoneknows andhowm uchheor h se ea rns.n Ice rasn igy l,t he benefis t os t ocit ey ofaneduct aedw ok r o frceareas lobee runderst ood,suchasgi anso fr e conom c i r gowtha nddeveo lpm ent ,poveryr t educo n,e tchnoloi gcalpo rgrs es,andgoodci zn e h sip. h T t a a sid,wesl lknowlileaboutthe“blakb c o” x ofeduc ao ns yt semsthem e slve—t s he dff ierentpoi ly c domai nsand c om ponens tthatmakeup h tew hoe l.n I thebest -cs ae s cenai ro, h tesecom ponens tshoud l l sott ogethero t e n u sreopporu tniesforthem ai xmumn um berof t sudent sa ndq uai lt yi nt srucont hate qui pst hem w t ihl aborm ark et-e radys kl is l. h T s isu tdy ag ruesthatt o m i pr oveeducao nquai lt y,iti scu rci altom onit oreachpoli cydom an i h tat makesupt hew hol e. Ae ral l,wec annotimprovew hatisnot measured.Toh ep l o gvernments ’ d ienf ybes- tprcc a epolc ie is,h te Wo l rdBankcreae tda r famew or k ob t enh c- markeducao nsyse tmp ol ii cesacr ossthew orl d.Theframew oki r scal ledSABER: y Sse tm As sessmentandBenchmarkn ig o frEduao c n Res u t l. sSABERam i sto d a dtoourm os ty l OECD- e cntrc istoe r ofn kow e ldgeon whatworksi n d e uao c n bygt ahern ig andanalz yingm l eyand i rgoo rusdatafo rmt hedeveo lpingword l. h T s istudypree sntsthefirstpl o i tappi c lao n ofSABERin14 economiesn i Eat s As ia—a rego in whereimprovn igtheq uai ly t ofeducao nrem ai nsthek eychal lenge.I t oul tn ies h teS ABER r famew ok r f oreighteduao c n poi ly c domain, saffordingcros s-counr tycom par io sn sn i o c mbi- nao nw t ihe vi denceo nw hi chp o rgramsw ork.U ndereachd omain,thelats et empii rcal v eidenceonh owthese poi c lis e andpracces re latetoeducao nq uai ly t i s pe rsented,ao lng wt ihthe n idia ctorsusedtom eau sre t hem .Therepot r conc ludesthatthekeytoa succes sful d e uao c ns ys temi snot j ustsucce ss na i nyoned om an i, butas loa no v eral l ai lg n ment of poi c liesinm anya specso t fthee ducao ns y stem. Them i anm essagesarea sfo l o lw s: Poi ly c ma er s.Notu srpr ii snl gyh ,ih g-n icomecounr ti es n oe hae v g oodeduao c npoi c lis e; howeve, r i ncertanp i olc iydomain, scera tindeveo lpi ngcounr ti essomem s e dobeer.Ths i findn ig s uge gsts thate e vn c ashs -tr apped counr te is can cath u c p w i th ther i i rcherper esby o fcusn igonbes- tpr acc epolc iy. h T ee r r aen h osot r-u ct. sAl ldev elopn igcounr te ishav e o t m i po rveeduao c npoi c lyn i a numberofares a.Wh l ieits i r tuethatdee vlopn i o gcuntr iesns i o meae rashav eadvane cd poi c lie si np lc ae,theyn ever thee lssneedt ofocuso ns v eeralae rasa tonet c o makethei rss ytemsunf iom r l ysr tong. h T eresut lsofh teE astAi saSABERp i lotwerepresn e e tda tajon itcone freneo c ft heU nie td Nao nsEducao na, lSce infi c,andC ut lua rlO g rani zao na ndtheWo r ldBankinB al i,u Jne 2011.Theposiverespone s o t t he A S BERtoob ly educ ao n pa rc o ner sfromtherego in has n e u sredthath tisproga rmw il lbecome apermanentsev ric e offee rdby h te Word lB a n , kdedi- a cted t o m onio tring p oi c lie s a nd i m pr ovn ig e du c ao n s yt sem p ero frm ane c c arossthedev elopn ig word l. CHAPTER 1 Struggling with Quality Kevin Macdonald SUMMARY Low- and middle-income economies in East Asia have made considerable progress toward achieving universal primary education, but quality still remains a challenge. Increasing per pupil public spending rarely corresponds to increasing quality: the education policies and practices that accompany spending are crucial. Education for All East Asia’s middle- and low- income economies are close to achieving the Education for 1 All (EFA) universal primary education goal, which is also a Millennium Development Goal. As shown in figure 1.1, all economies for which data are available have net primary enrollment rates of 89 percent or higher, with the exception of the Lao People’s Democratic Republic, at 82 percent. Net secondary enrollment rates stand at just under 70 percent for all middle-income economies except the Philippines, which is at 61 percent, while low-income economies have secondary enrollment rates of around 35 percent. Mongolia has the highest net secondary enrollment rate among middle-income economies, and among all economies in the region for which data exist, Mongolia ranks below only Japan and the Republic of Korea. Figure 1.1. Primary and Secondary Enrollment Rates by Economy 100 99 98 96 95 92 89 82 89 89 82 75 Net primary 72 enrollment 68 68 rate (percent) 61 Net secondary 34 36 enrollment rate (percent) Korea Malaysia Philippines SAR, China Mongolia Indonesia Cambodia Japan Thailand Hong Kong Lao PDR 2 Sources: UNESCO Institute of Statistics and World Bank EdStats. Note: No data for China, Singapore, or Vietnam. However, if enrollment rates counted only those students who were actually learning, the picture would change dramatically. Figure 1.2 presents the gross enrollment rate of grade 8 students—the number of grade 8 students in a country divided by the number of 14 year olds—and shows very high figures for all middle-income countries. However, when students who, according to the International Association for the Evaluation of Educational Achievement’s Trends in Mathematics and Science Study (TIMSS), do not have “some knowledge of whole numbers, operations, and basic graphs” are excluded from the calculation, the gross enrollment drops dramatically for Malaysia, Thailand, Indonesia, and the Philippines, all middle-income TIMSS participants. If we raise the standard even higher by excluding those who cannot “apply basic mathematical skills in straightforward situations,” the number drops to less than half of all 14 year olds in those four countries and even as low as 13 percent in the Philippines. TIMSS is typically conducted during grade 8, so it measures the knowledge and cognitive skills that children have acquired over the first seven years of their schooling. For this reason, it provides one measure of the quality of primary schooling. Cognitive ability is crucial to a person’s labor market opportunities and potential earnings (Patrinos and Sakellariou 2011; Moll 1998; Glewwe 1996; Murnane, Willett, and Levy 1995). A study of Canadians using data from the International Adult Literacy Survey found that a 20 percent increase in literacy among adults increases annual earnings by 35.6 percent, whereas the return on years of schooling after controlling for literacy is either zero or very small (Green and Riddell 2001). Figure 1.2. Gross Enrollment Adjusted for Education Quality Cambodia 58 China 101 Hong Kong SAR, China 91 101 107 Indonesia 17 44 92 Japan 87 97 100 Korea 93 102 104 3 Lao PDR 45 Malaysia 45 75 91 Mongolia 106 Philippines 13 36 91 Thailand 35 69 104 Vietnam 86 Sources: United Nations Educational, Scientific, and Cultural Organization (UNESCO) Institute of Statistics, World Bank Edstats, and Mullis, Martin, and Foy 2008. Grade 8 students Grade 8 students Grade 8 students enrolled as a enrolled with a basic enrolled who can apply percentage of all 14 knowledge of whole basic mathematical year olds numbers, decimals, knowledge in operations, and basic straightforward graphs as a percentage situations as a of all 14 year olds percentage of all 14 year olds Recent research is also establishing the importance of learning for economic growth and poverty reduction. Hanushek and Kimko (2000), as well as subsequent studies based on their approach (including Hanushek and Woessmann 2007 and OECD 2010), have shown a strong association between economic growth and education quality. The Organisation for Economic Co-operation and Development (OECD) has estimated that if its member countries were to boost their scores on its student assessment, the Programme for International Student Assessment, or PISA, by 0.25 standard deviations in the next 20 years (less than what Poland accomplished in 6 years), the net present value of benefits for the generation born in 2010 would be US$115 trillion (OECD 2010). Since 1999, TIMSS math scores have been far from static in East Asia, as is demonstrated in figure 1.3. Of all the TIMSS participants in the region, the Philippines increased education quality to a far greater extent than its neighbors—improving its math performance by almost 10 percent between 1999 and 2007—the largest percentage increase of all TIMSS economies over that period. Very small positive changes occurred in Korea and Indonesia, while there were small negative changes in Japan, Hong Kong SAR, China, and Singapore and more noticeable declines in Thailand and Malaysia, where TIMSS scores decreased by 2.8 and 4.4 percent, respectively. Figure 1.3. Average Growth in TIMSS Math Scores from 1999 to 2007 Average percentage change in 4 TIMSS math scores, 1999–2007 10 9.6 5 0.8 0.3 0 -0.9 -0.9 -0.8 -2.8 -5 -4.4 Malaysia Thailand Singapore Hong Kong Japan Indonesia Korea Philippines SAR, China SAR Sources: Mullis, Martin, and Foy 2008 and World Bank EdStats. Education Expenditure and Quality One obvious choice that policy makers have for improving education quality and subsequently learning is increasing public expenditure. But does spending more on education increase its quality? Figure 1.4 presents public expenditures on primary, secondary, and tertiary recurrent spending and non- allocated recurrent and non-recurrent spending as a percentage of gross domestic product (GDP) in the economies of East Asia. Vietnam stands out as the biggest spender of public funds on education overall with 5.3 percent of GDP. Indonesia spends the most public money on primary education with 1.8 percent of GDP, followed by Thailand with 1.4 percent. Korea is the largest spender on secondary schooling with 1.7 percent of GDP, followed by Mongolia with 1.6 percent, which also has the highest net secondary enrollment rate of all East Asian middle- and low-income economies. In terms of tertiary education, Malaysia is the largest public spender, with 1.4 percent of GDP, followed by Mongolia and Singapore with 0.9 percent each. Cambodia spends the least public money on education overall with only 1.6 percent of GDP, followed by Lao PDR at 2.3 percent, which ties with Singapore in spending the least public money on primary schooling at 0.5 percent of GDP. 6 Figure 1.4. Public Expenditure by Level 5 5.3 1.2 1.3 5 4 1.3 1.4 1.6 3.5 1.8 1.0 3 0.6 1.7 0.5 1.1 1.3 0.5 1.2 1.4 2 0.6 0.9 0.3 0.2 1.0 1.1 0.9 0.8 1.4 0.6 1.4 1 0.5 0.5 1.0 0.3 0.7 0.8 0.3 0.8 0.8 0.2 0.1 0.4 0.4 0 0.2 Singapore Vietnam Philippines Korea Hong Kong Indonesia Malaysia Mongolia Cambodia SAR, China Lao PDR Japan Thailand Sources: UNESCO Institute of Statistics and World Bank EdStats. Current public Current public Current public Current public expenditure, primary expenditure, secondary expenditure, tertiary expenditure, (percent of GDP) (percent of GDP) (percent of GDP) unallocated or not current (percent of GDP) Higher per student spending does not always lead to better education quality, however. Figure 1.5 plots TIMSS mathematics achievement and average annual per pupil spending—in purchasing power parity (PPP)–adjusted international currency—for primary schooling. The figure suggests a natural frontier that indicates which economies achieve the highest scores for their per student spending. Some East Asian economies are close to this frontier, suggesting that they are getting relatively good returns on their education investments. Others are clustered within the frontier, suggesting that they could spend less to achieve the same results. Figure 1.5. TIMSS Math Achievement and Expenditure per Primary Student 600 Korea Hong Kong SAR, China TIMSS 2007 mathematics score Japan 500 Malaysia Thailand Indonesia 6 400 Philippines 300 125 500 2,000 8,000 Public expenditure per pupil, primary (PPP constant 2005) Sources: UNESCO Institute of Statistics, World Bank EdStats, and Mullis, Martin, and Foy 2008. Note: Public expenditure per pupil, primary, is the average from 1999 to 2005 (that is, average per pupil spending while the tested cohort was in primary school). Data for the Philippines are from 2003. Increasing spending does not necessarily mean that education quality will increase, especially in the medium term. Figure 1.6 plots the growth in TIMSS mathematics achievement and the growth in per pupil primary education expenditure for the tested cohorts of students. Most of the economies in the region fall into the bottom right quadrant of the figure, indicating that although their per pupil primary education spending has increased, their TIMSS performance has declined—even though increases in spending in these economies have been quite large. Four out of the six East Asian economies with time- series data on spending that participate in TIMSS fall into this category. The exceptions are Korea and the Philippines, where there have been both high increases in spending and positive growth in TIMSS achievement. Figure 1.6. Change in Spending and Quality from TIMSS 1999 to 2007 10 Philippines Average change in TIMSS grade 8 mathematics (percent) Increase in TIMSS Score Increase in TIMSS Score Decrease in Per Pupil Spending Increase in Per Pupil Spending 5 Korea 0 Hong Kong SAR, China Japan -50 -25 0 25 50 Thailand Malaysia -5 Decrease in TIMSS Score Decrease in TIMSS Score Decrease in Per Pupil Spending Increase in Per Pupil Spending 7 -10 Growth in per student expenditure at the primary level for TIMSS cohorts Sources: UNESCO Institute of Statistics, World Bank EdStats, and Mullis, Martin, and Foy 2008. Note: Average change in per pupil primary public expenditure between TIMSS cohorts is the 1999 to 2007 average change from one TIMSS cohort to the next in the average per pupil public primary expenditure that occurred during grades 1 to 6 for the cohort. How Can Policy Makers Improve Quality? Spending alone has not necessarily improved education quality in East Asia. In fact, in many economies it has coincided with a decrease in quality. In other words, the policies and practices that accompany spending are crucial. But which policies and practices? The experience of other countries and the existing evidence base provide policy makers with some guidance. The purpose of this study is to help provide policy makers with guidance on how to improve education quality by benchmarking their policies and practices to that of other countries and to the evidence base on what works in education. References Glewwe, P. 1996. “The Relevance of Standard Estimates of Rates of Return to Schooling for Education Policy: A Critical Assessment.” Journal of Development Economics 51 (2): 267–90. Green, D. A., and W. C. Riddell. 2001. Literacy, Numeracy, and Labour Market Outcomes in Canada. Ottawa: Statistics Canada. Hanushek, E. A., and D. D. Kimko. 2000. “Schooling, Labor-Force Quality, and the Growth of Nations.” American Economic Review 90 (5): 1184–208. Hanushek, E. A., and L. Woessmann. 2007. “The Role of Education Quality for Economic Growth.” Policy Research Working Paper 4122, World Bank, Washington, DC. Moll, P. G. 1998. “Primary Schooling, Cognitive Skills, and Wages in South Africa.” Economica 65 (258): 263–84. Mullis, I. V. S., M. O. Martin, and P. Foy. 2008. TIMSS 2007 International Mathematics Report. Chestnut Hill, MA: TIMSS & PIRLS International Study Center. Murnane, R. J., J. B. Willett, and F. Levy. 1995. “The Growing Importance of Cognitive Skills in Wage Determination.” Review of Economics and Statistics 77 (2): 251–66. OECD (Organisation for Economic Co-operation and Development). 2010. The High Cost of Low 8 Educational Performance: The Long-Run Impact of Improving PISA Outcomes. Paris: OECD. Patrinos, H. A., and C. Sakellariou. 2011. “Quality of Schooling, Returns to Schooling, and the 1981 Vouchers Reform in Chile.” World Development 39 (12): 2245–56. CHAPTER 2 Learning from High-Performing Education Systems Kevin Macdonald and Harry Anthony Patrinos SUMMARY I n the previous chapter, we saw that improving education quality is not just a matter of additional resources; it also requires achieving the right policy mix. This is often difficult since policies and practices fit together to form the education system as a whole: if just one component is weak, it can undermine the whole system. The system may also fail if policy components are misaligned. This chapter examines the interaction between different policy areas in East Asia in an effort to understand the effectiveness of education system policies and practices as a whole. It benchmarks a few key components of the education system, as well as the alignment of these components. Generally, we see that higher-performing economies in international assessments tend to have fewer misalignments between key components than do lower-performing economies. 9 A Systems Approach Policy makers grappling with the challenge of improving education quality face a difficult task because of the lack of evidence on what works. While there is a growing evidence base on specific interventions that can improve learning and other education outcomes, there is less evidence on how these policies fit together to improve quality. Benchmarking education systems can fill this knowledge gap by allowing governments to compare their policies and find out what is working on the ground. One obstacle to doing this is that the education system itself is large and complex, and policies and practices cannot be pursued in isolation: having excellent policies in every component of the education system except for one, on which the other components rely, could lead to learning failure. The challenge is to find a system that is complete enough to not exclude crucial components but simple enough to provide clear policy guidance. One approach is to use the concept of cognitive production functions. Cognitive production functions are used extensively in the academic literature to model an education system. They treat the production of cognitive achievement analogously to that of a firm: various inputs are combined according to some type of production technology to create cognitive ability. The inputs of a cognitive production function include factors that could affect a child’s learning from the prenatal stage to the present. These factors could include pedagogical materials found in the home, support from parents and teachers, pedagogy at school, motivation, innate ability, and other inputs. The production function is a model of how these separate inputs combine to create learning. How the production function is modeled is crucial to identifying what distinguishes a high-performing education system. One insight from economics (Kremer 1993) is that production tasks must be done together for any of them to be of high value. The O-ring theory of economic development uses the metaphor of the space shuttle Challenger disaster in 1986, which was caused by the malfunctioning of a tiny component of the spaceship—the O-ring. This example illustrates the crucial concept of interdependence between components. Benchmarking Education Systems in East Asia When applied to education, the O-ring theory implies that even if most of the components of a system are excellent, if a single policy on which the success of the other components depends is deficient, then the system as a whole will fail. In the model underlying this chapter, teachers are the most important input into the education system, because they are the point of contact between the education system and the students. If an education system is unable to attract highly talented individuals into teaching, then the effectiveness of the system as a whole will likely be compromised regardless of the technical strength of the other components. Within the education system’s production function, this chapter focuses on the ability of schools to employ and retain good teachers—in other words, the degree of school autonomy and the decentralization of the hiring process. In particular, this chapter benchmarks the autonomy of schools, 10 student assessment information, and influence by the parents on school processes and considers how these factors interact to shape teacher hiring policy. Improving Teacher Policies To provide East Asian policy makers with guidance on how to improve education quality, one finds it useful to look at the policies involving the teaching profession that are in need of improvement. The success or failure of these policies also depends on the relative autonomy of schools and whether the complementary information and accountability mechanisms exist to ensure learning outcomes. Across the region, there are good levels of development in most of the teacher policy goals. However, there is room for improvement. In particular, policies need to be further developed to help (a) prepare teachers through useful training and experience and (b) empower school principals to lead teachers. The Role of Decentralization The decentralization of an organization is a distinguishing factor of any production process, especially an education system. However, moving the locus of decision making away from central government to the school level, in turn, increases the importance of information systems: with added responsibility comes the attendant need for greater accountability. Alignment between the degree of autonomy and the existence of information is crucial for decentralization to be effective. Table 2.1 measures the interaction of these two policies: the extent to which autonomous schools are able to manage their own budget or hire and fire teachers, and the existence of large-scale assessment mechanisms that hold the school or teachers accountable for learning outcomes. We note that the Republic of Korea, Malaysia, Thailand, and Vietnam allow school directors to manage their own budgets, but they also have a census-based assessment to hold schools accountable. In contrast, Cambodia and the Lao People’s Democratic Republic provide schools with the same level of financial autonomy but do not have an assessment system that is officially designed for school results. This finding suggests a misalignment. Indonesia, Mongolia, and Shanghai, China decentralize even further and allow schools to hire and fire teachers, but none of these economies has an assessment that holds teachers accountable. For these five cases, therefore, there is an apparent misalignment between the degree of autonomy and the information required to hold schools accountable. Other economies, such as China, Japan, the Philippines, and Singapore, have a more centralized system, so the issue of information is less applicable. Table 2.1. Is Student Assessment Information Aligned with Autonomy? − No autonomy School has School has 11 over budget or autonomy to autonomy to hiring and firing allocate its hire and fire teachers budget teachers A census-based assessment with an  Malaysia official purpose of teacher and school  Thailand accountability exists A census-based assessment with an  Vietnam  Indonesia official purpose of school accountability  Korea exists No census-based  Singapore assessment with an official purpose of  China  Lao PDR  Shanghai either school or  Philippines  Cambodia  Mongolia teacher accountability exists  Japan Source: Authors’ compilation Another type of decentralization is private sector education provision. Table 2.2 presents the extent to which public funding supports private schools and whether the corresponding information exists to hold those schools accountable. For example, Malaysia and Thailand provide public funding that follows students to the schools of their choice (for example, a voucher scheme). These systems also have a census-based assessment with an official purpose of providing information about school choice. Indonesia and Korea and the Philippines provide public funding to private schools and have an assessment system that is designed to hold schools accountable. China, Japan, and Shanghai, China, also provide public funding to schools but do not have such an assessment, suggesting a misalignment. Cambodia, Lao PDR, Mongolia, Singapore and Vietnam do not publicly fund private schools. Table 2.2. Is Student Assessment Aligned with Public Financing of Private Schools? No publically There exist Public funding funded private publically follows the schools funded private student to a schools chosen school A census-based assessment with an  Malaysia official purpose of promoting competition,  Thailand 12 orienting demand, school choice A census-based assessment with an  Indonesia official purpose of school accountability  Vietnam  Korea exisits  Philippines No census-based  Singapore  Shanghai assessment with an  Lao PDR official purpose of either  China school or teacher  Cambodia accountability exists  Mongolia  Japan Source: Authors’ compilation The Role of Parents in Ensuring Quality Education Decentralizing an education system weakens the influence of the central authority. To offset this problem, the influence of clients—particularly parents—is important. Table 2.3 presents the degree of school autonomy as compared to the influence of parents. For example, Shanghai, China, allows school directors to hire and fire teachers, and it also gives parents a degree of influence over this process. By contrast in Indonesia and Mongolia, where schools are also able to hire and fire teachers, parents have little influence. Cambodia, Korea, Lao PDR, Malaysia, and Thailand all allow school directors to manage their own budgets, and they also afford a degree of parental influence. In Vietnam, parents do not have such influence even though schools do. The remaining economies do not provide schools with either of these types of autonomy. Table 2.3. Are Accountability Policies Aligned with Autonomy? No autonomy School has over budget or School has autonomy to autonomy to hire hiring and firing allocate its budget teachers and fire teachers Parents have some influence over hiring and firing  Shanghai teachers Parents have some  Malaysia influence over  Thailand budget decisions  Lao PDR  Korea 13  Cambodia − Parents do not have  Singapore influence over  Philippines  Indonesia  Vietnam specific aspects of  China  Mongolia school autonomy  Japan Source: Authors’ compilation What Characterizes a High-Performing Education System in East Asia? Many economies in East Asia have well-developed teacher policies, as well as strong decentralization, assessment systems, or parental involvement; however, few economies have all these components properly aligned. In general, higher-performing economies tend to have better-aligned policies. For example, Japan, Korea, Malaysia, Singapore, and Thailand have relatively well-developed teacher policies and have no more than one misalignment between decentralization and information or parental influence. The other economies have many misalignments in a number of areas. The economies of East Asia are in a privileged position given that many of the best-performing education systems are located in the region and several developing economies in the region excel in specific policy domains. Moreover, all of these economies share some challenges. There is a clear need to focus on improving several policy areas at the same time in order to further holistic education reforms. It is not sufficient to excel in single policy domains. Different policy domains must be linked and reinforced through the following: 1. Accountability mechanisms—rewards and sanctions 2. A solid vision of where the system is headed—good stewardship 3. Feedback loops so that developments and lessons in one policy area are fed into and inform others Without clear rewards and sanctions, good system stewardship, and clear feedback loops, there is a danger that a country will develop efficient “islands” of activity while remaining trapped in a low- achievement environment. Toward Leading Indicators While gainful learning is one of the objectives of an education system, and although comparing and benchmarking country systems is a useful exercise, there is still a need for a subset of indicators. Moreover, these indicators should be useful for signaling change and furthering national education goals. In other words, they should be key or leading indicators. The leading indicators analogy is borrowed from the concept of economic indicators that signal shifts 14 before an economy has actually changed. Examples of leading indicators include production workweek, building permits, unemployment insurance claims, money supply, inventory changes, and stock prices. There are also coincident indicators, which change about the same time as the overall economy, and lagging indicators, which change after the overall economy, but these indicators are of minimal use as predictive tools. While the leading indicator concept has been used in other sectors, notably health, in education the only example that is well known is the Annenberg Institute’s approach, which works with central offices and community constituencies of local school system in the United States. Since improving outcomes and closing gaps takes time, policy makers need to assess systems and see progress before results show up in indicators such as test scores. Therefore, leading indicators enable education leaders to make more strategic and less reactive decisions. But leading indicators need to be timely and actionable; must be benchmarked to external standards; and, if changed, need to have the ability to improve outcomes. What is proposed here is an approach that attempts to use the concept of leading indicators to gauge national system progress. Leading indicators in education should provide early signs of progress toward academic achievement and will help governments make informed decisions about improving the system. In the future, the System Assessment and Benchmarking Education for Results (SABER) findings will be used to construct a system of leading indicators. References Kremer, M. 1993. “The O-Ring Theory of Economic Development.” Quarterly Journal of Economics 108 (3): 551–75. 15 16 CHAPTER 3 Information Systems Emilio Porta “Knowledge is power, and information is liberation.” Kofi Annan, Former United Nations Secretary-General SUMMARY The development of good education management information systems (EMISs) is an essential part of the effort to improve education quality. Governments must be able to gauge system capacity and produce and disseminate reliable education statistics to make informed policy decisions. The results of the SABER (System Assessment and Benchmarking Education for Results) pilot in East Asia show that overall the rate of available data in the region is higher than the world average, although there is substantial variance across economies, indicating opportunities for peer learning. There is still room for improvement, and EMISs across the region can often be improved by filling data gaps—particularly for education finance and higher education—and by focusing on the timely and widespread dissemination of results at the international, national, and local levels. 17 The Need for Reliable Data for School Systems The production and dissemination of reliable education statistics is an important global public good. Reliable data are essential for effective education sector planning and for monitoring progress toward meeting national and global education targets such as the Millennium Development Goals (MDGs). To evaluate the education sector in any country, analysts need to have access to performance indicators. They need to know about the education system’s internal efficiency, such as intake and completion rates, repetition and dropout rates, and student-teacher ratios. They also need to know about the context in which the system operates—such as educational expenditures by student and education level or private educational expenditures by households—to evaluate the potential winners and losers of changes in education policy. Data can also be used to strengthen school and policy-maker accountability and to promote better learning outcomes. A recent study by Bruns, Filmer, and Patrinos (2011) highlights how data have been used to change accountability relationships among the various actors in the education system to promote changes in behavior that improve outcomes. The majority of education projects funded by the World Bank over the past decade have included components for strengthening EMIS or school-mapping activities. Establishing a robust EMIS is no easy task, and a considerable number of economies still struggle to gather data on such essential indicators as the education MDGs (Porta and Klein 2010). As seen in figure 3.1, in 2007 just 61 percent of economies worldwide were able to report on their progress toward meeting the education MDGs. Although the situation in East Asia is slightly better, a considerable gap in the availability of information remains. Figure 3.1. Data Availability over Time for Education MDG Indicators 18 East Asia World Source: World Bank estimates. Data Quality Assessment Framework The International Monetary Fund (IMF) has developed a Data Quality Assessment Framework (DQAF) that can be applied to statistics in a range of different subject areas. The World Bank, in collaboration with the United Nation Educational, Scientific, and Cultural Organization (UNESCO) Institute for Statistics (UIS), used the DQAF to provide a flexible structure for the qualitative assessment of education statistics. The DQAF covers all aspects of the statistical environment in which the data are collected, processed, and disseminated. It is possible to not only assess the quality of educational statistics, but also identify those areas in need of strengthening. Using these data, the World Bank and UIS are working with governments to develop policies to build their capacity to collect useful and robust education statistics. (For a short history of the development of the DQAF and the IMF’s efforts to improve data quality, see Carson [1997]. The annotated DQAF used in this manual is based on the framework used by the World Bank and the UNESCO Institute for Statistics [Patel, Hiraga, and Wang 2007; Patel et al. 2003], which is an adaptation of the original framework developed by the IMF.) The DQAF assessment has a series of questions organized from the general to the specific, along six key dimensions: 1. Prerequisites of quality. This dimension measures the legal and institutional environment, including the extent of coordination between the Ministry of Education and other line ministries and departments, the adequacy of resources available for statistical work, and the overall awareness of the need for high-quality data for use in government. 2. Integrity. This dimension measures factors that ensure objectivity in the collection, compilation, and dissemination of statistics. It includes the institutional arrangements for ensuring professionalism in statistical policies and practices, transparency, and ethical standards. 3. Methodological soundness. This dimension measures the extent to which country EMISs use established methodologies and internationally accepted standards, guidelines, or good practices. Four specific elements are examined: concepts and definitions, scope, classification, and basis for recording. 4. Accuracy and reliability. This dimension measures the extent to which EMIS data accurately reflect reality on the ground and the ability of third-party sources—such as teachers—to collect data on the basis of sound statistical methods. It also measures the regularity of assessment, validation, and revision of source data, intermediate data, and statistical outputs. This dimension has five elements: 19 source data, statistical techniques, assessment and validation of source data, assessment and validation of intermediate data and statistical outputs, and revision studies. 5. Serviceability. This dimension measures factors that ensure education statistics are useful and relevant to users, such as timely and predictable dissemination, internal and external consistency with other major datasets, and formal revision policies. This dimension has four elements: relevance, timeliness and periodicity, consistency, and revision policy and practice. 6. Accessibility. This dimension measures the extent to which data and metadata are presented in a comprehensible manner, are easily accessible to all, and are regularly updated, as well as the extent to which user support services are readily available. This dimension has three elements: data accessibility, metadata accessibility, assistance to users, and leading indicators for benchmarking EMISs. Box 3.1 provides a case study of DQAF implementation. Box 3.1. The Implementation of the DQAF in the Lao People’s Democratic Republic The government of Lao PDR and the donor community have been working together toward the implementation of a sectorwide approach to harmonize their development assistance. Part of this effort involves strengthening the EMIS of the Ministry of Education. The government used the DQAF to identify those areas that needed strengthening and to put together a detailed action plan for implementing a fully functional EMIS by 2013. Prerequisites of Quality 1 0.75 0.5 Assurance of Accessibility 0.25 Integrity 0 20 Accuracy and Serviceability 2006 Reliability 2010 2013 Source: World Bank staff. Past experience has shown that it takes governments about two months to conduct a comprehensive data quality assessment, and since the changes take time to implement, it is not feasible to perform a DQAF annually. Therefore, to facilitate the benchmarking and monitoring of the EMIS on a regular basis, the World Bank has developed the SABER-EMIS Assessment Tool (SEAT) as part of the SABER initiative (Porta and Arcia 2011). The SEAT benefited from much contribution from UIS’s Assessment, Information Systems, Monitoring, and Statistics Unit in Bangkok. The assessment scale makes it possible to carry out a rapid assessment of the EMIS and provides clear scoring rules for evaluating data quality. The World Bank also recommends using the Leading Indicators of Data Quality, a simple, low-cost tool for the regular monitoring of progress in the development of an EMIS. This assessment is similar to use of leading economic indicators and can be used to monitor data quality between full assessment exercises. (For more information about SABER EMIS tools, visit http://go.worldbank.org/PX7O96H8F0.) Two of these indicators have been piloted in East Asia:  The percentage of UIS indicators that can be estimated with the data provided by the country. This indicator measures a country’s capacity to report key education indicators to the UIS. It consists of the ratio of the total number of UIS indicators that can be estimated with data provided by the country to the total number of UIS indicators. The UIS publishes indicators only for economies that report their data following international standards. Hence, this indicator also captures the quality of the reported data. 21  Feedback of information to schools and communities to promote accountability and to facilitate school-level planning. This indicator measures if economies report information back to schools. Giving statistical information to schools is very important because (a) it helps schools to see why their data are important for the creation of countrywide education statistics, (b) it gives schools information that they can use to compare themselves to national and regional standards, and (c) it gives schools information that they can use to increase their accountability to local communities. Analysis of Leading Indicators for East Asia We analyzed these two leading indicators using data collected through the UIS for the following economies: Cambodia; China; Hong Kong SAR, China; Indonesia; Japan; the Republic of Korea; Lao PDR; Malaysia; the Philippines; Singapore; Thailand; and Vietnam. Overall, the rate of available data in East Asia—defined as the ratio of the total number of indicators provided by East Asian economies to the total number of UIS indicators—is higher than the world average. As seen in table 3.1, the largest remaining information gaps are in the areas of education finance and International Standard Classification of Education (ISCED) 5–6 (higher education). It is important to note that this indicator is highly variable and that it has been declining since 2007. Table 3.1. Percentage of UIS Indicators That Can Be Estimated with Data Provided by the Country Education Higher education Education finance (ISCED 0–4) ISCED (5–6) 2007 2008 2009 2010 2007 2008 2009 2010 2007 2008 2009 2010 Cambodia 19 32 32 19 67 100 76 76 31 39 70 39 China 51 51 51 51 89 89 82 0 39 39 39 0 Hong Kong 38 41 41 68 85 85 93 93 96 96 96 37 SAR, China Indonesia 76 5 5 59 100 91 100 96 31 6 98 54 Japan 76 76 76 76 80 80 76 78 91 91 89 89 Korea, 68 68 65 84 100 100 100 100 100 100 100 100 Rep. Lao PDR 57 11 19 11 100 100 100 100 100 98 61 61 Malaysia 51 51 24 30 100 100 65 98 78 83 0 98 Philippines 73 59 0 84 100 100 100 82 67 63 63 41 Singapore 22 22 49 49 62 69 82 82 4 39 39 22 Thailand 32 16 16 41 84 93 89 62 28 24 46 43 Vietnam 14 16 91 91 58 58 81 81 91 74 AVERAGE 51 39 33 49 88 92 85 77 68 60 66 56 Source: UNESCO Institute of Statistics The biggest challenge economies face in terms of providing data to the UIS is in the area of education finance. In some economies, the minister of education is required to request the information from the minister of finance, which can cause delays and sometimes even noncompliance. Therefore, it is important to include arrangements for exchanging data as one of the topics in the DQAF. In fact, the availability of data on education finance could also be used as a leading indicator for assessing if these mechanisms are in place. As seen in figure 3.2, in 2010 three economies had a full range of data on education finance: Japan, Korea, and the Philippines. Korea and the Philippines made significant strides toward making data available in 2010, even moving ahead of Japan, which was the leader until 2008. The three economies with the least data on education finance were Cambodia, Lao PDR, and Vietnam. Also note that Lao PDR had a significant decrease in the availability of data between 2007 and 2010. Figure 3.2. Rate of Available Education Finance Data by Country estimated with data provided by country 90 Percentage of UIS indicators that can be 80 70 60 50 40 30 20 10 0 Philippines Malaysia Indonesia Singapore Cambodia Japan Thailand Korea Vietmam China Hong Kong SAR, China Lao PDR 23 2007 2008 2009 2010 Source: UNESCO Institute of Statistics. Providing information to schools is important for promoting accountability and enabling educational planning at the local level (Bruns, Filmer and Patrinos 2011; Arcia et al. 2011; Crouch 1997). Some governments are starting to send information back to schools so that students, parents, and teachers can find out how well their schools are performing compared with similar schools. Feeding information back to schools can improve data quality because the school community can identify and report any inaccuracies. Of the economies included in this analysis, Hong Kong SAR, China; the Philippines; and Singapore send information directly to parents, and Indonesia, Korea, Malaysia, and Vietnam send information to schools. Our analysis of the two indicators suggests that there is room for economies in the region to learn from each other about the process of developing EMISs. Some performed better than others by providing more complete data, and others did so by having better mechanisms for disseminating the data to parents and schools. Table 3.2. SABER EMIS Indicators Summary Percentage of UIS Indicators that can be Feeding estimated with data provided by the country information back (2011) to Education Preschool, finance primary, and Tertiary Schools Parents secondary Latent Advanced Emerging Cambodia    Established Latent Latent China    Hong Kong Established Advanced Emerging SAR, China      Established Advanced Established Indonesia     Advanced Advanced Advanced Japan    Korea, Advanced Advanced Advanced Rep.     24 Latent Advanced Established Lao PDR    Emerging Advanced Advanced Malaysia     Advanced Advanced Emerging Philippines      Emerging Advanced Emerging Singapore      Emerging Established Emerging Thailand    Latent Established Established Vietnam     Latent Emerging Established Advanced  = 0–25%  = 26–50%  = 51–75%  = 76–100% For example, Japan has high levels of data availability but is not feeding back this information to schools and parents. In contrast, although Vietnam has some challenges on the availability of data, the country is doing a good job of feeding information back into the system at the school level. Economies such as Hong Kong SAR, China; the Philippines; and Singapore both have high levels of data availability and are also feeding back information to schools and parents. Conclusion Often the development of EMISs can be left as an afterthought, given other pressing needs such as training and hiring good teachers or updating curricula to meet real-world needs. An EMIS can seem like a chore—an onerous undertaking that requires extra resources and expertise. However, a well-functioning EMIS is essential for any top-quality education system; it is the oil that greases the whole machine. An EMIS provides essential information to policy makers that allows them to take the pulse of the education system as a whole, to discern how existing policies and procedures are working, and to decide which elements need to be improved. Currently, East Asian economies are among the leaders in terms of the rate of available data, although there is substantial variance across economies. Hence, those economies that need to catch up can learn from the successful policies of their near-neighbors. Often governments collect the right kind of data but are less successful at dissemination. As transparency in education systems improves across the region, the quality of education will certainly improve as a consequence. 25 References Arcia, G., H. A. Patrinos, E. Porta, and K. Macdonald. 2011. “School Autonomy and Accountability.” Regulatory and Institutional Framework, System Assessment for Benchmarking Education for Results, Human Development Network, World Bank, Washington, DC. Bruns, B., D. Filmer, and H. A. Patrinos. 2011. Making Schools Work: New Evidence on Accountability Reforms. Washington, DC: World Bank. Carson, C. 1997. “Toward a Framework for Assessing Data Quality.” Statistics Department, International Monetary Fund, Washington, DC. Crouch, L. 1997. “Sustainable EMIS: Who Is Accountable?” In From Planning to Action: Government Initiatives for Improving School-Level Practices, ed. D. Chapman, L. Mählck, and A. Smulders, 211–39. Paris: United Nations Educational, Scientific, and Cultural Organization International Institute for Educational Planning. Patel, S., M. Hiraga, and L. Wang. 2007. “A Framework for Assessing the Quality of Education Statistics.” Development Data Group and Human Development Network, World Bank, Washington, DC, and United Nations Educational, Scientific, and Cultural Organization Institute of Statistics, Montreal. Patel, S., M. Hiraga, L. Wang, D. Drew, and D. Lynd. 2003. “A Framework for Assessing the Quality of Education Statistics.” Development Data Group and Human Development Network, World Bank , Washington, DC. 26 Porta, E., and G. Arcia. 2011. “Information Systems for Planning and Policy Dialogue: Assessing the Quality of Education Statistics.” Human Development Network, World Bank, Washington, DC. Porta, E., and J. Klein. 2010. “Increasing Education Data Availability for Knowledge Generation.” Background Paper for the Education Sector Strategy 2020, World Bank, Washington, DC. CHAPTER 4 Marguerite Clarke, Student Assessments Julia Liberman, and Maria-Jose Ramirez SUMMARY A ssessment is one of the least expensive innovations in education and a vital tool to gauge whether children are learning and achieving at school. This chapter provides an overview of the relative strength of student assessment systems in East Asia, focusing on a set of key indicators for which economies in the region provided data. The four main findings are that (a) most economies in the region report having a relatively strong enabling context (for example, formal policies, budgets) for their student assessment activities; (b) most economies report aligning their examinations and (where they exist) national large-scale, system-level assessment activities to their official learning standards or curriculum; (c) while the general quality and stability of international large-scale assessment activity in the region appears good, there is room for improvement in the dissemination and use of results; and (d) many of the lower-income economies in the region do not yet have mechanisms for monitoring or ensuring the quality of classroom- based assessment activities by teachers, something that needs to be addressed if they are to create truly effective student assessment systems that support learning for all. 27 Why Are Student Assessment Systems Important? It is not enough for children to be enrolled in school and sitting in classrooms; they must also be learning. Ensuring Assessment is the process of that children are learning requires strong systems for gathering and evaluating information on what students know, understand, assessing student achievement and learning outcomes. and can do so that an informed decision can be made about what to do Research has shown that the right kinds of assessment next in the educational process. activities and uses of assessment data can contribute to An assessment system is a group better learning outcomes and better-informed policy of policies, practices, and tools for decisions (for a review, see Clarke 2011). Evidence from this generating and using information on research has shown causal links between the following: student learning and achievement. Such systems can support a variety of  High-quality, formative classroom assessment decision-making activities, such as activities and better learning outcomes, as measured designing and evaluating policies, by student performance on standardized tests informing instruction, determining progress, and providing data for  Economies with exit examination policies and higher stakeholder accountability. performance levels on international large-scale assessments of educational achievement, such as the PISA (Programme for International Student Assessment) or TIMSS (Trends in International Mathematics and Science Study)  Use of results from large-scale, system-level assessments of educational achievement to hold schools and educators accountable and subsequent improvements in student learning outcomes Despite this growing evidence base, many governments have yet to establish fully effective student assessment systems. This is especially the case in many lower- and middle-income economies, which stand to benefit most from systematic efforts to measure student learning outcomes—particularly as testing is one of the least expensive innovations in education (Hoxby 2002; Wolff 2007). What Do Student Assessment Systems Look Like? Student assessment systems tend to be made up of three main kinds of activities that correspond to three key purposes or information needs:  Classroom assessments that provide real-time information to teachers and students to support teaching and learning in individual classrooms. These assessments are carried out by teachers and students in the course of daily activities in the classroom. They include a variety of tools and procedures for collecting and interpreting written, oral, and other forms of evidence on student learning and achievement. Research has shown a strong causal link between effective classroom assessment activities and better learning outcomes, as measured by students’ 28 performance on standardized tests, with the largest gains being made by low-achieving students (Black and Wiliam 1998).  Examinations, the results of which are used to make decisions about the progress of individual students through the education system. Examinations provide information to make crucial life- changing decisions about individual students—for example, whether they should be promoted to the next grade level, assigned to a particular type of school or academic program, graduate from high school, or gain admission to university (Heubert and Hauser 1999). The high-stakes nature of most examinations means that policy makers must carefully monitor the process to avoid unintended negative consequences. For example, some examinations can exert a “backwash effect” by encouraging such practices as “teaching to the test” rather than focusing on overall learning. These practices, in turn, affect the skills and knowledge profile of graduates.  Large-scale, system-level assessments, which provide policy- and practitioner-relevant information on the overall performance of the education system and the factors that contribute to that performance (Greaney and Kellaghan 2008). The results of these kinds of assessments can inform both educational policy and practice. Examples include international assessments of educational achievement such as the TIMSS and PISA; regional assessment programs such as the PASEC (Programme d’Analyse des Systèmes Educatifs, or Program on the Analysis of Education Systems) in francophone Africa; national-level assessments, such as ASLO (Assessment of Student Learning Outcomes) in the Lao People’s Democratic Republic; and subnational assessments such as the province- or state-level assessment programs in Canada and the United States. What Are Effective Student Assessment Systems? The effectiveness of a student assessment system depends on the quality of the information that it generates for decision making. The key factors governing the quality of information—and ultimately the effectiveness of the assessment system—are as follows (for a detailed discussion, see Clarke 2011):  Enabling context—the wider context in which the assessment activity takes place and the extent to which it supports the assessment. Enabling context refers to the broad policy framework within which the assessment is carried out, the institutional structures for carrying out the assessment or using the assessment results, the availability of sufficient and stable sources of funding, and the existence of effective human resources. It is important to get the enabling context right because it is a key driver of the long-term effectiveness of an assessment system and because no assessment system is sustainable in its absence.  System alignment—the extent to which the assessment activity is aligned with the rest of the education system. System alignment involves the connection between assessment activities and the education system’s learning goals, standards, and curriculum. Alignment involves more than a 29 simple match between what is tested and what is in the curriculum. For example, while there may be limited correspondence between a given country’s curriculum and what is tested on an international large-scale assessment such as the PISA or TIMSS, the assessment may still be aligned with—and may be useful for informing—the overall goals of the country’s education system. Indeed, the use of data from the TIMSS, PIRLS, and PISA to identify the factors that influence academic performance and to monitor the impact of reforms over time has been key to improving student achievement levels in countries as diverse as Brazil, Jordan, and Poland.  Assessment quality—the technical quality of the instruments and procedures used in the assessment activity. This factor applies not only to large-scale, system-level assessments but to any kind of assessment activity (AERA, APA, and NCME 1999). If an assessment is not sound in terms of its design, implementation, analysis, or use, it can contribute to poor decision making. Two technical issues that are vital for any assessment activity are reliability and validity: (a) does the assessment produce sufficiently precise data, a particularly important feature in high-stakes examinations and for monitoring trends over time, and (b) do the scores represent what they are supposed to and are they used in the intended ways with the intended outcomes? For example, validity might be undermined by a difference between the language of instruction and the language of testing, which would make it difficult for a child to show what he or she knows and can do. It is important in the context of validity to consider the consequences of how test scores are used, including social, economic, and other effects on different population groups. Benchmarking Student Assessment Systems in East Asia Table 4.1, which juxtaposes the three main assessment activities with the three key factors that drive the quality of the information produced by these activities, provides a basis for identifying indicators for reviewing student assessment systems and for planning ways to improve them. For the East Asia SABER (System Assessment and Benchmarking Education for Results) pilot, we selected and then collected data on a few key indicators (indicated in table 4.1) for economies in East Asia according to a combination of criteria, including professional testing standards and empirical research on effective assessment systems (for example, Airasian and Russell 2007; AERA, APA, and NCME 1999; Black and Wiliam 1998; Darling- Hammond and Wentworth 2010; Ferrer 2006; Ravela et al. 2009): Table 4.1. Conceptual Framework for Reviewing Student Assessment Systems Classroom Large-scale, Indicator assessment Examinations system-level assessment A policy framework for assessment activities    Enabling A regular budget for assessment 30 context activities   Training of teachers in assessment techniques    Evidence of an alignment System between assessment activities alignment and official learning standards or   curriculum Regular collection of system- level data on student learning  levels Assessment Timely dissemination of data on quality student learning achievement to  stakeholders Quality assurance of classroom assessment activities  Source: Authors’ compilation To be sure, these indicators do not capture every aspect of what is important in a student assessment system. At the same time, they serve as a useful initial “dipstick” of strengths and weaknesses in such a system that can act as a guide for subsequent, more detailed (and resource-intensive) data collection and review. Because of this focus on a few key indicators, results are not presented in the form of aggregate development-level ratings (that is, latent, emerging, established, advanced). Rather, data for individual indicators are discussed. Tools that allow for more extensive data collection on student assessment systems (including the calculation of development-level ratings) are available on the World Bank’s SABER website. The four main findings from the data collected are that (a) most economies in the East Asia region report having a relatively strong enabling context for their student assessment activities; (b) most economies report aligning their examinations and (where they exist) national large-scale, system-level assessment activities to their official learning standards or curriculum; (c) while the general quality and stability of international large-scale assessment activity in the region appears good, there is room for improvement in the dissemination and use of results; and (d) many of the lower-income economies in the region do not yet have mechanisms for monitoring or ensuring the quality of classroom-based assessment activities by teachers, something that needs to be addressed if they are to create truly effective student assessment systems that support learning for all. Finding 1: Most East Asian Economies Report Having a Relatively Strong Enabling Context for Their Student Assessment Activities 31 An important ingredient to creating a strong student assessment system is a system-level framework for assessment activities—for example, a legal document mandating a particular assessment activity. This type of framework better ensures continuity and stability of assessment activities over time as well as consistency in the uses of results. Most economies in the East Asia region report having a formal policy framework for large-scale, system-level assessment and examination activities (table 4.2). Approximately half of the economies in the region report having such a framework for classroom assessment activities. Another important element for securing the continuity and stability of an assessment system is a regular budget for assessment activities. Nine of the economies surveyed report having a regular budget to fund their participation in international large-scale assessments, typically PISA, TIMSS, or PIRLS (table 4.3). Eleven economies report having a regular budget for national large-scale, system-level assessment activities. The exception to this trend is Mongolia, which has only a short-term budget funded by loans for carrying out international and national large-scale, system-level assessment activities. Most economies also report having a regular budget for examination activities. In some economies (for example, Hong Kong SAR, China; Japan; the Republic of Korea; and Singapore), this budget is supplemented by student examination fees. For effective assessment to take place, especially at the classroom level, teachers must be knowledgeable about assessment techniques. In almost all the economies we surveyed, respondents reported that pre-service teacher training programs provide teachers with courses or workshops on classroom assessment techniques (table 4.4). In most East Asian economies, respondents reported that pre-service teacher training takes the form of specific required courses or workshops on classroom assessment. In other economies, such as Singapore, training programs were reported to also address the topic of classroom assessment in the context of other courses. For example, the course on mathematics instruction covers how to assess mathematics. It appears that in-service teacher training programs in several economies also include courses or workshops on large-scale, system-level assessment; classroom assessment; or examinations. Many economies also offer teachers the opportunity to work on short-term tasks related to national examinations. For example, in 11 out of the 14 economies surveyed, teachers are fully involved in writing examination questions, administering examinations, or scoring them. Table 4.2. Type of Document Mandating or Providing a Framework for the Assessment Activity* Law, official decree, or regulation Classroom Large-scale, system- assessment Examinations level assessment (national) Cambodia   China ** 32 Hong Kong SAR, China   Indonesia    Japan  Korea, Rep.    Lao PDR **   Malaysia    Mongolia    Philippines Shanghai,    China   Singapore ** Thailand **   Vietnam    Total 8 10 11 Source: Authors’ compilation *This question was not asked for international large-scale assessment. **Nonbinding, but official, document such as a set of assessment principles, standards, or recommendations. Table 4.3. Budget Situation for the Assessment Activity* There is a regular budget approved by law or decree or allocated at the discretion of the ministry or other authority Large-scale, Large-scale, system- Examinations system-level level assessment assessment (international) (national) Cambodia    China  Hong Kong SAR, China    Indonesia    Japan    33 Korea, Rep.    Lao PDR    Malaysia    Mongolia    Philippines    Shanghai, China    Singapore   Thailand    Vietnam    Total 10 11 9 Source: Authors’ compilation *This question was not asked for classroom assessment. Table 4.4. Teacher Training Programs on Assessment* Teacher training Classroom Large-scale, system- assessments Examinations level Assessment (National) Pre- In- In- Tasks In-service service service service Cambodia      China  **  Hong Kong SAR, China  **    Indonesia      Japan  **    Korea, Rep.  **    Lao PDR      34 Malaysia      Mongolia  **    Philippines      Shanghai,  China     Singapore  **    Thailand  **    Vietnam  **    Total 13 6 8 11 9 Source: Authors’ compilation *This question was not asked for international large-scale assessment. We also did not ask about the availability of pre-service teacher training in the specific areas of examinations or large-scale, system-level assessment. ** Widespread (covering two-thirds or more of all schools) audit or supervision system to review and evaluate the quality of classroom-based assessment activities. Finding 2: Most Economies Report Aligning Their Examinations and (Where They Exist) National Large-Scale, System-Level Assessment Activities to Their Official Learning Standards or Curriculum In general, it is important for examinations and national large-scale, system-level assessments to be closely aligned with official learning standards and curricula since their purpose is to measure or certify attainment at the student and system levels with respect to the stated learning goals of the system. (Alignment is less important in the case of international large-scale assessments, since these tend to serve as an external validity check with respect to the learning outcomes produced by an education system.) Respondents from 13 of the 14 economies surveyed reported that their examinations measure the official school curriculum or learning standards (table 4.5), although it appears that only three (Hong Kong SAR, China; Lao PDR; and Mongolia) include information from classroom- or school-based assessment activities in a student’s final score on the examination. Twelve of the economies surveyed also reported having a national large-scale, system-level assessment program based on their official school curriculum or learning standards (table 4.5). For eight of these economies, respondents reported that an independent review of the alignment between the national large-scale, system-level assessment and the official curriculum or learning standards has been conducted. 35 Box 4.1. Improving the Quality of University Entrance Examinations in Vietnam Prior to 1997, university entrance examinations in Vietnam were set by individual universities and were based on test questions made available ahead of time by the Ministry of Education and Training (MOET). These examinations promoted memorization and repetition as opposed to higher-order thinking skills. In 1997, the MOET stopped making examination questions available to the public, and in 2002, it took control of the student admission and placement process in universities in an effort to move toward a more holistic and equitable approach to evaluating student learning and achievement. In this regard, the MOET began to promote a “three things in common” policy for all universities: common examination questions, common organization of the examination, and common use of the examination results. As a consequence, a central examination board was established, and a single common university entrance examination was introduced. Today, students in Vietnam must take two examinations for full-time admission to higher education: the university entrance examination and the secondary school –leaving certificate. The secondary school– leaving certificate consists of a broad range of subjects, whereas the university entrance examination focuses on the subjects that the student intends to study at university. While the two examinations draw from the same item bank and are similar in format, they serve different purposes and uses. Consolidating the two examinations continues to be a topic of discussion, but a final decision on this issue has yet to be reached. Sources: Warwick 2009; Asian Development Bank 2010; MOET 2009; World Bank 2008, 2009c, 2010. Table 4.5. Alignment of Assessment Activities with Official Learning Standards or Curriculum* Assessment exercises are based on official school curricula or learning standards Examinations Large-scale, system-level assessment (national) Cambodia   China  Hong Kong SAR, China  ** Indonesia  ** Japan   Korea, Rep.  ** Lao PDR  ** 36 Malaysia  ** Mongolia  ** Philippines  ** Shanghai, China   Singapore  Thailand  ** Vietnam   Total 13 12 Source: Authors’ compilation *This question was not asked for classroom assessment or international large-scale assessment. **Independent review of the alignment between the assessment and the official curriculum or learning standards has been conducted. Finding 3: The General Quality and Stability of International Large-Scale Assessment Activity in East Asia Appears Good, but There Is Room for Improvement in the Dissemination and Use of Results One-time assessments may have shock value—particularly if test scores are worse than expected—and therefore create an opening for wider discussions of educational quality. However, only by creating sustained assessment systems are governments able to monitor learning trends over time and gain a full understanding of the contributions made by various inputs and educational practices to student learning outcomes. Many economies in the East Asia region have participated regularly in the main international large-scale assessment exercises—specifically PISA, PIRLS, and TIMSS—which have yielded a rich source of comparable data on student learning outcomes and trends for the entire region (table 4.6). The exceptions have been some lower-income economies that have not participated in these international exercises on a regular basis. In almost all the economies surveyed, copies of reports containing the results of the international assessment exercise are disseminated to key stakeholders, and press releases are issued to the public (table 4.7). However, there appear to be fewer mechanisms for ensuring the longer-term dissemination of, access to, and use of results. For example, apparently only four economies make the results available online, and only five provide the results to schools and educators. The same patterns appear to prevail with regard to the results of national large-scale, system-level assessment exercises. 37 Box 4.2. Taking a Systems Approach to Improving Assessment Activities in Korea Under the leadership of President Lee Myung-Bak, the Presidential Agenda has been introduced to improve assessment policies in Korea in three key areas: In 2008, to increase school and teacher accountability, the Korean government launched the National Assessment of Educational Achievement (NAEA) to systematically assess student achievement at the national level. The NAEA has been administered to all students in the sixth grade, to middle school seniors, and to high school sophomores every year since 2008. The specific objectives of the NAEA are to evaluate the educational achievements of all students in the applicable grades, to address weaknesses and formulate strategic plans for improving student achievement, to obtain baseline data to serve as a comparison for future assessments, to investigate the major factors that influence educational achievement, and to provide schools with examples of effective assessment methods. To enhance the quality of public education and reduce private tutoring expenses, the Korean government is aligning the examination for college admission, the College Scholastic Ability Test (CSAT), with the 2009 revised national curriculum. College and university admissions officers will be given more decision-making power and autonomy during the admissions process as CSAT scores will become one of the most important —as opposed to the most important—admission requirements. 38 To better manage the curriculum and assessment results at the system level, the Korean government launched the National Education Information System to enable the electronic storage and management of the School Achievement Records (SARs) of elementary, middle, and high school students. Teachers will use this newly implemented system to record student data electronically, and students’ SARs can now be viewed online by students and parents as well as by college admissions staff members during the college admission process. Source: Baek 2010. Table 4.6. Participation in International Large-Scale Assessments, 1995–2011 PISA TIMSS PIRLS 2000 2003 2006 2009 1995 1999 2003 2007 2011 2001 2006 2011 Cambodia            China        Hong Kong SAR, China  Indonesia    Japan     Korea     39 Lao PDR             Malaysia          Mongolia             Philippines           Shanghai,  China           Singapore     Thailand       Vietnam             Total 5 5 5 8 7 8 7 8 7 2 3 3 Source: Authors’ compilation Box 4.3: Vision and Leadership in Assessment in Singapore Since 1992, the Singaporean school curriculum has shifted from emphasizing memorization to focusing on higher-order skills, such as problem-solving and communication skills. Nonetheless, student assessment, especially in the form of national examinations, has continued to play an important role in the education system. With the shift in focus to higher-order skills, however, additional types of assessment tasks have been introduced into the national examination, including school-based assessment. Even with these changes in the competencies being assessed and in the ways they are assessed, the Ministry of Education of Singapore has continued to provide strong leadership and support for assessing student learning and continues to use assessment data to make the necessary adjustments in the education process and to improve student learning. Sources: Yeap 2008, 2009; World Bank 2011. 40 Table 4.7. Dissemination of Results from the Most Recent International Large-Scale Assessment Exercise Clear plan and products presentations available to provide feedback to schools or educators online or distributed International report to key stakeholders distributed to key distributed to key available online Press release to National report National report Brochures and communicate stakeholders stakeholders results Hong Kong SAR, China       Indonesia       41 Japan       Korea,    Rep.    Malaysia       Mongolia       Philippines       Shanghai,      China  Singapore       Thailand       Total 9 10 8 4 6 5 Source: Authors’ compilation Finding 4: Many Lower-Income Economies in the Region Lack Mechanisms for Monitoring or Ensuring the Quality of Classroom Assessment Activities by Teachers, Something They Must Address to Create Truly Effective Student Assessment Systems That Support Learning for All All 14 economies reported using classroom assessment activities primarily for supporting or promoting student learning. Despite a strong commitment to pre-service teacher t raining in classroom assessment, there appears to be less emphasis on in-service training or on actually observing the quality or effectiveness of classroom assessment activities as part of normal school inspection and supervision activities. Where such oversight does occur, it tends to be in the higher- income economies in the region (see table 4.4). Improving Student Assessment Systems in East Asia The overview of student assessment systems presented in this chapter shows that most economies in the East Asia region have created a relatively strong enabling context for their assessment activities in the form of clear policy frameworks, regular budgets, and opportunities for teachers to learn about— and develop skills in—assessment. However, findings also show that there is room for improvement in the dissemination and use of results 42 from large-scale assessment exercises, particularly with a focus on informing and supporting schools. For example, to date, economies in the East Asia region seem to have done a better job of informing policy makers and the media of the results of large-scale assessment exercises than informing schools and teachers. Of all assessment activities, classroom assessment has the biggest potential impact on student learning. However, ensuring the quality of classroom assessment activities in the region has received the least attention from policy makers, particularly in the lower-income economies. Mechanisms to oversee the quality of classroom assessment, such as audit or supervision exercises, are more common in the higher- income economies. As mentioned earlier, these findings rest on a set of key indicators that serve as a quick “dipstick” of strengths and weaknesses in a student assessment system. This limitation notwithstanding, there seems to be sufficient evidence to conclude that most East Asian economies have taken important steps in creating a sound foundation for their assessment systems. By building on this foundation and by prioritizing improvements in the dissemination and use of assessment results, as well as in the quality assurance of classroom assessment activities, East Asian economies should be well positioned to move toward the ultimate goal of learning and quality education for all. References AERA (American Educational Research Association), APA (American Psychological Association), and NCME (National Council on Measurement in Education). 1999. Standards for Educational and Psychological Testing. Washington, DC: AERA. Airasian, P., and M. Russell. 2007. Classroom Assessment: Concepts and Applications. 6th ed. New York: McGraw-Hill. Asian Development Bank. 2010. Vietnam: Preparing the Higher Education Sector Development Project (HESDP). Technical Assistance Consultant’s Report. Manila: Asian Development Bank. Baek, S. 2010. “Case Study on Korea’s Educational Assessment System.” Seoul National University, Seoul. Black, P., and D. Wiliam. 1998. “Assessment and Classroom Learning.” Assessment in Education: Principles, Policy, and Practice 5 (1): 7–73. Clarke, M. 2011. “Framework for Building an Effective Assessment System.” Discussion Paper, World Bank, Washington, DC. Darling-Hammond, L., and L. Wentworth. 2010. Benchmarking Learning Systems: Student Performance Assessment in International Context. Stanford, CA: Stanford Center for Opportunity Policy in Education, Stanford University. Ferrer, G. 2006. Educational Assessment Systems in Latin America: Current Practice and Future 43 Challenges. Washington, DC: Partnership for Educational Revitalization in the Americas. Greaney, V., and T. Kellaghan. 2008. Assessing National Achievement Levels in Education. Washington, DC: World Bank. Heubert, J., and R. Hauser. 1999. High Stakes: Testing for Tracking, Promotion, and Graduation. Washington, DC: National Academies Press. Hoxby, C. 2002. “The Cost of Accountability.” NBER Working Paper 8855, National Bureau of Economic Research, Cambridge, MA. MOET (Ministry of Education and Training). 2009. Report on the Development of Higher Education System, the Solutions to Ensure Quality Assurance and Improve Education Quality. Hanoi: Social Republic of Vietnam. Ravela, P., P. Arregui, G. Valverde, R. Wolfe, G. Ferrer, F. M. Rizo, M. Aylwin, and L. Wolff. 2009. “The Educational Assessments That Latin America Needs.” PREAL Working Paper 40, Partnership for Educational Revitalization in the Americas, Washington, DC. Warwick, E. 2009. “Draft Report on the Self-Diagnostic Survey of Education Assessment in Vietnam (for READ).” World Bank, Washington, DC. Wolff, L. 2007. “The Costs of Student Assessment in Latin America.” PREAL Working Paper 38 , Partnership for Educational Revitalization in the Americas, Washington, DC. World Bank. 2008. Vietnam: Higher Education and Skills for Growth. Washington, DC: Human Development Department, East Asia and Pacific Region, World Bank. ———. 2009c. Vietnam Self-Diagnosis Report. Washington, DC: World Bank. ———. 2010. Draft READ Annual Report. Washington, DC: World Bank. ———. 2011. “When Children Learn, Nations Prosper.” READ Video, Human Development Network Education, World Bank, Washington DC. Yeap, B. H. 2008. “A Seminar on Singapore Math.” Presented in Salt Lake City, UT. ———. 2009. Singapore’s Experience with Learning Assessment. Singapore: National Institute of Education, Nanyang Technological University. 44 CHAPTER 5 Teacher Policies Emiliana Vegas 1 SUMMARY This chapter presents an analysis of teacher policies in six East Asian education systems. Overall, the region has achieved acceptable levels of development in six of the eight core teacher policy goals that are measured under the System Assessment and Benchmarking Education for Results (SABER). Thus, policy makers in the region have succeeded in setting clear expectations for teachers, attracting the best into teaching, matching teachers’ skills with students’ needs, monitoring teaching and learning, supporting teachers to improve instruction, and motivating teachers to perform. Yet the region has room to improve in the goals of preparing teachers with useful training and experience and leading teachers with strong principals. The Importance of Teacher Policies Research suggests that teacher quality is the main school-based predictor of student achievement 45 and that several consecutive years of outstanding teaching can offset the learning deficits of disadvantaged students (Hanushek and Rivkin 2006; Nye, Konstantopoulos, and Hedges 2004; Park and Hannum 2001; Rivkin, Hanushek, and Kain 2005; Rockoff 2004; Sanders 1998; Sanders and Rivers 1996; Vignoles et al. 2000). However, it is not yet clear exactly which teacher policies can raise teacher effectiveness (Goldhaber 2002; Rivkin, Hanushek, and Kain 2005). Thus, devising effective policies to improve teaching quality remains a challenge. In East Asian and other economies, interest is increasing in attracting, retaining, developing, and motivating great teachers. While the World Bank has ample experience in supporting teacher policy reforms in the developing world, until recently there was no systematic effort to offer data and analysis that can provide policy guidance on teacher policies. SABER fills this gap and collects, analyzes, synthesizes, and disseminates comprehensive information on teacher policies in primary and secondary schools across a range of different education systems. The goal is to enable policy makers to learn how other governments address the same policy challenges related to teacher management and thus how to make well-informed policy choices that will lead to improved learning outcomes. Overview Data is collected on 10 core teacher policy areas in order to offer a comprehensive overview of teacher policies in each education system These policy areas are listed in box 5.1. It is important to highlight that the main focus is on the policies formally adopted by education systems. While in some cases the data collected also address how the teacher policy goals are implemented, the nature of the data collection approach (based on interviews with key informants and official document review) does not allow for a thorough assessment of policy implementation. Therefore, complementary research will be useful in most settings. Box 5.1. Key Teacher Policy Areas 1. Requirements to enter and remain in teaching 2. Initial teacher preparation 3. Recruitment and employment 4. Teachers’ workload and autonomy 46 5. Professional development 6. Compensation (salary and non-salary benefits) 7. Retirement rules and benefits 8. Monitoring and evaluation of teacher quality 9. Teacher representation and voice 10. School leadership The analysis focuses on eight teacher policy goals Three criteria were used to select the teacher policy goals: (a) policies were linked to performance through evidence provided by research and studies, (b) there was a high priority for resource allocation, and (c) policies were actionable. The eight teacher policy goals are presented in figure 5.1. Figure 5.1. Teacher Policy Goals 8. Motivating 1. teachers to Setting clear perform expectations for 7. teachers Supporting 2. teachers to Attracting improve the best into instruction teaching Effective 6. Teachers Monitoring 3. teaching and Preparing 47 learning teachers with useful training and 5. Leading experience teachers 4. Matching with strong teachers’ skills principals with students’ needs Source: Author’s compilation Since the eight teacher policy goals were determined by the existing research, there might well be other objectives that economies want to pursue to increase the effectiveness of teachers. For example, the tool collects information on teacher organizations as part of the policy area of “teacher representation and voice” and will make that information publicly available. Yet no clear trend has emerged regarding whether or how governments should engage with these organizations in policy formulation. To be sure, many studies have looked at the impact of unionization on schools’ productivity (Argys and Reese 1995; Eberts and Stone 1986; Hoxby 1996); student learning (Kingdon and Teal 2008; Kleiner and Petree 1988; Kurth 1987; Register and Grimes 1991; Steelman, Powell, and Carini 2000); teachers’ wages (Ballou and Podgursky 2002; Baugh and Stone 1982; Bee and Dolton 1995; Dolton and Robson 1996); working conditions (Eberts 1984; Murillo et al. 2002; Zegarra and Ravina 2003); and education policy (Goldschmidt and Stuart 1986; Woodbury 1985). But even top-performing countries differ widely in how much they engage, to what extent they regulate, and how they organize teachers’ unions. Data collected by SABER will offer guidance on how to approach these issues in the future. For a more detailed report on the eight teacher policy goals and the evidence supporting this selection, see Vegas et al. (2010). Regional Findings 48 Policy data were collected for Cambodia; Indonesia; Japan; Lao PDR; Malaysia; Mongolia; the Philippines; Shanghai, China; Singapore; the Republic of Korea; Thailand; and Vietnam. Table 5.1 presents the extent to which each education system has progressed in the eight policy goals. The analysis indicates that each teacher policy system has relative strengths and weaknesses. However, across all the systems, some general patterns were observed, which are discussed further in this chapter. Table 5.1. Levels of Development of Teacher Policies in East Asian Education Systems Philippines Cambodia Singapore Indonesia Shanghai, Mongolia Malaysia Thailand Vietnam POLICY Lao PDR Korea, Japan China Rep. GOAL 1. Setting clear Established Advanced Advanced Established Established Advanced Advanced Advanced Emerging Established Established Advanced expectations             for teachers 2. Established Emerging Established Established Emerging Established Established Established Established Established Established Emerging Attracting the best into             teaching 3. Preparing teachers with Latent Emerging Established Latent Established Emerging Established Emerging Emerging Emerging Established Emerging useful             training and experience 4. Matching 4 teachers’ Emerging Emerging Latent Latent Established Established Emerging Established Latent Latent Emerging Emerging skills with             9 9 students’ needs 5. Leading Established Emerging Emerging Emerging Emerging Established Emerging Established Emerging Established Established Emerging teachers with             strong principals 4 6. Established Emerging Emerging Established Advanced Established Advanced Established 9 Established Established Advanced Established Monitoring teaching and             learning 7. Supporting Emerging Latent Established Emerging Established Advanced Established Established Established Latent Established Established teachers to             improve instruction Established 8.  Motivating Emerging Emerging Emerging Emerging Established Established Established Emerging Established Established Emerging teachers to            perform Source: Author’s compilation. Setting Clear Expectations for Teachers This aspect of teacher policy is key for several reasons. First, expectations for student and teacher performance influence how potential entrants perceive the profession. Second, expectations guide teachers’ work. The more specific they are, the better teachers can organize their time and resources to meet them. Finally, expectations can help align the goals of key aspects of the profession (such as pre- service training, professional development, and teacher appraisals). The more institutionalized these expectations are, the more likely all of these aspects will be realized. There are three policy levers education systems can use to reach this goal:  Set clear expectations for what students should know and be able to do.  Set clear expectations for what teachers are supposed to do.  Ensure that teachers have enough time to fulfill their duties. All education systems in the East Asia region are established or advanced in this policy goal, which means they have (a) developed explicitly defined standards for what students should know and be able to do, as well as curricula to guide teaching and learning; (b) officially stipulated tasks for teachers; and (c) delineated official time allocations that enable teachers to fulfill their duties. (Shanghai, China has a slightly lower rating because has not delineated official time allocations for teachers.) Attracting the Best into Teaching (Guarino, Santibáñez, and Daley 2006) 50 First, more able individuals make better teachers (Boyd et al. 2009, 2010). Second, top candidates maximize the impact of teacher training. If the quality of student teachers is too low, training is likely to focus more on making up for their deficiencies in knowledge and skills and less on turning them into effective teachers. Finally, luring top talent into teaching has a multiplier effect: if teaching attracts qualified people, competitive candidates who had not considered teaching might be drawn to it. One issue that is not included in this list because of lack of clear guidance from available evidence on how to tackle it is that of the flexibility of the profession. Several studies find that some women prefer to teach because they can take leaves of absence to care for their families without incurring wage penalties when they come back (Flyer and Rosen 1997; Stinebrickner 1999a, 1999b, 2001a, 2001b). Yet it is unclear that education systems want to attract these candidates any more than other groups of candidates. There are four policy levers education systems can use to reach this goal:  Set appropriate entry requirements to attract talented candidates.  Provide appealing pay and benefits for talented candidates.  Ensure appealing working conditions for talented candidates.  Create attractive career opportunities. With the exception of Indonesia, Lao PDR, and Vietnam, all East Asian systems are established or advanced in meeting this policy goal, which means they have (a) entry requirements that allow screening of talented individuals, (b) attractive pay and benefits, (c) appealing working conditions, and (d) attractive career opportunities within the teaching profession. Preparing Teachers with the Skills They Need to Succeed in a Classroom Few (if any) individuals are born effective teachers. Teachers need subject-matter knowledge, classroom management skills, and lots of practice to be successful in a classroom. In addition, preparation puts all teachers on an equal footing, giving them a common framework and a base from which they can improve their practice. One aspect not included in this framework because no data on this indicator were collected is that of teacher certification. Although the definition of teacher certification varies by country—and sometimes within one country—studies have found that certification status is generally associated with higher teacher effectiveness (Darling-Hammond 1999; Darling-Hammond and Sykes 1999; Darling-Hammond, Barnett, and Thoreson 2001; Fuller 2000; Goldhaber and Brewer 2000; Hawk, Coble, and Swanson 1985). Education systems can use three policy levers to prepare teachers:  Set minimum standards for pre-service teaching training programs.  Require individuals to have prior classroom experience.  Ensure a smooth transition from pre-service training into the first teaching job. In this policy goal, the majority of systems are latent or emerging. 51 Matching Teachers’ Skills with Students’ Needs to Promote Equity and Efficiency Matching teachers’ skills with students’ needs is a way of ensuring that all students in an education system have equal opportunities to learn: without purposeful incentives, teachers tend to gravitate toward schools with better working conditions, which often serve better-off students (Boyd et al. 2005; Hanushek, Kain, and Rivkin 2004). Moreover, it is a way of ensuring that teachers are distributed efficiently—that is, to minimize the number of surplus teachers at a given grade, subject, or geographic area. Finally, ensuring that teachers are a good match for their school can also increase their effectiveness and reduce turnover rates (Boyd et al. 2002, 2005; Jackson 2010). Much research has been devoted to the issue of turnover. Several studies have noted that it is not always the most effective teachers who leave (Boyd et al. 2007; Hanushek et al. 2005; West and Chingos 2008). Yet these studies also concede that there is still considerable room for schools to enact targeted policies aimed at retaining only the most effective performers. Two policy levers can be used to match teachers’ skills with students’ needs:  Create incentives for teachers to work at hard-to-staff schools.  Create incentives for teachers to teach critical shortage subjects. A majority of East Asian systems are latent or emerging in this policy goal. Leading Teachers with Strong Principals Leading teachers with strong principals is important because the quality of school heads is second only to classroom teaching as a predictor of student learning (Eberts and Stone 1988; Leithwood et al. 2006) Quality principals attract and retain quality teachers (Boyd et al. 2009; Ingersoll 2001; Papa, Lankford, and Wyckoff 2002). Also, capable principals can spearhead much-needed change at the school level. Finally, good principals can facilitate teachers’ work and continuous improvement. The more capable a principal is, the more he or she can support teachers, create a sense of community, make teachers feel valued, and ease their anxiety about external pressures (Mulford and Silins 2003). There are three policy levers education systems can use to reach this goal:  Ensure requirements to become a principal attract talented candidates.  Create incentives for principals to perform well.  Allow principals to make key decisions to improve teaching. This policy goal remains a challenge in many systems. However, several systems, including those of Cambodia, Malaysia, the Philippines, Singapore, and Thailand, have set up mechanisms to provide autonomy to principals to make decisions related to instruction or personnel management for their schools––a factor that has been shown to enhance student learning outcomes in high-performing economies. 52 Monitoring Teaching and Learning to Improve Student Learning Outcomes Teacher and student evaluations help identify good practices, which can then be shared among teaching staff members to improve school performance. Also, identifying low-performing teachers and students is necessary to ensure that they are supported in a timely manner. Finally, such information is useful for accountability purposes. Two policy levers can be used to reach this goal:  Ensure that there are sufficient student achievement data to inform teaching.  Ensure that there are sufficient teacher performance data to inform teaching. With the exception of Indonesia and Japan, all other East Asian economies are either established or advanced in this policy goal, having put in place solid student learning assessment systems and teacher performance appraisal mechanisms. Supporting Teachers to Improve Instruction All teachers can improve—regardless of how effective they are at one point in time. Therefore, support mechanisms are necessary to help teachers reach their potential and perform at their best. Furthermore, changes in classroom assignments or student populations can pose new challenges to teachers. Thus, during periods of transition, teachers will need additional help to sustain their performance. Finally, support mechanisms can go a long way to prevent burnout and reduce turnover. Even motivated teachers may choose to leave if they are consistently ineffective, do not know how to improve, or receive little support. There are two policy levers education systems can use to reach this goal:  Use teacher performance data to improve teaching.  Provide professional development to improve practice. Many East Asian economies have achieved solid systems to support teachers for instruction. However, Cambodia, Indonesia, Korea, and Singapore have fewer systems in place for supporting teachers than do other East Asian systems. Motivating Teachers to Perform The more aligned incentives are with the behaviors and outcomes expected of teachers, the more likely teachers are to behave and perform as expected. Incentives are also a way of recognizing teachers’ 53 work. Teaching is a challenging job, and incentives can let teachers know that the results they have achieved are valued so that they continue working hard to sustain them. In addition, some types of incentives can influence the profile of the teaching profession and make it more competitive, dynamic, and performance driven. The presence of incentives can have an effect on the attractiveness of the teaching profession. There are three policy levers education systems can use to reach this goal:  Develop minimum mechanisms to hold teachers accountable.  Offer rewards for high-performing teachers.  Sanction low-performing teachers. In this policy goal, there is no clear regional trend. In many East Asian systems, there are minimum mechanisms to hold teachers accountable as well as rewards and sanctions for high- and low- performing teachers, respectively. Improving Teacher Policies: Priorities for East Asia East Asian education systems have clear strengths, as well as some areas for development with respect to their teacher policies. All economies have established clear expectations for teachers, which help set the standards for teaching and learning and help guide classroom practices. In addition, a majority of systems have attractive pay and working conditions and selective entry into the profession, thus enabling economies to attract strong professionals into teaching. Moreover, most education systems in the region have established mechanisms for monitoring teaching and learning. The policy goals where certain East Asian systems can devote attention and resources include preparing teachers with useful training and experience, matching teachers’ skills with students’ needs, leading teachers with strong principals, supporting teachers to improve instruction, and motivating teachers to perform. Preparing Teachers with Useful Training and Experience Several studies have shown that, while experience is not a good predictor of teacher quality, the first few years of a teacher’s experience considerably increase his or her effectiveness in the classroom, regardless of whether such experience is acquired through clinical practices or a probationary period (Boyd et al. 2009; Chingos and Peterson 2010; Hanushek et al. 2005; Hanushek and Rivkin 2010; Rivkin, Hanushek, and Kain 2005). Therefore, systems in the region should consider whether they would benefit from ensuring that all teachers who enter the profession have had some practical classroom experience. 54 Similarly, the way in which education systems manage the transition between a teacher’s training and his or her first job influences how effective he or she will be in the classroom. The smoother the transition, the more likely the teacher will succeed in school. For example, available evidence on induction and mentoring programs, which introduce new teachers to their schools and provide them with the support that they need to adapt to their new work environment, suggests that they can make teachers more effective in the classroom (Glazerman et al. 2010) and reduce teacher turnover (Rockoff 2008). Matching Teachers’ Skills with Students’ Needs One way education systems can foster a more equitable distribution of teachers is by using incentives to let the “price” of teachers vary to reflect their relative scarcity in different schools: if high-performing teachers find it financially attractive to teach in hard-to-staff schools, they are less likely to be concentrated in suburban schools that serve better-off students. Additionally, incentives can also offset information gaps: teachers often know little about the staffing needs of schools in remote areas; incentives can act as a signal to reflect these needs (Santiago 2004). Even in education systems with well-designed incentives to attract teachers into hard-to-staff schools, the distribution of teachers may still be inequitable. Thus, considering teacher experience when making transfer decisions, as is the case in many East Asian systems and around the world, can be detrimental to education equity. Leading Teachers with Strong Principals Giving school principals more authority over decisions that influence teaching and learning can have a positive influence on instruction. A growing body of evidence finds that when principals are allowed to hire teachers, they tend to hire candidates who turn out to be effective classroom instructors according to value-added estimates of teacher quality (Ballou and Podgursky 1998; Boyd et al. 2010; Rockoff et al. 2010). Several studies have also shown that principals make sound dismissal decisions if allowed to do so. Taking into account both effort and performance when dismissing teachers (Jacob 2010a), principals tend to dismiss teachers who are among the lowest performers, according to value-added estimates (Rockoff et al. 2010). Furthermore, their authority to dismiss teachers increases teacher effort over time (Jacob 2010b). Supporting Teachers to Improve Instruction Ensuring that teacher performance data are used to improve classroom instruction is useful because the more data are used to drive changes in instruction, the more likely these changes are to raise student learning. Similarly, the more professional development activities expose teachers to best practices in instruction, the more likely teachers are to adopt best practices in their classrooms. In East Asia, as in the rest of the world, there are varying mechanisms to support teachers in improving instruction and also differences in the extent to which teachers themselves are expected to bear the costs of 55 professional development. In general, it is useful to ground professional development activities on solid and comprehensive empirical data on student learning, to inform those areas where teachers are most in need of training. In addition, reducing the costs borne by teachers will likely increase their participation rates in professional development activities (Angrist and Lavy 2002; Borko 2004; Brown, Smith, and Stein 1996; Cohen and Hill 1997; Wenglinsky 2000; Wiley and Yoon 1995). Motivating Teachers to Perform Education systems can ensure that all teachers meet minimum standards of behavior, effort, and performance by tying these standards to accountability mechanisms: if there are consequences for not meeting these standards, teachers are more likely to do so. Education systems can also motivate teachers by rewarding outstanding performance: if teachers who attain above-average results are rewarded, they are likely to maintain or even raise their effort. Conversely, if other teachers want to attain that payoff, they are likely to work harder to obtain the same results as the high-performing teachers. While research is still needed to evaluate the effectiveness of various instruments to incentivize teachers, recent studies indicate that minimum accountability mechanisms and rewards for strong teacher performance can lead to improved student outcomes (Glewwe, Holla, and Kremer 2008; Muralidharan and Sundararaman 2011). The research also indicates that the design details of teacher incentives matter, suggesting that the extent to which teachers’ efforts can be linked to the desired outcomes is a key factor in determining the impact of teacher incentive mechanisms. References Angrist, J., and V. Lavy. 2002. “New Evidence on Classroom Computers and Pupil Learning.” Economic Journal 112 (482): 735–65. Argys, L. M., and D. I. Reese. 1995. “Unionization and School Productivity: A Reexamination.” Research in Labor Economics 14: 49–68. Ballou, D., and M. Podgursky. 1998. “Teacher Recruitment and Retention in Public and Private Schools.” Journal of Policy Analysis and Management 17 (3): 393–417. ———. 2002. “Returns to Seniority among Public School Teachers.” Journal of Human Resources 37 (4): 892–912. Baugh, W. H., and J. A. Stone. 1982. “Teachers, Unions, and Wages in the 1970s: Unionism Now Pays.” Industrial and Labor Relations Review 35 (3): 368–76. Bee, M., and P. J. Dolton. 1995. “The Remuneration of School Teachers: Time Series and Cross-Section Evidence.” Manchester School 63 (1): 11–22. Borko, H. 2004. “Professional Development and Teacher Learning: Mapping the Terrain.” Educational Researcher 33 (8): 3–15. Boyd, D., P. Grossman, M. Ing, H. Lankford, S. Loeb, and J. Wyckoff. 2009. “Teacher Preparation and Student Achievement.” Educational Evaluation and Policy Analysis 31 (4): 416–40. 56 Boyd, D., H. Lankford, S. Loeb, M. Ronfeldt, and J. Wyckoff. 2010. “The Role of Teacher Quality in Retention and Hiring: Using Applications-to-Transfer to Uncover Preferences of Teachers and Schools.” NBER Working Paper 15966, National Bureau of Economic Research, Cambridge, MA. Boyd, D., H. Lankford, S. Loeb, and J. Wyckoff. 2002. “Initial Matches, Transfers, and Quits: The Role of Teachers’ Career Decisions in the Disparities in Average Teacher Qualifications across Schools.” Research paper, Teacher Policy Research, Albany. ———. 2005. “Explaining the Short Careers of High-Achieving Teachers in Schools with Low-Performing Students. American Economic Review 95 (2): 166–71. ———. 2007. “Who Leaves? Teacher Attrition and Student Achievement.” Research report, Teacher Policy Research, Albany. Brown C., M. Smith, and M. Stein. 1996. “Linking Teacher Support to Enhanced Classroom Instruction.” Presented at the annual meeting of the American Educational Research Association, New York, April 8–12. Chingos, M. M., and P. E. Peterson. 2010. “It’s Easier to Pick a Good Teacher Than to Train One: Familiar and New Results on the Correlates of Teacher Effectiveness.” PEPG Working Paper 10-22, Program on Education Policy and Governance, Cambridge, MA. Cohen, D. K., and H. Hill. 1997. “Instructional Policy Classroom Performance: The Mathematics Reform in California.” University of Michigan, Ann Arbor. Darling-Hammond, L. 1999. “Teacher Quality and Student Achievement: A Review of State Policy Evidence.” Center for the Study of Teaching and Policy, Seattle, WA. Darling-Hammond, L., B. Berry, and A. Thoreson. 2001. “Does Teacher Certification Matter ? Evaluating the Evidence.” Educational Evaluation and Policy Analysis 23 (1): 57–77. Darling-Hammond, L., and G. Sykes, eds. 1999. Teaching as the Learning Profession: A Handbook of Policy and Practice. San Francisco, CA: Jossey-Bass. Dolton, P. J., and M. Robson. 1996. “Trade Union Concentration and the Determination of Wages: The Case of Teachers in England and Wales.” British Journal of Industrial Relations 34 (4): 539–56. Eberts, R. W. 1984. “Union Effects on Teacher Productivity.” Industrial and Labor Relations Review 37 (3): 346–48. Eberts, R. W., and J. A. Stone. 1986. “Teacher Unions and the Cost of Public Education.” Economic Inquiry 24 (4): 631–43. ———. 1988. “Student Achievement in Public Schools: Do Principals Make a Difference?” Economics of Education Review 7 (3): 291–99. Flyer, F., and S. Rosen. 1997. “The New Economics of Teachers and Education.” Journal of Labor Economics 15 (1): S104–39. Fuller, E. J. 2000. “Does Teacher Certification Matter? An Investigation of the Relationship between the Percentage of Properly Certified Teachers and Algebra I Achievement in Texas.” Presented at the annual meeting of the American Educational Research Association, New Orleans, LA, April 24– 28. 57 Glazerman, S., E. Isenberg, S. Dolfin, M. Bleeker, A. Johnson, and M. Grider. 2010. Impacts of Comprehensive Teacher Induction: Final Results from a Randomized Controlled Study. Washington, DC: Institute of Education Sciences. Glewwe, P., A. Holla, and M. Kremer. 2008. “Teacher Incentives in the Developing World.” In Performance Incentives: Their Growing Impact on K-12 Education, ed. M. Springer, 295–326. Washington, DC: Brookings Institution Press. Goldhaber, D. 2002. “The Mystery of Good Teaching: Surveying the Evidence on Student Achievement and Teachers’ Characteristics.” Education Next 2 (1): 50–55. Goldhaber, D., and D. Brewer. 2000. “Does Teacher Certification Matter? High School Teacher Certification Status and Student Achievement.” Educational Evaluation and Policy Analysis 22 (2): 129–45. Goldschmidt, S. M., and L. E. Stuart. 1986. “The Extent and Impact of Educational Policy Bargaining.” Industrial and Labor Relations Review 39 (3): 350–60. Guarino, C. M., L. Santibáñez, and G. A. Daley. 2006. “Teacher Recruitment and Retention: A Review of the Recent Empirical Literature.” Review of Educational Research 76 (2): 173–208. Hanushek, E. A., J. F. Kain, D. M. O’Brien, and S. G. Rivkin. 2005. “The Market for Teacher Quality.” NBER Working Paper 11154, National Bureau of Economic Research, Cambridge, MA. Hanushek, E. A., J. F. Kain, and S. G. Rivkin. 2004. “Why Public Schools Lose Teachers.” Journal of Human Resources 39 (2): 326–54. Hanushek, E. A., and S. G. Rivkin. 2006. “Teacher Quality.” In Handbook of the Economics of Education, vol. 2, ed. E. A. Hanushek and F. Welch, 1051–78. Amsterdam: Elsevier. ———. 2010. “Generalizations about Using Value-Added Measures of Teacher Quality.” Presented at the annual meetings of the American Economic Association, Atlanta, January 3–5. Hawk, P. P., C. R. Coble, and M. Swanson. 1985. “Certification: It Does Matter.” Journal of Teacher Education 36 (3): 13–15. Hoxby, C. M. 1996. “How Teachers’ Unions Affect Education Production.” Quarterly Journal of Economics 111 (3): 671–718. Ingersoll, R. M. 2001. “Teacher Turnover and Teacher Shortages: An Organizational Analysis.” American Educational Research Journal 38 (3): 499–534. Jackson, C. K. 2010. “Match Quality, Worker Productivity, and Worker Mobility: Direct Evidence from Teachers.” NBER Working Paper 15990, National Bureau of Economic Research, Cambridge, MA. Jacob, B. A. 2010a. “Do Principals Fire the Worst Teachers?” NBER Working Paper 15715, National Bureau of Economic Research, Cambridge, MA. ———. 2010b. “The Effect of Employment Protection on Worker Output: Evidence from Public Schooling.” PEPG Working Paper 10-10. Program on Education Policy and Governance, Cambridge, MA. 58 Kingdon, G., and F. Teal. 2008. “Teacher Unions, Teacher Pay, and Student Performance in India: A Pupil Fixed Effects Approach.” PEPG Working Paper 08-13, Program on Education Policy and Governance, Cambridge, MA. Kleiner, M., and D. Petree. 1988. “Unionism and Licensing of Public School Teachers: Impact on Wages and Educational Output.” In When Public Sector Workers Unionize, ed. R. Freeman and C. Ichiowski, 305–19. Chicago, IL: University of Chicago Press. Kurth, M. 1987. “Teachers’ Unions and Excellence in Education: An Analysis of the Decline in SAT Scores.” Journal of Labor Research 8 (4): 351–87. Leithwood, K., C. Day, P. Sammons, A. Harris, and D. Hopkins. 2006. “Successful School Leadership: What It Is and How It Influences Pupil Learning.” Nottingham, U.K.: National College for School Leadership. Mulford, B., and H. Silins. 2003. “Leadership for Organisational Learning and Improved Student Outcomes.” Cambridge Journal of Education 33 (2): 175–95. Muralidharan, K., and V. Sundararaman. 2011. “Teacher Performance Pay: Experimental Evidence from India.” Journal of Political Economy 119 (1): 39–77 Murillo, M. V., M. Tommassi, L. Ronconi, and J. Sanguinetti. 2002. “The Economic Effects of Unions in Latin America: Teachers’ Unions and Education in Argentina.” Inter-American Development Bank, Washington, DC. Nye, B., S. Konstantopoulos, and L. V. Hedges. 2004. “How Large Are Teacher Effects?” Educational Evaluation and Policy Analysis 26 (3): 237–57. Papa, F. C. Jr., H. Lankford, and J. Wyckoff. 2002. “The Attributes and Career Paths of Principals: Implications for Improving Policy.” State University of New York, Albany. Park, A., and E. Hannum. 2001. “Do Teachers Affect Learning in Developing Countries? Evidence from Matched Student-Teacher Data from China.” Presented at the conference on Rethinking Social Science Research on the Developing World in the 21st Century, Social Science Research Council, Park City, UT, June 7–11. Register, C. A., and P. W. Grimes. 1991. “Collective Bargaining, Teachers, and Student Achievement.” Journal of Labor Research 12 (2): 99–109. Rivkin, S. G., E. A. Hanushek, and J. F. Kain. 2005. “Teachers, Schools, and Student Achievement.” Econometrica 73 (2): 417–58. Rockoff, J. E. 2004. “The Impact of Individual Teachers on Student Achievement: Evidence from Panel Data.” American Economic Review 94 (2): 247–52. ———. 2008. “Does Mentoring Reduce Turnover and Improve Skills of New Employees? Evidence from Teachers in New York City.” Columbia Business School, New York. Rockoff, J. E., D. O. Staiger, T. J. Kane, and E. S. Taylor. 2010. “Information and Employee Evaluation: Evidence from a Randomized Intervention in Public Schools.” NBER Working Paper 16240, National Bureau of Economic Research, Cambridge, MA. Sanders, W. L. 1998. “Value Added Assessment.” School Administrator 11 (55): 24–27. 59 Sanders, W. L., and J. C. Rivers. 1996. “Cumulative and Residual Effects of Teachers on Future Student Academic Achievement.” Research progress report, University of Tennessee Value-Added Research and Assessment Center, Knoxville, TN. Santiago, P. 2004. “The Labour Market for Teachers.” In International Handbook on the Economics of Education, ed. J. Johnes and G. Johnes, 522–78. Cheltenham, U.K.: Edward Elgar. Steelman, L. C., B. Powell, and R. M. Carini. 2000. “Do Teacher Unions Hinder Educational Performance? Lessons Learned from State SAT and ACT Scores.” Harvard Educational Review 70 (4): 437–67. Stinebrickner, T. R. 1999a. “Estimation of a Duration Model in the Presence of Missing Data.” Review of Economics and Statistics 81 (3): 529–42. ———. 1999b. “Using Latent Variables in Dynamic, Discrete Choice Models: The Effect of School Characteristics on Teacher Decisions.” Research in Labor Economics 18: 141–76. ———. 2001a. “Compensation Policies and Teacher Decisions.” International Economic Review 42 (3): 751–79. ———. 2001b. “A Dynamic Model of Teacher Labor Supply.” Journal of Labor Economics 19 (1): 196– 230. Vegas, E., A. Ganimian, N. Goldstein, A. Paglayan, S. Loeb, and P. Romaguera. 2010. “What Are Teacher Policy Goals, How Can Education Systems Reach Them, and How Will We Know When They Do?” SABER-Teachers Background Paper 2, World Bank, Washington, DC. Vignoles, A., R. Levacic, J. Walker, S. Machin, and D. Reynolds. 2000. “The Relationship between Resource Allocation and Pupil Attainment: A Review.” Report 228, Centre for the Economics of Education, London School of Economics and Political Science, London. Wenglinsky, H. 2000. “How Teaching Matters: Bringing the Classroom Back into Discussions of Teacher Quality.” Princeton, NJ: Milken Family Foundation and Educational Testing Service. West, M., and M. Chingos. 2008. “Mobility and Attrition in Florida: A De scriptive Analysis.” PEPG Working Paper 08-01, Program on Education Policy and Governance, Cambridge, MA. Wiley, D., and B. Yoon. 1995. “Teacher Reports on Opportunity to Learn: Analyses of the 1993 California Learning Assessment Systems.” Educational Evaluation and Policy Analysis 17 (3): 355–70. Woodbury, S. 1985. “The Scope of Bargaining and Bargaining Outcomes in Public Schools.” Industrial and Labor Relations Review 38 (2): 195–210. Zegarra, E., and R. Ravina. 2003. “Teacher Unionization and the Quality of Education in Peru: An Empirical Evaluation Using Survey Data.” Inter-American Development Bank, Washington, DC. 60 CHAPTER 6 School Autonomy and Accountability Gustavo Arcia SUMMARY Evidence shows that the most successful schools are those that have pedagogical and budgetary autonomy and that enforce regular student assessments. This chapter presents a set of indicators to build strong autonomy and accountability mechanisms that will enhance education quality. The East Asian benchmarking exercise shows that budgetary autonomy at the school level is well established in most of the economies surveyed but may still require some changes to be fully achieved. In terms of school and teacher assessment, several of the economies are at the established and advanced stages, while only a few are still at the emerging stage. This finding suggests that with a little effort, most economies in the region could introduce these assessments and foster greater accountability throughout the school system. In terms of the participation of parents in school councils and the enforcement of accountability, some economies are doing very well, while several others are still at the emerging stage. In the case of autonomy over personnel 61 decisions, most economies are at only the latent or emerging stage. Why Do Autonomy and Accountability Matter? School autonomy and accountability are key components to ensure education quality. Studies have shown a clear causal link between school autonomy and efficiency in resource use (Barrera-Osorio et al. 2009). By transferring core managerial responsibilities to local schools, school autonomy fosters local accountability and increases the participation of parents and the community. As a result, the education provided tends to reflect local priorities, values, and needs (Bruns, Filmer, and Patrinos 2011). Also, as local accountability reveals areas for improvement in the school system—such as increasing the technical and pedagogical capacity of teachers or increasing the participation of parents in student life— education quality and learning outcomes tend to improve (Vegas and Umansky 2005). Although there is little formal evidence that teacher quality improves as a direct result of school-based management, there is a compelling argument that increasing school accountability is a necessary condition for improving teacher quality. The implementation of school-based management (SBM) can increase the support that parents and school councils give to good teachers—for example, by providing salary and nonsalary incentives and by setting the right conditions to attract the best candidates into the teaching profession (Arcia et al. 2011a). Autonomy and accountability are key areas of SBM and can be monitored through a series of indicators that allow rapid assessment and improvement of education systems. Using indicators also enables governments to compare their own policies and practices with those of other economies where students excel in international achievement tests. In East Asia, Hong Kong SAR, China, is a best-practice example of SBM reforms (See box 6.1). Box 6.1. School-Based Management in Hong Kong SAR, China: An Example of Best Practice The essence of Hong Kong SAR, China’s SBM reforms, which began in 1995, was to decentralize schools by devolving authority to all stakeholders, including the representatives of School-Sponsoring Bodies, principals, teachers, parents, and community members. School management committees were established to make key decisions and manage the direct transfer of funds from central 62 government to schools. Funding was based on the number of students enrolled in each school. In addition, schools in Hong Kong SAR, China, were highly decentralized in the areas of budgeting, student affairs, and the curriculum, but less decentralized in terms of personnel management, especially in setting teacher salaries. The role that school directors play in school management in Hong Kong SAR, China, is particularly important. They are the leaders in the school decision-making process, supported by members of the School-Sponsoring Body. In Hong Kong SAR, China, school directors and school board members are empowered to make decisions in consultation with departmental heads and teachers. The effectiveness of SBM is reflected in the 2009 PISA results, where Hong Kong SAR, China, ranked third among the 66 participating economies. Source: Ho (2005) Implementing Autonomy and Accountability for Improved Student Learning School-based management is a form of decentralization in which school personnel are in charge of making most managerial decisions, often in partnerships with parents and the community. Such an approach to school management creates the proper conditions for improving student learning in a sustainable way (Barrera, Fasih, and Patrinos 2009). SBM can encompass different activities and policies to improve student learning by involving both teachers and parents. By allowing more local control over school operations, SBM fosters a new social contract between parents and teachers by improving communication and increasing local cooperation and local accountability. For SBM to work effectively, it is useful to align the incentives of key stakeholders in a common strategy bound by the implicit incentives of the legal framework. Some components of SBM may take more time than others to be defined, whereas others may need to be implemented gradually until full autonomy and accountability are reached (Ho 2006). This gradual implementation requires the interplay of various managerial factors that determine autonomy and accountability at the school level, which can take several years to occur—see table 6.1. The available empirical evidence shows that it takes about eight years before school autonomy and accountability start affecting learning outcomes (Barrera, Fasih, and Patrinos 2009). 63 Table 6.1. Autonomy and Accountability at the School Level Strength Managerial factor Low Medium High Teacher and Centralized Regional hiring Local hiring and personnel hiring and and centralized firing. management firing. firing. Decentralized Decentralized at Centralized. budget with school level. Budget based regional Budget on payroll plus variations. Budget approved planning and an allotment Budget based on by the school approval for materials payroll and council and funds transferred directly and utilities. equity 64 to the school. considerations. Schools conduct routine Routine evaluations that Teacher None. evaluations; no provide teachers assessment direct and schools with accountability. incentives to perform better. Periodic standardized Routine Student None or based testing but standardized testing; assessment on local tests. results not made results made public. public. Source: Arcia et al. 2011. Autonomy and accountability have four key components: 1. The ability to hire and fire teachers and school directors 2. The participation of parents in the planning, approval, and monitoring of the school budget 3. The use of regular teacher assessments to make rewards conditional on performance 4. The use of assessments of student learning to measure school performance and enforce overall accountability Evidence also suggests that all of these critical components of SBM must be present and interconnected, so that the process amounts to more than just a series of discrete managerial activities (Arcia et al. 2011). The most successful combination of managerial components is still being studied, but an emerging body of practice points to a set of variables that foster managerial autonomy, the assessment of results, and the use of the assessment to promote accountability among stakeholders (Gertler, Patrinos, and Rubio-Codina 2007). These variables are used to assess the SBM indicators presented 65 here. The most recent empirical evidence on the effectiveness of SBM in improving learning comes from the analysis of the Programme for International Student Assessment (PISA) test results for 2009. PISA results cover 15-year-old students from 66 economies who took standardized tests in reading, mathematics, and science. Several Asian economies were among those that produced the best results, followed by several European economies, as shown in table 6.2 (OECD 2009). Table 6.2. 2009 PISA Scores for Asian and European Economies in the SBM Sample Economy Reading Math Science Ranking Shanghai, China 556 600 575 1 Korea, Rep. 539 546 538 2 Finland 536 541 554 3 Hong Kong SAR, 533 555 549 4 China Singapore 526 562 542 5 Japan 520 529 539 8 Netherlands 508 526 522 10 66 Poland 500 495 508 15 Denmark 495 503 499 24 Hungary 494 490 503 26 Spain 481 483 488 33 Bulgaria 429 428 439 46 Thailand 421 419 425 50 Indonesia 402 381 383 57 Source: OECD 2009. Table compiled by the author using data from http://www.pisa.oecd.org/dataoecd/54/12/46643496.pdf. With regard to school autonomy and accountability, an analysis of 2009 PISA data by the Organisation for Economic Co-operation and Development found that the most successful school systems in terms of academic achievement are those that give schools pedagogical and budgetary autonomy and that enforce regular student assessments. Moreover, to be successful, school systems do not necessarily have to tolerate education inequality. Specifically, the study found that  Economies where schools had more autonomy over teaching content and student assessment tended to perform better on the PISA test.  PISA scores in schools with autonomy over resource allocation were higher than in schools with less autonomy (in economies where schools publicized their test results).  Economies with standardized student assessment tended to do better than those without standardized student assessment.  PISA scores among schools with students from varying social backgrounds differed less in economies that use standardized tests for assessing student performance than in than in economies that did not use them.  Economies that allowed schools to compete for the best students did not obtain better PISA results than those that sought to reduce inequality in educational access. Key Indicators for the Development of School Autonomy 67 and Accountability Five SBM indicators can serve as a benchmark for judging policy intent and progress in the introduction of school autonomy and accountability: 1. School autonomy in budget planning and approval 2. School autonomy in personnel management 3. Participation of the school council in school finance 4. Assessment of school and student performance 5. School accountability to stakeholders Each of these indicators has a set of subindicators that make it possible to judge how far along each school is in the process of implementing each indicator (see table 6.3). Table 6.3. Indicators and Subindicators of Autonomy and Accountability 1. School autonomy in budget planning and approval 1A. Does the school director have the legal authority to manage its operational budget? 1B. Does the school director have the legal authority to set and manage staff and teacher salaries? Does the school director have the legal authority to raise other funds in addition to the transfers received 1C. from national or local governments? 2. School autonomy in personnel management 2A. Are decisions to hire and fire teachers managed by the school director? Do school councils (which may include the school director) have the legal authority to hire and fire 2B. teachers? 2C. Do school councils have the legal authority to hire and fire the school director? 3. Participation of the school council in school finance 3A. Does the school council assist the school director in the preparation of the school budget? 3B. Do school councils have the legal authority to approve the school budget? Is there a manual or set of instructions describing the participation of the school councils in the preparation 3C. of the school budget? 3D. Do school councils have the legal authority to supervise the implementation of the school budget? 68 If school councils participate in the preparation and approval of the school budget, is this budget used as an 3E. input in the general budget prepared by the Ministry of Education? 4. Assessment of school and student performance 4A. Do schools perform yearly assessments of school and student performance? Does the school use student assessments to make administrative or pedagogical decisions aimed at 4B. improving school and student performance? 4C. Do schools perform yearly assessments of learning outcomes using standardized tests? Are the results of standardized tests used to make administrative or pedagogical decisions aimed at 4D. improving school and student performance? 4E. Are the results of the assessment of school and student performance made public to parents? 5. School accountability to stakeholders Is there a manual regulating how the school council can use of the results of the yearly assessments of 5A. school and student performance? Is the school’s assessment of school and student performance part of a national or regional assessment 5B. system? 5C. Does the school use the assessments to compare school performance with schools in similar conditions? Do school councils have the legal authority to hire external auditors to carry out financial audits at the 5D. school? Is there a manual to guide the school council in how to use financial audits to evaluate school 5E. performance? For each economy in Asia, we assigned an overall SBM score of 1 (low), 2 (medium), or 3 (high). We also classified the progress of each country in achieving each subindicator according to four categories: latent, emerging, established, and advanced. A latent score indicates that the policy behind the indicator is not yet in place. An emerging score indicates that implementation of the program or policy is in progress. An established score indicates that the program or policy is in operation and meets the minimum standards. An advanced score indicates that the program or policy is in operation and reflects best practice. These scoring rules apply to most countries, but some high-performing countries in terms of learning outcomes may have formal mechanisms for teacher recruitment, deployment, and retention that function well enough. Parents in those countries may not be active participants in school governance, and their direct involvement may be considered a second-best solution, since the formal accountability systems work very well. As a result, some subindicators for these countries could be implicitly classified at the advanced stage. School Autonomy and School Accountability in East Asia The results of the SABER (System Assessment and Benchmarking Education for Results) exercise for 14 69 East Asian economies are summarized in table 6.4. Results show that budgetary autonomy at the school level is well established in most of the economies assessed but may still require some changes to be fully achieved. However, the fact that the majority of the economies in the region are at the established level of autonomy in budget management is a good sign. For the indicator of school and teacher assessment, several of the economies are at the established and advanced stages, while only a few are still at the emerging stage. This finding suggests that, with only a little effort, most economies could introduce these assessments, which would enable them to compare the educational performance of their schools and teachers and would foster accountability throughout the school system. The indicators for parental participation in the school council and of the enforcement of accountability are at an intermediate stage of development, with some economies doing very well but several others still at the emerging stage. In the case of the indicator for autonomy over personnel decisions, most economies are at only the latent or emerging stage. Table 6.4. Indicators of School Autonomy and Accountability in East Asia Budget Personnel Economy Participation Assessment Accountability Overall autonomy autonomy Established Emerging Latent Established Emerging Emerging Cambodia       Established Emerging Established Established Established Established China       Hong Kong Established Advanced Established Established Established Established SAR, China       Established Emerging Established Advanced Established Established Indonesia       Established Emerging Latent Emerging Emerging Emerging Japan       Established Latent Established Established Emerging Emerging Korea, Rep.       Established Emerging Emerging Emerging Emerging Emerging Lao PDR       70 Established Emerging Advanced Established Advanced Established Malaysia       Established Emerging Established Advanced Established Established Mongolia       Emerging Latent Emerging Emerging Latent Latent Philippines       Shanghai, Established Established Emerging Emerging Emerging Emerging China       Established Latent Advanced Advanced Established Established Singapore       Established Latent Advanced Advanced Established Established Thailand       Established Latent Latent Established Emerging Emerging Vietnam       Source: Prepared by the author. References Arcia, G., H. A. Patrinos, E. Porta, and K. Macdonald. 2011. “School Autonomy and Accountability.” Working paper for the series Benchmarking Education Systems for Results: Regulatory and Institutional Framework, Human Development Network, Washington, DC: World Bank. Barrera-Osorio, F., T. Fasih, H. A. Patrinos, and L. Santibañez. 2009. Decentralized Decision-Making in Schools: The Theory and Evidence on School-Based Management. Washington, DC: World Bank. Bruns, B., D. Filmer, and H. A. Patrinos. 2011. Making Schools Work: New Evidence on Accountability Reforms. Washington, DC: World Bank. Gertler, P., H. A. Patrinos, and M. Rubio-Codina. 2007. Impact Evaluation for School-Based Management Reform. Washington, DC: World Bank. ———. 2006. “Education Decentralization in Three Asian Societies: Japan, Korea, and Hong Kong.” Journal of Educational Administration 44 (6): 590–603. OECD (Organisation for Economic Co-operation and Development). 2009. PISA 2009 Results: What Students Know and Can Do. Paris: OECD. Vegas, E., and I. Umansky. 2005. Improving Teaching and Learning through Effective Incentives: What Can We Learn from Education Reforms in Latin America? Washington, DC: Human Development Network, World Bank. 71 72 CHAPTER 7 Engaging the Private Sector Laura Lewis SUMMARY Most school systems are predominantly state funded and therefore subject to top-down policy making that can sometimes stifle innovation and change at the local level. The involvement of nonstate players such as nongovernmental organizations (NGOs), communities, faith-based organizations, trade unions, private companies, and individual practitioners can therefore improve school performance through competition and autonomy—provided that the government ensures accountability mechanisms for all schools. This chapter outlines a framework for ensuring effective school provision, regardless of the type of provider—be it public or private—and then shows how the private sector can improve the quality of education and allow parents and students increased school choice. In terms of the SABER (System Assessment and Benchmarking Education for Results) pilot, the analysis of nonstate engagement in education in East Asia shows that although the public sector remains the dominant player, the private sector is increasingly active. Evidence suggests that economies in the region are willing to innovate in order to expand access and improve quality. However, a supportive and secure regulatory environment is crucial to engaging the private sector 73 effectively. The Role of the Private Sector in Education The Organisation for Economic Co-operation and Development (OECD) in its recent report on the Programme for International Student Assessment (PISA) 2009 defined superior education performance as high participation, high quality, high equity, and high efficiency (OECD 2010b). For governments to achieve this objective, Schleicher and Stewart (2009) have argued that they must ensure that education promotes the following: 1. High universal standards: by focusing on outcomes, raising aspirations, and defining educational excellence 2. Accountability and autonomy: by responding to local needs and strengthening accountability relationships 3. Teacher professionalism: by recruiting strong teacher candidates, preparing them well, and offering them ongoing professional development 4. Personalized learning: by ensuring that teachers are responsible for engaging constructively with the full range of students’ interests, capacities, and socioeconomic contexts Although the public sector remains the dominant player in education in East Asian economies, private sector programs and initiatives can complement state efforts to provide high-quality education that is accessible to all. There are ways in which the public and private sectors can join together to complement each other’s strengths in providing education services , meeting education goals, and improving learning outcomes. Given the ability of the private sector to innovate, these partnerships can be tailored to meet the needs of specific communities. A Model for Education Service Provision The recent World Bank publication by Patrinos, Barrera-Osorio, and Guáqueta (2009) cites evidence that engaging the private sector can  Create competition in the education market  Promote flexibility—for example, with teacher contracts that match the supply of education with demand  Enhance efficiency when providers are selected by means of an open bidding process and when contracts include measurable outcomes and cost-effectiveness measures  Increase risk sharing between the government and the private sector Building on the key elements of an effective system outlined by Patrinos, Barrera-Osorio, and 74 Guáqueta (2009); Schleicher and Stewart (2009); and the World Bank (2004) model of service provision, this chapter outlines a model developed by the World Bank that measures four key elements of effective school provision:  Autonomy: Autonomous schools that can tailor teaching and learning to meet the needs of all of students and that have control over the quality of educational professionals in the school  Competition: A competitive environment within which schools offer a range of models to meet the needs of all students  Parental engagement: Informed parents who can use both their voice and, if the system permits, their choice to ensure that the schooling supplied to their children is of the highest quality and who also hold the government accountable through the political process, whether at the local, regional, or national level  Accountability: A strong and transparent accountability system in which autonomy is accompanied by responsibility and policy makers hold all schools to account for the quality of their education Types of Private Sector Engagement The engagement of the private sector in education cannot be seen in isolation from other education policies and government interventions. The government must decide how it would like the private sector to participate and how such participation fits in its overall education strategy. The model does not advocate any specific arrangement for private sector engagement—there is no right answer. Each country should use the private sector in a way that suits its specific needs. The range of options open to governments includes • Private schools: Schools that are owned by the private sector (private businesses, churches, religious groups, NGOs, foundations, charities, or individuals) and are financed privately, typically through fees (in other words, they receive no public subsidies). • Privately funded schools: Schools that are owned and managed schools by the private sector (private businesses, churches, religious groups, NGOs, foundations, charities or individuals) that receive funding from the government, but this funding is not specified in a contract on a per student basis. • Privately contracted schools: Schools owned and managed by the private sector 75 that receives funding from the government and the funding is specified in a contract on a per student basis. The transfer of public funds to the school requires that the school satisfy specified conditions. • Privately managed schools: Schools owned by the government but operated or managed by private organizations (for example, charter schools, academies, and concession schools). • Market-contracted schools: Public schools, privately contracted schools, or privately managed schools that are implicitly contracted by the student in that public funding follows the students to their school of choice (for example, vouchers). Evidence A body of evidence from around the world shows how effective these different models of private sector engagement have been in terms of academic performance and cost-effectiveness (Lewis and Patrinos 2011). Private Schools The evidence with respect to private schools is mixed. The results of PISA 2009 suggest that there is no difference in the performance of students in public and private schools after controlling for their backgrounds. Another argument is that private schools increase competition, thus forcing all schools to improve. The degree to which this happens depends on the extent to which public schools view private schools as a threat for drawing students away. Nannestad (2004) analyzed whether private schools in Denmark improved the quality of local municipal schools and found no evidence that their existence increased competition. This finding is in direct contrast to that of Sandström and Bergström (2004), who found that increased competition arising from the existence of private schools in Sweden led to improvements in the standards of public schools. Nannestad partly explained this difference as being due to the different perceptions of the “threat” posed by private schools to the public school system in each country. As a consequence, one would expect the incentive effects of competition from private primary schools on actors in the public school system in Sweden to be stronger than they are in Denmark. This could partly explain why competition from private primary schools improves quality in public schools in 76 Sweden but not in Denmark (Nannestad 2004). Privately Funded Schools McEwan (2000) evaluated the impact of privately funded schools in Argentina and showed that Catholic school attendance lowers the probability of students repeating a grade by 4 percentage points. Privately Contracted Schools The Department of Education in the Philippines uses Education Service Contracts with qualified private high schools. These schools are paid a fixed amount per grantee to accept students who cannot be accommodated in nearby, highly congested public high schools but who are willing to pay the unsupported portion of their private school tuition and all other fees. An impact evaluation of the program showed that students in these contracted private schools scored 60 points higher on the TIMSS (Trends in International Mathematics and Science Study) test than did students who attended public schools. Even after controlling for student and family background, students at private schools scored 33 points higher (World Bank 2011). In Pakistan, Foundation-Assisted Schools are low-cost, privately contracted schools. To receive public funding, the schools must meet strict criteria. The program has had a positive influence on total test scores of between 0.361 and 0.457 standard deviations, particularly in math and science at grade 5 and in English at grade 8 (Barrera-Osorio 2010). Lefebvre and Merrigan (2009) examined privately contracted schools in Quebec. They found that the effect of changing from a public grade school to a private high school increases the percentile rank of a student’s math score by between 5 and 10 points and by between 0.13 to 0.35 of a standard deviation, depending on the specifications and samples. Privately Managed Schools Several evaluations of publicly funded and privately operated and managed public schools have found positive effects. Hoxby and Rockoff (2004) showed that students who won the lottery to attend a charter school in the United States achieved scores that were 11 percentage points higher in reading and 10 percentage points higher in math than those who did not win the lottery. Booker et al. (2008) found that initially students in charter schools perform worse on standardized tests than their peers in public schools, but that after three years, the scores of charter school students catch up with those of their public school peers. Barrera-Osorio (2007) reached similar conclusions for Colombia. He found that students in privately managed schools achieved higher test scores at the end of their basic education (grade 11) than those in public institutions. Clark (2009) found that schools in the United Kingdom that converted to grant-maintained status achieved as much as a quarter of a standard deviation improvement in pass rates on standardized examinations. Market-Contracted Schools Numerous empirical evaluations of voucher schemes have also found positive effects. In Colombia, 77 Angrist, Bettinger, and Kremer (2006) found that vouchers improved the test scores of both average students and those in the highest decile. Himmler (2007) found positive links between the intensity of competition that a voucher scheme creates and academic achievement in the Netherlands. Patrinos (forthcoming), who also examined market-contracted schools in the Netherlands, found private school effects on math, reading, and science achievement of 0.17, 0.28, and 0.18, respectively. The results for Chile’s voucher system, however, are mixed. Gallegos (2004) found that 1.0 standard deviation in private enrollment generates about a 0.20 standard deviation in test scores and 0.24 standard deviation in productivity. However, Contreras, Bustos, and Sepulveda (2008) found that, after they controlled for self-selection, there was no difference between public and private schools. Types of Private Sector Education in East Asia East Asian education systems all employ some private provision but in different ways. Private schools operate in all East Asian education systems. In China and Shanghai, China, privately funded schools operate with no explicit government contract; in Indonesia, Japan, the Republic of Korea, the Philippines, and Singapore, private schools are explicitly contracted and funded. Both Hong Kong SAR, China, and Thailand use vouchers to increase school choice (table 7.1). Table 7.1. Comparison of Types of Private Sector Engagement across East Asia Economy Private Privately Privately Privately Market funded contracted managed contracted Cambodia  Lao PDR  Malaysia  Mongolia  78 Vietnam  China   Shanghai, China   Indonesia    Japan    Korea, Rep.    Philippines    Singapore    Hong Kong SAR, China      Thailand      Source: Author’s compilation Figure 7.1 combines the types of private sector provision and enrollment data to outline how the private sector is engaged in the education sector in East Asia, the OECD, and other regions. The types of private sector engagement outlined in this chapter are not exclusive; however, the five key questions listed here can help discern the degree to which they are used. Answering “yes” to questions 1 to 3 automatically moves a country to the right of the figure as they are sequential. Questions 4 and 5 are nonsequential, so answering “no” to 4 does not mean an economy could not also engage the private sector by using market-based contracts. The key questions are as follows: 1. Are private schools allowed to operate? 2. Is public funding for private schools allowed? 3. Is there a contract that governs the conditions under which funds are transferred to nonpublic schools? 4. Is the private operation and management of public schools permitted? 5. Does public funding follow students to their school of choice? Figure 7.1. Private Enrollment and Types of Private Sector Engagement 75 79 Netherlands Belgium 55 Chile 50 Private enrollment 45 Indonesia Spain 30 Korea, Rep. 25 Phillipines France 20 Denmark Japan New Zealand Thailand 15 Portugal United States Shanghai, China Hong Kong Brazil Vietnam China Sweden 10 Mexico Greece Hungary England Scotland Germany Italy Mongolia Slovak Rep. Malaysia Norway Poland Estonia Singapore Finland Cambodia Ireland Lao PDR Czech Rep. Slovenia 0 Private schools Private funded Private contracted Private management Market contracted Type of Engagement Source: OECD 2010a, choice indicators; United Nations Educational, Scientific, and Cultural Organization Institute of Statistics; SABER East Asia Benchmarking Study; Shanghai Statistical Yearbook. For East Asian economies, information was provided by in-country experts. For the OECD and others, we consulted the OECD’s Education at Glance 2010 school choice indicators (OECD 2010a); however, because the OECD choice indicators do not cover question 4, experts were also interviewed. Combining types of private sector provision with private enrollment figures revealed a wide variation in the scale of private sector engagement across the world. Indonesia and Korea have a high level of private enrollment—comparable to the levels found in Spain. All other East Asian economies have over 10 percent private enrollment except for Cambodia, Lao People’s Democratic Republic, Malaysia, Mongolia, and Singapore. Even in Finland, where the quality of public schools is very high, the private sector plays a role, albeit a small one. Figure 7.1 also highlights the subjectivity involved in answering survey questions. In Ireland, 50 percent of secondary schools are aided by the Department for Education and Skills, of which 49 percent are Catholic-denominated schools. This statistic is not shown in OECD data because these schools are considered to be public. As a result, the private sector seems to play only a limited role in Ireland. This situation contrasts with that of France, where the extent of the role played by the Catholic Church in providing education is explicitly recognized in the data: France has one of the largest Catholic education systems in the world (Dreycke 2007: 1). Capturing the Potential Benefits of Private Sector Education 80 Provision Increasing the role of the private sector can have several potential advantages over the traditional public delivery of education. Whether these benefits are in practice realized depends greatly on how well designed the partnership between the public and private sector is, on the regulatory framework of the country, and on the capacity of the government to oversee and enforce its contracts and partnerships with the private sector (Patrinos, Barrera-Osorio, and Guáqueta 2009). We have developed a tool that categorizes East Asian economies on the basis of the policies they have implemented to regulate the private sector. We analyze these policies to see how well each supports the four key regulatory outcomes from our model of effective school provision: competition, accountability, autonomy, and information. Table 7.2 outlines the regulatory outcomes and poses questions that will examine the extent to which governments are currently meeting them. Table 7.2. Regulatory Outcomes Regulatory outcome Description How well does the system support different players in entering the Competition market, and to what extent does it ensure that there is no selection that constrains the efficiency of the market? What standards are private players required to meet and what is Accountability the accountability process? What degree of autonomy are schools given over curriculum, Autonomy delivery, and teacher policies? What information is available to parents when making their school Information choice, and to what extent are financial contributions hindering this selection? Source: Author’s compilation 81 Table 7.3 compares the extent to which each East Asian economy currently regulates private schools. There is a wide variation across the region, particularly in the degree of competition. In Cambodia and Korea, private school fees are set by the government, while in China; Hong Kong SAR, China; Shanghai, China; Thailand; and Vietnam, the government imposes a ceiling on tuition fees. Indonesia, Japan, Lao PDR, Malaysia, and the Philippines do not limit competition through regulated fees, nor do they restrict competition on the basis of ownership—both nonprofit and for-profit schools are allowed. All economies in the region set minimum standards for private schools to be established, and all except Cambodia have external inspections of schools. Korea, Malaysia, the Philippines, and Vietnam are considered to be advanced because the inspection regime is risk based; the frequency of future inspections depends on the results of the previous inspection. The tool does not currently assess the quality of the inspection regime or the professionalism of the inspectors. Most East Asian economies do not supply information to parents to enable them to make an informed choice when choosing a private school. In Cambodia; Japan; Lao PDR; Malaysia; Mongolia; the Philippines; Shanghai, China; and Vietnam, governments are not required to supply performance data to parents and students. In Hong Kong SAR, China, this requirement exists only for primary schools. In China, Indonesia, Korea, and Thailand, governments must supply this information at both the primary and secondary levels. The quality of the information supplied was not assessed in the tool, which is why both are considered advanced. (Autonomy is not included for private schools because it is considered outside the remit of public policy.) Table 7.3. The Extent of Regulation of Private Schools Indicator Competition Accountability Information Latent Emerging Latent Cambodia    Emerging Advanced Established China    Hong Kong SAR, Emerging Advanced Emerging China    Advanced Advanced Advanced Indonesia    Advanced Established Latent Japan    Latent Advanced Established Korea, Rep.    Established Established Latent Lao PDR    Advanced Advanced Latent Malaysia    Established Advanced Latent Mongolia    Advanced Established Latent Philippines    Emerging Established Latent 82 Shanghai, China    Emerging Established Established Thailand    Emerging Advanced Latent Vietnam    Source: Author’s compilation China and Shanghai, China, fund private schools with no explicit contract. Recent reforms have expanded the number of privately funded schools, but competition is restricted because the government controls tuition ceilings for nonfunded students and because schools are allowed to set selective admission criteria, thus allowing them to select the students who are the easiest to teach. In general, the Chinese government releases information on school performance, but this does not happen in Shanghai, China, specifically, although there are mandatory examinations at the secondary level and all schools are inspected annually. In China and Shanghai, China, autonomy is also inhibited by the requirement that schools have to follow a standardized curriculum. Seven of the East Asian education systems analyzed here use privately contracted schools (table 7.4). In Hong Kong SAR, China; Indonesia; Japan; Korea; the Philippines; Singapore; and Thailand, the contract does not require privately contracted schools to achieve certain outcomes (Singapore uses merit-based student admission criteria to enter the privately contracted schools.) In six of the seven education systems, privately contracted schools are constrained by contractual requirements on the inputs that they can use. Autonomy and competition can also be restricted if—as a precondition for public funding—governments require private schools to adhere to the same input standards that are applied in public schools, thus reducing their operational flexibility (Lundsgaard 2002). Table 7.4. Privately Contracted Schools: Contract Comparisons Contract based on Economy Inputs Outputs Outcomes Hong Kong SAR, Yes, support Yes, students No China services enrolled Yes, instruction Yes, students Indonesia and support enrolled No services Yes, instruction Yes, students Japan only enrolled No 83 Yes, instruction Yes, students Korea, Rep. and support enrolled No services Yes, instruction Yes, students Philippines only enrolled No Yes, students Singapore No enrolled No Yes, instruction Yes, students Thailand and support enrolled No services Source: Author’s compilation Thailand and Hong Kong SAR, China, both use market-contracted schools. Thailand uses vouchers targeted to certain geographic areas and to students from underprivileged socioeconomic backgrounds. Competition is hindered because schools often do not receive their funding on time, which makes it difficult for them to plan. They also receive less funding than do their public school counterparts. Parents are also restricted because they have to make compulsory financial contributions toward their children’s education. Thailand has an established accountability system in which the government sets minimum standards for teachers and annually inspects schools. Students at both the primary and the secondary level are required to take a mandatory exam to assess their academic progress. Schools are also required to report to the government as a condition for the continuation of their funding. However, despite this accountability system, school autonomy is hindered by restrictions imposed by the central government on schools regarding the curriculum and staffing. Conclusions The evidence shows that East Asian economies are willing to innovate to expand access and improve quality. However, while it is possible and often desirable to engage the private sector in this process, certain prerequisites are essential—in particular, a supportive and secure regulatory environment. Governments could ensure that the regulatory environment 84  Supports school autonomy and allows innovation in delivering education to meet individual students’ needs  Encourages multiple providers so there is a competitive environment  Gives parents access to information on the quality of the education system to ensure that they can make informed decisions and exercise their parental choice or their parental voice  Has a rigorous and transparent accountability system that engages all stakeholders References Angrist, J., E. Bettinger, and M. Kremer. 2006. “Long-Term Educational Consequences of Secondary School Vouchers: Evidence from Administrative Records in Colombia.” American Economic Review 96 (3): 847–62. Barrera-Osorio, F. 2007. “The Impact of Private Provision of Public Education: Empirical Evidence from Bogotá’s Concession Schools.” Policy Research Working Paper 4121, World Bank, Washington, DC. ———. 2010. ”Accountability: Lessons from a Test-Based Public Subsidy Program for Low-Cost Private Schools.” World Bank, Washington, DC. Booker, K., S. M. Gilpatric, T. Gronberg, and D. Jansen. 2008. “The Effect of Charter Schools on Traditional Public School Students in Texas: Are Children Who Stay Behind Left Behind?” Journal of Urban Economics 64 (1): 123–45. Clark, D. 2009. “The Performance and Competitive Effects of School Autonomy.” Journal of Political Economy 117 (4): 745–83. Contreras, D., S. Bustos, and P. Sepulveda. 2008. “When the Schools Are the Ones That Choose: Policy Analysis of the Screening in Chile.” University of Chile, Santiago. Department for Education and Skills, Ireland. “Key Statistics about the Department’s Customers.” Department for Education and Skills, Dublin. http://www.education.ie/home/home.jsp?maincat=17216&pcategory=17216&ecategory=1 85 7241&language=EN. Dreycke, H. 2007. “Catholic Schooling in France: Understanding ‘La Guerre Scolaire.’” International Handbook of Catholic Education: Challenges for School Systems in the 21st Century, ed. G. Grace and J. S. J. O’Keefe, 329–46. Dordrecht, Netherlands: Springer. Gallegos, F. 2004. “School Choice, Incentives, and Academic Outcomes: Evidence from Chile.” Boston: MIT Press. Himmler, O. 2007. “The Effects of School Choice on Academic Achievement in the Netherlands.” Georg-August-Universität Göttingen, Sweden. Hoxby, C., and J. E. Rockoff. 2004. “The Impact of Charter Schools on Student Achievement.” Harvard University, Cambridge, MA. Lefebvre, P., and P. Merrigan. 2009. “Public Subsidies to Private Schools Do Make a Difference for Achievement in Mathematics: Longitudinal Evidence from Canada.” Working Paper 09-35, Université du Québec à Montréal and CIRPÉE. Lewis, L., and H. A. Patrinos. 2011. “Framework for Engaging the Private Sector in Education: System Assessment and Benchmarking Education for Results (SABER).” World Bank, Washington, DC. Lundsgaard, J. 2002. “Competition and Efficiency in Publicly Funded Services.” OECD Economic Studies 35: 79–128. McEwan, P. 2000. “Private and Public Schooling in the Southern Cone: A Comparative Analysis of Argentina and Chile.” Occasional Paper 11, National Center for the Study of Privatization in Education, Teachers College, Columbia University, New York. Nannestad, P. 2004. “Do Private Schools Improve the Quality of Municipal Schooling? The Case of Denmark.” Presented at the European Public Choice Society Annual Meeting, Berlin, April 15–18. http://www.diw.de/documents/dokumentenarchiv/17/41438/Paper-108.pdf. OECD (Organisation for Economic Co-operation and Development). 2010a. Education at a Glance. Paris: OECD. ———. 2010b. PISA 2009 Strong Performers and Successful Reformers in Education: Lessons from PISA for the United States. Paris: OECD. Patrinos, H. A ., F. Barrera-Osorio, and J. Guáqueta. 2009. The Role and Impact of Public-Private Partnerships in Education. Washington, DC: World Bank. Patrinos, H. A . Forthcoming. “Private Education Provision and Public Finance: The Netherlands.” Education Economics. Sandström, F. M., and F. Bergström. 2004. “School Vouchers in Practice: Competition Will Not Hurt You.” Journal of Public Economics 89 (2–3): 351–80. Schleicher, A., and V. Stewart. 2009. “International Benchmarking.” Asia Society, New York. http://asiasociety.org/education-learning/learning-world/international-benchmarking. 86 World Bank. 2004. World Development Report 2004: Making Services Work for Poor People. Washington, DC: World Bank. ———. 2011. Philippines: Private Provision, Public Purpose—A Review of the Government’s Education Service Contracting Program. Washington, DC: World Bank. CHAPTER 8 Vocational Tracking Harry Anthony Patrinos 1 Prime Minister: Education in this country is a disaster. We re supposed to prepare children for work. Most of Permanent Secretary: the time they re bored stiff. I should ve thought that being bored stiff was an excellent preparation for work. —Antony Jay and Jonathan Lynn, Yes, Prime Minister, Episode 7, Series 2, BBC Television Program, originally aired 87 January 21, 1988 SUMMARY Reducing early tracking often leads to more equity, higher test scores, and more market-relevant results. This chapter presents case studies from Europe to support this assertion and uses a benchmarking tool to test the effectiveness of tracking policies in East Asia. Results of the East Asian pilot showed that tracking policies varied across the region substantially. For example, the Republic of Korea has no tracking (box 8.1), whereas high school education ends at grade 10 in the Philippines, and they both have a small vocational education sector. Japan, Mongolia, and Shanghai, China, all have late selection and continued access to tertiary education. Malaysia has late selection, while Lao PDR, Singapore, Thailand, and Vietnam, have early selection. In practice, however, few students switch between the vocational and academic tracks. Does Tracking Improve Learning Outcomes? The emphasis on vocational education in secondary schooling has been a mainstay of development policy for many decades. It is often argued that vocational skills are needed to encourage job creation and increase productivity. For decades, the received wisdom was that vocational education was necessary for a country to modernize and acquire the technical skills needed for economic development. As a result of these beliefs, several policies have been adopted to increase the proportion of students in vocational education programs. These policies have included policies that create youth employment schemes, policies that introduce technological knowledge courses, and policies that channel less able students into learning skills necessary to become mid-level technicians. Recent evaluations of these policies and programs have highlighted problems with the vocational argument, however. One of the major problems of this approach is opaque selection criteria. The reality is that in many cases a segment of the population is “selected” by the system for vocational studies irrespective of capability or personal choice. Moreover, there is little evidence to show that vocational studies lead to better job prospects. Vocational training while at school does not guarantee that students will acquire job-relevant skills or useful integration into the labor market. In sum, this approach has been proven to involve higher costs and lower benefits and to induce higher inequality than does traditional secondary academic schooling (Foster 1965; Psacharopoulos 1991, 1987). 88 Box 8.1. No Tracking in Korea Korea’s egalitarian education strategy successfully expanded opportunities for primary, secondary, and higher education by setting high standards for all and ameliorating differences across regions and socioeconomic groups. Introduced in 1951, Korea’s single -track educational system, which aimed to promote democracy and expand education for all, has seen little change since its inception. Today the system still ensures that anyone can receive elementary and secondary education and continue on to university. After nine years of schooling, students make a decision about what type of upper-secondary education —academic or vocational high school —they wish to pursue. Korea achieved universal primary and secondary education by the 1960s and 1970s, respectively, Key to this achievement were the No-Test Middle School Entrance System (1968) and the High School Equalization Policy (1974). The premise behind these programs was to encourage entry into middle school. The enrollment rates of elementary students into middle school increased from 51 percent in 1970 to 95 percent in 1980. The success of the No-Test program resulted in bottlenecks at the high school entrance stage. To cater for the huge influx of middle school students who wished to go on to high school, the High School Equalization Policy was introduced in 1974. The government attempted to reduce discrepancies in facilities, teachers, and finances among schools. As a result, high school enrollment rates soared from 41 percent in 1975 to 96 percent in 2000. To reduce coverage gaps, the government also provided targeted support for low-income and geographically marginalized students, providing free textbooks and free compulsory middle school education. Source: Sungmin Park Studies have found that  Early selection into vocational education leads to inequality by channeling poor, less able, and minority students into the less prestigious vocational track.  Early tracking reduces mean performance.  Tracking reinforces ethnic inequalities in educational attainment.  Vocationally oriented schooling systems do not prepare students for active citizenship. In addition, recent research shows that vocational students score poorly in subjects such as reading and math, which are precisely the skills that employers are demanding. Another issue is that students streamed into vocational studies are sometimes not allowed to switch later to another track, thus limiting their future opportunities and reducing their options for postsecondary schooling. Also, the earlier students get into the vocational track, the more limited their opportunities are for entering tertiary education. In several high-performing countries, evidence shows that extending comprehensive, nontracked schooling not only leads to significant improvements in the learning outcomes of those students who would otherwise have been enrolled in vocational secondary programs, but also causes overall learning outcomes to improve. In high-performing countries, education systems are flexible in that they allow students to switch back and forth between vocational and academic schools. Figure 8.1 shows the range of ages at which students are first selected to enter vocational tracks in Organisation for Economic Co- operation and Development (OECD) countries. 89 Figure 8.1. Age of First Selection into Vocational Track, OECD Countries USA 16 UK Sweden Spain Norway New Zealand Iceland Finland Denmark Australia Switzerland 15 Portugal Source: OECD Poland Japan Ireland Greece France Korea, Rep. 14 Italy Luxembourg 13 90 Canada Netherlands 12 Mexico Belgium Turkey 11 Slovak Rep. Hungary Czech Rep. Germany 10 Austria 0 2 4 6 8 10 12 14 16 Source: OECD The Rationale for More Schooling There is strong causal link between cognitive ability and labor market outcomes (OECD 2010). In addition, inadequate schooling lowers not only an individual’s future income, but also the returns to society from education. Because early tracking often prevents students from pursuing further education opportunities, the potential academic returns from higher education can also be limited by these policies. Education systems need to focus on producing the basic cognitive skills that employers demand—the type of “learning that begets learning” (Cunha and Heckman 2006). If schools do not teach the basics well, then investments in education will not translate into economic growth and poverty reduction. Thus, improving the quality of education in the short term will increase labor market success in the medium term (Bertschy, Cattaneo, and Wolter 2009). The Evidence in Favor of Reducing Early Tracking Reducing early tracking will lead to more equity, higher test scores, and more market-relevant results. Expanding comprehensive schooling—that is, reducing early vocational tracking—instills in students a demand for academic schooling, increases student effort, and ultimately results in higher test scores. 91 This happens because in academic schooling students spend more time on task, are subject to higher expectations, spend more hours learning tested subjects, and benefit from more opportunities to move onto higher education. Cross-country comparisons show that the variance in test scores is higher in countries where tracking takes place at an early age (Hanushek and Woessmann 2006). At the same time, early tracking seems to have generally negative effects on mean performance, though the evidence on these effects is less consistent (Brunello and Checchi 2007; Waldinger 2006). Experiences with Tracking: Case Studies from the European Union The economic transition in Eastern Europe led to a significant restructuring of school systems, including a decline in the share of students on the vocational track. In Romania, educational reform led to the postponement of tracking. As a result, more students from poor rural areas and with less educated parents are now significantly more likely to graduate from the academic track and become eligible to apply for university study. However, the postponement of tracking was not accompanied by an increase in the number of slots available at the university level, which means that access to higher education for disadvantaged groups did not change very much despite the fact that more disadvantaged students were eligible (Malamud and Pop-Eleches 2008). In Lithuania, there was an entire education system reform after independence. This reform included moving away from the old “Austrian Gymnasium” system of early selection and the introduction of extended compulsory comprehensive education and led to significant improvements in student test scores. Lithuania improved its TIMSS (Trends in International Mathematics and Science Study) scores by 34 points in math and 55 points in science—half of a standard deviation—between 1995 and 2007 (OECD 2002). In Poland, before the transition period, there was an eight-year primary school system that was followed by early vocational tracking. This system was changed to a nine-year comprehensive structure with tracking delayed by one year. Evaluations show that would-be vocational school students improved their learning outcomes by almost one standard deviation as a result of the reform. In fact, Poland went from being below the OECD average in 2000, to eighth place worldwide in reading scores on the PISA (Programme for International Student Assessment) test in 2006 (see box 8.2). In Germany, students must make an important decision about which educational track to follow at an early stage—after primary school at the age of 10. This practice has been shown to limit educational and career opportunities for many, primarily those from families with less educated parents. Parental background is strongly associated with what track is taken by a child in secondary school—and with his 92 or her subsequent educational achievements. This association between parental class and educational choice translates into substantial earnings differentials later in life (Dustmann 2004). In Finland, students were streamed at the age of 10—as in Germany and Austria today—until comprehensive school reform from 1972 to 1977 adjusted the streaming age to 16. The reform therefore replaced the old two-track system with a uniform nine-year comprehensive schooling, thus significantly reducing the degree of heterogeneity in Finnish primary and secondary education. An assessment of the reform found that it had a small positive effect on test scores, with significant increases in the scores of students from families where parents only had a basic education (Pekkarinen, Uusitalo, and Kerr 2009; see figure 8.2). In Northern Ireland, the academic track was expanded in 1978, leading to a 15 percentage point increase in the number of students entering secondary school. The number of students graduating from high school subsequently increased by 12 percentage points, and there was an increase of a similar magnitude in national examination scores (Maurin and McNally 2007). Box 8.2. The Impact of Delayed Tracking in Poland In Poland, delayed tracking improved test scores by taking students out of vocational programs with limited opportunities for jobs and future learning, by increasing their motivation to learn, and by exposing them to more hours of academic instruction. In 1999, Poland introduced education reforms that converted the traditional eight years of primary schooling followed by early vocational tracking, into a system that offered six years of primary education followed by three years of lower-secondary education. Only after nine years of schooling would a decision be made about what type of upper-secondary education —academic or vocational—a student would take. In other words, the new system postponed the choice of education track at the secondary level by one year. This reform was followed by a significant improvement in the performance of Polish students in international achievement tests. In math, Poland improved its PISA score from 470 in 2000 to 490 in 2003 and to 495 in 2006. This increase is equivalent to one-quarter of a standard deviation. Reading scores also steadily improved over time, from 479, to 490, to 508, an increase of 0.28 of a standard deviation. In science, Polish students scored 483, 498, and 498 over the same periods, an increase of 0.15 of a standard deviation. Among the countries in Eastern Europe that participate in PISA, Poland is today one of the best performers, with a solid record of improvement over time. These changes also constitute the largest improvement over time by a traditionally low-end performer. Measuring Impact 93 To test for the impact of the 1999 policy change in on Polish test scores over time, analysts used scores from the group of students who took the PISA 2000 test as a baseline —since most of the existing students were continuing their lower-secondary schooling under the old system —and then selected comparable individuals from a control group and compared the change in outcomes over time for likely vocational and nonvocational students in subsequent years. They assessed the change in the test scores of the likely vocational school students who were able to study in the general academic track because of the change in school policy. They found that the reform was associated with an improvement in the likely vocational students’ scores of about 100 points, or a whole standard deviation. Source: Jakubowski et al. 2010. Figure 8.2. Finnish School Systems Before and After the Comprehensive School Reform University University 18 Upper Upper 17 Secondary Vocational Secondary Vocational 16 School School School School 15 General 14 Secondary Civic 13 School School 12 Comprehensive 11 School 10 9 94 8 Primary School 7 BEFORE AFTER Source: Pekkarinen, Uusitalo and Kerr 2009 Postponing Tracking When students are tracked into vocational or academic secondary schools, usually only those who enter the latter track are eligible to go on to higher education. Postponing tracking gives prevocational students, especially those from disadvantaged backgrounds, additional time in school to catch up with their more privileged counterparts and to increase their academic attainment. However, if students’ ability and expectations are set at an early age, postponing tracking during adolescence may not have much effect. Indicators of Vocational Tracking There are three main indicators of vocational tracking (figure 8.3): 1. Selection 2. Separation of the vocational and academic pathways 3. Opportunities for further learning The key questions that relate to selection are as follows: At what age are students allocated to vocational schools? Is that selection conducted through competitive selection examinations or is it up to parents and students to choose? The following key question relates to pathways: Are students able to reenter the academic track once they are on the vocational pathway? In other words, even if there is early selection, is the system flexible enough so that students can switch back to academic or other tracks? Also, what is the ratio of enrollments in vocational to enrollments in general education at the stage immediately after the selection occurs or at the end of comprehensive education? This ratio is a measure of the stratification of the education system (Bertocchi and Spagat 2004). A further indicator is the percentage of low-income students on the academic track at the secondary level—the postcomprehensive stage after selection has occurred. The key question related to further learning is this: Are vocational students eligible for higher education? In other words, even if there is selection, and regardless of whether switching between tracks is allowed, are vocational students permitted to enter higher education institutions? A further 95 indicator might be the percentage of low-income students on the academic track in higher levels of schooling. Figure 8.3. Indicators of Vocational Tracking LEARNING PATHWAYS At what age are Are students able Are students able SELECTION FURTHER students allocated to re-enter the to re-enter the to vocational non-vocational non-vocational schools? track? track? Source: Author’s elaboration. East Asian Tracking Results We assessed the performance of secondary education systems in East Asia using these three indicators. Figure 8.4 presents the earliest age of selection. We found that all economies except for the Philippines track into vocational programs, but the ages at which tracking occurs varies. For example, the Lao People’s Democratic Republic and Vietnam have the earliest age of selection, at age 10, while Malaysia has the latest at age 17. The average age of selection is 14. Figure 8.4 also presents whether at this age students are selected into a vocational program that disqualifies them from entering tertiary education. In Korea and Indonesia, those students tracked into vocational programs will not be disqualified from entering academic tertiary programs, while those in Cambodia, Japan, Mongolia, and Shanghai, China, will be disqualified from entering academic tertiary but not technical tertiary. For the remaining economies shown in figure 8.4, students selected into vocational programs at the specified age are not qualified to apply for any tertiary program. Figure 8.4. Earliest Age of Selection Tracked into program whose Korea Rep. 15 graduates can enter any tertiary program (ISCED 3A) Indonesia 16 Cambodia 14 96 Japan 15 Tracked into program whose graduates can enter some tertiary programs (ISCED 3B) Shanghai, China 15 Mongolia 17 Lao PDR 10 Vietnam 10 Tracked into program whose China 12 graduates cannot enter all tertiary programs (ISCED 2B, Singapore 12 2C, 3C) Thailand 14 Malaysia 17 Philippines No Tracking 0 5 10 15 Source: SABER analysis At the upper secondary level, the proportion of students tracked into the vocational sector varies considerably across the region (see figure 8.5). As shown in figure 8.5, Indonesia and Korea and Indonesia track 39 and 24 percent of their students into vocational upper-secondary school, respectively, but graduates from these programs are still qualified for academic tertiary. For those economies where vocational graduates are qualified only for technical tertiary programs, the percentage of students in these programs ranges from just 2 percent in Lao PDR to 71 percent in Cambodia. Finally, for those economies where vocational graduates are not qualified for tertiary programs, the percentage also ranges greatly, from 35 percent in Vietnam to 86 percent in Malaysia. Figure 8.5. Percentage of Upper-Secondary Students Selected into a Vocational Track Tracked into program whose Indonesia 39 graduates can enter any tertiary program (ISCED 3A) Korea Rep. 24 Cambodia 71 Shanghai, China 48 Tracked into program whose Japan 23 graduates can enter some97 tertiary programs (ISCED 3B) Mongolia 15 Lao PDR 2 Malaysia 86 Singapore 56 Tracked into program whose graduates cannot enter all China 46 tertiary programs (ISCED 3C) Thailand 38 Vietnam 35 Philippines 0 No Tracking 0 20 40 60 80 100 Source: SABER analysis While some educational systems allow students to switch between vocational and academic tracks, switching does not happen very much in practice. In fact, there are very little data on the number of students who transfer from vocational programs to tertiary-oriented programs. Of the five economies that have students in nontertiary-oriented upper-secondary programs (ISCED 3C), only two report data on the number of students who transfer to tertiary-oriented programs: 10.3 percent of these students transferred, while in Vietnam 0.001 percent transferred (20 students). 98 References Bertocchi, G., and M. Spagat. 2004. “The Evolution of Modern Educational Systems: Technical vs. General Education, Distributional Conflict, and Growth.” Journal of Development Economics 73 (2): 559–82. Bertschy, K., M. A. Cattaneo, and S. C. Wolter. 2009. “PISA and the Transition into the Labour Market.” Labour 23 (1): 111–37. Brunello, G., and D. Checchi. 2007. “Does School Tracking Affect Equality of Opportunity? New International Evidence.” Economic Policy 22 (52): 781–861. Cunha, F., and J. J. Heckman. 2006. “Investing in Our Young People.” University of Chicago, Chicago, IL. Dustmann, C. 2004. “Parental Background, Secondary School Track Choice, and Wages.” Oxford Economic Papers 56 (2): 209–30. Foster, P. J. 1965. “The Vocational School Fallacy in Development Planning.” In Education and Economic Development, ed. C. A. Anderson and M. J. Bowman, 142–66. Chicago, IL: Aldine. Hanushek, E., and L. Woessmann. 2006. “Does Educational Tracking Affect Performance and Inequality? Differences-in-Differences Evidence across Countries.” Economic Journal 116 (510): C63–76. 99 Jakubowski, M., H. A. Patrinos, E. E. Porta, and J. Wisniewski. 2010. “The Impact of the 1999 Education Reform in Poland.” Policy Research Working Paper 5263, World Bank, Washington, DC. Malamud, O., and C. Pop-Eleches. 2008. “School Tracking and Access to Higher Education among Disadvantaged Groups.” NBER Working Paper 16914, National Bureau of Economic Research, Cambridge, MA. Maurin, E., and S. McNally. 2007. “Educational Effects of Widening Access to the Academic Track: A Natural Experiment.” IZA Discussion Paper 2596, Institute for the Study of Labor, Bonn, Germany. Organisation for Economic Co-operation and Development (OECD). 2002. Reviews of National Policies for Education: Lithuania 2002. Paris: OECD. ———. 2010. Pathways to Success: How Knowledge and Skills at Age 15 Shape Future Lives in Canada. Paris: OECD. Pekkarinen, T., R. Uusitalo, and S. Kerr. 2009. “School Tracking and Development of Cognitive Skills.” IZA Discussion Paper 4058, Institute for the Study of Labor, Bonn, Germany. Psacharopoulos, G. 1987. “To Vocationalize or Not to Vocationalize? That Is the Curriculum Question.” International Review of Education 33 (2): 187–211. ———. 1991. “Vocational Education Theory, Voced 101: Including Hints for ‘Vocational Planners.’” International Journal of Educational Development 11 (3): 193–99. Waldinger, F. 2006. “Does Tracking Affect the Importance of Family Background on Students’ Test Scores?” London School of Economics, London. 100 CHAPTER 9 Information and Communication Technologies Michael Trucano “You can see the computer age everywhere but in the productivity statistics.” Robert Solow, Nobel Prize–winning economist. SUMMARY This chapter explores issues and challenges related to the collection of data about the use of information and communication technologies (ICTs) in education in East Asia. Despite widespread efforts across the region to introduce ICTs in education systems, little comparable data exist to help guide policy makers as they make important investment decisions. This knowledge gap complicates efforts of many Asian countries to benchmark their progress and to ensure that their ICT investments for schools are strategic, are cost-effective, and have an impact. 101 Background and Context Across East Asia, enthusiasm for the use of computers and other information and communication technologies in education is undeniable and widespread. While the contexts for such use differ considerably—from the highly developed urban centers of Shanghai, China, and of Singapore to rural villages in Cambodia and in the Lao People’s Democratic Republic—the promise and potential of ICTs to help address a whole host of educational challenges are compelling. However, levels of actual ICT use in education systems in the region vary widely, not only between economies, but also within economies. Attempts to answer many of the pressing policy questions we have about the use and potential impact of ICTs in education are complicated by the fact that we still do not have reliable data to compare across regions. As hard as it may be to believe, given the large investments being made in ICTs and their increasing economic importance, basic answers to many fundamental questions about the use of ICT in schools remain largely unanswered. Some of these pressing questions include the following: To what extent—and how—are computers being used within classrooms? How connected are schools? What do ICT-related initiatives in the education sector cost, and what is the related impact? Many groups are trying to gather more data on ICT use in education, but until recently many of these efforts have been uncoordinated. Notably, the World Bank participates in the international Working Group on ICT Statistics in Education (WISE), which is led by the United Nations Educational, Scientific, and Cultural Organization (UNESCO) Institute for Statistics (UIS). This effort is part of a larger international, multistakeholder initiative called the Partnership on Measuring ICT for Development, which aims to improve the availability and quality of ICT data and indicators, particularly in developing countries. The partnership is an important component of the World Bank’s education work and, in particular, its Benchmarking Education Systems for Results initiative. Why Attempt to Benchmark ICT Use in Education? Creative and innovative applications of ICTs have long been seen as important tools to enable educational reform processes, with the potential to improve both the access to and the quality of education. The Internet explosion that occurred in the 1990s, the emergence of a variety of low-cost computing devices, and the increased diffusion of computers throughout society ushered in a wave of ICT and education policies and projects around the world. As an increasingly interconnected and digital world emerged—one which saw skills required by the industrial economy increasingly superseded by the demands of the knowledge economy—governments tried to adjust school curricula to equip their students with new, “21st-century” skills. In this regard, most education systems in Asia have generally been like their counterparts in other regions of the world—somewhat slow to adapt, tending to narrowly focus on technical ICT skills rather than the full range of skills needed. These programs require that students think creatively, be able to problem solve, effectively communicate, identify and analyze existing information, and create knowledge. 102 A range of pedagogical approaches have been proposed to help learners to develop those Information Age skills that are now in high demand. Based on research on how people learn, these methods include student-centered learning, active learning, project-based learning, and inquiry-based learning—to name just a few. Yet while the integration of ICTs into the learning process holds great potential to enhance these pedagogies, implementation to date has fallen short of this promise. In other words, ICTs are not yet transforming education, despite the high hopes of many reformers. That said, the increasing diffusion of a variety of ICTs throughout East Asian societies and education systems suggests that even if “transformation” has not yet occurred, many of the necessary preconditions that can help enable such a transformation are being put in place. Given the general lack of regionally comparable data on the magnitude of ICT education investments and their effectiveness, the first step is to benchmark the basic enabling environment in each economy. This means gauging the level of physical infrastructure and human resources in schools and education systems, as well as measuring the degree of access to and use of basic ICT infrastructure and the presence of trained teachers and technical specialists to facilitate that use. Collecting and analyzing data about this first-stage enabling environment provides a foundation on which larger, more fundamental investigations of the impact of the use of such infrastructure can be built. Measuring the Impact of ICT Investments There is much we still do not know about the impact of investments in ICT use in education and the related overall costs. We do not have a good handle on how to measure the types of impacts we hope to bring about through the introduction of things like “one-to-one computing”—where each student has his or her own computing device. Such efforts require different measurement instruments from the traditional standardized learning assessments. With very few exceptions, very limited data have been published to help us understand the costs of such initiatives, especially those related to the total cost of operation over time, and the way such costs are calculated is often not very transparent. Hence, collectively, we are often unable to answer a basic question posed by finance ministries seeking to discriminate between numerous worthy projects and initiatives contending for investment: How much “impact” will this initiative get me, and what will this “impact” cost? While there is much that we still do not know about the best or most effective models for the diffusion and use of ICTs to meet a variety of educational objectives, appropriate models and good practices are emerging. Knowledge about such models and practices is slowly diffusing, thanks in part to the use of ICTs themselves. Moreover, even when cost-benefit analysis is absent, policy makers are confronted with a further policy challenge: what is the cost of not investing in this area? If they delay, and do not invest now, many governments are afraid that they may be outpaced by other, more daring societies ready to make bold investments to compete in an increasingly technology-driven global economy. There is, regrettably, no simple answer to such questions. Some advocates for action recall the words of Nobel laureate in economics Robert Solow, who in 1987 remarked, “You can see the computer age everywhere 103 but in the productivity statistics.” The largely accepted economic imperatives for investment in ICTs, coupled with the perception of educational benefits, often result in sufficient political will for action in this area. Though educational reform is difficult, the purchase of computers for schools, for example, offers a tangible symbol of a government’s commitment to investment in change, and ICTs themselves can often be important vehicles to help bring about desired reforms. It is not just the government that can be held responsible for embracing the so-called ICT revolution; educational leaders, teachers, parents, and students can also help to achieve tangible results on the ground. These efforts should be aided by a strong commitment to evaluating the impact of various initiatives and practices and to learning from the results. What Should—or Can—We Measure? The UIS-led Working Group on ICT Statistics in Education has identified sets of more than 50 core and supplemental indicators that can be helpful in benchmarking the use of ICTs in East Asia. These indicators, summarized in figure 9.1, help address a number of key policy questions related to political commitment; infrastructure; teaching staff development; curriculum; usage; participation, skills, and output; and outcomes and impact. These factors should be understood within the context of a larger operational and conceptual framework for ICT integration in education (UIS 2009). Figure 9.1. Benchmarking ICT Use in Schools ICT for education policy, goals, and incentives Learners’ ICT facilities performance provision to INPUT 1 evaluation and INPUT 2 educational institutions 104 monitoring TO MEET POLICY OUTPUT GOALS OUTCOMES INPUT 3 Usage of ICT PROCESS Trained in teaching teachers on and learning ICT-entitled functions pedagogy Curriculum development and digital content provision Sources: Guide to Measuring Information and Communication Technologies (ICT) in Education, UNESCO Institute for Statistics, 2009. A subset of these indicators relates to the basic ICT infrastructure available in schools and within education systems more broadly. Specifically, these indicators are: 1. Learners-to-computer ratio To explore the opportunities or limits for using computers in schools to promote or expand computer- assisted instruction. This is the most requested data point on technology use in education around the world, and for that reason alone it is useful to collect regionally comparable data in this area. It is thought to be a useful simple proxy for the level of penetration of computers into schools. 2. Proportion of schools with Internet access, by type To measure the overall level of access to the Internet in schools and the opportunities or limits for the use of computers in primary and secondary schools, by type of Internet access. An important concern in East Asia relates to the use of the Internet as an access point to education materials. 3. Proportion of learners who have access to the Internet at school To measure Internet accessibility among learners for educational purposes. Anecdotal evidence suggests that where the Internet is available in schools, it may be used by only 105 certain segments of the school population or for administrative purposes. By building on indicators 1 and 2, this indicator is meant to help provide additional granularity to investigations of how broad Internet access is for students. 4. Proportion of ICT-qualified teachers in primary and secondary schools To measure the extent to which primary and secondary school teachers have the required ICT training to teach basic computer skills or computing classes. The preponderance of evidence suggests that teachers need to be trained if investments in ICTs for schools have any chance of having an impact on educational practices at a classroom level. Imparting basic ICT literacy is thought to be an important first step in this process. 5. Proportion of learners (by gender) enrolled at the postsecondary and tertiary level in ICT-related fields To measure the share of learners by gender in ICT-related fields of study in postsecondary and tertiary educational institutions. It is widely believed the introduction of ICTs into educational settings may raise profound equity issues. One of the proxies often used to measure gender disparities in ICT use is how many students intend to major in computer-related fields. Any attempt to benchmark ICT use in education across East Asia, let alone the impact of such use, must begin with the collection and analysis of such basic data. Benchmarking ICT Use in Education in East Asia A recent pilot survey coordinated by UNESCO-Bangkok that attempted to collect the type of data outlined in the previous section for a select number of economies shows just how far we have to go. Of all the policy domains in the education sector surveyed as part of the recent World Bank–UNESCO System Assessment and Benchmarking Education for Results (SABER) study, the ICT domain had the lowest completion rate, and much of the data collected were not current. Why was this the case? Probably the most significant factor is that there are very few systematic and formal initiatives in the region to collect even basic ICT-use data. When information does exist, it invariably comes from data sources outside of the education sector itself and does not appear to be gathered according to common methodologies and definitions. These factors are complicating efforts in the region to have policy and investment decisions driven by data rather than by untested conventional wisdom, political calculation, or the basic intuition of key decision makers. Figure 9.2. Korea, Korea Rep. Percentage of Rep. Primary and Secondary Schools Mongolia Mongolia Connected to the Internet 106 Indonesia Indonesia 0% 0% 20% 20% 40% 40% 60% 60% 80% 80% 100% 100% Source: Author’s compilation. Korea, Korea Rep. Figure 9.3. Rep. Nature of School Internet Access Indonesia as Percentage of Indonesia Schools Connected Fixed broadband Mongolia Mogolia Fixed narrowband 0% 0% 20% 20% 40% 40% 60% 60% 80% 80% 100% 100% Source: Author’s compilation. Across the region, as in the rest of the world, it is increasingly common to hear senior government officials call for “all schools to be connected to the Internet.” Of all the core indicators outlined by the UIS-led working group (WISE), progress toward the goal of “connecting schools” appears to be the easiest to measure, although defining what it means to be “connected” can often differ radically among countries. Let’s compare, for example, three countries: Indonesia, the Republic of Korea, and Mongolia. From one perspective, Indonesia is the outlier here, as only 11 percent of its schools are connected, compared to 87 percent in Mongolia and 100 percent in Korea. Dig a little deeper, however, and a different picture emerges. In Korea, 100 percent of schools are connected to broadband Internet at some of the fastest speeds in the world. Of the schools connected to the Internet in Indonesia and Mongolia, almost all of them use fixed narrowband connections—less than 256 kilobits per second. So when one evaluates the state of school connectivity in Mongolia, the answer depends on the goal, which may, on the one hand, be enabling teachers and students to communicate via e-mail or, on the other hand, be facilitating real-time access to rich media learning resources online. For the latter, a fast Internet connection is what really matters. Conclusions Almost a decade ago, UNESCO’s landmark Meta-Survey on the Use of Technologies in Education in Asia and the Pacific attempted to analyze ICT access and use in education in the region (UNESCO-Bangkok 107 2004). Since that time, there has been no significant regional data collection effort. For a region generally considered by many in the rest of the world to be at the vanguard of technology use, this omission may strike some as rather puzzling—especially given that regular data collection efforts related to technology use in education are under way in Europe, North America, and—more recently—South America. Absent basic data about how ICTs are being used in the education systems in their countries and how this usage compares with usage in other countries—both in the region and in the rest of the world—educational policy makers in East Asia and Pacific may face fundamental disadvantages compared to many of their counterparts in other regions of the world as they seek to ensure that their investments in this area are cost-effective and impactful. References Information for Development Program. 2005. Knowledge Maps: ICT in Education—What Do We Know about the Effective Uses of Information and Communication Technologies in Education in Developing Countries? Washington, DC: World Bank. ———. 2005. Monitoring and Evaluation of ICT in Education Projects. Washington, DC: World Bank. Scheuermann, F., and F. Pedró, eds. 2010). Assessing the Effects of ICT in Education: Indicators, Criteria, and Benchmarks for International Comparisons. Washington, DC: World Bank. Plomp, T., R. Anderson, N. Law, and A. Quale. 2009. Cross-National Information and Communication Technology Policies and Practices in Education. Rev. 2nd ed. Amsterdam: Information Age Publishing. UNESCO (United Nations Educational, Scientific, and Cultural Organization)–Bangkok. 2003. Developing and Using Indicators of ICT Use in Education. Bangkok: UNESCO. ———. 2004. Meta-Survey on the Use of Technologies in Education. Bangkok: UNESCO. ———. 2011. “ICT in Education.” http://www.unescobkk.org/education/ict/. 108 UIS (UNESCO Institute for Statistics). 2009. Guide to Measuring Information and Communication Technologies (ICT) in Education. Montreal: UNESCO. CHAPTER 10 Tertiary Education Jamil Salmi and Sunita Kosaraju “In questions of mind, there is no medium term: either we look for the best or we live with the worst.” John Gardner SUMMARY Tertiary education institutions have a critical role to play in supporting knowledge- driven economic growth strategies and in constructing democratic, socially cohesive societies. Comparisons of tertiary education systems across the world have revealed wide variations in their performance, even between those with similar funding levels and common country characteristics. They have also demonstrated that certain systems consistently outperform others in many critical areas. However, attempts to measure and analyze what works at the tertiary level have tended to emphasize the results of individual institutions as opposed to the system as a whole. The proliferation of international and national rankings has focused on the 109 relative standing of countries, using the position of top universities as proxy for the performance of the entire tertiary education system. But these rankings are flawed in their conceptual approach and suffer from serious methodological problems. They fail to measure the overall strength of a country’s tertiary education system and its contribution to economic and social development. There is, therefore, a need for a reliable and comprehensive benchmarking tool to evaluate tertiary education systems over time and the causal relationships between input and process variables and the outcomes of the system. This chapter outlines (a) an intellectual framework to underpin analysis of the performance of tertiary education systems, (b) a tool that can be used to analyze sets of data within this framework, and (c) the relevant indicators to evaluate these systems. Finally, types of comparisons that can be made with this tool are illustrated using key indicators from East Asian economies. Analytical Framework When the benchmarking tool was built, a fundamental distinction was made between the outcomes of tertiary education systems (system performance) and the drivers of performance that account for these results (system health), with the purpose of answering two questions: • How well does the tertiary education system actually produce expected outcomes at the current time (system performance)? • How well do the key inputs, processes, and enabling factors of the system reflect conditions that are known to bring about favorable outcomes? Furthermore, the tool was designed to be used to examine system evolution—in other words, the rate of change over time in the main dimensions of performance and in each key driver of this performance. System Performance System performance can be measured by looking at the key outcomes of a tertiary education system. Reflecting the various missions of tertiary education, the benchmarking tool includes the following outcomes:  Attainment refers to the stock of qualifications in a given population, measured by calculating the proportion of adults in the working-age population who have completed a tertiary degree.  Learning achievement refers to the quality and relevance of the education and training experience of tertiary-level graduates. This is one of the most difficult areas to measure in the absence of widely accepted metrics such as the Programme for International Student 110 Assessment (PISA) or Trends in International Mathematics and Science Study (TIMSS).  Equity refers to disparities in the results (attainment and academic trajectories) of disadvantaged groups (such as low-income groups, females, minorities, and people with disabilities).  Research outcomes refer to publications and advanced training, measured by the number of scientific journal citations relative to a country’s population and the capacity of the system to prepare doctoral graduates.  Knowledge and technology transfer represent the contribution of tertiary education institutions to the development of the regions that they serve. Some ways to measure this outcome include the number of patents registered by universities or the proportion of doctoral graduates working outside universities.  Values, behavior, and attitudes refer to the effectiveness of tertiary education in equipping graduates with positive values and citizenship skills. This area is also very difficult to measure, but the methodological challenges do not justify neglecting this important dimension of the role of education. System Health System health refers to the enabling conditions required for the tertiary system to produce these outcomes and to improve and sustain its performance over time. As figure 10.1 illustrates, these institutions operate in an environment that includes the following elements:  Macroenvironment: The overall political and economic situation of a country, together with the rule of law and the enforcement of basic freedoms, which influences the governance of tertiary education institutions (the appointment of university leaders), their level of funding, their academic freedom, and safety in the physical environment.  Leadership at the national level: The existence of a vision and a strategic plan to shape the future of tertiary education and the capacity to implement reforms.  Governance and regulatory framework: The governance structure and processes at the national and institutional levels that determine the degree of autonomy that tertiary education institutions enjoy and how and to what extent they are held accountable. This element is especially important for the human resource policies and management practices that allow tertiary education institutions to attract and keep qualified academics. 111  Quality assurance framework: The institutional setup and the instruments for assessing and enhancing the quality of research, teaching, and learning.  Financial resources and incentives: The absolute volume of resources available to finance tertiary education (mobilization of both public and private resources) and the way in which these resources are allocated to various institutions.  Articulation and information mechanisms: The links and bridges between high schools and tertiary education and between the various types of tertiary education institutions, all of which affect the academic characteristics of incoming students and their academic results within the tertiary education system.  Location: The infrastructure and the economic, social, and cultural characteristics of the geographic location of the institution, which determine its ability to attract outstanding scholars and talented students.  Digital and telecommunications infrastructure: The availability of broadband connectivity and end-user devices to enable tertiary education institutions to deliver educational, research, and administrative services in an efficient, reliable, and affordable way. Figure 10.1. Tertiary Education Ecosystem 112 Source: Salmi 2011. This analytical framework translates into specific inputs and process indicators that measure system health in the following way:  Inputs. To what extent do the resources invested in a tertiary system (such as its funding, the number and qualifications of its academics, the academic preparation of its incoming students, its curriculum, and its learning infrastructure) lead to positive outcomes?  Processes. How effective are a system’s processes or policies (such as its governance arrangements, resource allocation mechanisms, and accountability instruments) in producing positive outcomes? Comparing East Asian Countries on the Basis of Leading Indicators Leading indicators are those used to detect or predict important changes that are likely to occur in a tertiary education system. Table 10.1 lists the 10 leading indicators chosen for the summary analysis of the system performance and system health of tertiary education in East Asia. Table 10.1. Leading Indicators of Tertiary Education Systems Attainment: Proportion of the population (age 25–44) with a tertiary degree System Equity: Tertiary education attainment of females over attainment of males among the population (age 25+) performance Research output: Number of citations per 100,000 inhabitants Expansion: Tertiary education enrollment rate Equity: Female-to-male tertiary enrollment rate Quality: Proportion of accredited programs 113 System health (drivers of Financing: Total spending on tertiary education as a percentage of gross performance) domestic product (GDP) Investment in research: Public funding on research as a percentage of GDP Governance: Degree of autonomy of public universities Preparation of incoming students: Composite index of mean math, science, and reading PISA scores Source: Authors’ compilation The following approach was used to assess the performance and health of East Asian tertiary education systems. First, data were analyzed for each leading indicator in 1960, 1980, 2000, 2005, and the latest available year. Then data for each country were graphed to show the rate of change over time and relative to other economies. Where relevant, an East Asian average was calculated to give the reader a baseline comparison. So that the short-run and long-run growth rates of the various indicators could be analyzed and the trends in each country could be studied, the growth rate between 1980 and the most recent year for which data were available was calculated. Countries were grouped according to whether their growth rate on a particular indicator placed them in the top 25 percent, the middle 50 percent, or the bottom 25 percent. Within each percentage group, countries were listed alphabetically. Analysis of System Performance Indicators For the purpose of the analysis of performance, two leading indicators were selected: attainment and research citations. Attainment Figure 10.2 shows increasing levels of tertiary attainment in the adult population across the region. In 1960, in all economies in East Asia, fewer than 5 percent of the population had obtained a tertiary degree. By 2010, the spread has become much wider, with 4 out of 14 economies continuing under the 5 percent attainment range but with the Republic of Korea and Singapore, at the other end of the scale, having an attainment rate of 47 percent and 66 percent, respectively. There was rapid growth in tertiary degrees between 1980 and 2000 in Japan, Korea, and Singapore, but this growth has leveled off in the past 10 years. It would be useful to study what policies were in place that effectively allowed for attainment rates to quadruple within the span of 20 years. Figure 10.2. Proportion of the Population (Age 25–44) with a Tertiary Degree 114 70 Singapore 60 50 Korea, Rep. 40 Japan 30 Philippines Hong Kong SAR, China 20 Thailand 10 Mongolia Malaysia China Lao PDR Vietnam Indonesia Cambodia 0 1960 1980 2000 2010 Source: Barro and Lee (2010) Table 10.2 shows that Indonesia, the Lao People’s Democratic Republic, and Mongolia have the fastest- growing attainment rates, even though their current attainment rates are low relative to other economies such as Korea and Singapore. Table 10.2. Growth Rate in Adult Tertiary Attainment between 2000 and 2010 Korea, Rep. Top 25% Thailand Vietnam Cambodia China Hong Kong SAR, China 115 Middle 50% Japan Malaysia Philippines Singapore Indonesia Bottom 25% Lao PDR Mongolia Source: Barro and Lee 2010 and national statistics agencies of Hong Kong SAR, China; the Republic of Korea; and Singapore. Research Citations per 100,000 Inhabitants Figure 10.3 shows the research performance of economies in East Asia. Three economies stand out as high achievers in this area: Japan, Korea, and Singapore. All other economies continue to be in the initial or emerging phase of development in terms of research capacity, with fewer than 10 citations per 100,000 inhabitants. Figure 10.3. Science Citations per 100,000 Inhabitants 90 Singapore 80 70 60 50 Japan 40 116 Korea, Rep. 30 20 Hong Kong SAR, China 10 China Thailand Vietnam Philippines 0 1981 1990 2000 2005 (latest) Source: U.S. National Science Foundation via World Development Indicators. When grouped according to growth in research output between 1980 and 2005 (the latest year for which data are available), China, Korea, and Singapore are in the top 25 percent. Indonesia, Japan, Malaysia, the Philippines, Thailand, and Vietnam are in the middle 50 percent. Cambodia; Hong Kong SAR, China; Lao PDR; and Mongolia are in the bottom 25 percent. Analysis of System Health Indicators Three lead indicators were used to look at the drivers of outcomes: spending on tertiary education, spending on research, and degree of autonomy. Total Spending on Tertiary Education as a Share of GDP In those East Asian economies where data were available, the analysis noted that average public and private expenditure on tertiary education grew from 1.33 percent of GDP in 2000 to 1.92 percent in 2010. As shown in Figure 10.4, Mongolia was the leading spender on tertiary education in 2010, while Vietnam spent the least as a percentage of GDP. Figure 10.4. Public and Private Spending on Tertiary Education as a Percentage of GDP 2.93 Mongolia 1.22 117 2.50 Korea 2.21 1.92 EA average 1.33 1.50 Japan 1.10 2010 0.74 Vietnam 0.84 2000 Philippines 1.54 Hong Kong SAR, China 1.50 Thailand 0.87 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 Sources: UNESCO Institute for Statistics, HDNED–UNESCO Bangkok survey 2010 (SABER Analysis). Public Funding for Research as a Share of GDP Investment in research grew slowly between 2000 and 2007, the latest available year of data (the exception is Hong Kong SAR, China, for which data were available for the year 2008). In 2007, the average expenditure on research and development (R&D) in the region was 2.2 percent of GDP. The economies with the highest levels of R&D expenditure were Japan, Korea, and Singapore, while Indonesia (0.05 percent), Lao PDR (0.04 percent), and the Philippines (0.12 percent) were at the low end of the expenditure spectrum (figure 10.5). Figure 10.5. Research and Development Expenditure as a Percentage of GDP 4.0 3.5 Japan Korea, 3.0 Rep. 2.5 Singapore 118 2.0 East Asia average 1.5 China 1.0 Hong Kong Malaysia SAR, China 0.5 Thailand Philippines Indonesia Mongolia 0.0 2000 2005 2010 (or latest) Source: UNESCO Institute for Statistics Governance: Degree of Autonomy This indicator measures the level of management autonomy enjoyed by tertiary institutions. Economies were scored on a four-point scale depending on how many of the four characteristics of a fully autonomous system they exhibited according to key legal and regulatory documents such as higher education laws, ministry of higher education regulations, and similar documents. The four characteristics of a fully autonomous system are (a) independent management of finances; (b) independent appointment of leaders; (c) autonomy in recruitment of students and staff members; and (d) academic freedom, meaning autonomy over the formulation of academic content. Therefore, if an economy had no formal policies on the governance and management of tertiary institutions, it would receive a score of 1 and would be categorized as having “very low” policies on governance. An economy would receive a score of 2, defined as “low,” if its public institutions had one or two features of a completely autonomous system. An economy would be scored as 3, or “high,” if public institutions had three features of a completely autonomous governing system. Finally, economies whose public institutions had all four features of a completely autonomous governing system would be categorized as “very high” and given a score of 4. For example, Malaysia was categorized as having “low” policies in the area of governance and received two points out of four because Malaysian institutions still lack autonomy over such key areas as the independent appointment of leaders and because the government continues to maintain significant central control of public institutions (Raza 2010; see figure 10.6). 119 Figure 10.6. Governance: Degree of Autonomy 5 Very high 4 High 3 Low 2 Very low 1 0 Philippines Singapore Hong Kong SAR, China Mongolia Cambodia China Vietnam Lao PDR Thailand Korea Japan Indonesia Malaysia Source: Raza 2010. Conclusions This chapter introduces a conceptual framework for measuring the performance of a tertiary education system and identifying the degree to which an economy has aligned appropriate policies and resources to drive performance in the future. The application of this framework to the systems of 14 East Asian economies has highlighted a number of trends. First, Japan, Korea, and Singapore have established a leadership role in performance on attainment and research output between the 1980s and 2010. Not surprisingly, they achieved these results in a context of ample financial support (both total expenditure and R&D-specific investment in tertiary education) along with a favorable governance structure allowing for institutional autonomy. Second, the economies that are currently leading in performance are not necessarily the same as those that are currently leading in improvement over time. An analysis of the attainment dimension showed that only one current leader (Korea) has also shown significant improvement over the past 10 years. Other top growers, including Thailand and Vietnam, do not currently have a comparable level of financial support or an appropriate governance arrangement. It will be interesting to see what the ceiling on improvement will be for these countries without the advantage of relevant investments in institutional capacity. The System Assessment and Benchmarking Education for Results (SABER)–Tertiary tool can thus be used to focus the attention of policy makers on the key inputs and processes likely to stimulate improved performance of the system. 120 This chapter provided an overview of only two performance measures (attainment and research output) and a cursory sample of the many inputs driving change on these dimensions. For example, the relationship between the level of preparation of incoming students and its impact on the efficiency of tertiary education systems (in terms of time to degree) in producing graduates was not discussed because of the unavailability of data over time for most countries under study. Similarly, the relationship between quality and research output in terms of type and level of accreditation of programs was not explored both because of data constraints and because of the relative lower impact on research output of this variable compared to those discussed. Thus, there is a need for further data collection on specific inputs to tertiary education systems, including evidence-based research on the key inputs and policies required for system performance on the equity dimension. References Aghion, P., M. Dewatripont, C. Hoxby, A. Mas-Colell, and A. Sapir. 2009. “The Governance and Performance of Research Universities: Evidence from Europe and the U.S.” NBER Working Paper 14851, National Bureau of Economic Research, Cambridge, MA. Baye, A., and C. Mainguet. 2006. “Defining a Framework of Indicators to Measure the Social Outcomes of Learning.” In Measuring the Effects of Education on Health and Civic Engagement: Proceedings of the Copenhagen Symposium, ed. R. Desjardins and T. Schuller, 153–64. Paris: OECD. Barro, R., and J. Lee. 2010. Educational Attainment Dataset. November 12. http://www.barrolee.com. Campbell, David. 2006. “What Is Education’s Impact on Civic and Social Engagement?” In Measuring the Effects of Education on Health and Civic Engagement: Proceedings of the Copenhagen Symposium, ed. R. Desjardins and T. Schuller, 25–28. Paris: OECD. Organisation for Economic Co-operation and Development (OECD). 2009. Education at a Glance. Paris: OECD. Johnstone, B., and P. Marcucci. 2009. “Higher Education Finance and Cost Sharing Profiles by Country.” International Comparative Higher Education Finance and Accessibility Project, Center for Comparative and Global Studies in Education, State University of New York, Buffalo, NY. http://gse.buffalo.edu/org/inthigheredfinance/. 121 National Science Foundation. 2011. Science and Engineering Indicators via World Bank World Development Indicators. http://data.worldbank.org/indicator/IP.JRN.ARTC.SC. Raza, Reehana. 2010. “Higher Education Governance in East Asia.” Background paper for Putting Higher Education to Work: Skills and Research for Growth in East Asia. Washington, DC: World Bank. Salmi, J. 2011. “The Road to Academic Excellence: Lessons of Experience.” In The Road to Academic Excellence: Emerging Research Universities in Developing and Transition Countries, ed. P. Altbach and J. Salmi, 323–47. Washington, DC: World Bank. Salmi, J. 2009. The Challenge of Establishing World-Class Universities. Washington, DC: World Bank. Salmi, J., and A. Saroyan. 2007. “League Tables as Policy Instruments: Uses and Misuses.” Higher Education Management and Policy 19 (2): 42–62. 122 CHAPTER 11 Sequencing and Trade-offs Kevin Macdonald and Sungmin Park SUMMARY Comparing education policies with those of high-performing or “benchmark” systems provides some interesting insights into improving education quality, but what about context? The Republic of Korea has been considered a high-performing country given its strong performance in international assessments and, more interestingly, has been at the same stage of economic development as every low- and middle-income East Asian country included in this report. Consequently, an interesting question to ask is how do a country’s education policies today compare to Korea’s when Korea was at the same stage of development? Two differe nces emerge. First, no country is paying its teachers as well as Korea did when it was at the same stage of development. Second, Korea has never provided an education stream that disqualifies graduates from academic tertiary programs; many East Asian countries provide vocational programs that are dead-end. 123 Introduction Comparing education policies and practices to those of high-performing countries provides valuable lessons for improving the quality of education. Countries in East Asia, though, are at very different stages of development, and it is not clear whether the education policies used by wealthy, high- performing countries are necessarily suitable for those that have lower institutional capacity and higher rates of poverty and inequality. An interesting case, however, is the Republic of Korea. Korea has been— at one time or another—at the same stage of development as every low- and middle-income country in the region. Since the 1960s, Korea’s gross domestic product (GDP) increased 40 times, and this increase is largely attributed to the country’s education system (Lee 2008). Given the success of Korea’s education system to drive growth over the past 50 years, an interesting question to ask is how do low- and middle-income countries’ education policies now compare to those of Korea when Korea was at the same stage of development? This chapter shows that Korea has consistently paid teachers highly and has never streamed children into vocational programs that restrict them from entering academic tertiary programs throughout its development. Low- and middle-income countries are paying their teachers much lower relative rates than Korea was at the same stage of development; moreover, many stream students into vocational programs that prevent entry to academic programs or to tertiary education opportunities. Korea has adopted other policies and practices, such as school autonomy and the use of national assessments, more recently as the evidence base for their effectiveness has grown, and low- and middle-income countries are ahead of what Korea was doing at the same stage of development. Learning from Korea’s Past One of the most notable education strategies that Korea adopted was the sequenced expansion of its schooling system. First, primary enrollment expanded until it reached approximately 90 percent by the early 1960s. Then, secondary enrollment became the focus, and was expanded throughout the 1960s and 1970s. But it was not until middle and upper-secondary schooling became nearly universal—higher than 80 percent—that the focus turned to tertiary education. Between 1990 and 2005, the tertiary enrollment rate increased from 23 to 62 percent. Low- and middle- income countries, however, are not following this same sequence. Figure 11.1 presents the net secondary enrollment rate for East Asian countries aligned to when Korea had the same GDP per capita. For example, Thailand has a net secondary enrollment rate of 72 percent, whereas when Korea had the same GDP per capita, its net secondary enrollment rate was 57 percent. In fact, almost all low- and middle-income countries have secondary enrollment rates that exceed those reached by Korea at the same stage of development, suggesting that these countries are taking a different approach to that of Korea. Figure 11.1. What Was Korea’s Net Secondary Enrollment Rate When Korea Was at the Same Stage of Development as Other Countries? Net Secondary Enrolment Rate 124 100%  Japan  Korea, Rep.  Mongolia 75%  Hong Kong  Thailand SAR, China  Indonesia  Malaysia  Vietnam  Philippines 50%  Lao PDR 25% 0% 1955 When Korea was at the same Present stage of development Sources: KEDI 2007 and UNESCO Institute of Statistics/World Bank EdStats. Teacher Policy Researchers in Korea attribute its strong performance in international exams that test education quality, such as the Programme for International Student Assessment (PISA) and Trends in International Mathematics and Science Study (TIMSS), to being able to attract the “best” into teaching (Kim, Kim, and Han 2009). In Korea, the competition to be a teacher is tough—in 2006, only 15.3 percent of those who received teaching certificates after completing programs at secondary teacher education institutes were employed by public and private schools. Korean teachers have also received higher remuneration than teachers in other countries. In 1965, teachers with 15 years of experience were paid 3.9 times GDP per capita, and in 1985 the ratio peaked at 4.2 times. Figure 11.2 presents teacher salaries after 15 years as a percentage of GDP per capita throughout Korea since 1965; it plots this percentage for East Asia countries at the time in Korea’s past when they had the same GDP per capita. For example, teachers in the Philippines receive approximately 2.8 times GDP per capita. The GDP per capita overall in the Philippines is about US$1,200 (in constant 2000 dollars); Korea had the equivalent GDP per capita in 1964. In other words, when Korea was at the same stage of development as the Philippines is now, its teachers received salaries that were 3.9 times GDP per capita, which is higher than the salaries teachers receive in the Philippines. Although data for Korea go back only as far as 1965, the implication for client countries is that they are not paying teachers enough. 125 Figure 11.2. How Well Are Teachers in Other Countries Paid Compared to Teachers in Korea at the Same Stage of Development? Salary after 15 years (% GDP per capita) 400 300  Philippines 200  Vietnam  Korea, Rep  Mongolia  Japan par Lao PDR 0   Indonesia 1955 When Korea was at the same Present stage of development Source: UNESCO World Education Indicators 2006; SABER Teacher Data; KEDI 2007; Lee, Yoo, and Han 2000. Assessment Korea also uses a sophisticated national assessment to monitor the progress of education quality and to formulate policies and make decisions. Since 1999, Korea assesses all 6th, 9th, and 10th grade students using the National Assessment of Education Achievement (NAEA). Prior to 2008, the assessment was sample based, covering 1 to 5 percent of all students. Figure 11.3 compares the use of national assessments across different East Asian countries, looking at four different indicators. In addition to whether there has been a recent assessment, it also examines the enabling environment within the Ministry of Education by looking at whether there is a regular budget for the assessment or if funding is more ad hoc. Since high-stakes exams may not provide a reliable measure of student ability at the system level, figure 11.3 also looks at whether countries’ assessments are high stakes and, finally, at whether they are officially used for policy design or decision making. Korea’s NAEA achieves all four of these characteristics, but only since 1996. Before 1996, when Korea was at the same stage of development as the low- and middle-income countries included in this report, it did not have a national assessment; these countries are ahead of where Korea was in this respect. Figure 11.3. Comparing National Assessments to Korea when Korea was at the Same Stage of Development 126 ….and used officially  Vietnam  Korea, Rep. for policy design ….and that is not  Lao PDR  Philippines high stakes  Japan …with a regularized Cambodia budget   Indonesia  Thailand  Malaysia There has been an assessment in the  Mongolia past five years Hong Kong China SAR, China Singapore   1955 When Korea was at the same Present stage of development Source: Author’s compilation. Accountability to Parents The accountability of schools to parents is also a defining feature of the Korean education system. Since 1996, the school council system was introduced to every public school to supervise budget implementation; this measure was expanded to all schools in 1999. In 2005, school councils were given the legal authority to approve the school budget. Prior to 1996, parent-teacher associations existed but had a very limited role in overseeing the school’s financial performance. Consequently, parents in most East Asian countries have a much stronger influence over decision making at their school than Korean parents did when Korea was at the same stage of development. China, the Philippines, and Vietnam stand out as the only countries where parents do not even have a limited role in overseeing finances at the school (figure 11.4). Figure 11.4. How much Influence Do Parents Have Over the School’s Finances Compared to Korean Parents When Korea Was at the Same Stage of Development? School councils have Indonesia Malaysia Korea, Rep. legal authority to   Thailand    approve the school Hong Kong budget SAR, China 127 School councils have legal authority to Mongolia supervise the budget’s  implementation School councils have Lao PDR Singapore Japan some authority to Cambodia    oversee the school’s financial performance Vietnam Philippines China    1955 When Korea was at the same Present stage of development Source: Author’s compilation. Engaging the Private Sector Figure 11.5 shows that engaging the private sector with public funding has been a practice in Korea since 1971. Prior to 1971, tuition and PTA contributions were the main sources of funding for schools, and no public funding was allocated to them. Private school enrollment rates were quite high during Korea’s early development; in 1970, 48.6 percent of middle school students were enrolled in private schools, but this number had declined to 20 percent by 2004 (KEDI: 138). [[AQ: KEDI 2007, 138? Not in references.]] When Korea began providing private schools with public funding—initially teacher salaries—Korea’s GDP per capita was just over US$2,100 (in constant 2000 dollars). Most East Asian low- and middle- income countries shown in figure 11.5 follow the same pattern; those below this level of development do not fund private schools with public resources, while those above provide public resources. Figure 11.5. Comparing the Role of the Private Sector to That in Korea When Korea Was at the Same Stage of Development Hong Kong There is a contract Thailand Malaysia Korea, Rep. SAR, China governing the use of  Indonesia     public funding to Japan 128 private schools There are publically  China  Lao PDR funded private schools Private schools are legally able to Mongolia Philippines operate and provide   compulsory Vietnam Cambodia Singapore education  1955 When Korea was at the same Present stage of development Source: Author’s compilation. Tracking Tracking students into terminal degree, or “dead-end,” programs has been shown to be both inequitable and detrimental to student cognitive ability. Korea adopted a 6-3-3-4 single-track education system to ensure that high school graduates are still qualified to apply to universities. Tracking to vocational programs does exist in Korea; however, the difference between Korea and other countries is that vocational programs, while not oriented toward university admission, do not disqualify a student from entering. Vocational tracks in most East Asian countries, as shown in figure 11.6, disqualify students from all or at least some tertiary education programs; these are the ISCED (International Standard Classification of Education) B and C destinations as defined by the United Nations Educational, Scientific, and Cultural Organization. The key finding from figure 11.6 is that Korea has never at any stage of development produced secondary graduates that are not able to enter academic tertiary programs, let alone technical or vocational tertiary programs. Figure 11.6. How Do Tracking Policies Compare to Korea’s When Korea Was at the Same Stage of Development? Students can be 129 Singapore tracked at an age  Lao PDR  China  younger than 15 years old Students’ grades or Mongolia examination results   Cambodia  Thailand  Japan influence how student is tracked Students can be tracked into a program that restricts entry into some tertiary programs Philippines Korea, Rep.   1955 When Korea was at the same Present stage of development Source: Author’s compilation. Tertiary Education Spending The expansion of Korea’s education system is typically defined in stages and following a sequence starting with primary, then secondary, and now tertiary. As figure 11.7 shows, the amount of public expenditure as a percentage of GDP spent by Korea on tertiary education has grown from 0.3 percent in 1970 to 0.5 percent. Figure 11.6 also shows that some low- and middle-income country governments are spending about the same as Korea did at the same stage of development, while other countries— notably Mongolia and Malaysia—are spending much more. For example, Malaysia’s public expenditure on tertiary education is 1.4 percent, which is about 2.7 times more than what Korea spent at the same stage of development. Figure 11.7. Comparing Public Expenditure on Tertiary Education with Korea’s Spending at the Same Stage of Development Public Expenditure on Tertiary (% of GDP) 1.5  Malaysia 130 1.0  Singapore  Mongolia  Hong Kong SAR, China 0.5  Thailand  Korea, Rep.  Indonesia  Philippines  Lao PDR 0 1955 When Korea was at the same Present stage of development Sources: World Bank World Development Indicators; World Bank EdStats; UNESCO Institute of Statistics; World Bank 2000; Thomas et al. 1999. Are the Lessons from Korea’s Past Still Relevant Today? The limitation of comparing the education policies of a country today to Korea’s past policies is that the global context has changed dramatically over the past 55 years. The policies Korea was pursuing in the 1950s and 1960s may not be optimal for countries today, and there may be policies that Korea was not pursuing at the same stage of development that are needed now. Figure 11.8 reveals one such important difference: low- and middle-income East Asian countries are much more integrated into the global economy than Korea was at the same stage of development. Cambodia, Malaysia, and Thailand have manufactured exports equal to around 50 percent of GDP. As a result, Malaysia has a level of manufactured outputs as a percentage of GDP that is 1.8 times what Korea had at the same stage of development, and Thailand has a level that is 2.5 times. And if Korea’s manufactured exports as a percentage of GDP grew from 1955 to 1962, as the trend suggests, then Cambodia’s would be 120 times larger than what Korea had at the same stage of development. Unless there were dramatically higher exports in Korea prior to the earliest (1962) data, all low- and middle- income East Asian countries have a much higher level of manufactured exports as a percentage of GDP than Korea did at the same stage of development. And because these are manufactured exports, not agricultural or energy exports, the volume of these exports suggests a high degree of integration into the global knowledge economy that Korea did not have at the same stage of development. Human capital formation and the policies that drive it need to be stronger for these countries than for Korea at the same stage of development. 131 Figure 11.8. Comparing Manufactured Exports as a Percentage of GDP with Korea’s Exports at the Same Stage of Development Manufacturing Exports (% of GDP) 50%  Malaysia  Thailand  Cambodia 40%  Vietnam 30%  China  Korea  Philippines 20%  Lao  Japan 10%  Indonesia 0%  Mongolia 1955 When Korea was at the same Present stage of development Source: World Bank World Development Indicators. Are East Asia’s Education Policies on the Right Track? For some education policies, such as provision of public funds to private sector schools, low- and middle- income East Asian countries are pursuing a similar path as Korea. But for other policies, there are some stark differences. Teachers are paid much less than what Korea was paying at the same stage of development, and while Korea provides vocational education, it was never at a lower standard that disqualified graduates from applying to the same university programs as those students following the general curriculum, unlike in almost every low- and middle-income country today. Additionally, some policies and practices are more recent innovations in Korea. For example, the use of assessments and the accountability of schools to parents have been strengthened only in recent years. As a result, many low- and middle-income countries are more advanced in those two areas than Korea was at the same stage of development. Korea’s past education policies provide a benchmark to low- and middle-income countries facing similar development challenges. But as shown, low- and middle-income countries are much more export oriented in terms of manufactures than Korea—the model for export-oriented growth—ever was. Given their more robust integration into the global economy than Korea, human capital formation will be more crucial to their success, and consequently, their education policies will need to exceed the benchmark set by Korea throughout its rapid development. 132 References Korean Educational Development Institute (KEDI). 2007. Understanding Korean Education. Education and Korea's Development (5). Seoul: KEDI Kim, E., J. Kim, and Y. Han. 2009. Secondary Teacher Policy Research in Asia: Secondary Education and Teacher Quality in the Republic of Korea. Bangkok: United Nations Educational, Scientific, and Cultural Organization. Lee, C. J. 2008. “Education in the Republic of Korea: Approaches, Achievement, and Current Challenges.” In An African Exploration of the East Asian Education Experience, ed. B. Fredricksen and J. P. Tan, 155–217. Washington, DC: World Bank. Lee, J. H., K. Yoo, and Y. Han. 2000. Economic Analysis of Teacher Remuneration and Policy Reform. Seoul: Bibong. Thomas, V., M. Dailami, A. Dhareshwar, R. E. Lopez, Y. Wang, N. Kishor, and D. Kaufmann. 1999. The Quality of Growth. New York: Oxford University Press. United Nations Educational, Scientific and Cultural Organization (UNESCO). 2006. World Education Indicators 2006: Education Counts Benchmarking Progress in 19 WEI Countries. Montreal: UNESCO Institute for Statistics. World Bank. 2000. Higher Education in Developing Countries: Peril and Promise. Washington, DC: World Bank. CHAPTER 12 Capacity Development Shanti Jagannathan and Harry Anthony Patrinos SUMMARY The ability of a country to benefit from benchmarking depends largely on its capacity in a range of domains. Concerns about adequate capacity have been a significant part of the discourse on development cooperation. The approach of donors to capacity development has evolved over the years. Initially, external technical assistance filled the gaps to meet immediate capacity needs, and externally supported projects implemented ad hoc capacity development activities, often through training and short-term measures. Over time, the focus shifted to enabling country systems to become more capable of implementing reforms. The growing trend of using general and budget support approaches facilitated this approach. Capacity Development and Aid Effectiveness 133 The 2005 Paris Declaration on Aid Effectiveness gave top importance to country ownership and leadership in capacity development (CD). The Bonn Consensus in 2008 followed, with a list of six CD operational priorities, which became part of the Accra Agenda for Action: 1. Integrating capacity development as a core element in national, sector, and thematic strategies and development efforts 2. Supporting developing countries to take the lead in creating the enabling environment for capacity development by addressing systemic impediments to local capacity development 3. Responding to demand for technical cooperation from developing countries rather than taking a supply-side approach, and using local and regional resources, including South-South arrangements 4. Assessing, strengthening, and promoting greater use of country systems to implement policies and manage public resources, including procurement, financial management, results, statistics, and information systems 5. Enabling developing countries to undertake capacity development of civil society and the private sector so that they can play their developmental roles more fully 6. Assisting countries in fragile situations with tailored and coordinated capacity development for core functions earlier and for a longer period to help build or rebuild local institutions Capacity development work in the context of aid effectiveness seeks to strengthen CD methodologies in order to create pragmatic results and indicator frameworks that integrate CD results, that are operationally focused, and that explicitly leave room for adaptation during implementation (OECD/DAC 2011). Capacity development as a key concept for enhancing aid effectiveness often has three main dimensions of integration: (a) interactions among stakeholders at various levels; (b) links between different subsectors in the education sector; and (iii) links or combinations of different aid modalities— namely loans, grand aid, and technical cooperation (Hirosato and Kitamura 2009). In preparation for this year’s 2011 Busan High-Level Forum on Aid Effectiveness, a review of the Accra Agenda for Action for capacity development was undertaken. The review indicated how the Aid Effectiveness Agenda is helping to raise the profile of CD by focusing on country ownership, systems, and processes. However, while countries and development partners have made gains in coordinated, country-owned CD initiatives, there are still only a few examples of sector-specific plans and strategies 134 that address CD in an integral manner. Moreover, considerable work remains to be done to ensure harmonization and alignment of CD support by external partners at the sector level (OECD/DAC/LenCD 2011). There is now increasing recognition that capacity development is integral to achieving and sustaining education sector performance. The achievement of key educational objectives is contingent on national systems being able to strengthen sector capacity. For this to happen, CD needs to be an integral component of education sector policies and strategies. Ultimately, adequate sector capacity helps to translate inputs into outputs, outcomes, and impact. Capacity for National Benchmarking For evidence-based policy and planning, countries need comprehensive, reliable, and timely data covering all subsectors of the education system. Comprehensiveness, reliability, and timeliness are key factors for the effective use of data by countries to benchmark the performance of their education systems. Countries face the challenge of ensuring that all types of educational institutions, including those in the private sector, are comprehensively covered in a single national data base. In general, the education management information system (EMIS) tends to cover only public educational institutions and does not incorporate data from private sector providers, particularly if they are within the unregulated sector. Data reliability is also a constraint, as it is not uncommon to have inflated reporting of enrollment and other indicators. In addition to internal checks and balances, countries also need to ensure congruence between different sets of national data. Some dimensions of education data are collected through national population census and social sector survey data, and there could be substantial divergences. With growing emphasis on school-based data and household survey data for local planning, administrators are often challenged to triangulate information from different sources. Much can be gained from using a robust and decentralized matrix of data that captures different 135 dimensions of education to allocate resources, since the allocation of budgets is often norms based, rather than based on the degree of educational disadvantage arising from social exclusion, geographic remoteness, poverty, and other factors. Resources that are distributed “equitably” across regions, therefore, may not contribute to establishing equity in education. A vital prerequisite for identifying and supporting areas that are lagging educationally is the existence of reliable disaggregated databases, which can provide information on key educational development indicators (Jhingran and Sankar 2009). Developing counties also lack the capacity to collect and report internationally comparable data in a timely fashion. Although countries are providing an increasing amount of data for international reporting over time, there was a dip between 2009 and 2010. For example, the proportion of United Nations Educational, Scientific, and Cultural Organization (UNESCO) Institute for Statistics (UIS) indicators that are estimated using data provided by economies in East Asia is much lower at the tertiary level than at the primary and secondary levels. The capacity of economies to participate in benchmarking internationally requires attention to such capacity. Table 12.1 outlines key capacity indicators that benchmark CD. In addition to capacity for reliable data systems for national benchmarking, educational planners need to build institutional capacity for timely analysis of such data and their dissemination to local stakeholders. Triangulation of data and openness to putting data in the public domain and taking on board data from other sources are important ways by which the quality of data and their analysis and relevance can be strengthened. Table 12.1. Capacity Indicators in Data and Information Systems Capacity area Types of indicators and measures Percentage of educational institutions covered by the national education data system at the primary, secondary, and tertiary levels for key educational indicators Comprehensive education Public institutions data systems for evidence- Private institutions, including unregulated sector private based policy and planning sector Institutions run by charities, corporations, and other entities Institutions run through foreign franchises Online educational services Percentage of teachers covered in a teacher information database (regular and part-time teachers and “para” teachers) Existence of a national and decentralized teacher data Comprehensive data management information system that covers coverage on teachers  Teacher deployment  Teacher qualifications  Participation in training and professional 136 development  Career management information Effective use of data and information systems for Percentage of education budget allocated on the basis of a gap analysis and use of instruments such as an Education allocation of budgetary Development Index resources Third-party validation of Percentage of education data and education human education data and progress resources covered under alternative data-collecting and on education indicators data-reporting systems, including civil society Country leadership in data Percentage of data owned and managed by in-country mainstream institutions rather than by project-type EMIS; systems, analysis, and share of total funding for data collection, analysis, and reporting reporting included in regular education budgets Availability of internationally Percentage of UIS indicators estimated with data reported comparable data on key by the country education indicators Source: Author’s compilation. Box 12.1. Case Studies from India Using Data for Resource Allocation The Sarva Shiksha Abhiyan Elementary Education Program in India uses an Educational Development Index (EDI) at the national, state, and local levels for planning and resource allocation. The EDI captures a composite picture of the education systems, including the number of out-of-school children, gender gaps, student-classroom ratios, and student-teacher ratios, to allow a differentiated response strategy. This is a concrete example of using data from the District Information System for Education not only for allocating resources, but also for tracking the closing of gaps in key educational parameters. At the country level, the same database allows national policy makers to know which districts have the lowest EDI and to approve additional financial and technical resources to ensure progress over time. Independent impact evaluation of the program is carried out by 41 social science research institutes and universities. Reports are shared at all levels of government and with donors to ensure good practices, and experiences are circulated across the country. Citizens’ Report on State of Education The Annual State of Education Report (ASER) is a citizens’ report on the state of education in the India. The report has been produced every year since 2005 by Pratham, a nongovernmental organization. In 2010, in its sixth year, the ASER had covered 522 districts, more than 14,000 villages, approximately 300,000 households, and almost 700,000 children. Using a common methodology, the report not only tracks key educational indicators such as enrollments, attendance rates, and dropout rates, but also undertakes testing and reporting of the level of student learning, adapted for language and linked to curriculum standards of each state in the country. Source: http://www.sssa.nic.in; http://www.pratham.org. 137 Monitoring Quality and Tracking Improvements Economies in East Asia have made considerable progress in ensuring basic provisioning for education; however, the challenge to augment quality is still pressing. Unlike goals such as universal enrollment, which is a quantitative and terminal measure, quality is more complex and dynamic. Even at high levels of quality, there is always more to be done. Ensuring quality requires the interplay of several factors, which are often hard to measure. While monitoring is required to track sector performance as a whole, here we specially highlight quality as a more challenging component to measure and track for improvements, and more attention is needed to track quality improvements. However, it is possible for countries to develop a matrix of quality parameters. Teachers are considered the single most important factor in determining the quality of an education system. Countries invest considerable resources in teacher training, yet the results and impact of such training in classroom processes and on student learning are difficult to establish and complex to monitor. Most countries do not maintain a comprehensive database on teachers that includes information not just on their career management aspects, but also on their professional development path. By integrating teacher management information systems with school and student information management systems, quality planners can address the matrix of various factors that contribute to quality improvement. Quality assurance systems in East Asia are nascent. Countries are just beginning to articulate policies for minimum teacher and school standards and are only beginning to develop training programs to support these policies. The ultimate test of quality is derived from student learning for which countries implement national assessments of student learning. School quality assurance and accreditation mechanisms are being developed. However, school systems need to develop roadmaps that set realistic goals and targets within an overall time horizon. For example, in the Lao People’s Democratic Republic, the Schools of Quality program incorporates 70 indicators for 25 standards across six dimensions (UNICEF Lao PDR 2011). Measuring, monitoring, and ensuring continuous improvements across such a large matrix of quality standards requires not only a fairly sophisticated data management system, but also capacity at different levels to track, monitor, and improve conditions in schools against established benchmarks. Table 12.2 outlines key capacity indicators for education quality. Table 12.2. Capacity Indicators for Education Quality Capacity area Types of indicators and measures  Percentage of teachers meeting established Educators: minimum qualifications for different levels of education Building professional standards of  Percentage of teachers receiving annual in-service teachers and teacher cadre training relevant to their teaching assignments management  Percentage of teachers participating in quality 138 assurance initiatives of schools and student learning  Percentage of schools covered under a school quality assessment system Educational institutions:  Existence and coverage of an independent accreditation council or board for schools and Ensuring minimum quality technical and vocational education training (TVET) standards in educational institutions institutions  Existence and percentage coverage of an independent quality assurance and accreditation agency for higher education institutions  Existence and percentage coverage of an independent examinations board  Existence and coverage of a national system of student assessment of international standards  Number and percentage of TVET and higher education institutions covered under a national Learners: qualification framework Quality assurance of learners  Country participation in internationally comparable learning assessment systems such as the Programme for International Student Assessment, Trends in International Mathematical and Science Study, Progress in International Reading Literacy Study Source: Author’s compilation. Capacity for International Benchmarking Even as countries struggle to ensure capacity to benchmark performance nationally, there is growing evidence that they are starting to think about international benchmarking as well. These trends toward “internationalization” are attempts to augment national standards to reach internationally recognized levels and help students gauge their competitiveness in the labor market. In particular, a growing number of countries are participating in the Organisation for Economic Co-operation and Development (OECD) PISA (Programme for International Student Assessment), which provides the most universally recognized international standard for student learning. Internationally comparable tests such as PISA, TIMSS (Trends in International Mathematics and Science Study), and PIRLS (Progress in International Reading Literacy Study) are distinct from the benchmarking initiative, and countries need to build bridges between such tools to reach coherent and relevant conclusions on a possible way forward. Globalization of standards relates not only to students’ academic achievement, but also to the achievements of schools and educational institutions. In the Philippines, the Center for Educational Measurement (CEM) has partnered with the Australian Council for Educational Research (ACER) to introduce the International Benchmark Tests (IBTs). The IBTs seek to bring useful comparisons of student performance against international benchmarks. In Indonesia, the International Standard School project is a new education policy initiative that seeks to prepare students for global competitiveness. The policy accredits the status of an International Standard School to schools that have completed set standards. 139 International assessment tools and standards help schools to gauge the success of their students and benchmark their achievements against those of schools in other countries. Recently, the importance of global or mutual recognition of standards has gained ground in the vocational and higher education spheres, where there is mobility of students and faculty across borders. Another observable trend is the growth in regional networks that address quality assurance in the context of regional mobility, as well as the mutual recognition of qualifications. These networks include the Asian University Network, the Asian Pacific Association of International Education, University Mobility in Asia and the Pacific, the Association of South East Asian Institutions of Higher Learning, the Association of Universities of Asia and the Pacific, the Asia-Pacific Economic Cooperation Study Centres Consortium, and the Asia Pacific Quality Network. These trends will become more prominent over the next few years, and there is a need to strengthen the ability of country systems to ensure relevant, credible, and internationally benchmarked systems. Regional Initiatives for Peer Learning Significant gains can be made from regional collaborative efforts. The European Union bloc has implemented many initiatives for coordination and harmonization between education systems of its member countries and has established peer-learning platforms for teachers, education policy makers, and administrators. Within Asia, these types of initiatives are emerging, and much can be gained from strengthening cooperation efforts in education through regional networks of research and peer-learning platforms.  The Asian Network for Training and Research Institutions in Educational Planning (ANTRIEP) was set up with the objective of promoting peer learning and capacity building in participating institutions. ANTRIEP also addresses the growing and increasingly diversified need for skill development in educational planning and management in the region. Supported by the International Institute for Educational Planning (IIEP), ANTRIEP facilitates joint work between member institutions, particularly in the area of school management and head teachers. Annual meetings are used as venues for experience sharing and South-South cooperation. However, the network has lost momentum in the recent years.  Another regional network supported by the IIEP is the Southern and Eastern Africa Consortium for Monitoring Education Quality (SACMEQ). SACMEQ is a collaborative network of 15 education ministries to promote research that can be used by decision makers. With an important mandate for capacity development, IIEP stresses the importance of national ownership, long- term investment in CD, impact assessment of CD, and recognition of the professional character of education planning and management (De Grauwe 2009).  The Regional Network for Education Research initiative seeks to improve the capacity for evidence-based education policy formulation and reform in the Middle East and North Africa. The three-year initiative aims to create a virtual community of practice and continue to expand tailored training and capacity-building programs across the region. The initiative was funded by a World Bank endowment to the Jordan National Center for Human Resource Development. The initiative has supported a series of training seminars and workshops to build data management skills, as well as a mentoring program to help participants practice these skills. Country team participants have also been trained in education policy analysis and on the preparation of policy 140 notes using student assessment data collected by the countries’ participation in TIMSS. Capacity Development for Benchmarking A systematic approach to strengthening capacity for education needs concrete measures to increase school quality and specific indicators to assess outcomes. The following approaches to building this type of capacity could be further advanced in East Asia. 1. Incorporating CD in education system plans, along with proper budget allocation. Often CD initiatives are not adequately funded or supported by a comprehensive plan for implementation. 2. Establishing measurable targets and indicators for CD. Successful CD initiatives require concrete and measurable objectives. 3. Implementing demand-driven CD. CD activities could be instigated by international donors and national-level actors, but effective CD requires a buy-in from the ground-roots level. Generation of demand for CD requires foundational activities such as peer-sharing and exposure visits. 4. Diversifying the supply of CD. While public institutions have a strong role to play in CD activities, it is clear that provision can be diversified to include nongovernmental organizations and the private sector, provided that doing so ensures value for money, quality, and innovation. 5. Planning CD. A comprehensive CD plan should incorporate activities at the individual, institutional, and system level to enable sustainable institutional development and renewal of CD efforts. 6. Buttressing CD with adequate incentives and rewards. CD alone is not sufficient to guarantee quality education and must be complemented by a well-designed rewards and incentives system that encourages higher-level performance of all practitioners. Individuals Institutions Systems  Teaching professionals  Teacher iraining  Planning, (teachers, instructors, Institutions budgeting, and lecturers, professors) expenditure  Quality assurance management  Instructional leaders institutions for education (principals and head teachers, heads of  Regulatory and  Recruitment Incentives for performance faculty, heads of accreditation institutions and 141 Capacity development training institutions) performance  Examination board management of  Teacher educators educational  National authority on and  Teacher trainers and student assessment administrative supervisors personnel  Curriculum experts  Monitoring, review, and  Testing and certifying evaluation of experts education  Governance of education systems, including at decentralized levels References De Grauwe, A. 2009. Without Capacity, There Is No Development. Paris: International Institute for Educational Planning. Hirosato, Y., and Y. Kitamura, eds. 2009. The Political Economy of Educational Reforms and Capacity Development in Southeast Asia: Cases of Cambodia, Laos, and Vietnam. Springer. Jhingran, D., and D. Sankar. 2009. “Addressing Educational Disparity: Using District Level Education Development Indices for Equitable Resource Allocations in India.“ Policy Research Working Paper 4955, World Bank, Washington, DC. OECD (Organisation for Economic Co-operation and Development)/DAC (Development Assistance Committee). 2011. Capacity Development in Aid Business Processes: Getting it Right. Paris: OECD. OECD (Organisation for Economic Co-operation and Development)/DAC (Development Assistance Committee)/LenCD (Learning Network on Capacity Development). 2011. “Key Findings of Perspective Note on Sector Capacity Development.” http://sites.google.com/site/lencdorg/ UNICEF (United Nations Children’s Fund) Lao PDR. 2011. “Schools of Quality: A Case Study on Rights- Based Education Reform in the Lao People’s Democratic Republic.” UNICEF, Vientiane. 142 AboutSABER y SstemA ssesme s ntand Benchmarki ng o f rEduao c n Resut ls(SABER) s ian i nia v e h tathelpscountre issystema cal lyexaminean d t srengt h entheperfor ma nceoftheireducao nsyse tms toachiee v e larnn ig o f ral l.TheWo rldBank s iw ori kngw t ihpartnes r ar ound the word l todeveo l pdiagnosc toos lthatbenchma k r educao npolc iies ac cordingtoevidence- basedglobalst and ad rsand bestpa rcc e. Bylee vragingglobak ln owledge,S A BERfils l agapintheavaia lbii ly t of poi ly c dataand knowledgetoimprove t he qual iy t ofeducao n. hp/ :/www. wol rdbank .og r/d e uc aons /aber “ o f d e b r e u n A t h ao c e s o fi a s c i n el n a e a l x o l w p o r h e k e e r i n f o n e i ga r w n m c e a e t r nn d w s i a w g m y p s h t o e l t e i m m oi nd e m i p n a l e t p g m e r n r o o e o ee v s i n f n t e g p d d t o u u h i l ao c c e c i ao i r e o s n w a n s a l a p l ns it o s y y g l s s v i c i t t e e i e e s s m mi c s o Ic o .I na mp n a s t o c r d i h i s i n m o e kS p n r r s S e o A A h f B B e d E E n i Ra Ri s ff i v e s e r u u w e s e s n f e a t u f y e l u t l .I du o o f t a c o r i s - l onaloutc om essothat suce cssu fl pa rcc escanb eh g ihlg ihe td.I na ddi on,SABER o cntrb iutestothetr ackn ig ofpoi ly c m i pl em n e t ao n,showingtheimpactofthechanges. Andb ea cusethed ataares cun r iz ed, n ifom r ao nsystemsb eo c mem o ea r ccua rte” . Cei ci laMara í Vée lzWh t ief ,ormerMins ie trofEduao c nofCoo lmbi aandfom r e r e Sce r- a tryofEduao c no fBoo gtáC , oo lmba i “ dev e Wea s tak l e T h o p - h p p o eS n i r l g c e d A i e B c o a s r E u t Ri n et n a t n r h d i i e k s e av i n eWo yp ei h t l r a e dB r t sm n p a e u n r s r k a e u ’ g y l t i s e n o e a f ff c i n r o e dv e r . s l e s t ” a v u l a t a n ob b e c l e u .H n a i l dt d i e h g h x - c eS q l e A u e l a n B i l c E t e y Ra p n i s o t l i h y c e as i r h d e a a d e r a t uc dp r aec ao l n ao r u s r c y a i t s mw l e f m i o s t h r . o JukoSa rv, iPr a e cc e LaderE (duao c n, )As ianDev elopmentBank