Brunei Darussalam SABER Country Report STUDENT ASSESSMENT 2013 Key Policy Areas for Student Assessment Status 1. Classroom Assessment In Brunei, several formal, system-level documents provide guidelines for classroom assessment. In addition, there are system-level mechanisms to ensure that teachers develop the necessary skills and expertise in classroom assessment. These mechanisms include pre- and in-service teacher training programs and regular opportunities for teachers to participate in conferences and workshops. Teachers also gain experience through being involved in developing and scoring questions for large- scale assessments and examinations. Varied mechanisms are used to monitor and ensure the quality of classroom assessment. For example, at the secondary school level, a standardization process ensures that students are graded according to specified rubrics. Classroom assessment information is required to be disseminated to all key stakeholders, including school district and Ministry of Education officials, parents, and students. 2. Examinations The Brunei Cambridge General Certificate of Education (BC GCE) Examinations have been administered to students in years 12 and 13 since 1975. The purposes of these examinations, which cover more than 15 subject areas, include student certification for school cycle completion, student selection to higher education institutions, and the promotion of competition among schools. Funding for the examinations covers all core activities, including examination design, test administration, data analysis, and data reporting. Brunei offers some opportunities to prepare individuals to work on the examinations, such as funding to attend international programs or workshops on educational measurement and evaluation. However, there are no local courses or workshops that provide classroom teachers with up-to-date guidance and training on the examinations. 3. National Large-Scale Assessment (NLSA) Brunei carried out the National Study of Student Competencies in Mathematics and English (NSSCME) in 2008 (years 4, 6, and 8) and 2010 (year 10), with the goal of monitoring education quality at the system level; supporting schools and teachers; and informing policy design, evaluation, and decision making. While it was planned for the NSSCME to be conducted every two years, it has not been administered since 2010. There is no policy document authorizing the NSSCME and plans for future NSSCME rounds are conveyed through oral communications from Ministry of Education officers. The team in charge of the NSSCME includes assigned officers from the Ministry of Education who work on assessment activities on a temporary basis. Currently, there are no mechanisms in place to monitor the consequences of the NSSCME in terms of its impact on policy dialog, teaching, or learning. 4. International Large-Scale Assessment (ILSA) Brunei has not yet participated in an ILSA. However, it does plan to participate in PISA 2018. BRUNEI DARUSSALAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 SABER-Student Assessment methodology Introduction Brunei Darussalam has focused on increasing student The SABER-Student Assessment framework is built on learning outcomes by improving the quality of education the available evidence base for what an effective in the country. An effective student assessment system assessment system looks like. The framework provides is an important component of improving education guidance on how countries can build more effective quality and learning outcomes as it provides the student assessment systems. The framework is necessary information to meet stakeholders’ decision- structured around two main dimensions of assessment making needs. In order to gain a better understanding of systems: the types/purposes of assessment activities and the strengths and weaknesses of its existing assessment the quality of those activities. system, Brunei decided to benchmark this system using standardized tools developed under The World Bank’s Systems Approach for Better Education Results (SABER) Assessment types and purposes program. SABER is an evidence-based program to help countries systematically examine and strengthen the Assessment systems tend to be comprised of three main performance of different aspects of their education types of assessment activities, each of which serves a systems. different purpose and addresses different information needs. These three main types are: classroom assessment, examinations, and large-scale, system level What is SABER-Student Assessment? assessments. SABER-Student Assessment is a component of the SABER Classroom assessment provides real-time information to program that focuses specifically on benchmarking support ongoing teaching and learning in individual student assessment policies and systems. The goal of classrooms. Classroom assessments use a variety of SABER-Student Assessment is to promote stronger formats, including observation, questioning, and paper- assessment systems that contribute to improved and-pencil tests, to evaluate student learning, generally education quality and learning for all. on a daily basis. Examinations provide a basis for selecting or certifying National governments and international agencies are students as they move from one level of the education increasingly recognizing the key role that assessment of system to the next (or into the workforce). All eligible student learning plays in an effective education system. students are tested on an annual basis (or more often if The importance of assessment is linked to its role in: the system allows for repeat testing). Examinations cover (i) providing information on levels of student the main subject areas in the curriculum and usually learning and achievement in the system; involve essays and multiple-choice questions. (ii) monitoring trends in education quality over time; Large-scale, system-level assessments provide feedback (iii) supporting educators and students with real- on the overall performance of the education system in time information to improve teaching and particular years or age levels. These assessments learning; and typically cover a few subjects on a regular basis (such as (iv) holding stakeholders accountable for results. every 3 to 5 years), are often sample-based, and use multiple-choice and short-answer formats. They may be national or international in scope. Appendix 1 summarizes the key features of these main types of assessment activities. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 2 BRUNEI DARUSSALAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Quality drivers of an assessment system Table 1: Framework for building an effective assessment system, with indicator areas The key considerations when evaluating a student assessment system are the individual and combined quality of assessment activities in terms of the adequacy of the information generated to support decision making. There are three main drivers of information quality in an assessment system: enabling context, system alignment, and assessment quality. Enabling context refers to the broader context in which the assessment activity takes place and the extent to which that context is conducive to, or supportive of, the The indicators are identified based on a combination of assessment. It covers issues such as the legislative or criteria, including: policy framework for assessment activities; institutional and organizational structures for designing, carrying out, or using results from the assessment; the availability of x professional standards for assessment; sufficient and stable sources of funding; and the x empirical research on the characteristics of effective presence of trained assessment staff. assessment systems, including analysis of the characteristics that differentiate between the assessment systems of low- versus high-performing System alignment refers to the extent to which the nations; and assessment is aligned with the rest of the education x theory, that is, general consensus among experts system. This includes the degree of congruence between that it contributes to effective assessment. assessment activities and system learning goals, standards, curriculum, and pre- and in-service teacher training. Levels of development Assessment quality refers to the psychometric quality of The World Bank has developed a set of standardized the instruments, processes, and procedures for the questionnaires and rubrics for collecting and assessment activity. It covers issues such as design and evaluating data on the three assessment types and implementation of assessment activities, analysis and related quality drivers. interpretation of student responses to those activities, and the appropriateness of how assessment results are reported and used. The questionnaires are used to collect data on the characteristics of the assessment system in a particular country. The information from the questionnaires is then Crossing the quality drivers with the different applied to the rubrics in order to judge the development assessment types/purposes provides the framework and level of the country’s assessment system in different broad indicator areas shown in Table 1. This framework areas. is a starting point for identifying indicators that can be used to review assessment systems and plan for their improvement. The basic structure of the rubrics for evaluating data collected using the standardized questionnaires is summarized in Appendix 2. The goal of the rubrics is to provide a country with some sense of the development level of its assessment activities compared to best or recommended practice in each area. For each indicator, the rubric displays four development levels—Latent, SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 3 BRUNEI DARUSSALAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Emerging, Established, and Advanced. These levels are economic growth due to extensive petroleum and artificially constructed categories chosen to represent natural gas fields and is classified by the United Nations key stages on the underlying continuum for each as having the second-highest Human Development Index indicator. Each level is accompanied by a description of among all Southeast Asia nations. Brunei has prioritized what performance on the indicator looks like at that education in order to equip the country with the human level. resources required to support its economic diversification agenda for sustainable growth. In 2012, x Latent is the lowest level of performance; it Brunei committed 17 percent of government represents absence of, or deviation from, the desired expenditure to education. attribute. The formal school system in Brunei provides a minimum x Emerging is the next level; it represents partial of 12 years of education, specifically seven years of presence of the attribute. primary (inclusive of one year in pre-school) and five x Established represents the acceptable minimum years of secondary. In 2007, the Ministry of Education standard. mandated nine years of formal education for all children x Advanced represents the ideal or current best ages 6 to 15. As of 2012, the net school enrolment rate practice. at the primary level is 92 percent and the completion rate is 100 percent. A summary of the development levels for each assessment type is presented in Appendix 3. In 2009, the Ministry of Education introduced the National Education System for the 21st Century (SPN21). SPN21 aims to meet the social and economic challenges In reality, assessment systems are likely to be at of the 21st century and equip students with appropriate different levels of development in different areas. For 21st century skills, including a more balanced curriculum example, a system may be Established in the area of emphasizing assessment for learning. The development examinations, but Emerging in the area of large-scale, of the Ministry of Education Strategic Plan 2012-2017 system-level assessment, and vice versa. While intuition affirms the SPN21 and the Ministry’s commitment suggests that it is probably better to be further along in towards quality education by identifying three as many areas as possible, the evidence is unclear as interlinked strategic focus areas: (i) Teaching and to whether it is necessary to be functioning at Learning Excellence; (ii) Professionalism and Advanced levels in all areas. Therefore, one might view Accountability; and (iii) Efficiency and Innovativeness. the Established level as a desirable minimum outcome to The Education Strategy is part of the country’s overall achieve in all areas, but only aspire beyond that in those plan towards realizing the Brunei Vision 2035, which areas that most contribute to the national vision or aims to transform Brunei by 2035 into a nation widely priorities for education. In line with these considerations, recognized for, among other distinguishing the ratings generated by the rubrics are not meant to be characteristics, its educated and highly-skilled additive across assessment types (that is, they are not population. meant to be added to create an overall rating for an assessment system; they are only meant to produce an Detailed information was collected on Brunei’s student overall rating for each assessment type). The assessment system in 2013 using the SABER-Student methodology for assigning development levels is Assessment questionnaires and rubrics. It is important to summarized in Appendix 4. remember that these tools primarily focus on benchmarking a country’s policies and arrangements for assessment activities at the system or macro level. Education in Brunei Darussalam Additional data would need to be collected to determine actual, on-the-ground practices in Brunei, particularly by teachers and students in schools. The following sections Brunei Darussalam is a high-income, sovereign state discuss the findings by each assessment type, located in Southeast Asia. GDP per capita (current US$) accompanied by suggested policy options. The suggested is $41,126, with annual growth of about 2.2 percent. In policy options were determined in collaboration with key the past 20 years, Brunei has experienced significant SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 4 BRUNEI DARUSSALAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 local stakeholders based on Brunei’s immediate interests conduct SBAfL. Primary school teachers are undergoing and needs. Detailed, completed rubrics for each similar professional training so that comparable capacity assessment type in Brunei are provided in Appendix 5. will exist at the primary level in the near future. In addition, teachers have opportunities to participate in developing and scoring questions for large-scale Classroom Assessment assessments and examinations. Level of development There are a variety of systematic mechanisms in place to monitor the quality of classroom assessment practices. Classroom assessment is a required component of a teacher's performance evaluation, and part of school In Brunei, several formal, system-level documents inspection and teacher supervision. At the secondary provide guidelines for classroom assessment. These school level, a standardization process ensures that documents include: School Based Assessment for students are graded according to specified rubrics. A Learning (SBAfL) Guidebook For Years 7 and 8 Core similar mechanism will be developed for the primary Subjects (Islamic Religious Knowledge (IRK), Malay Islam level in the near future. In addition, a special task force Monarchy (MIB), Malay Language, English Language, has been set up at the national level to monitor the Mathematics & Science); Framework and Guidelines for implementation of SPN21 and classroom assessment in Curriculum and Assessment (by subject for years 1-8); schools. Government funding is being used to carry out and SBAfL Guidebook For Primary Schools (Asas research (by a local education consultancy) on how to (i) Pengetahuan Islam, Melayu Islam Beraja (MIB), Bahasa establish a SBAfL system in primary schools and (ii) Melayu, English Language, Mathematics & Science). develop a set of SBAfL Competency Standards for Brunei teachers. System-wide resources are in place for teachers to carry out classroom assessment activities. For example, the In addition, classroom assessment information is SBAfL Guidebook For Years 7 and 8 Core Subjects (Islamic required to be disseminated to all key stakeholders, Religious Knowledge (IRK), Malay Islam Monarchy (MIB), including school district and Ministry of Education Malay Language, English Language, Mathematics & officials, parents, and students. Science) document and the Framework and Guidelines for Curriculum and Assessment (by subject for years 1-8) Suggested policy options document were produced by the Curriculum Development Department (CDD) and distributed to all 1. Introduce an online system to support teachers in their schools. In addition, subject textbooks and workbooks classroom assessment activities. The online system are available to support classroom assessment activities. could, for example, provide sample scoring criteria and However, other resources, such as scoring criteria or rubrics for grading students’ work as well as on-demand rubrics for grading students' work, or online assessment training modules that are regularly updated. resources, are not systematically available to teachers. 2. Disseminate information on school-based assessment to key stakeholders by using targeted communication System-level mechanisms to ensure that teachers mechanisms. For example, develop radio and television develop skills and expertise in classroom assessment campaigns to target the public and prepare reports include pre-service teacher training for all teachers, in- specifically to address parents’ information needs. service teacher training, and opportunities for teachers to participate in conferences and workshops. At the secondary level (years 7 and 8), all core subject area teachers have undergone training in designing classroom assessment tasks and rubrics to enable them to carry out assessment activities in their classrooms. All secondary schools now have staff with the expertise to SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 5 BRUNEI DARUSSALAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Examinations all students. Reports on the strengths and weaknesses in student performance on a particular examination are Level of development available only to schools. There are no local, up-to-date courses or workshops that provide guidance and training to teachers on the examinations. However, teachers are involved in most The Brunei Cambridge General Certificate of Education tasks for the local subject areas included on the (BC GCE) Advanced Subsidiary (AS) and Advanced (A) examinations, such as Bahasa Melayu and Islamic Level Examinations have been administered to students Religious Studies. The tasks include creating and in years 12 and 13 since 1975. The examinations cover selecting examination questions, administering and the following subjects: Mathematics, Physics, Chemistry, scoring the examination, acting as a judge, supervising Biology, History, Geography, Sociology, Psychology, examination procedures, and resolving inconsistencies Thinking Skills, Economics, Accounting, Art and Design, between examination scores and school grades. Internal Literature in English, Computer Studies, Design and review panels and the use of observers during Technology, Business Studies, Applied ICT, Bahasa examination administration are the only mechanisms in Melayu, and Arabic Subjects. The purposes for which the place to ensure the quality of these local subject examination results are used include student examinations. certification for year and school cycle completion, student selection to higher education institutions, and Inappropriate behavior surrounding the examinations is promotion of competition among schools. marginal and infrequent. Reported incidents include the use of unauthorized materials, such as prepared answers Formal policy documents, including the Education and notes. Enactment 1976 and the Brunei New Education System (SPN 21) of 1975, authorize the examinations. There are some options for students who do not perform well on the examinations. Students may either retake the Funding for the examinations covers some core examinations or repeat the year. However, students are examination activities, including examination design, not provided with any special preparatory or remedial administration, data analysis, and data reporting. courses to prepare them to retake the examinations. Mechanisms to monitor the consequences of the The examinations office has been in charge of the BC GCE examinations are introduced at the discretion of the Examinations since 1975 and is accountable to Ministry of Education. Cambridge International Examinations (CIE), an international body. The examination office has state-of- the-art facilities and resources for carrying out the Suggested policy options examination. 1. Conduct regular workshops on the examinations for Brunei offers some opportunities that prepare teachers; provide targeted training for teachers of individuals to work on the examinations. For example, different subject areas as well as for teachers who funding for attending international programs, courses, or are involved in examination-related tasks. Make workshops on educational measurement and evaluation these trainings easily accessible to all teachers. is available to Ministry of Education staff and school teachers. 2. Introduce an external review process to ensure the In addition, there is comprehensive material available to quality of the examinations for local subjects such as most students to help them prepare to take the Bahasa Melayu and Islamic Religious Studies. Invite examinations. For example, a framework document the relevant subject-matter and assessment experts describing the content and topic areas covered by the to join the external review process. examinations as well as samples of the types of questions that appear on the examinations are widely available to SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 6 BRUNEI DARUSSALAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 National Large-Scale Assessment Opportunities to prepare for work on the NSSCME are available to select Ministry of Education officials and (NLSA) teachers. These opportunities include university courses Level of development on educational measurement and evaluation offered by Universiti Brunei Darussalam as well as funding to attend programs abroad that are specifically focused on educational measurement and evaluation. Teachers selected for professional training by the Ministry of Brunei Darussalam carried out the National Study of Education also have access to a mix of occasional and Student Competencies in Mathematics and English regular courses and workshops on the NSSCME. (NSSCME) in 2008 and 2010 with the assistance of the Australian Council for Educational Research (ACER). The While information from the NSSCME is purportedly used purposes of the NSSCME are to monitor education by stakeholder groups in a way that is consistent with the quality at the system level; support schools and teachers; stated purposes of the assessment, there are no formal and inform policy design, evaluation, and decision mechanisms in place to monitor the impact of the making. In 2008, all students in years 4, 6, and 8 were NSSCME on policy dialog, teaching, or learning. assessed in English and Mathematics. In 2010, all students in year 10 were assessed in English and Suggested policy options Mathematics. In order to include all students in the NSSCME, special plans were made to ensure that it was 1. Develop a policy document that authorizes the administered to students in hard-to-reach areas of the NSSCME. Also, develop a written plan for future country. In addition, the assessment was offered in the assessment activities to help ensure that these language of instruction for almost all student groups. activities are planned and carried out on a regular While the original plan was for the NSSCME to be carried basis, and results are used to inform education out every two years, it has not been administered since priorities. Identify budget requirements to carry out 2010. There is no policy document that authorizes the NSSCME activities and ensure there is continuous NSSCME, and plans for future assessment rounds are and predictable government funding to meet these conveyed only through oral communications from budget needs. Ministry of Education officers. 2. Create a core team that would be responsible for The general understanding is that funding for the next NSSCME activities and ensure that this team receives NSSCME will be made available by the Brunei appropriate and continuous training to successfully Government when the Ministry of Education takes the implement assessment activities during a particular decision to conduct the assessment. However, there is assessment round and from year to year. no formal provision for dedicated funding for the NSSCME. 3. Introduce mechanisms to monitor the impact of the NSSCME on policy dialog, teaching, and learning. For The group of people in charge of the NSSCME includes example, introduce a permanent oversight assigned officers from the Ministry of Education who committee and hold regular focus groups or surveys work on assessment activities on a temporary basis. This of key stakeholders. group of people is accountable to the Ministry of Education. The number of people directed to carry out 4. Consider extending the NCSSME to include subjects NSSCME activities is insufficient to effectively carry out such as Science and Bahasa Melayu. all of the required activities. Funding for the NSSCME has covered assessment design, administration, data analysis, and staff training, but not data reporting or research and development activities. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 7 BRUNEI DARUSSALAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 International Large-Scale Assessment (ILSA) Level of development Brunei Darussalam has not yet participated in an ILSA. However, Brunei does plan to participate in the Program for International Student Assessment (PISA) 2018. Suggested policy options: 1. Create an opportunity for high-level discussion among key stakeholders on education policy questions or problems for which ILSA data could be useful. 2. Determine the need for, and possible next steps in relation to, participation in an ILSA exercise. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 8 BRUNEI DARUSSALAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Appendix 1: Assessment Types and Their Key Differences Classroom Large-scale assessment Examinations Surveys National International Exit Entrance Purpose To provide To provide To provide To certify To select immediate feedback on feedback on the students as they students for feedback to overall health of comparative move from one further inform the system at performance of level of the educational classroom particular the education education opportunities instruction grade/age system at system to the level(s), and to particular next (or into the monitor trends grade/age workforce) in learning level(s) Frequency Daily For individual For individual Annually and Annually and subjects offered subjects offered more often more often on a regular on a regular where the where the basis (such as basis (such as system allows system allows every 3-5 years) every 3-5 years) for repeats for repeats Who is All students Sample or A sample of All eligible All eligible tested? census of students at a students students students at a particular grade particular grade or age level(s) or age level(s) Format Varies from Usually multiple Usually multiple Usually essay Usually essay observation to choice and short choice and short and multiple and multiple questioning to answer answer choice choice paper-and-pencil tests to student performances Coverage of All subject areas Generally Generally Covers main Covers main curriculum confined to a confined to one subject areas subject areas few subjects or two subjects Additional Yes, as part of Frequently Yes Seldom Seldom information the teaching collected from process students? Scoring Usually informal Varies from Usually involves Varies from Varies from and simple simple to more statistically simple to more simple to more statistically sophisticated statistically statistically sophisticated techniques sophisticated sophisticated techniques techniques techniques SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 9 BRUNEI DARUSSALAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Appendix 2: Basic Structure of Rubrics for Evaluating Data Collected on a Student Assessment System Development Level LATENT EMERGING ESTABLISHED (Absence of, or (On the way to (Acceptable deviation from, meeting minimum minimum ADVANCED Dimension attribute) standard) standard) (Best practice) Justification EC—ENABLING CONTEXT EC1—Policies EC2—Leadership, public engagement EC3—Funding EC4—Institutional arrangements EC5—Human resources SA—SYSTEM ALIGNMENT SA1—Learning/quality goals SA2—Curriculum SA3—Pre-, in-service teacher training AQ—ASSESSMENT QUALITY AQ1—Ensuring quality (design, administration, analysis) AQ2—Ensuring effective uses SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 10 BRUNEI DARUSSALAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Appendix 3: Summary of the Development Levels for Each Assessment Type Assessment Type LATENT EMERGING ESTABLISHED ADVANCED Absence of, or On the way to meeting Acceptable minimum Best practice deviation from, the minimum standard standard attribute There is no system- There is weak system-wide There is sufficient There is strong wide institutional institutional capacity to system-wide system-wide capacity to support support and ensure the institutional capacity institutional capacity CLASSROOM and ensure the quality of classroom to support and ensure to support and ensure ASSESSMENT quality of assessment practices. the quality of the quality of classroom classroom assessment classroom assessment assessment practices. practices. practices. There is no There is a partially stable There is a stable There is a stable standardized standardized examination standardized standardized examination in in place, and a need to examination in place. examination in place place for key develop institutional There is institutional and institutional decisions. capacity to run the capacity and some capacity and strong examination. The limited mechanisms mechanisms to EXAMINATIONS examination typically is of to monitor it. The monitor it. The poor quality and is examination is of examination is of high perceived as unfair or acceptable quality quality and is corrupt. and is perceived as perceived as fair and fair for most students free from corruption. and free from corruption. There is no NLSA in There is an unstable NLSA There is a stable NLSA There is a stable NLSA place. in place and a need to in place. There is in place and develop institutional institutional capacity institutional capacity capacity to run the NLSA. and some limited and strong NATIONAL (OR Assessment quality and mechanisms to mechanisms to SYSTEM-LEVEL) impact are weak. monitor it. The NLSA monitor it. The NLSA LARGE-SCALE ASSESSMENT is of moderate quality is of high quality and and its information is its information is disseminated, but not effectively used to always used in improve education. effective ways. There is neither a Participation in an ILSA has There is more or less There is stable history of been initiated, but there stable participation in participation in an participation in an still is need to develop an ILSA. There is ILSA and institutional ILSA nor plans to institutional capacity to institutional capacity capacity to run the INTERNATIONAL participate in one. carry out the ILSA. to carry out the ILSA. ILSA. The information LARGE-SCALE The information from from the ILSA is ASSESSMENT the ILSA is effectively used to disseminated, but not improve education. always used in effective ways. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 11 BRUNEI DARUSSALAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 6. Scores for certain key dimensions under ‘Enabling Appendix 4: Methodology for Assigning Context’ (in the case of EXAM, NLSA, and ILSA) and under Development Levels ‘System Alignment’ (in the case of CLASS) were set as ceiling scores, i.e., the overall mean score for the particular assessment type cannot be greater than the 1. The country team or consultant collects information score for these key dimensions. These key variables about the assessment system in the country. include formal policy, regular funding, having a permanent assessment unit, and the quality of 2. Based on the collected information, a level of assessment practices. development and score is assigned to each dimension in the rubrics: x Latent = 1 score point x Emerging = 2 score points x Established = 3 score points x Advanced = 4 score points 3. The score for each quality driver is computed by aggregating the scores for each of its constituent dimensions. For example: The quality driver, ‘Enabling Context,’ in the case of ILSA, has 3 dimensions on which a hypothetical country receives the following scores: Dimension A = 2 points; Dimension B = 2 points; Dimension C = 3 points. The hypothetical country’s overall score for this quality driver would be: (2+2+3)/3 = 2.33 4. A preliminary level of development is assigned to each quality driver. 5. The preliminary development level is validated using expert judgment in cooperation with the country team and The World Bank Task Team Leader. For scores that allow a margin of discretion (i.e., to choose between two levels of development), a final decision has to be made based on expert judgment. For example, the aforementioned hypothetical country has an ‘Enabling Context’ score of 2.33, corresponding to a preliminary level of development of ‘Emerging or Established.’ Based on qualitative information not captured in the rubric, along with expert judgment, the country team chooses ‘Emerging’ as the most appropriate level. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 12 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Appendix 5: SABER-Student Assessment Rubrics Brunei Darussalam This appendix provides the completed SABER-Student Assessment rubrics for each type of assessment activity in Brunei Darussalam. In each row of the rubric, the relevant selection is indicated by a thick border and an asterisk. The selection may include a superscript number that refers to the justification or explanation for the selection. The explanation or justification text can be located in the “Development-level rating justifications” section at the end of each rubric. If a row includes a superscript, but not a thick border and an asterisk, this means that insufficient information was available to determine the relevant selection in the row. Brunei Darussalam: Classroom Assessment SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 13 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ENABLING CONTEXT AND SYSTEM ALIGNMENT Overall policy and resource framework within which classroom assessment activity takes place in a country or system, and the degree to which classroom assessment activity is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT AND SYSTEM ALIGNMENT 1: Setting clear guidelines for classroom assessment There is no system-level document that There is an informal system-level There is a formal system-level document This option does not apply to this provides guidelines for classroom document that provides guidelines for that provides guidelines for classroom dimension. assessment. classroom assessment. assessment. 1 * This option does not apply to this This option does not apply to this The availability of the document is The document is widely available. 2 dimension. dimension. restricted. * ENABLING CONTEXT AND SYSTEM ALIGNMENT 2: Aligning classroom assessment with system learning goals There are no system-wide resources for There are scarce system-wide resources There are some system-wide resources There are a variety of system-wide teachers for classroom assessment. for teachers for classroom assessment. for teachers for classroom assessment. 3 resources available for teachers for classroom assessment. There is no official curriculum or There is an official curriculum or There is an official curriculum or * There is an official curriculum or standards document. standards document, but it is not clear standards document that specifies what standards document that specifies what what students are expected to learn or to students are expected to learn, but the students are expected to learn and to what level of performance. level of performance required is not clear. what level of performance. 4 ENABLING CONTEXT AND SYSTEM ALIGNMENT 3: * Having effective human resources to carry out classroom assessment activities There are no system-level mechanisms to This option does not apply to this There are some system-level mechanisms There are a variety of system-level ensure that teachers develop skills and dimension. to ensure that teachers develop skills and mechanisms to ensure that teachers expertise in classroom assessment. expertise in classroom assessment. 5 develop skills and expertise in classroom * assessment. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 14 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ASSESSMENT QUALITY Quality of classroom assessment design, administration, analysis, and use. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of classroom assessment Classroom assessment practices suffer Classroom assessment practices are Classroom assessment practices are Classroom assessment practices are from widespread weaknesses or there is known to be weak. known to be of moderate quality. known to be generally of high quality. 6 no information available on classroom assessment practices. There are no mechanisms to monitor the There are ad hoc mechanisms to monitor There are limited systematic mechanisms There are varied and systematic quality of classroom assessment the quality of classroom assessment to monitor the quality of classroom mechanisms in place to monitor the practices. practices. assessment practices. quality of classroom assessment practices. 7 * ASSESSMENT QUALITY 2: Ensuring effective uses of classroom assessment Classroom assessment information is not This option does not apply to this Classroom assessment information is Classroom assessment information is required to be disseminated to key dimension. required to be disseminated to some key required to be disseminated to all key stakeholders. stakeholders. stakeholders. 8 * There are no required uses of classroom There are limited required uses of There are adequate required uses of There are adequate required uses of assessment to support student learning. classroom assessment to support student classroom assessment to support student classroom assessment to support student learning. learning, excluding its use as an input for learning, including its use as an input for external examination results. external examination results. 9 * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 15 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Classroom Assessment: Development-level rating justifications 1. In Brunei, several formal system-level documents provide guidelines for classroom assessment. These documents include: SBAfL Guidebook For Years 7 and 8 Core Subjects (Islamic Religious Knowledge (IRK), Malay Islam Monarchy (MIB), Malay Language, English Language, Mathematics & Science), Framework and Guidelines for Curriculum and Assessment (by subject for years 1-8), and SBAfL Guidebook For Primary Schools (Asas Pengetahuan Islam, Melayu Islam Beraja (MIB), Bahasa Melayu, English Language, Mathematics & Science). 2. SBAfL Guidebook For Years 7 and 8 Core Subjects (Islamic Religious Knowledge (IRK), Malay Islam Monarchy (MIB), Malay Language, English Language, Mathematics & Science) and the Framework and Guidelines for Curriculum and Assessment (by subject for years 1-8) documents are publically available online, in teacher training colleges, and in-service teacher training courses. The SBAfL Guidebook For Primary Schools (Asas Pengetahuan Islam, Melayu Islam Beraja (MIB), Bahasa Melayu, English Language, Mathematics & Science) is being printed. 3. The SBAfL Guidebook For Years 7 and 8 Core Subjects (Islamic Religious Knowledge (IRK), Malay Islam Monarchy (MIB), Malay Language, English Language, Mathematics & Science) document and the Framework and Guidelines for Curriculum and Assessment (by subject for years 1-8) document have been produced by the Curriculum Development Department (CDD) and distributed to all schools. In all core subject documents, the General Learning Outcome (GLO) and Achievement Standard documents by years are used as reference by schools in developing their assessment tasks. (The General Learning Outcomes and Achievement Standards documents are available only for Years 7 and 8.) "Success criteria,” or performance indicators, are emphasized in teachers’ daily AfL lessons for primary Mathematics to help teachers in designing assessment tasks. In addition, some core subject documents have lists of skills to be taught and teachers are expected to use their own discretion to select some of these skills in designing assessment tasks. Subject textbooks, at all levels from Year 1 to Year 8, are written according to curricular and assessment requirements, with the exception that primary schools are allowed to choose their own English textbooks which cover curricular requirements, since only supplementary texts have been recommended by the CDD. While most subjects have workbooks to accompany the textbooks, there are no separate workbooks recommended for English (Years 1-6) and Mathematics (Years 7 and 8). These books provide sufficient examples and practices to indicate the depth and breadth of assessment of topics learned. However, there are no scoring criteria or rubrics for grading students' work, item banks or pools with examples of selection/multiple-choice or supply/open-ended questions, online assessment resources, computer-based testing with instant reports on students' performance systematically available to teachers for their classroom assessment practices. 4. The SBAfL Guidebook For Years 7 and 8 Core Subjects (Islamic Religious Knowledge (IRK), Malay Islam Monarchy (MIB), Malay Language, English Language, Mathematics & Science), and the Framework and Guidelines for Curriculum and Assessment (by subject for years 1-8) documents outline what students in different years/at different age levels are expected to learn and to what performance level. 5. System-level mechanisms to ensure that teachers develop skills and expertise in classroom assessment include pre-service teacher training that all teachers benefit from, in-service teacher training, and opportunities for teachers to participate in conferences and workshops. At the secondary level (Years 7 and 8), all core subject teachers have undergone special training (in 2010 and 2011) to design assessment tasks and rubrics so that they are able to carry out assessment for learning in their own classrooms. All secondary schools are now equipped with expertise to conduct SBAfL in their own schools. Primary school teachers are undergoing similar professional training and a similar mechanism will be established for the primary level in the near future. In addition, teachers have opportunities to participate in item development for, or scoring of, large-scale assessments or exams. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 16 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 6. While it is common for classroom assessment activities to be mainly about recalling information, it is uncommon for teachers to rely mainly on multiple-choice, selection-type questions or to not use explicit or a priori criteria for scoring or grading students' work. It is also uncommon to observe errors in the scoring or grading of students' work. It is rare for uneven application of standards for grading students’ work to be a serious problem, for grade inflation to be a serious problem, or for parents to be poorly informed of their children's grades. 7. Classroom assessment is a required component of a teacher's performance evaluation and school inspection or teacher supervision. At the secondary school level, social standardization is carried out to moderate the award of grades to students' based on agreed rubrics. A similar mechanism will be developed for the primary level in the near future. In addition, a special task force at the national level has been set up to monitor the implementation of SPN21 and classroom assessment at schools. Government funding was used to carry out a research initiative (through a local educational consultancy) to establish (i) a SBAfL system in primary schools in Brunei Darussalam and (ii) Brunei Teachers' SBAfL Competency Standards. 8. Classroom assessment information is required to be disseminated to all key stakeholders, including school district and Ministry of Education officials, parents, and students. 9. Classroom assessment information is required to be used for diagnosing student learning issues, providing feedback to students on their learning, informing parents about their child's learning, planning next steps in instruction, and grading students for internal classroom uses. In addition, classroom assessment information is required to be used for providing input to an external examination program, the Primary School Assessment, in Year 6. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 17 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 BRUNEI DARUSSALAM Examinations SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 18 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which assessment activity takes place in a country or system and the extent to which that framework is conducive to, or supportive of, the assessment activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies No standardized examination has taken The standardized examination has been The examination is a stable program that This option does not apply to this place. operating on an irregular basis. has been operating regularly. 1 dimension There is no policy document that There is an informal or draft policy * There is a formal policy document that This option does not apply to this authorizes the examination. document that authorizes the authorizes the examination.2 dimension. examination. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. the public. public. 3 dimension. This option does not apply to this This option does not apply to this * The policy document addresses some key The policy document addresses all key dimension. dimension. aspects of the examination. 4 aspects of the examination. * ENABLING CONTEXT 2: Having strong leadership All stakeholder groups strongly oppose Most stakeholder groups oppose the Most stakeholders groups support the All stakeholder groups support the the examination or are indifferent to it. examination. examination. examination. 5 * There are no attempts to improve the This option does not apply to this There are independent attempts to There are coordinated attempts to examination by stakeholder groups. dimension. improve the examination by stakeholder improve the examination by stakeholder groups. groups. 6 * Efforts to improve the examination are This option does not apply to this Efforts to improve the examination are This option does not apply to this not welcomed by the leadership in charge dimension. generally welcomed by the leadership in dimension. of the examination charge of the examination. 7 * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 19 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 (CONTINUED) LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having regular funding There is no funding allocated for the There is irregular funding allocated for There is regular funding allocated for the This option does not apply to this examination. the examination. examination.8 dimension. * This option does not apply to this Funding covers some core examination Funding covers all core examination This option does not apply to this dimension. activities: design, administration, data activities: design, administration, data dimension. processing or reporting. processing and reporting. 9 This option does not apply to this Funding does not cover research and This option does not apply to this * Funding covers research and dimension. development. 10 dimension. development. * ENABLING CONTEXT 4: Having strong organizational structures The examination office does not exist or is The examination office is newly The examination office is a stable This option does not apply to this newly established. established. organization. 11 dimension. * The examination office is not accountable This option does not apply to this The examination office is accountable to This option does not apply to this to an external board or agency. dimension. an external board or agency. 12 dimension. * Examination results are not recognized by Examination results are recognized by Examination results are recognized by Examination results are recognized by any certification or selection system. certification or selection system in the one certification or selection system in two or more certification or selection country. another country. system in another country. 13 * The examination office does not have the The examination office has some of the The examination office has all of the The examination office has state-of-the- required facilities to carry out the required facilities to carry out the required facilities to carry out the art facilities to carry out the examination. 14 examination. examination. examination. (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 20 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 5: Having effective human resources There is no staff to carry out the The examination office is inadequately The examination office is adequately The examination office is adequately examination. staffed to effectively carry out the staffed to carry out the examination staffed to carry out the assessment examination, issues are pervasive. effectively, with minimal issues. 15 effectively, with no issues. * The country does not offer opportunities This option does not apply to this The country offers some opportunities The country offers a wide range of that prepare for work on the dimension. that prepare for work on the opportunities that prepare for work on examination. examination. 16 the examination. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 21 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 SYSTEM ALIGNMENT Degree to which the assessment is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Aligning examinations with learning goals and opportunities to learn It is not clear what the examination This option does not apply to this There is a clear understanding of what the This option does not apply to this measures. dimension. examination measures.17 dimension. What the examination measures is This option does not apply to this What is measured by the examination is * This option does not apply to this questioned by some stakeholder groups. dimension. largely accepted by stakeholder groups. 18 dimension. * Material to prepare for the examination is There is some material to prepare for the There is comprehensive material to There is comprehensive material to minimal and it is only accessible to very examination that is accessible to some prepare for the examination that is prepare for the examination that is few students. students. accessible to most students. 19 accessible to all students. * SYSTEM ALIGNMENT 2: Providing teachers with opportunities to learn about the examination There are no courses or workshops on There are no up-to-date courses or There are up-to-date voluntary courses or There are up-to-date compulsory courses examinations available to teachers. workshops on examinations available to workshops on examinations available to or workshops on examinations for teachers. 20 teachers. teachers. * Teachers are excluded from all Teachers are involved in very few Teachers are involved in some Teachers are involved in most examination-related tasks. examination-related tasks. examination-related tasks. examination-related tasks. 21 * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 22 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ASSESSMENT QUALITY Degree to which the assessment meets quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring quality There is no technical report or other There is some documentation on the There is a comprehensive technical report There is a comprehensive, high quality documentation. 22 examination, but it is not in a formal but with restricted circulation. technical report available to the general report format. public. * There are no mechanisms in place to This option does not apply to this There are limited systematic mechanisms There are varied and systematic ensure the quality of the examination. dimension. in place to ensure the quality of the mechanisms in place to ensure the quality examination.23 of the examination. * ASSESSMENT QUALITY 2: Ensuring fairness Inappropriate behavior surrounding the Inappropriate behavior surrounding the Inappropriate behavior surrounding the Inappropriate behavior surrounding the examination process is high. examination process is moderate. examination process is low. examination process is marginal. 24 * The examination results lack credibility The examination results are credible for The examination results are credible for This option does not apply to this for all stakeholder groups. some stakeholder groups. all stakeholder groups. 25 dimension. * The majority of the students (over 50 A significant proportion of students (10 A small proportion of students (less than All students can take the examination; percent) may not take the examination percent to50 percent) may not take the 10 percent) may not take the examination there are no language, gender or other because of language, gender, or other examination because of language, because of language, gender, or other equivalent barriers. 26 equivalent barriers. gender, or other equivalent barriers. equivalent barriers. * (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 23 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 3: Using examination information in a fair way Examination results are not used in a Examination results are used by some Examination results are used by most Examination results are used by all proper way by all stakeholder groups. stakeholder groups in a proper way. stakeholder groups in a proper way. stakeholder groups in a proper way. 27 * 28 Student names and results are public. This option does not apply to this Students’ results are confidential. This option does not apply to this dimension. dimension. * ASSESSMENT QUALITY 4: Ensuring positive consequences of the examination There are no options for students who do There are very limited options for There are some options for students who There is a variety of options for students not perform well on the examination, or students who do not perform well on the do not perform well on the examination. who do not perform well on the 29 students must leave the education examination. examination. system. * There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to There is a variety of mechanisms in place monitor the consequences of the dimension. monitor the consequences of the to monitor the consequences of the examination. examination. 30 examination. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 24 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Examinations: Development-level rating justifications 1. In Brunei, the BC GCE AS and A Level Examinations have been administered since 1975 to students in years 12 and 13 in the following subjects: Mathematics, Physics, Chemistry, Biology, History, Geography, Sociology, Psychology, Thinking Skills, Economics, Accounting, Art and Design, Literature in English, Computer Studies, Design and Technology, Business studies, Applied ICT, Bahasa Melayu, and Arabic Subjects. The purposes of the examinations include student certification for school year/school cycle completion, selection to higher education institutions, as well as for promoting competition among schools. 2. Formal policy documents, the Education (Brunei Board of Examinations) Enactment of 1976 document and the Brunei New Education System (SPN 21) document authorize the examinations. 3. The Education (Brunei Board of Examinations), Enactment 1976 document is publically available and easily accessible to the public, while the Brunei New Education System (SPN 21) document is not publically available. 4. The policy documents include information on the governance, distribution of power, and responsibilities among key entities, and describe the purpose of the examinations and the authorized uses of examination results. The documents also outline procedures for special/disadvantaged students, specify who can sit for the examination, explain the format of the examination questions, and the alignment of the examination with curricula and standards. 5. All key stakeholder groups support the examinations, while teachers are neutral to them. 6. Coordinated efforts to improve the examinations have been made by stakeholder groups. 7. Efforts to improve the examination are generally welcomed by the leadership in charge of the examination. 8. Regular funding is allocated for the examination by the government. 9. Funding covers all core examination activities, including examination design, administration, data analysis, and data reporting. 10. Funding does not cover research and development. 11. The examination office has been in charge of the examinations since 1975. 12. The examination office is accountable to the Cambridge International Examinations (CIE), which is an international body. 13. Examination results are recognized in Brunei as well as in various selection systems abroad, such as in South Africa and Australia. 14. The examination office has state-of-the-art facilities and resources for carrying out the examination. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 25 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 15. Although there is permanent or full-time staff, it is insufficient in terms of quantity and quality to carry out the examination. However, there are no reported issues regarding the human resources responsible for the examination. 16. Funding for attending international programs, courses, and workshops on educational measurement and evaluation are available for Ministry of Education staff and school teachers. The country does not offer university graduate programs (masters or doctorate level) or courses (graduate or non-graduate) specifically focused on educational measurement and evaluation. The country also does not offer non university training courses or workshops on educational measurement and evaluation. Internships are not available in the examination office. 17. The examination measures national and internationally recognized guidelines or standards, specifically using the University of Cambridge International Examination Syndicate. 18. What is measured by the examination is largely accepted by stakeholders. 19. Examples of the types of questions that are on the examination and the framework document explaining what is measured on the examination are widely available to all students. The report on the strengths and weaknesses in student performance is available only to schools. Information on how to prepare for the examination is not available. 20. Only voluntary courses and workshops on the examination are available to teachers. These courses and workshops are not regularly updated. 21. Teachers are involved in most tasks for the local subjects in the examination, such as Bahasa Melayu and Islamic Religious Studies. Tasks include selecting and creating examination questions, administering and scoring the examination, acting as a judge, supervising examination procedures, and resolving inconsistencies between examination scores and school grades. 22. There is no technical report or other documentation. 23. Internal review and observers are the only mechanisms in place to ensure the quality of the examination. Other mechanisms, such as external review or observers, external certification or audit, or pilot or field testing do not take place. 24. Inappropriate behavior surrounding the examination is marginal. Only the use of unauthorized materials, such as prepared answers and notes, has taken place during the examination but very rarely. 25. Examination results are perceived as credible by all stakeholder groups. 26. All students can take the examination. 27. There is no evidence of improper use of examination results by any of the stakeholder groups. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 26 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 28. Only the student and persons with a legitimate, professional interest in the test taker can know the examination results. 29. Students who do not perform well on the examination can either retake the examination or repeat the year. Students cannot attend remedial or preparatory courses in order to prepare to retake the examination, or opt for less selective schools, universities, or tracks. 30. Mechanisms to monitor the consequences of the examination are introduced at the discretion of the Ministry of Education. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 27 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 BRUNEI DARUSSALAM National (or System-Level) Large-Scale Assessment (NLSA) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 28 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which NLSA activity takes place in a country or system and the extent to which that framework is conducive to, or supportive of, the NLSA activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies for NLSA No NLSA exercise has taken place. The NLSA has been operating on an The NLSA is a stable program that has This option does not apply to this irregular basis. 1 been operating regularly. dimension. * There is no policy document pertaining to There is an informal or draft policy There is a formal policy document that This option does not apply to this NLSA. 2 document that authorizes the NLSA. authorizes the NLSA. dimension. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. the public. public. dimension. There is no plan for NLSA activity. This option does not apply to this There is a general understanding that the There is a written NLSA plan for the dimension. NLSA will take place. 3 coming years. * ENABLING CONTEXT 2: Having strong public engagement for NLSA All stakeholder groups strongly oppose Some stakeholder groups oppose the Most stakeholders groups support the All stakeholder groups support the NLSA. the NLSA or are indifferent to it. 4 NLSA. NLSA. (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 29 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having regular funding for NLSA There is no funding allocated to the NLSA. There is irregular funding allocated to the There is regular funding allocated to the This option does not apply to this NLSA. 5 NLSA. dimension. * This option does not apply to this Funding covers some core NLSA activities: Funding covers all core NLSA activities: This option does not apply to this dimension. design, administration, analysis and design, administration, analysis and dimension. reporting. 6 reporting. * This option does not apply to this Funding does not cover research and This option does not apply to this Funding covers research and dimension. development activities. 7 dimension. development activities. * ENABLING CONTEXT 4: Having strong organizational structures for NLSA There is no NLSA office, ad hoc unit or The NLSA office is a temporary agency or The NLSA office is a permanent agency, This option does not apply to this team. group of people. 8 institution or unit. dimension. This option does not apply to this * Political considerations regularly hamper Political considerations sometimes Political considerations never hamper dimension. technical considerations. hamper technical considerations. technical considerations. 9 This option does not apply to this The NLSA office is not accountable to a The NLSA office is accountable to a clearly This option does not apply to this * dimension. clearly recognized body. recognized body. 10 dimension. * (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 30 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 5: Having effective human resources for NLSA There is no staff allocated for running an The NLSA office is inadequately staffed to The NLSA office is adequately staffed to The NLSA office is adequately staffed to NLSA. effectively carry out the assessment. 11 carry out the NLSA effectively, with carry out the NLSA effectively, with no minimal issues. issues. * The country does not offer opportunities This option does not apply to this The country offers some opportunities to The country offers a wide range of that prepare individuals for work on dimension. prepare individuals for work on the NLSA. opportunities to prepare individuals for 12 NLSA. work on the NLSA. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 31 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 SYSTEM ALIGNMENT Degree to which the NLSA is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Aligning the NLSA with learning goals It is not clear if the NLSA is based on This option does not apply to this The NLSA measures performance against This option does not apply to this curriculum or learning standards. dimension. curriculum or learning standards. 13 dimension. * What the NLSA measures is generally This option does not apply to this What the NLSA measures is questioned by What the NLSA measures is largely questioned by stakeholder groups. dimension. some stakeholder groups. accepted by stakeholder groups. 14 * There are no mechanisms in place to There are ad hoc reviews of the NLSA to There are regular internal reviews of the This option does not apply to this ensure that the NLSA accurately ensure that it measures what it is NLSA to ensure that it measures what it is dimension. measures what it is supposed to measure. intended to measure. intended to measure. 15 SYSTEM ALIGNMENT 2: * Providing teachers with opportunities to learn about the NLSA There are no courses or workshops on the There are occasional courses or There are some courses or workshops on There are widely available high quality NLSA. workshops on the NLSA. the NLSA offered on a regular basis. 16 courses or workshops on the NLSA offered on a regular basis. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 32 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ASSESSMENT QUALITY Degree to which the NLSA meets technical standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of the NLSA No options are offered to include all This option does not apply to this At least one option is offered to include all Different options are offered to include all groups of students in the NLSA. dimension. groups of students in the NLSA. 17 groups of students in the NLSA. * There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to There are a variety of mechanisms in ensure the quality of the NLSA. dimension. ensure the quality of the NLSA. place to ensure the quality of the NLSA. 18 There is no technical report or other There is some documentation about the There is a comprehensive technical report There is a comprehensive, high quality * documentation about the NLSA. 13 technical aspects of the NLSA, but it is not but with restricted circulation. 19 technical report available to the general in a formal report format. public. * ASSESSMENT QUALITY 2: Ensuring effective uses of the NLSA NLSA results are not disseminated. NLSA results are poorly disseminated. NLSA results are disseminated in an This option does not apply to this effective way. 20 dimension. * NLSA information is not used or is used in This option does not apply to this NLSA results are used by some NLSA information is used by all ways inconsistent with the purposes or dimension. stakeholder groups in a way that is stakeholder groups in a way that is the technical characteristics of the consistent with the purposes and consistent with the purposes and assessment. technical characteristics of the technical characteristics of the 21 assessment. assessment. There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to * There are a variety of mechanisms in monitor the consequences of the NLSA. 22 dimension. monitor the consequences of the NLSA. place to monitor the consequences of the NLSA. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 33 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 National (of System-Level) Large Scale Assessment (NLSA): Development-level rating justifications 1. Brunei Darussalam carried out the National Study of Student Competencies in Mathematics and English (NSSCME) in 2008 and 2010 with the assistance of ACER. The purposes of the NSSCME are to monitor education quality at the system level, and to support schools and teachers and policy design, evaluation, or decision making. In 2008, all students in years 4, 6, and 8 were assessed in English and Mathematics. In 2010, all students in year 10 were assessed in English and Mathematics. Future assessment rounds are planned to be carried out every two years. However, the NSSCME has not yet been institutionalized. 2. There is no policy document that authorizes the NSSCME. 3. Plans for future NSSCME assessment rounds are communicated through oral communications from Ministry of Education officers. 4. It is unclear to what extent stakeholders support or oppose the NSSCME. 5. Funding is made available from Brunei Government when the Ministry of Education takes the decision to conduct Large-Scale Assessment. However, dedicated and predictable funding for the NSSCME is not available. 5. Funding covers assessment design, administration, and data analysis, and staff training, but does not cover data reporting. 7. Funding does not cover research and development activities. 8. The group of people in charge of the NSSCME includes assigned officers from the Ministry of Education who work on the assessment activities on a temporary basis. 9. Political considerations never hamper technical considerations. 10. The group of people in charge of the NSSCME is accountable to the Ministry of Education. 11. There is an insufficient number of people allocated for carrying out all aspects of the NSSCME. 12. Selected Ministry of Education officers and teachers can attend University courses (graduate and non-graduate) on educational measurement and evaluation in Universiti Brunei Darussalam and at universities abroad. Funding is also available for selected Ministry of Education officers and teachers to pursue university graduate programs (masters or doctorate level) as well as to attend international programs or workshops specifically focused on educational measurement and evaluation abroad. Selected Ministry of Education officers and teachers may also attend professional development courses organized by the Ministry of Education. 13. The NSSCME measures performance against national curriculum guidelines and learning standards. 14. What is measured by the NSSCME is largely accepted by stakeholder groups. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 34 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 15. There have been regular internal reviews and independent reviews by qualified experts of the alignment between the assessment instrument and what it is supposed to measure. 16. Training courses, workshops and presentations are occasionally offered to teachers. Certain training courses and workshops are offered to teachers on a regular basis as well. The training courses and workshops are made available to teachers who are selected for professional training opportunities by the Ministry of Education. Most courses provide teachers with relevant resources that they can use in their classrooms. 17. In order to include all students in the NSSCME, special plans are made to ensure that the NSSCME is administered to students in hard-to-reach areas, and the assessment is offered in the language of instruction for almost all student groups. 18. A variety of mechanisms are in place to ensure the quality of the NSSCME. Specifically, a pilot is conducted before the main data collection takes place. There is also a standardized manual for the NSSCME administrator and scorers are trained to ensure high interrater reliability. All booklets are numbered, and discrepancies must be recorded on a standard sheet. Additionally, there is double processing of data and an external certification or audit takes place. External reviewers or observers are also brought on. 19. There is a comprehensive technical report, but with restricted circulation. 20. NSSCME results are disseminated within 12 months of the assessment being administered and reports with results are made available for all stakeholder groups. The main reports on the results contain information on overall achievement levels and trends over time by subgroups. The main reports on the results also contain standard errors. There are workshops or presentations for key stakeholders on the results, and results are featured in newspapers, magazines, radio, or television. 21. NSSCME information is used by some stakeholder groups in a way that is consistent with the stated purposes or technical characteristics of the assessment. 22. There are no mechanisms in place to monitor the consequences of the NSSCME. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 35 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 BRUNEI DARUSSALAM International Large-Scale Assessment (ILSA) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 36 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which ILSA takes place in a country or system and the extent to which that framework is conducive to, or supportive of, ILSA activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies for ILSA The country/system has not participated This option does not apply to this The country/system has participated in at The country/system has participated in in an ILSA in the last 10 years. 1 dimension. least one ILSA in the last 10 years. two or more ILSA in the last 10 years. * The country/system has not taken This option does not apply to this The country/system has taken concrete This option does not apply to this concrete steps to participate in an ILSA in dimension. steps to participate in at least one ILSA in dimension. the next 5 years. the next 5 years. There is no policy document that There is an informal or draft policy There is a formal policy document that This option does not apply to this addresses participation in ILSA. document that addresses participation in addresses participation in ILSA. dimension. ILSA. This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. the public. public. dimension. ENABLING CONTEXT 2: Having regular funding for ILSA There is no funding for participation in There is funding from loans or external There is regular funding allocated at There is regular funding approved by law, ILSA. donors. discretion. decree or norm. This option does not apply to this Funding covers some core activities of the Funding covers all core activities of the This option does not apply to this dimension. ILSA. ILSA. dimension. Funding does not cover research and This option does not apply to this This option does not apply to this Funding covers research and development activities. dimension. dimension. development activities. (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 37 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having effective human resources for ILSA There is no team or national/system There is a team or national/system There is a team and national/system This option does not apply to this coordinator to carry out the ILSA coordinator to carry out the ILSA coordinator to carry out the ILSA dimension. activities. activities. activities. This option does not apply to this The national/system coordinator or other The national/system coordinator is fluent This option does not apply to this dimension. designated team member may not be in the language of the assessment. dimension. fluent in the language of the assessment. This option does not apply to this The ILSA office is inadequately staffed or The ILSA office is adequately staffed or The ILSA office is adequately staffed and dimension. trained to carry out the assessment trained to carry out the ILSA effectively, trained to carry out the ILSA effectively, effectively. with minimal issues. with no issues. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 38 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 SYSTEM ALIGNMENT Degree to which the ILSA meets technical quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Providing opportunities to learn about ILSA The ILSA team has not attended The ILSA team attended some The ILSA team attended all international This option does not apply to this international workshops or meetings. international workshops or meetings. workshops or meetings. dimension. The country/system offers no This option does not apply to this The country/system offers some The country/system offers a wide range opportunities to learn about ILSA. dimension. opportunities to learn about ILSA. of opportunities to learn about ILSA. This option does not apply to this This option does not apply to this Opportunities to learn about ILSA are Opportunities to learn about ILSA are dimension. dimension. available to the country's/system's ILSA available to a wide audience, in addition team members only. to the country's/system's ILSA team members. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 39 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ASSESSMENT QUALITY Degree to which the ILSA meets technical quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of ILSA Data from the ILSA has not been The country/system met sufficient The country/system met all technical This option does not apply to this published. standards to have its data presented standards required to have its data dimension. beneath the main display of the presented in the main displays of the international report or in an annex. international report. The country/system has not contributed This option does not apply to this This option does not apply to this The country/system has contributed new new knowledge on ILSA. dimension. dimension. knowledge on ILSA. ASSESSMENT QUALITY 2: Ensuring effective uses of ILSA If any, country/system-specific results Country/system-specific results and Country/system-specific results and Country/system-specific results and and information are not disseminated in information are disseminated irregularly information are regularly disseminated in information are regularly and widely the country/system. in the country/system. the country/system. disseminated in the country/system. Products to provide feedback to schools This option does not apply to this Products to provide feedback to schools Products to provide feedback to schools and educators about the ILSA results are dimension. and educators about the ILSA results are and educators about ILSA results are not made available. sometimes made available. systematically made available. There is no media coverage of the ILSA There is limited media coverage of the There is some media coverage of the ILSA There is wide media coverage of the ILSA results. ILSA results. results. results. If any, country/system-specific results Results from the ILSA are used in a limited Results from the ILSA are used in some Results from the ILSA are used in a variety and information from the ILSA are not way to inform decision making in the ways to inform decision making in the of ways to inform decision making in the used to inform decision making in the country/system. country/system. country/system. country/system. It is not clear that decisions based on ILSA This option does not apply to this This option does not apply to this Decisions based on the ILSA results have results have had a positive impact on dimension. dimension. had a positive impact on students' students' achievement levels. achievement levels. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 40 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 International Large Scale Assessment (ILSA): Development-level rating justifications 1. Brunei has not participated in an ILSA. Brunei does plan to participate in PISA 2018. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 41 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Acknowledgements Department of Curriculum Development, This report was prepared by The World Bank SABER- Ministry of Education Student Assessment team in collaboration with Dr. x Dayang Hajah Kamlah binti Haji Daud, Acting Rosmawijah Jawawi, Deputy Dean, Sultan Hassanal Director of Examination, Department of Bolkiah Institute of Education (SHBIE), Universiti Brunei Examination, Ministry of Education Darussalam, and the following contributors: x Dayang Nadia Liew See Hoon binti Abdullah, Department of Planning, Development and x Dato Paduka Dr Haji Zulkarnain bin Haji Hanafi, Research, Ministry of Education Permanent Secretary (Higher Education), x Awang Haji Mohammad Ishak bin Haji Ahmad, Ministry of Education Department of Planning, Development and x Dr. Haji Junaidi bin Haji Abdul Rahman, Research, Ministry of Education Permanent Secretary (Core Education), x Wan Abdul Rahman bin Wan Ibrahim, Ministry of Education Department of Planning, Development and x Dr. Hajah Anita Binurul Zahrina binti Pehin Research, Ministry of Education Orang Kaya Laila Wijaya Dato Seri Setia Haji x Dayang Ramnah binti Haji Ismail, Department of Awang Abdul Aziz, Deputy Permanent Secretary Planning, Development and Research, Ministry (Higher Education), Ministry of Education of Education x Awangku Abdullah bin Pengiran Haji Tengah, x Dayang Norlina binti Haji Mohd Noor, Deputy Permanent Secretary (Higher Department of Planning, Development and Education), Ministry of Education (until Research, Ministry of Education November 2013) x Dayang Siti Noorihan binti Haji Daud, x Dr. Hajah Romaizah binti Haji Mohd Salleh, Department of Planning, Development and Deputy Permanent Secretary (Core Education), Research, Ministry of Education Ministry of Education x Pengiran Hajah Sarah binti Pengiran Haji x Dayang Hajah Noorhadizah binti Haji Sulaiman, Department of Planning, Development and Kamaluddin, Director General, Director General Research, Ministry of Education of Education’s Office, Mininstr of Education x Dayang Hajah Noor Khadizah binti Haji Abu x Dr. Hazri bin Haji Kifle, Acting Director of Naim, Department of Planning, Development Planning, Development and Research, Ministry and Research, Ministry of Education of Education x Dayang Titi Sulastri binti Haji Timbang, x Datin Dr. Hajah Asmah binti Haji Morni, Director Department of Planning, Development and of Schools, Department of Schools, Ministry of Research, Ministry of Education Education x Awang Cheong Huat Joo, Department of x Awang Haji Kassim bin Haji Mohammad Yassin, Schools, Ministry of Education Acting Director of Schools, Department of Schools, Ministry of Education (until December x Awang Haji Azis bin Haji Nayan, Department of 2013) Schools, Ministry of Education x Dayang Tan Lian Lian, Director of Schools’ x Awang Mohammad Riaz bin Haji Mohammad Inspectorate, Department of Schools’ Amin, Department of Schools’ Inspectotare, Inspectotare, Ministry of Education Ministry of Education x Awang Haji Abdul Rahman bin Haji Nawi, x Dayang Hajah Marzidah binti Haji Majid, Director of Curriculum Development, Department of Curriculum Development, Ministry of Education SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 42 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 x Awang Ali Hazis bin Haji Mohammad Tahir, Department of Examination, Ministry of Education x Awang Suhaili bin Haji Abdul Karim, Department of Examination, Ministry of Education x Dr. Jainatul Halida binti Jaidin, Universiti Brunei Darussalam x Associate Professor Dr. Saratha V. Sithamparam, Universiti Brunei Darussalam References Clarke, M. 2012. “What Matters Most for Student Assessment Systems: A Framework Paper.” READ/SABER Working Paper Series. Washington, DC: World Bank. Strategic Management Unit, Department of Planning, Development and Research, Ministry of Education. 2012. “The Ministry of Education Strategic Plan 2012-2017.” Brunei Darussalam. Data retrieved from http://www.mosbn.com/strategicplan.pdf on December 22, 2013. United Nations Development Programme. 2013. Human Development Report 2013. New York, NY. Data retrieved from http://hdr.undp.org/en/2013-report on December 22, 2013. World Bank. 2012. Brunei Darussalam Country Indicator Data. Washington, DC: World Bank. Data retrieved from http://data.worldbank.org/ on December 22, 2013. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 43 BRUNEI DARUSSALAM ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 www.worldbank.org/education/saber The Systems Approach for Better Education Results (SABER) initiative produces comparative data and knowledge on education policies and institutions, with the aim of helping countries systematically strengthen their education systems. SABER evaluates the quality of education policies against evidence-based global standards, using new diagnostic tools and detailed policy data. The SABER country reports give all parties with a stake in educational results—from administrators, teachers, and parents to policymakers and business people—an accessible, objective snapshot showing how well the policies of their country's education system are oriented toward ensuring that all children and youth learn. This report focuses specifically on policies in the area of student assessment. This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 44