ICRR 11847 Report Number : ICRR11847 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 09/14/2004 PROJ ID : P004805 Appraisal Actual Project Name : Th-university Science & Project Costs 261.5 153 Eng. Educ US$M ) (US$M) Country : Thailand Loan /Credit (US$M) Loan/ US$M ) 143.4 127.4 Sector (s): Board: ED - Tertiary Cofinancing education (100%) US$M ) (US$M) L/C Number : L4160 Board Approval 97 FY ) (FY) Partners involved : Australia (parallel) Closing Date 12/31/2002 12/31/2003 Prepared by : Reviewed by : Group Manager : Group : Helen Abadzi Denise A. Vaillancourt Alain A. Barbu OEDSG 2. Project Objectives and Components a. Objectives The overall objective of the Universities Science and Engineering Education Project was to improve the quality of undergraduate science and engineering programs. Specifically, the project aimed to: a) strengthen faculty teaching capabilities; b ) upgrade the existing science and engineering program content and broaden the range of programs relevant to Thailand ' s technological advancement; c ) modernize laboratories and strengthen their management; and d) improve resource utilization in engineering and science faculties and establish a system for large -scale equipment procurement. b. Components The project financed an institutional development program for 20 public universities which included : a) short term overseas training for selected academic and technical support staff in the educational use and maintenance of the project-financed equipment (US$2.6 million); b) upgrading the content of existing programs in science and engineering, and recommend new ones, improve the program resource management, develop effective procedures for large scale equipment procurement and strengthen project management (US$12.8 million); (c) modernizing laboratories and strengthen their management (US$130.3 million); and (d) improving resource utilization in the engineering and science faculties to establish a system of large-scale equipment procurement (US$7.3 million). c. Comments on Project Cost, Financing and Dates The project was completed after an extension of one year. Many activities were completed at a cost lower than expected, so the government cancelled US$14 million in 2001. Total project cost was 58% of appraisal estimate. Technical assistance was provided by Australia in a parallel -financed program. 3. Achievement of Relevant Objectives: The objectives were substantially achieved; numerical targets for various activities were attained and surpassed. Examples are as follows: (a) Strengthen faculty teaching capabilities - achieved. International fellowships, in-country training programs were financed for the teaching staff in all 36 recipient faculties through 139 staff development programs (47 in engineering and 92 in science). With the help of Australian funding, 3641 staff were trained in-country and 180 in Australia.. Also 2295 participants received training in pedagogy and curriculum development, 9,268 in English language, and 368 specialized short-term overseas fellowships were also awarded. (b) Upgrade the existing science and engineering programs - achieved. Faculties evaluated existing programs, identified new ones in need of support, and forged linkages with the industry and overseas universities. More than 3000 courses were enhanced or developed. In comparison to 1997, there are now 13% more undergraduate and 82% more graduate programs in engineering as well as 25% more undergraduate and 47% graduate programs in science. Because of limited communication among universities and departments, some efforts were unnecessarily duplicated . ( c) Modernize laboratories - achieved. About 38,671 equipment items were procured and delivered to 295 sites, 12,217 specification documents were written, and 798 staff received training in equipment maintenance. (d) Improve resource utilization in engineering and science faculties - achieved. Technical assistance was provided to the Ministry of University Affairs . All the public universities instituted a new budgeting system that incorporates depreciated costs of equipment and will make it possible to identify and plan for replacing equipment to optimize student learning . 4. Significant Outcomes/Impacts: An evaluation (referred to in the ICR but not specifically cited ) gave strong indications of client satisfaction with the implementation and outcomes of the project. Following training and linkages with foreign universities, teaching and administrative staff were better able to serve the academic needs of students and manage resources. A sector-wide quality assurance and asset management system was developed . The ICR reports that universities have substantially improved their capacity to design and update courses in science and engineering, plan and implement large -scale scientific procurement, operate and maintain sophisticated teaching equipment, and manage large-scale development projects in higher education . 5. Significant Shortcomings (including non-compliance with safeguard policies): The project suffered initial delays partly because staff were not experienced with Bank procurement . The Bank did not approve of an electronic bidding system until later in the project . Equipment procurement in large lots caused some problems and had hidden costs; some suppliers bid outside their scope of ability, did not maintain their contractual commitments, and some equipment did not function properly . Overly high appraisal estimates resulted in cost changes and cancellation of US$ 14 million. The design did not include clear indicators to help measure project outcomes . 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTE: NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. 7. Lessons of Broad Applicability: - With government and stakeholder commitment, it is possible for a large university system to be changed through targeted investments and well-conceived interventions. -The communication among university departments and various universities is not always optimal . Where faculties have similar or related academic interests, careful attention should be given to options for integration and sharing of equipment and other resources. - Existence of procurement advisors in field offices has advantages for projects which require frequent monitoring and evaluation . - In countries with a single language, the command of English language may not be extensive . Overestimating the capacity to read and understand English may cause delays and misunderstandings . -A clear definition of roles and responsibilities is critical for timely and successful project implementation . -Early establishment of baseline data as well as a monitoring and evaluation plan is most useful for tracking program performance. 8. Assessment Recommended? Yes No 9. Comments on Quality of ICR: The ICR is satisfactory overall . However, it could have given more information regarding the evaluation that was carried out and its findings (referred to on p. 4 and elsewhere). It is hoped that the evaluation to be completed in September 2004 (p. 6) will clarify the relationship between inputs and instructional quality improvement. Also, though the Staff Appraisal Report mentions 21 universities, the ICR mentions 20 and does not explain whether one was dropped.