Report No. 46324-PK Pakistan Sindh Province Baseline Indicators System Baseline Procurement Performance Assessment Report Government of Sindh and Development Partners UK's Department The European for International 1 Government of Sindh The World Bank Group Commission Development Asian Development Bank ABBREVIATIONS NAB National Accountability Bureau Sindh OECD-DAC Organization for Economic Co-operation and Development - Development Assistance Committee PEC Pakistan Engineering Council SBD standard bidding document SPPRA Sindh Public Procurement Regulatory Authority ii CONTENTS Assessment Results Summary ........................................................................................ 5 A. Background ..............................................................................................................10 B. Procurement Assessment Using the OECD-DAC Baseline Indicators System.................10 C. Planning and Preparing for the Assessment ...............................................................12 D. Assessment Results ..................................................................................................13 Pillar I: The legislative and regulatory framework 13 Indicator 1. Public procurement legislative and regulatory framework complies with applicable obligations with regard to national and international requirements. ............................................... 14 Indicator 2. Existence of implementing regulations and documentation. .... 18 Pillar II: Institutional framework and management capacity 21 Indicator 3. The public procurement system is mainstreamed and well integrated into the public sector governance system. .............. 21 Indicator 4. The province has a functional normative and/or regulatory body. ........................................................................................ 23 Indicator 5. Existence of institutional development capacity. ...................... 26 Pillar III: Procurement operations and market practices 27 Indicator 6. The provincial procurement operations and practices are efficient. ................................................................................... 27 Indicator 7. Functionality of the public procurement market. ...................... 29 Indicator 8. Existence of contract administration and dispute resolution provisions. ............................................................................... 30 Pillar IV: Integrity and transparency of the public procurement system 32 Indicator 9. The province has effective control and audit systems. ............. 32 Indicator 10. Efficiency of appeals mechanism. ............................................. 34 Indicator 11. Degree of access to information............................................... 36 Indicator 12. The province has ethics and anticorruption measures in place. 36 iii Being a joint product of the Development Partners (ADB, The World Bank, DfID and The European Commission) and the Government of Sindh, the formatting guidelines of Asian Development Bank were adopted for this report iv Assessment Results Summary Pillar and Indicator Sub-indicator Score Sub- Averaged indicator for Pillar & Indicator Pillar I: The legislative and regulatory framework 1.5 Indicator 1. 2 Public procurement legislative and regulatory framework complies with applicable obligations with regard to national and international requirements 1(a). Scope of application and coverage of the 2.5 legislative and regulatory framework 1(b). Procurement methods 2 1(c). Advertising rules and time limits 3 1(d). Rules on participation 1.5 1(e). Tender documentation and technical 2 specifications 1(f). Tender evaluation and award criteria. 2 1(g). Submission, receipt, and opening of tenders 3 1(h). Complaints 1 Indicator 2. 1 Existence of implementing regulations and documentation 2(a). Implementing regulations that provide 2 defined processes and procedures not included in higher-level legislation 2(b). Model tender documents for goods, works, 1.5 and services 2(c). Procedures for prequalification 2 2(d). Procedures suitable for contracting for 0 services or other requirements in which technical capacity is a key criterion 2(e). User's guide or manual for contracting 0 entities 2(f). General conditions of contracts for public 1.5 sector contracts covering goods, works, and services consistent with national requirements 5 Pakistan: Sindh Province--Baseline Indicators System and, when applicable, international requirements Pillar II: Institutional framework and management capacity 1 Indicator 3. 1 The public procurement system is mainstreamed and well integrated into the public sector governance system 3(a). Procurement planning and associated 0 expenditures are part of the budget formulation process and contribute to multiyear planning 3(b). Budget law and financial procedures 1 support timely procurement, contract execution, and payment 3(c). No initiation of procurement actions without 1 existing budget appropriations 3(d). Systematic completion reports are prepared 1 for certification of budget execution and for reconciliation of delivery with budget programming Indicator 4. 2 The province has a functional normative and/or regulatory body 4(a). The status and basis for the normative 1 and/or regulatory body is covered in the legislative and regulatory framework 4(b). The body has a defined set of 3 responsibilities 4(c). The body's organization, funding, staffing, Not and level of independence and authority scored sufficient and consistent with the responsibilities 4(d). The responsibilities should also provide for 3 separation and clarity so as to avoid conflict of interest and direct involvement in the execution of procurement transactions Indicator 5. 0 Existence of institutional development capacity 5(a). The province has a system for collecting and 1 disseminating procurement information including tender invitations, requests for proposals, and contract award information 6 Baseline Procurement Performance Assessment Report 5(b). The province has systems and procedures 0 for collecting and monitoring provincial procurement statistics 5(c). A sustainable strategy and training capacity 0 exists 5(d). Quality control standards are disseminated 0 and used to evaluate staff performance and address capacity development issues Pillar III: Procurement operations and market practices 1.25 Indicator 6. 1.25 The provincial procurement operations and practices are efficient 6(a). The level of procurement competence 0.5 among government officials within the entity is consistent with their procurement responsibilities 6(b). The procurement training and information 0 programs for government officials and for private sector participants are consistent with demand 6(c). There are established norms for the 2 safekeeping of records and documents related to transactions and contract management 6(d). There are provisions for delegating 2.5 authority to others who have the capacity to exercise responsibilities Indicator 7. 1.5 Functionality of the public procurement market 7(a). There are effective mechanisms for 1 partnerships between the public and private sector 7(b). Private sector institutions are well organized 2 and able to facilitate access to the market 7(c). There are no major systemic constraints 2 (e.g., inadequate access to credit, contracting practices, etc.) inhibiting the private sector's capacity to access the procurement market 7(d). There are clear and transparent rules for 1 determining whether to engage international or national markets, based on a sound development and business logic 7 Pakistan: Sindh Province--Baseline Indicators System Indicator 8. 1 Existence of contract administration and dispute resolution provisions 8(a). Procedures are clearly defined for 1 undertaking contract administration responsibilities that include inspection and acceptance procedures, quality control procedures, and methods to review and issue contract amendments in a timely manner 8(b). Contracts include dispute resolution 1 procedures that provide for an efficient and fair process to resolve disputes arising during the performance of the contract 8(c). Procedures exist to enforce the outcome of 1 the dispute resolution process Pillar IV: Integrity and transparency of the public procurement system 1 Indicator 9. 0.5 The province has effective control and audit systems 9(a). A legal framework, organization, policy, and 1 procedures for internal and external control and audit of public procurement operations are in place to provide a functioning control framework 9(b). Enforcement and follow-up on findings and 1 recommendations of the control framework provide an environment that fosters compliance 9(c). The internal control system provides timely 1 information on compliance to enable management action 9(d). The internal control systems are sufficiently 0 defined to allow performance audits to be conducted 9(e). Auditors are sufficiently informed about 0 procurement requirements and control systems to conduct quality audits that contribute to compliance Indicator 10. Not Scored Efficiency of appeals mechanism 8 Baseline Procurement Performance Assessment Report Indicator 11. 1 Degree of access to information 11(a). Information is published and distributed 1 through available media with support from information technology when feasible Indicator 12. 2 The province has ethics and anticorruption measures in place 12(a). The legal and regulatory framework for 2 procurement, including tender and contract documents, includes provisions addressing corruption, fraud, conflict of interest, and unethical behavior and sets out (either directly or by reference to other laws) the actions that can be taken with regard to such behavior 12(b). The legal system defines responsibilities, 1 accountabilities, and penalties for individuals and firms found to have engaged in fraudulent or corrupt practices 12(c). Evidence of enforcement of rulings and 2 penalties exists 12(d). Special measures exist to prevent and 2 detect fraud and corruption in public procurement 12(e). Stakeholders (private sector, civil society, 2.5 and the ultimate beneficiaries of procurement and/or end-users) support the creation of a procurement market known for its integrity and ethical behavior 12(f). The province should have in place a secure 2 mechanism for reporting fraudulent, corrupt, or unethical behavior 12(g). Existence of codes of conduct or codes of 2 ethics for participants that are involved in aspects of the public financial management systems that also provide for disclosure for those in decision-making positions 9 Pakistan: Sindh Province--Baseline Indicators System A. Background 1. This document provides an assessment of the public procurement system in Sindh Province using the baseline indicators system developed by the Development Assistance Committee of the Organisation for Economic Co- operation and Development (OECD-DAC). It is part of a broader public financial management and accountability assessment being undertaken by the government of Sindh with multilateral support from the Asian Development Bank, the World Bank, and the European Commission. 2. To prepare this assessment, interviews and discussions were held with stakeholders from the public and private sectors as well as civil society. Given the commitments of the government on the procurement reform agenda, this assessment provides a baseline against which future progress can be periodically benchmarked. There are no specific recommended next steps as the procurement road strategy adopted by the government includes the actions required to align systems with international best practice. Continued commitment from the government to implement the road map will facilitate sustained and efficient reforms in public procurement, with consequent positive impacts on provincial financial management. B. Procurement Assessment Using the OECD-DAC Baseline Indicators System 3. Developing-country governments and donors have a mutual interest in increasing the effectiveness, efficiency, and transparency of procurement systems. Both have worked together under the auspices of the World Bank and OECD-DAC Procurement Round Table Initiative to develop a set of tools and standards to support and guide improvements in procurement systems. These tools include (i) benchmarking for assessing the structure of public procurement systems, and (ii) monitoring and evaluation of public procurement systems. A key analytical tool developed for benchmarking is the baseline indicators system which comprises four pillars elaborated through 12 indicators which are further detailed using sub-indicators. Pillar I: The legislative and regulatory framework Indicator 1. Public procurement legislative and regulatory framework complies with applicable obligations with regard to national and international requirements. Indicator 2. Existence of implementing regulations and documentation. Pillar II: Institutional framework and management capacity Indicator 3. The public procurement system is mainstreamed and well integrated into the public sector governance system. 10 Baseline Procurement Performance Assessment Report Indicator 4. The province has a functional normative and/or regulatory body. Indicator 5. Existence of institutional development capacity. Pillar III: Procurement operations and market practices Indicator 6. The provincial procurement operations and practices are efficient. Indicator 7. Functionality of the public procurement market. Indicator 8. Existence of contract administration and dispute resolution provisions. Pillar IV: Integrity and transparency of the public procurement system Indicator 9. The province has effective control and audit systems. Indicator 10. Efficiency of appeals mechanism. Indicator 11. Degree of access to information. Indicator 12. The province has ethics and anticorruption measures in place. 4. Using the baseline indicators system, the scoring for assessment ranges from 3 to 0. A score of 3 indicates full achievement of the stated standard. A score of 2 indicates the system needs some improvements in the area being assessed, and a score of 1 indicates substantive work is needed for the system to meet the standard. A rating of 0 represents a complete failure to meet the proposed standard. In this assessment some of the sub-indicators are not scored because they are not relevant in this assessment, or because action has just commenced and it is not yet appropriate to comment on achievements 5. The tool helps to identify when an element of the public procurement system meets or exceeds the baseline or where it needs modification to meet the baseline. It is anticipated that the exercise of benchmarking using the baseline indicators system will contribute to improving public procurement in Sindh in the following ways: (i) The government will learn how much progress has been made towards creating procurement procedures that are effective, efficient, and transparent. (ii) The government will be able to create links between the contents of the procurement road map, and assess the benefits of its implementation. (iii) The government will be able to identify, prioritize, and focus on areas requiring immediate attention. (iv) The government will be able to measure the impact of reforms by comparing implementation progress against the results of the assessment. 11 Pakistan: Sindh Province--Baseline Indicators System (v) The exercise will provide the supporters of reform in the government with clear arguments for change, focus political attention, and mobilize commitment. (vi) The exercise will support the required harmonization of public procurement rules and procedures with recognized standards. (vii) The output of the assessment will provide valuable information to the overall assessment of the public finance system. (viii) The exercise will provide information to support strategic policy decision making by the government. (ix) Information on the structure of the government procurement system will enable a determination of the level of reliance that donors can place on the system. C. Planning and Preparing for the Assessment 6. The assessment is based on interviews held with (i) the special secretary finance, government of Sindh; (ii) the government of Sindh departments of health, irrigation and power, works and services, and audit; (iii) the Karachi Water and Sewerage Board (iv) local government staff; (v) the National Accountability Bureau Sindh (NAB); (vi) Transparency International; (vii) consultants; and (viii) contractors. 12 Baseline Procurement Performance Assessment Report D. Assessment Results Pillar I: The legislative and regulatory framework 7. In order to regulate public procurement of goods, works, services, and consulting services, and to promote fair, transparent, and open competition, the government took an interim measure to adopt the federal ordinance setting up the Public Procurement Regulatory Authority and the federal Public Procurement Rules 2004 through notification No. SO(C-1) SGA&CD/4-80/2006 dated 1 November 2006. The Sindh Public Procurement Regulatory Authority (SPPRA) was established with effect from 30 November 2006 (Government of Sindh Law Department, notification No. S.Legis: 1 (30)/2006 30 November 2006). 8. All procuring entities were informed through a notification signed by the chief secretary that (i) the above rules would override all existing manuals, instructions, orders, etc.; (ii) web posting of budget, procurement notices, and awards shall be necessary on www.sindh.gov.pk; and (iii) the practice of departmental enlistment or pre-registration of bidders should be stopped. This letter was addressed to all provincial departments, attached departments, autonomous and semi-autonomous bodies, all offices and subordinate offices of the government of Sindh, district governments, town municipal administrations, union councils, and all establishments associated with the government of Sindh. 9. The government takes cognizance of the shortcomings of the federal procurement rules, and has prepared a procurement road map in consultation with the major stakeholders to amend the said rules for better alignment with international best practice. Major milestones of this road map are (i) revision of the regulatory framework; (ii) preparation of standard bidding documents (SBDs), implementing regulations, and sector-specific SBDs; (iii) a staffing plan for, and operationalization of, the SPPRA; (iv) a long-term training strategy; (v) a reform management strategy; and (vi) e-procurement readiness. Score awarded 1.5 10. The above score for Pillar I is based on assessment of indicators 1 and 2 described below. 13 Pakistan: Sindh Province--Baseline Indicators System Indicator 1. Public procurement legislative and regulatory framework complies with applicable obligations with regard to national and international requirements. 11. This indicator covers the legal and regulatory instruments from the highest level (law, act, rules, directives, etc.) down to detailed regulations, procedures, and bidding documents formally in use. The assessment of this indicator is based on the considerations described in sub-indicators 1(a) to 1(h) described below. Score awarded 2 Sub-indicator 1(a): Scope of application and coverage. (i) The ordinance adopted by the government covers the formation of the SPPRA and gives it a mandate to formulate rules. These rules supersede all previous rules or directives pertaining to public procurement. The rules contain numerous references to regulations, but these regulations have yet to be framed. The SBDs have also yet to be adopted or notified. The hierarchy of various instruments, i.e., rules, regulations, and departmental guidelines, is not explicitly defined in the ordinance or the rules. However, reference is made to the regulations as an explanatory tool of the rules in several clauses (e.g., clauses 8 and 11). The SPPRA is also mandated to recommend modifications to the rules but the approving authority for such modifications is undefined. (ii) The rules are applicable to ministries, attached departments, autonomous bodies, districts, governments, and their subdivisions. No agencies or components of public expenditure are excluded from the application of the ordinance and/or the rules. Exception to the applicability of the rules, to the extent of the conflict, is allowed for procurements under international treaties or international financial institutions. Another exemption is given to the procurements related to national security, where publication may conflict with security interests (rule 14). (iii) The ordinance and rules are published through notification No. SO(C-1) SGA&CD/4-80/2006 dated 1 November 2006 and are posted on the government website. The rules have been extensively circulated to all user departments. Printed copies are available at a nominal cost. (iv) The rules apply to goods, works, and services. The rules make no distinction between the selection procedures for procurements with intellectual value (consultancy services) and without intellectual value (goods and works). The "bidder is defined as supplier, contractor, or consultant, and the only competitive selection method given is the lowest evaluated bid; this is regarded as restrictive in respect of the selection of consultancy services. Score awarded 2.5 14 Baseline Procurement Performance Assessment Report Sub-indicator 1(b): Procurement methods. (i) Clause 20 defines open competitive bidding as the default procurement method. Clause 38 gives lowest evaluated bid as the only method of acceptance of bids for goods, works, and services. This clause implies that the only competitive procedure allowed for selection of consultants is least-cost selection. (ii) Clause 42 defines alternative procurement procedures which are petty purchases, requests for quotations, and direct contracting. The situations for using the less competitive procedures are clearly defined. (iii) Under clause 8 it is mandatory for implementing agencies to prepare a procurement plan, and splitting and/or regrouping of the procurements is prohibited under clause 9. All executing agencies are required to develop their own system of authorization and delegation of powers for procurements under rule 11. (iv) International bidding is covered under clause 13 which provides an extended response time, i.e., 30 days, compared to 15 days for national competitive bidding. In addition, clause 24 mandates that procurement shall be without regard to nationality, unless national bidding only is specified, or if there is a restriction on certain nationalities under government policy. Clause 32 stipulates that no discriminatory condition can be put in the bids. The specific clauses which could define an unbiased environment for international bidding are implied to be covered under the SBDs, but as yet no SBDs have been developed by the SPPRA. Domestic preference is stipulated to be included as notified by the government in rule 24(2). (v) The practice for Public Sector Development Program projects (which are large) is to use the Pakistan Engineering Council (PEC) documents, which although quite close to best practice are focused on national bidding and do not include clauses such as tax equalization. Moreover, in these documents dispute resolution is limited to local laws, but currency exchange provisions are provided. Score awarded 2 Sub-indicator 1(c): Advertising rules and time limits. (i) In rule 12, an elaborate method for advertising procurement is given. Procurements costing between PRs100,000 and PRs2 million must be posted on the SPPRA website, and may also be published in the print media. Procurements above PRs2 million must be posted on the SPPRA website and published in at least three national daily newspapers. Executing agencies that have their own websites must use them for posting all procurement advertisements. (ii) Rule 13 specifies 15 days as a minimum response time for national bidding and 30 days for international bidding. Executing agencies are encouraged to decide adequate response times based upon the complexity of the procurement. Rule 13(3) states that if newspaper and website postings are not concurrent, the response time shall be calculated from the date of publication in newspapers. 15 Pakistan: Sindh Province--Baseline Indicators System (iii) Rule 12(4) stipulates that the procuring agency should ensure that the information posted on the website is complete and shall remain available on that website until the closing date for the submission of bids. Score awarded 3 Sub-indicator 1(d): Rules on participation. (i) Rule 15 gives the process of prequalification, which states that prequalification shall be solely based on the ability of the interested parties to perform. Rule 16 makes it mandatory for the procuring authority to announce the requirements of the prequalification application and the evaluation criteria. While it is specified that criteria should be unambiguous, a pass­fail basis is not mentioned. (ii) The disqualification process is covered under clause 18, which stipulates that any bidder can be disqualified if the information provided concerning the qualification is false, materially inaccurate, or incomplete. Clause 19 places the onus of devising a blacklisting process on the procuring agencies. The process is to be made public and communicated to the SPPRA. (iii) Rule 24 stipulates that the procuring agencies can limit the bidding to national firms, or prohibit participation of some nationalities in line with federal rules. Domestic preference in accordance with government polices is allowed. Although the rules do not stipulate any registration requirements, PEC registration rules are adopted for engineering sector firms. These rules require, under S.R.O. 568 (I)/87 byelaw 11, that every foreign contractor must enlist themselves with PEC before initiating any assignment in Pakistan. In addition, a firm interested in bidding for any project must obtain a project-specific license by submitting the enlistment certificate, a joint venture or association agreement with local firms, a scope of work, and the license fee. Similarly, for engineering consulting services, PEC byelaw 6 stipulates that foreign firms must be registered as consulting engineers for specified projects which need expertise not available with Pakistani consulting engineers. Foreign firms shall enter into a joint venture with a Pakistani consulting engineer and the services to be rendered by the foreign firm shall be limited to the expertise and knowledge not available with any Pakistani consulting engineer. These PEC byelaws are not referred to in the SPPRA rules but exemption is also not sought. (iv) There was a practice of pre-registration or enlistment in various departments which was supposed to be open the year round and firms were expected to register annually. Only departmentally enlisted firms were eligible to participate in bids. The process was not transparent as the qualification criteria and time frame for the review process were not disclosed. The government has abolished this process and all procuring entities are now required to carry out project- specific prequalification or post-qualification. (v) There are no explicit rules regarding participation of publicly owned entities, but clause 32 disallows any discrimination between bidders and eliminates any preferential treatment to a publicly owned entity. The issue of tax exemption should however be catered for in the SBDs to provide equalization between public and private sector organizations. 16 Baseline Procurement Performance Assessment Report (vi) The Sindh Purchase Manual 1991 (chapter III, paragraph 63) explicitly mentions that locally produced or manufactured goods are to be preferred, and the procuring entity has to seek exemption from the secretary to place orders for imported goods. Score awarded 1.5 Sub-indicator 1(e): Tender documentation and technical specifications. (i) Rule 15 provides the contents of the prequalification documents, and rule 23 provides the contents of the bidding documents. The contents of bidding documents are almost complete except that rule 23(2)(e) states general or special conditions of contracts, rather than mentioning both conditions. (ii) Guidelines for providing unbiased specifications are given in rule 10. (iii) The documents for prequalification and bidding are not yet standardized. Score awarded 2 Sub-indicator 1(f): Tender evaluation and award criteria. (i) Rule 29 stipulates that unambiguous award criteria should be specified in the documents, and rule 30 restricts the use of any criterion extraneous to the pre- specified criteria for evaluation. Negotiations are not allowed (rule 40) unless otherwise provided for in the regulations. However, there are no regulations as yet. (ii) It has been observed that in certain consultancy contracts of the Karachi Water and Sewerage Board and the Department of Irrigation and Power that the least- cost method has not been adopted. The top-ranking consulting firms in technical evaluations are invited to negotiate the contract, or quality and cost- based selection is adopted. This practice is quoted in various guidelines from the Planning Commission, PEC, and donor agencies. (iii) The award criterion to be used is the lowest evaluated price (rule 38), if not in conflict with any other, law, rules, regulations, or policy of the government. The procuring agencies have not yet formulated appropriate evaluation criteria listing all the relevant information against which a bid is to be evaluated and which should be an integral part of the SBDs. (iv) As discussed earlier, the evaluation/award criterion for consultancies is least evaluated cost which is not aligned to international best practices. (v) Any person directly involved in the prequalification process is authorized to receive, on request, a list of all prequalified participants. The procuring agency is required to communicate to unqualified participants the reasons for their disqualification (rule 16). It is mandatory to announce the results of a bid evaluation at least 10 days prior to the award (rule 35). Confidentiality of the evaluation process is ensured under rule 41. 17 Pakistan: Sindh Province--Baseline Indicators System Score awarded 2 Sub-indicator 1(g): Submission, receipt, and opening of tenders. (i) Rule 28 gives the details of bid opening and stipulates that the bids shall be publicly opened on the last day of bid submittal. In some cases the bids are opened immediately after the closing time, while in other cases there is 30­60 minutes between the closing and opening time. (ii) Rule 28(2) specifies that the minutes of the bid opening shall be recorded. (iii) Rule 47 specifies that as soon as the award has been made, the procuring agency shall make all documents related to the evaluation public. It is further stated that information of a proprietary nature can be withheld. There is no ambiguity in the modality of submitting tenders and their receipt. Score awarded 3 Sub-indicator 1(h): Complaint review procedures. (i) Complaint review is addressed in rule 48. Any participant is allowed to lodge a written complaint with the procuring agency within 15 days of the announcement of the evaluation. There is no provision to approach any agency other than the procuring agency, thus eliminating third party involvement in complaint review and redress. (ii) It is mentioned that the complaint can be lodged against any grievance caused by any act of the procuring agency after the bid submittal. (iii) Rule 48(3) gives a 15-day time frame for the review committee to settle complaints. Score awarded 1 Indicator 2. Existence of implementing regulations and documentation. 12. This indicator verifies the existence, availability, and quality of implementing regulations, operational procedures, handbooks, model tender documentation, and standard conditions of contract. Ideally, the higher-level legislation provides the framework of principles and policies that govern public procurement. Lower- level regulations and more detailed instruments supplement the law, make it operational, and indicate how to apply the law to specific circumstances. Score awarded 1 18 Baseline Procurement Performance Assessment Report 13. The above score for this indicator is based on assessment of the sub-indicators 2(a) to 2(f) described below. Sub-indicator 2(a): Implementing regulations that provide defined processes and procedures not included in higher-level legislation. (i) The rules set out the procedures for procurements; however, essential operational details are to be provided in the implementing regulations which are not yet developed. Model documents or SBDs are also not prepared yet. The rules have only recently been adopted, but the aspects requiring modification are clearly identified in the road map and revision is in process. (ii) Responsibility for preparing regulations is defined to be with the SPPRA, while implementing agencies are allowed to prepare their own implementation manuals. Score awarded 2 Sub-indicator 2(b): Model tender documents for goods, works, and services. (i) Rule 4 states that procuring agencies shall use SBDs as and when notified through the regulations by the SPPRA. Rule 23(1) states that procuring agencies shall formulate precise and unambiguous bidding documents that shall be made available to the bidders immediately after the publication of the invitation to bid. (ii) Contents of the bidding documents are defined in Rule 23(2) and include instructions to bidders, forms, evaluation criteria, general and special conditions, and specifications. Rule 23(3) stipulates that all amendments to SBDs should be publicly disseminated. (iii) As yet there are no model documents; however, under a directive from the federal government, the SBDs for works and goods prepared by PEC must be used for all federally funded projects. These documents are quite aligned to good practice and contain instructions to bidders, forms and schedules for bidding and guarantees, as well as general and special conditions of contract. However, some amendments are needed to make these documents applicable to international competitive bidding. With the exception of the Public Sector Development Program and a few very large contracts, most contracts are administered using brief departmental documents, which are heavily client focused. There are no model requests for proposals for consulting services, and implementing agencies use various donor agency standards at their discretion. Score awarded 1.5 Sub-indicator 2(c): Procedures for prequalification. (i) Rule 15 states that a procuring agency, prior to the floating of tenders, invitation to proposals, or offers in procurement proceedings, may engage in prequalification of bidders in the procurement of services, civil works, turnkey projects, and expensive and technically complex equipment to ensure that only firms having the technical, financial, and managerial capability are invited to 19 Pakistan: Sindh Province--Baseline Indicators System submit bids. Such prequalification shall solely be based on the ability of the interested parties to satisfactorily perform the services required. (ii) Rule 15 provides for the occasions where pre-qualification may be done by procuring agencies. Services, civil works, turnkey projects and complex goods re cited as procurements where pre-qualification could be done. (iii) Prequalification criteria suggested in rule 15(2) include relevant experience, physical resources, finances, managerial capacity, and any other relevant information. The use of pass­fail criteria is not specified. (iv) Separate guidelines for prequalification, procurement, and registration of biomedical firms have been prepared by the Health Department. Score awarded 2 Sub-indicator 2(d): Procedures suitable for contracting for services or other requirements in which technical capacity is a key criterion. (i) The sole award criteria is lowest evaluated cost and no cognizance is taken of the options of cost and quality, in its various combinations. There is no allowance for considering quality in making an award decision. There are no procedures combining assessment and scoring of a technical proposal with a cost proposal. (ii) It has been observed that, in certain consultancy contracts of the Karachi Water and Sewerage Board and the Irrigation and Power Department, the least-cost method has not been adopted. In these cases the top-ranking consulting firm in a technical evaluation was invited to negotiate the contract. Score awarded 0 Sub-indicator 2(e): User's guide or manual for contracting entities. (i) There is no mention of any procurement manual in the present law or rules. While the rules are being revised, the requirement for regulations and manuals is also being addressed. (ii) The Sindh Purchase Manual 1991 was used as a reference before the adoption of the new rules; however, its stipulations contradict the stipulations of these rules, rendering it irrelevant. Score awarded 0 20 Baseline Procurement Performance Assessment Report Sub-indicator 2(f): General conditions of contracts for public sector contracts covering goods, works, and services consistent with national requirements and, when applicable, international requirements. (i) The generally used departmental contract document is a brief tender document containing general conditions of contracts and a tender agreement form. Contractors for smaller contracts are accustomed to using this document, which is heavily client oriented. The PEC contract documents for goods, works, and consultancy services have more balanced general conditions of contracts, but the limited use of PEC documents diminishes the level of good practice. Score awarded 1.5 Pillar II: Institutional framework and management capacity 14. Pillar II looks at how the procurement system, as defined by the provincial legal and regulatory framework, is operating in practice through the institutions and management systems that are part of the overall public sector governance set up. The focus is on the extent and quality of the implementation of the regulatory system. Score awarded 1 15. The above score for Pillar II is based on assessment of indicators 3­5 described below. Indicator 3. The public procurement system is mainstreamed and well integrated into the public sector governance system. 16. This indicator looks at the procurement system to determine (i) its suitability to discharge the obligations prescribed in law without gaps or overlaps, (ii) whether the necessary links with other sectors of government affecting procurement exist, (iii) whether procurement operations are constrained by other external institutional factors, and (iv) whether the managerial and technical capacity of the system are adequate for procurement without unnecessary cost or delay. This indicator deals with the degree of integration of the procurement system with other parts of government and particularly with the financial management system, given the direct interaction between the two, including budget preparation and planning, and treasury operations for payments. Score awarded 1 17. The score for this indicator is based on assessment of the sub-indicators 3(a) to 3(d) described below. 21 Pakistan: Sindh Province--Baseline Indicators System Sub-indicator 3(a): Procurement planning and associated expenditures are part of the budget formulation process and contribute to multiyear planning. (i) As mentioned in PI 12 of the Public Financial Management Assessment Report, multiyear fiscal forecasting was recently introduced using the Medium-Term Fiscal Framework, which was a trigger of Sindh Education Developments Policy Credit. This framework covers the fiscal years 2006/07 to 2009/10. Previously the provincial budget only accommodated multiyear expenditures for projects which had completion periods extending beyond the annual fiscal window. (ii) There is no link between budgetary planning and the operational aspects of procurement. The packaging of various procurements, timelines, and dates of advertisement in order to meet the fiscal year targets is not done. (iii) Departments prepare a PC-I pro forma for each scheme and forward it to the Finance Department for approval and budget allocation. The Finance Department analyzes and scrutinizes the scheme in view of the available finances. The next step is approval from the planning departments (departmental, provincial, and federal). The Finance Department releases allocated funds in tranches for various schemes, which are generally not linked to any schedule of payments under a particular contract. Some departments reported receiving budgets for goods (including vehicles) in installments of 25%, which could not be spent as no supplier would deliver against part payment. 18. Budgets have no linkages to the implementation schedules of projects, and it is a common occurrence that releases are made and contracts are awarded in the last quarter of a fiscal year. Score awarded 0 Sub-indicator 3(b): Budget law and financial procedures support timely procurement, contract execution, and payment. (i) There are no standard timelines for committing budget funds for a particular contract, and actual timelines vary from 10 days to 10 months, depending on the priority of the scheme. Although it is allowed under the procedures to appropriate funds equal to the contract cost, it is seldom done. Allocations for every scheme in the last fiscal year were made on a quarterly basis creating limitations in implementation. (ii) There are no published business standards covering timely payment of invoices by government agencies. The contract documents under preparation are supposed to cover the procedure and timelines for the processing of invoices. The brief departmental standard documents do not specify a payment timeline. The PEC SBD does specify the payment time, but this is generally not complied with, and a contractor receiving interest on a delayed payment is unknown. (iii) In the absence of a link between budget release and contractual commitment, and the lack of standard procedures for making payments, there is no standard timeline for the authorization of payment following approval of invoices. 22 Baseline Procurement Performance Assessment Report Score awarded 1 Sub-indicator 3(c): No initiation of procurement actions without existing budget appropriations. (i) The rules require certification of the availability of funds before tenders are solicited. Tenders can not be called in excess of the actual budget provision made in development and recurrent (nondevelopment) budgets for projects creating unbudgeted liabilities in a subsequent financial year. (ii) However, in certain cases, where funds are not allocated in the budget, but considering the importance and/or urgency for execution of a specific scheme, a waiver can be obtained from the administrative secretary, finance minister, or chief minister / for tendering without a sanctioned budget. At the end of the year approval of a revised budget which includes all such projects is obtained. Score awarded 1 Sub-indicator 3(d). Systematic completion reports are prepared for certification of budget execution and for reconciliation of delivery with budget programming. (i) Although all procuring agencies are required to maintain a record of their procurement proceedings for a minimum of 5 years, as defined in rule 46(1), there is no effective reporting mechanism. Completion reports (PC-IVs) which are supposed to be prepared for every contract, are seldom developed, and even when a PC-IV is prepared the focus is not on effective budget utilization. Furthermore, contract costs (and scope of work) are generally adjusted to match allocations, rather than being fixed or altered on their own merit, thus leaving no room for reallocation of excess budget. Score awarded 1 Indicator 4. The province has a functional normative and/or regulatory body. 19. Although this indicator refers to a normative and/or regulatory body, what matters most is not the existence of a body but the existence of the functions within the public sector and the proper discharge and coordination of them (i.e., one agency may be responsible for policy while another can be doing the staff training and another might be taking care of the statistics). Thus, when the assessment criteria below refer to the regulatory body, this can be read to refer to the regulatory function if applicable to the particular assessment. Assessment of this indicator focuses on the existence of the functions, the independence of the regulatory function, the effectiveness of performance, and the degree of coordination between responsible organizations. Score awarded 2 23 Pakistan: Sindh Province--Baseline Indicators System 20. The score for this indicator is based on assessment of the sub-indicators 4(a) to 4(d) described below. Sub-indicator 4(a): The status and basis for the normative and/or regulatory body is covered in the legislative and regulatory framework. (i) Public Procurement Ordinance 2002 defines the regulatory functions of the SPPRA. However, the SPPRA is not yet fully functional and has not taken up its effective role. Provided the SPPRA is adequately resourced (including staff and funds), it has the legal basis to function effectively. (ii) The ordinance and rules under revision include further elaboration of the SPPRA's functions. These should be specified as custodian of the rules, provider of training, and a research unit rather than an organization responsible for transactional reviews. Score awarded 1 Sub-indicator 4(b): The body has a defined set of responsibilities. 21. The SPPRA has responsibility to act as custodian of the regulations, propose improvements and amendments to the regulatory framework, develop implementing regulations and standard documents, monitor functions and advise procuring entities on operational interpretations of the rules, establish and manage a procurement database, provide regular reporting, and provide training. Score awarded 3 Sub-indicator 4(c): The body's organization, funding, staffing, and level of independence and authority (formal power) to exercise its duties should be sufficient and consistent with the responsibilities. 22. The governing board of the SPPRA is headed by the Additional Chief Secretary Development; there are also five secretaries and representatives from the private sector on the board. The managing director of the SPPRA has been appointed and is assisted by one director; other staff is yet to be hired. Funding of the SPPRA is covered by the ordinance and an interim fund allocation has been made in the current year. The level of authority of the board seems appropriate, but a fair assessment can only be made after 1 year. 24 Baseline Procurement Performance Assessment Report Score awarded Not scored Sub-indicator 4(d): The responsibilities should also provide for separation and clarity so as to avoid conflict of interest and direct involvement in the execution of procurement transactions. 23. The SPPRA is not responsible for direct procurement operations and is free from other possible conflicts. Score awarded 3 25 Pakistan: Sindh Province--Baseline Indicators System Indicator 5. Existence of institutional development capacity. 24. The objective of this indicator is to assess the extent to which the country or agency has systems to support and monitor the performance of the entire system, and to formulate and implement improvement plans. This requires, among other things, the availability of information systems, a capacity for analysis, feedback mechanisms, and a planning capacity for the implementation of improvements. It is very important that responsibilities are clearly assigned and are being performed. Score awarded 0 25. The score for this indicator is based on assessment of the sub-indicators 5(a) to 5(d) described below. Sub-indicator 5(a): The province has a system for collecting and disseminating procurement information, including tender invitations, requests for proposals, and contract award information. 26. The government official website, at a minimum, provides up-to-date information and is easily accessible to all interested parties at no cost. Annual procurement plans are not made, and the information pertaining to contract awards and the status of contracts under implementation is not published. Although the implementing agencies are directed to post plans on the website, compliance is erratic. There is a disconnection between the SPPRA and the government website. Score awarded 1 Sub-indicator 5(b): The province has systems and procedures for collecting and monitoring provincial procurement statistics. 27. There is no integrated system to collect procurement data at the provincial level. Some departments have databases, but in the absence of a repository, retrieval is difficult. There is no system of data analysis in place so far, but when the SPPRA is fully operational it should be able to fill this gap. Score awarded 0 Sub-indicator 5(c): A sustainable strategy and training capacity exists to provide training, advice, and assistance to develop the capacity of government and private sector participants to understand the rules and regulations and how they should be implemented. (i) As part of implementing the road map the SPPRA is preparing a training strategy and, following approval by the board, training modules will be developed for specific target audiences. The capacity of the SPPRA itself to perform an advisory role is yet to be established. 26 Baseline Procurement Performance Assessment Report (ii) In the last 2 years a few workshops for capacity building have been arranged with the participation of Transparency International, the World Bank, and the NAB, but there are no defined schedules for such training exercises. Score awarded 0 Sub-indicator 5(d): Quality control standards are disseminated and used to evaluate staff performance and address capacity development issues. 28. There are no quality assurance standards or monitoring system for procurement processes. There are no benchmarks for the evaluation of staff performance related to procurement functions. Operational audits to monitor compliance with quality assurance standards are not carried out regularly. The audit staff is generally unaware of the public procurement ordinance. Score awarded 0 Pillar III: Procurement operations and market practices 29. Pillar III looks at the operational effectiveness and efficiency of the procurement system at the level of the implementing entity responsible for issuing individual procurement actions. It looks at the market as one means of judging the quality and effectiveness of the system when putting procurement procedures into practice. This pillar is distinguished from pillars I and II in that it is not looking at the legal/regulatory or institutional systems in a country but more at how they operate. Score awarded 1.25 30. The above score for Pillar III is based on assessment of indicators 6­8 described below. Indicator 6. The provincial procurement operations and practices are efficient. 31. This indicator looks at the efficiency of the operations and operational practices as implemented by the procuring agencies. Efficiency is considered to mean that the operational practices result in the timely award of contracts at competitive market prices as determined by effective and fair implementation of procurement procedures. Score awarded 1.25 32. The score for this indicator is based on assessment of the sub-indicators 6(a) to 6(d) described below. 27 Pakistan: Sindh Province--Baseline Indicators System Sub-indicator 6(a): The level of procurement competence among government officials within the entity is consistent with their procurement responsibilities. 33. Procurement is not considered a professional cadre, and there are no defined skills or knowledge profiles for specialized procurement positions. Recruitment processes appear transparently documented, but there are gray areas in execution. Staff undertake procurement activities on an ad hoc basis, with no systematic advancement or placement. There are exceptions in some departments (e.g., Irrigation and Power, Works and Services) and for larger projects where procurement committees tend to be well constituted with experienced staff, although formal guidelines for such committees are absent. Currently the staff responsible for recruitment have limited awareness of the procurement ordinance and rules. Score awarded 0.5 Sub-indicator 6(b): The procurement training and information programs for government officials and for private sector participants are consistent with demand. Systematic procurement training and information programs are not available at provincial level from government institutions. Score awarded 0 Sub-indicator 6(c): There are established norms for the safekeeping of records and documents related to transactions and contract management. (i) All procuring agencies are required to maintain a record of their procurement proceedings together with all associated documentation for a minimum period of 5 years, as defined in rule 46(1). However, the required details of such records are not described. (ii) Government departments are aware that record keeping is necessary for audit purposes although retrieval of records can become cumbersome. Generally, procurement records are kept in a separate file for each contract awarded and this record is particularly scrutinized for compliance by the auditors on an annual basis. There are no established security protocols to protect records, either physical or electronic. Score awarded 2 Sub-indicator 6(d): There are provisions for delegating authority to others who have the capacity to exercise responsibilities. 34. Delegation of decision-making authority is decentralized to various levels through departmental rules. The procurement rules do not address this in detail. Selection of consultancy firms is an exception, where authority lies with a provincial selection committee. Descriptions of accountability for decisions are subject to interpretation. 28 Baseline Procurement Performance Assessment Report Score awarded 2.5 Indicator 7. Functionality of the public procurement market. 35. The objective of this indicator is primarily to assess the market response to public procurement solicitations. This response is influenced by many factors such as the general economic climate, the private sector development environment and policies, the existence of strong financial institutions, the attractiveness of the public system as a good reliable client, the kind of goods or services being demanded, and the trust between private and public sectors. Score awarded 1.5 36. The score for this indicator is based on assessment of sub-indicators 7(a) to 7(d) described below. Sub-indicator 7(a): There are effective mechanisms for partnerships between the public and private sector. (i) There is no effective dialogue between the public and private sectors on improving procurement systems. Both sectors make observations regarding each other's capacity and professionalism, but no mechanism to discuss these issues exists. The few professional organizations which could support healthy market development make no proactive efforts to do so due to mutual mistrust. (ii) Public­private partnerships are not covered under any regulatory arrangement. (iii) There are public sector initiatives to encourage the development of small and medium-sized enterprises, but there is no data available to gauge the effectiveness of these interventions. Score awarded 1 Sub-indicator 7(b): Private sector institutions are well organized and able to facilitate access to the market. 37. The private sector is fairly large and professional associations exist, but they perform no effective role in creating a competitive market. Inefficient systems and a general mistrust of the public sector lead some market players to target the private sector for procurements, despite the fact that the public sector has the largest procurement outlays. Score awarded 2 29 Pakistan: Sindh Province--Baseline Indicators System Sub-indicator 7(c): There are no major systemic constraints (e.g., inadequate access to credit, contracting practices, etc.) inhibiting the private sector's capacity to access the procurement market. 38. The private sector perception of the public sector is one of compromised transparency and unequal opportunity. This perception is based on, among other factors, the vague qualification and evaluation criteria, the former registration requirements, biased contract conditions, unfair payment conditions, and delayed payments. The cost of prequalification, bidding documents, and requests for proposals are regarded as unreasonably high. The financial climate is such that banks generally provide guarantees against substantial deposits. Score awarded 2 Sub-indicator 7(d): There are clear and transparent rules for determining whether to engage international or national markets, based on a sound development and business logic. 39. The province has no clear rules for determining when to engage international or national markets; such decisions are usually made based on the size and complexity of the schemes. When foreign participation is proposed the provincial rules are not consistent with good practice, unless PEC documents are used. If foreign firms apply for a nationally offered contract, departments may allow such participation at their discretion; however, such events are rare given the size and conditions of such contracts. Score awarded 1 Indicator 8. Existence of contract administration and dispute resolution provisions. 40. The objective of this indicator is to assess the quality of contract administration practices which begin after the contract award and continue until acceptance and final payments. This is an area that many procurement systems fail to consider. It is also a period where many issues arise that can affect the performance of the contract and impact on service delivery. Score awarded 1 41. The score for this indicator is based on assessment of sub-indicators 8(a) to 8(c) described below. 30 Baseline Procurement Performance Assessment Report Sub-indicator 8(a): Procedures are clearly defined for undertaking contract administration responsibilities that include inspection and acceptance procedures, quality control procedures, and methods to review and issue contract amendments in a timely manner. (i) The procedures for acceptance of final products and for issuance of contract amendments are not a part of the legal or regulatory framework. However, PEC documents clearly contain such clauses, and the documents under preparation by the SPPRA will also cover these procedures. (ii) Quality checks are the responsibility of officers of the procuring or implementing agency, but unless assisted by a consulting firm, effectiveness of these quality checks is questionable. Supervision of large civil works is carried out by independent engineering firms, and for small contracts qualified government supervisors and inspectors check the quantities and to some extent the quality. However, the technical and technological capacity for in-house supervision is severely limited. Final payments are generally delayed, and contractors are never paid interest on delayed payments. Score awarded 1 Sub-indicator 8(b): Contracts include dispute resolution procedures that provide for an efficient and fair process to resolve disputes arising during the performance of the contract. (i) The rules stipulate that disputes between the parties to the contract shall be settled by arbitration, as defined in rule 49(1), and that the procuring agencies shall provide for a method of arbitration in the procurement contract that is consistent with the laws of Pakistan, as defined in rule 49(2). (ii) Pakistan has an Arbitration Act (1940), which is referred to in all major national and international contracts in which PEC SBDs are used. The departmental contract documents do not refer to arbitration. The act in general refers to accepted practices for neutrality of arbitrators, due process, expediency, and enforceability. (iii) International arbitration is not generally given as an option in the bidding documents. Alternative dispute resolution is also not a standard inclusion in the contracts. These points in particular contribute to the low score for this sub- indicator. Score awarded 1 Sub-indicator 8(c): Procedures exist to enforce the outcome of the dispute resolution process. 42. Arbitration awards are enforceable through a court order. Pakistan is member of the New York Convention on enforcement of international arbitration awards. However, the procedure to enforce the outcome of a dispute resolution is lengthy and cumbersome. The judicial system may even take decades before a case is decided, and even then the enforcement is not assured. There is no monitoring or organized database in respect of this issue, and no information is 31 Pakistan: Sindh Province--Baseline Indicators System available on the number of disputes or the time taken in their resolution and enforcement. The private sector does not view the dispute resolution process as a transparent or robust mechanism. Score awarded 1 Pillar IV: Integrity and transparency of the public procurement system 43. Pillar IV covers four indicators that are considered necessary to provide for a system that operates with integrity, has appropriate controls that support the implementation of the system in accordance with the legal and regulatory framework, and has appropriate measures in place to address the potential for corruption in the system. It also covers important aspects of the procurement system that include stakeholders as part of the control system. This pillar takes aspects of the procurement system and governance environment and seeks to ensure that they are defined and structured to contribute to integrity and transparency. Score awarded 1 44. The above score for Pillar IV is based on assessment of indicators 9­12 described below. Indicator 9. The province has effective control and audit systems. 45. The objective of this indicator is to determine the quality, reliability, and timeliness of the internal and external controls, preferably based on risk assessment and mitigation. Equally, the effectiveness of controls needs to be reviewed in terms of expediency and thoroughness of the implementation of auditors' recommendations. In addition to his or her own findings, the assessor should rely on the most current country financial accountability assessment that may be available. Score awarded 0.5 46. The score for this indicator is based assessment of sub-indicators 9(a) to 9(e) described below. Sub-indicator 9(a): A legal framework, organization, policy, and procedures for internal and external control and audit of public procurement operations are in place to provide a functioning control framework. 47. The Sindh auditor general is responsible for audit. Adequate independent control and audit mechanisms and institutions are available through pre-audit, external audit, and divisional accountants, but their focus is on financial matters. Provincial staff are not aware of the concept of a procurement audit, but creating 32 Baseline Procurement Performance Assessment Report such awareness is a major activity of the SPPRA under the procurement road map. Implementation of internal control mechanisms exists in individual agencies with clearly defined procedures. In addition to these controls, there is a chief minister's inspection team and an army monitoring team. Balance between timely and efficient decision making is difficult and awards can take as long as 1 year. Score awarded 1 Sub-indicator 9(b): Enforcement and follow-up on findings and recommendations of the control framework provide an environment that fosters compliance. 48. Follow up is carried out annually through post­fiscal year audit report compliance, but external audits are not usually carried out promptly at the end of each fiscal year. Internal audit mechanisms are not robust as the pre-audit staff is not independent of the transactional staff, except for autonomous bodies. Audit is done on the basis of sampling, and generally focuses on compliance with rules and regulations. The auditors are not aware of the new procurement rules, nor have they been specifically trained for conducting procurement audits. The external audit initially prepares an audit observation and seeks explanation from the executing units in their wrap-up meetings. All unresolved audit observations are discussed in the departmental accounts committee, and from there the observations are labeled as audit para and are taken to the public accounts committee. There are no service standards for this entire process. The recommendations of the public accounts committee range from recovery of losses sustained by government from the defaulting staff to criminal proceedings for punitive actions. There are major backlogs in the process and sometimes the public accounts committee may discuss actions dating back decades. 49. All the procuring entities have pre-audit staff who are supposed to pre-audit every payment; procurement awareness of this staff is low. 50. The details of this sub-indicator are explained in PI-26 of the main public financial management assessment report. Score awarded 1 33 Pakistan: Sindh Province--Baseline Indicators System Sub-indicator 9(c): The internal control system provides timely information on compliance to enable management action. 51. The term internal controls can be expanded to cover all operational and procedural rules and regulations, including audit. There are written standards for a few controls, covering some aspects, but timelines and reporting requirements are almost always missing. There are no written standards for internal audits or pre-audits. Reports are not regular and there is no stipulated time frame for reporting. Score awarded 1 Sub-indicator 9(d): The internal control systems are sufficiently defined to allow performance audits to be conducted. 52. There is no concept of a performance audit. The term audit is always viewed with a strong financial inclination and the performance aspect is rarely reviewed or documented. There are no service delivery standards and/or performance benchmarks. Consequently there is no concept of evaluating overall performance of any entity in terms of the efficiency, quality, and robustness of its stipulated functions. However, it would be unrealistic to report an absence of internal controls, so it can be stated that such controls are undefined and unmonitored. Score awarded 0 Sub-indicator 9(e): Auditors are sufficiently informed about procurement requirements and control systems to conduct quality audits that contribute to compliance. 53. The staff of the Sindh auditor general is not aware of the procurement rules in general. There is no regular training or capacity building in place for staff. Score awarded 0 Indicator 10. Efficiency of appeals mechanism. 54. The appeals mechanism was covered under Pillar I with regard to its creation and coverage by the legal regulatory framework. It is further assessed under this indicator for a range of specific issues regarding efficiency in contributing to the compliance environment in the province and the integrity of the public procurement system. Score awarded Not scored 55. There was no documented system in place prior to the recent legislation, which is yet to be implemented. Hence this indicator is not scored. However, the status of existing and promulgated systems is summarized for the sub-indicators 10(a) to 10(e) below. 34 Baseline Procurement Performance Assessment Report Sub-indicator 10(a): Decisions are made upon the basis of available information, and the final decision can be reviewed and ruled upon by a body (or authority) with enforcement capacity under the law. (i) There was no formal documented appeals system prior to development of the procurement rules. Although enacted (indicator 1(h) gives the details), these rules are yet to be implemented. Currently the director general (monitoring) is responsible for the review of complaints. Decisions are rendered on the basis of available evidence submitted by the parties and the director general (monitoring) has the authority to issue a final decision binding on the parties unless the decision is referred to an appeals body. (ii) The provincial ombudsman acts as an appeals body and has the authority to review decisions of the specified complaints body and issue final enforceable decisions. Timelines are specified for the submission and review of complaints and issuing of decisions that do not unduly delay the procurement process. Score awarded Not Scored Sub-indicator 10(b): The complaint review system has the capacity to handle complaints efficiently and a means to enforce the remedy imposed. 56. The SPPR rules pertaining to complaints provide a time frame but this system is not yet functional. The parallel systems, as described above, are currently the only resort for aggrieved parties. Score awarded Not Scored Sub-indicator 10(c): The system operates in a fair manner, with outcomes of decisions balanced and justified on the basis of available information. 57. Records were not readily available or retrievable to assess the quality of the appeals process. As informed by government officials, the decisions are based on information provided and are made on merit. However, the private sector has a differing opinion; they view the process as biased towards the public sector, where the aggrieved party has to go for a compromise in order to retain access to the market. 58. The mechanisms of the current appeal systems (both operational and regulatory aspects) are not independent as there is no third party involvement, or such involvement only occurs at a later or higher stage, which the aggrieved party is reluctant to approach. Score awarded Not Scored Sub-indicator 10(d): Decisions are published and made available to all interested parties and to the public. 35 Pakistan: Sindh Province--Baseline Indicators System 59. No decisions are publicly posted on the government website, but the information is provided to all interested parties. The procurement rules establish the publication of all such processes. Score awarded Not Scored Sub-indicator 10(e): The system ensures that the complaint review body has full authority and independence for resolution of complaints. 60. The departmental committees and the director general (monitoring) both have compromised neutrality. The ombudsman, however, is independent and autonomous in resolving complaints and has sufficient powers. Score awarded Not scored Indicator 11. Degree of access to information. 61. This indicator deals with the quality, relevance, ease of access, and comprehensiveness of information on the public procurement system. Score awarded 1 62. The score for this indicator is based on assessment of sub-indicator 11(a) described below. Sub-indicator 11: Information is published and distributed through available media with support from information technology when feasible. (i) Rule 47 states that as soon as a contract has been awarded the procuring agency shall make all documents related to the evaluation of the bid and award of contract public, provided that such disclosure does not include information of a proprietary nature, and is not against the public interest. (ii) This rule is not being implemented fully. Generally pre-award processes like prequalification notices and tender notices are available in the newspapers and on the website, but the tender evaluation report, status of awards, and complaints are not published widely. Score awarded 1 Indicator 12. The province has ethics and anticorruption measures in place. 63. This indicator assesses the nature and scope of the anticorruption provisions in the procurement system. 36 Baseline Procurement Performance Assessment Report Score awarded 2 64. The score for this indicator is based on the assessment of sub-indicators 12(a) to 12(g) described below. Sub-indicator 12(a): The legal and regulatory framework for procurement, including tender and contract documents, includes provisions addressing corruption, fraud, conflict of interest, and unethical behavior, and sets out (either directly or by reference to other laws) the actions that can be taken with regard to such behavior. (i) Rule 2(f) addresses corrupt and fraudulent practices. Currently the tender documents do not give this definition, but the SBDs being prepared by the SPPRA will include such provisions. (ii) Rule 19 stipulates that procuring agencies shall specify a mechanism and manner to permanently or temporarily bar suppliers and contractors who either consistently fail to provide satisfactory performances or are found to be indulging in corrupt or fraudulent practices. Such barring action shall be duly publicized and communicated to the SPPRA. The NAB ordinance 1999 also addresses this issue. (iii) There have been conflicting directives that apportion responsibility for deviations from the rules, which demonstrates the need for a well-devised program of awareness and capacity building in respect of conducting procurements. Score awarded 2 Sub-indicator 12(b): The legal system defines responsibilities, accountabilities, and penalties for individuals and firms found to have engaged in fraudulent or corrupt practices. 65. The rules do not explicitly cover the penalties applicable to public servants, and do not refer to any other regulatory framework in this regard. Rule 9 of the NAB ordinance defines corruption and corrupt practices and states that all offences under this ordinance shall be non-bailable and that, following review, the chair of NAB shall refer the matter to a court for approval and for the release of the accused, if in custody. The above is applicable to all involved from the public and private sector. The link with the responsibilities of an individual in the procurement cycle may need elaboration. Rule 10 of the NAB ordinance defines penalties. Score awarded 1 Sub-indicator 12(c): Evidence of enforcement of rulings and penalties exists. 66. As mentioned above, relevant law exists, but there needs to be clearer links with procurement rules and procedures. There is some evidence of enforcement. 37 Pakistan: Sindh Province--Baseline Indicators System Score awarded 2 Sub-indicator 12(d): Special measures exist to prevent and detect fraud and corruption in public procurement. 67. Various regulatory structures are explained in the sub-indicators above. Rule 33 of the NAB ordinance mandates the bureau to review rules, hold workshops, and give operational advice on anticorruption measures. Integrating or linking the procurement rules with the anticorruption regulatory framework could make the system more effective. The role of civil society is yet to be elaborated and the use of a panel of experts as a resource pool may enhance credibility of the system. Score awarded 2 Sub-indicator 12(e): Stakeholders (private sector, civil society, and the ultimate beneficiaries of procurement and/or end-users) support the creation of a procurement market known for its integrity and ethical behavior. 68. There is only one strong, effective, and credible civil society organization which exercises social audit and control, i.e., Transparency International. Its influence is expanding but is still limited to certain sectors. The print media also provides some oversight and comments, albeit with challengeable professionalism. The reaction of the public sector to such activities is varied; it has reservations, but it is because of champions within the public sector that media and advocacy organizations are deepening their impact on public procurement. Score awarded 2.5 Sub-indicator 12(f): The province should have in place a secure mechanism for reporting fraudulent, corrupt, or unethical behavior. 69. Apart from the informal reporting channels, there is no secure, accessible, and confidential system for public reporting of cases of fraud, unethical behavior, or corruption. The grievance redress mechanism given in procurement rules is yet to become known and functional, but work is under way for adding a second tier of appeals. Score awarded 2 Sub-indicator 12(g): Existence of codes of conduct or codes of ethics for participants that are involved in aspects of the public financial management systems that also provide for disclosure for those in decision-making positions. 70. A code of conduct exists but accountabilities are not adequately assigned, and the disclosure mechanism is yet to be disseminated and implemented. ESTACODE covers a code of conduct for all public servants without reference to 38 Baseline Procurement Performance Assessment Report responsibilities in relation to grade. Special directives in this regard are also issued from time to time by the chief minister or the secretariat governor. Score awarded 2 39