68345 The World Bank An Assessment Tool for Public Procurement Agencies in Health Sector in India February 6, 2009 This note was prepared at the request of the Ministry of Finance and Ministry of Health and Family Welfare, Government of India by a World Bank team led by Shanker Lal (Procurement Unit, South Asia Region) with the assistance of Center for Pharmaceutical Management, Management Sciences for Health (MSH) under the overall guidance of Barbara Kafka (Director, Operational Services and Quality) and Rachid Benmessaoud (acting Country Director, India). Peer reviewers included Yolanda Tayler (Regional Procurement Manager, Middle East and North Africa Region, World Bank), G.N.V.Ramana (Lead Public Health Specialist, South Asia Human Development Unit, World Bank) and Andreas Seiter (Senior Health Specialist, Human Development Network, World Bank). The team also received helpful comments from Bob Saum (Manager, Financial Management, South Asia region, World Bank), Gerard Martin La Forgia (Lead Health Specialist, South Asia Human Development Unit, World Bank), and Giovanna Prennushi (Economic Advisor, India Country Management Unit, World Bank). The comments and support from the Department of Economic Affairs (DEA), Government of India; Ministry of Health and Family Welfare, Government of India; Tamil Nadu Medical Services Corporation (TNMSC), Tamil Nadu Drug Control Authority, British Department for International Development (DFID), United Nations Office for Project Services (UNOPS) and World Health Organization (WHO) are also gratefully acknowledged. 2 An Assessment Tool for Public Procurement Agencies in Health Sector in India Background 1. The Government of India (GoI) under its Common Minimum Program is enhancing central funding to states for effective delivery of essential health services to underserved populations residing in the rural areas of the country. Flexible financing is being provided to states under the National Rural Health Mission (NRHM) to develop locally relevant strategies to improve service delivery and implement evidence-based interventions to enhance health outcomes of centrally sponsored schemes in the health sector. Such rapid scaling-up of central allocations for different health sector schemes will involve massive procurement of health sector goods by the states and districts that have capacity constraints to undertake health sector procurement The NRHM framework notes that continued central procurement is not a solution and proposes a phased decentralization of procurement to states and districts1. This note supports the proposed approach of phased decentralization through development of an assessment tool for systematically identifying the procurement capacity constraints at state level. 2. The rules for regulation of medical consumables in India are relatively well developed, but effective enforcement remains a challenge. Therefore, organizations that procure medicines and medical supplies in India are compelled to create their own quality assurance systems to address areas that are not adequately covered by the statutory bodies in charge of enforcing regulations. Another weak area is the procurement of medical equipment, particularly in the area of technical specifications development and checking compliance of the bids to technical specifications. 3. The supreme audit institution of Government of India (Comptroller and Auditor General of India) has also noted some deficiencies in procurement of health sector goods in a recent report2. Though there are some good examples of efficient procurement in the health sector in Indian states that have established their own agencies for undertaking transparent procurement complemented by appropriate quality assurance and logistic supply systems (such as Tamil Nadu Medical Services Corporation or TNMSC), so far there is no consistent standard against which the performance of such state procurement agencies could be measured. 4. This note provide the list of standards that need to be assessed before considering an agency for handling the procurement of health sector goods. This tool is intended to:  Identify strengths and weaknesses of a procurement agency  Develop appropriate capacity-building plans to address the identified weaknesses  Monitor the improvement over the time 1 NRHM Framework for Implementation (2005-2012), Ministry of Health and Family Welfare, Government of India 2 Performance Audit Report for Ministry of Health and Family Welfare (2007), Comptroller and Auditor General of India 3 5. The assessment tool is based on available national/international standards and best practices, and has been further customized for Indian conditions. The tool is modular in nature and though it is initially meant for procurement of goods (including drugs, pharmaceuticals, equipment, commodities, medical and non-medical supplies), additional modules could be added for using it for procurement of services and civil works as well. Similarly, a separate module (based on good storage and distribution practices such as those described by WHO3) may be included in the tool to assess supply chain, logistics and inventory management aspects. This is particularly important for state public sector agencies which are also responsible for these functions while handling procurement for their home states 6. The draft assessment tool was validated by the study team by applying it on TNMSC before it was finalized. However, it is important to keep this assessment tool dynamic so that the standards could be periodically reviewed and raised once the agencies achieve the minimum standards. Also it is intended that the agencies scoring lower scores could initially be assigned less complex/risky procurement while they could simultaneously work on addressing their weaknesses. 7. Though the assessment tool has been designed for India, it could be used with some modifications for other developing countries also. The procurement agencies should be encouraged to voluntarily conduct their self assessment using this tool for achieving continuous improvement in their operations. Assessment Tool Description and Applications 8. The assessment tool (Appendix 1) includes eight modules with two of the modules further divided into sub-modules. The modules assess 156 standards and out of them, 32 (noted in the tool with the symbol “ �) are considered mandatory requiring full compliance to these standards (either during the assessment or subsequently based on the capacity-building action plan) is required for an agency to be considered for handling tendering. For the remaining standards, a variable scoring system (described in detail below) is proposed which allows for flexibility in determining the agency’s eligibility to undertake procurements. 9. The specific assessment tool modules and sub-modules include the following:  General requirements o Physical resources o Organization, structure, and functions  Transparency  Procurement cycle management o Procurement planning o Bidding documents o Pre- and post-qualification of bidders o Advertisement and sale of bid document o Communication during the bidding process 3 World Health Organization. 2006. Good Distribution Practices for Pharmaceutical Products. WHO Technical Report Series, No. 937, Annex 5. Geneva: WHO. 4 o Receipt of bids and bid opening o Bid evaluation o Contract award o Contract administration  Support and control systems  Record-keeping  Human resources and personnel  General risk assessment  Supplier perception assessment 10. The tool format includes columns for standards, compliance rating, assessor observations and comments, and instructions for assessors. A four-scale scoring system (0, 1, 2, 3) displayed under the column heading “Compliance Rating� is used by the assessor to provide a quantitative basis for rating compliance to each standard. For mandatory 32 standards, only a two-scale compliant rating (not compliant: “0� or completely compliant: “3�) has been provided. Thus for these standards, the assessor’s rating is restricted to either completely compliant or noncompliant/unacceptable. 11. For each module or sub-module, a total minimum score is given at the end of that particular module. This score is based on a minimum rating of “substantial compliance� to all standards, and full or complete compliance for those that are mandatory indicated by a “0� or “3� rating and followed by the symbol ( ). Assessor’s combined total score for all modules must be equal to or greater than the total minimum score to meet the acceptable standards for procurement. If an agency’s total score is less than the acceptable level, a phased approach is recommended linked to an action plan for capacity-building. 12. By designing the tool with modules and sub-modules and assigning minimum scores to each, the strengths and weaknesses of the Agency become readily apparent. The identification of weaknesses requiring significant corrective action (i.e., a compliance rating of “1�) within individual modules can, particularly in the context of Agencies where deficiencies noted are not widespread, serve as areas to focus for strengthening activities. When implementation of such activities is successful, the Agency could commence tendering in line with internationally accepted best practices. 13. The purpose of this assessment is not to “pass� or “fail� an Agency by strictly applying the scoring tool. The tool may find greater utility as a means to identify agencies that have a reasonable possibility of addressing identified weaknesses progressively leading to internationally acceptable procurement standards. Approach to Conducting an Assessment Following important points need to be kept in mind when conducting an assessment: 5 The assessment team requires right composition Considering the comprehensive nature of the tool and an evaluation system that depends on professional judgment for several indicators, assessors should possess an extensive international or domestic procurement experience, or both. Since the assessment is concerned with procurement of medicines, supplies, and medical equipment, it would be helpful, if not mandatory, for at least one team member to have a background in medicine or pharmacy and biomedical engineering. At least one of the team members should also have experience with and understanding of the internal controls and systemic aspects of procurement. A team of three such experts will be able to complete the assessment in one week. Finally, due consideration should be given to real or potential conflicts of interest while selecting the assessment team members. Advance communication is necessary To make the most efficient use of assessor and Agency staff time, the assessment team should send a message to the agency before the visit regarding the data and documents required for the assessment and should ask the agency to have pertinent files and documents ready for review by the assessment team upon their arrival. These documents include, for example, complete files for recently executed tenders for medicines, supplies, and medical equipment as well as agency policies and procedures for all procurement-related activities, financial statements, internal and external audits, and any state laws and regulations pertaining to procurement. In addition, the team should request that personnel involved with key procurement functions [e.g., overall management, finance, administrative (i.e., contract management), the tender process (i.e., issuance, evaluation, and award), quality control, and management information systems] are available for interviews when they are visiting the Agency. The questionnaire prepared by the Bank4 with suitable modifications may be used for gathering information. Assessment of a agency is an iterative process—not just the physical application of the tool As one would expect, the team should become thoroughly familiar with local and country procurement laws and regulations, the assessment tool and criteria before arriving at the agency. A team leader should be appointed, and based on the background of team members; sections of the tool should be assigned to each of them. It is important not to immediately embark on assessing compliance to standards during the site visit. The tool should be set aside for the initial work period in favor of conducting a thorough review of a number of recent tenders in order to understand the process, any specific issues arising from the review of those tenders, and how the decisions were arrived at. This review may take one or two days depending on the clarity of the records and the availability of key staff for interviews and questioning. Once this review has been completed, each assessor should move through his or her respective sections of the tool to summarize his or her evaluation, standard by standard. Continuous discussion between assessment team members are also vital to think and talk through complex issues to ensure that they are evaluated judiciously through a consultative process rather than individual judgment. At the end of this phase, it is expected that residual gaps in information and imperfections in assessor understanding will be cleared which guide the next steps including additional data and information gathering from records and interviews. This iterative process of triangulating 4 Available on http://go.worldbank.org/2GBDECUNI0 6 conclusions and data as necessary should be continued until the evaluation is complete for each of the criteria. Willingness of the agency to share the information and introduce improvement is a must The assessment cannot be conducted unless the procurement agency shares the information and documentation required, participates in interviews and discussions, and is open to introducing improvements to address the areas of weaknesses identified during the assessment. Credible action plan to build the agency’s capacity is essential Finally, the team must help the agency in designing an action plan to address the areas of weaknesses identified in a time bound manner. The action plan should also include additional technical assistance and likely resources required and the mechanism to monitor the progress on capacity building plan (reassessment may be needed is some cases to monitor the progress). Delegation of procurement needs to be phased as every agency will have some or other areas of weakness. Although one must not discount the importance of the overall scoring, it is acknowledged that none of the agencies may meet all the criteria listed in the tool. Hence, the final decision to delegate the procurement to the assessed agencies should be taken based on the quality of the action plan proposed to address the weaknesses as well as agency’s willingness and track record of introducing the changes. Once the agreement on action plan is reached, less complex procurement may initially be delegated with intensive supervision and handholding from the Principal. Further delegation of complex procurement should be linked to the history of successful performance for initial procurement assigned and also achieving agreed milestones in the action plan. 7 Appendix-1: Assessment Tool Matrix Ratings: Compliance to Standards Completely Compliant = 3 points Substantially compliant with minor corrective action required = 2 points Some compliance and significant corrective action required = 1 point Noncompliant/unacceptable = 0 points Name of Procurement Agency: Date of Assessment: Members of Assessment Team: Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor Circle compliance Assessor comments are to be used for rating; justification of rating. For mandatory standards only two ratings are shown: 0 or 3 1.0 General Requirements 1.1 Physical Resources 1.1.1 Premises Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 1.1.1.1 Office is conveniently located for 0/1/2/3 There is unencumbered work space available for personnel and has sufficient storage space for employees. Records are stored in an orderly and documentation records, reports, product samples easily retrievable fashion (records/documents and other records relating to all procurement should all be filed); a lockable file/container is activities. available for storing vendor bids until bid opening. If medicine samples are stored at the agency, storage is secure and temperature controlled. 1.1.1.1.1 Office space is available for arranging 0/1/2/3 An office sufficient to seat typical number of pre-bid meeting, managing receipt of bids and bidders (based upon review of procurement records public bid opening. for past 2 years) is available. 1.1.1.2 Computers are available to facilitate 0/3 Computers and monitors are in good operating procurement procedures. condition and available to procurement personnel. 1.1.1.2.1 If computers are available, software is 0/1/2/3 Minimally, the office has Microsoft 2000 operating appropriate for the activities performed, staff is system (or equivalent) with word processing and adequately trained in their use, security systems spreadsheet software; financial management, are in place to prevent unauthorized access, back- purchase order processing, and sales order up systems are in place to prevent loss of data, processing software or system is available. Observe firewall is installed, and virus protection software personnel using computers and software, and is available and updated regularly. determine from interviews their familiarity with software. Document computer security and backup systems. Document if organization utilizes e- tendering and/or e-procurement, and if so, confirm via interview(s) that staff are familiar with procedures. 1.1.1.2.2 Computer technical support personnel 0/1/2/3 If support personnel are not onsite, does agency are either available onsite or are available locally. have a contract for technical support, or is there a record of support by a local company? 9 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 1.1.1.2.3 Computer hardware is sufficient to 0/1/2/3 Interview computer operators; request information operate software efficiently; there is sufficient concerning frequency of computer crashes or capacity and memory for intended use; printers downtime and operating condition of printers. are available and in proper working order. 1.1.1.2.4 Computer maintenance is performed 0/1/2/3 Verify that there are computer maintenance records regularly and/or personnel are familiar with general computer maintenance programs (i.e. Microsoft or equivalent system tools such as defragmenter, antiviral program) and that they are up to date. 1.1.1.3 There is adequate access to telephone and 0/1/2/3 Personnel with job responsibilities requiring access facsimile, and if computers are available, e-mail to phone or e-mail have easy access to equipment. and broadband connectivity to the Internet is available 24 hours per day. 1.1.1.4 Office equipment (e.g., copying machines, 0/1/2/3 Verify that routinely needed office equipment, scanner, paper shredder), supplies (e.g., furniture, and supplies are available [i.e., copier(s), stationery, computer supplies, printing paper), and desks, chairs, filing cabinets that can be secured, office furniture meet requirements of the printers, computer paper and supplies such as CDs, procurement office. printer ribbon, computer backup device, forms, paper punch, stapler(s)] are available onsite. 1.1.1.5 Transport required for performance of 0/1/2/3 The agency has a policy for travel reimbursement or official duties is available or expenditure is transport is provided by the agency. reimbursed to the staff. MINIMUM SCORE FOR RESOURCES 21 ACTUAL SCORE FOR RESOURCES 1.2 Organization, Structure, and Functions 10 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 1.2.1 The agency holds legal status (i.e., 0/3 State the status. government department, state corporation, registered society) and is accountable to the government. 1.2.2 Written standard operating procedures The organization has its own written procurement (SOPs) and policies are available for all rules and regulations and/or follows government procurement-related activities and are reviewed or legislated rules and regulations. The policies and updated regularly (annually, at a minimum). associated SOPs enumerated below shall be available for review. Assessor shall comment on policy and SOP availability and document annual review and/or revision, if any. If no formal written policy or procedures are available, assessor will evaluate any available documentation relative to the above. In addition, interviews with key personnel will be used to determine working knowledge of policies and SOPs. In rating compliance to standards for policy and SOP, only a written SOP together with supporting documentation and interviews indicative of adherence to the SOP shall be scored as completely compliant (�3"). The maximum score shall be 2 points (substantially compliant) when a written SOP is unavailable but other documentation exists to indicate there is a policy and an SOP in place AND interviews with key personnel indicate adherence to a less than formalized SOP. These instructions apply to 1.2.2.1 through 1.2.2.11. 11 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 1.2.2.1 SOP for Supplier and product 0/1/2/3 prequalification including facility inspection and product quality (prior to tender or as part of tender process) 1.2.2.2 SOP for Advertisement of tenders 0/1/2/3 1.2.2.3 SOP for Bidder questions and requests for 0/1/2/3 clarification prior to bid opening 1.2.2.4 SOP for Bid opening 0/1/2/3 1.2.2.5 T SOP for ender evaluation and award 0/1/2/3 1.2.2.6 SOP for Contract and price negotiation 0/1/2/3 1.2.2.7 SOP for Supplier dispute or complaint 0/1/2/3 resolution 1.2.2.8 SOP for Quality assurance for products 0/1/2/3 procured and operations 1.2.2.9 SOP for Contract management 0/1/2/3 1.2.2.10 SOP for Internal and external audits 0/1/2/3 1.2.2.11 SOP for Code of conduct and conflict of 0/1/2/3 interest 1.2.3 Agency personnel have experience with health commodity procurement for the commodity categories below (1.2.3.1–1.2.3.2) 1.2.3.1 Medicines, supplies including laboratory 0/1/2/3 Meets Standard Subst. Meets Min. Meets and medical supplies and test kits >USD 10 M USD 500K–10M USD 250K–500K 1.2.3.2 Medical equipment 0/1/2/3 Meets Standard Subst. Meets Min. Meets >USD 10 M USD 500K–10M USD 250K–500K 12 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 1.2.4 Delegation of thresholds for contracting 0/1/2/3 If written documentation not available, assessor powers within the organization (invitation and must determine thresholds from interview with acceptance of bids) are available and regularly managing director (or equivalent) or finance updated. director (or equivalent). 1.2.5 Records of all operations are maintained in a 0/1/2/3 Assessor must verify that procurement records are secure, easily retrievable manner and access is filed in an organized fashion and secure (in a locked limited to only authorized individuals. cabinet); if electronic records are maintained, ensure that they are password protected. 1.2.6 The finance staff follows procedures to 0/1/2/3 Procurement plan and records reflect availability of ensure that available funds are used efficiently funds and payment. and are allocated/ earmarked before tender is issued and released as per the approved purchase contract. 1.2.7 The quality staff assures that all categories 0/1/2/3 For all products procured, quality assurance is of products procured are of acceptable quality and carried out in accordance with terms dictated by the meet standards required by the funding agency. If Principal. If not specified, internal quality policy for none are specified, meet agency’s in-house health commodities provide for comprehensive standards. For all products procured, testing for quality assessment including visual inspection and quality is carried out in accordance with laboratory analysis where applicable. Principal’s (GoI or the state, whoever is using the services of the agency) instructions. If none are specified, the agency has a quality policy in place. For health products for which state or nationally approved testing facilities are available to perform analysis against product specification, such testing is carried out. 13 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 1.2.8 A management oversight committee with 0/1/2/3 Committee or equivalent is in place with written financial, legal and technical expertise reviews policies and procedures. contract disputes MINIMUM SCORE FOR ORGANIZATION, 39 Note: Minimum score per standard for 1.2.3.1 and 1.2.3.2 is 1 point STRUCTURE, AND FUNCTIONS ACTUAL SCORE ORGANIZATION, STRUCTURE, AND FUNCTIONS MINIMUM TOTAL SCORE FOR GENERAL 60 REQUIREMENTS ACTUAL TOTAL SCORE FOR GENERAL REQUIREMENTS 2.0 Transparency 2.1 Detailed records of all procurement 0/1/2/3 Randomly review a minimum of two recent proceedings are maintained. procurements (at least one for medicines that includes several dozen items and one for equipment), and document if detailed records are available describing, pre-bid conference (if any), bid opening, pre- and post-qualification, tender committee meetings, and contract award proceedings. 2.2 Bid openings are open to the public (i.e., the 0/1/2/3 Randomly review the record of at least two bid bidders), and all bidders receive an invitation to openings to confirm all bidders received an bid openings. invitation to attend. 2.3 The organization does not normally conduct 0/1/2/3 Random review of two procurement records reveals discussions or financial negotiations with bidders price negotiation does not normally take place on a after opening of the price bids. routine basis. 14 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 2.3.1 If discussion or financial negotiations take 0/1/2/3 Written rules and procedures are available and place after bids are opened, written rules and random review of three procurement records reveals procedures are strictly adhered to and a record is adherence to rules. available to ensure that such negotiation is a fair and transparent process. 2.3.2 The contract award information is disclosed 0/1/2/3 Review of two randomly chosen procurement to all companies who have submitted the bids records confirm adherence to standard. following review by the competent authority. 2.4 Open competitive bidding is the preferred or 0/1/2/3 Policy statement is available. default procurement method unless the organization carries out a supplier pre- qualification process. If manufacturer pre- qualification is standard, restricted tender shall be the default method. 2.5 Direct- or single-source procurement without 0/1/2/3 Request copies of single-source procurement tender process is utilized in instances where there documents and verify that the record reveals only is only one pre-qualified source for the product or one pre-qualified (or not) source was available and in case of extreme emergency. In such a case, that the price compares favorably with historical historical "reasonable" pricing is used to negotiate price or current market price. price with supplier. 2.6 Requirements for bid submission and bid and 0/3 Written (preferable) documentation is available. If performance securities are required of all bidders. state law requires exemption for bid security, rating is 3 2.7 If agency does not carry out pre-qualification 0/3 Policy should state that all tenders are �open.� of bidders, all suppliers are eligible to bid. 15 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 2.8 The process of bid examination and evaluation 0/3 Bid examination procedures that do not provide is rational and fair. opportunity for bidder discrimination are available, and a random review of tender evaluation committee minutes reveals adherence to procedures. 2.9 There are rules and procedures for bidder 0/1/2/3 Rules should also be specified in the procurement suspension and debarment (i.e., blacklisting) and contract and the bid document. these are known to suppliers. 2.10 Public disclosure of the complete 0/1/2/3 Verify, that disclosure is as prescribed by the Right procurement process is freely available (except to Information Act, state law or policy or agency for the commercially confidential information). policy 2.11 Regular meetings with the business 0/1/2/3 Verify record of meetings. community are held or are available to the business community to discuss procurement issues. A record of such meetings is maintained and includes follow-up actions, if any. MINIMUM SCORE FOR TRANSPARENCY 29 ACTUAL SCORE FOR TRANSPARENCY 3.0 Procurement Cycle Management 3.1 Procurement Planning 3.1.1 The hierarchy of sources for procurement 0/1/2/3 State the hierarchy. rules is well-established (i.e., laws, government regulations and procedures, the organization’s own rules and procedures). 3.1.2 The agency is or can be allowed by its 0/3 The mandate to act as a procurement agent is mandate to act as a procurement agent and follow evident from the business rules or articles of the procurement procedures of the Principal (i.e., association. GoI, or other states or donors in case of externally aided projects). 16 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.1.3 The agency has a procurement plan template 0/1/2/3 Agency follows a predetermined procurement plan that is closely adhered to after a procurement and examples are available for inspection. request is received; the plan should include, at a minimum, a schedule for advertising the tender, receipt of bids, bid opening, bid evaluation, and contract award. 3.1.3.1 There is a designated employee or team to 0/1/2/3 Employee or team is in place and interview reveals prepare/monitor the procurement plan that understanding of procurement planning. includes, at a minimum, scheduling of activities. 3.1.4. There is a designated employee assigned to 0/1/2/3 Personnel are in place. approve each stage of the procurement cycle and an overall director (or equivalent) responsible for managing the complete process. 3.1.5 There is general adherence to the dates 0/1/2/3 At least two procurement plans selected at random indicated in the procurement plans. and reviewed indicate adherence to the plan. MINIMUM SCORE FOR PROCUREMENT 13 PLANNING ACTUAL SCORE FOR PROCUREMENT PLANNING 3.2 Bidding Documents 3.2.1 Standard bidding documents are used for 0/1/2/3 Bidding documents are available. procurement of health sector goods. 3.2.1.1 Bidding documents are readily adaptable 0/1/2/3 Examples are available. to specific contract situations (i.e., modifications made through a bid data sheet, special conditions of contract, or similar). 17 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.2.2 Procurement staff are knowledgeable in 0/1/2/3 Interviews with procurement staff reveal familiarity preparation and modification of bidding with preparation and modification of standard documents. bidding documents. 3.2.2.1 Procurement staff is familiar with the 0/3 Interview procurement staff and check the procurement policies, guidelines, and standard correspondence with Principal, if already handling bidding documents of the Principal and has Principal-funded procurement. demonstrated experience with correctly completing the bid data sheet and drafting special conditions of contract. 3.2.3 Technical specifications for health 0/1/2/3 Perform random review of specifications for commodity products are clear, generic, neutral, medicines, medical supplies and kits, equipment, updated and accurate. and consumables. 3.2.3.1 Technical specifications are prepared by a 0/1/2/3 Technical specifications are provided by the committee that includes expert knowledge Principal, internal staff, outside committee, or a concerning products to be procured. combination of internal and external experts. 3.2.3.2 Technical specifications for medicines are 0/3 Specification includes International Pharmacopoeia, issued by the Principal and are included in bid British Pharmacopoeia, or United States documents. Pharmacy expertise is available for Pharmacopoeia standard. Document if pharmacy specification review and to request clarifications expertise is available in-house or from an outside of specifications issued by the Principal. When source. specifications are not provided, specifications are prepared with input from staff or consultants with pharmacy expertise. 3.2.3.2.1 Shelf life consideration is included in 0/3 Shelf life is included among contract terms. technical specifications for medicines or other supplies where applicable. 18 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.2.3.3 Technical specifications for medical 0/3 Review of tenders includes technical specifications. equipment and supplies or test kits are provided Document if medical equipment and supply by the Principal and are included in the bid expertise is available in-house or from an outside document. Equipment and supply expertise is source. available for specification review and to request clarification(s) for specifications issued by the Principal. 3.2.4 Instructions to bidders contain all 0/1/2/3 Bidder instructions are complete for equipment , information necessary to prepare responsive bids supplies, and pharmaceutical contracts. (i.e., they provide a clear understanding of eligibility requirements and how evaluation criteria will be applied). 3.2.4.1 Bid security required in bidding 0/1/2/3 Review the bidding instructions. instructions as a condition of responsiveness is appropriate (e.g., sliding scale but no more than 3% of total value). An absolute amount is used rather than percentage of bid amount MINIMUM SCORE FOR BIDDING 26 DOCUMENTS ACTUAL SCORE FOR BIDDING DOCUMENTS 3.3 Pre- and Post-qualification of Bidders 3.3.1 Supplier and product pre-qualification 0/1/2/3 Written pre- and post-qualification policy and and/or post-qualification is carried out prior to procedures for suppliers are available. A review of award of any health sector contract. three randomly selected procurements reveals documentation that pre- and/or post-qualification was carried out in accordance with policy and procedures. 19 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.1.1.1 Pre- and/or post-qualification 0/1/2/3 Detailed product specifications and pre- and post- requirements are clear and completely describe all qualification guidelines for submitting company and requisites for submitting a responsive application product information including agency evaluation and qualification requirements. criteria are stated in the bid invitation or solicitation. At least one bid invitation for each health sector product category (i.e., pharmaceutical, medical supply or kit, medical equipment) where an agency procures such products shall be reviewed for comprehensiveness. 3.3.1.2 The pre- and/or post-qualification process 0/1/2/3 A review of process procedures reveals is carried out in a fair and transparent manner and transparency, and a review of actual procurements decisions are made promptly. reveals that prescribed policies and procedures were followed. 3.3.1.3 Pre- and/or post-qualification includes 0/1/2/3 Documentation exists that technical and financial assessment of technical and financial capacity to capacity to supply requirement(s) is assessed as part supply required value of product(s). of the pre- and post-qualification process. 3.3.1.4 All evaluations and inspections are carried 0/1/2/3 Pharmaceutical and medical supply inspections are out by qualified evaluators who have signed a carried out by personnel trained to perform such declaration of interest (i.e., no situations of real, inspections. Inspections are carried out by a team of potential, or apparent conflict of interest are at least two people. Medical equipment evaluation known to them) preferably with participation of shall be carried out by a biomedical engineer. external experts. 3.3.1.5 For medicines and medical supplies or 0/1/2/3 Document the adherence to standards. kits, evaluation includes preparation of manufacturer site master file and product files that include existing certificates of inspection and/or quality (i.e., WHO GMP, Schedule M, ISO, most current inspection report). 20 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.3.2 Planning and preparation for manufacturer 0/1/2/3 Documentation available and interview with inspection includes notification of inspection date, representative inspector reveals understanding of completion of a site master file by the planning and preparation procedures. manufacturer, and agency familiarity with product information submitted by manufacturer. 3.3.3 Inspections are carried out in accordance 0/1/2/3 Inspection is carried out by a team of at least two with written procedures and include all aspects of inspectors; inspection may be outsourced to a GMP in line with GMP guidelines. competent agency including State Drug Control Authority.. 3.3.3.1 Waiving of inspections for procurement of 0/3 Review waiver policy to determine if in line with pharmaceuticals is permissible only if an standard and if waivers have been issued, waiver is inspection has been carried out by a Stringent appropriately issued. Drug Regulatory Authority - SDRA (i.e., International Conference on Harmonization or Pharmaceutical Inspection Cooperation Scheme member country) in the past 36 months and a GMP certificate has been issued without conditions (copy of inspection report and certificate is on file with the agency) or if the agency has carried out a GMP inspection in line with WHO GMP standards at the manufacturing facility in the past 36 months with the finding of no deficiencies and there have been no major changes to premises, equipment , or key personnel since the inspection (the latter applies to inspection by a SDRA as well). 21 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.3.4 Once all pre/post qualification data is 0/1/2/3 Record of prequalification evaluation is available collected and evaluation committee (or and is approved by the recognized agency equivalent) recommends qualification of a competent authority. bidder/product, final approval is issued by the Agency's appointed management authority. 3.3.5 Agency inspection report is communicated 0/1/2/3 Review three examples of agency inspection to the manufacturer and includes noncompliance reports and verify that reports include observed and observed and recommended corrective actions. recommended corrective actions and that reports have been forwarded to respective manufacturer. 3.3.6 Qualitative or physical and laboratory 0/1/2/3 Records reveal that as part of the pre-qualification analysis are performed as per pharmaceutical process, manufacturer samples are tested and meet product specifications on samples for all batches specifications by a laboratory equipped to perform by a laboratory operating in line with Good pharmaceutical analysis and certified by a national Practices for National Pharmaceutical or international organization (e.g., WHO) Laboratories. recognized for its certification competency. 3.3.6.1 Agency has records to demonstrate that 0/1/2/3 Procedure for sampling of batches received is samples from medicine batches supplied by the documented, and records are available that reveal manufacturer have undergone and passed the adherence to procedure. same testing used for samples during prequalification phase. Note: This requirement applies only if pre-qualification is carried out and sampling is included. If pre-qualification sampling is not carried out, agency has documentation that for batches received; full pharmacopoeial analysis is carried on a random basis as a minimum. 22 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.3.7 There is evidence that pre/post qualified 0/1/2/3 Obtain a list of pharmaceutical or supply manufacturers and products are re-evaluated at a manufacturers who have responded to tenders for minimum of every three years. more than three years, and verify that records are available documenting re-evaluation or inspection of manufacturing location and products. 3.3.8 The agency maintains a list of registered and 0/1/2/3 Verify that such a list is available and has been qualified suppliers that is updated at least updated annually. annually. 3.3.4.1 Evidence is available that, over the past 0/1/2/3 Documentation is available. two years, manufacturers have been added and/or deleted and/or temporarily disbarred for participation in agency procurements. MINIMUM SCORE FOR PRE- AND POST- 33 QUALIFICATION OF BIDDERS ACTUAL SCORE FOR PRE- AND POST- QUALIFICATION OF BIDDERS 3.4 Advertisement and Sale of Bid Document 3.4.1 Contracts awarded on the basis of 0/3 Written policy (may be included in state legislation) competitive bidding are advertised publicly in at is available that mandates competitive bidding, and least one national newspaper (in addition to local a copy of an ad placed in at least one national or internet advertising). If the agency does not newspaper is available for review. Interview with have its own website, it has access to one that is procurement personnel confirms adherence to freely accessible for publishing invitation for bids. policy. 3.4.2 Sufficient time (at least four weeks) is 0/1/2/3 Advertisement of tenders includes sufficient time provided to bidders for obtaining documents and for bidders to obtain documents. preparation of bids. 23 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.4.3 If an open tender process is utilized, sale of 0/3 Required bid documents are available to all who request them, and the price of bid document is reasonable. 3.4.3.1 Bid documents are always available on the 0/1/2/3 Required date specified in the advertised notice 3.4.3.2 Bid documents are available for sale or 0/3 Required available (e.g., from agency website) until the day of the bid opening. MINIMUM SCORE FOR ADVERTISEMENT 13 AND SALE OF BID DOCUMENT ACTUAL SCORE ADVERTISEMENT AND SALE OF BID DOCUMENT 3.5 Communications during the Bidding Process 3.5.1 Requests for clarifications are promptly and 0/1/2/3 Prompt, written clarification is required by agency completely provided in writing and shared with policy. Sampling of at least three procurement all bidders of record. records reveals that clarifications and responses to questions are provided within five days and shared with all bidders of record (i.e., a copy of the response is sent to all bidders is available). 3.5.2 Clarifications, minutes of the pre-bid 0/1/2/3 Records are available. conference, if any, and modifications to the bid documents are promptly communicated to all prospective bidders. 3.5.2.1 Bidders are provided ample opportunity to 0/1/2/3 When a major bid modification occurs, agency revise their bids following a modification to policy provides bidders with additional time to bidding documents. submit bids. Confirm this provision during interview with procurement personnel. 24 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.5.3 The agency maintains accurate records of all 0/1/2/3 Review the records. communications with bidders prior to and after the deadline for submission. MINIMUM SCORE FOR 8 COMMUNICATIONS DURING BIDDING PROCESS ACTUAL SCORE FOR COMMUNICATIONS DURING BIDDING PROCESS 3.6 Receipt of Bids and Bid Opening 3.6.1 Bids received prior to the deadline are 0/3 A secured "bid box" is available and access is securely stored and procedures are in place to limited to no more than two high-level agency ensure that no tampering can occur. officials. 3.6.2 The time elapsed between final day for 0/1/2/3 Time elapsed should be no more than one week. submission of bids and opening of bids shall be minimal and preferably shall occur on the same day. 3.6.3 All bid openings are conducted publicly, and 0/3 Required all firms that submitted a bid receive invitations to attend. 3.6.3.1 The names of all bidders and the price 0/3 The bid opening procedure is followed. Bids are quoted by them are read aloud to bid opening recorded in a register and available for review by all participants, or this information is provided in bidders and the public. written or electronic form and then recorded in a bid register. 3.6.4 Bids received after the deadline for 0/3 Required submission are returned unopened to bidders. 25 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.6.5 Bid procedure follows a single-envelope 0/3 If state legislation requires bid procedure other than process where technical and price bid are single-envelope process (technical and price provided unless otherwise permitted by state together), so state. legislation. If a two-envelope system is used, the price bid is securely stored until the technical bids are evaluated and opened in presence of technically qualified bidders. MINIMUM SCORE FOR RECEIPT OF BIDS 17 AND BID OPENING ACTUAL SCORE FOR RECEIPT OF BIDS AND BID OPENING 3.7 Bid Evaluation 3.7.1 For every procurement, bid evaluation is 0/1/2/3 Document the list of committee members and conducted by a committee composed of qualified background for evaluating the following types of members with expertise in pharmaceuticals and/or procurement: (1) medicines s, (2) medical supplies clinical medicine, other health care professionals, and test kits, (3) medical equipment, and (4) educators, medical equipment technical experts consumable supplies. State in comment section if (for equipment procurements). Some external there are external members on the team. A rating of members are desirable. complete compliance requires external members. 3.7.2 Bid responsiveness is determined solely on 0/3 A review of tender documents reveals that a the basis of meeting all requirements in the bid technical responsiveness checklist is available, and documents. the checklist reveals that all technical requirements have been received. 3.7.3 Among all bidders who meet previously 0/3 Required specified technical requirements, the lowest bidder is selected for award. 26 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.7.3.1 There shall be no negotiations with any 0/3 Required bidder including the lowest bidder unless this bid exceeds known current market price, is higher than historical pricing, or is otherwise permitted by state law. 3.7.3.2 In the event of default or inability to 0/3 Required, unless state law specifies otherwise supply as per contract by the lowest bidder, the second lowest bidder shall be notified and awarded the contract. The same system shall apply for the third lowest bidder in the event the second defaults or cannot supply, and so on. 3.7.4 The managing director, tender committee, or 0/3 Required other committee charged with this responsibility approves the award. If the Principal requires a "no objection," it shall be obtained prior to final approval. 3.7.5 A bid evaluation report is prepared for all 0/1/2/3 Review a minimum of two bid evaluation reports, procurements, and it includes all essential and verify documentation. information including a complete description of the evaluation process, reasons for rejecting bids as nonresponsive, the methodology for applying evaluation criteria, and verification of successful bidder's qualifications. 3.7.5.1 The bid evaluation report is maintained on 0/3 Required file for the duration of the contract at a minimum, and reasons for rejection of bids are provided to unsuccessful bidders, if requested. 27 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.7.6 The agency shall provide the following Assign point value as follows: 3 points if at least information to the assessor: total number and one contract within category is awarded annually value of contracts awarded in the past five years with a value of at least USD 1 million; 2 points if at (or less, if in operation less than five years) least one contract within category is awarded through ICB (global tender), NCB (open tender), annually with a value per category of USD 50K– limited tender, and single tender. Value shall be 1M; 1 point if at least USD 50K; 0 point if less than supplied separately for categories of medicines or USD 50 K. supplies and medical equipment or nonmedical supplies. 3.7.6.1 Medicines, medical supplies or kits 0/1/2/3 Assess on the basis of total value of tender. 3.7.6.2 Medical equipment or nonmedical 0/1/2/3 Assess on the basis of total value of tender. supplies 3.7.7 The agency shall provide an average time 0/1/2/3 Assess point value as follows: 1 point awarded if from date of bid opening to award of contract for the majority of contracts are awarded 60 days each category and shall list the number of beyond validity period, 2 points if they are awarded contracts awarded under each category within (1) up to 60 days, and 3 points if they are awarded the initial validity period, (2) 60 days after initial within the initial validity period. bid validity, and (3) beyond 60 days after initial bid validity. MINIMUM SCORE FOR BID EVALUATION 28 Note: Minimum score per standard for 3.7.6.1. and 3.7.6.2 is 1 point. ACTUAL SCORE FOR BID EVALUATION 3.8 Contract Award 3.8.1 The agency shall have a standard contract 0/1/2/3 Verify the availability and use of a standard form, which will be part of the bid document. contract form. 3.8.2 If the agency performs "shopping" (i.e., for 0/1/2/3 Verify the availability and use of a standard emergency or very low value procurements), a quotation form. standard quotation form and purchase order form is available. 28 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.8.3 Procurement rules dictate award to the 0/3 Rules shall be written and adherence is evidenced lowest qualified bidder unless otherwise by review of three procurement records. countermanded by state law (e.g., regulation pertaining to state-owned bidder or preference discount for suppliers operating within the state). 3.8.3.1 Should a low bidder be disqualified, this 0/1/2/3 During interview with management staff, ask if a disqualification is documented in the tender low bidder has ever been disqualified (i.e., due to record. quality issues). If affirmative, verify this disqualification is recorded in the tender record. 3.8.4 Unless otherwise directed by state or 0/3 Required national legislation, there is no negotiation with bidders regarding technical terms and conditions. 3.8.5 Performance security is required in an 0/1/2/3 Required appropriate amount and format. 3.8.6 Rules for repeat order system limit an 0/1/2/3 Required increase in contract value to no more than 25% with proper justification. MINIMUM SCORE FOR CONTRACT 16 AWARD ACTUAL SCORE CONTRACT AWARD 3.9 Contract Administration 3.9.1 A. manual or computerized procurement 0/1/2/3 Verify that there is an SOP for supplier process and contract monitoring is carried out (i.e. performance monitoring. Request that the agency , monitoring of direct and indirect product costs, provide examples of companies issuing a complaint, receipt monitoring [damage], quality monitoring, and verify that the complaint and action taken is monitoring of adherence to delivery schedules, documented. complaint monitoring). 29 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.9.2 The normal time lapse from invoice 0/1/2/3 Randomly select three awards and determine if submission to final payment indicates that payment is made in accordance with contract term. payment is generally made on time. 3.9.3 Contract changes or variations are not 0/3 Required permitted after award, unless they are in line with the provisions of contract. 3.9.4 Supplier claims are handled fairly and in 0/1/2/3 Required; review the complaints register. accordance with contract terms. 3.9.4.1 Procuring entities normally make a good 0/1/2/3 Determine, if possible, the number of disagreements faith effort to resolve disagreements through handled by formal negotiation or arbitration. informal negotiations, if it is permitted as per the dispute resolution procedure included in the contract. 3.9.4.2 If informal negotiations fail, there are 0/1/2/3 Document the procedures. established formal arbitration procedures which are described in the contract 3.9.5 Contracts are completed on schedule and 0/1/2/3 Review three randomly selected contracts and within the approved contract price. verify adherence. 3.9.6 Rejection of goods due to substandard 0/1/2/3 Required quality are referred to the Drug Control Authority for disposition. 3.9.6.1 Contract terms include rejection of goods 0/1/2/3 There is a product complaints registry or equivalent with compensation and/or blacklisting of supplier and review of records reveal that complaints are if evaluation findings of product complaint(s) handled promptly (i.e., within 30 days) and received from an end user reveals a defective/ disposition of complaint is recorded; all complaints substandard product (confirmed by Drug Control are thoroughly investigated as evidenced by a Authority). written record and reported to the Drug Control Authority for further evaluation and disposition. 30 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 3.9.7 There shall be a formal contract closing 0/1/2/3 Written procedure is available. procedure. MINIMUM SCORE FOR CONTRACT 21 ADMINISTRATION ACTUAL SCORE CONTRACT ADMINISTRATION MINIMUM TOTAL SCORE FOR 175 PROCUREMENT CYCLE MANAGEMENT ACTUAL TOTAL SCORE FOR PROCUREMENT CYCLE MANAGEMENT 4.0 Support and Control Systems 4.1 Internal and external auditing arrangements 0/1/2/3 Audit plan is available; internal and external audits are in place and established and overall audit plan take place. is available that describes audit goals, schedules, staffing, and reporting; audit reports are available for inspection (and are preferably published on the website). 4.1.1 Internal audits include at a minimum: (1) 0/1/2/3 Document the level of adherence. operational audits (e.g., compliance with Assess point value as follows: Completely procurement policies and procedures), (2) compliant for all three audit types = 3; substantially financial audits (e.g., review of agency's financial compliant for all three audit types = 2; some statements to determine whether financial compliance for one or more audit types = 1 point; statements fairly present the financial position as noncompliant for any type of audit = 0 points. of a certain date), and (3) information technology audits (assess the controls, accuracy, and integrity of an institution's electronic data processing and computer areas). 31 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 4.1.2 Audit reports contain a concise summary of 0/1/2/3 Document the level of adherence key results and conclusions, including identification of root causes of significant weaknesses, audit scope and objectives, detailed audit results, recommendations (if any), and management's commitments to correct material weaknesses. 4.1.2.1When audit recommendations are 0/1/2/3 Document the level of adherence. provided, appropriate action(s) are taken and recorded. There is demonstrated support from senior management/board on resolution of audit findings. Audit committee at Board level (headed by an independent director) is also desirable. 4.1.2.2 An external audit program performed by 0/1/2/3 Document the level of adherence. an outside independent auditor provides a financial statement audit, substantiation of internal controls over financial reporting, or other external audit procedures. 4.1.2.3 Procurement audits shall be carried out by 0/1/2/3 Document the level of adherence. the Accountant General's office or contracted third party. A copy of all audits is maintained in the agency office and is available for public inspection unless otherwise specified by the auditing authority. 32 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 4.2 Internal auditors are independent of activities 0/3 Required they audit and possess necessary knowledge and skill to successfully implement the audit program in a proficient and professional manner. Internal audits shall encompass compliance with financial, procurement, and information technology procedures. 4.3 The agency shall demonstrate that it has made 0/1/2/3 Agency shall have an attorney of record. arrangement for access to quality legal advice and input. MINIMUM TOTAL SCORE FOR SUPPORT 17 AND CONTROL SYSTEMS ACTUAL TOTAL SCORE FOR SUPPORT AND CONTROL SYSTEMS 5.0 Record-Keeping 5.1. The agency maintains a detailed contract 0/1/2/3 Obtain list of contract awards and randomly select award record for all competitive bids including three awards for review; document availability of (1) copies of all public advertisement, (2) the pre- these 10 documents as per standard. qualification evaluation report, (3) bidding documents, (4) a record of any pre-bid meetings, (5) minutes of bid opening, (6) a bid evaluation report including reasons for acceptance and rejection of bids, (7) appeals against procedures or award recommendations, (8) a signed copy of the final contract, (9) performance and advance payment securities issued, and (10) cross references to pertinent files. 33 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 5.2 Contract administration records are 0/1/2/3 Review a minimum of three contract administration maintained and include at a minimum: (1) records; document the degree of adherence. contractual notices issued by the supplier; (2) a detailed record of all change or variation orders issued affecting the scope, quantities, timing, or price of the contract; (3) records of invoice and payments; (4) progress reports; (5) certificates of inspection, acceptance, and completion; and (6) records of claims, disputes, and disputes outcome. 5.3 Periodic management reports are prepared 0/1/2/3 Records are available. summarizing overall procurement activities for a defined period of time (e.g., quarterly, semiannually, annually) and such reports are available for review. 5.4 A record of contract prices is maintained and 0/1/2/3 Verify the contract price record is maintained; the procurement unit regularly conducts periodic ensure that, at a minimum, prices can be easily supplier market surveys or reviews available retrieved. survey information (including international price reference sources) to update knowledge of product sources and prices for health sector goods. 5.5 Historical contract prices along with 0/1/2/3 To be determined from procurement policy and/or adjustment for subsequent changes are used to interview with personnel evaluate new bids. MINIMUM SCORE FOR RECORD- 10 KEEPING ACTUAL TOTAL SCORE FOR RECORD- KEEPING 34 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 6.0 Human Resources and Personnel 6.1 The procurement agency organization 0/1/2/3 There are offices, departments, committees, or includes sufficient management-level and support technical experts to carry out the Agency mandate. positions for all key procurement functions: There is a full-time managing director or equivalent overall management (managing director or chief and at least one employee is assigned to each key executive officer), financial, internal audit, procurement function. (Note: Ensure that the same contract administration and management, tender employee is not assigned to more than one key process (to issue, evaluate, and award), quality procurement function.) control, and management of administrative staff (i.e., management information systems and support staff). Reporting lines are clearly delineated (attach organizational chart). 6.1.1 Personnel responsible for specific 0/1/2/3 Department includes minimum of five key staff procurement functions have written job where there is one lead for management of product descriptions describing their tasks and specification, pre- or post-qualification, tender responsibilities and are familiar with applicable evaluation, quality assurance, and contract SOPs and policies (see 1.2.1). Staff is updated as management. Review job descriptions and to changes in rules and thresholds and other issues interview a minimum of three procurement relevant to their assigned responsibilities. employees to determine familiarity with SOPs; if there are no written SOPs, conduct interviews to determine employees’ familiarity with job responsibilities and procurement thresholds. 6.1.2 For each category of commodity 0/1/2/3 Review employees’ and consultants’ CVs to procurement (drugs, medical supplies / test kits, determine background and experience and utilize medical equipment, consumables), there is at least interviews to confirm that the employee’s or one employee or contracted outside consultant consultant’s status is current; if the CV is not on with requisite expertise to handle each respective file, probe for background and experience. Assessor commodity category undertaken by the agency. shall include in comments any commodity categories not procured by the agency. 35 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 6.1.3 For the volume of business, there are 0/1/2/3 There is no indication that procurement operations sufficient personnel who are appropriately trained, are impeded by lack of personnel (e.g., delays in educated, and experienced to perform key awarding contracts, supplier complaints regarding activities including pre-qualification, quality contract management, or an unusual number of assurance and management, finance and emergency procurements). If current staffing is administration, procurement, and management insufficient to satisfy additional procurement information services. volumes, assessor to determine agency flexibility to hire additional staff and provide compensation at market linked remuneration. 6.2 The pre- or post-qualification employee or 0/3 Required office is independent from the purchasing employee or office and is carried out by a team of qualified personnel. 6.3 The product evaluation employee or office is 0/3 Product evaluation responsibilities include receipt independent from manufacturer inspection of product information, screening and evaluation of employee office. product information, and communication of the results of product evaluation to the manufacturer. 6.4 An on-the-job training program or outside 0/1/2/3 Based upon interviews with employees, determine training exists for all staff for promotion of if such programs fit one of the following categories: professional development. well-organized, somewhat available, occasional training provided, or no training provided. 6.5 Procurement staff is experienced in 0/1/2/3 Determine the level of experience (i.e., highly international procurement, if they handle global experienced, moderate, little, none). tendering. 6.6 Procurement staff and other staff have training 0/1/2/3 Determine the level of expertise (i.e., highly or exposure in detecting indicators of fraud and experienced, moderate, little, none). corruption/collusion in the procurement process 36 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor MINIMUM TOTAL SCORE FOR HUMAN 20 RESOURCES AND PERSONNEL ACTUAL TOTAL SCORE FOR HUMAN RESOURCES AND PERSONNEL 7.0 General Risk Assessment 7.1 The agency shall be able to demonstrate that 0/1/2/3 Declaration of interest and code of ethics or conduct they have taken steps to curb or control corruption are in place. Interviews with a minimum of three (i.e., employees sign conflict of interest key employees directly involved with procurement statements and provide annual returns of assets reveal that there is a mechanism to report corrupt held by them). activities and that there is no fear of repercussion should anyone do so. Declaration of Interest form is available and inspectors or evaluators have a current, signed declaration on file (assessor to verify) stating that no real, potential, or apparent conflict of interest situations are known to them and that have disclosed if they have a financial or other interest in and/or relationship with parties who may have vested commercial interest in obtaining access to any confidential information disclosed to them. 7.1.1 The agency includes in its bid documents 0/1/2/3 Review bid documents for contract terms and contracts a provision describing steps that concerning bribery or corrupt practice. will be taken in the event of confirmed attempt of bribery or corrupt practice by a contractor or agency employee. 37 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 7.1.2 There are no unreasonable supplier 0/1/2/3 Request a list of manufacturers that have been disqualifications and/or noncompliance notices disqualified from consideration of bid or to whom a issued. notice of noncompliance has been issued. Review the records, and document reason(s) for disqualification or noncompliance. Are they trivial? Are they authorized by competent authority in the agency? 7.1.3 Agency analyzes individual supplier bids for 0/1/2/3 Agency tender analysis SOP discusses analysis of similarities in bid language, bid rotation, and/or bids for similarities in language and/or price. If no prices to other bidders. written policy exists, determine from interviews with at least two members of tender evaluation committee that this is a consideration. Review three manufacturer bid proposals for the same tender (look at three tenders), and determine if similarities in language or price. 7.1.4 Contract prices do not normally exceed 0/1/2/3 For 10 randomly selected contract prices, compare agency estimates. price to estimate. If the contract price far exceeds the estimate, it may be evidence of profiteering and corruption or lack of skills or information for preparing estimates. 7.2 Procurement staff members are held in high 0/1/2/3 Base this evaluation upon private sector interviews. regard within the agency. 7.3 Powers related to procurement are clearly 0/1/2/3 Written policy and procedures are available; if not delegated to the agency carrying out the written, personnel are aware of where powers are procurement process, and procurement procedures delegated. are clearly defined. 38 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor 7.4 The agency operates independently and is not 0/1/2/3 The agency has operational policies that do not required to consult/seek guidance from state require or include state government approval government in its day-to-day operation. (except for contract award or if the state has not authorized an independent procurement agency). 7.4.1 Any violation that comes to the attention of 0/1/2/3 A procedure is in place for handling bidder, the agency or government is promptly contractor, or supplier representations or complaints investigated and recorded, and a public record of before or after contract award. All complaints the complaint and disposition is maintained. whether settled informally or formally are documented. 7.4.1.1 A complaints registry exists within the 0/1/2/3 There is an easily retrievable complaints registry or agency. equivalent and review of records reveals that complaints are handled promptly (i.e., within 30 days); dispensation of the complaint is recorded. All bidder, contractor, or supplier complaints are recorded in the registry 7.5 There is a mechanism in place that allows for 0/1/2/3 A "whistle blowing" policy or equivalent is reporting of bribes and solicitation or extortion by available. procurement officials. 7.6 If the agency has previous experience of 0/3 Review previous Principal funded procurements handling procurement on behalf of Principal, no (three, if possible) for a record of complaints. or only minor issues have been documented with regard to timeliness, transparency, misprocurement, incidence of complaint, or reversal of decisions by the Principal MINIMUM TOTAL SCORE FOR GENERAL 25 RISK ASSESSMENT 39 Compliance Standard Rating Assessor Observations/Comments Instructions for Assessor ACTUAL TOTAL SCORE FOR GENERAL RISK ASSESSMENT 8.0 Supplier perception assessment 8.1 The agency procurement system is viewed as 0/1/2/3 Base the evaluation on interviews with at least 10 generally efficient and predictable. private sector organizations that have been successful and unsuccessful in obtaining awards from the agency. 8.2 The procurement process is transparent in all 0/1/2/3 regards. 8.3 Contract management is straightforward and 0/1/2/3 Interviews with private sector organizations confirm understandable, and the agency has a record of that contract management is reasonable, and a releasing payment within the time period review of contract records reveals that payments are stipulated in the contract. made as stipulated in the contract. 8.4 The agency is viewed as free of corruption. 0/1/2/3 MINIMUM TOTAL SCORE FOR SUPPLEIR 8 PERCEPTION ASSESSMENT ACTUAL SCORE FOR PRIVATE SUPPLIER PERCEPTION ASSESSMENT MINIMUM ASSESSMENT TOTAL SCORE 344 ACTUAL ASSESSMENT TOTAL SCORE Note: 1) There is a possibility that an agency does not presently procure medicines (and vice versa equipment, nonmedical supplies, or medical supplies and test kits) or that an agency utilizes only a particular procurement method (e.g., national tender or limited tender). In such circumstances, the standards that are not applicable should not be assessed, and for each standard not assessed, 2 points should be deducted from the total minimum required score. 40 2) Standards where scoring appears with the symbol are mandatory (total compliance required either during assessment or agreement to comply with the same in the Action Plan to be prepared to address the areas of weakness) 41 Recommended Reading for Assessors 1. World Health Organization. 2006. Annex: A Model Quality Assurance System For Procurement Agencies. In Recommendations for Quality Assurance Systems Focusing on Prequalification of Products and Manufacturers, Purchasing, Storage and Distribution of Products. WHO Technical Report Series, No. 937. Geneva: WHO. 2. World Health Organization. 2003. Annex 6: Procedure for Assessing the Acceptability. In Principle, of Procurement Agencies for use By United Nations Agencies, Technical Report Series, No. 917. Geneva: WHO. 3. World Bank. 2002. Revised Country Assessment Procurement Report Procedures (CPARs), May 2002. Washington, DC: The World Bank. 4. Union and state government’s procurement legislations, rules and manuals – particularly for the state where the agency being assessed is located - various