Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized March 2015 Solar Resource Mapping in Malawi MODEL VALIDATION REPORT This report was prepared by GeoModel Solar, under contract to The World Bank. It is one of several outputs from the solar resource mapping component of the activity Resource Mapping and Geospatial Planning Malawi [Project ID: P151289]. This activity is funded and supported by the Energy Sector Management Assistance Program (ESMAP), a multi-donor trust fund administered by The World Bank, under a global initiative on Renewable Energy Resource Mapping. Further details on the initiative can be obtained from the ESMAP website. This document is an interim output from the above-mentioned project. Users are strongly advised to exercise caution when utilizing the information and data contained, as this has not been subject to full peer review. The final, validated, peer reviewed output from this project will be the Malawi Solar Atlas, which will be published once the project is completed. Copyright © 2015 International Bank for Reconstruction and Development / THE WORLD BANK Washington DC 20433 Telephone: +1-202-473-1000 Internet: www.worldbank.org This work is a product of the consultants listed, and not of World Bank staff. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work and accept no responsibility for any consequence of their use. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for non-commercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: +1-202-522-2625; e-mail: pubrights@worldbank.org. Furthermore, the ESMAP Program Manager would appreciate receiving a copy of the publication that uses this publication for its source sent in care of the address above, or to esmap@worldbank.org. World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results World Bank Group, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi Project ID: P151289 Model Validation Report – Preliminary Results March 2015 GeoModel Solar, Pionierska 15, 831 02 Bratislava, Slovakia http://geomodelsolar.eu Reference No. (GeoModel Solar): 141-02/2015 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results TABLE OF CONTENTS Table of contents .....................................................................................................................................................4   Acronyms .................................................................................................................................................................5   Glossary ...................................................................................................................................................................7   1   Summary ............................................................................................................................................................8   2   Model quality indicators ...................................................................................................................................9   3   Inventory of solar atmospheric and meteorological validation data .........................................................10   3.1   Solar resource measurements .............................................................................................................10   3.2   Solar resource modelled data...............................................................................................................12   3.3   Atmospheric data..................................................................................................................................13   3.4   Meteorological measurements .............................................................................................................14   3.5   Meteorological models..........................................................................................................................15   4   Validation of aerosol data ..............................................................................................................................17   4.1   Evaluation of MACC-II Atmospheric Optical Depth data ......................................................................17   4.2   Seasonal variability of Atmospheric Optical Depth ...............................................................................20   5   Validation of solar resource data ..................................................................................................................23   5.1   Quality control of solar validation data..................................................................................................23   5.2   Validation of solar resource model .......................................................................................................26   6   Validation of meteorological data .................................................................................................................29   6.1   Validation sites .....................................................................................................................................29   6.2   Air temperature at 2 metres ..................................................................................................................29   6.3   Relative humidity ..................................................................................................................................30   6.4   Wind speed...........................................................................................................................................32   7   Uncertainty of the model estimates ..............................................................................................................34   7.1   Solar resource parameters ...................................................................................................................34   7.2   Meteorological data ..............................................................................................................................35   8   List of figures ..................................................................................................................................................36   9   List of tables ....................................................................................................................................................37   10   References .....................................................................................................................................................38   11   Background on GeoModel Solar .................................................................................................................39   World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results ACRONYMS AERONET The AERONET (AErosol RObotic NETwork) is a ground-based remote sensing network dedicated to measure atmospheric aerosol properties. It provides a long-term database of aerosol optical, microphysical and radiative parameters. AOD 670 Aerosol Optical Depth at 670 nm. This is one of atmospheric parameters derived from MACC-II database and used in SolarGIS. It has important impact on accuracy of solar calculations in arid zones. BSRN Baseline Surface Radiation Network CFS v2 Climate Forecast System. The meteorological model operated by the US service NOAA (National Oceanic and Atmospheric Administration) CFSR Climate Forecast System Reanalysis. The meteorological model operated by the US service NOAA. CMSAF Satellite Application Facility on Climate Monitoring (CMSAF) aims at the provision of satellite-derived geophysical parameter data sets suitable for climate monitoring. Several cloud parameters, surface albedo, radiation fluxes at the top of the atmosphere and at the surface as well as atmospheric temperature and humidity products form a sound basis for climate monitoring of the atmosphere are available. DIF Diffuse Horizontal Irradiation, if integrated solar energy is assumed. Diffuse Horizontal Irradiance, if solar power values are discussed. DNI Direct Normal Irradiation, if integrated solar energy is assumed. Direct Normal Irradiance, if solar power values are discussed. ECMWF European Centre for Medium-Range Weather Forecasts is independent intergovernmental organisation supported by 34 states, which provide operational medium- and extended-range global forecasts and a computing facility for scientific research. GAW Global Atmosphere Watch. It is a worldwide system established by the World Meteorological Organization to monitor trends in the Earth's atmosphere. GFS Global Forecast System. The meteorological model operated by the US service NOAA. GHI Global Horizontal Irradiation, if integrated solar energy is assumed. Global Horizontal Irradiance, if solar power values are discussed. GTI Global Tilted (in-plane) Irradiation, if integrated solar energy is assumed. Global Tilted Irradiance, if solar power values are discussed. MACC Monitoring Atmospheric Composition and Climate – meteorological model operated by the European service ECMWF (European Centre for Medium-Range Weather Forecasts) Meteonorm Meteonorm is a database with a monthly climate averages from meteorological stations around the world developed by Meteotest This includes global radiation, temperature, humidity, precipitation, days with precipitation, wind speed and direction and sunshine duration. Data for any geographical location is calculate by spatial interpolation of values from the nearby meteorological stations. Meteosat MFG and MSG Meteosat satellites operated by EUMETSAT organization. MSG: Meteosat Second Generation; MFG: Meteosat First Generation NCDC NOAA's National Climatic Data Center (NCDC) is responsible for preserving, monitoring, assessing, and providing public access to the climate and historical weather data and information. NCDC database contains mainly collection of data from the national networks belonging to World Meteorological Organization (WMO). page 5 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results NOAA NCEP National Oceanic and Atmospheric Administration, National Centre for Environmental Prediction PVGIS Photovoltaic Geographical Information System developed by Joint Research Centre (JRC), the European Commission. Online free solar photovoltaic energy calculator for stand-alone or grid-connected PV systems. PVGIS works for Europe, Africa and Asia. Solar electricity generator simulation and solar radiations maps. RSR Rotating Shadowband Radiometer TEMP Air Temperature at 2 metres page 6 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results GLOSSARY Aerosols Small solid or liquid particles suspended in air, for example soil particles, sea salts, pollen or air pollution such as smog or smoke. Bias Represents systematic deviation (over- or underestimation) and it is determined by systematic or seasonal issues in cloud identification algorithms, coarse resolution and regional imperfections of atmospheric data (aerosols, water vapour), terrain, sun position, satellite viewing angle, microclimate effects, high mountains, etc. KSI Kolmogorov-Smirnoff index. It characterizes representativeness of distribution of high frequency (e.g. hourly) values. Root Mean Square Represents spread of deviations given by random discrepancies between measured and Deviation (RMSD) modelled data and is calculated according to this formula: ! ! !!! 𝑋! !"#$%&"' − 𝑋!"#$%$# ! 𝑅𝑀𝑆𝐷 = 𝑛 On the modelling side, this could be low accuracy of cloud estimate (e.g. intermediate clouds), under/over estimation of atmospheric input data, terrain, microclimate and other effects, which are not captured by the model. Part of this discrepancy is natural - as satellite monitors large area (of approx. 3 x 4 km), while sensor sees only micro area of approx. 1 sq. centimetre. On the measurement side, the discrepancy may be determined by accuracy/quality and errors of the instrument, pollution of the detector, misalignment, data loggers, insufficient quality control, etc. Solar irradiance Solar power (instantaneous energy) falling on a unit area per unit time [W/m2]. Solar resource or solar radiation is used when considering both irradiance and irradiation. Solar irradiation Amount of solar energy falling on a unit area over a stated time interval [Wh/m2 or kWh/m2]. Spatial grid In digital cartography the term applies to the minimum size of the grid cell or in the other words resolution minimal size of the pixels in the digital map Uncertainty Is a parameter characterizing the possible dispersion of the values attributed to an estimated irradiance/irradiation values. In this report, uncertainty assessment of the solar resource estimate is based on a detailed understanding of the achievable accuracy of the solar radiation model and its data inputs (satellite, atmospheric and other data), which is confronted by an extensive data validation experience. The second important source of uncertainty information is the understanding of quality issues of ground measuring instruments and methods, as well as the methods correlating the ground-measured and satellite-based data. In this report, the range of uncertainty assumes 80% probability of occurrence of values. Thus, the lower boundary (negative value) of uncertainty represents 90% probability of exceedance, and it is also used for calculating the P90 value. Water vapour Water in the gaseous state. Atmospheric water vapour is the absolute amount of water dissolved in air. page 7 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 1 SUMMARY Background This Model Validation Report shows method and results of preliminary validation of solar resource and meteorological data for the Republic of Malawi, Phase 1 of solar resource mapping and measurement services. The project is a part of technical assistance in renewable energy development implemented by the World Bank in Malawi. It is being undertaken in close coordination with Ministry of Natural Resources, Energy and Mining of Malawi, the World Bank’s primary country counterpart for this project. The project is funded by the Energy Sector Management Assistance Program (ESMAP), a global knowledge and technical assistance program administered by the World Bank and supported by 11 bilateral donors. It is part of a major ESMAP initiative in support of renewable energy resource mapping and geospatial planning across multiple countries. Data and methods This report documents validation of solar resources calculated by satellite model SolarGIS, and meteorological data derived from the CFSR and CFSv2 models. Inventory in Chapter 3 identifies the existing data sources in the region: solar, aerosol and meteorological data. First, aerosol (Atmospheric Optical Depth, AOD) data from the MACC-II model is evaluated (Chapter 4, this data is used on the input to SolarGIS clear-sky model). Chapter 5 shows relative comparison of SolarGIS GHI and DNI to other modelled databases. Next, validation to high-quality solar resource measurements demonstrates stable performance of SolarGIS in geographic conditions, similar to Malawi. Chapter 6 validates meteorological parameters that are used for site-specific data and for maps. Chapter 7 summarizes validation results in the estimate of uncertainty. Results Validation demonstrates stable performance of SolarGIS model in Africa. The validation and previous experience indicate that using high-quality local measurements, the SolarGIS model output have significant potential for reduction of uncertainty in geography of tropical Africa. page 8 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 2 MODEL QUALITY INDICATORS The performance of satellite-based models, for a given site, is characterized by the following indicators: 1. Bias characterizes systematic model deviation at a given site; 2. Root Mean Square Deviation (RMSD), Standard deviation (SD) and Mean Average Deviation (MAD), which indicate spread of error for instantaneous values (typically hourly or sub-hourly); 3. Kolmogorov-Smirnoff index (KSI) characterizes representativeness of distribution of values. This indicator is applied only for solar resource data. Focus of this report is validation and uncertainty assessment of SolarGIS solar resource data that are derived in the form of spatial and site-specific data products. The meteorological data are also validated as they are used in the site-specific times series and Typical Meteorological Year data (TMY). Air temperature is also used as a spatial data product. Only quality-controlled measurements from high-standard sensors can be used for objective validation of satellite-based solar model, as issues in the ground measured data result in skewed evaluation. Typically, bias is considered as the first indicator of the model accuracy, however the model should be interpreted analyzing also all other accuracy measures. While knowing bias helps to understand a possible error of longer-term estimate, MAD and RMSD are important for estimating the accuracy of energy simulation and operational calculations (monitoring, forecasting). KSI reveals issues in the model’s ability to represent specific solar radiation conditions. This is especially important in the CSP modelling, as the response of these systems is non-linear to irradiance levels. Even if bias of different satellite-based models is similar, other accuracy characteristics (RMSD, MAD and KSI) may indicate substantial differences in their performance. Validation statistics for one site may not provide representative picture of the model performance in the given geographical conditions. The reason is that one particular site may be affected by a local microclimate or by hidden issues in the ground-measured data. Therefore, the model should be evaluated at several validation sites. If ability of the model to estimate longterm values, at least two measures are to be considered [1]: • Mean bias deviation, which indicates whether the model has overall tendency to overestimate or to underestimate the measured values. • Standard deviation of biases, which shows the range of deviation of the model estimates (statistically one standard deviation characterizes 68% probability of occurrence). Good satellite models are consistent in space and time, and thus the validation at several sites within one geography provides a robust indication of the model accuracy in geographically comparable regions elsewhere. Besides bias and RMSD, the ability of the model to simulate representatively sub-hourly values for all conditions (especially high and low light conditions) is very important for optimisation of the solar power plants. Two evaluation studies have been conducted independently by University of Geneva [1, 2]. Both studies analyse features of existing solar radiation models based on processing of satellite data. The studies show that SolarGIS model demonstrates robust and harmonized performance in all indicators. page 9 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 3 INVENTORY OF SOLAR ATMOSPHERIC AND METEOROLOGICAL VALIDATION DATA 3.1 Solar resource measurements Public data Solar radiation, unlike other basic meteorological parameters, is measured only at few meteorological stations in Africa. Solar measurements are collected by various organizations: by international or regional professional networks, meteorological agencies or universities. Access to these data may be restricted by data usage polices. Inventory shows that only few sources provide data with sufficient quality required for the validation of SolarGIS model (Tab 3.1) in geographic conditions comparable to those of Malawi. Table 3.1: Sources of solar resource validation data Network Description AMMA African Monsoon Multidisciplinary Analyses. AMMA is an international project to improve the knowledge and understanding of the West African Monsoon (WAM) and its variability. http://amma-international.org/ BSRN Baseline Surface Radiation Network (BSRN) provides near-continuous, long-term, in situ-observed broadband irradiances (solar and thermal infrared) and certain related parameters from a network of more than 50 globally diverse sites. Data usually include GHI, DIF and DNI measurements. Data from De Aar and Tamanrasset meteorological stations are used. http://www.bsrn.awi.de/ http://www.bsrn.awi.de/en/data/data_retrieval_via_pangaea/ SASSCAL Southern African Science Service Centre for Climate Change and Adaptive Land Management in Southern Africa conducts a problem-oriented research in the area of adaptation to climate change and sustainable land management. Solar radiation measurements are part of the project monitoring activities. http://www.sasscal.org/ http://www.sasscalweathernet.org/ SAURAN SAURAN (Southern African Universities Radiometric Network) is an initiative of the Centre for Renewable and Sustainable Energy Studies (CRSES) at Stellenbosch University and the Group for Solar Energy Thermodynamics (GSET) at the University of KwaZulu-Natal. The network provides high-resolution, ground- based solar radiometric data available from stations located across the Southern African region, including South Africa, Namibia, Botswana and Reunion Island. In this report, four SAURAN data sets were used. http://www.sauran.net/ SolRad-Net SolRad-Net (Solar Radiation Network) is an established network of ground-based sensors providing high- frequency solar flux measurements in quasi-realtime to the scientific community and various other end users. This network was implemented as a companion to AERONET and its instrumentation are invariably collocated with AERONET sites. http://solrad-net.gsfc.nasa.gov/ STERG Sonbesie is a station operated by STERG (Solar Thermal Energy Research Group), a research group housed in the Department of Mechanical and Mechatronic Engineering in Stellenbosch University and affiliated with the Centre for Renewable and Sustainable Energy Studies, Stellenbosch University. STERG has performed measurements on one site considered in this report. http://weather.sun.ac.za/ Before decision was made whether the ground-measurements are to be used for the model validation, the data was quality controlled (Chapter 5.1). In general, measurements from networks such as AMMA, BSRN, SAURAN and Sonbesie show quality that allows using them for the model validation. BSRN, SAURAN SolRad-Net and Sonbesie produce data in 1, 2 and 10 minute time step, AMMA data are available in hourly time step only. page 10 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results Preliminary quality control of measurements from SASSCAL measurement network has shown quality issues and high uncertainty of this data, therefore they were excluded from evaluation of SolarGIS model (Chapter 5). List of public solar resource measuring stations, checked for possible use for validation of SolarGIS model (Chapter 5.2), is summarized in Tab. 3.2. Their position is shown in Figure 3.1. Figure 3.1: Position of solar sites in Africa that were used for validation in this report Table 3.2: Solar measuring stations in Africa, used for validation of SolarGIS Site name Country Source Latitude Longitude Altitude GHI DNI Period [°] [°] [m a.s.l.] Tamanrasset Algeria BSRN 22.7833 5.5137 1378 YES YES 03/2000 – 12/2006 Bamba Mali AMMA 17.0990 -1.4018 272 YES NO 2006 Agoufu Mali AMMA 15.3445 -1.4791 290 YES NO 2006 M Bour Senegal AMMA 14.3940 -16.9590 5 YES NO 2006 Banizoumbou Niger AMMA 13.5311 2.6613 211 YES NO 2006 Djougou Benin AMMA 9.6920 1.6620 438 YES NO 2006 Nairobi Kenya SolRad-Net -1.3388 36.8653 1650 YES NO 12/2005 – 03/2009 De Aar South Africa BSRN -30.6667 24.0000 1331 YES YES 06/2000 – 12/2004 Durban South Africa SAURAN -29.8710 30.9769 150 YES YES 02/2013 – 07/2014 Graaff-Reinet South Africa SAURAN -32.4855 24.5858 660 YES YES 12/2013 – 09/2014 Port Elizabeth South Africa SAURAN -34.0086 25.6653 35 YES YES 12/2012 – 07/2014 Sonbesie South Africa STERG -33.9282 18.8651 144 YES YES 02/2012 – 07/2014 Vanrhynsdorp South Africa SAURAN -31.6175 18.7383 130 YES YES 11/2013 – 09/2014 page 11 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results Private initiatives A number of solar measuring stations are deployed by companies active in development of solar energy projects in the region. The measured data are used for commercial and technological assessment of solar resource for particular projects and they are not publically available. 3.2 Solar resource modelled data Public databases There are several modelled databases available in the region (Table 3.3). In general, the databases based on the interpolation of ground-measured data, such as Meteonorm [3] are less reliable in areas with sparse availability of meteorological stations. PVGIS HelioClim-1 [4, 5] is calculated from daily HelioClim-1 data with limited reliability. The global database NASA SSE [6] is computed by empirical models from satellite and atmospheric data and shows only global climate patterns at coarse-resolution. SWERA/NREL database has medium spatial resolution and is computed using CSR model by NREL [7], thus only showing overview perspective. The closest to SolarGIS is satellite-based database PVGIS CMSAF, however the data are not updated regularly and are available only as long-term averages [8]. Implementation of these databases is static, and they are not updated regularly. Table 3.3: Inventory of solar resource models for Malawi Model Data source Data spatial Parameter Time resolution Period resolution of available data NASA SSE Satellite + model 110 km x 110 km GHI, DNI Long-term monthly 1983 – 2005 Interpolation and Meteonorm 7.1 Ground + satellite GHI, DNI Long-term monthly 1981 – 2010 satellite data PVGIS Daily MFG satellite 30 x 30 km GHI Long-term monthly 1985 – 2004 HelioClim-1 data MFG and MSG Long-term monthly PVGIS CMSAF 3 km x 4 km GHI, DNI 1998 – 2011 satellites (hourly) SWERA/NREL Model 40 km x 40 km GHI, DNI Long-term monthly 1985 – 1991 MSG/MFG PRIME SolarGIS* 3 km x 4 km GHI, DNI 15 and 30 minutes 1994 – 2015* satellites * SolarGIS database is continuously updated on daily basis Commercial satellite-based databases On the market there are few solar databases developed and maintained by commercial entities that provide solar radiation data to customers for a fee. Most of these databases are based on the use of satellite data, but they differ in the model implementation and use of input data (e.g. aerosols, water vapour), therefore results may significantly differ. These databases differ also in spatial coverage, spatial and temporal resolution, operational update and other parameters. Besides quality of data, important for a user is easy access, ability of the system to deliver updated data, and support by services, such as site adaptation, derived data products (e.g. TMY), bankable solar resource assessment, map services and others. To our understanding, for Malawi, besides SolarGIS, the following commercial databases are available: SOLEMI, 3TIER, IrSOLaV and HelioClim-3. More information about these databases is available in [1, 2]. page 12 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 3.3 Atmospheric data Along with the clouds, aerosols are the most influential factor controlling GHI and DNI irradiance in the region, especially during cloud-free weather situations. The atmospheric turbidity is mostly influenced by burning biomass, soil particles, locally by human activities (agriculture, industry, transport and urbanization) and particles transported from other regions. Complex geography creates specific conditions for local distribution and transport of aerosols. The combined influence of these factors results in varying atmospheric pollution both spatially and temporarily. The accurate description of aerosols is difficult due to several factors: • Aerosols have high spatial and temporal variability, • There is insufficient number of aerosol-specialized meteorological stations, and often they have only short period of measurements, • In most of Africa, there are limited possibilities for detailed description of aerosol sources for chemical- transport atmospheric models, • Arid and semiarid conditions make it difficult to use satellite measurements of aerosols, • Dynamics of aerosols increases in a complex terrain. For aerosol characterization a data from chemical transport model MACC-II is used in SolarGIS. The original data with resolution of ca. 85 km and 125 km is post-processed by a) regional adaptation to remove systematic regional deviation of MACC-II database and b) altitude correction to better reflect local terrain conditions. The understanding of nature of the modelled aerosol data helps to indirectly evaluate the satellite based SolarGIS model. For this purpose a model input aerosol data was compared to aerosol measurements from AERONET [9]. Figure 3.2 shows location of the stations used for atmospheric data validation. No station is located in Malawi, though several stations exist in the wider region, having reasonably long period of measurements: • Mongu (Zambia) • Nairobi (Kenya) • Malindi (Kenya) • Mbita (Kenya) • Gorongosa (Mozambique) • Skukuza (South Africa) • Pretoria (South Africa) • Elandsfontein (South Africa) • Wits University (South Africa) • Henties Bay (Namibia) Fit of aerosol data to ground measurements is important indirect indicator of performance of satellite-based model (for SolarGIS evaluated in Chapter 4.2). page 13 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results Figure 3.2: Position of AERONET stations 3.4 Meteorological measurements The validation procedure was carried out by comparison of modelled data with ground-measured data at seven meteorological stations in the region (source NOAA NCDC), five stations in Malawi operated by Department of Climate Change and Meteorological Services of Malawi (DCCMS), and two other in the region operated by Southern African Science Service Centre for Climate Change and Adaptive Land Management (SASSCAL) (Figure 3.3) It must be noted that time period of data comparison for the SASSCAL and DCCMS network is relatively short (Table 3.4). Bvumbwe station has less than half a year of valid data and this station could be not used in the comparison (moreover other 3 stations − Chichiri, Mimosa and Chileka − are located close to Bvumbwe). Comparison with NCDC meteorological stations in the region is performed for a time period 2008 to 2010 (CFSR model) and for 2011 to 2014 (CFSv2 model) [10, 11]. Table 3.4: Meteorological stations in the region considered for validation of CFSR and CFSv2 model outputs Latitude Longitude Elevation Meteorological station Data source Time period [º] [º] [m a.s.l.] Lilongwe intl. airport, Malawi NOAA 01/2008 – 12/2014 -13.7830 33.7670 1229 Chileka, Malawi NOAA 01/2008 – 12/2014 -15.6830 34.9670 767 Lusaka intl. airport, Zambia NOAA 01/2008 – 12/2014 -15.3170 28.4500 1154 Harare, Kutsaga, Zimbabwe NOAA 01/2008 – 12/2014 -17.9170 31.1330 1480 Songea, Tanzania NOAA 01/2008 – 12/2014 -10.6600 35.5830 1036 Mbeya, Tanzania NOAA 01/2008 – 12/2014 -8.9330 33.4670 1758 Chimoio, Mozambique NOAA 01/2008 – 12/2014 -19.1170 33.4670 732 Kabwe, Zambia SASSCAL 10/2013 – 10/2014 -14.2926 28.5663 1143 Samfya, Zambia SASSCAL 10/2013 – 10/2014 -11.3712 29.5606 1197 Chitedze, Malawi DCCMS 05/2013 – 12/2014 -13.9846 33.6403 1148 Chichiri, Malawi DCCMS 03/2013 – 12/2014 -15.8000 35.0333 1101 Mangochi, Malawi DCCMS 01/2013 – 09/2014 -14.4833 35.2667 487 Mimosa, Malawi DCCMS 01/2013 – 12/2014 -16.0833 35.6200 653 Nkhotakota, Malawi DCCMS 01/2013 – 12/2013 -11.6167 34.2500 536 Bvumbwe, Malawi DCCMS 02/2013 – 06/2013 -15.9200 35.0700 1168 page 14 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results Figure 3.3: Position of meteorological stations considered for validation of CFSR and CFSv2 model outputs 3.5 Meteorological models Table 3.5 gives an overview of selected modelled meteorological data available for the region. Chapter 4.2 of Interim Solar Modelling Report 141-01/2015 gives more insight into global meteorological models. These models are run by meteorological organisations, such as US National Oceanic and Atmospheric Administration (NOAA), European Center for Medium range Weather Forecasting (ECMWF) or Canadian Meteorological Centre (CMC). Global meteorological models serve for purposes such as weather forecasting, modelling long-term climate processes and helping to understand weather phenomena in a global scale. Accuracy of modelled meteorological data for a specific geographical location cannot compete with the accuracy of well-maintained on-site meteorological sensors. However advantages of the modelled data are numerous: they cover large territories (some are global), they are free of maintenance and calibration issues, in case of reanalysis product they ensure seasonal and long term stability, long history, almost 100% availability both spatially and temporally and they offer data from any location on the Earth. This makes them a good choice for preliminary solar energy simulations. Table 3.5: Selected meteorological models available in the region. Database name Source Spatial resolution Time resolution Period CFSR NOAA, model 0.312° x 0.312° 1 hour 1979 to 2010 CFSv2 NOAA, model 0.20° x 0.20° 1 hour 2011 to present GFS NOAA, model 0.20° x 0.20° 3 to 6 hours 1991 to present ERA-Interim ECMWF, model 0.75°x 0.75° 6 hours 1979 to present GDPS CMC, model 0.225° x 0.225° 3 hours 2010 to present Meteonorm Ground-measurements Interpolation Long-term monthly 2000 to 2009 page 15 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results Meteonorm database is also mentioned in Table 3.6. It is a different type of weather database, based on ground measurements from a number (8325) of meteorological stations, where site-specific information is calculated by interpolation of monthly averages. Monthly averages are, in the second step statistically disaggregated to synthetic hourly data representing one year. This approach has limitations due to its static character (there is no systematic update) and limited performance in areas with sparse network of meteorological stations. Although this database was historically popular, with today’s computing and modelling options, this approach is overcome. In the delivery for Malawi, the meteorological parameters are derived from CFSR and CFS v2 models. Water vapour parameter - for solar resource model - is partially derived also from the GFS database. page 16 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 4 VALIDATION OF AEROSOL DATA Along with clouds, aerosol data is one of the most important parameters as it controls accuracy of solar models in arid and semiarid zones. Atmospheric aerosols include liquid and solid particles originating from different sources, e.g. soil particles, sea salts, burning biomass, industrial and traffic pollution and pollen. Aerosols have high spatial and temporal variability and complex behaviour in terms of absorption and scattering of solar irradiance. Increased aerosol concentrations in the atmosphere can reduce GHI in the range of 0% to 7%, occasionally up to 10%. In case of DNI, the variable aerosols can reduce daily DNI as much as 40% or even more. In solar modelling, aerosols are represented by the parameter called Atmospheric Optical Depth (AOD). 4.1 Evaluation of MACC-II Atmospheric Optical Depth data MACC-II aerosol data [12, 13], used in the SolarGIS model, provide good representation of temporal as well as spatial variability of aerosol load. Despite these qualities, the data may experience systematic deviation in some regions [14]. Here, we evaluate MACC-II AOD data using measurements from AERONET stations [9] located in a wider region (Figure 4.1). There is no station to be used for direct evaluation in Malawi, all sites show aerosol accuracy in the wider spatial context. Figure 4.1: AERONET sites used for MACC-II model evaluation. The SolarGIS model is based on aerosol data, which are regionally adapted for larger systematic deviations and they are also adapted for local altitude by empirical height correction [15]. Figures 4.2 and 4.3 demonstrate an accuracy of the MACC-II database used in the SolarGIS model. Comparison of post-processed daily MACC-II data and 15-minute AERONET ground measurements for Mongu site in Zambia shows very good fit. Seasonal profiles, as well as short extreme situations, are well represented. However, in conditions with very low aerosol load a slight overestimation of MACC-II AOD is found. On the other hand, for some of high load situations, the modelled AOD may be slightly underestimated. The discrepancy visible in the plot also arises from comparing high frequency (15-min) site-specific AERONET values with daily summaries of regionally-smoothed MACC-II data. page 17 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results The comparison of AERONET sites in a broader region shows generally a good representation of the AOD variability. At some sites, the higher bias of MACC-II aerosols (Henties Bay, Elandsfontein, Gorongosa, Malindi) may influence resulting accuracy of SolarGIS DNI and GHI irradiation. These sites are relatively far from Malawi, but may indicate potential issues of the MACC-II model in Malawi. Figure 4.2: Comparison of daily summaries from MACC-II model with 15-min AERONET data. Mongu AERONET station in Zambia Some discrepancies in individual stations may be attributed to the coarse resolution of MACC-II database, which is not capable representing the specific local conditions recorded in the AERONET data with sufficient accuracy. Also the spread of values may be partially explained by natural differences arising from the comparison of the exact point measurements (AERONET) with regionally averaged values (MACC-II). page 18 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results Figure 4.3: Comparison of Aerosol Optical Depth AERONET 675 nm (x-axis) and MACC 670 nm (y-axis); blue points: original MACC data; red points: MACC data regionally-adapted by SolarGIS method To understand potential issues with AOD in regions, where AERONET data are not available, the MACC-II data for Lilongwe were compared with other data sources from several satellite missions [16, 17] (Figure 4.4): • Terra MODIS, • Aqua MODIS, • Terra MISR, • Envisat MERIS page 19 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results All compared databases (except Envisat MERIS) show the same seasonal pattern with increased aerosol load in a period from August to October – the same trend was identified also in the Mongu AERONET station. Range of values between databases is also similar, only differences in extremely high values are identified. The spatial distribution of ground-measured data does not allow evaluating quality of aerosol data directly for Malawi. In general, analysis of available AOD data shows good representativeness of MACC-II AOD database in the Mongu site. In other sites some discrepancy can be in other sites could be observed. These uncertainties may be reduced or removed in Phase 2 of the ESMAP project, using high resolution and high–quality local measurements. Aqua MODIS (550nm, monthly) Terra MODIS (550nm, monthly) Terra MISR (555 nm, monthly) Envisat MERIS (550 nm, monthly) MACC-II (670nm, daily) Figure 4.4: Aerosol Optical Depth for Lilongwe from five different AOD databases Plots of satellite-based data were produced with by Giovanni online data system, NASA Langley ASCD [18] 4.2 Seasonal variability of Atmospheric Optical Depth SolarGIS uses AOD input data, at the wavelength 670 nm, derived from the MACC-II model. The MACC-II model captures high temporal variability of aerosols, thus it reduces uncertainty of instantaneous GHI and DNI estimates. Figure 4.5 shows typical monthly variability of aerosols in central and southern Africa. Malawi is located in a transition zone between region with high aerosol load in equatorial and sub-equatorial Africa and regions with low aerosol load in the southern part of continent. The seasonal pattern is influenced by high page 20 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results aerosol load events having their centre outside of the Malawian territory in the Northwest. The lowest aerosol concentration in the atmosphere can be observed from December to June, and the highest in September and October. From the global perspective, Malawi is a region with relatively low aerosol load (Figure 4.6), but due to seasonal increase still having significant influence on the dynamics of solar resource, especially DNI. Figure 4.5: Monthly-averaged aerosol maps (AOD 670) derived from the MACC-II database and adapted for the SolarGIS model. Period 2003 to 2013 page 21 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results Figure 4.6: Malawi in the world context – average annual aerosols (represented by AOD 670 nm) computed by the MACC-II model and adapted for SolarGIS. Period 2003 to 2012. Values are dimensionless. page 22 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 5 VALIDATION OF SOLAR RESOURCE DATA 5.1 Quality control of solar validation data Validation of satellite-based data was performed using ground measurements from the wider region. Prior to comparison with satellite-based solar resource data, the ground-measured irradiance was quality- controlled by GeoModel Solar. Quality control (QC) was based on methods defined in SERI QC procedures and Younes et al. [19, 20] and developed by GeoModel Solar. The ground measurements were inspected also visually, mainly for identification of shading and other regular data error patterns. Figure 5.1 shows an example of results of such quality control in two meteorological stations: Durban (South Africa) and Nairobi (Kenya). The colours indicate the following flags: • Blue: data excluded by visual inspection - mainly shading, incorrect tracking and calibration issues • Green: data passed all tests • Grey: sun below horizon • White strips: missing data • Red and violet: GHI, DNI and DIF consistency problem or problems with physical limits Figure 5.1: Quality control of data measured at Durban and Nairobi stations Top: Durban, South Africa, bottom: Nairobi Kenya. X-axis: date of measurement, Y-axis: time of measurements; colour – various QC flags. The example of Durban (Figure 5.1 top) shows relatively low occurrence of issues: for short periods we have identified an inconsistency between GHI, DNI and DIF component (red colour); for a short period in June 2013 we identified missing data and non-systematic time shifts. The example for Nairobi (Figure 5.1 bottom) shows several longer periods of missing data, or invalid data excluded by visual check (blue). Quality control shows that all measured solar radiation data are affected by disturbances. The most typical errors are: missing values, inconsistency between the solar components, and occurrence of values out of the page 23 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results physical limits. In many cases shading from surrounding terrain or objects is also observed. These errors were identified, to a various extent, in all stations. Measurements of all three components (GHI, DNI and DIF) allow performing more complex consistency tests, which help to reveal various issues in data that may otherwise remain hidden (Fig 5.2). Important is also visual control of the data that is used to identify systematic issues such shading, reflections or problems with calibration of instruments (Fig 5.3). Figure 5.2: Example of incorrect DNI (DIF) measurements due to problems of sun tracking. Figure 5.3: Example of incorrect DNI measurements due to issues with calibration. Quality-control procedures were used for evaluation of those stations that were pre-qualified for validation of the SolarGIS model. All affected data readings were flagged for these stations and excluded from further analyses. Table 5.1 summarizes percentage of data that passed through the quality control tests. page 24 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results Table 5.1: Data for each solar measuring station that passed through quality control [%] Type of test (numbers show percent of total volume of data)] Sun below Test for physical Visual test Consistency test Total excluded data horizon limits (GHI – DNI – DIF) samples Tamanrasset 49.3 0.0 9.3 1.1 10.4 Bamba 49.2 0.5 0.0 0.0 0.5 Agoufu 49.1 0.05 9.0 0.0 9.1 M Bour 49.7 0.3 12.6 0.0 12.9 Banizoumbou 49.7 0.5 0.0 0.0 0.5 Djougou 49.7 0.05 0.0 0.0 0.05 Nairobi 3.0 0.0 6.1 0.0 6.1 De Aar 48.9 0.1 0.6 5.9 6.6 Sonbesie 49.5 0.0 3.4 0.3 3.7 Durban 50.8 0.0 0.4 0.2 0.6 Graaff-Reinet 50.5 0.0 0.8 0.5 1.3 Port Elizabeth 50.2 0.0 16.1 1.0 17.1 Vanrhynsdorp 50.2 0.0 1.2 0.9 2.1 Based on our experience we propose the following recommendations for running routine measurement campaign: • Data used for model validation must be measured by high quality instruments; detailed description of instruments must be available. Regular maintenance of sensors must be arranged. • Use of just one or two sensors (GHI, DNI), without redundant (DIF) measurements does not allow applying valuable quality control algorithms. • Instruments should be preferably mounted about 1 m to 1.5 m above ground or roof surface on a stable concrete or metal platform. • Many analysed stations are influenced by shading from surrounding structures. This should be avoided if possible, or affected values should be flagged. • Data cleaning should be systematic and logged. • Data should be quality checked on a continuous basis. Some types of data logger have software, which can automatically pre-flag errors. Data should be provided for the end users with flags indicating the above-mentioned problems, to avoid their mistaken use. • Regular service visits will prevent common issues with tracker misalignment, sensor levelling, PV and battery supply, desiccant exchange, etc. page 25 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 5.2 Validation of solar resource model 5.2.1 Comparison of SolarGIS to other models Solar irradiance for Malawi is calculated by the SolarGIS model (Chapter 3.2 of Interim Solar Modelling Report 141-01/2015). In this Chapter, the annual SolarGIS average is compared to five other data sources with different temporal and spatial resolution and time coverage (Table 3.4). Seven representative sites are used, as described in Chapter 6.1 of Interim Solar Modelling Report 141-01/2015). Comparison of the databases shows discrepancies (Tables 5.2 and 5.3), which are determined by their specific characteristics: • Applied model approaches • Type and quality of the input data • Time representation • Spatial and temporal resolution of the output databases. In general, higher uncertainties have to be expected when comparing data representing different time periods due to short-term weather variability and climate cycles, but also due to fluctuating atmospheric conditions (e.g. concentration of aerosols). When comparing with ground observations from the previous decades, one has to consider that these may have been measured with instruments of lower accuracy and under application of less- stringent measuring standards. Table 5.2: Comparison of SolarGIS long-term yearly GHI average with five different data sources. 2 Global Horizontal Irradiation [kWh/m ] Database Karonga Mzuzu Mzimba Chitedze Mangochi Blantyre Nsanje NASA SSE 2071 1958 2115 2045 1980 1947 1972 Meteonorm 7 2272 2018 2081 2072 2093 2009 2058 PVGIS Helioclim 2200 2180 2110 2060 2040 2030 2060 PVGIS CMSAF 2360 1890 2140 2130 2230 2020 2200 NREL 2039 1810 1933 1977 1970 1890 1968 SolarGIS 2215 2000 2125 2061 2125 1984 2032 Standard deviation of GHI annual values 5.5% 6.4% 3.7% 2.4% 4.7% 2.7% 4.1% Schematic assessment of GHI uncertainty (80% confidence) 7.1% 8.2% 4.7% 3.1% 6.1% 3.4% 5.3% Expected SolarGIS uncertainty (80% confidence) 6.0% 6.0% 6.0% 6.0% 6.0% 6.0% 6.0% Table 5.3: Comparison of SolarGIS long-term yearly DNI averages with four different data sources. 2 Direct Normal Irradiation [kWh/m ] Database Karonga Mzuzu Mzimba Chitedze Mangochi Blantyre Nsanje NASA SSE 2104 1899 2206 2107 1983 1936 1994 Meteonorm 7 2284 1823 1959 1981 1959 1870 1887 PVGIS CMSAF 2473 1596 2027 2013 2272 1852 2275 NREL 1702 1221 1546 1650 1610 1442 1640 SolarGIS 1933 1675 1939 1808 1912 1717 1772 Standard deviation of DNI annual values 14.3% 16.1% 12.5% 9.5% 12.1% 11.1% 12.6% Schematic assessment of DNI uncertainty (80% confidence) 18.3% 20.6% 16.0% 12.2% 15.5% 14.3% 16.2% Expected SolarGIS uncertainty (80% confidence) 12.0% 12.0% 12.0% 12.0% 12.0% 12.0% 12.0% page 26 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results Tables 5.2 and 5.3 show dispersion of yearly GHI and DNI values between six different databases, including SolarGIS. This comparative approach is a simplified way how to assess the solar resource uncertainty. Important is to understand the risk of possible great geographical dispersion of the estimates. For comparison we show an uncertainty estimate for SolarGIS based on the available validation data in the region (more in Chapters 5.2.2 and 7.1). The modern satellite-based models, such as SolarGIS, are more accurate as they are based on modern algorithms, which generate data outputs at high spatial and temporal resolution. However such data should be systematically updated and quality controlled, in order to fulfil all needs of the solar energy industry: during the prefeasibility project stage, design optimisation and financing, as well as for operation and management of solar power plants. In this context, high quality ground measurements play key role for validation and adaptation of satellite-based models: this is planned in next step for Phase 2 of this project. Based on the analysis of sites from Europe, North Africa and Middle East, two intercomparison studies [1, 2] independently analyse the accuracy of satellite-based models, besides SolarGIS, some of them available also for Malawi: 3Tier, SOLEMI, HelioClim-3 and IrSOLaV. Tables 5.4 to 5.7 show that SolarGIS has very good performance in all statistical indicators for both Global Horizontal Irradiation and Direct Normal Irradiation. Occasionally, some databases show slightly lower Mean Bias, but this indicator may hide problems (high bias) in individual sites, which may compensate each other in the final figure. From a user’s perspective important parameter is Standard Deviation of biases that indicates geographical stability of the model. Full comparison can be found in both studies. Table 5.4: GHI quality indicators related to satellite-based solar radiation models [2] Global Horizontal Irradiance, GHI Mean bias Standard deviation Standard deviation 2 [W/m ] [%] of biases [%] of hourly values [%] SolarGIS 3 1 2.7 17 HelioClim v3 4 1 5.3 21 3Tier 4 1 3.4 21 IrSOLaV 1 0 4.0 33 Table 5.5: DNI quality indicators related to satellite-based solar radiation models [2] Direct Normal Irradiance, DNI Mean bias Standard deviation Standard deviation 2 [W/m ] [%] of biases [%] of hourly values [%] SolarGIS -11 -4 5.9 35 HelioClim v3 25 8 16.1 50 3Tier 17 5 12.1 49 IrSOLaV -3 -1 - 54 Table 5.6: GHI quality indicators related to satellite-based solar radiation models [1] Global Horizontal Irradiance, GHI Mean bias Standard deviation Standard deviation [%] [%] of biases [%] of hourly values [%] SolarGIS 0 0 2.1 17 HelioClim v3 5 1 5.1 20 SOLEMI (Aerocom aerosols) 6 2 4.8 23 IrSOLaV 2 1 4.2 24 Table 5.7: DNI quality indicators related to satellite-based solar radiation models [1] Direct Normal Irradiance, DNI Mean bias Standard deviation Standard deviation 2 [W/m ] [%] of biases [%] of hourly values[%] SolarGIS -6 -2 5.9 34 HelioClim v3 21 6 13.9 47 SOLEMI (Aerocom aerosols) -40 -11 14.5 49 IrSOLaV -1 0 12.0 49 page 27 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 5.2.2 Validation at sites with high-quality GHI and DNI measurements Compared to high-quality ground measurements (Tables 5.8 and 5.9) that passed through quality control (Chapter 5.1), the SolarGIS model slightly underestimates DNI in Southern Africa. The bias for GHI is in general lower, but more variable between the sites. At the level of individual sites, bias (systematic deviation) of the model values is found in a narrow range (typically within ±10% for yearly DNI and ±4% for yearly GHI), thus it correspond to the expected uncertainty of the SolarGIS model [21]. However due to absence of high-quality measurements in tropical Africa, our proposed model uncertainty is rather conservative (see Chapter 7.1). Terms Bias and RMSD are explained in Glossary. Absolute values of bias are calculated for daytime hours only. Prior to data comparison all data were harmonized into hourly time step. Number of data pairs in Tables 5.8 and 5.9 represent all valid hourly daytime data samples from which statistical measures were calculated. Table 5.8: Global Horizontal Irradiance: bias and RMSD for validation sites Site name Country Bias Root Mean Square Deviation KSI Data (RMSD) pairs Hourly Daily Monthly 2 [W/m ] [%] [%] [%] [%] [-] Tamanrasset Algeria 0 0.0 8.5 4.6 1.8 10 216 M Bour Senegal 1.9 11.2 6.4 3.3 3 293 Bamba Mali -2.2 12.0 7.7 5.1 4 403 Agoufu Mali -1.0 10.9 6.1 2.9 3 660 Banizoumbou Niger -1.8 12.3 7.5 4.8 4 360 Djougou Benin 2.7 16.8 9.6 5.4 4 330 Nairobi Kenya 11 2.0 18.0 7.3 3.5 48 5 835 De Aar South Africa 8 1.8 11.5 6.9 2.5 2 729 Durban South Africa -13 -3.3 17.4 9.1 4.6 70 5 756 Sonbesie South Africa -6 -1.2 10.5 4.8 2.2 49 13 981 Vanrhynsdorp South Africa -4 -0.8 8.8 3.5 1.3 23 3 279 Port Elizabeth South Africa -9 -2.3 11.8 5.8 3.7 21 4 484 Graaff-Reinet South Africa -1 -0.3 11.6 4.9 0.9 11 2 975 Table 5.9: Direct Normal Irradiance: bias and RMSD for validation sites Site name Bias Root Mean Square Deviation KSI Data (RMSD) pairs Hourly Daily Monthly 2 [kWh/m ] [%] [%] [%] [%] [-] Tamanrasset Algeria 24 3.9 21.6 16.4 5.6 10 216 De Aar South Africa -6 -1.0 16.8 9.9 2.4 2 729 Aggeneys South Africa -36 -4.9 16.1 10.7 6.3 203 5 743 Durban South Africa -41 -9.7 30.2 18.1 10.2 187 5 756 Port Elizabeth South Africa -9 -2.0 23.5 13.7 5.5 75 4 484 Sonbesie South Africa -23 -4.2 24.3 17.8 6.3 292 13 981 Vanrhynsdorp South Africa -29 -4.6 18.7 12.8 5.7 114 3 279 Graaff-Reinet South Africa -32 -5.5 21.2 11.6 6.2 79 2 975 Further information about the data and methodology and detailed analysis of uncertainty can be consulted in [22, 23]. Comparison of validation statistics computed for solar meteorological sites in Africa with similar geographical conditions shows stability of the SolarGIS model outputs, and provides confidence about the estimated uncertainty of GHI and DNI. page 28 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 6 VALIDATION OF METEOROLOGICAL DATA 6.1 Validation sites The validation procedure was carried out by comparison of meteorological-model data with ground-measured data at the selected meteorological stations in the region. The validation data come from three different databases: • DCCMS: Department of Climate Change and Meteorological Services of Malawi • NOAA NCDC: National Climatic Data Center • SASSCAL: Southern African Science Service Centre for Climate Change and Adaptive Land Management Comparison is performed for various periods of time for different data sources: • 6 years of data for NCDC (2008 to 2010 for CFSR model and 2011 to 2014 for CFSv2 model) • 1 year of data for DCCMS and SASSCAL networks. For details on the applied meteorological models please refer to Chapter 4.2 in Interim Solar Modelling Report 141-01/2015. Position of the meteorological stations is shown in Table 3.5 and Figure 3.3. 6.2 Air temperature at 2 metres Air temperature is derived from both meteorological models by postprocessing and disaggregation from the original model resolution to 1-km grid (Table 6.1). Considering spatial and time interpolation, the deviation of the model values compared to ground observations is acceptable. Variability of the modelled data matches the variability recorded by the ground measurements. However, it is to be noted that the model data represent larger area, they are smoothed and therefore they are not capable to represent exact values of the local microclimate (as measured on a meteorological station). Table 6.1: Air temperature at 2 m: accuracy indicators of the meteorological model [ºC]. CFSR model (2008 to 2010) CFSv2 model (2011 to 2014)* Bias Bias Bias RMSD RMSD RMSD Bias Bias Bias RMSD RMSD RMSD mean min max hourly daily monthly mean min max hourly daily monthly Lilongwe -0.3 -0.5 -0.2 1.9 1.5 0.5 -1.0 -1.4 -1.0 2.4 1.8 1.2 Chileka -0.2 -0.7 0.1 2.5 2.1 0.5 -1.2 -2.0 -0.6 2.8 2.1 1.5 Lusaka 0.7 2.3 -0.1 4.4 2.7 1.4 -1.3 -1.2 -0.9 2.5 1.7 1.3 Harare -0.5 -1.5 1.0 2.6 1.3 0.7 -1.3 -2.1 0.1 2.8 1.7 1.4 Songea -1.0 -1.6 -0.6 2.4 1.8 1.1 -2.9 -3.1 -2.8 3.5 3.2 2.9 Mbeya -0.5 0.6 -0.8 2.3 1.4 0.9 -1.5 0.1 -2.0 2.7 1.9 1.6 Chimoio -0.4 -2.0 0.9 3.0 1.5 0.6 -1.3 -2.9 -0.3 3.2 2.3 1.5 Kabwe - - - - - - -1.0 0.2 -1.8 2.7 1.9 1.2 Samfya - - - - - - -1.8 -1.3 -1.7 2.5 2.1 2.0 Chitedze - - - - - - -1.1 -0.8 -0.9 2.2 1.4 1.2 Chichiri - - - - - - -1.7 -2.2 -0.8 2.8 2.3 1.9 Mangochi - - - - - - -3.0 -3.4 -1.9 3.6 3.1 3.0 Mimosa - - - - - - -1.7 -1.3 -2.2 3.0 2.2 1.9 Nkhotakota - - - - - - -2.3 -1.0 -3.5 3.0 2.6 2.4 * Time period for SASSCAL, DCCMS and NOAA networks is shown in Table 3.5 page 29 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results The difference in yearly average maximum temperature between modelled and measured data is relatively small. For the minimum night temperature the meteorological models show bias -3.4ºC (Mangochi station in Mulanje mountains) in the worst case (model temperature is lower than ground measurement). The daytime temperature is represented with better accuracy except one station (Nkhotakota). It can be observed that in general the accuracy of the model air temperature in the region is varies. Both daily and seasonal variability is well represented. Figure 6.1 shows the data fit for the Lilongwe airport meteorological station. Figure 6.1: Scatterplots of air temperature at 2 m at the Lilongwe airport meteorological station. Measured values (horizontal axis) and meteorological model values (vertical axis) 6.3 Relative humidity Relative humidity for a period 1999 to 2014 is calculated from the specific humidity, air pressure and air temperature. Original time resolution is 1-hour. The indirect calculation of relative humidity from the meteorological models may result in higher deviation, especially for the night values with high relative humidity. The validation results are summarized in Table 6.2. Similarly to the case of air temperature, relative humidity exhibits higher bias for average maximum values (night time), as it is a temperature-dependent meteorological variable. Daytime values are represented with better accuracy. Accuracy of the modelled data is relatively stable throughout the year. Figure 6.2 shows the data fit for the Lilongwe airport meteorological station. page 30 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results Table 6.2: Relative humidity: accuracy indicators of the model outputs [%]. CFSR model (2008 to 2010) CFSv2 model (2011 to 2014)* Bias Bias Bias RMSD RMSD RMSD Bias Bias Bias RMSD RMSD RMSD mean min max hourly daily monthly mean min max hourly daily monthly Lilongwe -1 0 -2 11 9 4 2 3 0 12 9 5 Chileka -4 -5 -3 14 13 5 3 3 5 13 10 6 Lusaka -13 -5 -13 20 16 14 -8 -4 -9 14 11 9 Harare -2 -4 1 13 8 4 0 -1 0 12 7 4 Songea 3 3 3 12 9 5 11 11 9 15 14 12 Mbeya 0 3 -3 14 10 7 3 7 -2 12 8 6 Chimoio -10 -11 -8 16 13 11 -5 -4 -4 14 10 6 Kabwe - - - - - - -1 4 -8 14 11 7 Samfya - - - - - - -2 -1 -3 10 7 5 Chitedze - - - - - - 5 5 5 11 7 6 Chichiri - - - - - - 7 8 3 13 10 8 Mangochi - - - - - - 8 9 6 14 12 10 Mimosa - - - - - - 6 11 1 13 10 7 Nkhotakota - - - - - - 9 17 2 14 12 11 * Time period for SASSCAL, DCCMS and NOAA networks is shown in Table 3.5 Figure 6.2: Scatterplots of relative humidity at 2 m at the Lilongwe airport meteorological station. Measured values (horizontal axis) and meteorological model values (vertical axis) page 31 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 6.4 Wind speed Wind speed for the period 1999 to 2014 is calculated from the CFSR and CFSv2 models, from 10-metre wind u- and v- components with the original 1 hourly time step resolution. The results of comparison of modelled wind speed with on-site ground measurements are summarized in Table 6.3 and Figure 6.4. Table 6.3: Wind speed: accuracy indicators of the model outputs [m/s]. CFSR model (2008 to 2010) CFSv2 model (2011 to 2014)* Bias Bias Bias RMSD RMSD RMSD Bias Bias Bias RMSD RMSD RMSD mean min max hourly daily monthly mean min max hourly daily monthly Lilongwe -1.3 -0.9 -1.9 2.2 2.0 1.4 -1.6 -0.8 -2.7 2.3 1.9 1.6 Chileka -1.6 -1.3 -1.9 2.2 2.0 1.6 -1.9 -1.1 -3.1 2.4 2.0 1.9 Lusaka 0.9 1.0 -0.1 1.9 1.5 1.2 0.4 0.6 -0.3 1.6 1.2 0.8 Harare -0.5 0.4 -2.0 1.6 1.0 0.6 -0.9 0.0 -2.1 1.6 1.1 0.9 Songea -2.2 -1 -4.0 2.9 2.4 2.3 -2.4 -1.2 -4.2 3.0 2.6 2.4 Mbeya -1.0 0.2 -3.1 2.6 2.0 1.4 -2.0 -0.7 -3.6 2.9 2.4 2.0 Chimoio -0.5 -0.5 -0.6 2.0 1.5 0.6 -1.2 -1.0 -1.4 1.9 1.7 1.2 Kabwe - - - - - - 1.6 1.6 0.8 2.0 1.8 1.7 Samfya - - - - - - -0.6 0.2 -2.1 1.7 0.9 0.7 Chitedze - - - - - - 1.6 1.5 0.3 1.9 1.7 1.6 Chichiri - - - - - - 0.0 0.6 -1.3 1.3 0.6 0.2 Mangochi - - - - - - 0.1 0.8 -2.1 1.3 0.5 0.2 Mimosa - - - - - - 0.7 0.9 -0.4 1.3 0.9 0.7 Nkhotakota - - - - - - -0.5 0.9 -0.7 10.5 9.1 2.8 * Time period for SASSCAL and NOAA networks is shown in Table 3.5 Wind speed (together with wind direction) parameter is strongly determined by the local microclimate. From the comparison it can be seen that modelled wind speed does not fit well the measured values. Figure 6.3 compares wind speed from the CFSv2 model with the measurements. The model represents regional values for 10 m height. The modelled wind speed deviates from the measured data. Figure 6.3: Comparison of duration curves of wind speed data at the Lilongwe airport meteorological station. CFSR/CFSv2 model versus local measurements (data represent period 2008 to 2014). page 32 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results Figure 6.4: Scatterplots of wind speed at 2 m at Lilongwe airport meteorological station. Measured values (horizontal axis) and meteorological model values (vertical axis) Wind direction (together with wind speed) is represented by wind rose, and this parameter is strongly determined by local microclimate (Figure 6.5). From the comparison it can be seen that modelled wind direction represents in this case well the measured values, deviation is only caused by wind speed. Figure 6.5: Comparison of wind direction derived from the CFSv2 modes (left) with local measurements (right) at the Lilongwe meteorological station (data represent period 01/2008 to 12/2014). The reason why the model wind speed values differ from the measured data is low spatial resolution of the CFSR and CFSv2 meteorological models, which represents only regional effects, while data at micro-scale may differ from the regional scale. Since meteorological model represent larger area, the highest wind speed modelled are not present in the measured data. page 33 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 7 UNCERTAINTY OF THE MODEL ESTIMATES 7.1 Solar resource parameters In Malawi, the uncertainty of DNI and GHI is determined by the combined uncertainty of the SolarGIS model and of the ground measurements [21], more specifically: 1. Parameterization of numerical models integrated in SolarGIS for the specific data inputs and their ability to generate accurate results for various geographical conditions: • Data inputs into SolarGIS model (accuracy of satellite data, aerosols, water vapour and terrain). • Clear-sky model and its capability to properly characterize various states of the atmosphere • Simulation accuracy of the satellite model and cloud transmittance algorithms, being able to properly distinguish different types of surface, clouds, fog, vegetation, occasional flooding, etc. • Diffuse and direct decomposition models 2. Uncertainty of the ground-measurements, which is determined by: • Accuracy of the instruments • Maintenance practices, including sensor cleaning, calibration • Data post-processing and quality control procedures. Statistics, such as bias and RMSD (Chapter 5.2.3) characterize accuracy of SolarGIS model in a given validation points, relative to the ground measurements. The validation results are determined by local geography and by quality and reliability of the ground-measured data. It is to be noted that validation for one single site can provide only indicative information. Consistent understanding of the model performance and uncertainty can only be developed by analysis of several validation sites representing similar geographic conditions. From the user’s perspective, the information about the model uncertainty has probabilistic nature, which can be considered at different confidence levels. Tables 7.1 and 7.2 show expert estimate of the model uncertainty assumed at 80% probability of occurrence (an equivalent to 90% exceedance) of values. Table 7.1: Interim uncertainty of SolarGIS model estimate for GHI, GTI and DNI Yearly uncertainty Monthly uncertainty Global Horizontal Irradiation (GHI) ±6% ±8% Global Tilted Irradiation (GTI) ±7% ±9% Direct Normal Irradiation (DNI) ±12% ±15% Table 7.2: Uncertainty of estimate of yearly solar resources: ground instruments vs. SolarGIS model Best sensors and 2 1 SolarGIS data professional maintenance DNI: Rotating Shadowband Radiometer (RSR) ±3.5% ±12% DNI: First class pyrheliometer ±1% GHI: Rotating Shadowband Radiometer (RSR) ±3.5% ±6% GHI: Secondary standard pyranometer ±2% 1 Range of uncertainty depends on climate, measurements practices and post-processing 2 Depends on the geographical ability of SolarGIS model and input data to reflect the local solar climate page 34 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 7.2 Meteorological data The quality of the modelled meteorological parameters stored in the SolarGIS database was assessed by comparison with ground measurements in the geographic region. Model meteorological data are derived from two different numerical models covering periods from 1994 to 2010 (CFSR model) and 2011 to 2014 (CFSv2). Taking into account the results of the comparison, the uncertainty is estimated in Table 7.3. It was found that the modelled air temperature fits quite well the measured data with occasional larger discrepancies in minimum night-time or maximum day-time temperature. Wind speed and wind direction data from the meteorological model represent larger region and are smoothed in a comparison to the site measurements at a meteorological station. Although modelled wind speed and wind direction usually fit the patterns of the site-measured data, the maximum values are often not present accurately in the modelled data. Table 7.3: Expected uncertainty of modelled meteorological parameters in region. Unit Annual Monthly Hourly <3.0 (night time) Air temperature at 2 m [°C] <1.0 <1.5 <2.0 (day time) <20 (night time) Relative humidity at 2 m [%] < 10 <15 <10 (day time) Average wind speed at 10 m [m/s] <1.5 <2.0 <5.0 page 35 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 8 LIST OF FIGURES Figure 3.1: Position of solar sites in Africa that were used for validation in this report ......................................... 11   Figure 3.2: Position of AERONET stations ........................................................................................................... 14   Figure 3.3: Position of meteorological stations considered for validation of CFSR and CFSv2 model outputs.... 15   Figure 4.1: AERONET sites used for MACC-II model evaluation. ........................................................................ 17   Figure 4.2: Comparison of daily summaries from MACC-II model with 15-min AERONET data. ......................... 18   Figure 4.3: Comparison of Aerosol Optical Depth ................................................................................................ 19   Figure 4.4: Aerosol Optical Depth for Lilongwe from five different AOD databases ............................................. 20   Figure 4.5: Monthly-averaged aerosol maps (AOD 670) derived from the MACC-II database ............................ 21   Figure 4.6: Malawi in the world context – average annual aerosols ..................................................................... 22   Figure 5.1: Quality control of data measured at Durban and Nairobi stations ...................................................... 23   Figure 5.2: Example of incorrect DNI (DIF) measurements due to problems of sun tracking. ............................. 24   Figure 5.3: Example of incorrect DNI measurements due to issues with calibration. ........................................... 24   Figure 6.1: Scatterplots of air temperature at 2 m at the Lilongwe airport meteorological station. ....................... 30   Figure 6.2: Scatterplots of relative humidity at 2 m at the Lilongwe airport meteorological station. ..................... 31   Figure 6.3: Comparison of duration curves of wind speed data at the Lilongwe airport meteorological station. .. 32   Figure 6.4: Scatterplots of wind speed at 2 m at Lilongwe airport meteorological station. ................................... 33   Figure 6.5: Comparison of wind direction derived from the CFSv2 modes (left) .................................................. 33   page 36 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 9 LIST OF TABLES Table 3.1:   Sources of solar resource validation data ........................................................................................ 10   Table 3.2:   Solar measuring stations in Africa, used for validation of SolarGIS ................................................. 11   Table 3.3:   Inventory of solar resource models for Malawi ................................................................................ 12   Table 3.4:   Meteorological stations in the region considered for validation ....................................................... 14   Table 3.5:   Selected meteorological models available in the region. ................................................................. 15   Table 5.1:   Data for each solar measuring station that passed through quality control [%] ............................... 25   Table 5.2:   Comparison of SolarGIS long-term yearly GHI average with five different data sources. ............... 26   Table 5.3:   Comparison of SolarGIS long-term yearly DNI averages with four different data sources. ............. 26   Table 5.4:   GHI quality indicators related to satellite-based solar radiation models [2] ..................................... 27   Table 5.5:   DNI quality indicators related to satellite-based solar radiation models [2]..................................... 27   Table 5.6:   GHI quality indicators related to satellite-based solar radiation models [1] ..................................... 27   Table 5.7:   DNI quality indicators related to satellite-based solar radiation models [1]...................................... 27   Table 5.8:   Global Horizontal Irradiance: bias and RMSD for validation sites ................................................... 28   Table 5.9:   Direct Normal Irradiance: bias and RMSD for validation sites ......................................................... 28   Table 6.1:   Air temperature at 2 m: accuracy indicators of the meteorological model [ºC]. ............................... 29   Table 6.2:   Relative humidity: accuracy indicators of the model outputs [%]. .................................................... 31   Table 6.3:   Wind speed: accuracy indicators of the model outputs [m/s]. .......................................................... 32   Table 7.1:   Interim uncertainty of SolarGIS model estimate for GHI, GTI and DNI ............................................ 34   Table 7.2:   Uncertainty of estimate of yearly solar resources: ground instruments vs. SolarGIS model ........... 34   Table 7.3:   Expected uncertainty of modelled meteorological parameters in region. ........................................ 35   page 37 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 10 REFERENCES [1] Ineichen P., 2014. Long Term Satellite Global, Beam and Diffuse Irradiance Validation. Energy Procedia, 48, 1586–1596. [2] Ineichen P. Five satellite products deriving beam and global irradiance validation on data from 23 ground stations, university of Geneva/IEA SHC Task 36, 2011: http://www.unige.ch/cuepe/pub/ineichen_valid-sat-2011-report.pdf [3] Meteonorm handbook, Version 6.12, Part II: Theory. Meteotest, 2010 [4] Šúri M., Huld T., Cebecauer T., Dunlop E.D., 2008. Geographic Aspects of Photovoltaics in Europe: Contribution of the PVGIS Web Site. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 1, 1, 34-41. [5] Šúri M., Huld T., Dunlop E.D., Albuisson M., Wald L, 2006. Online data and tools for estimation of solar electricity in Africa: the PVGIS approach. Proceedings of the 21st European Photovoltaic Solar Energy Conference and Exhibition, 4-8 October 2006, Dresden, Germany [6] Surface meteorology and Solar Energy (SSE) release 6.0, Methodology, Version 2.4, 2009. [7] SWERA web site. Monthly and annual average global data at 40 km resolution for Africa, NREL, 2006. [8] Huld T., Müller R., Gambardella A., 2012. A new solar radiation database for estimating PV performance in Europe and Africa, Solar Energy, 86, 6, 1803-1815. [9] Aerosol Robotic Network (AERONET), NASA. http://aeronet.gsfc.nasa.gov/ [10] CFSV2 model. http://www.nco.ncep.noaa.gov/pmb/products/CFSv2/ [11] CSFR data web site http://cfs.ncep.noaa.gov/cfsr/ [12] Morcrette J., Boucher O., Jones L., Salmond D., Bechtold P., Beljaars A., Benedetti A., Bonet A., Kaiser J.W., Razinger M., Schulz M., Serrar S., Simmons A.J., Sofiev M., Suttie M., Tompkins A., Uncht A., GEMS- AER team, 2009. Aerosol analysis and forecast in the ECMWF Integrated Forecast System. Part I: Forward modelling. Journal of Geophysical Research, 114. [13] Benedictow A. et al. 2012. Validation report of the MACC reanalysis of global atmospheric composition: Period 2003-2010, MACC-II Deliverable D_83.1. [14] Cebecauer T., Šúri M., 2012. Correction of Satellite-Derived DNI Time Series Using Locally-Resolved Aerosol Data.. Proceedings of the SolarPACES Conference, Marrakech, Morocco, September 2012. [15] Gueymard C., Thevenard D., 2009. Monthly average clear-sky broadband irradiance database for worldwide solar heat gain and building cooling load calculations, Solar Energy, 83, 1998-2018. [16] NASA Multiangle Imaging Spectroradiometer (MISR). https://www-misr.jpl.nasa.gov/ [17] NASA Moderate Resolution Imaging Spectroradiometer (MODIS). http://modis.gsfc.nasa.gov/ [18] NASA Goddard Earth Sciences, Data and Information Services Center (GES DISC), Giovanni - Interactive Visualization and Analysis. http://disc.sci.gsfc.nasa.gov/giovanni [19] NREL, 1993. User’s Manual for SERI_QC Software-Assessing the Quality of Solar Radiation Data. NREL/TP-463-5608. Golden, CO: National Renewable Energy Laboratory. [20] Younes S., Claywell R. and Muneer T, 2005. Quality control of solar radiation data: Present status and proposed new approaches. Solar Energy 30, 1533-1549. [21] Šúri M., Cebecauer T., 2014. Satellite-based solar resource data: Model validation statistics versus user’s uncertainty. ASES SOLAR 2014 Conference, San Francisco, 7-9 July 2014. [22] Perez R., Cebecauer T., Suri M., 2014. Semi-Empirical Satellite Models. In Kleissl J. (ed.) Solar Energy Forecasting and Resource Assessment. Academic press. [23] Cebecauer T., Šúri M., Perez R., High performance MSG satellite model for operational solar energy applications. ASES National Solar Conference, Phoenix, USA, 2010. page 38 of 40 World Bank, Global ESMAP Initiative, Renewable Energy Resource Mapping: Solar − Malawi (Project ID: P151289) Model Validation Report – Preliminary results 11 BACKGROUND ON GEOMODEL SOLAR Primary business of GeoModel Solar is in providing support to the site qualification, planning, financing and operation of solar energy systems. We are committed to increase efficiency and reliability of solar technology by expert consultancy and access to our databases and customer-oriented services. The Company builds on 25 years of expertise in geoinformatics and environmental modelling, and 15 years in solar energy and photovoltaics. We strive for development and operation of new generation high-resolution quality-assessed global databases with focus on solar resource and energy-related weather parameters. We are developing simulation, management and control tools, map products, and services for fast access to high quality information needed for system planning, performance assessment, forecasting and management of distributed power generation. Members of the team have long-term experience in R&D and are active in the activities of International Energy Agency, Solar Heating and Cooling Program, Task 46 Solar Resource Assessment and Forecasting. ® GeoModel Solar operates a set of online services, integrated within SolarGIS information system, which includes data, maps, software, and geoinformation services for solar energy. http://geomodelsolar.eu http://solargis.info GeoModel Solar is ISO 9001:2008 certified company for quality management since 2011 page 39 of 40