98032 ESMAP Solar Resource Mapping for Zambia Interim Model Validation Report Renewable Energy Resource Mapping and Geospatial Planning – Zambia [P145271] November 2014 GeoModel Solar, Pionierska 15, 831 02 Bratislava, Slovakia http://geomodelsolar.eu Reference No. (GeoModel Solar): 128-02/2014 TABLE OF CONTENTS Table of contents .................................................................................................................................................... 4 Acronyms ................................................................................................................................................................ 5 Glossary .................................................................................................................................................................. 6 1 Summary ...................................................................................................................................................... 7 2 Model quality indicators .............................................................................................................................. 8 3 Inventory of solar atmospheric and meteorological validation data ....................................................... 9 3.1 Solar resource measurements .............................................................................................................. 9 3.2 Solar resource modelled data ............................................................................................................. 11 3.3 Atmospheric data ................................................................................................................................ 12 3.4 Meteorological measurements ............................................................................................................ 13 3.5 Meteorological models ........................................................................................................................ 14 4 Validation of Aerosol data......................................................................................................................... 16 4.1 Evaluation of MACC-II Atmospheric Optical Depth data ..................................................................... 16 4.2 Seasonal variability of Atmospheric Optical Depth .............................................................................. 19 5 Validation of solar resource data ............................................................................................................. 21 5.1 Quality control of solar validation data................................................................................................. 21 5.2 Validation of solar resource model ...................................................................................................... 25 6 Validation of meteorological data ............................................................................................................ 28 6.1 Validation sites .................................................................................................................................... 28 6.2 Air temperature at 2 metres ................................................................................................................. 28 6.3 Relative humidity ................................................................................................................................. 29 6.4 Wind speed ......................................................................................................................................... 31 7 Uncertainty of the model estimates ......................................................................................................... 33 7.1 Solar resource parameters .................................................................................................................. 33 7.2 Meteorological data ............................................................................................................................. 34 8 List of figures ............................................................................................................................................. 35 9 List of tables............................................................................................................................................... 36 10 References ................................................................................................................................................. 37 11 Background on GeoModel Solar .............................................................................................................. 38 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report ACRONYMS AERONET The AERONET (AErosol RObotic NETwork) is a ground-based remote sensing network dedicated to measure atmospheric aerosol properties. It provides a long-term database of aerosol optical, microphysical and radiative parameters. AOD 670 Aerosol Optical Depth at 670 nm. This is one of atmospheric parameters derived from MACC-II database and used in SolarGIS. It has important impact on accuracy of solar calculations in arid zones. CFS v2 Climate Forecast System. The meteorological model operated by the US service NOAA (National Oceanic and Atmospheric Administration) CFSR Climate Forecast System Reanalysis. The meteorological model operated by the US service NOAA. DIF Diffuse Horizontal Irradiation, if integrated solar energy is assumed. Diffuse Horizontal Irradiance, if solar power values are discussed. DNI Direct Normal Irradiation, if integrated solar energy is assumed. Direct Normal Irradiance, if solar power values are discussed. ECMWF European Centre for Medium-Range Weather Forecasts is independent intergovernmental organisation supported by 34 states, which provide operational medium- and extended-range global forecasts and a computing facility for scientific research. GFS Global Forecast System. The meteorological model operated by the US service NOAA. GHI Global Horizontal Irradiation, if integrated solar energy is assumed. Global Horizontal Irradiance, if solar power values are discussed. GTI Global Tilted (in-plane) Irradiation, if integrated solar energy is assumed. Global Tilted Irradiance, if solar power values are discussed. MACC Monitoring Atmospheric Composition and Climate – meteorological model operated by the European service ECMWF (European Centre for Medium-Range Weather Forecasts) Meteosat MFG and MSG Meteosat satellites operated by EUMETSAT organization. MSG: Meteosat Second Generation; MFG: Meteosat First Generation NOAA NCEP National Oceanic and Atmospheric Administration, National Centre for Environmental Prediction RSR Rotating Shadowband Radiometer TEMP Air Temperature at 2 metres page 5 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report GLOSSARY Aerosols Small solid or liquid particles suspended in air, for example soil particles, sea salts, pollen or air pollution such as smog or smoke. Bias Represents systematic deviation (over- or underestimation) and it is determined by systematic or seasonal issues in cloud identification algorithms, coarse resolution and regional imperfections of atmospheric data (aerosols, water vapour), terrain, sun position, satellite viewing angle, microclimate effects, high mountains, etc. Root Mean Square Represents spread of deviations given by random discrepancies between measured and Deviation (RMSD) modelled data and is calculated according to this formula: ∑𝑛 𝑘 𝑘 𝑘=1(𝑋 𝑚𝑒𝑎𝑠𝑢𝑟𝑒𝑑 − 𝑋𝑚𝑜𝑑𝑒𝑙𝑒𝑑 ) 2 𝑅𝑀𝑆𝐷 = √ 𝑛 On the modelling side, this could be low accuracy of cloud estimate (e.g. intermediate clouds), under/over estimation of atmospheric input data, terrain, microclimate and other effects, which are not captured by the model. Part of this discrepancy is natural - as satellite monitors large area (of approx. 3 x 4 km), while sensor sees only micro area of approx. 1 sq. centimetre. On the measurement side, the discrepancy may be determined by accuracy/quality and errors of the instrument, pollution of the detector, misalignment, data loggers, insufficient quality control, etc. Solar irradiance Solar power (instantaneous energy) falling on a unit area per unit time [W/m 2]. Solar resource or solar radiation is used when considering both irradiance and irradiation. Solar irradiation Amount of solar energy falling on a unit area over a stated time interval [Wh/m 2 or kWh/m2]. Spatial grid In digital cartography the term applies to the minimum size of the grid cell or in the other words resolution minimal size of the pixels in the digital map Uncertainty Is a parameter characterizing the possible dispersion of the values attributed to an estimated irradiance/irradiation values. In this report, uncertainty assessment of the solar resource estimate is based on a detailed understanding of the achievable accuracy of the solar radiation model and its data inputs (satellite, atmospheric and other data), which is confronted by an extensive data validation experience. The second important source of uncertainty information is the understanding of quality issues of ground measuring instruments and methods, as well as the methods correlating the ground-measured and satellite-based data. In this report, the range of uncertainty assumes 80% probability of occurrence of values. Thus, the lower boundary (negative value) of uncertainty represents 90% probability of exceedance, and it is also used for calculating the P90 value. Water vapour Water in the gaseous state. Atmospheric water vapour is the absolute amount of water dissolved in air. page 6 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 1 SUMMARY Background This Model Validation Report shows method and results of preliminary validation of solar resource and meteorological data for the Republic of Zambia, Phase 1 of solar resource mapping and measurement services. The project is a part of technical assistance in renewable energy development implemented by the World Bank in Zambia. It is being undertaken in close coordination with the Ministry of Energy and Water Development of Zambia, Department of Energy, the World Bank’s primary country counterpart for this project. The project is funded by the Energy Sector Management Assistance Program (ESMAP), a global knowledge and technical assistance program administered by the World Bank and supported by 11 bilateral donors. It is part of a major ESMAP initiative in support of renewable energy resource mapping and geospatial planning across multiple countries. Data and methods This report documents validation of solar resources calculated by satellite model SolarGIS, and meteorological data derived from the CFSR and CFSv2 models. Inventory in Chapter 3 identifies the existing data sources in the region: solar, aerosol and meteorological data. First, aerosol (Atmospheric Optical Depth, AOD) data from the MACC-II model is evaluated (Chapter 4, this data is used on the input to SolarGIS clear-sky model). Chapter 5 shows relative comparison of SolarGIS GHI and DNI to other modelled databases. Next, validation to high- quality solar resource measurements demonstrates stable performance of SolarGIS in geographic conditions, similar to Zambia. Chapter 6 validates meteorological parameters that are used for site-specific data and for maps. Chapter 7 summarizes validation results in the estimate of uncertainty. Results Validation demonstrates stable performance of SolarGIS model in Africa. The validation and previous experience indicate that using high-quality local measurements, the SolarGIS model output have significant potential for reduction of uncertainty in geography of tropical Africa. page 7 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 2 MODEL QUALITY INDICATORS The performance of satellite-based models, for a given site, is characterized by the following indicators: 1. Bias characterizes systematic model deviation at a given site; 2. Root Mean Square Deviation (RMSD), Standard deviation (SD) and Mean Average Deviation (MAD), which indicate spread of error for instantaneous values (typically hourly or sub-hourly); 3. Kolmogorov-Smirnoff index (KSI) characterizes representativeness of distribution of values. This indicator is applied only for solar resource data. Focus of this report is validation and uncertainty assessment of SolarGIS solar resource data that are derived in the form of spatial and site-specific data products. Meteorological data are also validated as they are used in the site-specific times series and Typical Meteorological Year data (TMY). Air temperature is also used as a spatial data product. Only quality-controlled measurements from high-standard sensors can be used for objective validation of satellite-based solar model, as issues in the ground measured data result in skewed evaluation. Typically, bias is considered as the first indicator of the model accuracy, however the model should be interpreted analyzing also all other accuracy measures. While knowing bias helps to understand a possible error of longer-term estimate, MAD and RMSD are important for estimating the accuracy of energy simulation and operational calculations (monitoring, forecasting). KSI reveals issues in the model’s ability to represent specific solar radiation conditions. This is especially important in the CSP modelling, as the response of these systems is non-linear to irradiance levels. Even if bias of different satellite-based models is similar, other accuracy characteristics (RMSD, MAD and KSI) may indicate substantial differences in their performance. Validation statistics for one site may not provide representative picture of the model performance in the given geographical conditions. The reason is that one particular site may be affected by a local microclimate or by hidden issues in the ground-measured data. Therefore, the model should be evaluated at several validation sites. If ability of the model to estimate longterm values, at least two measures are to be considered [1]:  Mean bias deviation, which indicates whether the model has overall tendency to overestimate or to underestimate the measured values.  Standard deviation of biases, which shows the range of deviation of the model estimates (statistically one standard deviation characterizes 68% probability of occurrence). Good satellite models are consistent in space and time, and thus the validation at several sites within one geography provides a robust indication of the model accuracy in geographically comparable regions elsewhere. Besides bias and RMSD, the ability of the model to simulate representatively sub-hourly values for all conditions (especially high and low light conditions) is very important for optimisation of the solar power plants. Two evaluation studies have been conducted independently by University of Geneva [1, 2]. Both studies analyse features of existing solar radiation models based on processing of satellite data. The studies show that SolarGIS model demonstrates robust and harmonized performance in all indicators. page 8 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 3 INVENTORY OF SOLAR ATMOSPHERIC AND METEOROLOGICAL VALIDATION DATA 3.1 Solar resource measurements Public data Solar radiation, unlike other basic meteorological parameters, is measured only at few meteorological stations in Africa. Solar measurements are collected by various organizations: by international or regional professional networks, meteorological agencies or universities. Access to these data may be restricted by data usage polices. Inventory shows that only few sources provide data with sufficient quality required for the validation of SolarGIS model (Tab 3.1) in geographic conditions comparable to those of Zambia. Tab. 3.1: Sources of solar resource validation data Network Description AMMA African Monsoon Multidisciplinary Analyses. AMMA is an international project to improve the knowledge and understanding of the West African Monsoon (WAM) and its variability. http://amma-international.org/ BSRN Baseline Surface Radiation Network (BSRN) provides near-continuous, long-term, in situ-observed broadband irradiances (solar and thermal infrared) and certain related parameters from a network of more than 50 globally diverse sites. Data usually include GHI, DIF and DNI measurements. Data from De Aar and Tamanrasset meteo stations are used. http://www.bsrn.awi.de/ http://www.bsrn.awi.de/en/data/data_retrieval_via_pangaea/ SASSCAL Southern African Science Service Centre for Climate Change and Adaptive Land Management in Southern Africa conducts a problem-oriented research in the area of adaptation to climate change and sustainable land management. Solar radiation measurements are part of the project monitoring activities. http://www.sasscal.org/ http://www.sasscalweathernet.org/ SAURAN SAURAN (Southern African Universities Radiometric Network) is an initiative of the Centre for Renewable and Sustainable Energy Studies (CRSES) at Stellenbosch University and the Group for Solar Energy Thermodynamics (GSET) at the University of KwaZulu-Natal. The network provides high-resolution, ground- based solar radiometric data available from stations located across the Southern African region, including South Africa, Namibia, Botswana and Reunion Island. In this report, four SAURAN data sets were used. http://www.sauran.net/ STERG Sonbesie is a station operated by STERG (Solar Thermal Energy Research Group), a research group housed in the Department of Mechanical and Mechatronic Engineering in Stellenbosch University and affiliated with the Centre for Renewable and Sustainable Energy Studies, Stellenbosch University. STERG has performed measurements on one site considered in this report. http://weather.sun.ac.za/ Before decision was made if the ground-measurements to be used for the model validation, the data was quality controlled (Chapter 5.1). In general, measurements from networks such as AMMA, BSRN, SAURAN and Sonbesie show quality that allows using them for the model validation. BSRN, SAURAN and Sonbesie produce data in 1 and 10 minute time step, AMMA data are in available hourly time step only. SASSCAL meteorological network is not focused primarily on high-accuracy solar radiation measurements. The instruments have high uncertainty and our analysis identified higher occurrence of issues, which excluded these measurements from further consideration. List of public solar resource measuring stations, checked for possible use for validation of SolarGIS model (Chapter 5.2), is summarized in Tabs. 3.2 and 3.3. Their position is shown in Fig. 3.1. page 9 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Fig. 3.1: Position of solar validation sites in Africa Tab. 3.2: Solar measuring stations in Africa, used for validation of SolarGIS Site name Country Source Latitude Longitude Altitude GHI DNI Period [°] [°] [m a.s.l.] Tamanrasset Algeria BSRN 22.7833 5.5137 1378 YES YES 03/2000 – 12/2006 Bamba Mali AMMA 17.0990 -1.4018 272 YES NO 2006 Agoufu Mali AMMA 15.3445 -1.4791 290 YES NO 2006 M Bour Senegal AMMA 14.3940 -16.9590 5 YES NO 2006 Banizoumbou Niger AMMA 13.5311 2.6613 211 YES NO 2006 Djougou Benin AMMA 9.6920 1.6620 438 YES NO 2006 De Aar South Africa BSRN -30.6667 24.0000 1331 YES YES 06/2000 – 12/2004 Durban South Africa SAURAN -29.8710 30.9769 150 YES YES 02/2013 – 07/2014 Graaff-Reinet South Africa SAURAN -32.4855 24.5858 660 YES YES 12/2013 – 09/2014 Port Elizabeth South Africa SAURAN -34.0086 25.6653 35 YES YES 12/2012 – 07/2014 Sonbesie South Africa STERG -33.9282 18.8651 144 YES YES 02/2012 – 07/2014 Vanrhynsdorp South Africa SAURAN -31.6175 18.7383 130 YES YES 11/2013 – 09/2014 page 10 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Tab. 3.3: Solar measuring stations operated by SASSCAL network in Zambia Site name Country Source Latitude Longitude Altitude GHI DNI Period [°] [°] [m a.s.l.] Samfya Zambia SASSCAL -11.3712 29.5606 1194 YES NO 10/2013 – 10/2013 Mwinilunga Zambia SASSCAL -11.7400 24.4310 1360 YES NO 12/2103 – 06/2014 Serenje Zambia SASSCAL -13.2267 30.2151 1395 YES NO 12/2013 – 10/2014 Kasempa Zambia SASSCAL -13.4570 25.8337 1227 YES NO 12/2013 – 10/2014 Zambezi Zambia SASSCAL -13.5337 23.1079 1066 YES NO 12/2013 – 10/2014 Kabwe Mulungushi Zambia SASSCAL -14.2926 28.5663 1142 YES NO 01/2013 – 10/2013 Kalabo Zambia SASSCAL -14.9890 22.6818 1018 YES NO 12/2013 – 10/2014 Lusaka Int. Airport Zambia SASSCAL -15.3193 28.4405 1149 YES NO 10/2013 – 10/2014 Lusaka, Uni. Zambia Zambia SASSCAL -15.3912 28.3320 1260 YES NO 10/2013 – 10/2014 Sesheke Zambia SASSCAL -17.4711 24.3013 944 YES NO 12/2013 – 10/2014 Private initiatives A number of solar measuring stations are deployed by companies active in development of solar energy projects in the region. The measured data are used for commercial and technological assessment of solar resource for particular projects and they are not publically available. 3.2 Solar resource modelled data Public databases There are several modelled databases available in the region (Tab. 3.4). In general, the databases based on the interpolation of ground-measured data, such as Meteonorm [3] are less reliable in areas with sparse availability of meteorological stations. PVGIS HelioClim-1 [4, 5] is calculated from daily HelioClim-1 data with limited reliability. The global database NASA SSE [6] is computed by empirical models from satellite and atmospheric data and shows only global climate patterns at coarse-resolution. SWERA/NREL database has medium spatial resolution and is computed using CSR model by NREL [7], thus only showing overview perspective. The closest to SolarGIS is satellite-based database PVGIS CMSAF, however the data are not updated regularly and are available only as long-term averages [8]. Implementation of these databases is static, and they are not updated regularly. Tab. 3.4: Inventory of solar resource models for Zambia. Model Data source Data spatial Parameter Time resolution Period resolution of available data NASA SSE Satellite + model 110 km x 110 km GHI, DNI Long-term monthly 1983 – 2005 Interpolation and Meteonorm 7.1 Ground + satellite GHI, DNI Long-term monthly 1981 – 2010 satellite data PVGIS Daily MFG satellite 30 x 30 km GHI Long-term monthly 1985 – 2004 HelioClim-1 data MFG and MSG Long-term monthly PVGIS CMSAF 3 km x 4 km GHI, DNI 1998 – 2011 satellites (hourly) SWERA/NREL Model 40 km x 40 km GHI, DNI Long-term monthly 1985 – 1991 MSG/MFG PRIME SolarGIS* 3 km x 4 km GHI, DNI 15 and 30 minutes 1994 – 2013* satellites * SolarGIS database is continuously updated on daily basis page 11 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Commercial satellite-based databases On the market there are few solar databases developed and maintained by commercial entities that provide solar radiation data to customers for a fee. Most of these databases are based on the use of satellite data, but they differ in the model implementation and use of input data (e.g. aerosols, water vapour), therefore results may significantly differ. These databases differ also in spatial coverage, spatial and temporal resolution, operational update and other parameters. Besides quality of data, important for a user is easy access, ability of the system to deliver updated data, and support by services, such as site adaptation, derived data products (e.g. TMY), bankable solar resource assessment, map services and others. To our understanding, for Zambia, besides SolarGIS, the following commercial databases are available: SOLEMI, 3TIER, IrSOLaV and HelioClim-3. More information about these databases is available in [1, 2]. 3.3 Atmospheric data Along with the clouds, aerosols are the most influential factor controlling GHI and DNI irradiance in the region, especially during cloud-free weather situations. The atmospheric turbidity is mostly influenced by burning biomass, soil particles, locally by human activities (industry, transport and urbanization) and particles transported from other regions. Complex geography creates specific conditions for local distribution and transport of aerosols. Combined influence of these factors results in varying atmospheric pollution both spatially and temporarily. The accurate description of aerosols is difficult due to several factors:  Aerosols have high spatial and temporal variability,  There is insufficient number of aerosol-specialized meteo stations, and often they have only short period of measurements,  In most of Africa, there are limited possibilities for detailed description of aerosol sources for chemical- transport atmospheric models,  Arid and semiarid conditions make it difficult to use satellite measurements of aerosols,  Dynamics of aerosols increases in a complex terrain. For aerosol characterization a data from chemical transport model MACC-II is used in SolarGIS. The original data with resolution of ca. 85 km and 125 km is post-processed by a) regional adaptation to remove systematic regional deviation of MACC-II database and b) altitude correction to better reflect local terrain conditions. The understanding of nature of the modelled aerosol data helps to indirectly evaluate the satellite based SolarGIS model. For this purpose a model input aerosol data was compared to aerosol measurements from AERONET [9]. Fig. 3.2 shows location of the stations used for atmospheric data validation. Only one station is located in Zambia, though several stations exist in the wider region, having reasonably long period of measurements:  Mongu (Zambia)  Nairobi (Kenya)  Malindi (Kenya)  Mbita (Kenya)  Gorongosa (Mozambique)  Skukuza (South Africa)  Pretoria (South Africa)  Elandsfontein (South Africa)  Wits University (South Africa)  Henties Bay (Namibia) Fit of aerosol data to ground measurements is important indirect indicator of performance of satellite-based model (for SolarGIS evaluated in Chapter 4.2). page 12 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Fig. 3.2: Position of AERONET stations 3.4 Meteorological measurements The validation procedure was carried out by comparison of modelled data with ground-measured data at eight sites operated by Southern African Science Service Centre for Climate Change and Adaptive Land Management (SASSCAL), and also with other six meteo data measured in the region (source NOAA NCDC). It must be noted that time period of data comparison for the SASSCAL network is relatively short (Tab. 3.5). Comparison with other meteo stations in the region is performed for a time period 2008 to 2010 (CFSR model) and for 2011 to 2013 (CFSv2 model) [11, 12]. Tab. 3.5: Meteo stations in the region considered for validation of CFSR and CFSv2 model outputs Meteo station Data source Time period Latitude [º] Longitude [º] Elevation [m a.s.l.] Lusaka intl. airport NOAA 01/2008 – 12/2013 -15.3170 28.4500 1154 Kasane NOAA 01/2008 – 12/2013 -17.8170 25.1500 1000 Harare, Kutsaga NOAA 01/2008 – 12/2013 -17.9170 31.1330 1480 Gweru NOAA 01/2008 – 12/2013 -19.4500 29.8500 1429 Mbeya NOAA 01/2008 – 12/2013 -8.9330 33.4670 1758 Chimoio NOAA 01/2008 – 12/2013 -19.1170 33.4670 732 Sesheke SASSCAL 11/2013 – 10/2014 -17.4711 24.3013 953 Kalabo SASSCAL 12/2013 – 10/2014 -14.9890 22.6818 1030 Kabwe Mulungushi SASSCAL 10/2013 – 10/2014 -14.2926 28.5663 1143 Zambezi SASSCAL 11/2013 – 10/2014 -13.5337 23.1079 1077 Kasempa SASSCAL 11/2013 – 10/2014 -13.4570 25.8337 1240 Serenje SASSCAL 12/2013 – 10/2014 -13.2267 30.2151 1390 Mwinilunga SASSCAL 12/2013 – 10/2014 -11.7400 24.4310 1368 Samfya SASSCAL 10/2013 – 10/2014 -11.3712 29.5606 1197 page 13 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Fig. 3.3: Position of meteo stations considered for validation of CFSR and CFSv2 model outputs 3.5 Meteorological models Tab. 3.6 gives an overview of selected modelled meteorological data available for the region. Chapter 4.2 of Interim Solar Modelling Report 128-01/2014 gives more insight into global meteorological models. These models are run by meteorological organisations, such as US National Oceanic and Atmospheric Administration (NOAA), European Center for Medium range Weather Forecasting (ECMWF) or Canadian Meteorological Centre (CMC). Global meteorological models serve for purposes such as weather forecasting, modelling long-term climate processes and helping to understand weather phenomena in a global scale. Accuracy of modelled meteorological data for a specific geographical location cannot compete with the accuracy of well-maintained on-site meteorological sensors. However advantages of the modelled data are numerous: they cover large territories (some are global), they are free of maintenance and calibration issues, in case of reanalysis product they ensure seasonal and long term stability, long history, almost 100% availability both spatially and temporally and they offer data from any location on the Earth. This makes them a good choice for preliminary solar energy simulations. Tab. 3.6: Selected meteorological models available in the region. Database name Source Spatial resolution Time resolution Period CFSR NOAA, model 0.312° x 0.312° 1 hour 1979 to 2010 CFSv2 NOAA, model 0.20° x 0.20° 1 hour 2011 to present GFS NOAA, model 0.20° x 0.20° 3 to 6 hours 1991 to present ERA-Interim ECMWF, model 0.75°x 0.75° 6 hours 1979 to present GDPS CMC, model 0.225° x 0.225° 3 hours 2010 to present Meteonorm Ground-measurements Interpolation Long-term monthly 2000 to 2009 page 14 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Meteonorm database is also mentioned in Tab. 3.6. It is a different type of weather database, based on ground measurements from a number of (8325) meteorological stations, where site-specific information is calculated by interpolation of monthly averages. Monthly averages are, in the second step statistically disaggregated to synthetic hourly data representing one year. This approach has limitations due to its static character (there is no systematic update) and limited performance in areas with sparse network of meteorological stations. Although this database was historically popular, with today’s computing and modelling options, this approach is overcome. In the delivery for Zambia, the meteorological parameters are derived from CFSR and CFS v2 models. Water vapour parameter - for solar resource model - is partially derived also from the GFS database. page 15 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 4 VALIDATION OF AEROSOL DATA Along with clouds, aerosol data is one of the most important parameters as it controls accuracy of solar models in arid and semiarid zones. Atmospheric aerosols include liquid and solid particles originating from different sources, e.g. soil particles, sea salts, burning biomass, industrial and traffic pollution and pollen. Aerosols have high spatial and temporal variability and complex behaviour in terms of absorption and scattering of solar irradiance. Increased aerosol concentrations in the atmosphere can reduce GHI in the range of 0% to 7%, occasionally up to 10%. In case of DNI, the variable aerosols can reduce daily DNI as much as 40% or even more. In solar modelling, aerosols are represented by the parameter called Atmospheric Optical Depth (AOD). 4.1 Evaluation of MACC-II Atmospheric Optical Depth data MACC-II aerosol data [13, 14], used in the SolarGIS model, provide good representation of temporal as well as spatial variability of aerosol load. Despite these qualities, the data may experience systematic deviation in some regions [15]. Here, we evaluate MACC-II AOD data using measurements from AERONET stations [29] located in a wider region (Fig. 3.2). Only one station (Mongu) can be used for direct evaluation in Zambia, other sites show aerosol accuracy in the wider spatial context. The SolarGIS model is based on aerosol data, which are regionally adapted for larger systematic deviations and they are also adapted for local altitude by empirical height correction. Fig. 4.1 and 4.2 demonstrate an accuracy of the MACC-II database used in the SolarGIS model. Comparison of post-processed daily MACC-II data and 15-minute AERONET ground measurements for Mongu site shows very good fit. Seasonal profiles, as well as short extreme situations, are well represented. But in conditions with very low aerosol load a slight overestimation of MACC-II AOD is found. On the other hand, for some of high load situations, the modelled AOD may be slightly underestimated. The discrepancy visible in the plot also arises from comparing high frequency (15-min) site-specific AERONET values with daily summaries of regionally-smoothed MACC-II data. The comparison of AERONET sites in a broader region shows generally good representation of the AOD variability. At some sites higher bias of MACC-II aerosols (Henties Bay, Elandsfontein, Gorongosa, Malindi) may influence resulting accuracy of SolarGIS DNI and GHI irradiation. These sites are relatively far from Zambia, but may indicate issues of the MACC-II model in Eastern Zambia, which is far from the Mongu site. Fig. 4.1: Comparison of daily summaries from MACC-II model with 15-min AERONET data. Mongu AERONET station in Zambia page 16 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Some discrepancies in individual stations may be attributed to the coarse resolution of MACC-II database, which is not capable representing the specific local conditions recorded in the AERONET data with sufficient accuracy. Also the spread of values may be partially explained by natural differences arising from the comparison of exact point measurements (AERONET) with regionally-averaged values (MACC-II). page 17 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Fig. 4.2: Comparison of Aerosol Optical Depth AERONET 675 nm (x-axis) and MACC 670 nm (y-axis); blue points: original MACC data; red points: MACC data regionally-adapted by SolarGIS method To understand potential issues with AOD in regions, where AERONET data are not available, the MACC-II data for Lusaka were compared with two other data sources (Fig. 4.3):  Data computed from several satellite missions: Terra MODIS, Aqua MODIS, Terra MISR, Envisat MERIS [16, 17]  Data computed by chemical transport model GOCART (NASA, Langley ASCD) [18]. All compared databases (except Envisat MERIS) show the same seasonal pattern with increased aerosol load in a period from August to October – the same trend was identified also in the Mongu AERONET station. Range of values between databases is also similar, only differences in extremely high values are identified. Limited number of ground-measured data does not allow evaluating quality of aerosol data for the whole Zambia. In general, analysis of available AOD data shows good representativeness of MACC-II AOD database in the Mongu site. Discrepancies seen in other sites could be observed in some regions of Zambia. These discrepancies may be reduced or removed in Phase 2 of the ESMAP project, by use of high resolution and high–quality local measurements. page 18 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Aqua MODIS (550nm, monthly) Terra MODIS (550nm, monthly) Terra MISR (555 nm, monthly) Envisat MERIS (550 nm, monthly) GOCART (550 nm, monthly) MACC-II (670nm, daily) Fig. 4.3: Aerosol Optical Depth for Lusaka from six different AOD databases Plots of satellite-based data were produced with by Giovanni online data system, NASA Langley ASCD [19] 4.2 Seasonal variability of Atmospheric Optical Depth SolarGIS uses AOD input data, at the wavelength 670 nm, derived from the MACC-II model. The MACC-II model captures high temporal variability of aerosols, thus it reduces uncertainty of instantaneous GHI and DNI estimates. Fig. 4.4 shows typical monthly variability of aerosols in central and southern Africa. Zambia is located in a transition zone between region with high aerosol load in equatorial and sub-equatorial Africa and regions with low aerosol load in the southern part of continent. The seasonal pattern is influenced by high aerosol load events having their centre outside of the Zambian territory in the Northwest. The lowest aerosol concentration in the atmosphere can be observed from December to June, and the highest in September and October. Highest concentrations are seen in the Western part of the country. From the global perspective, Zambia is a region with relatively low aerosol load (Fig. 4.5), but still having significant influence on the dynamics of solar resource, especially DNI. page 19 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Fig. 4.4: Monthly-averaged aerosol maps (AOD 670) derived from the MACC-II database and adapted for the SolarGIS model. Period 2003 to 2013 Fig. 4.5: Zambia in the world context – average annual aerosols (represented by AOD 670 nm) computed by the MACC-II model and adapted for SolarGIS. Period 2003 to 2012. Values are dimensionless. page 20 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 5 VALIDATION OF SOLAR RESOURCE DATA 5.1 Quality control of solar validation data Prior to comparison with satellite-based solar resource data, the ground-measured irradiance was quality- controlled by GeoModel Solar. Quality control (QC) was based on methods defined in SERI QC procedures and Younes et al. [20, 21] and developed by GeoModel Solar. The ground measurements were inspected also visually, mainly for identification of shading and other regular data error patterns. Fig. 5.1 shows an example of results of such quality control in one meteo station: Durban (South Africa). The colours indicate the following flags:  Blue: data excluded by visual inspection - mainly shading, incorrect tracking and calibration issues  Green: data passed all tests  Grey: sun below horizon  White strips: missing data  Red and violet: GHI, DNI and DIF consistency problem or problems with physical limits Fig. 5.1: Quality control of data measured at Durban measuring station in South Africa X-axis: date of measurement, Y-axis: time of measurements; colour – various QC flags. The example of Durban (Fig. 5.1) shows relatively low occurrence of issues: for short periods we have identified an inconsistency between GHI, DNI and DIF component (red colour); for a short period in June 2013 we identified missing data and non-systematic time shifts. Quality control shows that all measured solar radiation data is affected by disturbances. The most typical errors are: missing values, inconsistency between the solar components, and occurrence of values out of the physical limits. In many sites shading from surrounding terrain or objects is also observed. These errors were identified, to a various extent, in all stations. Measurements of all three components (GHI, DNI and DIF) allow performing more complex consistency tests, which help to reveal various issues in data that may otherwise remain hidden (Fig 5.2). Important is also visual control of the data that is used for identification of systematic issues such shading, reflections or problems with calibration of instruments (Fig 5.3). page 21 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Fig. 5.2: Example of incorrect DNI (DIF) measurements due to problems of sun tracking. Fig. 5.3: Example of incorrect DNI measurements due to issues with calibration. For the territory of Zambia the ground measurements were available for eleven locations from the SASSCAL network. The data are available through the project web page, however no additional information was available: about instruments, calibration, cleaning plan and maintenance log. Therefore it was difficult to assess nominal accuracy of these measurements. The visual inspection of this data revealed issues, such as shading, missing data, incorrect levelling of instruments (Figs. 5.4 and 5.5). The global horizontal irradiation observations from the SASSCAL network show systematic positive bias for satellite-based yearly GHI, from 4% to 12%, which indicates systematic issues with measured data (instrument levelling and sensor soiling). Fig. 5.6 shows an example of how issues in ground measurements can be identified when compared to the model data. Two stations show very distinct behaviour. While data from Sonbesie show good fit, the data from Kasempa show high dispersion of values due to lower quality of ground measurements. The performance of the data from the SASSCAL network is variable and cannot be used for objective validation of satellite model performance. Taking into consideration also fact that these stations are equipped with lower standard sensors, which are not primarily meant for high-accuracy measurements of solar radiation, the data were excluded from further consideration. page 22 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Fig. 5.4: Two examples of data issues identified at meteo stations Top: asymmetric profile and shading, Bottom: incorrect data. Fig. 5.5: Example of missing data Fig. 5.6: Quality of GHI measurements for two stations Left: Sonbesie, Stellenbosch, South Africa is an example of good-quality ground measurements; Right: Kasempa, Zambia shows issues in the measured data. page 23 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Quality-control procedures were used for evaluation of those stations that were pre-qualified for validation of the SolarGIS model (data from the SASSCAL network were excluded from further analysis). All affected data readings were flagged for these stations and excluded from further analyses. Tab. 5.1 summarizes percentage of data passed through the quality control tests. Tab. 5.1: Data for each solar measuring station that did not pass through quality control [%] Type of test (numbers show percent of total volume of data)] Sun below Test for physical Visual test Consistency test Total excluded data horizon limits (GHI – DNI – DIF) samples Tamanrasset 49.3 0.0 9.3 1.1 10.4 Bamba 49.2 0.5 0.0 0.0 0.5 Agoufu 49.1 0.05 9.0 0.0 9.1 M Bour 49.7 0.3 12.6 0.0 12.9 Banizoumbou 49.7 0.5 0.0 0.0 0.5 Djougou 49.7 0.05 0.0 0.0 0.05 De Aar 48.9 0.1 0.6 5.9 6.6 Sonbesie 49.5 0.0 3.4 0.3 3.7 Durban 50.8 0.0 0.4 0.2 0.6 Graaff-Reinet 50.5 0.0 0.8 0.5 1.3 Port Elizabeth 50.2 0.0 16.1 1.0 17.1 Vanrhynsdorp 50.2 0.0 1.2 0.9 2.1 Based on our experience we propose the following recommendations for running routine measurement campaign:  Data used for model validation must be measured by high quality instruments; detailed description of instruments must be available. Regular maintenance of sensors must be arranged.  Use of just one or two sensors (GHI, DNI), without redundant (DIF) measurements does not allow applying valuable quality control algorithms.  Instruments should be preferably mounted about 1 m to 1.5 m above ground or roof surface on a stable concrete or metal platform.  Many analysed stations are influenced by shading from surrounding structures. This should be avoided if possible, or affected values should be flagged.  Data cleaning should be systematic and logged.  Data should be quality checked on a continuous basis. Some types of data logger have software, which can automatically pre-flag errors. Data should be provided for the end users with flags indicating the above-mentioned problems, to avoid their mistaken use.  Regular service visits will prevent common issues with tracker misalignment, sensor levelling, PV and battery supply, desiccant exchange, etc. page 24 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 5.2 Validation of solar resource model 5.2.1 Comparison of SolarGIS to other models Solar irradiance for Zambia is calculated by the SolarGIS model (Chapter 3.2 of Interim Solar Modelling Report 128-01/2014). In this Chapter, the annual SolarGIS average is compared to five other data sources with different temporal and spatial resolution and time coverage (Tab. 3.4). Six representative sites are used, as described in Chapter 6.1 of Interim Solar Modelling Report 128-01/2014). Comparison of the databases shows discrepancies (Tab. 5.2 and 5.3), which are determined by their specific characteristics:  Applied model approaches  Type and quality of the input data  Time representation  Spatial and temporal resolution of the output databases. In general, higher uncertainties have to be expected when comparing data representing different time periods due to short-term weather variability and climate cycles, but also due to fluctuating atmospheric conditions (e.g. concentration of aerosols). When comparing with ground observations from the previous decades, one has to consider that these may have been measured with instruments of lower accuracy and under application of less- stringent measuring standards. Tab. 5.2: Comparison of SolarGIS long-term yearly GHI average with five different data sources. Global Horizontal Irradiation [kWh/m2] Database Longe Lusaka Misamfu Mochipapa Msekera Mutanda NASA SSE 2137 2093 2148 2089 2115 2089 Meteonorm 7 2137 1990 2030 2209 2090 2004 PVGIS 3 2144 2086 2286 2071 2078 2129 PVGIS CMSAF 2224 2159 2232 2133 2126 2191 SWERA/NREL 2052 2023 1976 2030 2008 1986 SolarGIS 2200 2120 2170 2137 2071 2150 Standard deviation of GHI annual values 2.8% 3.0% 5.5% 3.0% 2.0% 3.9% Schematic assessment of GHI uncertainty (80% confidence) 3.6% 3.8% 7.1% 3.8% 2.6% 5.0% Expected SolarGIS uncertainty (80% confidence) 6.0% 6.0% 6.0% 6.0% 6.0% 6.0% Tab. 5.3: Comparison of SolarGIS long-term yearly DNI averages with four different data sources. Direct Normal Irradiation [kWh/m2] Database Longe Lusaka Misamfu Mochipapa Msekera Mutanda NASA SSE 2367 2264 2286 2279 2246 2224 Meteonorm 7 2217 1884 1799 2323 2014 1873 PVGIS CMSAF 2290 2129 2188 2129 1980 2170 SWERA/NREL 1822 1778 1586 1828 1730 1652 SolarGIS 2117 2000 1895 2103 1801 1909 Standard deviation of DNI annual values 9.8% 9.6% 14.7% 9.1% 10.3% 11.9% Schematic assessment of DNI uncertainty (80% confidence) 12.5% 12.3% 18.8% 11.7% 13.2% 15.2% Expected SolarGIS uncertainty (80% confidence) 12.0% 12.0% 12.0% 12.0% 12.0% 12.0% page 25 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Tabs. 5.2 and 5.3 show dispersion of yearly GHI and DNI values between six different databases, including SolarGIS. This comparative approach is a simplified way how to assess the solar resource uncertainty. Important is to understand the risk of possible great geographical dispersion of the estimates. For comparison we show an uncertainty estimate for SolarGIS based on the available validation data in the region (more in Chapters 5.2.3 and 7.1). The modern satellite-based models, such as SolarGIS, are more accurate as they are based on modern algorithms, which generate data outputs at high spatial and temporal resolution. However such data should be systematically updated and quality controlled, in order to fulfil all needs of the solar energy industry: during the prefeasibility project stage, design optimisation and financing, as well as for operation and management of solar power plants. In this context, high quality ground measurements play key role for validation and adaptation of satellite-based models: this is planned next step for Phase 2 of this project. Based on the analysis of sites from Europe, North Africa and Middle East, two intercomparison studies [1, 2] independently analyse the accuracy of satellite-based models, besides SolarGIS, some of them available also for Zambia: 3Tier, SOLEMI, HelioClim-3 and IrSOLaV. Tabs. 5.4 to 5.7 show that SolarGIS has very good performance in all statistical indicators for both Global Horizontal Irradiation and Direct Normal Irradiation. Occasionally, some databases show slightly lower Mean Bias, but this indicator may hide problems (high bias) in individual sites, which may compensate each other in the final figure. From a user’s perspective important parameter is Standard Deviation of biases that indicates geographical stability of the model. Full comparison can be found in both studies. Tab. 5.4: GHI quality indicators related to satellite-based solar radiation models [2] Global Horizontal Irradiance, GHI Mean bias Standard deviation Standard deviation [W/m2] [%] of biases [%] of hourly values [%] SolarGIS 3 1 2.7 17 HelioClim v3 4 1 5.3 21 3Tier 4 1 3.4 21 IrSOLaV 1 0 4.0 33 Tab. 5.5: DNI quality indicators related to satellite-based solar radiation models [2] Direct Normal Irradiance, DNI Mean bias Standard deviation Standard deviation [W/m2] [%] of biases [%] of houry values [%] SolarGIS -11 -4 5.9 35 HelioClim v3 25 8 16.1 50 3Tier 17 5 12.1 49 IrSOLaV -3 -1 - 54 Tab. 5.6: GHI quality indicators related to satellite-based solar radiation models [1] Global Horizontal Irradiance, GHI Mean bias Standard deviation Standard deviation [%] [%] of biases [%] of hourly values [%] SolarGIS 0 0 2.1 17 HelioClim v3 5 1 5.1 20 SOLEMI (Aerocom aerosols) 6 2 4.8 23 IrSOLaV 2 1 4.2 24 Tab. 5.7: DNI quality indicators related to satellite-based solar radiation models [1] Direct Normal Irradiance, DNI Mean bias Standard deviation Standard deviation [W/m2] [%] of biases [%] of hourly values[%] SolarGIS -6 -2 5.9 34 HelioClim v3 21 6 13.9 47 SOLEMI (Aerocom aerosols) -40 -11 14.5 49 IrSOLaV -1 0 12.0 49 page 26 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 5.2.3 Validation at sites with high-quality GHI and DNI measurements Compared to high-quality ground measurements (Tabs. 5.8 and 5.9) that passed through quality control (Chapter 5.1), the SolarGIS model slightly underestimates DNI in Southern Africa; in other sites and for GHI the bias is more variable. At the level of individual sites, bias (systematic deviation) of the model values is found in a narrow range (typically within ±10% for yearly DNI and ±4% for yearly GHI), thus it correspond to the expected uncertainty of the SolarGIS model [22]. However due to absence of high-quality measurements in tropical Africa, our proposed model uncertainty is rather conservative (see Chapter 7.1). Terms Bias and RMSD are explained in Glossary. Absolute values of bias are calculated for daytime hours only. Prior to data comparison all data were harmonized into hourly time step. Number of data pairs in Tabs. 5.8 and 5.9 represent all valid hourly daytime data samples from which statistical measures were calculated. Tab. 5.8: Global Horizontal Irradiance: bias and RMSD for validation sites Site name Bias Root Mean Square Deviation KSI Data (RMSD) pairs Hourly Daily Monthly [W/m2] [%] [%] [%] [%] [-] Tamanrasset Algeria 0 0.0 8.5 4.6 1.8 10 216 M Bour Senegal 1.9 11.2 6.4 3.3 3 293 Bamba Mali -2.2 12.0 7.7 5.1 4 403 Agoufu Mali -1.0 10.9 6.1 2.9 3 660 Banizoumbou Niger -1.8 12.3 7.5 4.8 4 360 Djougou Benin 2.7 16.8 9.6 5.4 4 330 De Aar South Africa 8 1.8 11.5 6.9 2.5 2 729 Durban South Africa -13 -3.3 17.4 9.1 4.6 70 5 756 Sonbesie South Africa -6 -1.2 10.5 4.8 2.2 49 13 981 Vanrhynsdorp South Africa -4 -0.8 8.8 3.5 1.3 23 3 279 Port Elizabeth South Africa -9 -2.3 11.8 5.8 3.7 21 4 484 Graaff-Reinet South Africa -1 -0.3 11.6 4.9 0.9 11 2 975 Tab. 5.9: Direct Normal Irradiance: bias and RMSD for validation sites Site name Bias Root Mean Square Deviation KSI Data (RMSD) pairs Hourly Daily Monthly [kWh/m2] [%] [%] [%] [%] [-] Tamanrasset Algeria 24 3.9 21.6 16.4 5.6 10 216 De Aar South Africa -6 -1.0 16.8 9.9 2.4 2 729 Aggeneys South Africa -36 -4.9 16.1 10.7 6.3 203 5 743 Durban South Africa -41 -9.7 30.2 18.1 10.2 187 5 756 Port Elizabeth South Africa -9 -2.0 23.5 13.7 5.5 75 4 484 Sonbesie South Africa -23 -4.2 24.3 17.8 6.3 292 13 981 Vanrhynsdorp South Africa -29 -4.6 18.7 12.8 5.7 114 3 279 Graaff-Reinet South Africa -32 -5.5 21.2 11.6 6.2 79 2 975 Further information about the data and methodology and detailed analysis of uncertainty can be consulted in [23, 24]. Comparison of validation statistics computed for solar meteo sites in Africa with similar geographical conditions shows stability of the SolarGIS model outputs, and provides confidence about the estimated uncertainty of GHI and DNI. page 27 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 6 VALIDATION OF METEOROLOGICAL DATA 6.1 Validation sites The validation procedure was carried out by comparison of meteorological-model data with ground-measured data at the selected meteorological stations in the region. The validation data come from two different databases: NOAA NCDC and SASSCAL. Comparison is performed for a time period 2008 to 2010 (CFSR model) and for 2011 to 2014 (CFSv2 model). The time period for SASSCAL network covers up to 1 year only. For details on the applied meteorological models please refer to Chapter 4.2 in Interim Solar Modelling Report 128-01/2014. Position of the meteorological stations is shown in Tab. 3.5 and Fig. 3.3. 6.2 Air temperature at 2 metres Air temperature is derived from both meteorological models by postprocessing and disaggregation from the original model resolution to 1-km grid (Tab. 6.1). Considering spatial and time interpolation, the deviation of the model values compared to ground observations is acceptable. Variability of the modelled data matches the variability recorded by the ground measurements. However, it is to be noted that the model data represent larger area, they are smoothed and therefore they are not capable to represent exact values of the local microclimate (as measured on a meteo station). Tab. 6.1: Air temperature at 2 m: accuracy indicators of the meteorological model [ºC]. CFSR model (2008 to 2010) CFSv2 model (2011 to 2013)* Bias Bias Bias RMSD RMSD RMSD Bias Bias Bias RMSD RMSD RMSD mean min max hourly daily monthly mean min max hourly daily monthly Lusaka Intl. 0.7 2.3 -0.1 4.4 2.7 1.4 -1.3 -1.2 -0.9 2.5 1.7 1.3 Kasane 0.6 0.7 1.0 2.7 1.8 1.0 -0.4 -0.6 0.5 2.7 1.6 0.8 Harare, Kutsaga -0.5 -1.5 1.0 2.6 1.3 0.7 -1.3 -2.1 0.1 2.8 1.7 1.4 Gweru 0.3 -0.2 1.0 2.2 1.6 0.6 -0.6 -1.3 0.2 2.3 1.4 1.0 Mbeya -0.5 0.6 -0.8 2.3 1.4 0.9 -1.5 0.1 -2.1 2.7 1.9 1.6 Chimoio -0.4 -2.0 0.9 3.0 1.5 0.6 -1.3 -2.9 -0.3 3.2 2.3 1.5 Sesheke 0.7 2.0 -0.1 2.9 1.6 0.8 Kalabo -0.5 -0.2 0.0 2.5 1.7 0.6 Kabwe -1.0 0.2 -1.8 2.7 1.9 1.2 Zambezi 0.3 1.3 -0.3 2.4 1.4 1.0 Kasempa -0.8 0.2 -1.0 2.5 1.5 1.0 Serenje -1.8 -1.5 -1.9 2.7 2.2 2.0 Mwinilunga -1.3 -0.1 -1.4 2.6 1.7 1.7 Samfya -1.8 -1.3 -1.7 2.5 2.1 2.0 * Time period for SASSCAL network is shown in Tab. 3.5 page 28 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report The difference in yearly average maximum temperature between modelled and measured data is relatively small. For the minimum night temperature the meteorological models show bias -2.9ºC, in the worst case (model temperature is lower than ground measurement) for one meteorological station. The daytime temperature is represented with much better accuracy: with bias below ±2ºC for all but one meteorological station. It can be observed that accuracy of the model air temperature in the region is good. Both daily and seasonal variability is well represented. Fig. 6.1 shows the data fit for the Lusaka airport meteo station. Fig. 6.1: Scatterplots of air temperature at 2 m at the Lusaka airport meteo station. Measured values (horizontal axis) and meteorological model values (vertical axis) 6.3 Relative humidity Relative humidity for a period 1999 to 2014 is calculated from the specific humidity, air pressure and air temperature. Original time resolution is 1-hour. The indirect calculation of relative humidity from the meteorological models may result in higher deviation, especially for the night values with high relative humidity. The validation results are summarized in Tab. 6.2. Similarly to the case of air temperature, relative humidity exhibits higher bias for average maximum values (night time), as it is a temperature-dependent meteorological variable. Daytime values are represented with better accuracy. Accuracy of the modelled data is relatively stable throughout the year. Fig. 6.2 shows the data fit for the Lusaka airport meteo station. page 29 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Tab. 6.2: Relative humidity: accuracy indicators of the model outputs [%]. CFSR model (2008 to 2010) CFSv2 model (2011 to 2013)* Bias Bias Bias RMSD RMSD RMSD Bias Bias Bias RMSD RMSD RMSD mean min max hourly daily monthly mean min max hourly daily monthly Lusaka Intl. -13 -5 -13 20 16 14 -8 -4 -9 14 11 9 Kasane -10 -6 -15 16 14 11 -7 -5 -9 14 11 8 Harare, Kutsaga -2 -4 1 13 8 4 0 -1 0 12 7 4 Gweru -7 -6 -6 13 11 8 -3 -3 -2 12 8 4 Mbeya 0 3 -3 14 10 7 3 7 -2 12 8 6 Chimoio -10 -11 -8 16 13 11 -5 -4 -4 14 10 6 Sesheke -11 -4 -15 17 13 11 Kalabo -6 -3 -9 14 11 7 Kabwe -1 4 -8 14 11 7 Zambezi -9 -3 -13 16 12 10 Kasempa -3 -1 -7 13 9 5 Serenje 4 5 2 11 8 5 Mwinilunga -1 2 -8 11 6 2 Samfya -2 -1 -3 10 7 5 * Time period for SASSCAL network is shown in Tab. 3.5 Fig. 6.2: Scatterplots of relative humidity at 2 m at the Lusaka airport meteo station. Measured values (horizontal axis) and meteorological model values (vertical axis) page 30 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 6.4 Wind speed Wind speed for the period 1999 to 2014 is calculated from the CFSR and CFSv2 models, from 10-metre wind u- and v- components with the original 1 hourly time step resolution. The results of comparison of modelled wind speed with on-site ground measurements are summarized in Tab. 6.3 and Fig. 6.4. Tab. 6.3: Wind speed: accuracy indicators of the model outputs [m/s]. CFSR model (2008 to 2010) CFSv2 model (2011 to 2013)* Bias Bias Bias RMSD RMSD RMSD Bias Bias Bias RMSD RMSD RMSD mean min max hourly daily monthly mean min max hourly daily monthly Lusaka Intl 0.9 1.0 -0.1 1.9 1.5 1.2 0.4 0.6 -0.3 1.6 1.2 0.8 Kasane -0.9 0.3 -2.5 1.8 1.2 0.9 -1.1 0.0 -2.6 1.8 1.3 1.1 Harare, Kutsaga -0.5 0.4 -2.0 1.6 1.0 0.6 -0.9 0.0 -2.1 1.6 1.1 0.9 Gweru 0.7 0.9 0.1 2.1 1.5 0.8 0.6 0.9 0.0 2.0 1.4 0.7 Mbeya -1.0 0.2 -3.1 2.6 2.0 1.4 -2.0 -0.7 -3.6 2.9 2.4 2.0 Chimoio -0.5 -0.5 -0.6 2.0 1.5 0.6 -1.2 -1.0 -1.4 1.9 1.7 1.2 Sesheke 0.0 0.7 -1.3 1.2 0.5 0.1 Kalabo 0.3 0.6 -0.2 1.1 0.7 0.4 Kabwe 1.6 1.6 0.8 2.0 1.8 1.7 Zambezi -0.5 0.0 -1.7 1.2 0.7 0.5 Kasempa 0.9 1.1 0.0 1.3 1.1 0.9 Serenje -0.4 0.3 -1.2 1.2 0.7 0.5 Mwinilunga -0.4 0.3 -0.9 4.0 3.7 1.0 Samfya -0.6 0.2 -2.1 1.7 0.9 0.7 * Time period for SASSCAL network is shown in Tab. 3.5 Wind direction (together with wind speed) parameter is strongly determined by the local microclimate. From the comparison it can be seen that modelled wind speed and wind direction do not fit well the measured values. Fig. 6.3 compares wind speed from the CFSv2 model with the measurements. The model represents regional values for 10 m height. The modelled wind speed deviates from the measured data. Fig. 6.3: Comparison of duration curves of wind speed data at the Lusaka airport meteorological station. CFSR/CFSv2 model versus local measurements (data represent period 2008 to 2013). page 31 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report Fig. 6.4: Scatterplots of wind speed at 2 m at Lusaka airport meteo station. Measured values (horizontal axis) and meteorological model values (vertical axis) Wind direction (together with wind speed) is represented by wind rose, and this parameter is strongly determined by local microclimate (Fig. 6.5). From the comparison it can be seen that modelled wind speed and wind direction deviate from the measured values. Wind speed data for the other meteorological stations exhibit similar characteristics, with the annual bias below 1 m/s. Fig. 6.5: Comparison of wind direction derived from the CFSv2 modes (left) with local measurements (right) at the Lusaka meteorological station (data represent period 01/2008 to 12/2013). The reason why the model wind speed values differ from the measured data is low spatial resolution of the CFSR and CFSv2 meteorological models, which represents only regional effects, while data at micro-scale may differ from the regional scale. Since meteorological model represent larger area, the highest wind speed modelled are not present in the measured data. page 32 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 7 UNCERTAINTY OF THE MODEL ESTIMATES 7.1 Solar resource parameters In Zambia, the uncertainty of DNI and GHI is determined by the combined uncertainty of the SolarGIS model and of the ground measurements [22], more specifically: 1. Parameterization of numerical models integrated in SolarGIS for the specific data inputs and their ability to generate accurate results for various geographical conditions:  Data inputs into SolarGIS model (accuracy of satellite data, aerosols, water vapour and terrain).  Clear-sky model and its capability to properly characterize various states of the atmosphere  Simulation accuracy of the satellite model and cloud transmittance algorithms, being able to properly distinguish different types of surface, clouds, fog, vegetation, occasional flooding, etc.  Diffuse and direct decomposition models 2. Uncertainty of the ground-measurements, which is determined by:  Accuracy of the instruments  Maintenance practices, including sensor cleaning, calibration  Data post-processing and quality control procedures. Statistics, such as bias and RMSD (Chapter 5.2.3) characterize accuracy of SolarGIS model in a given validation points, relative to the ground measurements. The validation results are determined by local geography and by quality and reliability of the ground-measured data. It is to be noted that validation for one single site can provide only indicative information. Consistent understanding of the model performance and uncertainty can only be developed by analysis of several validation sites representing similar geographic conditions. From the user’s perspective, the information about the model uncertainty has probabilistic nature, which can be considered at different confidence levels. Tabs. 7.1 and 7.2 show expert estimate of the model uncertainty assumed at 80% probability of occurrence (an equivalent to 90% exceedance) of values. Tab. 7.1: Interim uncertainty of SolarGIS model estimate for GHI, GTI and DNI Yearly uncertainty Monthly uncertainty Global Horizontal Irradiation (GHI) ±6% ±8% Global Tilted Irradiation (GTI) ±7% ±9% Direct Normal Irradiation (DNI) ±12% ±15% Tab. 7.2: Uncertainty of estimate of yearly solar resources: ground instruments vs. SolarGIS model Best sensors and SolarGIS data2 professional maintenance1 DNI: Rotating Shadowband Radiometer (RSR) ±3.5% ±12% DNI: First class pyrheliometer ±1% GHI: Rotating Shadowband Radiometer (RSR) ±3.5% ±6% GHI: Secondary standard pyranometer ±2% 1 Range of uncertainty depends on climate, measurements practices and post-processing 2 Depends on the geographical ability of SolarGIS model and input data to reflect the local solar climate page 33 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 7.2 Meteorological data The quality of the modelled meteorological parameters stored in the SolarGIS database was assessed by comparison with ground measurements in the geographic region. Model meteorological data are derived from two different numerical models covering periods from 1994 to 2010 (CFSR model) and 2011 to 2014 (CFSv2). Taking into account the results of the comparison, the uncertainty is estimated in Tab. 7.3. It was found that the modelled air temperature fits quite well the measured data with occasional larger discrepancies in minimum night-time or maximum day-time temperature. Wind speed and wind direction data from the meteorological model represent larger region and are smoothed in a comparison to the site measurements at a meteorological station. Although modelled wind speed and wind direction usually fit the patterns of the site-measured data, the maximum values are often not present accurately in the modelled data. Tab. 7.3: Expected uncertainty of modelled meteorological parameters in region. Annual Monthly Hourly <3.0 (night time) Air temperature at 2 m [°C] <1.5 <2.0 <2.0 (day time) <25 (night time) Relative humidity at 2 m [%] < 10 <15 <15 (day time) Average wind speed at 10 m [m/s] <1.0 <1.5 <2.0 page 34 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 8 LIST OF FIGURES Fig. 3.1: Position of solar validation sites in Africa ............................................................................................... 10 Fig. 3.2: Position of AERONET stations .............................................................................................................. 13 Fig. 3.3: Position of meteo stations considered for validation of CFSR and CFSv2 model outputs ..................... 14 Fig. 4.1: Comparison of daily summaries from MACC-II model with 15-min AERONET data. ............................ 16 Fig. 4.2: Comparison of Aerosol Optical Depth ................................................................................................... 18 Fig. 4.3: Aerosol Optical Depth for Lusaka from six different AOD databases .................................................... 19 Fig. 4.4: Monthly-averaged aerosol maps (AOD 670) derived from the MACC-II database ................................ 20 Fig. 4.5: Zambia in the world context – average annual aerosols ........................................................................ 20 Fig. 5.1: Quality control of data measured at Durban measuring station in South Africa ..................................... 21 Fig. 5.2: Example of incorrect DNI (DIF) measurements due to problems of sun tracking. ................................. 22 Fig. 5.3: Example of incorrect DNI measurements due to issues with calibration. .............................................. 22 Fig. 5.4: Two examples of data issues identified at meteo stations ..................................................................... 23 Fig. 5.5: Example of missing data ....................................................................................................................... 23 Fig. 5.6: Quality of GHI measurements for two stations ...................................................................................... 23 Fig. 6.1: Scatterplots of air temperature at 2 m at the Lusaka airport meteo station. .......................................... 29 Fig. 6.2: Scatterplots of relative humidity at 2 m at the Lusaka airport meteo station. ......................................... 30 Fig. 6.3: Comparison of duration curves of wind speed data at the Lusaka airport meteorological station. ........ 31 Fig. 6.4: Scatterplots of wind speed at 2 m at Lusaka airport meteo station........................................................ 32 Fig. 6.5: Comparison of wind direction derived from the CFSv2 modes (left) ...................................................... 32 page 35 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 9 LIST OF TABLES Tab. 3.1: Sources of solar resource validation data ......................................................................................... 9 Tab. 3.2: Solar measuring stations in Africa, used for validation of SolarGIS ................................................ 10 Tab. 3.3: Solar measuring stations operated by SASSCAL network in Zambia ............................................. 11 Tab. 3.4: Inventory of solar resource models for Zambia. .............................................................................. 11 Tab. 3.5: Meteo stations in the region considered for validation of CFSR and CFSv2 model outputs ........... 13 Tab. 3.6: Selected meteorological models available in the region. ................................................................. 14 Tab. 5.1: Data for each solar measuring station that did not pass through quality control [%] ....................... 24 Tab. 5.2: Comparison of SolarGIS long-term yearly GHI average with five different data sources. ............... 25 Tab. 5.3: Comparison of SolarGIS long-term yearly DNI averages with four different data sources. ............. 25 Tab. 5.4: GHI quality indicators related to satellite-based solar radiation models [2] ..................................... 26 Tab. 5.5: DNI quality indicators related to satellite-based solar radiation models [2]..................................... 26 Tab. 5.6: GHI quality indicators related to satellite-based solar radiation models [1] ..................................... 26 Tab. 5.7: DNI quality indicators related to satellite-based solar radiation models [1] ..................................... 26 Tab. 5.8: Global Horizontal Irradiance: bias and RMSD for validation sites ................................................... 27 Tab. 5.9: Direct Normal Irradiance: bias and RMSD for validation sites ......................................................... 27 Tab. 6.1: Air temperature at 2 m: accuracy indicators of the meteorological model [ºC]. ............................... 28 Tab. 6.2: Relative humidity: accuracy indicators of the model outputs [%]. .................................................... 30 Tab. 6.3: Wind speed: accuracy indicators of the model outputs [m/s]. .......................................................... 31 Tab. 7.1: Interim uncertainty of SolarGIS model estimate for GHI, GTI and DNI ........................................... 33 Tab. 7.2: Uncertainty of estimate of yearly solar resources: ground instruments vs. SolarGIS model ........... 33 Tab. 7.3: Expected uncertainty of modelled meteorological parameters in region. ........................................ 34 page 36 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 10 REFERENCES [1] Ineichen P., 2014. Long Term Satellite Global, Beam and Diffuse Irradiance Validation. Energy Procedia, 48, 1586–1596. [2] Ineichen P. Five satellite products deriving beam and global irradiance validation on data from 23 ground stations, university of Geneva/IEA SHC Task 36, 2011: http://www.unige.ch/cuepe/pub/ineichen_valid-sat-2011-report.pdf [3] Meteonorm handbook, Version 6.12, Part II: Theory. Meteotest, 2010 [4] Šúri M., Huld T., Cebecauer T., Dunlop E.D., 2008. Geographic Aspects of Photovoltaics in Europe: Contribution of the PVGIS Web Site. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 1, 1, 34-41. [5] Šúri M., Huld T., Dunlop E.D., Albuisson M., Wald L, 2006. Online data and tools for estimation of solar electricity in Africa: the PVGIS approach. Proceedings of the 21st European Photovoltaic Solar Energy Conference and Exhibition, 4-8 October 2006, Dresden, Germany [6] Surface meteorology and Solar Energy (SSE) release 6.0, Methodology, Version 2.4, 2009. [7] SWERA web site. Monthly and annual average global data at 40 km resolution for Africa, NREL, 2006. [8] Huld T., Müller R., Gambardella A., 2012. A new solar radiation database for estimating PV performance in Europe and Africa, Solar Energy, 86, 6, 1803-1815. [9] Aerosol Robotic Network (AERONET), NASA. http://aeronet.gsfc.nasa.gov/ [10] GFS model. http://www.nco.ncep.noaa.gov/pmb/products/gfs/ [11] CFSV2 model. http://www.nco.ncep.noaa.gov/pmb/products/CFSv2/ [12] CSFR data web site http://cfs.ncep.noaa.gov/cfsr/ [13] Morcrette J., Boucher O., Jones L., Salmond D., Bechtold P., Beljaars A., Benedetti A., Bonet A., Kaiser J.W., Razinger M., Schulz M., Serrar S., Simmons A.J., Sofiev M., Suttie M., Tompkins A., Uncht A., GEMS- AER team, 2009. Aerosol analysis and forecast in the ECMWF Integrated Forecast System. Part I: Forward modelling. Journal of Geophysical Research, 114. [14] Benedictow A. et al. 2012. Validation report of the MACC reanalysis of global atmospheric composition: Period 2003-2010, MACC-II Deliverable D_83.1. [15] Cebecauer T., Šúri M., 2012. Correction of Satellite-Derived DNI Time Series Using Locally-Resolved Aerosol Data.. Proceedings of the SolarPACES Conference, Marrakech, Morocco, September 2012. [16] NASA Multiangle Imaging Spectroradiometer (MISR). https://www-misr.jpl.nasa.gov/ [17] NASA Moderate Resolution Imaging Spectroradiometer (MODIS). http://modis.gsfc.nasa.gov/ [18] NASA Goddard Chemistry Aerosol Radiation and Transport (GOCART). http://croc.gsfc.nasa.gov/gocart/ [19] NASA Goddard Earth Sciences, Data and Information Services Center (GES DISC), Giovanni - Interactive Visualization and Analysis. http://disc.sci.gsfc.nasa.gov/giovanni [20] NREL, 1993. User’s Manual for SERI_QC Software-Assessing the Quality of Solar Radiation Data. NREL/TP-463-5608. Golden, CO: National Renewable Energy Laboratory. [21] Younes S., Claywell R. and Munner T, 2005. Quality control of solar radiation data: Present status and proposed new approaches. Solar Energy 30, 1533-1549. [22] Šúri M., Cebecauer T., 2014. Satellite-based solar resource data: Model validation statistics versus user’s uncertainty. ASES SOLAR 2014 Conference, San Francisco, 7-9 July 2014. [23] Perez R., Cebecauer T., Suri M., 2014. Semi-Empirical Satellite Models. In Kleissl J. (ed.) Solar Energy Forecasting and Resource Assessment. Academic press. [24] Cebecauer T., Šúri M., Perez R., High performance MSG satellite model for operational solar energy applications. ASES National Solar Conference, Phoenix, USA, 2010. page 37 of 39 ESMAP Solar Resource Mapping for Zambia (P145271) Interim Model Validation Report 11 BACKGROUND ON GEOMODEL SOLAR Primary business of GeoModel Solar is in providing support to the site qualification, planning, financing and operation of solar energy systems. We are committed to increase efficiency and reliability of solar technology by expert consultancy and access to our databases and customer-oriented services. The Company builds on 25 years of expertise in geoinformatics and environmental modelling, and 14 years in solar energy and photovoltaics. We strive for development and operation of new generation high-resolution quality-assessed global databases with focus on solar resource and energy-related weather parameters. We are developing simulation, management and control tools, map products, and services for fast access to high quality information needed for system planning, performance assessment, forecasting and management of distributed power generation. Members of the team have long-term experience in R&D and are active in the activities of International Energy Agency, Solar Heating and Cooling Program, Task 46 Solar Resource Assessment and Forecasting. ® GeoModel Solar operates a set of online services, integrated within SolarGIS information system, which includes data, maps, software, and geoinformation services for solar energy. http://geomodelsolar.eu http://solargis.info GeoModel Solar is ISO 9001:2008 certified company for quality management. page 38 of 39