WPS7759 Policy Research Working Paper 7759 Cheaper, Faster, and More Than Good Enough Is GPS the New Gold Standard in Land Area Measurement? Calogero Carletto Sydney Gourlay Siobhan Murray Alberto Zezza Development Economics Development Data Group July 2016 Policy Research Working Paper 7759 Abstract In rural societies of low- and middle-income coun- has held promise for balancing cost and precision. Guided tries, land is a major measure of wealth, a critical input by purposely collected primary data from Ethiopia, Nige- in agricultural production, and a key variable for assess- ria, and Tanzania (Zanzibar), and with consideration for ing agricultural performance and productivity. In the practical household survey implementation, the paper absence of cadastral information to refer to, measures of assesses the nature and magnitude of measurement error land plots have historically been taken with one of two under different measurement methods and proposes a approaches: traversing (accurate, but cumbersome), and set of recommendations for plot area measurement. The farmers’ self-report (cheap, but marred by measurement results largely point to the support of GPS measurement, error). Recently, the advent of cheap handheld GPS devices with simultaneous collection of farmer self-reported areas. This paper is a product of the Development Data Group, Development Economics. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at gcarletto@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team   Cheaper, Faster, and More Than Good Enough: Is GPS the New Gold Standard in Land Area Measurement? By Calogero Carletto, Sydney Gourlay, Siobhan Murray, and Alberto Zezza (World Bank)   Keywords: Land, Agriculture, Measurement, Surveys. JEL Classification: Q12, Q15, C8     Acknowledgements This document was made possible by generous funding from UK Aid. The authors would like to thank Raka Banerjee, Flavio Bolliger, Klaus Deininger, Sam Desiere, Stein Holden, Talip Kilic, Gbemisola Oseni, Marco Tiberti, Abhilasha Vaid, Colin Williams, and Ismael Yacoubou Djima for their inputs during fieldwork implementation and preparation and review of this paper and companion guidebook (Carletto et al., 2016). Fieldwork for the methodological experiments was executed in partnership with the Central Statistical Agency of Ethiopia, the National Bureau of Statistics in Nigeria, and the Ministry of Agriculture and Natural Resources in Zanzibar, Tanzania. Our greatest appreciation goes to these partners for their dedication to the projects. This paper has large areas of overlap with a companion guidebook (Carletto et al., 2016), which serves as a practical guide for implementation of land area measurement in household surveys.    2    I. Introduction Land area measurement has been of concern to humanity since the dawn of time. Some of the buildings constructed in ancient Egypt as early as 2700 BC testify to the knowledge of land surveying techniques. Evidence of boundary surveying in Mesopotamia and the Nile Valley from around 1400 BC have also been found (Lyman and Wright, 2015). The word ‘geometry’ originated in ancient Greece from the Greek words for ‘measure’ and ‘Earth’. Similarly, ancient traces of the command of surveying techniques have been found in China (Swetz, 1992) and India (Joseph, 1997). The need to measure land originated from concerns as diverse as the construction of buildings, tax collection, and the need to demarcate properties and boundaries. Those concerns are still valid today, and many of the basic principles and techniques have remained remarkably similar over the millennia, even though technology has certainly advanced and allows for measurements that are both easier and more precise than they might have once been. With over 70 percent of the developing world’s poor residing in rural areas where agriculture is the primary means of livelihood, high quality agricultural data and analysis are paramount to informing policy aimed at poverty reduction (IFAD, 2010). Land is a key measure of absolute and relative farmer wealth, a critical input in production, and a key variable for normalizing agricultural input use and output measures. Although easily overlooked by analysts, the quality of land area measurement can have non- trivial implications for agricultural statistics, economics, and policy analysis (Carletto et al., 2013 and 2015; Dillon et al., 2016). Area measurement holds significant value in the developed country context as well. In Europe, national authorities conduct a regular Farm Structure Survey employing a common methodology devised by Eurostat (Istat, 2008). The EU FIELDFACT Project is one of many schemes that aim to raise farmer awareness on the use of GPS in land measurement for the purposes of precision agriculture and more accurate and transparent subsidy claims through the EU’s Common Agricultural Policy.1 Frequent land measurement is encouraged by several developed country governments (e.g. USA, UK, Germany, Australia) at the individual farmer level in promotion of precision agriculture, whereby the production process is tailored to farmland size estimates obtained via GPS/GNSS and remote sensing. By knowing exact area measurements, farmers are able to adjust input use accordingly and, thus, optimize yields while cutting down on costs. Effective land area measurement, therefore, also contributes to a more focused application of fertilizers and pesticides that could, in turn, alleviate environmental degradation and pollution.                                                              1 http://www.gsa.europa.eu/introduction-and-promotion-gnss-agriculture  3    The implications of land area measurement extend well beyond agricultural productivity. Disaggregated land ownership data are an input into analyses of agrarian structures and how these evolve with economic and demographic change, the related analyses of land inequality, and how this might be related to income inequality and its trends. Notable examples are the literature on the relationship between asset distribution and income inequality (Deininger and Squire, 1998; Deininger and Olinto, 1999), or the related work on agrarian structures and inequality (Carter, 2000). Unequal distribution of land has been linked to less pro- poor growth, participation in and occurrence of civil strife, and delayed long-run human capital development (Deininger and Squire, 1998; Macours 2011; André and Platteau, 1998; Baten and Juif, 2013). Carletto et al. (2015) find that the area measurement methodology used in calculating the land Gini coefficient has consequences on the level of inequality observed, with self-reported area estimates resulting in underestimated land inequality. Failure to adequately measure land limits the ability to analyze the agricultural economy and its relationship with land inequality. Land registration and titling programs require high quality land area measurement for fair program implementation. Such programs are frequently met with opposition and accusations of corruption or favoritism. Ongoing land certification reform efforts in Ethiopia have recently moved from the first stage of certification, which consisted of identification of plots by land markings and neighbor recall, to the second stage certification in which GPS measurements would replace the first stage data. A recent study analyzing the demand for the second stage certification concluded that the majority of the demand for such area measurement comes from land administrators, as households exhibited a low and declining willingness to pay over the period 2007 to 2012 (Bezu and Holden, 2013). International organizations have also emphasized the importance of objective area measurement in land registration and redistribution. In its support to the Zimbabwe Ministry of Lands and Rural Resettlement 2014-2016 Action Plan (UNDP, 2014), the United Nations Development Programme explicitly mentions GPS and remote sensing as requirements for the successful tracking of the redistributed, post-2000 Land Reform Program parcels through the creation of a national land information database. The methodological menu for collecting land area measurements is diverse and selection of the appropriate method depends on several factors. This paper focuses on the methods that hold relevance for agricultural and household surveys.2 Readers interested in a broader approach to the measurement of agricultural land are referred to FAO (1982) and Sud et al. (2015). This is an important distinction because several measures of agricultural land that are important for agricultural statistics can be collected separately from information about the holding or the household (e.g. when the goal is to estimate crop                                                              2 The focus will also be on low income countries with little or no cadastral or administrative information to integrate this type of data collection.  4    land or area under specific crops at the national or other administrative level). For the analysis of household level processes and outcomes, on the other hand, it is vital that the land area being measured can then be linked to other variables concerning the agricultural production, or welfare outcomes, or other variables of interest for the same household or holding.3 The main types of surveys for which these measurements are relevant are agricultural sample surveys, agricultural censuses, multi-topic surveys that cover agriculture (such as most Living Standard Measurement Study (LSMS) surveys), and smaller scale household surveys carried out for research purposes. This paper aims to provide some elements to inform the selection of measurement methods, based on empirical evidence gathered by the Living Standard Measurement Study – Integrated Surveys on Agriculture (LSMS-ISA) team of the World Bank in Ethiopia, Tanzania, and Nigeria during methodological fieldwork aimed at understanding the relationship between farmer estimates, GPS measurement and the traditional compass and rope method. This paper focuses on the analysis of area measurement data from the abovementioned studies. For more detailed guidance on implementation and practical considerations for area measurement in household surveys, refer to the complementary guidebook (Carletto et al., 2016). II. Methods for land area measurement in surveys Achieving accurate area measures in a household survey setting with limited financial resources and a potentially strict fieldwork schedule is a challenging endeavor. Numerous considerations need to be balanced in designing the survey and selecting the appropriate method including the logistics of transportation to plots, the length of the full questionnaire and the likelihood of respondent and/or enumerator fatigue, and the security of teams and equipment. Three main methods are used in the context of agricultural statistics data collection for the measurement of land area: compass and rope, traditionally held as the ‘gold standard’; respondent self-reported land area; and, the most recent addition, GPS-based measurement. Each area measurement option has unique costs and benefits that need to be carefully assessed in view of the scale of the data collection of which they are a part, the intended use of the data, and the characteristics of both the plots to be measured and the respondents to the survey. Different methods also present different challenges in terms of their implementation: a potentially accurate method can become highly inaccurate if poorly implemented in the field, or it may simply not be feasible on a                                                              3 Similarly, for plot level productivity analysis it is essential that input and output data can be combined at the plot level. 5    larger scale. The specific limitations, challenges, and benefits of each of the abovementioned measurement methods are addressed in detail below.  A. Self-Reported Area Estimation   Household surveys, particularly in developing countries, have historically relied on subjective measurements of land area, and for good reason. The marginal cost of adding one or two questions to a survey that is already being administered to a household is trivial and the exercise can be completed in a matter of minutes, with no need to visit the plot. As a result, item non-response for this item is usually negligible in existing surveys. The minimal financial investment required by this method, however, does not come without challenges, particularly in terms of data quality. Several factors influence the accuracy of subjective farmer self-reported estimates of area, including respondent characteristics, plot characteristics, and the land registration or titling system. Land in developing countries is often passed down from generation to generation or distributed by the community, and rarely are there property rights or documentation to inform farmers of their true land area. In many cases measurement units are not standardized and determining accurate conversion factors is a time- consuming and often ill-fated exercise. The most worrying aspect of some of the measurement error associated with self-reporting is that it may be systematic, and associated with key variables of interest. The literature has shown the accuracy of subjective estimates to be sensitive to respondent characteristics. More educated farmers might be more numerate and more at ease at quantifying their own land area, while absentee landlords, or respondents for whom farming is only a secondary activity, may be less aware of the characteristics of their plots. Using data from Uganda, Carletto et al. (2013) analyze the determinants of the difference between farmer self-reported plot area and GPS measurements, finding that the age of the household head has a significant and positive relationship with measurement bias. The quality of data collected through farmer self-reporting is also significantly degraded by the natural inclination of respondents to round off numbers. Distributional analysis of GPS and self-reported areas of the 2010/2011 Malawi Integrated Household Survey shows clear evidence of heaping at whole numbers and common fractions, such as 0.5 acres (Figure 1). Carletto et al. (2013 and 2015) and Desiere and D’Haese (2015) find rounding to be a significant factor in the discrepancy between GPS area and farmer self-reported estimates. Plot characteristics, such as boundary delineation and the existence of property rights, are known to significantly influence the measurement bias (Carletto et al. 2013, 2015). Plot slope or crop-type may also play a role in the ability of a farmer to estimate plot size. De Groote and Traoré  (2005) assessed the accuracy of a method in which farmer self-report is elicited during a visit to the plot, 6    and a discussion with a trained enumerator. Comparing this method to rope and compass in southern Mali they find that on average plots areas were underestimated by 11 percent, with smaller plots being overestimated and larger plots underestimated, and measurement error being smaller for cotton fields than for cereals. Farmer self-reported area estimates are influenced not only by plot and respondent characteristics, but also by a variety of cultural considerations and logistics of survey implementation. One major such factor is the prevalence of traditional or non-standard units. In many countries, respondents are often not familiar with standard measurement units such as acres, square meters, or hectares as they are used to express area measures in traditional units. Those units are not standardized and may vary in size by region, or even across villages or farms. In Ethiopia, for example, one of the most common non-standards units is the timad, traditionally defined as the amount of land a pair of oxen can plough in one day. This measure will vary significantly by region, and even by farm. The soil texture and moisture content, plot slope and strength of the oxen will have an impact on the ease with which the oxen move and hence on the size of a timad. For Eastern Ghana, Goldstein and Udry (1999) report a correlation between self- reported and GPS measured plot size of just 0.15, which they attribute to the agricultural history of the region where local field measurements are traditionally based on length rather than area, and respondents are not accustomed to converting them to two dimensional area measures. Recent studies emphasize not only the presence of large measurement error in self-reported measures, but also the systematic association of the magnitude and sign of the error with important plot characteristics. Carletto et al (2015), using LSMS-ISA data from Malawi (2010/11), Tanzania (2010/11), Niger (2011) and Uganda (2009/10), identify a common trend in the magnitude of measurement bias, defined as self- reported minus GPS area for plots for which both measures were available and taken independently. The smallest of plots (less than 0.5 acres) are systematically over-reported. The degree to which these are over-reported varies but in all countries the mean self-reported area is overestimated by at least 90% of the mean GPS area of plots in that particular class. With increasing plot size the degree of over-estimation decreases and, eventually, converts to under-estimation for the largest plots. Carletto et al (2013) find the same trend in 2005/6 data from Uganda while Dillon et al. provide evidence of similar systematic trends in data from Nigeria (2016). In a methodological study conducted in Southern Mali comparing area estimates with compass and rope measurement on larger plots (average 0.816 ha, maximum 8.78 ha) De Groote and Traoré (2005) find that the same is true: farmers (while aided by expert observers) are inclined to over-estimate the area of plots less than one hectare, while the degree of area under-estimation increases with plot size. 7    B. Measurement with Handheld GPS Devices   Measurement with Global Navigation Satellite Systems, such as the Global Positioning System, collectively referred to hereafter as GPS, requires that the enumerator first traverse and clear the plot boundary with the farmer. The enumerator then begins at a designated corner of the plot, starts the GPS area measurement function, paces the perimeter at the recommended speed (pausing at all corners to allow for coordinate capture) and completes the area measurement upon returning to the initial corner (instructions may vary by GPS unit). GPS technology and GPS-enabled devices offer a practical approach to objective area measurement (Kelly and Donovan, 2008). Measuring land area with portable GPS devices is becoming increasingly popular among survey practitioners around the world. The method is relatively cheap, accurate and precise if careful measuring protocols are devised and implemented. The main concerns with the method relate to the measurement of plots that cannot be visited by survey staff (resulting in missing data), and the accuracy of the measurement on very small plots. Advancements in GPS technology show promise for increased accuracy in the coming years as more satellites are launched and the availability of satellite augmentation systems spreads to reach all world regions (at the time of writing this report augmentation systems do not currently extend across Africa to any useful degree).4 Time can often be the most restrictive resource in survey implementation and existing studies comparing the time use for GPS and compass and rope find that compass and rope can take approximately 3.5 times as long as required for GPS (Schoning et al., 2005; Keita and Carfagna, 2009). Keita and Carfagna (2009) provide a discussion of the area measurement performance of different GPS devices compared to traversing. Their discussion is informed by a field experiment, the results of which indicate that the GPS-                                                              4 In 2011 the Russian Global Navigation Satellite System (GLONASS), which works seamlessly with the United States’ GPS network, became globally operational with 24 satellites. Augmentation systems can improve the accuracy and speed of GPS measurement in the field. The Wide Area Augmentation System (WAAS), a real-time correction based on ground stations, has been proven to increase position accuracy by as much as five times according to a leading manufacturer. The WAAS system is only operational in North America, while Europe and Asia have their own regional solutions (Euro Geostationary Navigation Overlay Service (EGNOS) and Japanese Multi-Functional Satellite Augmentation System (MSAS), respectively). India’s regional augmentation system (GAGAN) was cleared for navigational use in early 2014. 8    based area measurement is a reliable alternative to traversing and that 80 percent of the sample plots were measured with negligible error. One major advantage of GPS, as with any objective measure, is that of being immune of the potential biases linked to respondent characteristics and the use of non-standard measurement units. Despite the great potential of GPS technology, GPS-based coordinates are subject to known types of measurement error stemming from satellite position, signal propagation, and receivers. Approximate contributions of these factors to the overall position error are significant, ranging from 0.5 to 4 meters (Hofmann- Wellenhof et al., 2008). The number of satellites, in particular, can cause the distribution of position error to be elliptical, rather than spherical (van Diggelen, 2007). Additional factors that may be expected to influence the quality of GPS measures include the presence of dense tree canopy or cloud cover, which may interfere with the signal. The quality of the GPS device used also has non-negligible impact on the magnitude and distribution of measurement error (Palmegiani, 2009). Although position estimates are subject to a certain level of inaccuracy and may be distributed in a non-spherical manner, in theory the error associated with area measurement should be random—that is, the factors that cause non-spherical position error are largely macro level factors that are unlikely to change in the short period of time required to pace the perimeter of a plot, rendering the position error distribution consistent at all points along the perimeter. A study by Bogaert, Delince and Kay (2005) using simulated coordinates and European Geostationary Navigation Overlay Service (EGNOS) augmentation concluded that the position error can be reasonably assumed to be normally distributed. The literature suggests some concern that errors in GPS measures may vary systematically with key plot characteristics, namely plot size, slope, and shape. Few published studies have tested the use of GPS measurement against the gold-standard measure, the traditional compass and rope method. Recent research by FAO points out possible effects of slope on the accuracy of GPS-based area measurement (Keita et al., 2010). Slope-related effects on area measurement are rooted in the fact that the actual area should be the horizontal projection of the plot, as opposed to the plot area itself (Muwanga-Zake, 1985). The difference between actual area and projection appears to be particularly important for slopes greater than 10 degrees (Fermont and Benson, 2011). Bogaert, Delincè  and Kay (2005), based on modeling and simulations conclude that “for GPS/EGNOS measurements made by an operator moving along the border of a field, area measurement error is linked both to the operator speed and to the acquisition rate of the GPS device. For typical field sizes found in the European Union, ranging from 0.5 ha to 5 ha, the coefficient of variation (CV) for area measurement errors is about 1% to 5%. These results depend on the field area, but they can be considered to be 9    insensitive with respect to the field shape. They also show that field area measurement errors can be limited if an appropriate combination of operator speed and GPS acquisition rate is selected.” Fasbender and Lucau (2012) also find that plot shape as well as plot size affects GNSS area measurement error. In their synthetic simulation of four distinct parcel shapes (square, rectangular, elongated narrow rectangle, and irregular polygon), with simulations for areas from 1 m2 to 10 ha, they find that the variance of the measurement on the elongated rectangle and irregularly shaped polygon are the most amplified. Specifically, they suggest that the error on such irregular parcels, which are common amongst low-income regions’ small-holder farmers, is primarily attributable to operator (enumerator) error (Fasbender and Lucau, 2012). One concern with GPS measures in large scale surveys, which does not apply to self-reported measures, is the rate of missingness in the data. Missingness rates of 20-30 percent are not uncommon in existing datasets, and the pattern of missingness is not random but tends to be correlated with both plot and respondent characteristics. Kilic et al. (2013) show, with national data for Uganda and Tanzania, how plot distance from the interviewed household is the main factor determining what plots get measured, as field protocols normally include a provision not to measure plots beyond a given distance. Partly due to that, the plots that are not measured also differ systematically from those for which a GPS measure is taken in a number of (self-reported) characteristics, such as self-reported plot size, level of input use, and titling. Furthermore, some respondent characteristics are associated with higher missingness rates: plots belonging to older, less educated, poorer household heads, owning fewer plots are more likely to be measured than other plots. This is a drawback of GPS data that raises concerns about possible biases introduced by relying on observed GPS plot measures alone. In presence of rates of missingness this high, imputations are often necessary for analysts to be able to work with complete case datasets. Self-reported land areas measures have been shown to be an important predictor of GPS area measure (Kilic et al., 2013). For that reason it is recommended that GPS measures be taken in addition to, not instead of, self-reported ones. C. The ‘Gold-Standard’: Compass and Rope Measurement Traversing, also known as the compass and rope method, is widely used in farm surveys and is often considered to be the gold standard (FAO, 1982). When properly implemented, traversing returns highly accurate measures that can also provide a benchmark against which to judge the precision of other 10    methods. However, its implementation is time-consuming and burdensome, and is often unfeasible in the context of national household surveys and censuses. As in the GPS measurement, this method requires that the enumerator and respondent travel to the plot, and clear its boundaries from obstacles to the extent possible. Before the measurement can begin the farmer must pace the perimeter of the plot with the enumerator in tow to ensure the measurement captures the proper area. The enumerator will note the corners of the plot, where clearly available. When plots are irregularly shaped enumerators use their best judgment in declaring the corner points. The boundary of bushy plots will need to be cleared (with the permission of the farmer) prior to commencing measurement in order for the ranging poles to be visible. Only then can the enumerator start the task of measuring the plot. An example of instructions for completing the compass and rope measurement can be found in FAO  (1982), while an example questionnaire format can be found in Carletto et al. (2016). To some degree, the measurement error associated with traversing is observable. The closure (or closing) error is an important element of quality control in the compass and rope method. It is a measure of the gap between the reported start and end points of the constructed polygon, and gives an indication of the accuracy of the measurement. If the closing error is calculated while in the field, the measurement can be conducted again when found to be above a pre-determined threshold. The closing error will not confirm that the plot corners have been accurately assessed, however, only that the bearings and distances recorded form a full closed figure. The precision of the measurement is still subject to human error as identifying the plot corners can be a burdensome task on its own, particularly for irregularly shaped plots. When corners are not clearly defined (as is the case in many irregularly shaped plots) enumerators must plot the “best” corner they can, and take the compass bearings. With each additional corner, therefore, there is additional room for error in the misreading of the compass or the measurement of the distance between two corners. Misreading of the compass by one or two degrees on one corner is not likely to result in material changes to the area measurement. However, aggregated over several plot corners, these small deviations add up, implying that the area calculation is not for the true plot boundaries. While field protocols usually include closing error thresholds beyond which the measurement needs to be retaken, this adds to the time necessary to take the measurement. The limitations to the compass and rope method lie primarily in the burdensome nature and time required to complete the measurement. The time requirements will vary by plot size but the compass and rope method is expected to be consistently and significantly more time-intensive than GPS. However, when properly implemented this method is considered the gold standard and, as such, it offers the benchmark against which to evaluate the trade-offs with competing, less time-intensive methods. 11    III. Data: The LSMS Methodological Validation Program (MVP) In order to address the gaps in the area measurement literature and extend the applicability of studies to the plot conditions common to developing countries, the Living Standard Measurement Study – Integrated Surveys on Agriculture (LSMS-ISA) of the World Bank has prioritized land area measurement in its research agenda. The Global Strategy to Improve Agricultural and Rural Statistics has also identified improving the measurement of crop productivity, and by necessity farm area measurement, as a top priority (World Bank et al., 2010). Because agricultural statistics are often marred by controversy over methods and overall quality, stringent validation of the available measurement methodologies is essential. With financial support from UK Aid, the Living Standards Measurement Study (LSMS) has partnered with national statistical offices in the design and implementation of methodological validation studies. The methodological studies completed through the LSMS have a particular focus on the feasibility of implementation in large-scale household surveys, thus the recommendations on best practices are made in consideration for both the highest data quality and practicality of implementation under the constraints common in this type of survey. To date, methodological validation studies on land area measurement have been conducted in Tanzania (Zanzibar), Ethiopia and Nigeria. This paper uses the data collected within these studies, which are briefly described in what follows. Data for Zanzibar, Tanzania come from the Measuring Cassava Productivity (MCP) study. The MCP focused on testing several methods for measuring cassava production, including crop-cutting, harvest diary, various means of assisted harvest diary, and various recall periods. To complement the measurement of cassava production, cassava plots were also measured using three methods of area measurement. Fieldwork extended from June 2013 to May 2014, with area measurement completed from August 2013 to January 2014. The study was conducted in two districts, one on Unguja and one on Pemba Island, Zanzibar. The sample consisted of 1,247 households, with 1,932 cassava plots measured for area. Partners in the study included the Ministry of Agriculture and Natural Resources, Zanzibar, the Office of the Chief Government Statistician, Zanzibar, and the World Bank. The handheld GPS Unit used in the study was a Garmin eTrex 30. The second data set used in this paper comes from the Ethiopia Land and Soil Experimental Research (LASER) study. The LASER study involved methodological validation of plot area measurement, soil fertility testing, and measurement of maize production. Area measurement and soil fertility testing was conducted on up to two randomly selected plots per household (where applicable, one pure-stand maize plot was selected for crop-cutting). The questionnaires were administered using computer-assisted personal interviewing. Fieldwork was conducted in multiple waves. Post-planting activities were 12    conducted from September to December 2013. Post-harvest activities were conducted from January to early March 2014. Crop-cutting was conducted at any point during this period when the maize was deemed ready for harvest by the respondent. Area measurement was conducted in the post-planting visit. The data collection for LASER was conducted in 3 zones of the Oromia region in Ethiopia. In total, 85 enumeration areas (EAs) were randomly selected using the Central Statistical Agency of Ethiopia’s Agricultural Sample Survey (AgSS) as the sampling frame. Within each EA, 12 households were randomly selected from the AgSS household listing completed September 2013. Partners in the study include the Central Statistical Agency of Ethiopia, the World Agroforestry Centre (ICRAF), and the World Bank. This study also used Garmin eTrex 30 units to collect GPS land area data. The last batch of data used in this analysis comes from the Nigeria Area Measurement Validation Study. The primary focus of this study was validation of area measurements. Extraordinary differences between GPS and farmer self-reported areas were observed in the first wave of the Nigeria General Household Survey, invoking the need to validate the methodologies. This study was conducted on a subsample of the General Household Survey panel households. After the measurement conducted in the second wave of the national survey a special team was deployed to re-measure a subsample of plots using three area measurement methods. Fieldwork for the area measurement validation study ran from March to May 2013. Four states were selected for inclusion in the study based on safety, location, and previous performance of farmer self-reported area and GPS area. The plot selection was stratified on plot size to ensure a complete range of plot sizes included. In total, 211 households were selected, including 518 plots. The study was implemented by the National Bureau of Statistics, Nigeria, and the World Bank. The GPS Unit utilized for this study was the Garmin GPS Maps 62. In each of the three studies, agricultural plots were measured first by farmer self-reported estimate, then by compass and rope, and finally by GPS. The order of measurements was deliberate and great attention was paid to this in the field. Farmer estimation must be recorded prior to any objective measurement so as not to influence the farmer. Enumerators were instructed in all studies not to influence the farmer’s estimate. Although in each of the three studies the training of methods was conducted in the same way, with the same LSMS staff present at each training, there were two differences in implementation worth noting. First, in each survey enumerators were required to repeat the compass and rope measurement if the closing error was 5% or more. However, in the Ethiopia experiment the closing error calculation was done on the spot by the enumerators, possible because of the use of computer-assisted personal interviewing. In Tanzania and Nigeria, the closing error was calculated by the supervisors (in some 13    instances the supervisors in Nigeria were present at the plot at the time of measurement). This may influence the comparison of compass and rope with GPS measurement if, for example, an enumerator had to re-visit the plot in order to take the new compass and rope measurement and unintentionally identified the borders differently during the second visit. In Ethiopia, on the other hand, all re-measurements were completed at the same time of the initial measurement as the closing error was calculated before leaving the plot. The instance of closing error greater than 5%, however, was rare and therefore not expected to influence the analysis (in the Ethiopia experiment, only 5% of fields were measured more than once). Second, the skill level of the enumerators varied. In Tanzania, the enumerators were the agricultural extension officers for the local area. These enumerators were very familiar with the agricultural practices but generally inexperienced in survey administration and the use of the particular measurement tools. These enumerators also happened to be significantly older and several had poor vision (requiring the purchase of glasses in order to read the compass and GPS). Because of the ongoing and intensive nature of the cassava measurement component of the Tanzania experiment, the extension officers were the preferred enumerators for the existing infrastructure and established relationships within the community. In Ethiopia, professional enumerators were hired based on past performance with the Central Statistical Agency and previous experience with computer-assisted personal interviewing. In this particular study the enumerators all held bachelor’s degrees and were relatively young. In Nigeria, staff from the head office of the National Statistics Bureau were trained and sent to the field rather than the enumerators used to conduct the national panel survey. In each of these studies the health, education, and skill level varied, as did the incentive structure and duration of fieldwork. Rather than discredit the comparability of the data collected from each of these studies, the consistency observed in the comparison of methods should lend confidence in the applicability across survey environments. IV. Methods The first step in analyzing the different methods is comparing the measurements obtained. To that end we construct two measures of deviation between the GPS and CR measures, defined as follows:   Bias GPS – CR Relative Bias * 100 The bias is the simple difference between the GPS measure and the CR measure, expressed in acres. The relative bias is the simple difference between the GPS measure and the CR measure, in acres, divided by 14    the CR measure, expressed in percentage terms. The absolute value of both measures is also used in the analysis. 5 Although the main focus on what follows will be on the deviation of the GPS from the CR measure, we will occasionally employ measures of deviation of the self-reported (SR) from the CR measure, employing a terminology analogous to the one just described for the deviation of GPS from CR measures.6 The analysis will be based initially on a bivariate comparison of the means of the above variables for particular portions of the sample cross-tabulated with a broad range of variables of interest. The second part of the analysis will explore the determinants of the different measures of bias. We will estimate two main regression models. The first model is an OLS regression specified as: (1) Yi=Li+Ci+Si+SATi+Ti+Wi+ei Where Y is one of the four measures of bias defined above, L is the measure of the plot taken using CR, C is the closing error of the CR measure, S is a vector of proxies for the shape of the plot (including the number of corners and the ratio of the perimeter/area, SAT is the number of satellites the GPS device was fixed on at the time of measurement, T is a vector of dummy variables related to tree canopy cover (the reference being no canopy cover), W is a vector of dummy variables related to weather conditions at the time of the measurement (the reference being clear or partly cloudy sky), and e is a random error with the usual desirable characteristics. To focus specifically on plots for which large deviations are observed between GPS and CR we then estimate a probit model to capture the factors likely to increase the probability that a plot be measured with a relative bias larger than ten percent (in absolute value). We estimate three versions of this model for each country dataset, so as to investigate whether under- and over-estimation by large margins are driven by different factors. The model is specified as follows: 2 Pr 1| Φ where , , , , and Φ is the standard cumulative distribution function. In equation (2), Yi is one of three outcomes: a plot having absolute relative bias greater than 10%; a plot having relative bias greater than 10%; a plot having relative bias smaller than -10%.                                                              5  Mean difference presented as [ 100 at each level.  6 To limit the influence of outliers, 78 observations which fell in the top 1% in terms of absolute value of relative bias (for either GPS vs CR or SR vs GPS) of the individual country data sets were dropped.   15    V. Results A. Compass and rope: How golden is the gold standard? The error associated with the compass and rope measurement is to some degree observable through the closing error. The average closing error in our data across all measurement is around 2 percent, with the range going from 1.6 percent in Nigeria to 2.2 percent in Ethiopia (Table 1). Regression analysis aimed at determining the factors that contribute to closing error is presented in Table 2. At the individual country level, plot size appears to influence closing error but with varying results. In Ethiopia, the data with the highest concentration of plots smaller than 0.05 acres, the plot area cubed has a significant and negative impact on closing error, while in Tanzania the linear area term is negative and significant, implying that closing error is smaller on larger plots. In Nigeria, where we have the largest plots, the linear term is positive while the squared term is negative. When data from the three experiments are pooled, plot size has no significant effect on the closing error. The number of corners on the plot (as measured by the number of vertices captured in the compass and rope measurement) exhibits a negative and significant coefficient in Ethiopia and the pooled data – contrary to the expectation that more corners lead to higher closing error. Tree cover proves to have little effect on the closing error, as none of the individual experiments exhibit significant coefficients. Had plots been randomly assigned to enumerators, enumerator effects could have been used to control for enumerator skill level and other idiosyncrasies, but plots were assigned primarily based on geographic proximity and thus enumerators often measured plots with very similar geographic properties, rendering enumerator effects inappropriate. Ultimately, there seems to be little evidence that closing error is systematic. This is comforting for the analysis that follows, as we move to explore systematic sources of error in other area measures, namely deviation from the CR method. From the perspective of survey practitioners and national statistical offices, considerations about accuracy need to be accompanied by considerations related to time (and hence cost) of each methods’ implementation. The reason why the choice of method should matter for survey practitioners is compellingly conveyed by Figure 2, which shows the measurement time for GPS and compass and rope measurements by plot size classes, moving from small plots on the left to large plots on the right. Compass and rope requires significantly more time than GPS with time increasing exponentially with plot size, while the additional time required for GPS measurement for plots of the size included in these studies is negligible. In both the Ethiopia and Tanzania experiments, the compass and rope measurement 16    took approximately four times the time required for GPS.7 In Ethiopia, GPS required 13.9 minutes on average, while the compass and rope measurement on the same plots required an average of 57 minutes. In Tanzania, the duration averages were 7.4 minutes and 29.3 minutes for GPS and compass and rope respectively. These findings are consistent with previous studies such as Schoning et al. (2005) and Keita and Carfagna (2009) who find that compass and rope takes approximately 3.5 times as long as GPS on average. To put the time considerations into context, given the sample size and average measurement durations in Ethiopia, the field teams spent a total of 416 hours measuring plots with GPS (1797 plots * 13.89 minutes) and 1,707 hours measuring with compass and rope. Using GPS instead of compass and rope, therefore, saved 1,291 hours of labor – over 160 person/days (at 8 hours per day). This estimate of time savings is for a relatively small-scale methodological experiment; savings in nationally representative household surveys would be proportionally larger. Minimizing the amount of time required to collect quality land area data can significantly reduce costs and improve the flow of fieldwork. B. Comparison of competing measurements 1. Compass and Rope vs. GPS In the literature, the main reservation regarding the use of GPS measurement in surveys is its performance on small plots. Furthermore, Keita and Carfagna (2009), Schoning et al. (2005), and Palmegiani (2009) all found that GPS tends on average to err on the negative side, i.e. to understate the area with respect to compass and rope. Table 3 presents descriptive statistics on the GPS and compass and rope area measurements completed as part of the methodological studies. Mean plot size is small in all countries, ranging from 0.27 acres in Tanzania to 1.31 acres in Nigeria. The mean difference between compass and rope and GPS measurement is very small. The sample mean bias in all three countries is plus or minus 0.01 acre, which translates in a 1 to 3 percent difference when expressed in relative terms (note that the values are not expressed in absolute value and as such negative and positive figures are averaged). Mean GPS and CR measurements are significantly different at the 1% level in Ethiopia, Tanzania, and the pooled data. In Nigeria, GPS and CR area measurements are significantly different at 10% level. Furthermore, in Nigeria the GPS and                                                              7  Data on measurement duration are not available for Nigeria. 17    compass and rope measurements are significantly different at the 5% level for all plot size levels reported in Table 3 except for the largest plots (level 6), in which the measurements are not significantly different. Notably, GPS and CR measurements on the smallest plots (level 1) are not found to be significantly different in Ethiopia, Tanzania or the pooled data. Unlike previous studies, the data do not exhibit any clear trends in terms of GPS underestimating plot size compared to CR, neither on average, nor across the distribution of plot sizes. In Nigeria, GPS under- estimates plot size compared to CR, but only slightly, while in Tanzania and Ethiopia GPS averages are somewhat larger than compass and rope measurements. Moreover, the magnitude as well as the sign of the error seem both to be unrelated to plot size, being small in all of the plot size classes. The concern of GPS accuracy at small plot sizes is also discounted. While some literature suggests that plots smaller than 0.5 hectares (1.24 acres) have significantly different GPS and compass and rope measurements with much lower correlation (Schoning et al., 2005), results from the methodological validation experiments suggest otherwise. In the pooled data, the difference between the average GPS measurement and average compass and rope measurement for plots ranging from 0.05 – 0.15 acres was less than 0.002 acres or 2% of the average compass and rope area. Even for the smallest plots, those less than 0.05 acres (202.3 square meters or 0.02 hectares), the average measurements are extremely consistent. In Ethiopia, the average GPS measurement of 390 plots in this size range is 0.0216 while the average compass and rope measurement for the same plots is 0.0215 acres. The differences that are recorded do not appear to bear any clear trend with plot size. In Tanzania, the smallest and largest plot classes have the smallest and largest average relative bias, but the figures are not large, and the number of observations in these two classes fairly small. The correlation coefficients between GPS and CR are in excess of 0.99 in all three studies, and 0.87 or larger in all classes with n larger than 50 (Table 4). The results presented here suggest that average GPS measures are not much different from compass and rope even for very small plots, and even from a fairly small n, and that is despite the difference in enumerator skill levels and plot characteristics of the different studies. This is confirmed by an inspection of the scatter plots in the left side of Figure 3, where GPS measures are plotted against compass and rope with measures tightly clustered around the equality line. This all lends support to the argument that GPS is an acceptable substitute of compass and rope measures across the range of plot sizes in our samples, at least if the goal is that of estimating average plot size for groups with sufficient numerosity. 18    2. GPS measures: Exploring deviations from the gold-standard Previous studies have also raised the issue of how factors other than plot size may affect the quality of GPS measures, as was recalled earlier in this paper. None of the studies have provided compelling, conclusive evidence on the impact of these factors on measurement quality. Some have explicitly called for further research to systematically investigate this matter. Our data allow for analysis on a number of factors including plot shape, slope, and tree cover, weather conditions, and number of GPS satellites acquired at the time of measurement, via a comparison of the GPS measurement to the ‘gold standard’ of CR measurement. Annex Table 1 reports summary statistics of how GPS and compass and rope measures compare with varying satellite acquisition, canopy cover and weather conditions. The global navigation system requires, at a minimum, the acquisition of four satellites to triangulate the 3D position of the GPS receiver. The acquisition of additional satellites can improve position error.8 Enumerators in both the Tanzania and Ethiopia experiments recorded the number of satellites fixed at the start of the GPS measurement. In training, enumerators were instructed to wait until at least four satellites were acquired, with further instruction that they must wait until the “GPS accuracy” figure on the GPS device stabilized (thereby allowing time for maximum satellite acquisition). Descriptive statistics suggest that the difference between GPS and CR measurement tends to decline the higher the number of satellites, but average difference remains small across all the distribution of plot areas. In Ethiopia, the difference between measurements is 1.6% (but not statistically significant) on the plots with fewer than 16 satellites and 1.2% on plots with the 20 or more satellites, though the trend is not linear as the middle category has an average of 1.8% bias. In Tanzania, the differences are 2.9% and 2.6%, respectively. Various geographic and atmospheric conditions can impact the satellite acquisition. Dense canopy cover and weather conditions at the time of measurement have been found or argued to impair the precision of the GPS measurement. To address the concern over canopy density and the impact on GPS area measurement accuracy, the methodological validation studies included a subjective measure of canopy density. Contrary to expectations, descriptive analysis reveals that the relative difference between the GPS and CR measurements was slightly higher on plots with no tree cover, with the level of bias decreasing with increasing canopy density. In Ethiopia and Nigeria, there was no statistically significant difference in measurements found on plots reported with partial or heavy tree cover. The lack of statistical significance in the groups with canopy cover is also likely linked to the smaller sample size for these groupings. This could also be attributable to plot size, enumerator characteristics or other factors, which                                                              8 http://www8.garmin.com/aboutGPS/ 19    are not controlled for in these simple descriptive statistics. The sections below will further explore the influence of tree cover on measurement. Higher level atmospheric conditions which can impact the satellite signals influence the precision of GPS point estimates. For this reason, the literature on GPS measurements points to weather conditions as a potential source of error. In order to address this discrepancy, all three methodological studies included a subjective measure of weather at the time of measurement, ranging from all clear to rainy. No clear trend emerges in terms of systematic association of the differences between the two measures and weather conditions. In Ethiopia, the relative bias in measurements hovers around 2% for plots measured in conditions “mostly clear” or better, and the difference in mean measurements is only significantly different on plots measured clear or mostly clear conditions. Similarly, in Nigeria significant differences in GPS and CR measurements are only observed on plots measured in “clear/sunny” conditions. The descriptive statistics from Nigeria and Tanzania provide little evidence that weather conditions have an adverse effect on GPS area measurement. It should be noted that the majority of plots were measured in conditions “partly cloudy” or clearer. Keita and Carfagna (2010) and Muwanga-Zake (1985) explain that plot slope can influence the difference between GPS and compass and rope measured areas, as the GPS measures the horizontal plane and traversing measures the surface area. Fermont and Benson (2011) note that plot slopes greater than 10 degrees will result in significantly different measurements. The LASER study incorporated the use of clinometers for slope measurement. Descriptive statistics on the slope and measurement bias are reported in Table 5. For plots of slope less 5 degrees or less, the mean relative difference is 1.3% whereas for plots of 6 – 15 degrees it is 2.2% and for plots of slope greater than 15 degrees it is 2.4%. Having ascertained that the average difference between GPS and CR is small does not rule out that for individual measurements, there may be observations with errors of significant magnitude. To investigate this aspect we plot the percentage and absolute differences between GPS and CR measures over plot area (Figure 4). A number of considerations emerge from a visual analysis of these graphs. First, the GPS measurement error in percentage terms is often far from negligible, in some instances larger than plus or minus 50 percent. Second, large percentage errors appear to be roughly equally distributed above or below the zero line, which explains why we do not observe differences in the means for the two measures. Thirdly, the magnitude of the percentage errors is much larger for the small size classes, and decreases rapidly as plot size increase. Those trends are clearly mirrored by the graphs with the absolute bias, which show no clear correlation with plot size and fairly constant dispersion both sides of the zero line, with most values within the plus/minus 0.5 acre range. That seems to suggest that it is the inherent imprecision 20    of GPS devices that causes percentage error to matter much more for very small plots. We therefore turn to investigating more in depth the extent and nature of the errors for observation with an arbitrary set value of plus or minus 10 percent. Table 6 reports on some key the characteristics of the plots and the measurements, slicing the sample according to whether the bias is below or above the 10 percent threshold, and splitting the latter portion of the sample in observations where GPS over- or under-reports land area. The number of observations with such large errors is far from negligible, ranging from 17 percent in Nigeria to 31 percent in Ethiopia. Again, no strong systematic bias emerges in terms of GPS over- or under-reporting: differences in average acreage between GPS and CR are small, yet statistically significant, even for the high-error portion of the sample in all countries. In two of the three experiments (Ethiopia and Tanzania) there are more GPS observations with large percentage over-reporting compared to under-reporting, in Nigeria the opposite is true. Desiere and D’Haese (2015) present more optimistic results in their robust sample of over 50,000 parcel-level observations in Burundi, as 90% of plots greater than 550m2 were measured with less than 10% absolute value of relative error. Table 6 also reports the average values for several of the variables that are expected to influence the quality of GPS measurement. We do not observe any substantial difference for some of the factors that are often cited as important for GPS measurement, such as number of plot corners, number of satellites and tree canopy cover. In Ethiopia there is no statistically significant difference in closing error between plots measured with high bias versus those not measured with high bias, while the number of plot corners is not significantly different in Tanzania or Nigeria. We do, however, observe some difference in the perimeter/area ratio, which approximates the complexity of a plot shape. In all three experiments this is higher in plots that are substantially underestimated by the GPS measure, compared to the plots that are measured with greater accuracy. Ethiopia is the only country for which such a difference, albeit of much smaller magnitude, is also observed for plots with size over-reported by more than 10 percent. Shape complexity therefore does seem to affect GPS precision, resulting mostly in under-reporting of the plot size. Some differences are observed also for the walking speed of the enumerators, but with results that are more difficult to interpret. In Ethiopia the walking speed is lower on the plots measured with larger errors. In Tanzania we observe no sizeable difference. In Nigeria plots that are under-estimated by more than 10 percent are associated with lower average walking speed, while plots that are over-estimated tend to record higher average walking speed. We are not able to draw any conclusions from this mixed evidence. 21    One last variable for which we do observe systematic differences is the magnitude of the CR closing error, although differences are not significantly different across high-bias and non-high bias plots in Ethiopia. In both Ethiopia and Tanzania there is a gradient between plots that are underestimated by a large margin (which have the smallest closing error), plots with error below 10 percent (which have moderate closing error), and plots with large GPS over-estimate (with the largest closing error). In Nigeria the plots with larger over-estimates are also the ones with the largest closing error, but the ranking of the other two groups is inverted. What we conclude from these observations is that for the cases in which we observe substantial deviations between GPS and CR measures, part of the explanation is likely to rest in noise in the CR measures. In that sense the inaccuracy in the GPS measures may be somewhat less serious than if one just looked at the prevalence of high bias cases, and that as observed earlier the gold standard is also bound to be imperfect. 3. Compass and Rope vs. Self-Reported Estimations With an understanding of the comparability of GPS and compass and rope objective measurements, we now explore the difference in subjective (self-reported) and objective (CR) measurement.9 Table 7 presents mean plot areas as measured by farmer self-reported estimation and compass and rope for all three methodological experiments. The data is grouped by compass and rope plot size class. While the mean plot areas as measured by GPS and compass and rope differ by only as much as 3% on average, the mean self-reported and compass and rope measurements differ by as much as 143% on average (Tanzania). The mean difference is smaller in Ethiopia and Nigeria, at 23% and 5% respectively, but still considerably larger than the divergence observed between the objective measurements. Self-reported measures result not only in higher average deviations, but in dramatic systematic error as the size of small plots is overestimated by anywhere from 30% (Nigeria) up to a factor of six (Tanzania), with the over-estimation declining almost monotonically as plot size increases and eventually results in under-estimation in the larger plot size classes in Nigeria and Ethiopia. These results comparing objective and subjective area measures are in line with findings of previous literature (Carletto et al. 2015; de Groote and Traore, 2005). The scatter plots on the right side of Figure 3 convey the same message in graphic form.                                                              9 Comparing self-reported to GPS yields exactly the same results, even when analyzed using data from the nationally representative LSMS surveys in Malawi (2010/11) and Tanzania (2010/11), therefore we limit the comparison to self-reported and CR. 22    C. Regression analysis of the differences between competing measures   1. Comparison of CR and GPS The descriptive statistics presented above are aimed at comparing the two primary objective area measurement options, GPS and compass and rope. In this section, regression analysis is used to explore the determinants of measurement bias, defined here as the difference between GPS measured area and compass and rope measured area, which is used as the benchmark. The results in Table 8 include four specifications per dataset, the difference among them being the dependent variable, which is: (i) bias (GPS – CR), (ii) absolute value of bias, (iii) relative bias (bias as a percentage of the CR area), and (iv) absolute value of relative bias. Recall from the descriptive statistics that the observed error is generally small, and little evidence of systematic variation with many of the factors that are a priori expected to influence GPS measurement precision was found. It is therefore not surprising that the explanatory power of these regressions (as captured by their R2 values) is low, and that the majority of the estimated coefficients are not statistically significant. The main variables of interests are the set of terms (levels, quadratic, cubic) related to the plot size itself, as measured by CR, graphic representations of which are available in Figure 5. In the first specification, there appears to be a relationship between plot size and measurement error in Ethiopia, where the shape of the relationship is that of an inverted U, with the predicted bias being positive on very small plots, peaking at about 0.7 acres, and becoming negative for plots larger than about 1.7 acres. The coefficients are small, so that the predicted error is in the plus/minus 0.02 acres range. In Tanzania, a linear relationship is exhibited in which larger plot size results in larger bias (in terms of acres). In Nigeria there is no statistically significant relationship between bias and plot size, and in the pooled data there is very little, controlling for other factors. When the absolute level of bias is considered the relationship with plot size becomes monotonically positive, with a small curvature (significant quadratic term) only in Nigeria and the pooled data. Values are somewhat larger, up to about 0.2 acres in the observed plot size range, but still small. When the percentage bias is considered (third specification) the relationship with plot size becomes an L- shaped quadratic curve (the cubic term is significant only in Ethiopia and the pooled data, tilting the curve up around the 2 acres mark). The last specification has the absolute bias expressed in percentage terms as 23    the dependent variable, with the relationship with plot size being again best characterized as L-shaped. Nigeria is an exception in that no statistically significant relationship with plot size is revealed by these two specifications. These results are in line with the earlier descriptive analysis in that the overall distribution of the bias does not seem to bear much relationship with plot size, as they are largely equally distributed on the positive and negative side. The absolute magnitude of the error does, however, increase somewhat with plot size but less than proportionally. For that reason, in percentage terms the bias actually declines fairly rapidly as plot size increase, stabilizing as plot size reaches the 1-2 acres range. Of the covariates reflecting physical characteristics that are expected to affect the quality of GPS measures (cloud and canopy cover, plot slope) hardly any are consistently significant across country. What appears to matter most are closing error and the perimeter/area ratio. The former reflects inaccuracy in the CR measure, while the latter is a proxy for the complexity of the plot shape which is likely to affect the accuracy of GPS measures, but can in principle also be capturing noise in the CR measure besides what is captured by the closing error. In Ethiopia and Tanzania (as well as in the fourth specification of the pooled sample regression), there is evidence that heavy canopy cover does increase relative bias. An unsystematic comparison of plot outlines computed from the CR method and collected in the GPS also suggests that it may often be the case that enumerators may tend to simplify the shape of the plot more when collecting CR than GPS data (Figure 6, see Carletto et al. 2016, for additional evidence). Although the difference between the two objective measurements is relatively small on average, it is worth digging into the problem cases in which the deviation is much larger. Table 9 reports results of a probit model (see equation 2) estimating the probability of a plot being a “problem plot” – defined here as having a relative bias greater than 10% (in absolute value). In all countries we find evidence of a cubic relationship between the probability of GPS overestimating area by more than 10 percent and plot size. That translates into the probability being highest for very small plots, and decreasing fast as plot size increases, before flattening fairly quickly (and eventually tilting up somewhat) for larger plot sizes. The same relationship is found for plots under-estimated by GPS in Ethiopia, but not in the other two experiments. As in the OLS regression, in the probit model the other covariates that appear to be playing a role are closing error and perimeter/area ratio. Tree canopy cover appears to play more of a role in these regressions, implying that the effects of canopy cover are not felt equally throughout the distribution of the bias variable, but that they kick-in in particular regions of the distribution. Weather at the time of GPS measurement has a more limited effect. In Tanzania, plots that are measured during “mostly cloudy”, “all cloudy”, or “rainy” weather are slightly more likely to be over-stated by the GPS by 10% or more 24    (compared to plots measured during “partly cloudy” or clearer weather). In all other countries and specifications the cloudy or rainy weather does not have a statistically significant effect on the probability of area being measured with high bias. 2. Comparison of SR and Objective Measurements While Table 7 illustrates the degree to which farmer self-reported estimates differ from compass and rope measurements, it does not offer any explanation as to why the two systematically diverge. For this we turn to regression analysis. Table 10 presents the results of four OLS regression models: the first on the measurement bias (farmer self-reported estimate minus CR measured area), the second on the absolute value of this bias, the third on the relative bias, and the fourth on the absolute value of the relative bias (in percentage terms). The claim of plot area affecting the direction and degree of error associated with self-reported area estimates is supported by the regression results. In the first specification (on bias) the coefficient on plot area is negative quadratic and positive in the cubic term in Ethiopia, Nigeria and the pooled data. In the second specification (on absolute value of bias), the coefficients on plot area are positive suggesting that as plot size increases the degree of farmer over-reporting shrinks while at the same time the absolute value of the bias increases. In this second specification, the Tanzania data exhibit a negative quadratic term and positive cubic term. When looking at the relative bias and absolute value of relative bias, the linear term is negative and the quadratic term positive in each country but at very different magnitudes, potentially driven by the difference in average plot size observed across the countries. The distance from the plot to the dwelling holds significant explanatory power in the Ethiopia data, but not in Tanzania. The results from Ethiopia suggest that self-reported estimates of area diverge more from compass and rope measurements on those plots that are further from the household. This could be theoretically explained by assuming the farmer spends less time on plots more distant from the household and does not have the opportunity to view these plots to make his/her area estimate, should he/she prefer to do so. Consistent with Carletto et al. (2015), the existence of property rights (proxied here by the possession of a title or certificate of ownership or the ability to sell or use the plot as collateral) has a significant, negative relationship with the relative bias in Ethiopia and the pooled data, suggesting that on plots where the household has some form of property rights they are better able to estimate the area. Household characteristics such as the gender, age, and education of the household head play out differently across countries. Results from Nigeria and Tanzania (but not Ethiopia) suggest that measurement bias is greater in households with older household heads. Contrary to expectations, the 25    education and literacy status of the household head does not hold consistent results across country. In Ethiopia, literacy of the household head reduces measurement bias (in acreage terms), but years of education has the opposite effect. In fact, the education of the household head significantly increases the relative bias in Ethiopia (but is not statistically significant in Nigeria or Tanzania). One can speculate that, controlling for literacy, education may be associated with increasing opportunity cost of time or decreasing involvement in agriculture, which could make the respondent less attentive or knowledgeable regarding the measurement or reporting of plot area. Finally, one key decision point when collecting self-reported area data is whether to allow respondents to use non-standard units, or to force them (or enumerators) to convert responses from traditional to standard units at the moment of the interview. In Table 11, we compare deviations between self-reported and GPS data separately splitting the sample between observations where respondents used traditional and standard units in the Ethiopia and Nigeria sample. When land area is collected using non-standard units, as opposed to forcing respondents or enumerators to perform a conversion to standard units at the time of the interview, data from self-report appears to approximate the preferred GPS measures much better. This finding supports the idea that it is best to allow non-standard units to be used at interview time, while organizing complementary collection of adequate conversion factors to translate all the data in a common metric at the data processing stage. VI. Conclusions Several important findings emerge forcefully from this analysis, which translate into clear implications for future survey design and implementation. The first result is that our experimental data confirm what we already knew about the presence of large, systematic measurement error in farmers’ self-reported estimates of land area, and on its direction, correlates and determinants (which include land area itself, introducing potentially large biases at the data analysis stage). While not a novel finding, this is a useful reminder of the urgency to find alternative measures that are both accurate and usable in the context of large scale household surveys. GPS measurement is the obvious candidate. Much of the focus of the paper has, therefore, been on assessing the fitness for purpose of GPS measures. In this respect an important finding of the study is that on average GPS measures return very accurate estimates of plot size, even for very small plots, and even for reasonably small samples. We also do not detect any evidence that GPS systematically under-reports land size, as is the case in earlier studies. That should suffice to make GPS an attractive method for land area data collection for most household survey 26    practitioners. This conclusion becomes even more forceful when taken together with the comparison of the time required for GPS compared to CR measurement, with our data showing GPS to lead CR by several orders of magnitude. This strong message in support of the adoption of GPS in survey fieldwork, is, however, mediated by a number of considerations regarding outstanding challenges with GPS measurement. One that emerges from the analysis is that while the GPS measurement error is almost universally small in magnitude (only 5 percent of observations recording a discrepancy with CR of more than 0.09 acre), in relative terms a discrepancy of plus or minus 10 percent is not uncommon. However, as illustrated, the CR method is not immune to measurement error thus some of the problematic cases could be attributable to noise in the CR data. Considering that GPS measures in large scale surveys are often plagued by a missingness rate in the range of 15-30 percent, which suggests that a large scale dataset of GPS plot measurements could be plagued by as much as 50 percent problem cases. Complementing data collection with information that can aid in identifying those problem cases, as well as enforcing sound field implementation protocols, can significantly reduce both bias and missingness in GPS measures. Despite the evidence that subjective measurements can be riddled with problems, farmer self-reported estimates of area should still be included in household surveys, but not as the primary measurement method. Objective measurements come with their own challenges, including time and equipment requirements, questions of accuracy at small-plot levels, and feasibility of full plot sample measurement. Subjective measurements have negligible fieldwork costs, and, more importantly, they can serve as a baseline for imputation where objective measurements may be missing (Kilic et al. 2013). Therefore, we recommend GPS measurement (where feasible) complemented by farmer self-reported estimated area (for all plots). An ancillary story emerging from the data concerns self-reported data. While confirming all the known issues with measurement error in self-reported land data, the analysis presented here provides at least one suggestion for limiting the scope of this error in the future. When land area is collected using non- standard units, as opposed to forcing respondents or enumerators to perform a conversion to standard units at the time of the interview, data from self-report appears to be a better approximation of the benchmark preferred measure. Furthermore, our analysis casts some shadows on the benchmark compass and rope measurement. It appears that a good deal of what we labeled for simplicity as GPS measurement error, may in fact be 27    linked to noise in the CR data. This is hardly surprising since CR does in fact require a good deal of precision that, no matter how careful the training, will be hard to reach for survey enumerators that are not professional land surveyors. In terms of specific suggestions for CR measurements, we do observe an increase in discrepancy between CR and GPS when the CR closing error is above 3 percent. Translated into recommendations for survey work, this means that 3 percent offers a good rule of thumb for instructing enumerators to re-take CR measurement. Finally, little research is to date available on the use of remote sensing imagery for area measurement in household surveys. As technology advances and image resolution improves along with affordability, the use of this method becomes more feasible, and is likely to hold promise particularly for the measurement of large plots. Future research on how to effectively integrate remote sensing and household survey data for plot area measurement, including fieldwork challenges and respondent ability to identify plots, is highly encouraged. 28    VII. References Andre, C. and Platteau, J.-P. (1998). “Land Relations Under Unbearable Stress: Rwanda Caught in the Malthusian Trap.” Journal of Economic Behavior & Organization, vol. 34, pp.1 – 47. Baten, J., and Juif, D. (2013). “A story of large landowners and math skills: Inequality and human capital formation in long-run development, 1820–2000.” Journal of Comparative Economics, http://dx.doi.org/10.1016/j.jce.2013.11.001 Bezu, S. and Holden, S. (2013). "Unbundling Land Administrative Reform: Demand for Second Stage Land Certification in Ethiopia,"CLTS Working Papers 3/13, Centre for Land Tenure Studies, Norwegian University of Life Sciences. Bogaert, P., J. Delincè and S. Kay. (2005). “Assessing the error of polygonal area measurements: a general formulation with applications to agriculture.” Meas. Sci. Technol. 16 (2005) 1170–1178 doi:10.1088/0957-0233/16/5/017. Carletto, G., Gourlay, S., and Winters, P. (2015). “From Guesstimates to GPStimates: Land Area Measurement and Implications for Agricultural Analysis,” Journal of African Economies, first published online May 29, 2015 doi:10.1093/jae/ejv011. Carletto, G., Savastano, S., and Zezza, A. (2013). "Fact or artifact: The impact of measurement errors on the farm size–productivity relationship," Journal of Development Economics, Elsevier, vol. 103(C), pages 254-261. Carletto, G., Gourlay, S., Murray, S., Zezza, A. (2016). “Land Area Measurement in Household Surveys: A Guidebook, Empirical Evidence and Practical Guidance for Effective Data Collection”. Washington, DC: The World Bank. Carter, M. (2000). Land Ownership Inequality and the Income Distribution Consequences of Economic Growth. UNU Wider Working Papers 201. Helsinki: UNU Wider Institute for Development Economic Research. De Groote, H. and O. Traoré. (2005). “The cost of accuracy in crop area estimation”. Agricultural Systems. 84 (2005) 21–38. Deininger, Klaus and P. Olinto (1999). Asset Distribution, Inequality, and Growth. World Bank Policy Research Working Paper No. 2375. Washington, DC: The World Bank. Deininger, K., and Lyn Squire. (1998). “New ways of looking at old issues: inequality and growth.” Journal of Development Economics, Volume 57, Issue 2, 1998, Pages 259-287. Desiere, S., and M. D’Haese. (2015). “Area Measurement in Agricultural Surveys: GPS or Compass and Rope?”. In Desiere, S. From raw numbers to robust evidence. Finding fact, avoiding fiction. Doctoral dissertation, Ghent University. 29    Dillon, A., Gourlay, S., McGee, K., and G. Oseni. (2016). “Land Measurement Bias and its Empirical Implications: Evidence from a Validation Exercise”, World Bank Policy Research Working Paper 7597. Washington, DC: The World Bank. FAO (1982). Estimation of crop areas and yields in agricultural statistics, FAO, Rome. Fasbender, D., C. Lucau (2012). “Identifying and modelling sources of uncertainty in GNSS area measurements.” Proceedings of the 17th GeoCAP Annual Conference. Fermont, A., T. Benson (2011). "Estimating yield of food crops grown by smallholder farmers: A review in the Ugandan context.” Uganda Strategy Support Program Working Paper No. USSP 05, IFPRI. Goldstein, M. and C. Udry (1999) ‘Agricultural Innovation and Risk Management in Ghana’, Unpublished, Final report to IFPRI, IFPRI, Washington DC. Hofmann-Wellenhof, B., H. Lichtenegger, E. Wasle. (2008). “GNSS – Global Navigation Satellite Systems.” Springer-Verlag: New York, NY. IFAD (2010). Rural poverty report 2011 New realities, New challenges: New opportunities for tomorrow's generation. Rome, Italy: IFAD. Istat (2008). Farm Structure Survey 2009/2010. Survey on agricultural production methods 2009/2010. National Methodological Report (NMR). Rome: Istat. Joseph, George Gheverghese (1997). Geometry in India. In H. Selin. “Encyclopaedia of the History of Science Technology and Medicine in Non-Western Cultures”, Springer. Keita, N., and Carfagna. E. (2009). “Use of modern geo-positioning devices in agricultural censuses and surveys: Use of GPS for crop area measurement.” Bulletin of the International Statistical Institute, the 57th Session, (2009), Proceedings, Special Topics Contributed Paper Meetings (STCPM22), Durban. Keita, N., Carfagna, E., and Mu’Ammar, G. (2010). “Issues and guidelines for the emerging use of GPS and PDAs in agricultural statistics in developing countries.” The Fifth International Conference on Agricultural Statistics; Kampala, Uganda. Kelly, V. and C. Donovan (2008). “Agricultural Statistics in Sub-Saharan Africa: Differences in Institutional Arrangements and their Impacts on Agricultural Statistics Systems A Synthesis of Four Country Case Studies.” Michigan State University International Development Working Paper No. 95, East Lansing, Michigan. Kilic, T., Zezza, A., Carletto, G., and Savastano, S. (2013). “Missing(ness) in Action: Selectivity Bias in GPS-Based Land Area Measurements,” World Bank Policy Research Working Paper 6490. Washington, DC: The World Bank. Lyman, John, and John Wilfrid Wright. 2015. “Surveying”. In Encyclopaedia Britannica. Encyclopaedia Britannica Online Academic Edition. Encyclopædia Britannica Inc., 2015. Web. 20 Mar. 2015. 30    Macours, K. (2011). “Increasing Inequality and Civil Conflict in Nepal.” Oxford Economic Papers, vol. 63, n. 1, pp. 1-26. Muwanga-Zake, E. S. K. (1985). “Sources of Possible Errors and Biases in Agricultural Statistics in Uganda: A Review.” Kampala, Uganda: Institute of Statistics and Applied Economics, Makerere University. Palmegiani, G. (2009). Measuring cultivation parcels with GPS: a statistical evidence. Mimeo, LUISS University: Rome. Schoning, P., Apuuli, J. B. M., Menyha, E., Zake-Muwanga, E. S. K. (2005). “Handheld GPS equipment for agricultural statistics surveys: Experiments on area-measurement and geo-referencing of holdings done during fieldwork for the Uganda Pilot Census of Agriculture.” Statistics Norway Report 2005/29. Sud, U.C., Ahmad, T., Gupta, V.K., Chandra, H., Sahoo, P.M., Aditya, K., Singh, M., Biswas, A., and Indian Agricultural Statistics Research Institute (2015). Gap Analysis and Proposed Methodologies for Estimation of Crop Area and Crop Yield under Mixed and Continuous Cropping. Global Strategy Working Paper No. 4. FAO: Rome. Swetz, F. (1992). “The Sea Island Mathematical Manual: Surveying and Mathematics in Ancient China.” The Pennsylvania State University Press; University Park, Pennsylvania. UNDP (2014). Support to the Ministry of Lands and Rural Resettlement Action Plan 2014-2016 Action Plan, Harare, Zimbabwe. Van Diggelen, F. (2007). “GNSS Accuracy – Lies, Damn Lies and Statistics”, GPS World, 18(1): 1-4. World Bank, FAO, United Nations (2010). Global Strategy to Improve Agricultural and Rural Statistics. Report No. 56719-GLB. Washington DC: The World Bank. 31    VIII. Tables     Table 1 ‐ Closing Error and Plot Shape  Ethiopia Tanzania Nigeria Pooled Bias     Bias     Bias     Plot Shape Closing  (GPS‐CR,  Closing  (GPS‐CR,  Closing  Bias        Closing  (GPS‐CR,  N Error (%) acres) N Error (%) acres) N Error (%) (GPS‐CR) N Error (%) acres) < = 4 sides 662 2.32 0.01 358 2.03 0.00 71 1.46 ‐0.01 1091 2.17 0.01 5 ‐ 9 sides 875 2.19 0.00 980 2.00 0.01 215 1.63 ‐0.01 2070 2.04 0.00 > = 10 sides 228 2.09 0.01 570 1.97 0.01 199 1.68 ‐0.02 997 1.94 0.01 Total 1765 2.23 0.01 1908 2.00 0.01 485 1.62 ‐0.01 4158 2.05 0.00                     32    Table 2 – Determinants of Closing Error   OLS Regression Ethiopia Tanzania Nigeria Pooled CR Area (acres) 0.020 ‐ 0.291** 0.096 ‐0.032 2 CR Area 0.166 ‐ ‐0.005 ‐ 3 CR Area ‐0.039** ‐ ‐ ‐ Number of  Corners  ‐ 0.032*** 0.006 ‐0.004 ‐0.013*** Slope  (clinometer) 0.014*** ‐ ‐ ‐ Treecover: Partial ‐ 0.003 ‐0.052 ‐0.128 ‐0.143*** Heavy 0.183 ‐0.163 0.331 0.021 Weather: Mostly Cloudy ‐ Rainy ‐ 0.014 ‐ 0.128** 0.036 ‐0.048 Constant 2.298*** 2.118*** 1.597*** 2.238*** N 1765 1908 485 4158 R2 0.015 0.007 0.019 0.008 *p<.1; ** p<.05; *** p<.01      33      Table 3 – GPS vs Compass and Rope (CR) measures, by plot size classes  Means; Acres Ethiopia Tanzania Mean  Mean  Level  (CR) Bias /  Difference   Bias /  Difference   N GPS CR Bias Mean CR in means N GPS CR Bias Mean CR in means 1    (< 0.05 acres) 390 0.02 0.02 0.00 0% ‐ 45 0.04 0.04 0.00 ‐3% ‐ 2    (< 0.15 acres) 400 0.10 0.09 0.00 2% *** 631 0.11 0.11 0.00 2% *** 3    (< 0.35 acres) 365 0.24 0.24 0.01 3% *** 823 0.23 0.23 0.01 2% *** 4    (< 0.75 acres) 328 0.52 0.51 0.01 2% *** 326 0.51 0.49 0.02 4% *** 5    (< 1.25 acres) 182 0.98 0.96 0.02 2% *** 63 0.94 0.92 0.02 2% *** 6   (>= 1.25 acres) 100 1.91 1.89 0.02 1% ‐ 20 1.91 1.81 0.09 5% ‐ Total 1765 0.38 0.38 0.01 2% *** 1908 0.28 0.27 0.01 3% *** Nigeria Pooled Mean  Mean  Level  (CR) Bias /  Difference   Bias /  Difference   N GPS CR Bias Mean CR in means N GPS CR Bias Mean CR in means 1    (< 0.05 acres) ‐ ‐ ‐ ‐ ‐ ‐ 436 0.02 0.02 0.00 ‐1% ‐ 2    (< 0.15 acres) 21 0.11 0.11 ‐ 0.01 ‐7% *** 1052 0.10 0.10 0.00 2% *** 3    (< 0.35 acres) 73 0.24 0.25 ‐ 0.01 ‐4% *** 1261 0.24 0.23 0.01 2% *** 4    (< 0.75 acres) 129 0.52 0.53 ‐ 0.01 ‐2% ** 783 0.51 0.50 0.01 2% *** 5    (< 1.25 acres) 108 0.97 0.99 ‐ 0.02 ‐2% *** 353 0.97 0.96 0.01 1% ‐ 6   (>= 1.25 acres) 153 2.86 2.87 ‐ 0.01 0% ‐ 273 2.44 2.43 0.01 0% ‐ Total 485 1.30 1.31 ‐ 0.01 ‐1% * 4158 0.44 0.44 0.00 1% *** *p<.1; ** p<.05; *** p<.01   Note: Results for categories in which there are fewer than 20 observations are not reported.  The same is true for all tables presented in the paper.  34      Table 4 – Correlation Coefficient (GPS & CR)  Level  (CR) Ethiopia Tanzania Nigeria Pooled 1    (< 0.05 acres) 0.95 0.81 ‐ 0.95 2    (< 0.15 acres) 0.91 0.92 0.91 0.92 3    (< 0.35 acres) 0.90 0.95 0.90 0.93 4    (< 0.75 acres) 0.91 0.96 0.87 0.92 5    (< 1.25 acres) 0.93 0.95 0.91 0.92 6   (>= 1.25 acres) 0.98 0.96 1.00 1.00 Total 0.996 0.993 0.997 0.997       Table 5 – Bias and slope (measured with a clinometer)  Plot Slope Ethiopia / Difference in (degrees) N GPS CR Bias Mean CR means 0‐5 1076 0.36 0.35 0.00 1.3% *** 6 ‐ 15 562 0.44 0.43 0.01 2.2% *** > 15 127 0.37 0.36 0.01 2.4% ** Total 1765 0.38 0.38 0.01 1.7% *** *p<.1; ** p<.05; *** p<.01             35      Table 6 –Descriptive statistics for high‐bias observations    Bias = GPS ‐ CR (acres) Ethiopia Tanzania   Difference Difference GPS over ‐ GPS under‐ in Means GPS over ‐ GPS under‐ in Means   Relative reported reported Relative ( >10% Bias Relative reported reported Relative ( >10% Bias Bias > 10% by > 10% by > 10% Bias < 10% All Plots vs. < 10% ) Bias > 10% by > 10% by > 10% Bias < 10% All Plots vs. < 10% )   N: 542 305 237 1223 1765 388 251 137 1520 1908 % of Total Plot Sample 31% 17% 13% 69% 100% 20% 13% 7% 80% 100%   Average: CR Area (acres) 0.22 0.23 0.20 0.45 0.38 *** 0.21 0.23 0.17 0.28 0.27 *** GPS Area (acres) 0.23 0.28 0.16 0.45 0.38 *** 0.22 0.27 0.14 0.29 0.28 ***   Bias (GPS ‐ CR) 0.01 0.05 ‐0.03 0.00 0.01 *** 0.02 0.04 ‐0.03 0.01 0.01 *** |% Bias| 22.74 23.18 22.18 3.41 9.34 *** 16.95 16.61 17.57 4.13 6.74 ***   Closing Error (%)   2.24 2.31 2.14 2.22 2.23 ‐ 2.09 2.22 1.86 1.97 2.00 * Number of Corners 5.94 5.98 5.89 6.48 6.32 *** 8.09 8.25 7.80 8.35 8.30 ‐ Per : Area Ratio (GPS) 0.41 0.27 0.59 0.20 0.26 *** 0.19 0.16 0.24 0.15 0.16 *** Number of Satellites 17.0 17.1 16.8 17.4 17.3 *** 16.5 16.6 16.4 16.8 16.7 ** Walking Speed (m/min) 37.0 37.8 36.1 43.5 41.5 *** 42.1 42.4 41.6 44.3 43.9 *** Treecover: Partial (n) 139 77 62 292 431 207 132 75 739 946 (%) 26% 25% 26% 24% 24% ‐ 53% 53% 55% 49% 50% * Heavy (n) 39 20 19 50 89 26 16 10 78 104 (%) 7% 7% 8% 4% 5% *** 7% 6% 7% 5% 5% ‐ Nigeria Pooled Difference Difference GPS over ‐ GPS under‐ in Means GPS over ‐ GPS under‐ in Means Relative reported reported Relative ( >10% Bias Relative reported reported Relative ( >10% Bias Bias > 10% by > 10% by > 10% Bias < 10% All Plots vs. < 10% ) Bias > 10% by > 10% by > 10% Bias < 10% All Plots vs. < 10% ) N: 83 30 53 402 485 1013 586 427 3145 4158 % of Total Plot Sample 17% 6% 11% 83% 100% 24% 14% 10% 76% 100% Average: CR Area (acres) 0.94 1.31 0.72 1.38 1.31 ** 0.27 0.29 0.25 0.49 0.44 *** GPS Area (acres) 0.96 1.56 0.61 1.37 1.30 * 0.29 0.34 0.21 0.49 0.44 *** Bias (GPS ‐ CR) 0.02 0.25 ‐0.11 ‐0.02 ‐0.01 ** 0.01 0.05 ‐0.04 0.00 0.00 *** |% Bias| 18.96 23.60 16.33 3.93 6.50 *** 20.21 20.39 19.97 3.82 7.82 *** Closing Error (%) 2.05 2.68 1.69 1.54 1.62 *** 2.16 2.29 1.99 2.01 2.05 *** Number of Corners 9.17 9.50 8.98 10.24 10.06 ‐ 7.03 7.13 6.89 7.87 7.66 *** Per : Area Ratio (GPS) 0.13 0.08 0.15 0.09 0.09 *** 0.30 0.21 0.42 0.16 0.20 *** Number of Satellites ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ Walking Speed (m/min) 58.6 67.5 53.3 61.8 61.3 * 40.7 41.3 39.9 46.0 44.7 *** Treecover: Partial (n) 50 18 32 228 278 396 227 169 1259 1655 (%) 60% 60% 60% 57% 57% ‐ 39% 39% 40% 40% 40% ‐ 36    Heavy (n) 9 3 6 48 57 74 39 35 176 250 (%) 11% 10% 11% 12% 12% ‐ 7% 7% 8% 6% 6% ** *p<.1; ** p<.05; *** p<.01 Table 7 – Comparison of Self‐Reported and CR measures  Acres Ethiopia Tanzania Mean  Mean  Level  (CR) Bias /  Difference   Bias /  Difference   N SR CR Bias Mean CR in means N SR CR Bias Mean CR in means 1    (< 0.05 acres) 352 0.09 0.02 0.07 307% *** 44 0.32 0.04 0.28 661% *** 2    (< 0.15 acres) 392 0.27 0.09 0.18 188% *** 622 0.41 0.11 0.31 288% *** 3    (< 0.35 acres) 351 0.40 0.23 0.17 72% *** 816 0.62 0.23 0.39 173% *** 4    (< 0.75 acres) 316 0.66 0.51 0.15 29% *** 323 0.98 0.49 0.49 100% *** 5    (< 1.25 acres) 179 0.95 0.97 ‐0.02 ‐ 2% ‐ 63 1.53 0.92 0.61 66% *** 6   (>= 1.25 acres) 99 1.42 1.90 ‐0.47 ‐25% *** 20 2.05 1.81 0.24 13% ‐ Total 1689 0.47 0.38 0.09 23% *** 1888 0.65 0.27 0.38 143% *** Nigeria Pooled Mean  Mean  Level  (CR) Bias /  Difference   Bias /  Difference   N SR CR Bias Mean CR in means N SR CR Bias Mean CR in means 1    (< 0.05 acres) ‐ ‐ ‐ ‐ ‐ ‐ 397 0.12 0.03 0.09 371% *** 2    (< 0.15 acres) 21 0.15 0.11 0.03 30% ‐ 1035 0.35 0.10 0.25 247% *** 3    (< 0.35 acres) 73 0.39 0.25 0.14 55% *** 1240 0.55 0.23 0.32 136% *** 4    (< 0.75 acres) 129 0.79 0.53 0.26 50% *** 768 0.82 0.50 0.31 62% *** 5    (< 1.25 acres) 108 1.31 0.99 0.32 33% *** 350 1.16 0.96 0.20 21% *** 6   (>= 1.25 acres) 153 2.56 2.87 ‐0.30 ‐11% ‐ 272 2.11 2.44 ‐0.32 ‐13% ** Total 485 1.38 1.31 0.07 5% ‐ 4062 0.66 0.44 0.22 51% *** *p<.1; ** p<.05; *** p<.01         37    Table 8 – Determinants of Bias (GPS – CR)   OLS Regression Bias = GPS ‐ CR (acres) Ethiopia Tanzania Nigeria Pooled {Bias/CR }   {|Bias|/CR }     {Bias/CR }  {|Bias|/CR }   {Bias/CR }   {|Bias|/CR }     {Bias/CR }   {|Bias|/CR }     Dependent Variable: Bias |Bias| * 100 * 100 Bias |Bias| * 100 * 100 Bias |Bias| * 100 * 100 Bias |Bias| * 100 * 100 CR Area (acres) 0.049*** 0.040*** ‐9.186*** ‐9.013*** 0.055** 0.069*** ‐8.169*** ‐ 5.421*** ‐ 0.005 0.055*** ‐0.331 ‐ 0.015 0.011 0.050*** ‐ 1.898*** ‐ 3.344*** 2 CR Area ‐0.037*** ‐ 4.025*** 4.557*** ‐ ‐ 3.057** 2.406** ‐ ‐ 0.002*** ‐ ‐ ‐ 0.001*** ‐ 0.002*** 0.352** 0.507*** 3 CR Area 0.006*** ‐ ‐ 0.477** ‐0.589*** ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐0.012** ‐ 0.017*** Closing Error (%) 0.003** 0.001 0.862*** 0.156 0.002* 0.002** 0.494*** 0.282** 0.006 0.019*** 1.327*** 1.260*** 0.003* 0.004*** 0.762*** 0.370*** Number of Corners  0.001 0.001 ‐0.048 0.026 0.000 0.000 0.056 0.090** ‐ 0.001 0.000 ‐ 0.101* ‐ 0.009 0.000 0.000 ‐ 0.002 0.055* Per : Area Ratio (GPS) 0.002 ‐0.002 ‐13.927*** 10.551*** ‐ 0.003 0.036 ‐38.537*** 12.955*** ‐ 0.13 0.109 ‐58.831*** 43.445*** ‐ 0.006 0.001 ‐ 13.442*** 11.534*** Number of Satellites 0.000 ‐0.001 0.107 ‐0.114 0.000 0.000* 0.027 ‐ 0.004 ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ Slope  (clinometer) 0.000 0.000 0.018 0.045 ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ Treecover: Partial ‐0.005** 0.000 ‐0.074 0.318 ‐ 0.001 0.002 ‐ 0.729* 0.779** 0.013 ‐ 0.004 0.126 0.721 ‐ 0.001 0.001 ‐ 0.217 0.692** Heavy ‐ 0.006 0.009* ‐1.019 3.804** ‐0.006* 0.001 ‐1.339 1.116* 0.023 ‐ 0.012 ‐1.237 0.56 ‐ 0.002 0.004 ‐ 0.870 2.170*** Weather: Mostly Cloudy ‐ Rainy ‐0.005** 0.003 ‐0.085 1.206* 0.001 0.000 0.569 0.527* ‐ 0.029 0.016 ‐0.274 ‐ 0.773 ‐ 0.004 0.003 0.219 0.772** Constant ‐ 0.009 0.007 3.897 9.213*** ‐ 0.007 ‐0.021 8.317*** 3.867** 0.000 ‐ 0.033 2.856 0.107 ‐ 0.002 ‐0.008* 3.979*** 5.731*** Includes Country Dummies ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ Yes Yes Yes Yes N 1765 1765 1765 1765 1908 1908 1908 1908 485 485 485 485 4158 4158 4158 4158 R2 0.046 0.262 0.117 0.199 0.151 0.263 0.071 0.051 0.017 0.304 0.127 0.136 0.026 0.316 0.095 0.162 *p<.1; ** p<.05; *** p<.01             38      Table 9 – Determinants of High Bias   Probit (reporting marginal effects), Error Clustered on Enumerator ID Bias = GPS ‐ CR (acres) Ethiopia Tanzania Nigeria Pooled GPS over‐ GPS under‐ GPS over‐ GPS under‐ GPS over‐ GPS under‐ GPS over‐ GPS under‐ Percent  reported     reported     Percent  reported     reported     Percent  reported     reported     Percent  reported     reported     Dependent Variable: Bias > 10% by > 10% by > 10% Bias > 10% by > 10% by > 10% Bias > 10% by > 10% by > 10% Bias > 10% by > 10% by > 10% CR Area (acres) ‐0.899*** ‐0.941*** ‐0.339*** ‐0.674*** ‐0.871*** ‐0.036 0.006 ‐0.193*** 0.008 ‐0.275*** ‐0.257*** ‐0.119*** 2 CR Area 0.794*** 1.056*** 0.281*** 0.555** 0.642*** ‐ ‐ 0.060*** ‐ 0.069*** 0.100*** 0.023*** 3 CR Area ‐0.198*** ‐0.349*** ‐0.059** ‐0.117* ‐0.129** ‐ ‐ ‐0.005*** ‐ ‐0.004*** ‐0.010*** ‐0.001*** Closing Error (%) 0.003 0.01 ‐0.007 0.014* 0.018** ‐0.005 0.053*** 0.038*** 0.015** 0.017** 0.021*** ‐0.004 Number of Corners  0.006 0.002 0.004 0.005 0.006* 0.001 ‐0.001 ‐0.001 0.000 0.004 0.002 0.002 Per : Area Ratio (GPS) 0.173*** ‐0.144** 0.175*** 0.428*** ‐0.911*** 0.558*** 1.700*** ‐1.435*** 1.696*** 0.254*** ‐0.072** 0.191*** Number of Satellites ‐0.003 0.001 ‐0.004 ‐0.007 ‐0.003 ‐0.003 ‐ ‐ ‐ ‐ ‐ ‐ Slope  (clinometer) 0.002 0.002 0.000 ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ Treecover: Partial 0.002 0.005 0.003 0.043* 0.014 0.027* 0.052 0.016 0.044 0.029 0.012 0.019 Heavy 0.111* 0.042 0.078** 0.080* 0.031 0.045 0.047 ‐0.017 0.060 0.081** 0.026 0.055** Weather: Mostly Cloudy ‐ Rainy 0.026 0.009 0.016 0.044 0.035* 0.012 ‐0.026 0.007 ‐0.019 0.035* 0.023 0.012 Includes Country Dummies ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ Yes Yes Yes Psuedo‐R2 0.096 0.047 0.128 0.050 0.036 0.120 0.106 0.152 0.172 0.076 0.039 0.111 N 1765 1765 1765 1908 1908 1908 485 485 485 4158 4158 4158 Standard errors clustered at enumerator level *p<.1; ** p<.05; *** p<.01 39      Table 10 – Determinants of Bias (SR – CR)   OLS Regression Ethiopia Tanzania Nigeria Pooled {Bias/CR }     {|Bias|/CR }   {Bias/CR }    {|Bias|/CR }   {Bias/CR }     {|Bias|/CR }   {Bias/CR }     {|Bias|/CR }   Dependent Variable: SR‐ CR |SR‐ CR| * 100 * 100 SR‐ CR  |SR‐ CR| * 100 * 100 SR‐ CR  |SR‐CR| * 100 * 100 SR‐ CR  |SR‐CR| * 100 * 100 CR Area (acres) 0.043 0.278*** ‐775.385*** ‐724.752*** ‐0.070 0.821*** ‐1208.4*** ‐ 1172.4*** 0.112 0.454*** ‐ 25.308*** ‐ 10.551***  ‐ 0.014 0.341***  ‐ 255.718*** ‐229.597***   2 CR Area ‐0.247*** ‐ 425.846*** 408.956*** ‐ ‐ 0.863*** 966.7*** 942.4*** ‐ 0.127*** 0.017*** 0.966*** 0.532***   ‐0.101*** 0.022*** 42.172*** 39.887*** 3 CR Area 0.038*** ‐ ‐56.161*** ‐54.541*** ‐ 0.264*** ‐ 209.4*** ‐ 202.0*** 0.004*** ‐ ‐ ‐ 0.003*** ‐ ‐ 1.457*** ‐1.392*** Number of  Corners ‐0.011*** 0.002 ‐10.179*** ‐8.710*** 0.027*** 0.022*** 7.738*** 7.490*** 0.048*** 0.017 1.968** 1.517**  0.026*** 0.014*** ‐ 0.110 ‐0.156 Distance  from dwelling 0.012*** 0.014*** 5.047*** 4.902*** 0.000 0.000 ‐0.144 ‐ 0.153 ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ⱡ Number cultivated plots in HH ‐0.014*** ‐0.011*** ‐4.841* ‐ 3.735 ‐0.002 0.000 2.129 2.024 0.061 0.088*** 0.799 2.861 ‐0.016*** ‐ 0.007** ‐ 2.768 ‐1.726 Slope  (clinometer) 0.001 0.001 ‐1.865 ‐ 1.656 ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ Soil Quality (SR): Fair ‐0.090*** ‐0.076*** ‐69.345*** ‐66.665*** ‐0.032 ‐ 0.023 2.226 3.229 ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ Poor ‐0.049 ‐0.057* ‐130.936*** ‐123.089*** 0.120 0.102 43.172 42.997 ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ Property Rights⁰ ‐0.012 ‐ 0.024 ‐32.988* ‐ 33.235* 0.010 ‐ 0.002 ‐2.068 ‐ 3.612 0.082 ‐ 0.133 8.284 ‐7.938 0.017 ‐ 0.033** ‐ 9.745 ‐13.003 Treecover: Partial 0.131*** 0.094*** 124.283*** 117.589*** 0.064*** 0.039* 5.838 3.351 ‐ 0.273*** ‐ 0.114 ‐ 39.153*** ‐ 21.355** 0.03 0.045*** 46.016*** 45.218*** Heavy 0.162*** 0.165*** 76.105** 74.343** ‐0.019 ‐ 0.024 ‐ 11.779 ‐ 12.629 0.518** 0.415** ‐ 9.628 ‐2.723 0.243*** 0.172*** 47.628*** 43.547** HH Head Characterisitcs: Female ‐0.038 ‐ 0.028 ‐28.272 ‐28.918 ‐0.029 ‐ 0.019 15.763 15.81 0.348 0.117 8.628 ‐5.546 ‐ 0.008 ‐ 0.025 ‐ 4.621 ‐6.532 Yrs education 0.021*** 0.016*** 9.683** 8.871** ‐0.002 ‐ 0.001 ‐1.735 ‐ 1.734 0.019 ‐ 0.002 0.551 ‐0.059 0.005 0.000 ‐ 0.352 ‐0.558 Age ‐0.001 ‐ 0.001 ‐0.271 ‐ 0.249 0.001* 0.001** 0.744 0.776 0.010** 0.003 0.520 0.14 0.001 0.000 ‐ 0.102 ‐0.129 Literate ‐0.102*** ‐0.114*** ‐23.008 ‐ 23.86 ‐0.052 ‐ 0.051* ‐ 15.767 ‐ 15.068 0.397 0.074 31.090* 9.958 0.024 ‐ 0.031 2.714 ‐1.544 Constant 0.317*** 0.231*** 456.555*** 452.797*** 0.158** 0.053 338.02*** 335.02*** ‐ 1.186*** ‐ 0.317 8.794 66.538***   ‐ 0.066 0.057 253.352*** 270.263*** Includes Country Dummies ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ Yes Yes Yes Yes N 1689 1689 1689 1689 1737 1737 1737 1737 485 485 485 485 3931 3931 3931 3931 R2 0.231 0.279 0.206 0.183 0.073 0.150 0.170 0.160 0.529 0.625 0.111 0.032 0.386 0.510 0.114 0.090 *p<.1; ** p<.05; *** p<.01   ⱡ In Tanzania, number of plots owned or cultivated   ⁰Property rights defined here  as: HH has title  or certificate, HH has ability to sell  land, or  HH can use  land as collateral.     40        Table 11 – Standard and Non‐Standard Area Units  Acres Ethiopia Nigeria Experiment Standard Units Non‐Standard Units U Standard Uni its U Non‐Standard Units Level (GPS) Mean Bias Mean Bias Mean Bias Mean Bias / Mean / Mean Difference / Mean / Mean Difference N SR GPS GPS N SR GPS GPS in Bias^ N SR GPS GPS N SR GPS GPS in Bias 1 (< 0.05 acres) ‐ ‐ ‐ ‐ 375 0.09 0.02 276.1% ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ 2 (< 0.15 acres) 31 0.67 0.09 643.6% 356 0.24 0.10 150.0% *** ‐ ‐ ‐ ‐ 25 0.15 0.11 38.1% ‐ 3 (< 0.35 acres) 24 1.02 0.25 302.4% 349 0.36 0.23 53.6% *** ‐ ‐ ‐ ‐ 69 0.33 0.25 33.9% ‐ 4 (< 0.75 acres) 20 1.32 0.52 152.4% 309 0.60 0.52 16.3% *** 28 1.49 0.53 181.4% 112 0.67 0.54 24.4% *** 5 (< 1.25 acres) ‐ ‐ ‐ ‐ 163 0.87 0.96 ‐9.8% ‐ 25 2.24 1.01 121.2% 71 1.06 0.99 7.3% *** 6 (>= 1.25 acres) ‐ ‐ ‐ ‐ 93 1.19 1.71 ‐30.3% ‐ 76 3.51 3.24 8.4% 76 1.61 2.51 ‐35.9% ** Total 120 1.16 0.63 84.2% 1645 0.42 0.37 12.9% *** 131 2.80 2.19 28.2% 354 0.85 0.97 ‐12.2% ***   41    IX. Figures   Figure 1 – Plot Size Distribution  Malawi Integrated Household Survey 2010/11 Plot Area Distribution SR GPS 30 P e rce n t 20 10 0 0 1 2 3 Acres *Limited to plots <=3 acres                         42      Figure 2 – Time taken for GPS and CR measurement by plot size (minutes)        43    Figure 3 – Scatter plots of Compass and Rope vs GPS (left) and Self‐Reported (right) land area measures, acres  Ethiopia 6 6 4 4 CR 2 2 0 0 0 2 4 6 0 2 4 6 GPS SR Tanzania 3 3 2 2 CR 1 1 0 0 0 1 2 3 4 0 2 4 6 8 GPS SR Nigeria 20 20 15 15 10 10 CR 5 5 0 0 0 5 10 15 20 0 5 10 15 20 GPS SR Note: red line is line of equality between measurements.    44    Figure 4 – Scatter plots of relative (%) and absolute (acres) bias over plot size (acres)  Plot Size & Relative Bias (%) Ethiopia Tanzania Nigeria 100 100 100 50 50 50 0 0 0 -50 -50 -50 -100 -100 -100 0 2 4 6 0 1 2 3 0 5 10 15 20 CR(acres) CR(acres) CR(acres) Plot Size & Absolute Bias Ethiopia Tanzania Nigeria 1 1 1 .5 .5 .5 Bias (acres) 0 0 0 -.5 -.5 -.5 -1 -1 -1 -1.5 -1.5 -1.5 0 2 4 6 0 1 2 3 0 5 10 15 20 CR(acres) CR(acres) CR(acres)                 45    Figure 5 – Graphic representation of land area coefficients found in Table 8  Graphs consider only land area terms and constant       Ethiopia Bias (acres) | Bias | (acres) Relative Bias (%) | Relative Bias | (%) .0 1 4 9 .1 .0 8 8 0 2 .0 6 7 -. 0 1 .0 4 6 0 -. 0 2 .0 2 5 -2 -. 0 3 0 4 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 Plot Size (acres, CR) Plot Size (acres, CR ) Plot Size (acres, CR) Plot Size ( acres, CR) Tanzania .1 5 .2 10 8 .15 8 6 .1 .1 4 6 .0 5 .05 2 4 0 0 0 2 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 Plot Size (acres, CR) Plot Size (acres, CR) Plot Size (acre s, CR) Plo t Size (acres, CR) Nigeria 0 3 .3 .1 -. 0 1 2 .2 .0 5 -. 0 2 1 -. 0 3 .1 0 -. 0 4 0 0 -. 0 5 - .0 5 -1 0 2 4 6 8 10 0 2 4 6 8 10 0 2 4 6 8 10 0 2 4 6 8 10 Plot Size (acres, CR) Plot Size (acres, CR) Plot Size (acres, CR) Plot Size (acres, CR) Pooled .0 3 .2 4 6 .1 5 4 .0 2 3 .1 2 .0 1 2 .0 5 0 0 0 -2 1 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 Plot Size (acres, CR) Plot Size (acres, CR) Plot Size (acres, CR) Plot Size (acres, CR) *Specifications in which all coefficients on land area are not statistically significant at the 10% level are highlighted in gray. 46    Figure 6 – GPS and Compass and Rope Plot Outlines  Red outlines were constructed from compass and rope measurements.  Gray shapes are GPS plot outlines.     a) and b) not high‐bias plots, included for reference   c) a large plot with 19 sides and obvious closing error   d) CR shape simplification – number of vertices in GPS is twice the number of CR corners   e) appears to be enumerator/field delineation error, with CR shape simplification   f) appears to be error in CR bearing (front and back bearings were likely switched)    47    X. Annex Table   Annex Table 1 – Summary Statistics on Weather Conditions, Satellite Acquisition, and Canopy Cover   Acres Ethiopia Tanzania Mean Bias / Difference in Mean Bias / Difference in N GPS CR Bias Mean CR means N GPS CR Bias Mean CR means Weather Conditions at Measurement Clear/Sunny 576 0.50 0.49 0.01 1.9% *** 800 0.30 0.29 0.01 3.3% *** Mostly Clear 733 0.33 0.33 0.01 2.0% *** 217 0.28 0.27 0.00 1.3% ** Partly Cloudy 334 0.32 0.31 0.00 0.6% ‐ 705 0.25 0.24 0.01 2.8% *** Mostly Cloudy 94 0.34 0.34 0.00 ‐0.1% ‐ 76 0.22 0.21 0.01 3.1% ** Completely Cloudy ‐ ‐ ‐ ‐ ‐ ‐ 41 0.26 0.25 0.00 2.0% ** Rainy ‐ ‐ ‐ ‐ ‐ ‐ 69 0.29 0.28 0.01 2.7% *** Tree Canopy Cover None 1245 0.40 0.39 0.01 2.0% *** 858 0.27 0.26 0.01 3.2% *** Partial 431 0.35 0.35 0.00 0.7% ‐ 946 0.28 0.27 0.01 2.8% *** Heavy 89 0.28 0.28 0.00 1.0% ‐ 104 0.34 0.33 0.01 1.6% * GPS Satellites Acquired <= 15 305 0.23 0.22 0.00 1.6% ‐ 488 0.31 0.30 0.01 2.9% *** 16 ‐ 19 1204 0.40 0.39 0.01 1.8% *** 1266 0.27 0.26 0.01 2.9% *** > = 20 256 0.49 0.48 0.01 1.2% ** 154 0.24 0.24 0.01 2.6% *** Total 1765 0.38 0.38 0.01 1.7% *** 1908 0.28 0.27 0.01 2.9% *** Nigeria Pooled Mean Bias / Difference Mean Bias / Difference in N GPS CR Bias Mean CR in means N GPS CR Bias Mean CR means Weather Conditions at Measurement Clear/Sunny 317 1.31 1.32 ‐0.01 ‐0.9% ** 1693 0.56 0.55 0.01 1.0% *** Mostly Clear 101 1.35 1.34 0.01 0.5% ‐ 1051 0.42 0.41 0.01 1.5% *** Partly Cloudy 62 1.13 1.16 ‐0.03 ‐2.6% ‐ 1101 0.32 0.32 0.00 1.0% * Mostly Cloudy ‐ ‐ ‐ ‐ ‐ ‐ 174 0.30 0.30 0.00 0.5% ‐ Completely Cloudy ‐ ‐ ‐ ‐ ‐ ‐ 54 0.25 0.25 0.00 1.3% ‐ Rainy ‐ ‐ ‐ ‐ ‐ ‐ 85 0.34 0.33 0.01 2.1% * Tree Canopy Cover None 150 0.92 0.94 ‐0.02 ‐2.1% ‐ 2253 0.38 0.38 0.01 1.6% *** Partial 278 1.42 1.43 ‐0.01 ‐0.6% ‐ 1655 0.49 0.48 0.00 0.7% ** Heavy 57 1.66 1.66 0.00 ‐0.1% ‐ 250 0.62 0.62 0.00 0.4% ‐ GPS Satellites Acquired <= 15 NA NA NA NA NA NA NA NA NA NA NA NA 16 ‐ 19 NA NA NA NA NA NA NA NA NA NA NA NA > = 20 NA NA NA NA NA NA NA NA NA NA NA NA Total 485 1.30 1.31 ‐0.01 ‐0.9% * 4158 0.44 0.44 0.00 1.1% *** *p<.1; ** p<.05; *** p<.01   48