Policy Research Working Paper 8828 Auction Length and Prices Evidence from Random Auction Closing in Brazil Alexandre Borges de Oliveira Abdoulaye Fabregas Mihály Fazekas Governance Global Practice April 2019 Policy Research Working Paper 8828 Abstract Electronic reverse auctions are the most used competitive are run with a wide set of fixed effects as well as covariates method for procurement of goods and non-consulting capturing competition. The findings point at alternative services by the Federal Government of Brazil. These auc- strategies to optimize auction design: simple actions such as tions are closed randomly, which perfectly satisfies fairness increasing the average and minimum length of the random considerations but may be suboptimal from an efficiency phase can result in 2.8 and 0.6 percent price savings, respec- perspective. There are concerns that tenders are closed too tively, or R$540 million and R$116 million per year; or early and randomness favors bidders with algorithmic bid- more complex designs such as setting the length to the ding software, leading to high prices. Hence, this paper maximum for the random phase if there are 15 bidders investigates what would happen if the random closing rule or more can yield 2.6 percent or R$ 500 million a year in was replaced by another rule. The paper uses the complete price savings, or doing the same if a large discount is placed data set of completed electronic actions in 2015–17 com- within three minutes to closing can yield 1.1 percent lower prising 112 million bids for 0.9 million items purchased. prices or R$ 210 million a year in savings. Exploiting the random closing rule, simple OLS models This paper is a product of the Governance Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at aoliveira@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Auction Length and Prices: Evidence from Random Auction Closing in Brazil  Alexandre Borges de Oliveira (World Bank), Abdoulaye Fabregas (World Bank), Mihály Fazekas (Central European University and Government Transparency Institute) JEL classification: C53, D44, C57, C55, C61 Keywords: auctions, Brazil, high frequency bidding, random closing 1. Introduction Reverse auctions were introduced to the Brazilian public procurement system in 1997 and almost instantly became the favorite method of government agencies and suppliers for its speed when compared to traditional procurement methods: you could award a contract in about two weeks. The possibility of submitting electronic bids for the first time amplified the success of reverse auctions in Brazil. Not only government agencies and suppliers were happy with them, but also the public at large and oversight agencies touted its transparency, as the entire process can be followed online by anyone with a computer and internet connection. In 2018, reverse auctions were the most used competitive procurement method by federal government agencies in Brazil by far. But, more recently, this impressive history of reverse auctions in Brazil has been threatened by the rise of high-frequency bids, which almost always lower the price by only decimals. It is widely believed that high- frequency bids are placed by algorithms, or “bots” and there are strong concerns that they are impacting not only prices in public procurement but also unsettling the playing field. Once known for breaking monopolies and oligopolies in public procurement, reverse auctions are now on the spotlight over concerns that bidders with the capability of placing bids using bots have a leg up over bidders that use labor power to place their bids. The widespread use of bots overwhelms the IT infrastructure and makes it hard for humans to put a bid through when the system must handle a high frequency of bids placed by machines. At the core of the problem is the approach to wrap up a reverse auction in the Federal Government of Brazil: reverse auctions are closed randomly by computers, following a period of time that can vary between one second and 30 minutes. This period, known as “random phase” in Brazil, is when fierce competition for contracts take place and bidders that do not have the capability of placing bids with bots claim that they do not see their bids through among the high frequency of bids placed by bots. In addition, government agencies and the public now question the wisdom of a random closing to reverse auctions: a great deal of processes are closed at the heat of the bidding or within very few seconds, sparking concerns that the random closing might be cutting short bidding that would otherwise lower prices for the government. Some governments in Brazil have started taking action against high-frequency bids. For instance, the government of the State of São Paulo will automatically extend the time allowed for submission of bids if one is received within three minutes prior to closing. The state also requires new bids to lower the bidder’s previous bid by a minimum amount, usually 0.5% of the previous price – this is to counter bids that lower the price by cents, just enough to get ahead of the competition. Given the central importance of auction length, we investigate what would happen if the random closing rule was replaced by another rule? Would government agencies get lower prices under different scenarios? This paper will present both a conceptual framework and novel empirical evidence to answer these questions. We used data on all electronic reverse auctions done by the Federal Government of Brazil during 2015-2017 which encompassed over 112 million bids for 7 million purchased items. Our research revealed that longer random phases resulted in lower prices, but the effect tapers off at the tail end of the 30-minute period. Thus, it seems that closing the auction randomly does not produce optimal prices, especially when closing within seconds. Unsurprisingly, we found that the size of the discount had a material effect on prices; more specifically, larger discounts within the last three minutes of the random phase resulted in lower prices. In addition, the size of the discounts mattered more for the final auction price than the frequency of discounts. This appears to indicate that the high-frequency bidding attributed to the use of bots did not help to achieve the lowest prices, which came when discounts were larger rather than more frequent. Further to these results, we also found that a larger number of bidders resulted in lower 2  prices, albeit with a non-linear effect. In fact, when there were many bidders in the auction, the length of the random phase had little to no effect at all on prices. Based on these findings, some related to the length of the random closing, some to the interaction of random phase length and indices of competition such as the number of bidders, we propose a mix of policy options to maximize results in electronic reverse auctions: simple actions as increasing the average and minimum length of the random phase can result in 2.8% and 0.6% price savings respectively. More elaborate and adaptive procurement tactics, such as automatically setting the maximum time allowed for the random phase upon certain events can produce robust savings: if there are 15 bidders or more, setting the maximum length for the random phase can yield 2.6% price savings, and doing the same if a large discount is placed within three minutes to closing can yield 1.1% price savings. The paper is organized as follows: the next section explains the debut and evolution of reverse auctions in the context of public procurement in Brazil and why it became such a darling of public procurement. Then, we describe the random closing process, which is at the heart of today’s issues with electronic reverse auctions. Next, the paper presents the main findings of a review of the literature relevant to this research. After the literature review, the paper presents details of the data available for this work followed by the methodology used in the research. Finally, the paper will wrap up with one section to present the main results followed by the conclusions and policy recommendations. 2. Reverse auctions in the Brazilian procurement system The Brazilian government procurement system is characterized by three complementary sets of processes and procedures. Procurement of civil works and consulting services is primarily carried out via the two- envelope system set by law 8.666 enacted in 1993. In addition, the “RDC”, Regime Diferenciado de Contratações or “Special Procurement Regime”, was introduced in 2011 specifically for procurement of strategic construction projects, including those related to the world cup and Olympic games, as well as key works projects in the health sector and in the “growth acceleration program” or PAC. And then, there is the reverse auction system, regulated by law 10.520 issued in 2002, which is the default method for procurement of goods and non-consulting services. The latter is the main focus of this paper. Specifically, it will address electronic reverse auctions, which is the most important mechanism for procurement of goods and non-consulting services in Brazil. Face-to-face reverse auctions are not considered in this paper, and neither is procurement of construction works of any kind. Electronic reverse auctions are by far the most used competitive method for procurement of goods and non- consulting services by the Federal Government of Brazil: in 2018, 95% or R$ 19.3 billion (about US$ 2.7 billion) of competitive procurement for goods and non-consulting services were awarded through electronic reverse auctions. There is also a substantial R$ 27.5 billion (about US$ 7.9 billion) of goods and non- consulting services that was awarded using non-competitive methods, named “dispensa” and “inexegibilidade”. Reverse auctions were first used in Brazil by Anatel, the telecommunications regulatory agency, in 1997 and its use has spread dramatically since then. Savings achieved by the early adopters prompted the federal government to adopt the use of reverse auctions by all federal agencies in 2000 and then expand its use to all state and municipal governments in 2002. Then electronic reverse auctions became mandatory for all federal government agencies in 2005, unless there was a solid justification to do a face-to-face reverse auction. Since then a large number of electronic reverse auction systems in Brazil have been implemented; besides the federal government, most states developed their own electronic reverse auction systems such as the system in São Paulo as well as some municipal governments. In addition to systems developed by 3  government administrations, private sector companies also offered government agencies electronic platforms to carry out reverse auctions, most notably the one from Banco do Brasil, which is used by several subnational governments, and the platform offered by the commodities exchange. This paper only covers electronic reverse auctions implemented by federal agencies using the Comprasnet platform. It does not include any procurement done by state and municipal governments nor procurement done by federal agencies that do not use Comprasnet, most notably companies owned fully or partially by the government, such as Petrobrás, Banco do Brasil and others. Regulations for reverse auctions limited their use only to off-the-shelf goods and non-consulting services. Their use for procurement of construction works is expressly forbidden. An attempt by government officials to include works failed around 2004. But reverse auction regulations introduced a novelty by not limiting its use to any monetary threshold: as long as it is procurement of off-the-shelf goods or non-consulting services, reverse auctions are the default procurement method regardless of the contract value. A. The advantages of electronic reverse auctions: Speed, transaction costs, and transparency The speed of procuring through reverse auctions is one of its most celebrated advantages over traditional procurement methods. During 2014-2016, half of the reverse auctions were completed in just 11 days and three-quarters of them in just 13 days. Even very large contracts, over R$ 30 million, which typically take longer to award, were completed in 11 days or less half of the time. Traditional competitive procurement methods in Brazil such as concorrência or open bidding, and tomada de preços (a restricted bidding) take months to be completed. The speed of reverse auctions can be credited to a procedure named inversão das fases or “reversal of phases”, which was a major breakthrough in procurement introduced by reverse auctions. The reversal of phases significantly cut down the time spent evaluating bids as well as lowered the number of complaints, especially those which only aimed at delaying the process. Traditional procurement methods from law 8.666 require bids to be presented in two separate envelopes. The first envelope contains the technical proposal along with legal, financial, fiscal and technical qualification documents, while the second envelope solely contains the price. Consequently, bids are evaluated in two stages. First, only the envelopes containing the technical proposal and qualifications documents are opened and assessed by government officials with the aim of determining which of the bidders are qualified to execute a given contract. This is done for all envelopes received, regardless of what the corresponding price envelope might contain (reviewing a single technical proposal is a time-consuming task as it is dense in information). Bidders that do not meet the requirements are disqualified at this first stage and their price envelopes are returned unopened. Thus, the first stage acts like a filter through which only qualified bidders—as per the requirements set forth in the bidding documents—will move forward to the second stage. Next, the price envelopes of those bidders that were successful at the first stage are opened and the qualified bidder with the lowest evaluated price is automatically awarded the contract. Besides having to go through the entire documentation of all bids, traditional procurement methods are known to be prone to complaints, often many of them in a single process, as any bidder that is rejected will try to reinstate its bid and other bidders will put their lawyers to work with the goal of trying to eliminate as many competitors as possible before prices are known. The dynamics of evaluating documents without knowing prices contribute to a legalistic approach to bid evaluation. The combination of all these factors results in a long process to get through bid evaluation to traditional procurement methods. 4  Electronic reverse auctions changed these dynamics by reversing the phases and starting with the price. Then, only the documentation of the lowest price bidder is evaluated and reviewed. Besides the obvious element that reviewing the documentation of one bid – the winner - is faster than reviewing for all of them regardless of ranking, the number of complaints that so significantly delay a process came down substantially, as low-ranked bidders will not make an effort and spend money for a contract they are unlikely to get. Knowing prices upfront has proven to be a factor in speeding up procurement processes and lowering the number of frivolous complaints. A second tangible advantage of reverse auctions over traditional procurement methods is that it could be implemented electronically from end-to-end. This capability was celebrated by all stakeholders: government agencies praised its quickness, suppliers liked the ability of bidding for contracts all over the country and at a lower cost, and oversight agencies and the public at large stressed the transparency of reverse auctions. The entire process can be followed in real time, online if you have a computer with internet access. Furthermore, electronic reverse auctions automatically generate and publish all relevant documents and information online. A full electronic process can only work effectively for the kind of items purchased using reverse auctions, off-the-shelf goods and non-consulting services, as bids are simple and easy to evaluate. It is much harder, or even not possible, to achieve the same level of automation for construction works or complex equipment that requires much more careful and detailed analysis of technical documentation and understanding the cost-quality trade-offs. Electronic reverse auctions do not require bidders to submit bid securities, that is money which is withheld by the government until contract signature as a deterrent for bidders to fail to sign a contract or to sell its place in the ranking. Bid securities are monies that are frozen for quite a while for suppliers and thereby a cost to compete for government contracts. Eliminating bid securities made it cheaper for bidders to compete for government contracts and the possibility of submitting electronic bids lowered participation costs even further, as bidders did not have to travel to submit their bids in person and did not have to put together large hard copies of bids. B. Auction design and the random closing set‐up An electronic reverse auction kicks off with publication of an advertisement for bids at the Comprasnet website. Bidders will have at least eight days to prepare and submit their bids electronically through the system. At the date and time indicated in the bidding documents, an auctioneer, who is a government official certified to carry out this type of procurement, will open up the session for bids. The starting price of each bidder will be the price quoted in the bids they uploaded to the system. Bidders will have a fixed period which is set by the buyer between one and 60 minutes to lower their prices, after which a random phase kicks in to close the reverse auction. The random phase is controlled by the system and it can vary from one second to 30 minutes. The widely held perception of the government is that the random phase is when the real bidding takes place in electronic reverses auctions for federal procurement. This is also confirmed by descriptive statistics: in one of the few existing studies conducted on Comprasnet electronic reverse auctions, Celiktemur and Szerman (2012) observe that in a typical Comprasnet auction, a bidder places on average 1.95 bids, of which 1.36 bids, representing 70% of the total, are placed in the random phase. While it is possible to argue that bidders should place their best prices during the fixed 30 minutes that precede the random phase to avoid being cut short, the more intensive bidding observed in the random phase, however, suggests that bidders ‘save’ their best offers to the random phase, probably because they want to lower their price as little 5  as possible so as to maximize their profits. Malaga et al. (2010) provide theoretical grounding to this observed bidding behavior by establishing that, in a random close setting, the bidder with the highest bidding frequency has a higher chance of winning the auction.1 This assertion implies that it is therefore rational for bidders to delay bidding during the constant phase and to start bidding intensively during the random phase to maximize their probability of winning the auction. 3. Literature review Economic research on electronic reverse auctions remains scant, in part because the use of this procurement method is still relatively incipient, and because detailed transactional data from existing electronic reverse auction systems are rarely available to the public. In addition, their analysis requires specific computational infrastructure given the large volumes of data involved. Traditional auctions, on the other hand, are one of the oldest forms of economic institutions and have therefore been subject to extensive research. Auction theory, the branch of applied economics dealing with auction markets, provides some insights that are readily transposable to electronic reverse auctions given their very similar, but inverse, features. Over the past 15 years, a growing and equally relevant body of empirical work has also emerged on Internet electronic commerce (e-commerce) auctions, taking advantage of the large volume of transactional data made available by the leading e-commerce web platforms such as eBay or Amazon. The auction closing rule constitutes an important feature of the design of any auction system and has thereby attracted attention from researchers and practitioners alike. In fact, a diversity of auction ending designs is currently used in electronic auctions in an effort to strike the right balance between maximizing auction revenue and minimizing the costs associated with long auction processes. Broadly speaking, the existing literature distinguishes three types of auction closing rules in second-price ascending (English) auctions: (i) the hard close (HC), corresponding to a predefined time limit known to all bidders at the start of the auction; (ii) the soft close (SC), which automatically extends the auction time for a predetermined amount of time when new bids are received near the end of the auction; and (iii) the random close (RC), which assigns a time limit to each auction based on a random distribution and prevents bidders from anticipating the auction closing time. Several papers have examined the effect of the closing rule on bidders’ behavior. Cassady (1967) describes a conventional auction with a time limit controlled by an hour glass and observes that bidders wait for the sand to nearly run out before they start placing their bids. This practice, known as late bidding or “sniping”, can still be observed in modern-day Internet electronic auctions and has since then been confirmed by empirical analysis. Ockenfels and Roth (2003) analyze data from the two leading Internet auction platforms, eBay and Amazon, which use respectively a hard close (eBay) and a soft close (Amazon): they find that the difference in auction ending rules is sufficient to explain the late bidding observed in the data. This is consistent with Malaga et al. (2010), which suggest that a random close design can discourage late bidding, enabling all bidders to compete more fairly and potentially increasing the revenue of the seller. 1  Malaga et al. (2010) imagine a random close auction with two bidders, A and B, in which A bids with a time delay  of hA and B with a time delay of hB. They posit that the probability that B wins the auction is equal to hA divided by  hA plus hB. In equation form: Pr[B wins] = hA/[hA+hB].  6  Random close designs, such as the one used in Comprasnet’s e-reverse auctions, are not common in modern-day online auctions. However, they have far predated the emergence of the Internet: Cassidy (1967) provides several examples of candle auctions, a type of auctions used in Great Britain in the 17th and 18th centuries, in which the random closing time is determined by a lit candle and the auction ends when the flames die. In Comprasnet, the auction duration is randomly determined by the system based on a uniform distribution. According to Celiktemur and Szerman (2012), the fact that the random close is rarely observed outside Comprasnet could be due in part to IBM’s patent of the system. Using transactional data from Comprasnet, Celiktemur and Szerman (2012) report interesting patterns of bidding behavior in Comprasnet’s random close auctions. They find that (i) bidders tend to defer bidding to the end of the auction; (ii) a large part of the auctions is resolved early; (iii) bid increments are usually small; and (iv) large increments are more likely to occur early in the auction. They conclude that random close designs do not necessarily prevent late bidding and consider that random close mechanisms may lead to suboptimal prices when auctions are closed in the heat of bidding. Finally, they suggest that this effect could be offset or outweighed by increased participation since randomness increases the chances of weak bidders to win the auction. Our paper aims to complement previous work by: (i) evaluating how Comprasnet’s random close design impacts the prices of goods and services purchased by the Federal Government of Brazil and (ii) estimating the savings that could be generated if the closing rule was set differently compared to the current random setting. Building upon a framework for measuring fiscal efficiency in public procurement that has been tested and refined through implementation in eight Latin American countries, including Brazil (World Bank, 2017), we propose a model that extends the specifications used in the previous studies, using additional controls and taking into account the interactions between competition and auction length. To the best of our knowledge, this paper represents the first attempt to quantify the effects of a random close design on the outcomes of electronic reverse auctions. It is also the first estimation of the potential savings that could be generated by using big data and artificial intelligence to optimize the auction closing rule. A. Hypotheses  To answer the research question presented in the introduction, we test several hypotheses relating to and addressing the existing literature. This enables us to evaluate several auction ending designs and to estimate the impact of each of these variants on the prices of the goods purchased through Comprasnet’s electronic reverse auctions. H1: Auctions with a longer random phase result in lower unit prices. According to this premise, we expect longer random phase durations to be associated with lower prices since longer times increase the opportunities for bidders to outbid their competitors and subsequently reduce the probability of ending the auction in the heat of bidding. Conversely, auctions with short random phase are at higher risk of closing before the market clearing price (in this case the lowest price acceptable to the bidders) is reached, impeding the government from reaping all the benefits from the competition. In order to test this first hypothesis, we examine the independent effect of the random phase duration on unit prices. This estimation of potential savings enables the pricing of the impacts of a simple variant of Comprasnet’s random close with a longer duration of the random phase. 7    H2: Auctions with a longer random phase result in lower unit prices, especially when competition is strong with many bidders present. Under this hypothesis, we posit that the length of the random phase results in lower prices especially when competition is sufficiently strong in the auction. In a low competition setting, with few bidders participating in the auction, an extended random phase would not necessarily translate into better prices for the government. The specification used to test this hypothesis allows us to simulate a “smart” variant of the random ending rule, whereby the duration of the random phase is determined by a specific parameter reflecting the level of competition attained in the auction, in this case the number of bidders actively taking part in the auction. H3: Auctions with longer random phase result in lower unit prices, especially when competition is intense with a large number of bids submitted. This hypothesis is similar to the previous one but looks at the interacted effects of the duration of the random phase and bidding intensity in terms of the number of bids submitted in the last minutes of the random phase. We consider several resolutions for the independent variable measuring the number of bids submitted: 5 minutes, 3 minutes and 30 seconds before the end of the random phase. This variant allows us to evaluate another “smart” design of the random ending rule in which the duration of the random phase is determined by the number of bids submitted before the close of the random phase. However, if the frequency of bidding is driven by bots or algorithms, then we can expect this hypothesis not to be supported by the data. H4: Auctions with a longer random phase result in lower unit prices, especially when competition is intense with larger bid discounts. This hypothesis takes into consideration the potential distortion introduced by the widespread use of bots in Comprasnet’s electronic reverse auctions. Since bots are configured to conduct high frequency bidding and to outbid the previous best bid by the minimum amount allowed, we postulate that extending the random phase based solely on the number of bidders taking part in the auction may not be optimal. Instead what is crucial for auction extension is the discounts offered. To test this hypothesis, we look at the interaction between the duration of the random phase and an aggregate reflecting the bid discount obtained in the last minutes of the random phase. Our specification allows us to simulate an auction random close design where the length of the random phase is determined by the magnitude of the bid discount observed in the last minutes of the auction. To summarize, the 4 hypotheses we test correspond to variants of Comprasnet’s random closing rule. H1 allows to test a simple variant with a longer duration of the random phase, while the “smart” variants used to test H2, H3 and H4 can be understood as a combination between a random close and a soft close, whereby the duration of the random phase is determined by specific parameters, such as the number of bidders participating in the auction (H2), the number of bids received in the last minutes of the random phase (H3), or the bid discount registered in the last minutes of the random phase (H4). On the other hand, we chose 8    not to evaluate hard close designs since a significant body of literature already suggests that hard close endings lead to suboptimal prices.  4. Data  A. Data preparation  For the purposes of our analysis, we obtained three years (2015-2017) of detailed transactional data from Comprasnet’s online reverse auction system provided by the Brazilian Ministry of Planning, Budget and Management. For each year, we were provided with the following data sets: (i) a bidder-level data set containing general information about each auction, representing a total of 7 million observations across the three years; (ii) an item-level data set containing detailed information about each bid received during the auction, representing a total of 112 million observations; and (iii) a data set holding the milestone dates and times of all the electronic reverse auctions processes carried out through Comprasnet (0.9 million purchased items). The data preparation phase consisted of three main steps. The first step involved merging the raw data provided by the government into a single data set by linking data from the same auctions across the different tables. It also required summarizing the data to obtain a final data set in which each observation represents an auction conducted for a specific item. The second step was to clean the data to ensure its consistency as well as the coherence of the final sample used for the analysis. This involved removing observations with inconsistent dates and times, duplicated identifiers or missing values. Following the advice of the Brazilian Ministry of Planning, Budget and Management, we also removed all the data from agencies that are not part of the SISG (Sistema de Serviços Gerais). All agencies that are part of SISG must use Comprasnet and SIASG for all their procurement purchases, while non-SISG agencies can choose to use or not to use these systems. When the use of the Comprasnet system is voluntary we do not observe the full purchasing activity of a buying organization potentially biasing the sample. The third step was to generate new aggregates used in our models. The main variables generated during the data preparation phase included measures of: (i) price; (ii) duration of the auction’s random phase; and (iii) competition characteristics. Several control variables reflecting market characteristics and trends were also included in the final data set. Besides the typical challenges associated with the data cleaning process, the large volume of transactional data managed in Comprasnet also stretched the capacity of the server-based infrastructure2 provided by the Brazilian Ministry of Planning, Budget and Management. Eventually, an upgrade of the R server from 2 GB to 112 GB of RAM significantly improved the computational capacity, allowing us to successfully complete all the tasks of the data preparation phase. B. Indicators used in the analysis  To test the four hypotheses mentioned above, we use the unit price (in reais) as our dependent variable, and the duration of the random phase (in seconds) as our main independent variable. To test the second, third                                                              2 An R instance running on a remote server with 2 GB of RAM. 9    and fourth hypotheses specifically, we also examine the interaction between the duration of the random phase and several indicators reflecting competition characteristics, namely the number of bidders participating in an auction (H2), the number of bids submitted (H3) and a measure of bid discount (H4). In all variants (H1, H2, H3 and H4), we also use a number of control variables, such as the year, the quantity purchased, market characteristics, location of procuring entity and supervisory ministry. Table 1 below presents a list of the key variables used in our analysis as well as their role in each specification. Table 1: List of key variables used in the analysis Variable description Variable role Price DV (H1, H2, H3, H4) Unit price of the winning bid Auction characteristics IV (H1, H2, H3, H4) Duration of the auction random phase (in seconds) Competition characteristics IV (H2) - Number of bidders Number of unique bidders participating in each auction Number of active bidders in the last 30 seconds of the random phase - Bidding intensity IV (H3) Number of bids received in the last 5 minutes of the random phase Number of bids received in the last 3 minutes of the random phase Number of bids received in the last 30 seconds of the random phase - Bid discount Percentage discount in the random phase IV (H4) Percentage discount in the last 5 minutes of the random phase Percentage discount in the last 3 minutes of the random phase Percentage discount in the last 30 seconds of the random phase Controls Control variables Quantity purchased (H1, H2, H3, H4) Year Market Procuring entity Location of procuring entity Supervisory ministry The final sample used in our analysis consists of 560,163 observations, with each row in the dataset representing an auction process for the purchase of an item. Table 2 presents a set of descriptive statistics about the main variables used in the analysis. The data reveal that the Federal Government of Brazil acquires a wide range of products and services through Comprasnet’s e-reverse auctions. Between 2015 and 2017, the government purchased more than 54,000 distinct products, with unit prices ranging from less than 1 real to more than 5 million reais. 10    Table 2: Summary statistics Variable N N distinct Median Mean Std. Dev. Min. Max. Price Unit price 560,163 20.00 1,178.25 22,635.69 0.00 5,079,000.00 Duration Random phase duration (in seconds) 560,163 897.00 898.64 519.44 0.00 1,800.00 Competition characteristics: - Number of bidders Number of bidders per auction 560,163 6.00 7.19 5.00 1.00 74.00 Number of active bidders in the last 30 sec. of random phase 445,898 0.00 0.62 1.06 0.00 13.00 - Bid intensity Number of bids in the last 5 min. of random phase 445,898 2.00 6.76 10.56 0.00 133.00 Number of bids in the last 3 min. of random phase 445,898 1.00 4.15 6.65 0.00 86.00 - Bid discount Bid discount in the constant phase 571,867 -0.02 115.44 35,072.43 -1.00 21,631,204.67 Bid discount in the random phase 453,559 -0.10 38.05 11,278.45 -1.00 6,399,999.00 Bid discount in the last 5 min. of random phase 289,713 -0.02 53.86 13,877.97 -1.00 6,399,999.00 Bid discount in the last 3 min. of random phase 248,757 -0.01 9.90 4,054.49 -1.00 1,991,999.00 Bid discount in the last 30 sec. of random phase 146,945 0.00 0.02 3.40 -1.00 983.13 Controls Year 560,163 3 Quantity purchased 560163 50.00 19,193.48 3,334,774.01 1.00 1,500,000,000.00 Market 560,163 8,653 Procuring entity 560,163 190 Supervisory ministry 560,163 29 Location of procuring entity 560,163 27 Notes: This table reports summary statistics of the main variables in the sample dataset. For numeric variables, we display the number of valid observations, the median, the mean, the standard deviation and the minimum and maximum values. The duration of the random phase is indicated in seconds. For categorical variables, we only show the number of observations and number of distinct values. The 190 procuring entities included in our sample present volumes of purchases ranging from as low as 13,164 reais to as high as 8,2 billion reais, an additional indication of the diversity of the demand channeled through Comprasnet’s e-reverse auctions. On the supply side, the data shows that auction processes receive an average of 7 bidders, and a median of 6 bidders. The bidders represented in our sample come from Brazil’s 27 states. Our sample includes 8,653 markets with different characteristics. The number of contracts per market varies between 1 and more than 36,000, with a median of 434, and the volume of purchase per market ranges from a few reais to more than 1,3 billion reais, with a median of 4,1 million reais. For the purposes of our analysis, we define large markets as those with more than 250 contracts or a volume of purchases superior or equal to 500,000 reais. As shown in Figure 1, the data confirm that the duration of the auction’s random phase follows a random uniform distribution, a result consistent with the technical specifications of Comprasnet’s random close system. As expected, the average duration of the random phase is 15 minutes. In a typical auction, the last bids are received 102 seconds before the end of the constant phase, and 131 seconds before the end of the auctions’ random phase. 11    Figure 1: Distribution of random phase duration According to the data in our sample data set, higher discounts tend to be obtained during the random phase (median of -10%) than during the constant phase (-2%). This pattern could indicate that most bidders are reluctant to disclose their real bid price to other competitors, and therefore delay bidding to the last phase of the auction to reduce the opportunities for other bidders to respond. An exploratory analysis of the data seems to confirm that the duration of the random phase influences the number of bids received. Figure 2 shows that the proportion of auctions which did not receive bids in the random phase is higher in auctions with very short random phases. The effect, however, seems to taper off over 500 seconds: for example, nearly half (48%) of the auctions with a random phase of 100 seconds or less received no bids in the random phase, but this number goes down to 19% in auctions whose random phase lasted at least 500 seconds. Based on these descriptive statistics, we conjecture that Comprasnet’s current random close setting could lead to suboptimal auction outcomes for the government for two main reasons. First, since bidders tend to delay bidding to the end of the auction, very short random phases may not provide enough time for competition to achieve the market clearing price which maximizes savings for the government. Second, even in auctions with longer random phases, the system may put an end to an auction in the heat of bidding, thereby limiting competition. 12    Figure 2: Percentage of auctions with no bids received in the random phase over random phase duration In line with usual trends observed in public procurement data sets, we also observe that the unit price, contract value and quantity variables found in our sample follow a non-normal empirical distribution (Table 3). For our analysis, we thus use some monetary and quantity variables measured in logarithms (Figure 3) to satisfy the normality assumption of our linear regression model. Table 3: Quantile distribution Variable Min. 25% 50% 75% Max. Unit price 0.00 4.43 20.00 103.95 5,079,000.00 Awarded contract value 0.00 288.75 1,232.40 5,994.00 199,792,000.00 Quantity 0.00 8.00 48.00 300.00 1,500,000,000.00 Discount in the constant phase -1.00 -0.19 -0.02 0.00 21,631,200.00 Discount in the random phase -1.00 -0.27 -0.10 0.00 6,399,999.00 Discount in last 5 minutes of random phase -1.00 -0.08 -0.02 0.00 6,399,999.00 Discount in last 3 minutes of random phase -1.00 -0.04 -0.01 0.00 1,991,999.00 Discount in last 30 seconds of random phase -1.00 0.00 0.00 0.00 983.13 13    Figure 3: Distribution of log unit prices 5. Methodology  In order to adequately address our main research question and to evaluate and compare alternative auction closing designs, we have to identify the independent as well as interacted effects of random closing phase length and competitive characteristics. Hence the variables denoting the length of the random closing and the characteristics of competition will be of crucial interest, allowing us to assess both the current random closing design under different parameters as well as variants of the soft closing design discussed above. Exploiting the fully random feature of auction lengths and striving to keep the analysis simple, we estimate a straightforward ordinary least squares regression (OLS), of the following form: Pri = αi + ꞵ1*X1i + ꞵ2*X2i + ꞵ3*X1i*X2i + ꞵ4*X3i + εi (1) where Pri represents the log unit price of the ith item bought (Figure 3); X1i stands for the auction random closing phase length; X2i encompasses a matrix of competition characteristics such as the % discounts offered in the last 3 mins; X3i denotes the matrix of control variables accounting for policy influenceable factors (e.g. quantity bought) and structural conditions (e.g. year of purchase) (World Bank, 2017); and εi stands for the error term of the regression model. For a full list of each variable in each of these groups see Table 1 above. In order to identify the causal impact of the auction length on prices, we exploit the random uniform distribution of the length variable (Figure 1). While some characteristics of the competitive environment 14    such as the number of bidders qualifying for the auction are clearly exogenous3; others such as the discounts offered in the last 3 minutes are most likely endogenous to the dependent variable (i.e. final price is determined by the amount of discounts, but these discounts are also dependent on the price level in the auction with higher prices enabling larger discounts). Under such scenario the marginal effect of the exogenous variable remains unbiased hence we can interpret it as causal (Bun-Harrison, 2014). This is because the auction length is truly random hence its effect conditional on different competitive conditions such as more or fewer bidders will still be appropriately identified. 6. Results  We estimate regression equation 1 above in steps, starting from the simplest models but reporting only the final, best models with the parameters used for evaluating policy scenarios. In each model, we include the same set of control variables as highlighted in Table 1: purchased quantity, fixed effects for supervisory ministry, product market, location of the buyer, and the year of auction. While omitted variable bias cannot be ruled out as a concern for the non-random predictors, we consider the high explanatory power of the models, reaching 60-63%, as a sign that this problem is relatively contained. First, we estimate the independent effect of random phase length on unit prices with as well as without controls for competitive conditions (H1) (Table 4, models 1-4). As expected, we see a significant negative coefficient which is quite substantial in addition. For example, holding all else constant, an extra 10 minutes of auction time would result in approximately 6% price reduction across the Brazilian federal government. This linear effect is also depicted in Figure 4.                                                              3  This claim rests on the assumption that initial reference prices are determined in a standard way for all auctions hence there is no way the auction price could influence bidder participation before the auction. The authors’ understanding of the Brazilian federal procurement system and our informal interviews with officials at the Brazilian Ministry of Planning, Budget and Management confirm this assumption.  15    Table 4. Main regression results-OLS predicting log unitprice (each regression includes controls for purchased quantity, fixed effects for supervisory ministry, product market, location of the buyer, and the year of auction) ====================================================================================================================================================================================== Dependent variable: ---------------------------------------------------------------------------------------------------------------------------------------- log unitprice (1) (2) (3) (4) (5) (6) (7) -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- random phase length -0.0001*** -0.0001*** -0.0001*** -0.0001*** 0.00002 -0.0001*** -0.0001*** (0.00000) (0.00000) (0.00001) (0.00001) (0.00002) (0.00001) (0.00002) number of bids in the last 5 mins 0.018*** 0.014*** (0.0003) (0.001) log % discount in last 3 mins -0.086*** -0.074*** (0.002) (0.004) number of bidders 0.051*** 0.061*** 0.092*** 0.073*** 0.061*** 0.091*** (0.002) (0.002) (0.004) (0.004) (0.002) (0.004) number of bidders squared -0.0004*** -0.001*** -0.002*** -0.001*** -0.001*** -0.002*** (0.0001) (0.0001)    (0.00000) (0.00000) (0.00000) (0.00000) random phase length*number of bidders -0.00002*** random phase length*number of bidders sq. 0.00000*** (0.00000) random phase length*nr.of bids in last 5 mins 0.00000*** (0.00000) random phase length*log % discount -0.00002*** (0.00000) -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Observations 334,086 334,086 266,983 106,301 334,086 266,983 106,301 R2 0.612 0.617 0.634 0.629 0.617 0.634 0.629 Adjusted R2 0.612 0.617 0.633 0.627 0.617 0.633 0.627 Residual Std. Error 1.437 (df = 333641) 1.428 (df = 333639) 1.402 (df = 266535) 1.345 (df = 105854) 1.428 (df = 333637) 1.401 (df = 266534) 1.345 (df = 1058 ====================================================================================================================================================================================== Note: *p<0.1; **p<0.05; ***p<0     16      Figure 4: Marginal effect of random phase length on log unit prices, model 1 in Table 4. 3.2 3.15 Predictive Margins with 95% CIs log unitprice 3.1 3.05 3 0 180 360 540 720 900 1080 1260 1440 1620 1800 random phase length (seconds)   Second, our specification estimates the interaction between random phase length and the number of bidders showing up for the online auction (H2) (Table 4, model 5). As the number of bidders participating in an auction is determined prior to bidding activities and the final auction price, the criticism that it is endogenous can be largely fended off. We also enter bidder number in quadratic form to account for the diminishing effect of an additional bidder, hence the interaction effects are non-linear too. In line with expectations, random phase length continues to have a significant negative and sizeable effect on prices which increases in size as the number of bidders increases, albeit at a diminishing rate. For example, a 10 minutes longer random auction phase results in 7% lower prices when the auction has 11.5 bidders (1 sd above average), compared to only 3% lower prices when there are 2.5 bidders (1 sd below average). Third, in order to also gauge the random phase length’s interactions with bidding intensity we estimate a specification in which we use the number of bids submitted in the last 5 minutes of the auction as the main competitive characteristic indicator (H3) (Table 4, model 6). For the number of bids variable, we find a counterintuitive result, more bids predicting higher prices, which could be driven by endogeneity bias (i.e. higher prices driving more intense bidding, rather than intense bidding driving down prices). However, this result is also consistent with the story of bots submitting a high number of bids but decreasing prices only marginally driving out genuine competition. The interacted effect of the random phase length which remains well identified, changes substantially with bidding intensity. For example, a 10 minutes longer random auction phase results in 1% lower prices when the auction has about 17 bids submitted in the last 5 minutes (1 sd above average), compared to 6% lower prices when the auction has virtually no bid submitted in the last 5 minutes (1 sd under average). 17    Fourth, in order to also gauge the random phase length’s interactions with bidding intensity we also include a specification in which we use the logged percent discount offered in the last 3 minutes of the auction as the main competitive characteristic indicator (H4) (Table 4, model 7). While conceptually this specification is highly attractive, it potentially suffers from endogeneity. Nevertheless, the effect of random phase length is once again in line with expectations, significant and large. When the discounts in the last minute are 1 sd above average, a 10 minutes lengthening of the random phase results in a 7% decrease in prices, while if discounts are 1 sd below average, the same effect size shrinks to 2%. To visually demonstrate these effects, see Figure 5 below. Figure 5: Marginal effect of random phase length on log unit prices interacted with log discounts in the last 3 minutes, model 4 in Table 4. Predictive Margins with 95% CIs 4 3.5 log unitprice 3 2.5 0 180 360 540 720 900 1080 1260 1440 1620 1800 random phase length (seconds) log %disc.in last 3 mins=-10 log %disc.in last 3 mins=-8 log %disc.in last 3 mins=-6 log %disc.in last 3 mins=-4 log %disc.in last 3 mins=-2 log %disc.in last 3 mins=0     7. Conclusions and policy recommendations  The main conclusion is that the current government policy of closing reverse auctions randomly does not produce optimal prices. In our models, longer random phases contribute to lower prices, suggesting that those auctions which had a very short random phase did not give bidders enough time to lower their offers. As presented earlier in this paper, the data showed an even distribution of random phase lengths, and thus a good number of auctions closing very quickly. In fact, we recommend increasing the average and minimum length of the random phase by 450 seconds (or 7.5 minutes) and that would bring savings of 2.8% and 0.6% respectively. This would represent R$ 540 million and R$ 116 million based on 2018 spending levels. 18    A second eminent conclusion is that the size of discounts during the random phase mattered more to prices than the frequency or number of bids. For instance, one bid lowering the price significantly was better than multiple bidders lowering the price by just a few cents, or just enough to get ahead on the rankings. This appears to indicate that high-frequency bidding attributed to bots does not contribute to awarding at the lowest prices. Measures such as the ones taken by the State of São Paulo that may require a minimum reduction over the bidder’s previous bid might be productive to counter high-frequency bidding when combined with other adaptive measures, such as extending the length of the random phase. We estimate that the federal government could save 1.1% or R$ 210 million per year if the random phase was extended to its maximum every time a price reduction in the last three minutes is bigger than the 90th percentile. Again, this shows the impact of the length of the random phase on prices, which is too great to be left as random. The number of bidders had a material impact on prices and more bidders contributed to drive prices down, albeit with a non-linear relationship. In this vein, we recommend maxing out the length of the random phase whenever there are 15 bidders or more in the auction. That tactic can produce substantial 2.6% savings or R$ 500 million based on 2018 spending. As a benchmark, we also include here the potential savings effect of a policy, not related to auction design: demand aggregation. The quantity purchased in an auction can have an impact on prices, as buying in bulk can lower prices compared to small purchases. In another study (World Bank, 2017), we estimated that the federal government can save 8% with bulk buying of high volume, low complexity items. That amount corresponds to about R$ 4 billion based on 2018 spending value. Thus, good planning that allows for leveraging the demand of the federal government as a whole when buying high-volume items should certainly be part of a strategy to achieve lower prices in public procurement. Table 5. below summarizes the main conclusions and corresponding policy recommendations. Table 5. Summary of policy options and estimated savings potential Variable  Policy option  Potential  Yearly savings  savings  (2018 baseline)  Length of random  Increase average length by 450 seconds  2.8%  R$ 540 million  phase  Increase minimum length by 450 seconds  0.6%  R$ 116 million  Maximum time for random phase if 15+  Number of bidders  2.6%  R$ 500 million  bidders  Maximum time for random phase if  Size of discount  discount in last three minutes bigger than  1.1%  R$ 210 million  90th percentile  Aggregating all purchases into larger  R$ 4 billion  Buying in bulk  tenders which are below 361 units per  8%  (*not only e‐reverse  tender  auctions)    While this research delivered a range of policy‐relevant and robust results, it merely represents the first  steps towards better understanding competitive behavior in online auctions and devising ways to better  tap  into  the  potential  of  healthy  competition.  As  a  logical  next  step,  further  research  could  validate  19    whether the predicted results arise when one of the above scenarios is implemented. Moreover, more  broadly,  a  more  comprehensive  assessment  of  competitive  behavior,  bidder  profiles,  and  potentially  collusive behavior could be carried out based on the rich data set available for the Federal Government  of  Brazil  with  directly  policy  relevant  insights.  Finally,  if  the  widespread  use  of  algorithms  for  bidding  remains a challenge, bidding activity‐based indicators could be developed for limiting excessively frequent  bidding and other unwanted behaviors.       20    References  Ariely, D. and Ockenfels, A. and Roth, A. E. (2003). An Experimental Analysis of Ending Rules in Internet Auctions. CESifo Working Paper Series No. 987; Harvard NOM Working Paper No. 03-42; MIT Sloan Working Paper No. 4419-03. Bun, M. J. G. and Harrison, T. D. (2014) OLS and IV estimation of regression models including endogenous interaction terms. LeBow College of Business, Drexel University, School of Economics, Working Paper Series, WP 2014-3. Cassady, R. (1967). Auctions and Auctioneering. University of California Press. Celiktemur, C., Szerman, D. (2012). Auctions with Random Ending Time. Unpublished manuscript, accessed on November 20th, 2019 from https://eesp.fgv.br/sites/eesp.fgv.br/files/file/Dimitri_Szerman.pdf. Ely, Jeffrey C., and Tanjim Hossain. 2009. “Sniping and Squatting in Auction Markets.” American Economic Journal: Microeconomics, 1 (2): 68-94. Fullbrunn, S. and Sadrieh, A. (2012). Sudden termination auctions: An experimental study. Journal of Economics & Management Strategy, 21 (2): 519-540. Hasker, K. and Sickles, R. (2010). eBay in the economic literature: Analysis of an auction marketplace. Review of Industrial Organization, 37 (1): 3-42. Malaga, R., Porter, D., Ord, K., and Montano, B. (2010). A new end-of-auction model for curbing sniping. Journal of the Operational Research Society, 61:1265-1272. Milgrom, P. R. (1985). Auction Theory. Cowles Foundation Discussion Papers,Cowles Foundation for Research in Economics, Yale University, 779. Milgrom, P. R. and Weber, R. J. (1982). A theory of auctions and competitive bidding. Econometrica, 50 (5): 1809-1122. Roth A. E. and Ockenfels, A. (2002). Last-minute bidding and the rules for ending second-price auctions: Evidence from eBay and Amazon auctions on the Internet. American Economic Review, 92 (4): 1093-1103. Trevathan, J. and Read, W. (2011). Disarming the bid sniper. Journal of Electronic Commerce Research, 12 (3): 176-186. World Bank. 2017. A fair adjustment : efficiency and equity of public spending in Brazil : Volume 1 - Overview (English). Washington, D.C. : World Bank Group. 21