WWW 2007 / Track: Search Session: Advertisements and Click Estimates Dynamics of Bid Optimization in Online Advertisement Auctions Microsoft Research Redmond, WA Christian Borgs Microsoft Research Redmond, WA Jennifer Chayes Omid Etesami U.C. Berkeley Berkeley, CA borgs@microsoft.com Nicole Immorlica Microsoft Research Redmond, WA jchayes@microsoft.com Kamal Jain etesami@cs.berkeley.edu Mohammad Mahdian Yahoo! Research Santa Clara, CA Microsoft Research Redmond, WA nickle@microsoft.com ABSTRACT kamalj@microsoft.com mahdian@yahoo-inc.com General Terms Algorithms, Economics, Theory We consider the problem of online keyword advertising auctions among multiple bidders with limited budgets, and study a natural bidding heuristic in which advertisers attempt to optimize their utility by equalizing their return-on-investment across all keywords. We show that existing auction mechanisms combined with this heuristic can experience cycling (as has been observed in many current systems), and therefore propose a modified class of mechanisms with small random perturbations. This perturbation is reminiscent of the small time-dependent perturbations employed in the dynamical systems literature to convert many types of chaos into attracting motions. We show that the perturbed mechanism provably converges in the case of first-price auctions and experimentally converges in the case of second-price auctions. Moreover, the point of convergence has a natural economic interpretation as the unique market equilibrium in the case of first-price mechanisms. In the case of second-price auctions, we conjecture that it converges to the "supply-aware" market equilibrium. Thus, our results can be alternatively described as a t^tonnement process for convergence to mara ket equilibrium in which prices are adjusted on the side of the buyers rather than the sellers. We also observe that perturbation in mechanism design is useful in a broader context: In general, it can allow bidders to "share" a particular item, leading to stable allocations and pricing for the bidders, and improved revenue for the auctioneer. Keywords Sponsored Search, Advertisement Auctions, Bidding Agent, Equilibrium Analysis 1. INTRODUCTION Categories and Subject Descriptors F.2 [Theory of Computation]: ANALYSIS OF ALGORITHMS AND PROBLEM COMPLEXITY Work performed while author was an intern at Microsoft Research. Work performed while author was a postdoc at Microsoft Research. Copyright is held by the International World Wide Web Conference Committee (IW3C2). Distribution of these papers is limited to classroom use, and personal use by others. WWW 2007, May 8­12, 2007, Banff, Alberta, Canada. ACM 978-1-59593-654-7/07/0005. Online search engine advertising is become an increasingly important and costly component of the marketing and sales strategies of many businesses. The corresponding auctions are the main source of revenue for many search engines and other Internet-related businesses. It is therefore of tremendous interest to understand and analyze the behavior of these auction systems, and to try and ensure that the system functions smoothly. In this paper, we consider the advertisement auction system as a whole and from a dynamic perspective. We first define a simple and natural bidding heuristic for budget-limited advertisers based on equalizing the "return-on-investment" (ROI) across keywords. We then observe that, when used by a set of advertisers, multiple copies of this heuristic may induce cycling behavior into the system. We propose circumventing this undesirable effect by introducing random perturbations, and see that this modified system converges to the market equilibrium (provably for first-price auctions and experimentally for second-price auctions). Thus our results may alternatively be interpreted as providing a t^tonnement process for convergence to mara ket equilibrium in which prices are adjusted on the side of the buyers rather than the sellers. Online search engine advertising is typically sold via keyword auctions (see, for example, Google's AdWords, Yahoo's Search Marketing, and MSN's AdCenter). Each prospective advertiser chooses a set of keywords relevant to his products, and for each keyword submits a bid representing an estimate of his utility for a click when that word is displayed. He also submits a maximum budget which must be respected for the chosen time period. When each keyword appears, it is auctioned among all interested advertisers with remaining 531 WWW 2007 / Track: Search budget, typically using a first-price or second-price auction mechanism (see [5, 1, 11, 16] for a comparison of these approaches). As bidders have limited budgets, the bid optimization problem they face is essentially a discrete separable resource allocation problem [7]. One of the most popular metrics to assess the efficiency of various investment strategies is marginal "return-on-investment," which in this context can be taken as the derivative of the utility with respect to the price. (See Section 3 for precise definitions.) Here we use an easily computable approximation to this quantity, namely the ratio rather than the derivative. For a particular advertiser, we define the ROI of a keyword at a given bid to be the ratio of the utility of this word to the price of the word, both at the given bid. In the bidding heuristic we consider, each budget-constrained advertiser bids an amount such that his ROI is equal across all keywords. Such heuristics are common in practice and have been proposed in other theoretical contexts as well [15]. Assume that the above bidding heuristic is employed by a set of advertisers. Two questions immediately arise. First, does there exist an underlying mechanism which causes these algorithms to converge? Second, if a convergent mechanism does exist, to what does it converge? In particular, how does this system impact revenue for the search engine provider? It is important to note that we consider these questions in light of the bidding dynamics defined by the specified heuristics, assuming all bidders adhere to these heuristics and use them truthfully regardless of the optimality of such a strategy. In particular, we do not study the properties of these systems in a strategic equilibrium.1 The first question, namely the existence of a convergent mechanism, is more than just a theoretical question. Indeed, what appears to be chaotic cycling behavior has been observed in actual search engine auctions [11].2 Moreover, for straightforward mechanisms used in conjunction with the ROI bidding heuristic, we can easily construct two-bidder examples which exhibit cycling, with the allocation oscillating between the bidders. These observations and examples are not surprising in light of the general phenomenon of heteroclinic cycles that can occur in both continuous [6] and discrete [14] dynamic systems with symmetry, sometimes leading to cycling chaos [3, 13]. In order to overcome this, we introduce an online random bid perturbation into our algorithm. In some sense, this perturbation is reminiscent of the small time-dependent perturbations employed in the dynamical systems literature to convert many types of chaos into attracting motions [12]. In mechanism design, perturbation has been proposed previously as a solution to spiteful bidding (bidding strategies which attempt to drive out competition by exhausting their budgets) [10]. Our results further motivate the introduction of perturbations to mechanism design as a technique 1 A ma jor difficulty in studying this setting as a strategic game is the repeated nature of the game. Folklore theorems show that repeated games (such as this one) have a plethora of equilibria, thereby making equilibrium analysis (without any restriction on the set of available strategies) unsuitable for predicting the behavior of the system. In this work, we are taking a different route: we fix a particular bidding strategy (whose variants are used in practice) and analyze the equilibrium of this strategy. 2 For an alternative justification of observed cycling patterns see [17]. Session: Advertisements and Click Estimates for smoothing the dynamics of the system and permitting bidders to "share" items in arbitrary ratios. Indeed, in the case of a first-price auction, we prove that the introduction of random perturbations causes the mechanism to converge. This is by far the most technically complex part of the paper. We conjecture that the random perturbations will also eliminate cycling behavior and lead to convergence of an analogous second-price auction, a conjecture which is supported by simulations in Section 5. Furthermore, we can prove that, in the case of the perturbed first-price auction, the prices (and hence revenue) of our system converges to the unique market equilibrium. As a side note, this also gives an algorithm for computing the market equilibrium in our setting (incidentally, the algorithm is quite similar to that of Devanur et al. [4] for computing market equilibria), as well as a t^tonnement process for cona vergence to market equilibrium in which prices are adjusted on the side of the buyers rather than the sellers. All of our results are supported by simulations, which we discuss in Section 5. 2. MODEL When a user performs a search, the search engine often displays advertisements alongside search results. These advertisements appear in a dedicated area of the search results page, each one in a particular fixed subarea, or slot. An online advertisement auction is a mechanism for selling these slots based on the keyword which the user provided to the search engine. We consider a setting in which m advertisers bid for the advertising slots of n keywords. Each keyword j has l slots and appears qj (t) times on day t (by "day" we mean some fixed unit of time; it does not necessarily have to be 24 hours). Advertiser i has a value vij for each click received when his advertisement is displayed on keyword j . Note that while advertisers value clicks, our auction is actually selling impressions, or the chance to appear in a keyword slot. We can convert the values per click to an expected value per impression uij k by taking the product of vij with the probability cij k that advertiser i receives a click when displayed in slot k of keyword j . This probability is called the click-through-rate. We assume these click-through-rates factor, that is, there exist ij for each bidder i and keyword j , and k for each slot k (independent of the advertiser and keyword)3 such that cij k = ij k . Thus the per impression bid uij k for the k'th slot can be written as k uij for some uij . We number slots in order of decreasing click-throughrate so 1 2 . . . l and without loss of generality assume 1 = 1. Each advertiser submits a bid bij for each keyword representing the amount he is willing to pay for one impression in slot 1 of keyword j (i.e., uij above). By extension, we assume he is willing to pay k bij for an impression in slot k of keyword j .4 Advertisers additionally submit a daily budget Bi indicating the maximum amount they are willing 3 The assumption that the click-through-rate can be decomposed in this way is a reasonable assumption and is used in practice. 4 Note we could just have easily described our results for a setting where advertisers submit a bid per click if we assume the click-through-rates of advertisers and slots are known or estimated. 532 WWW 2007 / Track: Search to spend in a given day. Although in general these parameters may be adjusted at arbitrary times, for simplicity we assume they are updated at most daily and in the beginning of the day. Upon a search for a particular keyword j , the advertisement auction then selects up to l advertisers i1 , . . . , il and assigns them to slots 1, . . . , l, respectively. It then computes a price pj k for each advertiser ik {i1 , . . . , il }. The auction guarantees no bidder is charged more than his bid nor exceeds his budget. Furthermore, no bidder is awarded more than one slot per search query. We focus our attention on two particular auction mechanisms quite common in practice. The first is a first-price mechanism in which advertisers are awarded slots in a priority order determined by their bids. Advertisers are then charged a price equal to the minimum of their bid and remaining budget. The second mechanism is a generalization of the second-price mechanism. The allocation rule of this mechanism is identical to that of the first-price mechanism, but the pricing scheme is different. Each advertiser is now charged a price equal to the minimum of his remaining budget and the bid of the advertiser in the next slot. The pseudocode of these two mechanisms appears in Figure 1. For our theoretical results, we simplify the model in the following ways. First, we study a setting in which there is only one slot per keyword. The single-slot setting is rich enough to capture the chaotic behavior our results circumvent and thus suffices to illustrate our main points.5 Second, we consider a continuous-time version of the auction: for each keyword j , there are a constant number qj of searches each day, and these searches are evenly spaced throughout the day. We assume qj 's are large and therefore we can model this process as one in which all keywords arrive continuously at a uniform rate throughout the day. The daily budget of advertiser i is Bi , and the total utility of advertiser i for showing his ad on keyword j throughout the entire day is uij (thus, his utility for being shown during an fraction of the day is uij ). Without loss of generality we will j uij . assume Bi Session: Advertisements and Click Estimates bj is the existence of a constant (the Lagrangian multii plier) such that for all j with Uj (bj ) > 0, i d Uj /d Pj |bij =bj = i if such derivatives exist. This derivative is known as the marginal return-on-investment (marginal ROI) and measures how the net utility of an advertiser changes as he modifies his investment. Thus, for an optimal set of bids {bj }, we i know advertiser i has the same marginal ROI at bj across i all keywords. This marginal ROI is exactly the Lagrangian multiplier above. The marginal ROI is usually difficult to estimate, and is even undefined when Pj or Uj are discontinuous. Thus, it is useful to approximate the marginal ROI of keyword j at bid b by the ROI of keyword j at that bid, where ROI is defined as ROIj (b) = Uj (b)/Pi (b). This suggests one method for optimizing the bids of the advertiser: set the bids bij such that ROIj (bij ) approximately equals some constant ROI for all j . If the prices were fixed and known to the advertiser, determining an optimal bidding vector would be a simple calculation. Suppose the price of the kth slot for keyword j is pj k . We further introduce an artificial slot l + 1 with price zero and utility zero indicating that the advertiser does not appear in any slot on that keyword. A bidding strategy is now a selection of affordable slots sj {1, . . . , l + 1} for each keyword j , where a selection is affordable if the sum of prices is at most the budget of the advertiser. This problem is a natural extension of the knapsack problem [8] and has a similar FPTAS. In fact, the idea of the ROI heuristic is similar to the well-known 2-approximation algorithm for knapsack that is based on sorting items by the ratio of their value to their size (which, in our setting, corresponds to the ROI). One way to implement the ROI heuristic is through a t^tonnement-like process, where the advertiser iteratively a incrementing bids on keywords with relatively large ROI and decrementing bids on keywords with relatively small ROI by small increments. The advantage of this method is that it requires the minimal amount of information. In particular, it does not even need to know the price of the slots above and below the current slot. It is easy for an advertiser to calculate the ROI for each keyword in hindsight at the end of the day. Based on this idea, we consider the following ROI-based heuristic bidding algorithm for advertiser i. Algorithm 1. On each day t, al l bids of advertiser i are determined by a single parameter Ri (t) (0, 1].7 The parameter Ri (t) is adjusted based on the performance of advertiser i's bids on the previous day. Starting from an arbitrary Ri (0) (0, 1] for day t = 0, advertiser i sets i f i runs out of money Ri (t)e- Ri (t + 1) = before the end of day t min(Ri (t)e , 1) otherwise where > 0 is a smal l constant. Final ly, he sets the bid bij (t) of keyword j to bij (t) = Ri (t)uij . Note since Ri (t) (0, 1], bij (t) uij . This parameter is related to the target return-oninvestment by Ri (t) = 1/(ROI + 1) where ROI is the target return on investment of advertiser i. 7 3. BID OPTIMIZATION HEURISTICS In this section we describe a natural bidding heuristic for optimizing the utility of the advertisers. We consider the following abstraction of the bid optimization problem for advertiser i. We want to specify a bid bij on each keyword j . We assume that if advertiser i bids bij on keyword j then his day-long charge and net utility (i.e., total value minus total charge) on that keyword is given by Pj (bij ) and Uj (bij ) respectively.6 The optimization problem is now to j choose {bij } such that Uj (bij ) is maximized sub ject to j Pj (bij ) Bi . Through the use of Lagrangian relaxation, we see that a necessary condition for the optimality of bids 5 In fact, it is straightforward to generalize our convergence result (Theorem 1) to the multi-slot setting (essentially the only thing that needs to be changed is Equation 1). However, the point to which the system converges can no longer be characterized as a market equilibrium. 6 Note that we assume the charge and net utility of advertiser i for keyword j is a function of his bid for keyword j alone and does not depend on the bids of i for other keywords. Although this is not strictly true, it is a reasonable approximation and serves to develop our intuition for our heuristic. 533 WWW 2007 / Track: Search Session: Advertisements and Click Estimates First-Price Mechanism Let S be the set of bidders {i : si < Bi }. For k = 1 to l do Let i = argmaxiS (bij ), Set S = S - {i}, Assign i to slot k, Charge i price min(k bij , Bi - si ). Second-Price Mechanism Let S be the set of bidders {i : si < Bi }. For k = 1 to l do Let i = argmaxiS (bij ), Set S = S - {i}, Assign i to slot k, Charge i price min(k m x bi j , Bi - si ). a i S Figure 1: Pseudo co de for the first and second-price auctions, resp ectively. The parameter si is the current total daily charge of advertiser i. Before discussing the dynamics of this algorithm, let us note that an added advantage of the above bidding heuristic is that it can be adapted to cases where an advertiser only knows her budget and the relative utility of various keywords (i.e., the ratio of uij 's), and not the exact value of the utilities. In this case, the bidding algorithm for advertiser i can initially set her largest utility to Bi and the other utilities according to the specified ratios, and then adjust these values by changing Ri (t). This is useful in practice since for an advertiser estimating the ratio of the values of various keywords is a considerably simpler task than estimating the exact utilities. coupled with multiple copies of the bid optimization algorithm presented in Section 3, converges to a fixed allocation and set of prices corresponding to the market equilibrium. We conjecture a similar result for the perturbed second-price auction, supporting our conjecture with simulation results in Section 5. 4.1 Perturbations 4. DYNAMICS OF THE SYSTEM In Section 3 we defined a heuristic for bidding in an advertisement auction. In order to better understand the properties of a system where bidders are using such a heuristic, we need to analyze the interplay of bidding algorithms of various bidders. One might wonder if such a system could ever stabilize, and whether the resulting prices would be logical in some sense (i.e., be simultaneously "reasonable" for the advertisers and generate sufficient revenue for the search engine). In fact, the following example shows that the combination of the first-price auction with the ROI heuristic may result in an unstable situation with low prices. Example 1. Suppose there is just one keyword with one slot and 1000 impressions. There are two advertisers a and b, each advertiser with a budget of $500 and a utility of $1 for each impression of the keyword. Consider the first-price auction mechanism. Assume a bids $0.5e , and b bids $0.5. Bidder a is going to win al l the impressions until he runs out of the budget around the end of the day, but he is going to decrease his bid for tomorrow to $0.5, since he ran out o of budget today. On the other hand, b is going to bid $0.5e n the fol lowing day. Thus, a and b wil l interchange roles. This way the al location of the impressions alternates between a and b daily. It is easy to see that the above example works for the second price mechanism as well. The results of Section 5 confirm that such examples arise in a variety of plausible scenarios, resulting in oscillating allocations and dampened revenue. We avoid such situations by applying a random perturbation to the bids of the advertisers in determining the allocation, as defined below. In this section we study variants of the first and second-price auctions with perturbations. We prove that the perturbed first-price auction, In order to get rid of situations like the one explained in Example 1, we modify the auction mechanism to slightly perturb the bids before running the auction, thereby giving the bidder with a smaller bid some chance of winning if his bid is close to the largest bid. The perturbations are defined as follows. On each day t, advertiser i bids a value bij (t) for the day-long possession of keyword j . When a search on keyword j occurs, we perturb the bids as follows: bij = bij (t)e-i , where i is a uniformly random number in [0, ]8 , independently generated for each bidder/query pair, and > 0 is a constant. The auction mechanisms are run exactly as described in Section 2, but the allocation is determined according to the perturbed bids bij (t). Perturbations essentially allow advertisers to bid such that they share the keyword in any portion they please. That is, fixing the bids of other advertisers on a particular keyword, a given advertiser can choose to receive in expectation any fraction of the day-long procession of the keyword by adjusting his bid appropriately. Note that such a sharing property can not be achieved by introducing a randomized tiebreaking rule; applying the perturbation to the bids themselves is significantly more powerful. Notice how this affects the advertisers in Example 1. Example 2. Again, consider the scenario from the previous example. However, now suppose the bids are perturbed as described above and notice the instability we observed before won't happen. Indeed a and b share the impressions almost equal ly in expectation, and so neither bidder runs out of budget. Therefore, they wil l increase their bids until their bids get close to $1 at which time both the price and al locations remain stable. In this case the perturbation both removed the cycling and improved auctioneer's revenue by a factor of two. The choice of the distribution for perturbation is essentially arbitrary, and our results hold for other reasonable perturbation models (e.g., Gaussian perturbations) as well. 8 534 WWW 2007 / Track: Search Session: Advertisements and Click Estimates Theorem 2. Given > 0 and > 0, let t = t(, ) t0 , where t0 is defined as in Theorem 1. Let pj (t) be the maximum price at which keyword j is sold in day t, and let xij (t) be the fractional daily al location of word j to advertiser i on day t. As , go to zero, the price vector pj (t) converges to that of the market equilibrium, and the total utilities of the j advertisers including their unused budgets, Li + uij xij (t), converge to the utilities of an equilibrium al location. Notice that convergence of the priice vector implies also convergence of the total revenue pi for the auctioneer. The proof of Theorem 2, which makes substantial use of the stability results in Theorem 1, is deferred to Appendix A. Proof of Theorem 1. We first show Statement 1, i.e. that after some finite time nobody runs out of budget early. s ore precisely, we will show that for every 0 < < 1, M mall enough and t T (where T is a constant depending on ), we have i (t) 1 - for all 1 i n. Let k(t) be the first advertiser who finishes his budget on day t. The proof of Statement 1 follows from the following two claims. Claim 1. If k(t) (t - 1) < 1, then k(t) (t) min(e k(t) (t 4.2 Convergence to Equilibria We now discuss our main theoretical results, namely the convergence properties of our perturbed mechanisms with multiple bid optimization algorithms. Throughout the remainder of this section, we assume there is just one slot per keyword.9 We consider both perturbed first-price and perturbed secondprice auctions. In each of these auctions, the allocation rule awards the keyword slot to the bidder with the highest perturbed bid bij . The winning advertiser is then charged a price equal to the minimum of his remaining budget and unperturbed bid bij in the case of the first-price auction10 , or the minimum of his remaining budget and the perturbed bid of the closest competitor in the case of the second-price auction. Once the spending of an advertiser during a day reaches his daily budget, he is withdrawn from all further auctions during that day. We now state our principal result. Namely, we prove that in a perturbed first-price auction where bidders bid according to the ROI heuristic, Algorithm 1 of Section 3, both the prices and the daily utilities of the advertisers, and hence the revenue of the auctioneer, converge to that of the market equilibrium in the sense of Arrow and Debreu [2] when goods correspond to the ad spaces and the money (see Appendix A). More formally, let si (t) [0, Bi ] denote the spending of advertiser i on day t. Let i (t) [0, 1] denote the moment during day t when advertiser i spends all his budget (or 1 if he does not spend all his budget). Finally let ri (t) denote the spending rate of advertiser i in the beginning of the day before anyone runs out of budget. In other words, j n bij (t) i ri (t) = Pr bij (t)e-x > bi j (t)e-i ]dx (1) i [ 0 = =1 i - 1), 1). Claim 2. If k(t) (t-1) = 1, then k(t) (t) 1-, provided is chosen in such a way that 2C e . To see that these two claims imply Statement 1 of the theorem, set min (t) = mini i (t). Claims 1 and 2 together imply min (t) min(1 - , e min (t - 1)). We know that j uij ) = 1/C . Therefore for t T = min (1) mini Bi /( -1 log(C (1 - )), we have min (t) 1 - , as required. Proof of Claim 1. Throughout this proof, let k = k(t). If k (t) = 1, then the claim is true. Assume k (t) < 1. Note that since k (t - 1) < 1, Rk (t) = Rk (t - 1)e- and for i = k, Ri (t) Ri (t - 1)e- . Consider an imaginary scenario in ^ which on day t, Ri (t) = Ri (t - 1)e- for all bidders i. By (1), the spending rate rk (t) of bidder k in the imaginary ^ scenario is at least that of the real scenario (rk (t) rk (t)). ^ Furthermore, rk (t) = rk (t - 1)e- since advertisements in ^ the imaginary scenario are sold to advertisers with the same probabilities as day t - 1 and at a price e- times the price of day t - 1. Therefore, we have w Note that the rate of spending only increases as other advertisers run out of budget, and therefore we have si (t) ri (t)i (t). We first show these parameters converge, namely, that after some time no advertiser runs out of budget early and each advertiser either spends most of his budget or is bidding nearly his utility on all keywords. The proof of the following theorem appears at the end of this section. Theorem 1. Given utilities uij , budgets Bi , and constants > 0 and > 0, there exist constants > 0 and t0 < , such that for al l t t0 and al l i, we have 1. i (t) 1 - , and 2. si (t) (1 - )Bi or Ri (t) 1 - . Here and t0 can be chosen as = ( min{1, C 2 }) and j uij 2 t0 = log C - log(mini Ri (0)) with C = maxi ( ). Bi rk (t - 1)k (t - 1) Bk = k (t)rk (t) k (t)rk (t - 1)e- hich implies Claim 1. In order to prove Claim 2, we first prove the following lemma. Lemma 1. For al l t and al l i, we have |ri (t) - ri (t - 1)| (2C e / )Bi . Proof. Note that Ri (t) Ri (t-1)e and Ri (t) Ri (t- 1)e- for i = i. Consider an imaginary scenario in which ^ ^ on day t, Ri (t) = Ri (t - 1)e2 and Ri (t) = Ri (t - 1) ^ for i = i. Then, Ri (t) e Ri (t) and for every i = i, ^ ^ Ri (t)/Ri (t) Ri (t)/Ri (t), which implies that now ri (t) ^ . ri (t)e We couple the perturbed bids ^i j (t) of the imagb inary scenario with the perturbed bids bi j (t - 1) of day The above theorem allows us to characterize the equilibrium of our system. Let Li (t) = Bi - si (t) be the unused portion of advertiser i's budget at the end of day t. Then the following theorem holds. 9 It is not hard to see that Theorem 1 holds for the multi-slot case with essentially the same proof. 10 Note that our results hold if the pricing rule charges the winning bidder his perturbed bid bij as well. 535 WWW 2007 / Track: Search t - 1 in such a way that ^i j (t) = bi j (t - 1) if i = i and b ^ij (t) = bij (t - 1)] = 2 /. Namely, we set Pr[b bi if bij (t - 1) ^ij (t)e- b j (t - 1) ^ij (t) = b i b j (t - 1)e otherwise As the ratio of ^ij (t) to bij (t - 1) is e2 , it is easy to see that b this coupling results in the desired probability. Thus, even if advertiser i wins all auctions in which ^ij (t) = bij (t - 1) b (which happens at most a 2 / fraction of the times), we have 2 2j U ri (t) ri (t - 1) + ^ uij e2 ri (t - 1) + C Bi e2 sing that ri (t) ri (t)e , this implies ri (t) ri (t - 1) + ^ (2C e / )Bi . The matching upper bound on ri (t - 1) in terms of ri (t) is proved by exchanging the roles of t and t - 1. Proof of Claim 2. Let k = k(t). By the previous lemma and our condition on , we have rk (t) rk (t - 1) + Bk Bk (1 + ) = rk (t)k (t)(1 + ) where we used the assumption k (t - 1) = 1 to conclude that rk (t - 1) Bk . This gives k (t) 1/(1 + ) 1 - , proving the claim. Now we will prove Statement 2. Note that ri (t) Bi (1 - ) implies si (t) Bi (1 - ) (this is because either si (t) = Bi or i (t) = 1 in which case si (t) ri (t)). Therefore, it is enough to show that for all t 2T - log(mini Ri (0)) and all i, one of the following holds: ri (t) Ri (t) (1 - )Bi , e- (2) ( 3) Session: Advertisements and Click Estimates 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 1st/2nd price without perturbation 1st/2nd price with perturbation 0.2 0 50 100 150 200 250 300 350 400 450 500 Figure 2: Change in bids, Examples 1 and 2 proof technique fails for the second price auction. Given the convergence result, in Theorem 2 (whose proof is presented in Appendix A) we show that for the first price auction, the prices converge to the market equilibrium prices. For the second-price auction, assuming our conjecture on the convergence of the system, we can similarly show that the prices tend to approximate equilibria for a new notion of market equilibrium, called the self-competition-free or supply-aware market equilibrium (see [9]). A supply-aware equilibrium for a market with additive utilities is a regular market equilibrium for a modified setting in which the utility of each buyer for each item is capped to the utility they derive by buying the entire supply of the item. The simulations in Section 5 support our intuitions. 5. SIMULATIONS so long as is less than . We first prove the following claim. Claim 3. For 2C , 4C e we have si (t - 1) - ri (t) Bi . , and (t - 1) T , Proof. By Statement 1, min (t -j1) (1 - ), and thereuij ri (t - 1) + Bi /2 fore si (t - 1) ri (t - 1)(1 - ) + provided 2C . Moreover, by Lemma 1 and our condition on , we have ri (t - 1) ri (t) + Bi /2. Therefore si (t - 1) ri (t) + Bi . The proof of Statement 2 now follows by backwards induction. First suppose neither (2) nor (3) holds on day t and t - 1 T . We will show neither inequalities holds on day t - 1. Indeed, by the above claim, si (t - 1) ri (t) + Bi < Bi and hence Ri (t) = min(Ri (t - 1)e , 1) Ri (t - 1). Therefore (3) did not hold on day t - 1 as well, which implies that Ri (t) = Ri (t - 1)e . Now using an argument similar to Claim 1, we can show that ri (t) ri (t - 1)e . It follows that (2) did not hold on day t - 1 either. For the base case, notice that as long as neither (2) nor (3) holds, we saw in the above paragraph that Ri (t) = Ri (t - 1)e and so for t 2T - log(mini Ri (0)), inequality (3) will hold. The above result shows that the prices in a perturbed firstprice mechanism converge. We believe that a similar result holds for a perturbed second price auction (see next section for evidence of this in simulation results). However, our In this section, we present the results of simulating the bid optimization algorithm of Section 3 for various auction mechanisms. In particular, we compare the behavior of the bid optimization algorithm in the equilibrium for the first and second-price auctions with and without perturbation. Parameters of the simulation: We have implemented the simulation program in Matlab. In all our simulations, we assume that k = 1/k (i.e., click-through rates of different slots follow a power law with exponent -1). We assume that throughout the day, each keyword is searched for 1000 times, and these searches occur in a random order. At the end of each day, the bid optimization algorithm is run to update the bids of each advertiser. For most simulations, the parameters (determining the aggressiveness of the bid optimization algorithm in changing bids) and (determining the extent of the perturbations for perturbed mechanisms) are set to 0.01 and 0.1, respectively. A small example: We start by showing the outcome of the simulation for the instance explained in Examples 1 and 2 for 500 days. In this instance, there are two advertisers and one keyword with one ad slot. Each advertiser has a utility of $1 and a daily budget of $500. Both advertisers start by bidding $0.20 on each keyword. The graph of the bid of the first advertiser as a function of time for each of the four mechanisms is shown in Figure 2 (the second advertiser has similar bids). As we see in this figure, in unperturbed mechanisms, the bids of the advertisers grow only to $0.50, and after that remain constant, whereas in perturbed mech- 536 WWW 2007 / Track: Search Session: Advertisements and Click Estimates 1000 900 800 700 600 500 400 300 200 100 1st price 2nd price 1st Perturbed 2nd Perturbed 0 100 200 300 400 500 5400 5200 5000 4800 4600 4400 4200 4000 3800 3600 1st price 2nd price 1st Perturbed 2nd Perturbed 0 100 200 300 400 500 Figure 3: Change in revenue, Examples 1 and 2 Figure 5: Change in revenue, random instance 7200 7000 6800 6600 6400 6200 6000 1st price 2nd price 1st Perturbed 2nd Perturbed 0 100 200 300 400 500 5800 Figure 4: Change in efficiency, random instance anisms, the bids grow to $1. The revenue of the mechanisms are compared in Figure 3.11 Since the utilities in this example are equal, the efficiency of all mechanisms are constant over time. A larger example: We have simulated the bid optimization algorithm with different mechanisms on larger instances generated at random. Figures 4 and 5 show the changes in the efficiency and the revenue of the auctions (per day) as a function of the day for an instance with n = 20 bidders, m = 10 keywords, and one slot per keyword. In this instance, each advertiser bids on each keyword with probability 1/3, and the value of the bids are drawn uniformly at random from [0, 1]. The daily budgets of the advertisers are 3000, 3000/2, 3000/3, . . . , 3000/20.12 Figure 6 shows the changes in the bids on one of the keywords as a function of time. As this figure shows, the mechanisms with perturbation avoid having bids that are almost equal and frequently change order, whereas in mechanisms without perturbation, 11 The decrease in the revenue of the perturbed second-price auction (compared to the first-price) is due to the fact that after a short while, the randomness in the system could cause the bid of one of the advertisers to be slightly more than the other, resulting in the advertiser running out of budget earlier than the other advertiser, and the other advertiser getting the remaining ad spaces in that day for free. 12 The choice of budgets as a power law distribution with exponent -1 is motivated by the classical observation that income distribution often follows such a power law. such situations are common. This can be observed from the diagram of efficiency in Figure 4, where it can be observed that the efficiency of the allocation on odd-numbered days are significantly lower than the efficiency of the mechanism on even-numbered days. Random instances: We have simulated the bid optimization algorithm with each of the four auction mechanisms on a set of 150 randomly generated instances to measure the average behavior of the algorithm in different auctions. The instances are generated similar to the way described in the previous example, with 10 bidders, 5 keywords, and 3 slots per keyword. We have simulated the auctions for 300 days, and measured the following parameters: the convergence of system, and the efficiency and the revenue of the auction. Convergence. To measure the convergence, we check the properties required in the statement of Theorem 1, and compute the fraction of bidders for whom both of these properties are satisfied at the end of the simulation. We say we have perfect convergence if these conditions (for = 0.1) are satisfied for all bidders and good convergence if they are satisfied for 90% (i.e., all but at most one) of the bidders after 1000 steps. Figure 7 shows the distribution of the number of converged bidders, and Figure 8 compares the percentage of the times perfect or good convergence is achieved on the four mechanisms. In this figure, mechanisms 1, 2, 3, and 4 represent the first price, the second price, the perturbed first price, and the perturbed second price mechanisms, respectively. These figures confirm our result that perturbed mechanisms are significantly more likely to converge to an equilibrium. Revenue and Efficiency. The comparison of the revenue and the efficiency of the mechanisms reveals that in this set of instances, the revenue and the efficiency of the perturbed mechanisms are consistently (between 79% and 97% of the times) more than the unperturbed mechanisms. However, the difference is small (between 1.5% and 5% on average). 6. ACKNOWLEDGMENTS We would like to thank Max Chickering, Uri Feige, and Chris Meek for many fruitful discussions regarding the systems proposed in this paper. 537 WWW 2007 / Track: Search Session: Advertisements and Click Estimates Bids for keyword 6 in 1st price auction 0.45 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 0.45 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 500 0 Bids for keyword 6 in 2nd price auction 0 100 200 300 400 100 200 300 400 500 Bids for keyword 6 in perturbed 1st price auction 0.45 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 0.45 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 500 0 Bids for keyword 6 in perturbed 2nd price auction 0 100 200 300 400 100 200 300 400 500 Figure 6: Change in the bids in a random instance 1st price 100 80 60 40 20 0 1 2 3 4 5 6 7 8 9 10 perturbed 1st price 100 80 60 40 20 0 1 2 3 4 5 6 7 8 9 10 100 80 60 40 20 0 100 80 60 40 20 0 2nd price 1 2 3 4 5 6 7 8 9 10 perturbed 2nd price 1 2 3 4 5 6 7 8 9 10 Figure 7: Distribution of the numb er of converged bidders 538 WWW 2007 / Track: Search Session: Advertisements and Click Estimates Perfect convergence 1 0.8 0.6 0.4 0.2 0 1 2 3 4 1 0.8 0.6 0.4 0.2 0 1 Good convergence 2 3 4 Figure 8: Distribution of the numb er of converged bidders 7. REFERENCES [1] G. Aggarwal, A. Goel, and R. Motwani. Truthful auctions for pricing search keywords. In Proceedings of the 7th ACM Conference on Electronic Commerce, 2006. [2] K. Arrow and G. Debreu. Existence of an equilibrium for a competitive economy. Econometrica, 22:265­290, 1954. [3] M. Dellnitz, M. Field, M. Golubitsky, A. Hohmann, and J. Ma. Cycling chaos. Intern. J. Bifur. & Chaos, 5:1243­1247, 1995. [4] N.R. Devanur, C.H. Papadimitriou, A. Saberi, and V.V. Vazirani. Market equilibrium via a primal-dual-type algorithm. In Proceedings of the 43rd Symposium on Foundations of Computer Science, 2002. [5] J. Feng, H.K. Bhargava, and D. Pennock. Comparison of allocation rules for paid placement advertising in search engines. In Proceedings of the 5th International Conference on Electronic Commerce, 2003. [6] M.J. Field. Equivariant dynamical systems. Trans. Amer. Math. Soc., 259:185­205, 1980. [7] T. Ibaraki and N. Katoh. Resource Al location Problems: Algorithmic Approaches. MIT Press, Cambridge, MA, 1988. [8] O.H. Ibarra and C.E. Kim. Fast approximation algorithms for the knapsack and sum of subset problems. Journal of the ACM, 22(4):463­468, 1975. [9] K. Jain and K. Talwar. Truth revealing market equilibria. Manuscript, 2004. [10] C. Meek, D. Chickering, and D. Wilson. Stochastic and contingent-payment auctions. In 1st Workshop on Sponsored Search Auctions, 2005. [11] M. Ostrovsky, B. Edelman, and M. Schwarz. Internet advertising and the generalized second price auction: Selling billions of dollars worth of keywords. In 2nd Workshop on Sponsored Search Auctions, 2006. [12] E. Ott, C. Grebogi, and J.A. Yorke. Controlling chaos. Physics Review Letters, 64, 1990. [13] A. Palacios. Cycling chaos in one-dimensional couple interated maps. Intern. J. Bifur. & Chaos, 12m:1859­1868, 2002. [14] A. Palacios. Heteroclinic cycles in coupled systems of difference equations. J. Difference Eqs & Appl., 9:671­686, 2002. [15] P. Rusmevichientong and D. Williamson. An adaptive algorithm for selecting profitable keywords for search-based advertising services. In Proceedings of the 7th ACM Conference on Electronic Commerce, 2006. [16] H. Varian. Position auctions. Manuscript, 2006. [17] X.M. Zhang and J. Feng. Price cycles in online advertising auctions. In Proceedings of the 26th International Conference on Information Systems (ICIS), 2005. APPENDIX A. PROOF OF THEOREM 2 We start by recalling some standard definitions, as applied to our setting. Given the prices pj for keywords, an optimal allocation xij for advertiser i is any solution to the following linear program: j maximize Li + uij xij sub ject to Li + j pj xij = Bi j : xij 0 Li 0. Here xij is the fractional amount of word j assigned to the advertiser i, and Li is the amount of money unspent by i. A price vector is called a market equilibrium price vector if there exist allocations xij that satisfy the following two conditions: · At the given price vector, xij is an optimal allocation for each advertiser i. i · For each keyword j , we have xij = 1 (recall that the supply of each keyword was assumed to be 1). The next theorem follows from the classical results in the economic literature (see, for example, Arrow and Debreu[2]) by considering the market with one commodity for each keyword and an additional commodity termed "money". The proof of this theorem is deferred to the full version of the paper. 539 WWW 2007 / Track: Search Theorem 3. There exists an equilibrium price vector in the market defined above. Moreover, the market equilibrium prices are unique, and can be characterized as the set specified by the fol lowing convex program. j Li + B uij xij uij i, j : (4) pj i j Li + B uij xij i : 1 (5) i i xij 1 (6) j : j pj + i Li i Bi (7) (8) (9) Session: Advertisements and Click Estimates no advertiser buys any keyword at a higher price than his utility, and the other three constraints are satisfied because Algorithm 1 always computes a feasible allocation and nonnegative prices. Therefore, the only constraints that we need to check are (4) and (7). But it is easy to use Theorem 1 to show that these constraints are satisfied approximately, i.e., there is a value (, ) that approaches zero as and approach zero so that: j Li + B uij xij u ij i, j : (1 - (, )) pj i j pj + i Li (1 + (, )) i Bi . i, j : xij 0 j : pj 0. Now let us return to the proof of Theorem 2. We show that as and approach zero, the constraints in Theorem 3 becomes satisfied. In fact, constraints (5), (6), (8), and (9) are always satisfied: constraint (5) is satisfied because The prices and allocation of our algorithm must satisfy these constraints. Consider the convex region specified by these relaxed constraints. As and go to zero, the constraints approach those of Theorem 3, implying that the price and utility vectors converge to the unique equilibrium price and utility vectors, respectively. This completes the proof of Theorem 2. 540