Optimal online learning in bidding for sponsored search auctions

Sponsored search advertisement auctions offer one of the most accessible automated bidding platforms to online advertisers via human-friendly web interfaces. We formulate the learning of the optimal bidding policy for sponsored search auctions as a stochastic optimization problem, and model the auctions to build a simulator based on a real world dataset focusing on the simulated sponsored search auctions to show hourly variations in auction frequency, click propensity, bidding competition, and revenue originating from advertisements. We present several bidding policies that learn from bidding results and that can be easily implemented. We also present a knowledge gradient learning policy that can guide bidding to generate samples from which the bidding policies learn. The bidding policies can be trained with a small number of samples to achieve a significant performance in advertising profit. We show that a hybrid policy that makes the optimal switching from learning the bidding policies to exploiting the learned bidding policies achieves 95% of optimal performance. Also, our result suggests that using knowledge gradient learning policy may provide robustness in guessing when to switch from learning the bidding policies to exploiting the learned bidding policies.