2. Methodology¶

2.1. Generals in Bayesian optimization approach¶

\[\int_a^{b}f=\left[f(a+\dfrac{dx}{2})+f(a+dx+\dfrac{dx}{2})+(f(a+2dx+\dfrac{dx}{2})+\cdots \right]dx\]

2.1.1. Gaussian process¶

2.1.2. Acquisition Functions for Bayesian Optimization¶

2.2. Technical features in COMBO¶

2.2.1. Thompson sampling [CL11]¶

2.2.2. Random Feature Map [RR08]¶

2.2.3. Automatic hyperparameter tuning [CER06]¶

2.3. References¶

[CER06]

Christopher K. I. Williams Carl Edward Rasmussen. Gaussian Processes for Machine Learning. volume of Adaptive computation and machine learning. MIT Press, edition, 2006. URL: http://www.gaussianprocess.org/gpml/.

[CL11]

Olivier Chapelle and Lihong Li. An empirical evaluation of thompson sampling. In J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 24, pages 2249–2257. Curran Associates, Inc., 2011. URL: http://papers.nips.cc/paper/4321-an-empirical-evaluation-of-thompson-sampling.pdf.

[GGMS72]

Phillip E. Gill, Gene H. Golub, Walter A. Murray, and Michael A. Saunders. Methods for modifying matrix factorizations. Math. Comp., 28:505–535, 1972. URL: http://www.ams.org/journals/mcom/1974-28-126/S0025-5718-1974-0343558-6/, doi:http://dx.doi.org/10.1090/S0025-5718-1974-0343558-6.

[Moc74]

Jonas Mockus. On bayesian methods for seeking the extremum. In Proceedings of the IFIP Technical Conference, 400–404. London, UK, UK, 1974. Springer-Verlag. URL: http://dl.acm.org/citation.cfm?id=646296.687872.

[RR08]

Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In J. C. Platt, D. Koller, Y. Singer, and S. T. Roweis, editors, Advances in Neural Information Processing Systems 20, 1177–1184. Curran Associates, Inc., 2008. URL: http://papers.nips.cc/paper/3182-random-features-for-large-scale-kernel-machines.pdf.

[URH+16]

Tsuyoshi Ueno, Trevor David Rhone, Zhufeng Hou, Teruyasu Mizoguchi, and Koji Tsuda. Combo: an efficient bayesian optimization library for materials science. Materials Discovery, ():–, 2016. URL: http://www.sciencedirect.com/science/article/pii/S2352924516300035, doi:http://dx.doi.org/10.1016/j.md.2016.04.001.