中文标题#
基於潛在 Kronecker 高斯過程的學習曲線預測的連續分半方法
英文标题#
Successive Halving with Learning Curve Prediction via Latent Kronecker Gaussian Processes
中文摘要#
successive halving 是一種流行的超參數優化算法，它將指數級更多的資源分配給有前景的候選者。 然而，該算法通常依賴於中間性能值來做出資源分配決策，這可能導致過早地剪枝那些起步較慢但最終可能成為最佳候選者的模型。 我們研究了是否可以通過基於潛在 Kronecker 高斯過程的學習曲線預測來引導 successive halving，從而克服這一限制。 在一項涉及不同神經網絡架構和點擊預測數據集的大規模實證研究中，我們將這種預測方法與基於當前性能值的標準方法進行了比較。 我們的實驗表明，儘管預測方法表現具有競爭力，但與將更多資源投入到標準方法相比，它並不是帕累托最優的，因為它需要完全觀測到的學習曲線作為訓練數據。 然而，通過利用現有的學習曲線數據，可以緩解這一缺點。
英文摘要#
Successive Halving is a popular algorithm for hyperparameter optimization which allocates exponentially more resources to promising candidates. However, the algorithm typically relies on intermediate performance values to make resource allocation decisions, which can cause it to prematurely prune slow starters that would eventually become the best candidate. We investigate whether guiding Successive Halving with learning curve predictions based on Latent Kronecker Gaussian Processes can overcome this limitation. In a large-scale empirical study involving different neural network architectures and a click prediction dataset, we compare this predictive approach to the standard approach based on current performance values. Our experiments show that, although the predictive approach achieves competitive performance, it is not Pareto optimal compared to investing more resources into the standard approach, because it requires fully observed learning curves as training data. However, this downside could be mitigated by leveraging existing learning curve data.
文章页面#
基於潛在 Kronecker 高斯過程的學習曲線預測的連續分半方法
PDF 獲取#
抖音掃碼查看更多精彩內容