WO2014186488A3 - Tuning hyper-parameters of a computer-executable learning algorithm - Google Patents

Tuning hyper-parameters of a computer-executable learning algorithm Download PDF

Info

Publication number
WO2014186488A3
WO2014186488A3 PCT/US2014/038039 US2014038039W WO2014186488A3 WO 2014186488 A3 WO2014186488 A3 WO 2014186488A3 US 2014038039 W US2014038039 W US 2014038039W WO 2014186488 A3 WO2014186488 A3 WO 2014186488A3
Authority
WO
WIPO (PCT)
Prior art keywords
hyper
learning algorithm
configurations
parameters
parameter
Prior art date
Application number
PCT/US2014/038039
Other languages
French (fr)
Other versions
WO2014186488A2 (en
Inventor
Mikhail Bilenko
Alice Zheng
Original Assignee
Microsoft Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Corporation filed Critical Microsoft Corporation
Publication of WO2014186488A2 publication Critical patent/WO2014186488A2/en
Publication of WO2014186488A3 publication Critical patent/WO2014186488A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • G06N5/025—Extracting rules from data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Complex Calculations (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

Technologies pertaining to tuning a hyper-parameter configuration of a learning algorithm are described. The learning algorithm learns parameters of a predictive model based upon the hyper-parameter configuration. Candidate hyper-parameter configurations are identified, and statistical hypothesis tests are undertaken over respective pairs of candidate hyper-parameter configurations to identify, for each pair of candidate hyper-parameter configurations, which of the two configurations is associated with better predictive performance. The technologies described herein take into consideration the stochastic nature of training data, validation data, and evaluation functions.
PCT/US2014/038039 2013-05-15 2014-05-15 Tuning hyper-parameters of a computer-executable learning algorithm WO2014186488A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US13/894,429 2013-05-15
US13/894,429 US9330362B2 (en) 2013-05-15 2013-05-15 Tuning hyper-parameters of a computer-executable learning algorithm

Publications (2)

Publication Number Publication Date
WO2014186488A2 WO2014186488A2 (en) 2014-11-20
WO2014186488A3 true WO2014186488A3 (en) 2015-04-16

Family

ID=50979881

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2014/038039 WO2014186488A2 (en) 2013-05-15 2014-05-15 Tuning hyper-parameters of a computer-executable learning algorithm

Country Status (2)

Country Link
US (1) US9330362B2 (en)
WO (1) WO2014186488A2 (en)

Families Citing this family (52)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9773041B2 (en) 2013-03-06 2017-09-26 Oracle International Corporation Methods and apparatus of shared expression evaluation across RDBMS and storage layer
US10102480B2 (en) * 2014-06-30 2018-10-16 Amazon Technologies, Inc. Machine learning service
US10963810B2 (en) * 2014-06-30 2021-03-30 Amazon Technologies, Inc. Efficient duplicate detection for machine learning data sets
US10275719B2 (en) 2015-01-29 2019-04-30 Qualcomm Incorporated Hyper-parameter selection for deep convolutional networks
US9135559B1 (en) 2015-03-20 2015-09-15 TappingStone Inc. Methods and systems for predictive engine evaluation, tuning, and replay of engine performance
US10713594B2 (en) 2015-03-20 2020-07-14 Salesforce.Com, Inc. Systems, methods, and apparatuses for implementing machine learning model training and deployment with a rollback mechanism
US10649973B2 (en) * 2015-08-14 2020-05-12 Tibco Software Inc. Method for performing in-database distributed advanced predictive analytics modeling via common queries
US11443206B2 (en) 2015-03-23 2022-09-13 Tibco Software Inc. Adaptive filtering and modeling via adaptive experimental designs to identify emerging data patterns from large volume, high dimensional, high velocity streaming data
US9984315B2 (en) * 2015-05-05 2018-05-29 Condurent Business Services, LLC Online domain adaptation for multi-object tracking
US10511489B2 (en) * 2015-09-30 2019-12-17 Hitachi, Ltd. Storage operational management service providing apparatus, storage operational management service providing method, and storage operational management system
US11093826B2 (en) 2016-02-05 2021-08-17 International Business Machines Corporation Efficient determination of optimized learning settings of neural networks
WO2018227277A1 (en) * 2017-06-12 2018-12-20 Royal Bank Of Canada System and method for adaptive data visualization
JP6703264B2 (en) * 2016-06-22 2020-06-03 富士通株式会社 Machine learning management program, machine learning management method, and machine learning management device
US20180082212A1 (en) 2016-09-20 2018-03-22 Intel Corporation Optimizing machine learning running time
US11106994B1 (en) * 2016-09-30 2021-08-31 Amazon Technologies, Inc. Tuning of machine learning models using accuracy metrics selected to increase performance
JP2018147280A (en) * 2017-03-07 2018-09-20 株式会社日立ソリューションズ Data analysis device and data analysis method
JP6824450B2 (en) * 2017-06-28 2021-02-03 リキッド バイオサイエンシズ，インコーポレイテッド Iterative feature selection method
US11227188B2 (en) * 2017-08-04 2022-01-18 Fair Ip, Llc Computer system for building, training and productionizing machine learning models
US11397887B2 (en) * 2017-09-26 2022-07-26 Amazon Technologies, Inc. Dynamic tuning of training parameters for machine learning algorithms
US11120368B2 (en) 2017-09-27 2021-09-14 Oracle International Corporation Scalable and efficient distributed auto-tuning of machine learning and deep learning models
US11176487B2 (en) 2017-09-28 2021-11-16 Oracle International Corporation Gradient-based auto-tuning for machine learning and deep learning models
US11544494B2 (en) 2017-09-28 2023-01-03 Oracle International Corporation Algorithm-specific neural network architectures for automatic machine learning model selection
CN109800884B (en) * 2017-11-14 2023-05-26 阿里巴巴集团控股有限公司 Model parameter processing method, device, equipment and computer storage medium
US20190213475A1 (en) * 2018-01-10 2019-07-11 Red Hat, Inc. Reducing machine-learning model complexity while maintaining accuracy to improve processing speed
US10810512B1 (en) * 2018-03-21 2020-10-20 Verily Life Sciences Llc Validating a machine learning model prior to deployment
US10600005B2 (en) * 2018-06-01 2020-03-24 Sas Institute Inc. System for automatic, simultaneous feature selection and hyperparameter tuning for a machine learning model
US11451565B2 (en) 2018-09-05 2022-09-20 Oracle International Corporation Malicious activity detection by cross-trace analysis and deep learning
US11082438B2 (en) 2018-09-05 2021-08-03 Oracle International Corporation Malicious activity detection by cross-trace analysis and deep learning
US11218498B2 (en) 2018-09-05 2022-01-04 Oracle International Corporation Context-aware feature embedding and anomaly detection of sequential log data using deep recurrent neural networks
US11061902B2 (en) 2018-10-18 2021-07-13 Oracle International Corporation Automated configuration parameter tuning for database performance
CN109508455B (en) * 2018-10-18 2021-11-19 山西大学 GloVe super-parameter tuning method
CN111178533B (en) * 2018-11-12 2024-04-16 第四范式（北京）技术有限公司 Method and device for realizing automatic semi-supervised machine learning
CN111178535A (en) * 2018-11-12 2020-05-19 第四范式（北京）技术有限公司 Method and device for realizing automatic machine learning
US11526746B2 (en) 2018-11-20 2022-12-13 Bank Of America Corporation System and method for incremental learning through state-based real-time adaptations in neural networks
US11429895B2 (en) 2019-04-15 2022-08-30 Oracle International Corporation Predicting machine learning or deep learning model training time
US11620568B2 (en) 2019-04-18 2023-04-04 Oracle International Corporation Using hyperparameter predictors to improve accuracy of automatic machine learning model selection
US11562178B2 (en) 2019-04-29 2023-01-24 Oracle International Corporation Adaptive sampling for imbalance mitigation and dataset size reduction in machine learning
CN110110861B (en) * 2019-05-09 2021-11-26 北京市商汤科技开发有限公司 Method and device for determining model hyper-parameters and training model and storage medium
US11868854B2 (en) 2019-05-30 2024-01-09 Oracle International Corporation Using metamodeling for fast and accurate hyperparameter optimization of machine learning and deep learning models
WO2021000244A1 (en) * 2019-07-02 2021-01-07 Alibaba Group Holding Limited Hyperparameter recommendation for machine learning method
US11853388B1 (en) * 2019-09-05 2023-12-26 Amazon Technologies, Inc. Recalibration frequency determination for state space models
CN110705715B (en) * 2019-09-27 2023-04-18 北京迈格威科技有限公司 Hyper-parameter management method and device and electronic equipment
TWI769418B (en) 2019-12-05 2022-07-01 財團法人工業技術研究院 Method and electronic device for selecting neural network hyperparameters
US11567495B2 (en) * 2019-12-11 2023-01-31 Toyota Motor Engineering & Manufacturing North America, Inc. Methods and systems for selecting machine learning models to predict distributed computing resources
JP6774129B1 (en) 2020-02-03 2020-10-21 望 窪田 Analytical equipment, analysis method and analysis program
CN111582374A (en) * 2020-05-09 2020-08-25 北京百度网讯科技有限公司 Hyper-parameter searching method, device, equipment and storage medium
US11544561B2 (en) 2020-05-15 2023-01-03 Microsoft Technology Licensing, Llc Task-aware recommendation of hyperparameter configurations
US11775878B2 (en) 2020-12-22 2023-10-03 Sas Institute Inc. Automated machine learning test system
CN112862107A (en) * 2021-01-26 2021-05-28 贝式计算(天津)信息技术有限公司 Automatic modeling operation system and operation method
US20230101023A1 (en) * 2021-09-17 2023-03-30 C3.Ai, Inc. Ai-based hyperparameter tuning in simulation-based optimization
CN114118414A (en) * 2021-11-30 2022-03-01 广东海洋大学 Deep learning algorithm hyper-parameter optimization method based on sparse response surface
CN115329661B (en) * 2022-07-22 2023-06-23 上海环保（集团）有限公司 Intelligent dosing model modeling, intelligent dosing system creation and dosing method

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7624074B2 (en) 2000-08-07 2009-11-24 Health Discovery Corporation Methods for feature selection in a learning machine
US20030018457A1 (en) * 2001-03-13 2003-01-23 Lett Gregory Scott Biological modeling utilizing image data
US20050234761A1 (en) * 2004-04-16 2005-10-20 Pinto Stephen K Predictive model development
WO2010148238A2 (en) 2009-06-17 2010-12-23 Board Of Regents, The University Of Texas System System and method for solving multiobjective optimization problems
US20110231356A1 (en) 2009-07-01 2011-09-22 Quantum Leap Research, Inc. FlexSCAPE: Data Driven Hypothesis Testing and Generation System
US8694303B2 (en) * 2011-06-15 2014-04-08 Language Weaver, Inc. Systems and methods for tuning parameters in statistical machine translation

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
JAMES BERGSTRA ET AL: "Algorithms for Hyper-Parameter Optimization", NEURAL INFORMATION PROCESSING SYSTEMS NIPS PROCEEDINGS 2011, 31 December 2011 (2011-12-31), XP055168492 *
JASPER SNOEK ET AL: "Practical Bayesian Optimization of Machine Learning Algorithms", ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 25 (NIPS 2012), 31 December 2012 (2012-12-31), XP055168760, Retrieved from the Internet <URL:http://arxiv.org/abs/1206.2944> [retrieved on 20150210] *
MALCOLM J A STRENS ET AL: "Policy Search using Paired Comparisons", JOURNAL OF MACHINE LEARNING RESEARCH, 31 December 2002 (2002-12-31), pages 921 - 950, XP055168497, Retrieved from the Internet <URL:http://www.jmlr.org/papers/volume3/strens02a/strens02a.pdf> [retrieved on 20150209] *
NELDER J A ET AL: "A SIMPLEX METHOD FOR FUNCTION MINIMIZATION", THE COMPUTER JOURNAL, vol. 7, 30 April 1964 (1964-04-30), pages 308 - 313, XP000574677 *
PAULO GASPAR ET AL: "On the parameter optimization of Support Vector Machines for binary classification", JOURNAL OF INTEGRATIVE BIOINFORMATICS, vol. 9, no. 3, 31 December 2012 (2012-12-31), XP055168467, Retrieved from the Internet <URL:http://journal.imbio.de/articles/pdf/jib-201.pdf> [retrieved on 20150209], DOI: 10.2390/biecoll-jib-2012-201 *

Also Published As

Publication number Publication date
US9330362B2 (en) 2016-05-03
US20140344193A1 (en) 2014-11-20
WO2014186488A2 (en) 2014-11-20

Similar Documents

Publication Publication Date Title
WO2014186488A3 (en) Tuning hyper-parameters of a computer-executable learning algorithm
IL261245A (en) Structure learning in convolutional neural networks
IL274424A (en) Meta-learning for multi-task learning for neural networks
EP3523766A4 (en) Machine learning classifier and prediction engine for artificial intelligence optimized prospect determination on win/loss classification
IL260372B (en) Machine learning based webinterface generation and testing system
EP3479033A4 (en) Battery life prediction and monitoring
CN106663037A8 (en) Characteristic processing balance management
BR112014008823A2 (en) adaptive learning method, one or more non-transient storage and switch system
EP3241172A4 (en) Learning based on simulations of interactions of a customer contact center
BR112014003581A2 (en) method and apparatus for neural temporal coding, learning and recognition
KR102238248B9 (en) Battery diagnostic methods using machine learning
FR3006116B1 (en) BIPOLAR LI-ON BATTERY WITH IMPROVED SEALING AND METHOD OF MAKING THE SAME.
EP3198454A4 (en) Link retraining based on runtime performance characteristics
WO2014013390A3 (en) Prediction, scoring, and classification of magnetic resonance contrast using contrast signal scoring equation
PL3335266T3 (en) Improved separators, batteries, battery strings with improved performance, and related methods
DK3443610T3 (en) FORECAST MODEL FOR A FUEL CELL BASED ON A CORRESPONDING CIRCUIT DIAGRAM
EP3285326A4 (en) Secondary battery, evaluation method and production method therefor, and charge-discharge control device
EP3852023A4 (en) Machine learning device, inference device, program, and learned model
DK3273262T3 (en) SECONDARY MONOIMPULS MONITORING RIDES
EP3563161C0 (en) Electricity meter with an impedance learning algorithm
EP3420342A4 (en) Redundant polymer analysis by translocation reversals
GB201419992D0 (en) An associative memory learning device
EP3485260C0 (en) Counting photoactive cells
BR112015000887A2 (en) glove unit, and method of driving a glove unit installed on the production pipe &#34;.
MX2016013085A (en) Prioritized cell identification and measurement method.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14731865

Country of ref document: EP

Kind code of ref document: A2

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
122 Ep: pct application non-entry in european phase

Ref document number: 14731865

Country of ref document: EP

Kind code of ref document: A2