Dec. 3, 2018, Tokyo Japan – Preferred Networks, Inc. (“PFN”, Head Office: Tokyo, President & CEO: Toru Nishikawa) has released the beta version of Optuna™, an open-source automatic hyperparameter optimization framework.

In deep learning and machine learning, it is essential to tune hyperparameters since they control how an algorithm behaves. The precision of a model largely depends on tuning the hyperparameters. The number of hyperparameters tends to be high especially in deep learning. They include the numbers of training iterations, neural network layers and channels, learning rate, batch size, and others. Nevertheless, many deep learning researchers and engineers manually tune these hyperparameters and spend a significant amount of their time doing so.

Optuna automates the trial-and-error process of optimizing the hyperparameters. It automatically finds optimal hyperparameter values that enable the algorithm to give excellent performance. Optuna can be used not only with the Chainer™ open-source deep learning framework, but also with other machine learning software.

Main features of Optuna are:

• Define-by-Run style API

Optuna can optimize complex hyperparameters while maintaining high modularity.

• Pruning of trials based on learning curves

Optuna predicts the result of training with an iterative algorithm based on a learning curve. It halts unpromising trials to enable an efficient optimization process.

• Parallel distributed optimization

Optuna supports asynchronous distributed optimization and simultaneously performs multiple trials using multiple nodes.

Optuna is used in PFN projects and with good results. One example is the second place award in the Google AI Open Images 2018– Object Detection Track competition. PFN will continue to develop Optuna, while prototyping and implementing advanced functionalities.

• Official Website https://optuna.org
• Official Document https://optuna.readthedocs.io/en/stable/
• Tutorial https://optuna.readthedocs.io/en/stable/tutorial/index.html
• GitHub https://github.com/pfnet/optuna
• Developers Blog https://research.preferred.jp/2018/12/optuna-release/

* Chainer™ and Optuna™ are the trademarks or the registered trademarks of Preferred Networks, Inc. in Japan and elsewhere.