Activation Functions and Learnable Parameters: An Evaluation on Spiral Classification Task

Mentor 1

Dexuan Xie

Start Date

28-4-2023 12:00 AM

Description

In recent years, deep learning has led to a revolution in machine learning, with artificial neural networks emerging as a fundamental component of deep learning systems. PyTorch and TensorFlow have played a significant role in this development, making it easier to build and train complex models. However, the choice of activation function can significantly impact the accuracy and speed of model training. This research project aims to evaluate the performance of various activation functions on a spiral classification task, a widely used benchmark dataset for evaluating the performance of machine learning algorithms. By analyzing the impact of different activation function parameters, we can determine which function yields the best performance for this task. In addition to examining standard and popular activation functions such as ReLU, Tanh and Softmax, we also investigate the use of custom activation functions derived from base functions. By comparing their performance metrics, we can determine if custom functions provide any significant benefits over standard functions. In summary, this research sheds light on the importance of activation functions in neural network training and performance, providing insights into the best practices for designing and optimizing deep learning models.

This document is currently not available here.

Share

COinS
 
Apr 28th, 12:00 AM

Activation Functions and Learnable Parameters: An Evaluation on Spiral Classification Task

In recent years, deep learning has led to a revolution in machine learning, with artificial neural networks emerging as a fundamental component of deep learning systems. PyTorch and TensorFlow have played a significant role in this development, making it easier to build and train complex models. However, the choice of activation function can significantly impact the accuracy and speed of model training. This research project aims to evaluate the performance of various activation functions on a spiral classification task, a widely used benchmark dataset for evaluating the performance of machine learning algorithms. By analyzing the impact of different activation function parameters, we can determine which function yields the best performance for this task. In addition to examining standard and popular activation functions such as ReLU, Tanh and Softmax, we also investigate the use of custom activation functions derived from base functions. By comparing their performance metrics, we can determine if custom functions provide any significant benefits over standard functions. In summary, this research sheds light on the importance of activation functions in neural network training and performance, providing insights into the best practices for designing and optimizing deep learning models.