Activation Functions and Learnable Parameters: An Evaluation on Spiral Classification Task
Mentor 1
Dexuan Xie
Start Date
28-4-2023 12:00 AM
Description
In recent years, deep learning has led to a revolution in machine learning, with artificial neural networks emerging as a fundamental component of deep learning systems. PyTorch and TensorFlow have played a significant role in this development, making it easier to build and train complex models. However, the choice of activation function can significantly impact the accuracy and speed of model training. This research project aims to evaluate the performance of various activation functions on a spiral classification task, a widely used benchmark dataset for evaluating the performance of machine learning algorithms. By analyzing the impact of different activation function parameters, we can determine which function yields the best performance for this task. In addition to examining standard and popular activation functions such as ReLU, Tanh and Softmax, we also investigate the use of custom activation functions derived from base functions. By comparing their performance metrics, we can determine if custom functions provide any significant benefits over standard functions. In summary, this research sheds light on the importance of activation functions in neural network training and performance, providing insights into the best practices for designing and optimizing deep learning models.
Activation Functions and Learnable Parameters: An Evaluation on Spiral Classification Task
In recent years, deep learning has led to a revolution in machine learning, with artificial neural networks emerging as a fundamental component of deep learning systems. PyTorch and TensorFlow have played a significant role in this development, making it easier to build and train complex models. However, the choice of activation function can significantly impact the accuracy and speed of model training. This research project aims to evaluate the performance of various activation functions on a spiral classification task, a widely used benchmark dataset for evaluating the performance of machine learning algorithms. By analyzing the impact of different activation function parameters, we can determine which function yields the best performance for this task. In addition to examining standard and popular activation functions such as ReLU, Tanh and Softmax, we also investigate the use of custom activation functions derived from base functions. By comparing their performance metrics, we can determine if custom functions provide any significant benefits over standard functions. In summary, this research sheds light on the importance of activation functions in neural network training and performance, providing insights into the best practices for designing and optimizing deep learning models.
