Hyper-Parameter Tuning for MLP Classifier

Hyper-parameter tuning is the process of enhancing a machine learning model's performance by selecting the optimal set of hyper-parameters. These values are defined before the training begins and play a crucial role in guiding the model's learning process. Unlike the model's parameters (such as neural network weights), which are learned during the training phase, hyper-parameters are set manually and significantly impact the model’s accuracy and generalization ability.

Benefits of Hyper-Parameter Tuning:

• Improved Model Performance: Proper tuning helps the model achieve better accuracy and generalization by identifying the most suitable configuration.
• Prevents Overfitting/Underfitting: Optimizing hyper-parameters such as regularization and model complexity can help avoid overfitting (when a model learns noise) or underfitting (when a model fails to learn).
• Efficient Resource Use: Well-tuned models make efficient use of computational resources, saving time and reducing the number of iterations needed for convergence.

Code for hyperparameter tuning:Hyperparameter_tuning_mlp_using_gridsearchcv.ipynb

In this code, hyper-parameter tuning is applied to optimize an MLPClassifier using the GridSearchCV method, which automates the process of testing different parameter combinations to find the best configuration.

Data Preparation:

• The dataset is loaded and split into features (X) and labels (y).
• The data is divided into training and testing sets (80% for training, 20% for testing).

Hyper-Parameter Space:

• hidden_layer_sizes:This specifies the number of neurons in each hidden layer of the MLP (Multilayer Perceptron).
• activation: The activation function is used to introduce non-linearity to the model, helping it learn complex patterns.
• solver: Compares solvers like 'adam' and 'sgd' for optimizing the model.
• learning_rate and learning_rate_init: Investigates different initial learning rates and rate schedules.
• max_iter: Defines the maximum number of training iterations.

GridSearchCV Setup:

• GridSearchCV systematically searches through all hyper-parameter combinations using 5-fold cross-validation to avoid overfitting.
• It selects the combination yielding the best accuracy, reducing guesswork and ensuring robustness.

Model Training and Evaluation:

• The MLPClassifier is trained using the best hyper-parameters identified by GridSearchCV.

• The model's performance is evaluated on the testing set, reporting the final accuracy.The test results, shows great reduction on the confusions it makes in number predictions but still there are some confusions.Inorder to better handle those confusion, decided to train separate model for numbers.

Landmarks from both hands are combined and processed, enabling the system to interpret numerical gestures.Number model is also passed through hyperparameter tuning ranging same accurary level.This completely avoid confusion between number and sign.

References

[1]      Modeling of Multilayer Perceptron Neural Network Hyperparameter Optimization and Training.” https://www.researchgate.net/publication/368485199\_Modeling\_of\_Multilayer \_Perceptron\_Neural\_Network\_Hyperparameter\_Optimization\_and\_Training (accessed Aug. 25, 2024).

[2]      Pirjatullah, D. Kartini, D. T. Nugrahadi, Muliadi, and A. Farmadi, “Hyperparameter Tuning using GridsearchCV on the Comparison of the Activation Function of the ELM Method to the Classification of Pneumonia in Toddlers,” Proc. - 2021 4th Int. Conf. Comput. Informatics Eng. IT-Based Digit. Ind. Innov. Welf. Soc. IC2IE 2021, pp. 390–395, 2021, doi: 10.1109/IC2IE53219.2021.9649207.