Sweep Parameters

Selecting the sweep parameters

We often test with various parameters when we develop the ML model. Creating multiple experiments with a combination of hyperparameter values is a tedious task. Instead, we can create one sweep job that tries all the possible combinations and selects the best model. We will be using SVM here and using a sweep job for hyperparameter tuning.

The hyperparameters need to be defined using searchspace. The option type specifies how to choose different values. We use choice for categorical values, which means choosing one of the values defined. We can use distribution methods like uniform and randint for integer values.

• Kernel: We’ll tune with poly, rbf (radial basis function), and linear kernel functions.

• Regularization Parameter (C value): If the C value is low (close to zero), we might have underfitting. On the other hand, we might overfit if the C value is high (close to infinity). For this specific example, we will choose a reasonable C value (0<C<1) that provides high accuracy.

• coef0: This parameter is for poly and sigmoid kernel functions.

Selecting the sampling algorithm

The following is a list of the sampling algorithms to use over the search space:

• Random sampling: This sampling algorithm tries random samples of all parameter combinations.
• Grid sampling: This sampling algorithm tries all possible parameter combinations.
• Bayesian sampling: This algorithm optimizes the hyperparameter selection based on the previous results.

We represent them in the YAML file, as shown below: