Machine learning models learn from data, but humans still choose many settings that guide training. These settings are called hyperparameters. Choosing them well can improve model quality, reduce overfitting, and make training more efficient.
Vertex Vizier helps automate this search.
Quick Answer
Use hyperparameter tuning when model quality depends on settings such as learning rate, batch size, number of hidden units, regularization, or training epochs. Vertex Vizier can run multiple trials and search for better values using grid search, random search, or Bayesian optimization.
Key Takeaways
- Hyperparameters are settings chosen before or during training.
- Tuning tests different values and compares model performance.
- The objective metric tells the tuning job what to optimize.
- Grid search is simple but can become expensive.
- Random search is often faster for large spaces.
- Bayesian optimization uses past results to guide future trials.
What Are Hyperparameters?
Examples of hyperparameters:
- learning rate,
- batch size,
- number of epochs,
- optimizer choice,
- regularization strength,
- number of hidden layers,
- number of neurons,
- dropout rate.
These are different from model parameters, which the model learns from data.
Why Tuning Matters
Bad hyperparameters can cause:
- slow training,
- poor accuracy,
- overfitting,
- underfitting,
- unstable training,
- wasted compute cost.
Tuning helps search for settings that perform better on validation data.
Common Search Methods
| Method | How it works | Best for |
|---|---|---|
| Grid search | Tests fixed combinations | Small search spaces |
| Random search | Tests random combinations | Larger search spaces |
| Bayesian optimization | Uses past trials to choose better future trials | Efficient tuning with limited trials |
Objective Metric
The tuning job needs a metric to optimize.
Examples:
- maximize validation accuracy,
- minimize validation loss,
- maximize F1 score,
- minimize RMSE,
- maximize AUC.
Choose a metric that matches the real problem. Do not tune for accuracy if the business problem cares more about recall, precision, or cost of false positives.
Practical Workflow
- Choose the model and baseline.
- Pick the hyperparameters to tune.
- Define the allowed range for each hyperparameter.
- Choose the objective metric.
- Run tuning trials.
- Compare the best trial with the baseline.
- Review whether improvement is meaningful.
- Save the chosen configuration.
What To Tune First
Start with a small number of important hyperparameters.
Good first candidates:
- learning rate,
- batch size,
- regularization,
- number of epochs,
- hidden units or layer size.
Avoid tuning too many values at once. A huge search space can waste time and cost.
Common Mistakes
- tuning without a baseline
- optimizing the wrong metric
- using too small a validation set
- tuning too many hyperparameters at once
- ignoring training cost
- choosing the best trial without checking overfitting
- not saving the final configuration
Bottom Line
Hyperparameter tuning is a disciplined way to improve model training. Vertex Vizier helps automate the search, but the user still needs to choose the right metric, search space, and validation approach. Start small, compare against a baseline, and keep the best configuration traceable.