K‑Fold Cross Validation in Machine Learning

K‑Fold Cross Validation is a widely used evaluation method in machine learning that helps determine how well a model can generalize to new, unseen data. Instead of relying on a single train–test split, this technique divides the dataset into K equally sized portions, known as folds. The model is trained and tested repeatedly across these folds, producing a more stable and trustworthy performance estimate.

K - Fold train and test split

Understanding Cross Validation

Cross validation plays an important role in model assessment because it:

• Prevents overfitting — ensuring the model performs well not only on training data but also on unseen samples.

• Provides a more reliable evaluation — results are averaged across multiple runs, reducing bias and variance.

• Maximizes data usage — especially useful when the dataset is small or expensive to collect.

How K‑Fold Cross Validation Works

The K‑Fold procedure can be summarized in three main steps:

1. Divide the dataset into K folds Each fold contains roughly the same number of samples.

2. Iterative training and testing For each iteration:

   • Train the model using K − 1 folds

   • Test the model using the remaining fold Every fold serves as the test set exactly once.

3. Combine the results Compute a performance metric (accuracy, precision, recall, etc.) for each fold and average them to obtain the final score.

Mathematical Representation

Let the dataset be $D$, split into $K$ folds $F_k$.

Training set for fold $k$:

$$D_{\text{train}}(k)=D\setminus F_k$$

Test set for fold $k$:

$$D_{\text{test}}(k)=F_k$$

Overall model performance:

$$\text{Performance}=\frac{1}{K}\sum _{k=1}^K\text{Metric}(M_k,F_k)<strong><br></strong>$$

$$<strong><br></strong>$$

Choosing the Right Value of K 

The value of K influences both computation time and the stability of the evaluation:

• Small K (2–5) Faster but may produce higher variance in results.

• Large K (10 or more) More stable estimates but requires more computation.

• K = n (Leave‑One‑Out Cross Validation) Extremely thorough but computationally expensive.

A commonly used setting is 10‑Fold Cross Validation, which balances accuracy and efficiency for most applications.

Variants of K‑Fold Cross Validation

• Stratified K‑Fold Ensures each fold maintains the same class distribution as the full dataset — ideal for imbalanced classification problems.

• Repeated K‑Fold Runs K‑Fold multiple times with different splits, producing even more stable performance estimates.

• Leave‑One‑Out (LOOCV) Uses one sample per fold; suitable for very small datasets.

Benefits of K‑Fold Cross Validation

• Efficient use of data — every sample is used for both training and testing.

• More dependable evaluation — reduces the chances of overfitting or underfitting.

• Suitable for small datasets — especially when data collection is costly.

For time‑series data, a modified version called Time Series Cross Validation is used, where training always precedes testing in chronological order.

Limitations of K‑Fold Cross Validation

• High computational cost — the model must be trained K times.

• Risk of data leakage — preprocessing must be done carefully to avoid contaminating the test folds.

• Not ideal for time‑series — unless adapted with time‑based splits.

Source: https://www.geeksforgeeks.org/machine-learning/k-fold-cross-validation-in-machine-learning

Selamat Maju Jaya pada Pelajar FYP bagi Sidang Akademik 2025/2026

Alhamdulillah, pada sidang kali ini, saya menerima enam (6) pelajar tahun akhir yang diletakkan di bawah seliaan dan tanggungjawab saya. Pelbagai cabaran dan dugaan telah ditempuh bersama-sama. Sepanjang penyeliaan, saya sedar ada masanya saya memarahi mereka atau meninggikan suara kepada mereka, tetapi kesemuanya dilakukan bagi menjadikan mereka pelajar FYP yang hebat dan mendapat markah terbaik. Saya sentiasa berdoa dan berharap semua pelajar seliaan saya ini berjaya dan menjadi orang yang hebat dalam bidang yang mereka ceburi nanti.

Jumpa lagi!.