Machine Learning with scikit-learn#

Purpose#

Scikit-learn (scikit-learn: machine learning in Python - scikit-learn 1.7 documentation) is one of the most popular and powerful open-source machine learning libraries for Python. Built on top of NumPy, SciPy, and matplotlib, it provides a simple and efficient interface for a wide range of machine learning tasks.

About this tutorial#

In this tutorial, you will learn how to build and evaluate machine learning models using scikit-learn in three steps:

  • Installation

  • Model training, prediction and evaluation

  • Using model in ACS RTDI

Compatibility#

  • Ubuntu 20.4 / SmarTest 8 / RHEL79 / Nexus 3.1.0 / Edge 3.4.0-prod / Unified Server 2.3.0

Procedure#

Installation#

Ensure you have the following prepared:

  • An ACS RTDI virtual environment containing an Ubuntu Server VM.

You can use the following commands to download the example Jupyter notebooks and set up the runtime environment:

Click to expand! 
# 1.Download artifacts
cd ~
curl http://10.44.5.139/apps/scikit-learn-notebook_2.0.0.0.tar.gz -O
tar -zxf scikit-learn-notebook_2.0.0.0.tar.gz

# 2.Install required system packages
sudo apt update -y
sudo apt install -y make build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm \
libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev \
liblzma-dev python3-openssl git

# 3.Install Python 3.10.12 and create virtual environment
PYTHON_VERSION="3.10.12"
PYTHON_DIR="$HOME/.local/python310"
VENV_DIR="$HOME/jupyter310_env"

if [ ! -d "$PYTHON_DIR" ]; then
    wget "https://www.python.org/ftp/python/$PYTHON_VERSION/Python-$PYTHON_VERSION.tgz"
    tar -xzf "Python-$PYTHON_VERSION.tgz"
    cd "Python-$PYTHON_VERSION" || exit
    ./configure --prefix="$PYTHON_DIR" --enable-optimizations
    make -j $(nproc)
    make install
    cd .. || exit
    rm -rf "Python-$PYTHON_VERSION" "Python-$PYTHON_VERSION.tgz"
else
    echo "Python 3.10.12 already installed, skipping..."
fi

if [ ! -d "$VENV_DIR" ]; then
    "$PYTHON_DIR/bin/python3.10" -m venv "$VENV_DIR"
else
    echo "Virtual environment already exists, skipping..."
fi

# 4.Install JupyterLab
source "$VENV_DIR/bin/activate"
pip install --upgrade pip -q
pip install jupyterlab
jupyter lab --generate-config -y
echo "c.ServerApp.port = 8890" >> "$HOME/.jupyter/jupyter_lab_config.py"
deactivate

Start JupyterLab#

Launch Jupyter Lab using the following commands. You will find the notebooks listed below:

1.sample_feature_engineering.ipynb - Feature encoding 2.sample_training_and_eval.ipynb - Model training and evaluation

source "$HOME/jupyter310_env/bin/activate"
jupyter lab --ip=0.0.0.0 --port=8890 ~/scikit-learn-notebook_2.0.0.0

Feature Engineering#

Feature engineering is the process of creating, transforming, or selecting input variables (features) to improve the performance of machine learning models.

The notebook 1.sample_feature_engineering.ipynb encodes the categorical variables using label encoding (assigning each category a unique integer) in sample.train.anom.csv and sample.test.anom.csv into numerical representations to facilitate the next step: model training.

The files generated after feature engineering are: 1.sample_encoded_features.train.anom.csv and 1.sample_encoded_features.test.anom.csv.

Example: Data before feature engineering:

test_suite_name

x_spec

test_suite_name_042

x_spec_010

test_suite_name_042

x_spec_010

test_suite_name_042

x_spec_010

test_suite_name_023

x_spec_005

test_suite_name_023

x_spec_005

test_suite_name_023

x_spec_005

Example: Data after feature engineering:

test_suite_name_encoded

x_spec_encoded

2

0

2

0

2

0

34

1

34

1

34

1

For code implementation, please see 1.sample_feature_engineering.ipynb

Training and Evaluation#

The notebook 2.sample_training_and_eval.ipynb contains the code for training and evaluation. It uses 1.sample_encoded_features.train.anom.csv as the training dataset and 1.sample_encoded_features.test.anom.csv as the evaluation dataset.

This section describes the process of training a LightGBM regression model with hyperparameter tuning using Optuna, and logging the results with MLflow.

Hyperparameter Tuning with Optuna for LightGBM#

The model’s performance is optimized by searching for the best combination of hyperparameters using Optuna. The objective function defines the search space and evaluation logic for each trial.

Objective Function Definition:

def objective(trial):
    params = {
        'n_estimators': trial.suggest_int('n_estimators', 100, 500),
        'learning_rate': trial.suggest_loguniform('learning_rate', 0.005, 0.1),
        'max_depth': trial.suggest_int('max_depth', 4, 10),
        'min_child_samples': trial.suggest_int('min_child_samples', 10, 50),
        'min_data_in_leaf': trial.suggest_int('min_data_in_leaf', 20, 100),
        'subsample': trial.suggest_float('subsample', 0.6, 1.0),
        'colsample_bytree': trial.suggest_float('colsample_bytree', 0.6, 1.0),
        'random_state': 42
    }

Hyperparameter Search Space:

Parameter

Description

Range / Distribution

n_estimators

Number of boosting rounds

Integer: 100 to 500

learning_rate

Step size shrinkage

Log-uniform: 0.005 to 0.1

max_depth

Maximum tree depth

Integer: 4 to 10

min_child_samples

Minimum samples per leaf

Integer: 10 to 50

min_data_in_leaf

Minimum data points in a leaf

Integer: 20 to 100

subsample

Row sampling ratio

Float: 0.6 to 1.0

colsample_bytree

Feature sampling ratio per tree

Float: 0.6 to 1.0

These parameters control the model’s complexity, regularization, and generalization ability.

Model Training and Evaluation#

The target variable is log-transformed before training to reduce skewness:

y_train_log = np.log1p(y_train)
model.fit(X_train, y_train_log)

Predictions are transformed back to the original scale:

y_pred = np.expm1(model.predict(X_test))

Model performance is evaluated using:

  • MAE (Mean Absolute Error)

  • RMSE (Root Mean Squared Error)

  • R2 Score

The R2 score is returned to Optuna to guide the optimization process.

study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=50)

Optuna runs 50 trials, each testing a different hyperparameter configuration to maximize the R2 score.

Final Model Training#

After optimization, the best parameters are used to train a final model:

best_params = study.best_params
final_model = LGBMRegressor(**best_params)
final_model.fit(X_train, np.log1p(y_train))

For code implementation, please see 2.sample_training_and_eval.ipynb

Using model in ACS RTDI#

We provide an example Test Program and application. You can use this example to understand how the Test Program sends data to the application for prediction.

You can use the following commands to download the example:

cd ~/apps/
curl http://10.44.5.139/apps/application-lgbm-v3.1.0.tar.gz -O
tar -zxf application-lgbm-v3.1.0.tar.gz

The Test Program sends prediction requests to the application#

// Connect to the container application
NexusTPI.target("lgbm-app").timeout(10);
...
// Use Nexus TPI to send a request to the container application
NexusTPI.request(jsonPayload);
// Receive the response
String response = NexusTPI.getResponse();
System.out.println("Response: "+response);

Logs generated by the Test Program during execution#

{
  "request": "Predict Target",
  "data": "acs_lot,acs_wafer,acs_ecid_x,acs_ecid_y,ecid_start,ecid_end,ecid_time,bin,final_bin_txt,ecid_lot_id,acs_tester_names,acs_last_test_suite,ecid_parametric_1,ecid_parametric_2,test_suite_name,x_spec,FBVDDQ,VDD0,VDD1,VDD2,VDD3,VDDMS\nacs_lot_003,1,0,2,2024/12/17 21:55,2024/12/17 22:24,558465.5,1,final_bin_txt_001,ecid_lot_id_002,acs_tester_names_001,acs_last_test_suite_002,1688.446289,2125.089242,test_suite_name_060,x_spec_001,1.35,0.9,0.9,0.9,0.9,0.9"
}
TPI res: 0
Response: Predicted:0.4830730316533709

For more details, please refer to the code.

You can follow the steps below to run this example:

In this example, we use Unified Server as the image repository

Please refer to “Use Unified Server as a Container Registry” and configure Unified Server as a Docker image registry, you need to follow these steps from the documentation:

  • Configure the hosts for the Host Controller

  • Get the docker certificates from the Unified Server

  • Configure the Project and Account in Harbor

  • Configure Edge Server

Build the application

cd ~
curl http://10.44.5.139/docker/ubuntu_20.04.tar -O
sudo docker load -i ubuntu_20.04.tar
cd ~/apps/application-lgbm-v3.1.0/rd-lgbm-app/
sudo docker build ./ --tag=unifiedserver.local/example-project/example-repo:lgbm

Push the application

sudo docker push unifiedserver.local/example-project/example-repo:lgbm

Configure the Nexus for container application deployment to the Edge Server:

gedit /opt/acs/nexus/conf/images.json

{
    "selector": {
        "device_name": "demo RTDI"
    },
    "edge": {
        "address": "<Edge IP>",
        "registry": {
            "address": "unifiedserver.local",
            "user": "robot$example_account",
            "password": "<Password>"
        },
        "containers": [
            {
                "name": "lgbm-app",
                "image": "example-project/example-repo:lgbm",
                "environment" : {
                    "ONEAPI_DEBUG": "3",
                    "ONEAPI_CONTROL_ZMQ_IP": "<Host Controller IP>"
                }
            }
        ]
    }
}

gedit /opt/acs/nexus/conf/acs_nexus.ini

[Auto_Deploy]
Enabled=false
...
[GUI]
Auto_Popup=true

To apply the modified configuration, restart Nexus:

sudo systemctl restart acs_nexus

Run the Test Program

cd ~/apps/application-lgbm-v3.1.0/
sh start_smt8.sh

For instructions on how to run the Test Program, please refer to “DPAT demo application on RTDI with SmarTest 8” -> “Run the SmarTest test program”.

On this page