[Bug Report] XGBoost problem in ml_ops/sm-mlflow_pipelines/sm-mlflow_pipelines.ipynb

[sdelahaies]

Link to the notebook
ml_ops/sm-mlflow_pipelines/sm-mlflow_pipelines.ipynb

Describe the bug
Pipeline execution fails due to two different version installations of XGboost. One by Conda from the Sagemaker Distribution Image. The other from Pip in requirements.txt.

error log in CloudWatch:

  * XGBoost is first installed with anaconda then upgraded with pip. To fix it please remove one of the installations.

To reproduce
Run the notebook on SageMaker Studio.

Fix

1. remove version number for XGBoost in requirements.txt
xgboost==1.7.6 -> xgboost

2. move early_stopping_rounds=5 to the instanciation of the XGBClassifier

xgb = XGBClassifier(n_estimators=num_round, early_stopping_rounds=5, **param)
xgb.fit(
    train_df,
    y_train,
    eval_set=[(validation_df, y_validation)]
)