Cloud Pak for Data/WML/notebooks/model risk management/MRM E2E with Openpages on CP4D.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "968913f3-8cb5-42a3-99d0-a40ef9b6f18a"
   },
   "source": [
    "<img src=\"https://github.com/pmservice/ai-openscale-tutorials/raw/master/notebooks/images/banner.png\" align=\"left\" alt=\"banner\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4f7234e8-ee83-41dd-9596-89b0b50274fe"
   },
   "source": [
    "This notebook should be run using Default Spark 3.3.x & Python 3.10 runtime environment. If you are viewing this in Watson Studio and do not see Python 3.10.x with Spark in the upper right corner of your screen, please update the runtime now. It requires service credentials for the following services:\n",
    "\n",
    "<li>Watson OpenScale\n",
    "<li>Watson Machine Learning\n",
    "<li>Db2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8a116e3e-792c-47a3-b287-e0c69d8f0cbe"
   },
   "source": [
    "## Credentials for IBM Cloud Pak for Data\n",
    "In the following code boxes, you need to replace the sample data with your own credentials. You can acquire the information from your system administrator or through the Cloud Pak for Data dashboard."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b91dea91-2604-47fc-897f-ead599a28538"
   },
   "source": [
    "### Obtaining your Watson OpenScale credentials\n",
    "You can retrieve the URL by running the following command: `oc get route -n namespace1 --no-headers | awk '{print $2}'`\n",
    "Replace the `namespace1` variable with your namespace.\n",
    "\n",
    "You should have been assigned a username and password when you were added to the Cloud Pak for Data system. You might need to ask either your database administrator or your system administrator for some of the information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "ea185c7c-00e5-4217-871e-2932711a5a89"
   },
   "outputs": [],
   "source": [
    "############################################################################################\n",
    "# Paste your Watson OpenScale credentials into the following section and then run this cell.\n",
    "############################################################################################\n",
    "WOS_CREDENTIALS = {\n",
    "      \"url\": \"<CP4D cluster URL>\",\n",
    "      \"username\": \"*****\",\n",
    "      \"password\": \"*****\"\n",
    "  }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "62a99f39-d6c4-4b3c-8629-829a71f226bd"
   },
   "source": [
    "### Your Watson OpenScale GUID\n",
    "For most systems, the default GUID is already entered for you. You would only need to update this particular entry if the GUID was changed from the default."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "febe1223-e2c8-4087-9447-b04435c6ac7f"
   },
   "outputs": [],
   "source": [
    "WOS_GUID=\"00000000-0000-0000-0000-000000000000\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3d4a05b2-7a03-46af-b833-e38c8f5ce433"
   },
   "source": [
    "### Your Watson OpenScale GUID\n",
    "For most systems, the default GUID is already entered for you. You would only need to update this particular entry if the GUID was changed from the default."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "b4b6654e-7ae0-4831-8728-422c2ede1de0"
   },
   "outputs": [],
   "source": [
    "WML_CREDENTIALS = WOS_CREDENTIALS.copy()\n",
    "WML_CREDENTIALS['instance_id']='openshift'\n",
    "WML_CREDENTIALS['version']='4.7' #If your env is CP4D 4.x.x then specify \"4.x.x\" instead of \"4.7\"\n",
    "WML_CREDENTIALS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2f472554-d55b-4314-addb-34dd6c23ddb6"
   },
   "source": [
    "### Your database credentials and schema\n",
    "Normally, you must obtain the database credentials and schema name from your database administrator, however, if you deployed one of the database options from the Cloud Pak for Data UI, you should be able to retrieve credentials yourself by going to My data and right-clicking on the database tile."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "f17f892f-1d20-4171-a9e4-e2f9bd2e9498"
   },
   "outputs": [],
   "source": [
    "####################################################################################\n",
    "# Paste your database credentials into the following section and then run this cell.\n",
    "####################################################################################\n",
    "DATABASE_CREDENTIALS = {\n",
    "      \"db\": \"SAMPLE\",\n",
    "      \"db_type\": \"db2\",\n",
    "      \"hostname\": \"<host>\",\n",
    "      \"password\": \"******\",\n",
    "      \"port\": \"<port>\",\n",
    "      \"username\": \"*****\"\n",
    "  }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "130f9cab-b28e-4387-a050-bc419ec7b639"
   },
   "outputs": [],
   "source": [
    "SCHEMA_NAME = \"<schema_name>\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "a9996c97-00bc-4669-8be9-9602db090306"
   },
   "source": [
    "## Integration system credentials (IBM OpenPages MRG) \n",
    "As part of the closed beta, you were sent an email message from IBM that provides the URL, username, and password of an IBM OpenPages system that you can use. Replace the following credentials with the ones that you received in that email message. Please make sure to provide the location of Openpages instance as \"location\" and fill it with \"ibmcloud\" or \"cp4d\". By default the value for location will be \"cp4d\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "58179477-a7ea-46d3-8639-7e7a71575137"
   },
   "outputs": [],
   "source": [
    "#########################################################################################\n",
    "# Paste your IBM OpenPages credentials into the following section and then run this cell.\n",
    "#########################################################################################\n",
    "OPENPAGES_CREDENTIALS = {\n",
    "     \"url\": \"<Openpages URL>\",\n",
    "     \"username\": \"*****\",\n",
    "     \"password\": \"*****\"\n",
    "     # \"location\": \"*****\"\n",
    "  }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bea08a1d-efcb-41ea-a745-4267c9ac9ff7"
   },
   "source": [
    "## Model ID\n",
    "For the model that you create, copy the `model_id` from the OpenPages Model URL from the browser address bar. For example, if you create a model and view it in the OpenPages UI, you might see a URL such as \"https://mrgbeta.op-ibm.com/app/jspview/react/grc/task-view/7916?\". In this example, the `model_id` is **7916**. Paste that value into the following code cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "b7cd4e79-3812-45f2-a2af-c9cc1f725a58"
   },
   "outputs": [],
   "source": [
    "####################################################################\n",
    "# Paste your model ID in the following field and then run this cell.\n",
    "####################################################################\n",
    "openpages_model_id=\"<model_id>\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b35b37cf-dc1b-4cd7-93f3-e9428294c3a2"
   },
   "source": [
    "## Package installation\n",
    "The following opensource packages must be installed into this notebook instance so that they are available to use during processing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "0a4906d8-8155-4983-a314-914e9fe9cd98"
   },
   "outputs": [],
   "source": [
    "!rm -rf /home/spark/shared/user-libs/python3.6*\n",
    "\n",
    "#If you are running this notebook in non IBM Watson Studio env then uncomment the below pip statements and run it\n",
    "#!pip install --upgrade pyspark==3.3.0  --no-cache | tail -n 1\n",
    "#!pip install numpy==1.20.1 --no-cache | tail -n 1\n",
    "#!pip install scikit-learn==1.1 --no-cache | tail -n 1\n",
    "#!pip install --upgrade pandas==1.2.3 --no-cache | tail -n 1\n",
    "#!pip install --upgrade SciPy --no-cache | tail -n 1\n",
    "\n",
    "!pip install --upgrade ibm-watson-machine-learning --user | tail -n 1\n",
    "!pip install --upgrade ibm_watson_openscale --no-cache | tail -n 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c3c3c3f2-34a2-4d8b-9a8e-0ba26b56eaeb"
   },
   "source": [
    "## Load the training data from Github\n",
    "So you don't have to manually generate training data, we've provided a sample and placed it in a publicly available Github repo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "b7baf81e-49e9-4658-ae58-43a8d494dd94"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2024-08-08 05:50:34--  https://raw.githubusercontent.com/pmservice/ai-openscale-tutorials/master/assets/historical_data/german_credit_risk/wml/german_credit_data_biased_training.csv\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.109.133, 185.199.110.133, 185.199.108.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.109.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 689622 (673K) [text/plain]\n",
      "Saving to: ‘german_credit_data_biased_training.csv’\n",
      "\n",
      "german_credit_data_ 100%[===================>] 673.46K  --.-KB/s    in 0.09s   \n",
      "\n",
      "2024-08-08 05:50:34 (7.19 MB/s) - ‘german_credit_data_biased_training.csv’ saved [689622/689622]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!rm german_credit_data_biased_training.csv\n",
    "!wget https://raw.githubusercontent.com/pmservice/ai-openscale-tutorials/master/assets/historical_data/german_credit_risk/wml/german_credit_data_biased_training.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "78fdc41f-fedd-4c67-a5a6-6369ece571c5"
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import requests\n",
    "import base64\n",
    "from requests.auth import HTTPBasicAuth\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "59d18c2d-87b6-40b9-83d1-7d5cdb7f8738"
   },
   "source": [
    "# Deploy the models\n",
    "\n",
    "The following cells deploy both the pre-prod and challenger models into the Watson Machine Learning instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "3a9a92db-3283-4a32-bbdd-efd1d3229733"
   },
   "outputs": [],
   "source": [
    "PRE_PROD_MODEL_NAME=\"German Credit Risk Model - PreProd\"\n",
    "PRE_PROD_DEPLOYMENT_NAME=\"German Credit Risk Model - PreProd\"\n",
    "\n",
    "PRE_PROD_CHALLENGER_MODEL_NAME=\"German Credit Risk Model - Challenger\"\n",
    "PRE_PROD_CHALLENGER_DEPLOYMENT_NAME=\"German Credit Risk Model - Challenger\"\n",
    "\n",
    "PRE_PROD_SPACE_NAME=\"wml_preprod\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "33f8c143-b900-4616-9f5b-287891dde7ea"
   },
   "source": [
    "## Initialize WML client and set the space_id parameter\n",
    "Analytic spaces are used to categorize assets and can be associated with a project inside of Watson Studio."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "6ab80789-141f-4404-9f27-c1097a979edf"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.10/site-packages/ibm_watson_machine_learning/client.py:184: UserWarning: The provided version: 4.7 is different from the current CP4D version: 5.0. Correct the wml_credentials with proper CP4D version number.\n",
      "  warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://cpd-cpd-instance.apps.wos415nfs2672.cp.fyre.ibm.com\n",
      "Note: 'limit' is not provided. Only first 50 records will be displayed if the number of records exceed 50\n",
      "------------------------------------  -----------------------------------------------------------------------  ------------------------\n",
      "ID                                    NAME                                                                     CREATED\n",
      "498630dc-3e0b-4470-b44d-583f2a393c0c  wml_preprod_space                                                        2024-08-08T00:19:18.705Z\n",
      "00969846-29b6-446a-959e-587df21992e0  prod                                                                     2024-08-07T06:33:55.297Z\n",
      "06873ce9-baf3-46a6-86c4-54c1e3d84429  tutorial_space                                                           2024-08-06T13:29:18.489Z\n",
      "e8b87647-a0e9-4932-920f-1e8f1d1f383d  tutorial-space                                                           2024-08-06T12:55:07.962Z\n",
      "832d6bbe-a2ea-451f-8d94-853da7c52b00  FrauDScore                                                               2024-08-05T07:48:21.490Z\n",
      "8d3a3e65-4875-4a5e-a6bd-617d2deeb9ac  Neelima                                                                  2024-08-05T05:42:10.490Z\n",
      "3a051b57-d5f6-4290-a2ac-5f66a6ed6d65  openscale-express-path-preprod-00000000-0000-0000-0000-1722503677398532  2024-08-01T09:56:04.456Z\n",
      "3d10da0f-497d-43d6-a8cd-71d15b833511  openscale-express-path-00000000-0000-0000-0000-1722503677398532          2024-08-01T09:55:43.160Z\n",
      "45683c18-10ea-4b25-8888-cf3c12646b8d  hk_pre_prod                                                              2024-07-27T12:34:07.223Z\n",
      "7f4afba9-1a6b-4f27-9a0a-507182adf269  goSales deployment space                                                 2024-07-09T14:42:26.982Z\n",
      "47dab2f7-5fe2-49bb-bf69-2cad9c7c5aa8  Notebook demo space                                                      2024-07-09T07:44:12.138Z\n",
      "a4bc50f8-8acf-47b5-b82d-103f311a5fa4  GCR Auto AI prod space                                                   2024-07-04T06:40:01.094Z\n",
      "4a0ad576-aad5-43ee-9457-11b4d2f39cf6  GCR AutoAI space Demo                                                    2024-07-03T14:26:17.440Z\n",
      "a8233d23-2f8b-40fc-b9a2-45aec8b53b83  openscale-express-path-preprod-00000000-0000-0000-0000-1719914010257861  2024-07-02T09:59:03.652Z\n",
      "3e3e2ae1-23d9-468d-abff-02c127d052ef  openscale-express-path-00000000-0000-0000-0000-1719914010257861          2024-07-02T09:58:32.157Z\n",
      "dcb2cb00-cb6f-47fe-8543-cffd5df0786a  notebooks CPD                                                            2024-07-01T17:10:55.025Z\n",
      "c3905865-4e73-43d1-a18f-24cf3a8b9163  wml_preprod                                                              2024-06-29T17:49:53.207Z\n",
      "7e5a8be6-9103-4c22-9c43-b66f3d8364de  poojitha_notebooks_space                                                 2024-06-29T13:53:46.645Z\n",
      "16ccd855-46bd-43ed-8219-5f00ac565d08  shreya-space                                                             2024-06-26T04:29:17.302Z\n",
      "bc3b9797-c509-4fb4-a424-f67b1e2ed4be  QUALITY_WMLV4_PREPROD                                                    2024-06-23T12:23:04.790Z\n",
      "e396e187-2977-47b4-ade3-1539f9f10adc  QUALITY_WMLV4_PROD                                                       2024-06-23T12:22:54.422Z\n",
      "40c4d032-0339-4da6-bfec-4bdb096c9650  shreya                                                                   2024-06-20T10:54:20.088Z\n",
      "088c142e-f35e-4e48-a30c-ad55a6edeecc  notebooks 5.0                                                            2024-06-13T04:42:07.336Z\n",
      "b9b3d3b4-6e26-4e16-807d-e8bf5e7d6984  MRM_WMLV4_PREPROD                                                        2024-06-12T15:49:26.571Z\n",
      "d22e2b6b-917c-4427-a40c-1a439352a742  MRM_WMLV4_PROD                                                           2024-06-12T15:49:16.185Z\n",
      "ce15e0f6-be30-4349-af47-35ae15983bf1  openscale-express-path-preprod-00000000-0000-0000-0000-000000000000      2024-06-04T05:18:51.988Z\n",
      "6264dc0e-087a-4dea-bcbc-6bd872b510fb  openscale-express-path-00000000-0000-0000-0000-000000000000              2024-06-04T05:18:30.811Z\n",
      "------------------------------------  -----------------------------------------------------------------------  ------------------------\n",
      "498630dc-3e0b-4470-b44d-583f2a393c0c\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'SUCCESS'"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from ibm_watson_machine_learning import APIClient\n",
    "wml_client = APIClient(WML_CREDENTIALS)\n",
    "print(wml_client.service_instance.get_url())\n",
    "\n",
    "wml_client.spaces.list()\n",
    "\n",
    "# Find and set the default space\n",
    "space_name=PRE_PROD_SPACE_NAME\n",
    "spaces = wml_client.spaces.get_details()['resources']\n",
    "space_id = None\n",
    "for space in spaces:\n",
    "    if space['entity']['name'] == space_name:\n",
    "        space_id = space[\"metadata\"][\"id\"]\n",
    "        print(space_id)\n",
    "if space_id is None:\n",
    "    space_id = wml_client.spaces.store(\n",
    "        meta_props={wml_client.spaces.ConfigurationMetaNames.NAME: space_name})[\"metadata\"][\"id\"]\n",
    "\n",
    "wml_client.set.default_space(space_id)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "35c292be1eee4a87806e83bfa8c96500"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0.360'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wml_client.version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cb16f23a-8466-4b34-a264-837ea7889d95"
   },
   "source": [
    "## Deploy the Spark Credit Risk Model to WML\n",
    "\n",
    "The following cell deploys the Spark version of the German Credit Risk Model to the specified Machine Learning instance in the specified deployment space. You'll notice that this version of the German Credit Risk model has an auc-roc score around 71%."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "76334df0-8942-4ba5-9798-7d0d33990741"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "24/08/08 05:50:54 WARN Utils: Your hostname, Nelwins-MacBook-Pro.local resolves to a loopback address: 127.0.0.1; using 192.168.0.103 instead (on interface en0)\n",
      "24/08/08 05:50:54 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address\n",
      "Setting default log level to \"WARN\".\n",
      "To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).\n",
      "24/08/08 05:50:55 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of records for training: 4485\n",
      "Number of records for evaluation: 515\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "24/08/08 05:51:05 WARN InstanceBuilder: Failed to load implementation from:dev.ludovic.netlib.blas.JNIBLAS\n",
      "24/08/08 05:51:05 WARN InstanceBuilder: Failed to load implementation from:dev.ludovic.netlib.blas.VectorBLAS\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy = 0.726321\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "24/08/08 05:51:08 WARN GarbageCollectionMetrics: To enable non-built-in garbage collector(s) List(G1 Concurrent GC), users should configure it(them) to spark.eventLog.gcMetrics.youngGenerationGarbageCollectors or spark.eventLog.gcMetrics.oldGenerationGarbageCollectors\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--  ----  -------  ----  ----------  ----------------\n",
      "ID  NAME  CREATED  TYPE  SPEC_STATE  SPEC_REPLACEMENT\n",
      "--  ----  -------  ----  ----------  ----------------\n",
      "{'name': 'German Credit Risk Model - PreProd', 'description': 'German Credit Risk Model - PreProd', 'software_spec': 'd11f2434-4fc7-58b7-8a62-755da64fdaf8', 'type': 'mllib_3.3'}\n",
      "Note: Warnings!! :  Software specification spark-mllib_3.3 specified for the wml_model is deprecated and will be removed in the future. We recommend you use spark-mllib_3.4 instead. For details see Supported Frameworks https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=specifications-supported-deployment-frameworks\n",
      "{'entity': {'hybrid_pipeline_software_specs': [], 'label_column': 'Risk', 'pipeline': {'id': '6474bdbb-3344-48b0-b58b-664913fb21eb'}, 'schemas': {'input': [{'fields': [{'metadata': {}, 'name': 'CheckingStatus', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanDuration', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'CreditHistory', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanPurpose', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanAmount', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'ExistingSavings', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'EmploymentDuration', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'InstallmentPercent', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Sex', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'OthersOnLoan', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'CurrentResidenceDuration', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'OwnsProperty', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Age', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'InstallmentPlans', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Housing', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'ExistingCreditsCount', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Job', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Dependents', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Telephone', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'ForeignWorker', 'nullable': True, 'type': 'string'}], 'id': '1', 'type': 'struct'}], 'output': []}, 'software_spec': {'id': 'd11f2434-4fc7-58b7-8a62-755da64fdaf8', 'name': 'spark-mllib_3.3'}, 'training_data_references': [{'id': '1', 'location': {}, 'schema': {'fields': [{'metadata': {}, 'name': 'CheckingStatus', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanDuration', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'CreditHistory', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanPurpose', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanAmount', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'ExistingSavings', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'EmploymentDuration', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'InstallmentPercent', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Sex', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'OthersOnLoan', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'CurrentResidenceDuration', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'OwnsProperty', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Age', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'InstallmentPlans', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Housing', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'ExistingCreditsCount', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Job', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Dependents', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Telephone', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'ForeignWorker', 'nullable': True, 'type': 'string'}, {'metadata': {'modeling_role': 'target'}, 'name': 'Risk', 'nullable': True, 'type': 'string'}], 'id': '1', 'type': 'struct'}, 'type': 'fs'}], 'type': 'mllib_3.3'}, 'metadata': {'created_at': '2024-08-08T00:21:19.987Z', 'description': 'German Credit Risk Model - PreProd', 'id': 'fe7ed7dd-2856-42d6-83ab-51c13079f78d', 'modified_at': '2024-08-08T00:21:28.415Z', 'name': 'German Credit Risk Model - PreProd', 'owner': '1000331001', 'resource_key': '921895f7-f546-4e05-b477-aae7c7bf889d', 'space_id': '498630dc-3e0b-4470-b44d-583f2a393c0c'}, 'system': {'warnings': [{'message': 'Software specification spark-mllib_3.3 specified for the wml_model is deprecated and will be removed in the future. We recommend you use spark-mllib_3.4 instead. For details see Supported Frameworks https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=specifications-supported-deployment-frameworks'}]}}\n",
      "------------------------------------  ----------------------------------  ------------------------  ---------  ----------  ----------------\n",
      "ID                                    NAME                                CREATED                   TYPE       SPEC_STATE  SPEC_REPLACEMENT\n",
      "fe7ed7dd-2856-42d6-83ab-51c13079f78d  German Credit Risk Model - PreProd  2024-08-08T00:21:19.002Z  mllib_3.3  deprecated  spark-mllib_3.4\n",
      "------------------------------------  ----------------------------------  ------------------------  ---------  ----------  ----------------\n",
      "This method is deprecated, please use get_model_id()\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.10/site-packages/ibm_watson_machine_learning/repository.py:563: UserWarning: This method is deprecated, please use get_model_id()\n",
      "  warn(\"This method is deprecated, please use get_model_id()\")\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Deploying model...\n",
      "\n",
      "\n",
      "#######################################################################################\n",
      "\n",
      "Synchronous deployment creation for uid: 'fe7ed7dd-2856-42d6-83ab-51c13079f78d' started\n",
      "\n",
      "#######################################################################################\n",
      "\n",
      "\n",
      "initializing\n",
      "Note: Software specification spark-mllib_3.3 is deprecated. Use spark-mllib_3.4 software specification instead when saving a spark model. For details, see https://www.ibm.com/support/producthub/icpdata/docs/content/SSQNUZ_latest/wsj/wmls/wmls-deploy-python-types.html.\n",
      "\n",
      "ready\n",
      "\n",
      "\n",
      "------------------------------------------------------------------------------------------------\n",
      "Successfully finished deployment creation, deployment_uid='42b79776-470c-4d05-9115-76c4f5fffeab'\n",
      "------------------------------------------------------------------------------------------------\n",
      "\n",
      "\n",
      "Model id: fe7ed7dd-2856-42d6-83ab-51c13079f78d\n",
      "Deployment id: 42b79776-470c-4d05-9115-76c4f5fffeab\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'predictions': [{'fields': ['CheckingStatus',\n",
       "    'LoanDuration',\n",
       "    'CreditHistory',\n",
       "    'LoanPurpose',\n",
       "    'LoanAmount',\n",
       "    'ExistingSavings',\n",
       "    'EmploymentDuration',\n",
       "    'InstallmentPercent',\n",
       "    'Sex',\n",
       "    'OthersOnLoan',\n",
       "    'CurrentResidenceDuration',\n",
       "    'OwnsProperty',\n",
       "    'Age',\n",
       "    'InstallmentPlans',\n",
       "    'Housing',\n",
       "    'ExistingCreditsCount',\n",
       "    'Job',\n",
       "    'Dependents',\n",
       "    'Telephone',\n",
       "    'ForeignWorker',\n",
       "    'CheckingStatus_IX',\n",
       "    'CreditHistory_IX',\n",
       "    'EmploymentDuration_IX',\n",
       "    'ExistingSavings_IX',\n",
       "    'ForeignWorker_IX',\n",
       "    'Housing_IX',\n",
       "    'InstallmentPlans_IX',\n",
       "    'Job_IX',\n",
       "    'LoanPurpose_IX',\n",
       "    'OthersOnLoan_IX',\n",
       "    'OwnsProperty_IX',\n",
       "    'Sex_IX',\n",
       "    'Telephone_IX',\n",
       "    'features',\n",
       "    'rawPrediction',\n",
       "    'probability',\n",
       "    'prediction',\n",
       "    'predictedLabel'],\n",
       "   'values': [['no_checking',\n",
       "     13,\n",
       "     'credits_paid_to_date',\n",
       "     'car_new',\n",
       "     1343,\n",
       "     '100_to_500',\n",
       "     '1_to_4',\n",
       "     2,\n",
       "     'female',\n",
       "     'none',\n",
       "     3,\n",
       "     'savings_insurance',\n",
       "     46,\n",
       "     'none',\n",
       "     'own',\n",
       "     2,\n",
       "     'skilled',\n",
       "     1,\n",
       "     'none',\n",
       "     'yes',\n",
       "     0.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     [21,\n",
       "      [1, 3, 5, 13, 14, 15, 16, 17, 18, 19, 20],\n",
       "      [1.0, 1.0, 1.0, 13.0, 1343.0, 2.0, 3.0, 13.0, 46.0, 2.0, 1.0]],\n",
       "     [0.37923501094948475, -0.37923501094948475],\n",
       "     [0.6810214668555962, 0.31897853314440383],\n",
       "     0.0,\n",
       "     'No Risk'],\n",
       "    ['no_checking',\n",
       "     24,\n",
       "     'prior_payments_delayed',\n",
       "     'furniture',\n",
       "     4567,\n",
       "     '500_to_1000',\n",
       "     '1_to_4',\n",
       "     4,\n",
       "     'male',\n",
       "     'none',\n",
       "     4,\n",
       "     'savings_insurance',\n",
       "     36,\n",
       "     'none',\n",
       "     'free',\n",
       "     2,\n",
       "     'management_self-employed',\n",
       "     1,\n",
       "     'none',\n",
       "     'yes',\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     [21,\n",
       "      [2, 3, 9, 10, 13, 14, 15, 16, 17, 18, 19, 20],\n",
       "      [1.0, 2.0, 2.0, 2.0, 24.0, 4567.0, 4.0, 4.0, 24.0, 36.0, 2.0, 1.0]],\n",
       "     [0.40678948974997703, -0.40678948974997703],\n",
       "     [0.6928716383154149, 0.3071283616845851],\n",
       "     0.0,\n",
       "     'No Risk'],\n",
       "    ['0_to_200',\n",
       "     26,\n",
       "     'all_credits_paid_back',\n",
       "     'car_new',\n",
       "     863,\n",
       "     'less_100',\n",
       "     'less_1',\n",
       "     2,\n",
       "     'female',\n",
       "     'co-applicant',\n",
       "     2,\n",
       "     'real_estate',\n",
       "     38,\n",
       "     'none',\n",
       "     'own',\n",
       "     1,\n",
       "     'skilled',\n",
       "     1,\n",
       "     'none',\n",
       "     'yes',\n",
       "     2.0,\n",
       "     3.0,\n",
       "     3.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     1.0,\n",
       "     2.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     [2.0,\n",
       "      3.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      3.0,\n",
       "      1.0,\n",
       "      1.0,\n",
       "      2.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      26.0,\n",
       "      863.0,\n",
       "      2.0,\n",
       "      2.0,\n",
       "      26.0,\n",
       "      38.0,\n",
       "      1.0,\n",
       "      1.0],\n",
       "     [1.08446701248213, -1.08446701248213],\n",
       "     [0.8974248814242887, 0.10257511857571133],\n",
       "     0.0,\n",
       "     'No Risk'],\n",
       "    ['0_to_200',\n",
       "     14,\n",
       "     'no_credits',\n",
       "     'car_new',\n",
       "     2368,\n",
       "     'less_100',\n",
       "     '1_to_4',\n",
       "     3,\n",
       "     'female',\n",
       "     'none',\n",
       "     3,\n",
       "     'real_estate',\n",
       "     29,\n",
       "     'none',\n",
       "     'own',\n",
       "     1,\n",
       "     'skilled',\n",
       "     1,\n",
       "     'none',\n",
       "     'yes',\n",
       "     2.0,\n",
       "     4.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     [21,\n",
       "      [0, 1, 5, 7, 13, 14, 15, 16, 17, 18, 19, 20],\n",
       "      [2.0, 4.0, 1.0, 2.0, 14.0, 2368.0, 3.0, 3.0, 14.0, 29.0, 1.0, 1.0]],\n",
       "     [1.4143966876630347, -1.4143966876630347],\n",
       "     [0.9442120764123368, 0.05578792358766316],\n",
       "     0.0,\n",
       "     'No Risk'],\n",
       "    ['0_to_200',\n",
       "     4,\n",
       "     'no_credits',\n",
       "     'car_new',\n",
       "     250,\n",
       "     'less_100',\n",
       "     'unemployed',\n",
       "     2,\n",
       "     'female',\n",
       "     'none',\n",
       "     3,\n",
       "     'real_estate',\n",
       "     23,\n",
       "     'none',\n",
       "     'rent',\n",
       "     1,\n",
       "     'management_self-employed',\n",
       "     1,\n",
       "     'none',\n",
       "     'yes',\n",
       "     2.0,\n",
       "     4.0,\n",
       "     4.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     [2.0,\n",
       "      4.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      4.0,\n",
       "      1.0,\n",
       "      0.0,\n",
       "      2.0,\n",
       "      0.0,\n",
       "      1.0,\n",
       "      2.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      4.0,\n",
       "      250.0,\n",
       "      2.0,\n",
       "      3.0,\n",
       "      4.0,\n",
       "      23.0,\n",
       "      1.0,\n",
       "      1.0],\n",
       "     [1.5173690203089494, -1.5173690203089494],\n",
       "     [0.9541190318258127, 0.0458809681741873],\n",
       "     0.0,\n",
       "     'No Risk'],\n",
       "    ['no_checking',\n",
       "     17,\n",
       "     'credits_paid_to_date',\n",
       "     'car_new',\n",
       "     832,\n",
       "     '100_to_500',\n",
       "     '1_to_4',\n",
       "     2,\n",
       "     'male',\n",
       "     'none',\n",
       "     2,\n",
       "     'real_estate',\n",
       "     42,\n",
       "     'none',\n",
       "     'own',\n",
       "     1,\n",
       "     'skilled',\n",
       "     1,\n",
       "     'none',\n",
       "     'yes',\n",
       "     0.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     [21,\n",
       "      [1, 3, 7, 13, 14, 15, 16, 17, 18, 19, 20],\n",
       "      [1.0, 1.0, 2.0, 17.0, 832.0, 2.0, 2.0, 17.0, 42.0, 1.0, 1.0]],\n",
       "     [1.1122314956561121, -1.1122314956561121],\n",
       "     [0.9024248873010724, 0.0975751126989276],\n",
       "     0.0,\n",
       "     'No Risk'],\n",
       "    ['no_checking',\n",
       "     33,\n",
       "     'outstanding_credit',\n",
       "     'appliances',\n",
       "     5696,\n",
       "     'unknown',\n",
       "     'greater_7',\n",
       "     4,\n",
       "     'male',\n",
       "     'co-applicant',\n",
       "     4,\n",
       "     'unknown',\n",
       "     54,\n",
       "     'none',\n",
       "     'free',\n",
       "     2,\n",
       "     'skilled',\n",
       "     1,\n",
       "     'yes',\n",
       "     'yes',\n",
       "     0.0,\n",
       "     2.0,\n",
       "     2.0,\n",
       "     4.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     4.0,\n",
       "     1.0,\n",
       "     3.0,\n",
       "     0.0,\n",
       "     1.0,\n",
       "     [0.0,\n",
       "      2.0,\n",
       "      4.0,\n",
       "      4.0,\n",
       "      2.0,\n",
       "      0.0,\n",
       "      1.0,\n",
       "      3.0,\n",
       "      0.0,\n",
       "      2.0,\n",
       "      0.0,\n",
       "      1.0,\n",
       "      0.0,\n",
       "      33.0,\n",
       "      5696.0,\n",
       "      4.0,\n",
       "      4.0,\n",
       "      33.0,\n",
       "      54.0,\n",
       "      2.0,\n",
       "      1.0],\n",
       "     [-1.3184238002172723, 1.3184238002172723],\n",
       "     [0.06680429265446867, 0.9331957073455314],\n",
       "     1.0,\n",
       "     'Risk'],\n",
       "    ['0_to_200',\n",
       "     13,\n",
       "     'prior_payments_delayed',\n",
       "     'retraining',\n",
       "     1375,\n",
       "     '100_to_500',\n",
       "     '4_to_7',\n",
       "     3,\n",
       "     'male',\n",
       "     'none',\n",
       "     3,\n",
       "     'real_estate',\n",
       "     37,\n",
       "     'none',\n",
       "     'own',\n",
       "     2,\n",
       "     'management_self-employed',\n",
       "     1,\n",
       "     'none',\n",
       "     'yes',\n",
       "     2.0,\n",
       "     0.0,\n",
       "     1.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     8.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     [2.0,\n",
       "      0.0,\n",
       "      8.0,\n",
       "      1.0,\n",
       "      1.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      2.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      2.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      13.0,\n",
       "      1375.0,\n",
       "      3.0,\n",
       "      3.0,\n",
       "      13.0,\n",
       "      37.0,\n",
       "      2.0,\n",
       "      1.0],\n",
       "     [0.996270448775496, -0.996270448775496],\n",
       "     [0.8800116928980676, 0.11998830710193242],\n",
       "     0.0,\n",
       "     'No Risk']]}]}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy \n",
    "numpy.version.version\n",
    "\n",
    "import pandas as pd\n",
    "import json\n",
    "\n",
    "from pyspark import SparkContext, SQLContext\n",
    "from pyspark.ml import Pipeline\n",
    "from pyspark.ml.classification import RandomForestClassifier,GBTClassifier\n",
    "from pyspark.ml.evaluation import BinaryClassificationEvaluator\n",
    "from pyspark.ml.feature import StringIndexer, VectorAssembler, IndexToString\n",
    "from pyspark.sql.types import StructType, DoubleType, StringType, ArrayType\n",
    "\n",
    "from pyspark.sql import SparkSession\n",
    "from pyspark import SparkFiles\n",
    "\n",
    "spark = SparkSession.builder.getOrCreate()\n",
    "pd_data = pd.read_csv(\"german_credit_data_biased_training.csv\", sep=\",\", header=0)\n",
    "spark_df = spark.read.csv(path=\"german_credit_data_biased_training.csv\", sep=\",\", header=True, inferSchema=True)\n",
    "spark_df.head()\n",
    "\n",
    "(train_data, test_data) = spark_df.randomSplit([0.9, 0.1], 24)\n",
    "print(\"Number of records for training: \" + str(train_data.count()))\n",
    "print(\"Number of records for evaluation: \" + str(test_data.count()))\n",
    "\n",
    "si_CheckingStatus = StringIndexer(inputCol='CheckingStatus', outputCol='CheckingStatus_IX')\n",
    "si_CreditHistory = StringIndexer(inputCol='CreditHistory', outputCol='CreditHistory_IX')\n",
    "si_LoanPurpose = StringIndexer(inputCol='LoanPurpose', outputCol='LoanPurpose_IX')\n",
    "si_ExistingSavings = StringIndexer(inputCol='ExistingSavings', outputCol='ExistingSavings_IX')\n",
    "si_EmploymentDuration = StringIndexer(inputCol='EmploymentDuration', outputCol='EmploymentDuration_IX')\n",
    "si_Sex = StringIndexer(inputCol='Sex', outputCol='Sex_IX')\n",
    "si_OthersOnLoan = StringIndexer(inputCol='OthersOnLoan', outputCol='OthersOnLoan_IX')\n",
    "si_OwnsProperty = StringIndexer(inputCol='OwnsProperty', outputCol='OwnsProperty_IX')\n",
    "si_InstallmentPlans = StringIndexer(inputCol='InstallmentPlans', outputCol='InstallmentPlans_IX')\n",
    "si_Housing = StringIndexer(inputCol='Housing', outputCol='Housing_IX')\n",
    "si_Job = StringIndexer(inputCol='Job', outputCol='Job_IX')\n",
    "si_Telephone = StringIndexer(inputCol='Telephone', outputCol='Telephone_IX')\n",
    "si_ForeignWorker = StringIndexer(inputCol='ForeignWorker', outputCol='ForeignWorker_IX')\n",
    "si_Label = StringIndexer(inputCol=\"Risk\", outputCol=\"label\").fit(spark_df)\n",
    "label_converter = IndexToString(inputCol=\"prediction\", outputCol=\"predictedLabel\", labels=si_Label.labels)\n",
    "\n",
    "va_features = VectorAssembler(\n",
    "inputCols=[\"CheckingStatus_IX\", \"CreditHistory_IX\", \"LoanPurpose_IX\", \"ExistingSavings_IX\",\n",
    "           \"EmploymentDuration_IX\", \"Sex_IX\", \"OthersOnLoan_IX\", \"OwnsProperty_IX\", \"InstallmentPlans_IX\",\n",
    "           \"Housing_IX\", \"Job_IX\", \"Telephone_IX\", \"ForeignWorker_IX\", \"LoanDuration\", \"LoanAmount\",\n",
    "           \"InstallmentPercent\", \"CurrentResidenceDuration\", \"LoanDuration\", \"Age\", \"ExistingCreditsCount\",\n",
    "           \"Dependents\"], outputCol=\"features\")\n",
    "\n",
    "classifier=GBTClassifier(featuresCol=\"features\")\n",
    "\n",
    "pipeline = Pipeline(\n",
    "stages=[si_CheckingStatus, si_CreditHistory, si_EmploymentDuration, si_ExistingSavings, si_ForeignWorker,\n",
    "        si_Housing, si_InstallmentPlans, si_Job, si_LoanPurpose, si_OthersOnLoan,\n",
    "        si_OwnsProperty, si_Sex, si_Telephone, si_Label, va_features, classifier, label_converter])\n",
    "\n",
    "model = pipeline.fit(train_data)\n",
    "predictions = model.transform(test_data)\n",
    "evaluator = BinaryClassificationEvaluator(rawPredictionCol=\"prediction\")\n",
    "auc = evaluator.evaluate(predictions)\n",
    "\n",
    "print(\"Accuracy = %g\" % auc)\n",
    "\n",
    "# Remove existing model and deployment\n",
    "MODEL_NAME=PRE_PROD_MODEL_NAME\n",
    "DEPLOYMENT_NAME=PRE_PROD_DEPLOYMENT_NAME\n",
    "\n",
    "deployment_details = wml_client.deployments.get_details()\n",
    "# for deployment in deployment_details['resources']:\n",
    "#     deployment_id = wml_client.deployments.get_id(deployment)\n",
    "#     model_id = deployment['entity']['asset']['id']\n",
    "#     if deployment['entity']['name'] == DEPLOYMENT_NAME:\n",
    "#         print('Deleting deployment id', deployment_id)\n",
    "#         wml_client.deployments.delete(deployment_id)\n",
    "#         print('Deleting model id', model_id)\n",
    "#         wml_client.repository.delete(model_id)\n",
    "wml_client.repository.list_models()\n",
    "\n",
    "# Save Model\n",
    "software_spec_uid = wml_client.software_specifications.get_id_by_name(\"spark-mllib_3.3\")\n",
    "model_props_rf = {\n",
    "    wml_client.repository.ModelMetaNames.NAME: MODEL_NAME,\n",
    "    wml_client.repository.ModelMetaNames.DESCRIPTION: MODEL_NAME,\n",
    "    wml_client.repository.ModelMetaNames.SOFTWARE_SPEC_UID: software_spec_uid,\n",
    "    wml_client.repository.ModelMetaNames.TYPE: 'mllib_3.3'\n",
    "}\n",
    "print(model_props_rf)\n",
    "published_model_details = wml_client.repository.store_model(model=model, meta_props=model_props_rf, training_data=train_data, pipeline=pipeline)\n",
    "print(published_model_details)\n",
    "\n",
    "# List models in the repository\n",
    "wml_client.repository.list_models()\n",
    "\n",
    "# Get the model UID\n",
    "pre_prod_model_uid = wml_client.repository.get_model_uid(published_model_details)\n",
    "pre_prod_model_uid\n",
    "\n",
    "\n",
    "# Deploy model\n",
    "wml_deployments = wml_client.deployments.get_details()\n",
    "pre_prod_deployment_uid = None\n",
    "for deployment in wml_deployments['resources']:\n",
    "    if DEPLOYMENT_NAME == deployment['entity']['name']:\n",
    "        pre_prod_deployment_uid = wml_client.deployments.get_id(deployment)\n",
    "        break\n",
    "\n",
    "if pre_prod_deployment_uid is None:\n",
    "    print(\"Deploying model...\")\n",
    "    meta_props = {\n",
    "        wml_client.deployments.ConfigurationMetaNames.NAME: DEPLOYMENT_NAME,\n",
    "        wml_client.deployments.ConfigurationMetaNames.DESCRIPTION: DEPLOYMENT_NAME,\n",
    "        wml_client.deployments.ConfigurationMetaNames.ONLINE: {}\n",
    "    }\n",
    "    deployment = wml_client.deployments.create(artifact_uid=pre_prod_model_uid, name=DEPLOYMENT_NAME, meta_props=meta_props)\n",
    "    pre_prod_deployment_uid = wml_client.deployments.get_id(deployment)\n",
    "\n",
    "print(\"Model id: {}\".format(pre_prod_model_uid))\n",
    "print(\"Deployment id: {}\".format(pre_prod_deployment_uid))\n",
    "\n",
    "pre_prod_deployment_uid=wml_client.deployments.get_id(deployment)\n",
    "pre_prod_deployment_uid\n",
    "\n",
    "fields = [\"CheckingStatus\",\"LoanDuration\",\"CreditHistory\",\"LoanPurpose\",\"LoanAmount\",\"ExistingSavings\",\"EmploymentDuration\",\"InstallmentPercent\",\"Sex\",\"OthersOnLoan\",\"CurrentResidenceDuration\",\"OwnsProperty\",\"Age\",\"InstallmentPlans\",\"Housing\",\"ExistingCreditsCount\",\"Job\",\"Dependents\",\"Telephone\",\"ForeignWorker\"]\n",
    "values = [\n",
    "  [\"no_checking\",13,\"credits_paid_to_date\",\"car_new\",1343,\"100_to_500\",\"1_to_4\",2,\"female\",\"none\",3,\"savings_insurance\",46,\"none\",\"own\",2,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"no_checking\",24,\"prior_payments_delayed\",\"furniture\",4567,\"500_to_1000\",\"1_to_4\",4,\"male\",\"none\",4,\"savings_insurance\",36,\"none\",\"free\",2,\"management_self-employed\",1,\"none\",\"yes\"],\n",
    "  [\"0_to_200\",26,\"all_credits_paid_back\",\"car_new\",863,\"less_100\",\"less_1\",2,\"female\",\"co-applicant\",2,\"real_estate\",38,\"none\",\"own\",1,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"0_to_200\",14,\"no_credits\",\"car_new\",2368,\"less_100\",\"1_to_4\",3,\"female\",\"none\",3,\"real_estate\",29,\"none\",\"own\",1,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"0_to_200\",4,\"no_credits\",\"car_new\",250,\"less_100\",\"unemployed\",2,\"female\",\"none\",3,\"real_estate\",23,\"none\",\"rent\",1,\"management_self-employed\",1,\"none\",\"yes\"],\n",
    "  [\"no_checking\",17,\"credits_paid_to_date\",\"car_new\",832,\"100_to_500\",\"1_to_4\",2,\"male\",\"none\",2,\"real_estate\",42,\"none\",\"own\",1,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"no_checking\",33,\"outstanding_credit\",\"appliances\",5696,\"unknown\",\"greater_7\",4,\"male\",\"co-applicant\",4,\"unknown\",54,\"none\",\"free\",2,\"skilled\",1,\"yes\",\"yes\"],\n",
    "  [\"0_to_200\",13,\"prior_payments_delayed\",\"retraining\",1375,\"100_to_500\",\"4_to_7\",3,\"male\",\"none\",3,\"real_estate\",37,\"none\",\"own\",2,\"management_self-employed\",1,\"none\",\"yes\"]\n",
    "]\n",
    "\n",
    "payload_scoring = {\"fields\": fields,\"values\": values}\n",
    "payload = {\n",
    "    wml_client.deployments.ScoringMetaNames.INPUT_DATA: [payload_scoring]\n",
    "}\n",
    "scoring_response = wml_client.deployments.score(pre_prod_deployment_uid, payload)\n",
    "\n",
    "scoring_response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "60def73f-e179-4083-91c7-215debf39eba"
   },
   "source": [
    "## Deploy the Scikit-Learn Credit Risk Model to Watson Machine Learning\n",
    "\n",
    "The following cell deploys the Scikit-learn version of the German Credit Risk Model to the specified Machine Learning instance in the specified deployment space. This version of the German Credit Risk model has an auc-roc score around 85% and will be called the \"Challenger.\"\n",
    "\n",
    "Note : If you are running this notebook on python 3.10, then use `runtime-23.1-py3.10` and `scikit-learn_1.1` as software specifications name and ModelMetaNames.TYPE respectively"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "50193e3df87b49008440e9693efc2a47"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.1.1'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import sklearn\n",
    "sklearn.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "178d5505-d7d6-44c7-83cd-3d72fe821adb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of records for training: 4485\n",
      "Number of records for evaluation: 515\n",
      "Accuracy = 0.726321\n",
      "------------------------------------  -------------------------------------  ------------------------  ----------------  ----------  ----------------\n",
      "ID                                    NAME                                   CREATED                   TYPE              SPEC_STATE  SPEC_REPLACEMENT\n",
      "f94340ae-57bf-4e49-be5c-7156cd0ec09a  German Credit Risk Model - Challenger  2024-08-08T00:24:24.002Z  scikit-learn_1.1  supported\n",
      "fe7ed7dd-2856-42d6-83ab-51c13079f78d  German Credit Risk Model - PreProd     2024-08-08T00:21:19.002Z  mllib_3.3         deprecated  spark-mllib_3.4\n",
      "------------------------------------  -------------------------------------  ------------------------  ----------------  ----------  ----------------\n",
      "{'name': 'German Credit Risk Model - PreProd', 'description': 'German Credit Risk Model - PreProd', 'software_spec': 'd11f2434-4fc7-58b7-8a62-755da64fdaf8', 'type': 'mllib_3.3'}\n",
      "Note: Warnings!! :  Software specification spark-mllib_3.3 specified for the wml_model is deprecated and will be removed in the future. We recommend you use spark-mllib_3.4 instead. For details see Supported Frameworks https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=specifications-supported-deployment-frameworks\n",
      "{'entity': {'hybrid_pipeline_software_specs': [], 'label_column': 'Risk', 'pipeline': {'id': '59dd3942-ecc2-41c1-b2da-09c869fa909f'}, 'schemas': {'input': [{'fields': [{'metadata': {}, 'name': 'CheckingStatus', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanDuration', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'CreditHistory', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanPurpose', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanAmount', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'ExistingSavings', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'EmploymentDuration', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'InstallmentPercent', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Sex', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'OthersOnLoan', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'CurrentResidenceDuration', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'OwnsProperty', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Age', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'InstallmentPlans', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Housing', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'ExistingCreditsCount', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Job', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Dependents', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Telephone', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'ForeignWorker', 'nullable': True, 'type': 'string'}], 'id': '1', 'type': 'struct'}], 'output': []}, 'software_spec': {'id': 'd11f2434-4fc7-58b7-8a62-755da64fdaf8', 'name': 'spark-mllib_3.3'}, 'training_data_references': [{'id': '1', 'location': {}, 'schema': {'fields': [{'metadata': {}, 'name': 'CheckingStatus', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanDuration', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'CreditHistory', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanPurpose', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanAmount', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'ExistingSavings', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'EmploymentDuration', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'InstallmentPercent', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Sex', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'OthersOnLoan', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'CurrentResidenceDuration', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'OwnsProperty', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Age', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'InstallmentPlans', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Housing', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'ExistingCreditsCount', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Job', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Dependents', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Telephone', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'ForeignWorker', 'nullable': True, 'type': 'string'}, {'metadata': {'modeling_role': 'target'}, 'name': 'Risk', 'nullable': True, 'type': 'string'}], 'id': '1', 'type': 'struct'}, 'type': 'fs'}], 'type': 'mllib_3.3'}, 'metadata': {'created_at': '2024-08-08T00:27:25.050Z', 'description': 'German Credit Risk Model - PreProd', 'id': 'b9cc4287-c830-4856-8d42-96347cc5e89e', 'modified_at': '2024-08-08T00:27:33.307Z', 'name': 'German Credit Risk Model - PreProd', 'owner': '1000331001', 'resource_key': '12c7396a-a261-48f3-81ab-bd4044a48f82', 'space_id': '498630dc-3e0b-4470-b44d-583f2a393c0c'}, 'system': {'warnings': [{'message': 'Software specification spark-mllib_3.3 specified for the wml_model is deprecated and will be removed in the future. We recommend you use spark-mllib_3.4 instead. For details see Supported Frameworks https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=specifications-supported-deployment-frameworks'}]}}\n",
      "------------------------------------  -------------------------------------  ------------------------  ----------------  ----------  ----------------\n",
      "ID                                    NAME                                   CREATED                   TYPE              SPEC_STATE  SPEC_REPLACEMENT\n",
      "b9cc4287-c830-4856-8d42-96347cc5e89e  German Credit Risk Model - PreProd     2024-08-08T00:27:25.002Z  mllib_3.3         deprecated  spark-mllib_3.4\n",
      "f94340ae-57bf-4e49-be5c-7156cd0ec09a  German Credit Risk Model - Challenger  2024-08-08T00:24:24.002Z  scikit-learn_1.1  supported\n",
      "fe7ed7dd-2856-42d6-83ab-51c13079f78d  German Credit Risk Model - PreProd     2024-08-08T00:21:19.002Z  mllib_3.3         deprecated  spark-mllib_3.4\n",
      "------------------------------------  -------------------------------------  ------------------------  ----------------  ----------  ----------------\n",
      "This method is deprecated, please use get_model_id()\n",
      "Model id: b9cc4287-c830-4856-8d42-96347cc5e89e\n",
      "Deployment id: 42b79776-470c-4d05-9115-76c4f5fffeab\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'predictions': [{'fields': ['CheckingStatus',\n",
       "    'LoanDuration',\n",
       "    'CreditHistory',\n",
       "    'LoanPurpose',\n",
       "    'LoanAmount',\n",
       "    'ExistingSavings',\n",
       "    'EmploymentDuration',\n",
       "    'InstallmentPercent',\n",
       "    'Sex',\n",
       "    'OthersOnLoan',\n",
       "    'CurrentResidenceDuration',\n",
       "    'OwnsProperty',\n",
       "    'Age',\n",
       "    'InstallmentPlans',\n",
       "    'Housing',\n",
       "    'ExistingCreditsCount',\n",
       "    'Job',\n",
       "    'Dependents',\n",
       "    'Telephone',\n",
       "    'ForeignWorker',\n",
       "    'CheckingStatus_IX',\n",
       "    'CreditHistory_IX',\n",
       "    'EmploymentDuration_IX',\n",
       "    'ExistingSavings_IX',\n",
       "    'ForeignWorker_IX',\n",
       "    'Housing_IX',\n",
       "    'InstallmentPlans_IX',\n",
       "    'Job_IX',\n",
       "    'LoanPurpose_IX',\n",
       "    'OthersOnLoan_IX',\n",
       "    'OwnsProperty_IX',\n",
       "    'Sex_IX',\n",
       "    'Telephone_IX',\n",
       "    'features',\n",
       "    'rawPrediction',\n",
       "    'probability',\n",
       "    'prediction',\n",
       "    'predictedLabel'],\n",
       "   'values': [['no_checking',\n",
       "     13,\n",
       "     'credits_paid_to_date',\n",
       "     'car_new',\n",
       "     1343,\n",
       "     '100_to_500',\n",
       "     '1_to_4',\n",
       "     2,\n",
       "     'female',\n",
       "     'none',\n",
       "     3,\n",
       "     'savings_insurance',\n",
       "     46,\n",
       "     'none',\n",
       "     'own',\n",
       "     2,\n",
       "     'skilled',\n",
       "     1,\n",
       "     'none',\n",
       "     'yes',\n",
       "     0.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     [21,\n",
       "      [1, 3, 5, 13, 14, 15, 16, 17, 18, 19, 20],\n",
       "      [1.0, 1.0, 1.0, 13.0, 1343.0, 2.0, 3.0, 13.0, 46.0, 2.0, 1.0]],\n",
       "     [0.37923501094948475, -0.37923501094948475],\n",
       "     [0.6810214668555962, 0.31897853314440383],\n",
       "     0.0,\n",
       "     'No Risk'],\n",
       "    ['no_checking',\n",
       "     24,\n",
       "     'prior_payments_delayed',\n",
       "     'furniture',\n",
       "     4567,\n",
       "     '500_to_1000',\n",
       "     '1_to_4',\n",
       "     4,\n",
       "     'male',\n",
       "     'none',\n",
       "     4,\n",
       "     'savings_insurance',\n",
       "     36,\n",
       "     'none',\n",
       "     'free',\n",
       "     2,\n",
       "     'management_self-employed',\n",
       "     1,\n",
       "     'none',\n",
       "     'yes',\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     [21,\n",
       "      [2, 3, 9, 10, 13, 14, 15, 16, 17, 18, 19, 20],\n",
       "      [1.0, 2.0, 2.0, 2.0, 24.0, 4567.0, 4.0, 4.0, 24.0, 36.0, 2.0, 1.0]],\n",
       "     [0.40678948974997703, -0.40678948974997703],\n",
       "     [0.6928716383154149, 0.3071283616845851],\n",
       "     0.0,\n",
       "     'No Risk'],\n",
       "    ['0_to_200',\n",
       "     26,\n",
       "     'all_credits_paid_back',\n",
       "     'car_new',\n",
       "     863,\n",
       "     'less_100',\n",
       "     'less_1',\n",
       "     2,\n",
       "     'female',\n",
       "     'co-applicant',\n",
       "     2,\n",
       "     'real_estate',\n",
       "     38,\n",
       "     'none',\n",
       "     'own',\n",
       "     1,\n",
       "     'skilled',\n",
       "     1,\n",
       "     'none',\n",
       "     'yes',\n",
       "     2.0,\n",
       "     3.0,\n",
       "     3.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     1.0,\n",
       "     2.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     [2.0,\n",
       "      3.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      3.0,\n",
       "      1.0,\n",
       "      1.0,\n",
       "      2.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      26.0,\n",
       "      863.0,\n",
       "      2.0,\n",
       "      2.0,\n",
       "      26.0,\n",
       "      38.0,\n",
       "      1.0,\n",
       "      1.0],\n",
       "     [1.08446701248213, -1.08446701248213],\n",
       "     [0.8974248814242887, 0.10257511857571133],\n",
       "     0.0,\n",
       "     'No Risk'],\n",
       "    ['0_to_200',\n",
       "     14,\n",
       "     'no_credits',\n",
       "     'car_new',\n",
       "     2368,\n",
       "     'less_100',\n",
       "     '1_to_4',\n",
       "     3,\n",
       "     'female',\n",
       "     'none',\n",
       "     3,\n",
       "     'real_estate',\n",
       "     29,\n",
       "     'none',\n",
       "     'own',\n",
       "     1,\n",
       "     'skilled',\n",
       "     1,\n",
       "     'none',\n",
       "     'yes',\n",
       "     2.0,\n",
       "     4.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     [21,\n",
       "      [0, 1, 5, 7, 13, 14, 15, 16, 17, 18, 19, 20],\n",
       "      [2.0, 4.0, 1.0, 2.0, 14.0, 2368.0, 3.0, 3.0, 14.0, 29.0, 1.0, 1.0]],\n",
       "     [1.4143966876630347, -1.4143966876630347],\n",
       "     [0.9442120764123368, 0.05578792358766316],\n",
       "     0.0,\n",
       "     'No Risk'],\n",
       "    ['0_to_200',\n",
       "     4,\n",
       "     'no_credits',\n",
       "     'car_new',\n",
       "     250,\n",
       "     'less_100',\n",
       "     'unemployed',\n",
       "     2,\n",
       "     'female',\n",
       "     'none',\n",
       "     3,\n",
       "     'real_estate',\n",
       "     23,\n",
       "     'none',\n",
       "     'rent',\n",
       "     1,\n",
       "     'management_self-employed',\n",
       "     1,\n",
       "     'none',\n",
       "     'yes',\n",
       "     2.0,\n",
       "     4.0,\n",
       "     4.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     [2.0,\n",
       "      4.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      4.0,\n",
       "      1.0,\n",
       "      0.0,\n",
       "      2.0,\n",
       "      0.0,\n",
       "      1.0,\n",
       "      2.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      4.0,\n",
       "      250.0,\n",
       "      2.0,\n",
       "      3.0,\n",
       "      4.0,\n",
       "      23.0,\n",
       "      1.0,\n",
       "      1.0],\n",
       "     [1.5173690203089494, -1.5173690203089494],\n",
       "     [0.9541190318258127, 0.0458809681741873],\n",
       "     0.0,\n",
       "     'No Risk'],\n",
       "    ['no_checking',\n",
       "     17,\n",
       "     'credits_paid_to_date',\n",
       "     'car_new',\n",
       "     832,\n",
       "     '100_to_500',\n",
       "     '1_to_4',\n",
       "     2,\n",
       "     'male',\n",
       "     'none',\n",
       "     2,\n",
       "     'real_estate',\n",
       "     42,\n",
       "     'none',\n",
       "     'own',\n",
       "     1,\n",
       "     'skilled',\n",
       "     1,\n",
       "     'none',\n",
       "     'yes',\n",
       "     0.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     [21,\n",
       "      [1, 3, 7, 13, 14, 15, 16, 17, 18, 19, 20],\n",
       "      [1.0, 1.0, 2.0, 17.0, 832.0, 2.0, 2.0, 17.0, 42.0, 1.0, 1.0]],\n",
       "     [1.1122314956561121, -1.1122314956561121],\n",
       "     [0.9024248873010724, 0.0975751126989276],\n",
       "     0.0,\n",
       "     'No Risk'],\n",
       "    ['no_checking',\n",
       "     33,\n",
       "     'outstanding_credit',\n",
       "     'appliances',\n",
       "     5696,\n",
       "     'unknown',\n",
       "     'greater_7',\n",
       "     4,\n",
       "     'male',\n",
       "     'co-applicant',\n",
       "     4,\n",
       "     'unknown',\n",
       "     54,\n",
       "     'none',\n",
       "     'free',\n",
       "     2,\n",
       "     'skilled',\n",
       "     1,\n",
       "     'yes',\n",
       "     'yes',\n",
       "     0.0,\n",
       "     2.0,\n",
       "     2.0,\n",
       "     4.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     4.0,\n",
       "     1.0,\n",
       "     3.0,\n",
       "     0.0,\n",
       "     1.0,\n",
       "     [0.0,\n",
       "      2.0,\n",
       "      4.0,\n",
       "      4.0,\n",
       "      2.0,\n",
       "      0.0,\n",
       "      1.0,\n",
       "      3.0,\n",
       "      0.0,\n",
       "      2.0,\n",
       "      0.0,\n",
       "      1.0,\n",
       "      0.0,\n",
       "      33.0,\n",
       "      5696.0,\n",
       "      4.0,\n",
       "      4.0,\n",
       "      33.0,\n",
       "      54.0,\n",
       "      2.0,\n",
       "      1.0],\n",
       "     [-1.3184238002172723, 1.3184238002172723],\n",
       "     [0.06680429265446867, 0.9331957073455314],\n",
       "     1.0,\n",
       "     'Risk'],\n",
       "    ['0_to_200',\n",
       "     13,\n",
       "     'prior_payments_delayed',\n",
       "     'retraining',\n",
       "     1375,\n",
       "     '100_to_500',\n",
       "     '4_to_7',\n",
       "     3,\n",
       "     'male',\n",
       "     'none',\n",
       "     3,\n",
       "     'real_estate',\n",
       "     37,\n",
       "     'none',\n",
       "     'own',\n",
       "     2,\n",
       "     'management_self-employed',\n",
       "     1,\n",
       "     'none',\n",
       "     'yes',\n",
       "     2.0,\n",
       "     0.0,\n",
       "     1.0,\n",
       "     1.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     8.0,\n",
       "     0.0,\n",
       "     2.0,\n",
       "     0.0,\n",
       "     0.0,\n",
       "     [2.0,\n",
       "      0.0,\n",
       "      8.0,\n",
       "      1.0,\n",
       "      1.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      2.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      2.0,\n",
       "      0.0,\n",
       "      0.0,\n",
       "      13.0,\n",
       "      1375.0,\n",
       "      3.0,\n",
       "      3.0,\n",
       "      13.0,\n",
       "      37.0,\n",
       "      2.0,\n",
       "      1.0],\n",
       "     [0.996270448775496, -0.996270448775496],\n",
       "     [0.8800116928980676, 0.11998830710193242],\n",
       "     0.0,\n",
       "     'No Risk']]}]}"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import json\n",
    "import sys\n",
    "import numpy\n",
    "import sklearn\n",
    "import sklearn.ensemble\n",
    "numpy.set_printoptions(threshold=sys.maxsize)\n",
    "#from sklearn.ensemble.gradient_boosting import GradientBoostingClassifier\n",
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "from sklearn.utils.multiclass import type_of_target\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.impute import SimpleImputer\n",
    "from sklearn.preprocessing import StandardScaler, OrdinalEncoder\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.model_selection import cross_validate\n",
    "from sklearn.metrics import get_scorer\n",
    "from sklearn.model_selection import cross_validate\n",
    "from sklearn.metrics import classification_report\n",
    "\n",
    "data_df=pd.read_csv (\"german_credit_data_biased_training.csv\")\n",
    "\n",
    "data_df.head()\n",
    "\n",
    "target_label_name = \"Risk\"\n",
    "feature_cols= data_df.drop(columns=[target_label_name])\n",
    "label= data_df[target_label_name]\n",
    "\n",
    "# Set model evaluation properties\n",
    "optimization_metric = 'roc_auc'\n",
    "random_state = 33\n",
    "cv_num_folds = 3\n",
    "holdout_fraction = 0.1\n",
    "\n",
    "if type_of_target(label.values) in ['multiclass', 'binary']:\n",
    "    X_train, X_holdout, y_train, y_holdout = train_test_split(feature_cols, label, test_size=holdout_fraction, random_state=random_state, stratify=label.values)\n",
    "else:\n",
    "    X_train, X_holdout, y_train, y_holdout = train_test_split(feature_cols, label, test_size=holdout_fraction, random_state=random_state)\n",
    "\n",
    "# Data preprocessing transformer generation\n",
    "\n",
    "numeric_transformer = Pipeline(steps=[\n",
    "    ('imputer', SimpleImputer(strategy='median')),\n",
    "    ('scaler', StandardScaler())])\n",
    "categorical_transformer = Pipeline(steps=[\n",
    "    ('imputer', SimpleImputer(strategy='most_frequent')),\n",
    "    ('OrdinalEncoder', OrdinalEncoder(categories='auto',dtype=numpy.float64 ))])\n",
    "\n",
    "numeric_features = feature_cols.select_dtypes(include=['int64', 'float64']).columns\n",
    "categorical_features = feature_cols.select_dtypes(include=['object']).columns\n",
    "\n",
    "preprocessor = ColumnTransformer(\n",
    "    transformers=[\n",
    "        ('num', numeric_transformer, numeric_features),\n",
    "        ('cat', categorical_transformer, categorical_features)])\n",
    "\n",
    "# Initiate model and create pipeline\n",
    "model=GradientBoostingClassifier()\n",
    "gbt_pipeline = Pipeline(steps=[('preprocessor', preprocessor), ('classifier', model)])\n",
    "model_gbt=gbt_pipeline.fit(X_train, y_train)\n",
    "\n",
    "y_pred = model_gbt.predict(X_holdout)\n",
    "\n",
    "\n",
    "# Evaluate model performance on test data and Cross validation\n",
    "scorer = get_scorer(optimization_metric)\n",
    "scorer(model_gbt,X_holdout, y_holdout)\n",
    "\n",
    "# Cross validation -3 folds\n",
    "cv_results = cross_validate(model_gbt,X_train,y_train, scoring={optimization_metric:scorer})\n",
    "numpy.mean(cv_results['test_' + optimization_metric])\n",
    "\n",
    "print(classification_report(y_pred, y_holdout))\n",
    "\n",
    "\n",
    "# Remove existing model and deployment\n",
    "MODEL_NAME=PRE_PROD_CHALLENGER_MODEL_NAME\n",
    "DEPLOYMENT_NAME=PRE_PROD_CHALLENGER_DEPLOYMENT_NAME\n",
    "\n",
    "deployment_details = wml_client.deployments.get_details()\n",
    "for deployment in deployment_details['resources']:\n",
    "    deployment_id = wml_client.deployments.get_id(deployment)\n",
    "    model_id = deployment['entity']['asset']['id']\n",
    "    if deployment['entity']['name'] == DEPLOYMENT_NAME:\n",
    "        print('Deleting deployment id', deployment_id)\n",
    "        wml_client.deployments.delete(deployment_id)\n",
    "        print('Deleting model id', model_id)\n",
    "        wml_client.repository.delete(model_id)\n",
    "wml_client.repository.list_models()\n",
    "\n",
    "# Store Model\n",
    "software_spec_uid = wml_client.software_specifications.get_id_by_name(\"runtime-23.1-py3.10\")\n",
    "model_props_gbt = {\n",
    "    wml_client.repository.ModelMetaNames.NAME: MODEL_NAME,\n",
    "    wml_client.repository.ModelMetaNames.DESCRIPTION: MODEL_NAME,\n",
    "    wml_client.repository.ModelMetaNames.SOFTWARE_SPEC_UID: software_spec_uid,\n",
    "    wml_client.repository.ModelMetaNames.TYPE: \"scikit-learn_1.1\"\n",
    "}\n",
    "\n",
    "published_model_details = wml_client.repository.store_model(model=model_gbt, meta_props=model_props_gbt, training_data=feature_cols,training_target=label)\n",
    "print(published_model_details)\n",
    "\n",
    "# List models in the repository\n",
    "wml_client.repository.list_models()\n",
    "\n",
    "# Get the model UID\n",
    "challenger_model_uid = wml_client.repository.get_model_id(published_model_details)\n",
    "print('challenger_model_uid ', challenger_model_uid)\n",
    "\n",
    "\n",
    "# Deploy model\n",
    "wml_deployments = wml_client.deployments.get_details()\n",
    "challenger_deployment_uid = None\n",
    "for deployment in wml_deployments['resources']:\n",
    "    if DEPLOYMENT_NAME == deployment['entity']['name']:\n",
    "        challenger_deployment_uid = wml_client.deployments.get_id(deployment)\n",
    "        break\n",
    "\n",
    "if challenger_deployment_uid is None:\n",
    "    print(\"Deploying model...\")\n",
    "    meta_props = {\n",
    "        wml_client.deployments.ConfigurationMetaNames.NAME: DEPLOYMENT_NAME,\n",
    "        wml_client.deployments.ConfigurationMetaNames.DESCRIPTION: DEPLOYMENT_NAME,\n",
    "        wml_client.deployments.ConfigurationMetaNames.ONLINE: {}\n",
    "    }\n",
    "    deployment = wml_client.deployments.create(artifact_uid=challenger_model_uid, meta_props=meta_props)\n",
    "    challenger_deployment_uid = wml_client.deployments.get_id(deployment)\n",
    "\n",
    "print(\"Model id: {}\".format(challenger_model_uid))\n",
    "print(\"Deployment id: {}\".format(challenger_deployment_uid))\n",
    "\n",
    "challenger_deployment_uid=wml_client.deployments.get_id(deployment)\n",
    "challenger_deployment_uid\n",
    "\n",
    "\n",
    "# Sample scoring\n",
    "fields = [\"CheckingStatus\",\"LoanDuration\",\"CreditHistory\",\"LoanPurpose\",\"LoanAmount\",\"ExistingSavings\",\"EmploymentDuration\",\"InstallmentPercent\",\"Sex\",\"OthersOnLoan\",\"CurrentResidenceDuration\",\"OwnsProperty\",\"Age\",\"InstallmentPlans\",\"Housing\",\"ExistingCreditsCount\",\"Job\",\"Dependents\",\"Telephone\",\"ForeignWorker\"]\n",
    "values = [\n",
    "  [\"no_checking\",13,\"credits_paid_to_date\",\"car_new\",1343,\"100_to_500\",\"1_to_4\",2,\"female\",\"none\",3,\"savings_insurance\",46,\"none\",\"own\",2,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"no_checking\",24,\"prior_payments_delayed\",\"furniture\",4567,\"500_to_1000\",\"1_to_4\",4,\"male\",\"none\",4,\"savings_insurance\",36,\"none\",\"free\",2,\"management_self-employed\",1,\"none\",\"yes\"],\n",
    "  [\"0_to_200\",26,\"all_credits_paid_back\",\"car_new\",863,\"less_100\",\"less_1\",2,\"female\",\"co-applicant\",2,\"real_estate\",38,\"none\",\"own\",1,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"0_to_200\",14,\"no_credits\",\"car_new\",2368,\"less_100\",\"1_to_4\",3,\"female\",\"none\",3,\"real_estate\",29,\"none\",\"own\",1,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"0_to_200\",4,\"no_credits\",\"car_new\",250,\"less_100\",\"unemployed\",2,\"female\",\"none\",3,\"real_estate\",23,\"none\",\"rent\",1,\"management_self-employed\",1,\"none\",\"yes\"],\n",
    "  [\"no_checking\",17,\"credits_paid_to_date\",\"car_new\",832,\"100_to_500\",\"1_to_4\",2,\"male\",\"none\",2,\"real_estate\",42,\"none\",\"own\",1,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"no_checking\",33,\"outstanding_credit\",\"appliances\",5696,\"unknown\",\"greater_7\",4,\"male\",\"co-applicant\",4,\"unknown\",54,\"none\",\"free\",2,\"skilled\",1,\"yes\",\"yes\"],\n",
    "  [\"0_to_200\",13,\"prior_payments_delayed\",\"retraining\",1375,\"100_to_500\",\"4_to_7\",3,\"male\",\"none\",3,\"real_estate\",37,\"none\",\"own\",2,\"management_self-employed\",1,\"none\",\"yes\"]\n",
    "]\n",
    "\n",
    "payload_scoring = {\"fields\": fields,\"values\": values}\n",
    "payload = {\n",
    "    wml_client.deployments.ScoringMetaNames.INPUT_DATA: [payload_scoring]\n",
    "}\n",
    "scoring_response = wml_client.deployments.score(challenger_deployment_uid, payload)\n",
    "\n",
    "scoring_response\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6d66e647-0b3e-483e-8223-55d79934f20c"
   },
   "source": [
    "# Configure OpenScale \n",
    "The notebook will now import the necessary libraries and set up a Python OpenScale client."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "94363246-f6e0-4a95-af54-9d8513854dbe"
   },
   "outputs": [],
   "source": [
    "from ibm_cloud_sdk_core.authenticators import CloudPakForDataAuthenticator\n",
    "from ibm_watson_openscale import APIClient\n",
    "from ibm_watson_openscale.utils import *\n",
    "from ibm_watson_openscale.supporting_classes import *\n",
    "from ibm_watson_openscale.supporting_classes.enums import *\n",
    "from ibm_watson_openscale.supporting_classes.payload_record import PayloadRecord\n",
    "import uuid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "id": "9c822f42-7763-4fe5-bbf7-ed6fb6ae4722"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'3.0.39'"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "authenticator = CloudPakForDataAuthenticator(\n",
    "        url=WOS_CREDENTIALS['url'],\n",
    "        username=WOS_CREDENTIALS['username'],\n",
    "        password=WOS_CREDENTIALS['password'],\n",
    "        disable_ssl_verification=True\n",
    "    )\n",
    "\n",
    "wos_client = APIClient(service_url=WOS_CREDENTIALS['url'],authenticator=authenticator)\n",
    "wos_client.version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "444021dc-f467-4b55-9c19-2a42e2725985"
   },
   "source": [
    "## Create schema and datamart\n",
    "\n",
    "### Set up datamart\n",
    "Watson OpenScale uses a database to store payload logs and calculated metrics. If an OpenScale datamart exists in Db2, the existing datamart will be used and no data will be overwritten."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "78d73bc1b8f84765bf0eb36e4c637d17"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>Data Marts</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>name</th><th style='border: 1px solid #dddddd'>description</th><th style='border: 1px solid #dddddd'>internal_database</th><th style='border: 1px solid #dddddd'>status</th><th style='border: 1px solid #dddddd'>created_at</th><th style='border: 1px solid #dddddd'>id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>AIOSFASTPATHICP-00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>Data Mart created by OpenScale ExpressPath</td><td style='border: 1px solid #dddddd'>False</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-06-04 05:19:03.698000+00:00</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "wos_client.data_marts.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "id": "af7f338d-cd56-45f9-8817-40bb176d523d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using existing datamart 00000000-0000-0000-0000-000000000000\n"
     ]
    }
   ],
   "source": [
    "data_marts = wos_client.data_marts.list().result.data_marts\n",
    "if len(data_marts) == 0:\n",
    "    if DATABASE_CREDENTIALS is not None:\n",
    "        if SCHEMA_NAME is None: \n",
    "            print(\"Please specify the SCHEMA_NAME and rerun the cell\")\n",
    "\n",
    "        print(\"Setting up external datamart\")\n",
    "        added_data_mart_result = wos_client.data_marts.add(\n",
    "                background_mode=False,\n",
    "                name=\"WOS Data Mart\",\n",
    "                description=\"Data Mart created by WOS tutorial notebook\",\n",
    "                database_configuration=DatabaseConfigurationRequest(\n",
    "                  database_type=DatabaseType.DB2,\n",
    "                    credentials=PrimaryStorageCredentialsLong(\n",
    "                        hostname=DATABASE_CREDENTIALS[\"hostname\"],\n",
    "                        username=DATABASE_CREDENTIALS[\"username\"],\n",
    "                        password=DATABASE_CREDENTIALS[\"password\"],\n",
    "                        db=DATABASE_CREDENTIALS[\"db\"],\n",
    "                        port=DATABASE_CREDENTIALS[\"port\"]\n",
    "                    ),\n",
    "                    location=LocationSchemaName(\n",
    "                        schema_name= SCHEMA_NAME\n",
    "                    )\n",
    "                )\n",
    "             ).result\n",
    "    else:\n",
    "        print(\"Setting up internal datamart\")\n",
    "        added_data_mart_result = wos_client.data_marts.add(\n",
    "                background_mode=False,\n",
    "                name=\"WOS Data Mart\",\n",
    "                description=\"Data Mart created by WOS tutorial notebook\", \n",
    "                internal_database = True).result\n",
    "        \n",
    "    data_mart_id = added_data_mart_result.metadata.id\n",
    "    \n",
    "else:\n",
    "    data_mart_id=data_marts[0].metadata.id\n",
    "    print(\"Using existing datamart {}\".format(data_mart_id))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "4a6c18b3207f4ff98bf21621df5a2ec9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'00000000-0000-0000-0000-000000000000'"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_mart_id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "59cf3a5b-78be-4d10-8ae8-069a98b48fa2"
   },
   "source": [
    "## Generate an ICP token\n",
    "\n",
    "The following is a function that will generate an ICP access token used to interact with the Watson OpenScale APIs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "id": "9e6c7b03-7506-4ad4-b656-7f29e2349ea8"
   },
   "outputs": [],
   "source": [
    "import requests\n",
    "import urllib3\n",
    "from http import HTTPStatus\n",
    "\n",
    "def get_iamtoken(url, username, password):\n",
    "    fqdn = urllib3.util.parse_url(url).netloc\n",
    "    domain = '.'.join(fqdn.split('.')[1:])\n",
    "    token_url = 'https://cp-console.{}/idprovider/v1/auth/identitytoken'.format(domain)\n",
    "    data = {\n",
    "        'grant_type': 'password',\n",
    "        'username': username,\n",
    "        'password': password,\n",
    "        'scope': 'openid'\n",
    "    }\n",
    "    return requests.post(token_url, data, verify=False)\n",
    "\n",
    "def get_accesstoken(url, username, iamtoken):\n",
    "    url = '{}/v1/preauth/validateAuth'.format(url)\n",
    "    headers = {\n",
    "        'Content-type': 'application/json',\n",
    "        'username': username,\n",
    "        'iam-token': iamtoken\n",
    "    }\n",
    "    return requests.get(url, headers=headers, verify=False)\n",
    "\n",
    "def generate_access_token():\n",
    "    url=WOS_CREDENTIALS['url']\n",
    "    username=WOS_CREDENTIALS['username']\n",
    "    password=WOS_CREDENTIALS['password'] \n",
    "    response = get_iamtoken(url,username,password)\n",
    "    #service is not available when iamintegration=false so fall back to old way of generating code\n",
    "    if response.status_code==HTTPStatus.SERVICE_UNAVAILABLE:\n",
    "        url = '{}/v1/preauth/validateAuth'.format(url)\n",
    "        headers = {'Content-type': 'application/json'}\n",
    "        data = {\n",
    "            'grant_type': 'password',\n",
    "            'username': username,\n",
    "            'password': password\n",
    "        }\n",
    "        return requests.get(url, headers=headers, auth= (username,password),verify=False).json()['accessToken']\n",
    "        \n",
    "    else:\n",
    "        return get_accesstoken(url,username, response.json()['access_token']).json()['accessToken']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4b0c9239-899e-4748-80e2-b901a4d26741"
   },
   "source": [
    "## Bind WML machine learning instance as Pre-Prod\n",
    "\n",
    "Watson OpenScale needs to be bound to the Watson Machine Learning instance to capture payload data into and out of the model. If a binding with name \"WML Pre-Prod\" already exists, this code will delete that binding a create a new one.\n",
    "\n",
    "**Note**: Binding with name `WML Pre-Prod` is assumed to be only created by this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "d97136a95b184de8b794d14318367aad"
   },
   "outputs": [],
   "source": [
    "SERVICE_PROVIDER_NAME = \"WML Pre-Prod\"\n",
    "SERVICE_PROVIDER_DESCRIPTION = \"Added by tutorial WOS notebook.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "id": "33c581b9-fc49-4885-90d9-d25e2f78762a"
   },
   "outputs": [],
   "source": [
    "service_providers = wos_client.service_providers.list().result.service_providers\n",
    "for service_provider in service_providers:\n",
    "    service_instance_name = service_provider.entity.name\n",
    "    if service_instance_name == SERVICE_PROVIDER_NAME:\n",
    "        service_provider_id = service_provider.metadata.id\n",
    "        wos_client.service_providers.delete(service_provider_id)\n",
    "        print(\"Deleted existing service_provider for WML instance: {}\".format(service_provider_id))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "id": "60add123-1273-4190-97bd-1c3ff5769fba"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "=================================================================================\n",
      "\n",
      " Waiting for end of adding service provider 33397d85-dcae-4703-907d-4668932f7df0 \n",
      "\n",
      "=================================================================================\n",
      "\n",
      "\n",
      "\n",
      "active\n",
      "\n",
      "-----------------------------------------------\n",
      " Successfully finished adding service provider \n",
      "-----------------------------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "added_service_provider_result = wos_client.service_providers.add(\n",
    "        name=SERVICE_PROVIDER_NAME,\n",
    "        description=SERVICE_PROVIDER_DESCRIPTION,\n",
    "        service_type=ServiceTypes.WATSON_MACHINE_LEARNING,\n",
    "        deployment_space_id = space_id,\n",
    "        operational_space_id = \"pre_production\",\n",
    "        credentials=WMLCredentialsCP4D(),\n",
    "        background_mode=False\n",
    "    ).result\n",
    "service_provider_id = added_service_provider_result.metadata.id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "id": "3bb0b46e-a5d3-4fff-a746-7d3aae5a7d1f"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>Service Providers</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>instance_id</th><th style='border: 1px solid #dddddd'>status</th><th style='border: 1px solid #dddddd'>name</th><th style='border: 1px solid #dddddd'>service_type</th><th style='border: 1px solid #dddddd'>created_at</th><th style='border: 1px solid #dddddd'>id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>99999999-9999-9999-9999-999999999999</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>WML Pre-Prod-2</td><td style='border: 1px solid #dddddd'>watson_machine_learning</td><td style='border: 1px solid #dddddd'>2024-08-08 00:28:02.977000+00:00</td><td style='border: 1px solid #dddddd'>33397d85-dcae-4703-907d-4668932f7df0</td></tr><tr><td style='border: 1px solid #dddddd'>99999999-9999-9999-9999-999999999999</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>Watson Machine Learning V2-2</td><td style='border: 1px solid #dddddd'>watson_machine_learning</td><td style='border: 1px solid #dddddd'>2024-08-07 09:57:15.265000+00:00</td><td style='border: 1px solid #dddddd'>cf31400f-925c-4180-b4b2-77e53440d9a7</td></tr><tr><td style='border: 1px solid #dddddd'>99999999-9999-9999-9999-999999999999</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>WML - for training data -2</td><td style='border: 1px solid #dddddd'>watson_machine_learning</td><td style='border: 1px solid #dddddd'>2024-08-07 08:42:56.271000+00:00</td><td style='border: 1px solid #dddddd'>1b7f6643-f6ac-4754-9f39-e8dbe5b09232</td></tr><tr><td style='border: 1px solid #dddddd'>99999999-9999-9999-9999-999999999999</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>Watson Machine Learning V2_test-2</td><td style='border: 1px solid #dddddd'>watson_machine_learning</td><td style='border: 1px solid #dddddd'>2024-08-07 06:51:26.963000+00:00</td><td style='border: 1px solid #dddddd'>b9fd3cb5-c7a4-4b5c-a90c-684676276352</td></tr><tr><td style='border: 1px solid #dddddd'>99999999-9999-9999-9999-999999999999</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>WML Prod</td><td style='border: 1px solid #dddddd'>watson_machine_learning</td><td style='border: 1px solid #dddddd'>2024-08-07 06:34:58.909000+00:00</td><td style='border: 1px solid #dddddd'>f4931b54-2103-40d9-b564-9ba3b1982fdb</td></tr><tr><td style='border: 1px solid #dddddd'>99999999-9999-9999-9999-999999999999</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>WML Pre-Prod2</td><td style='border: 1px solid #dddddd'>watson_machine_learning</td><td style='border: 1px solid #dddddd'>2024-08-07 06:03:11.179000+00:00</td><td style='border: 1px solid #dddddd'>81655d73-6ea3-44ae-9876-86df1fa6390d</td></tr><tr><td style='border: 1px solid #dddddd'>99999999-9999-9999-9999-999999999999</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>WML AI function -  WOS notebook</td><td style='border: 1px solid #dddddd'>watson_machine_learning</td><td style='border: 1px solid #dddddd'>2024-08-06 13:32:38.884000+00:00</td><td style='border: 1px solid #dddddd'>e413a76d-802c-44f3-8c55-300c5aa2659b</td></tr><tr><td style='border: 1px solid #dddddd'>99999999-9999-9999-9999-999999999999</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>Watson Machine Learning - Indirect Bias Demo-2</td><td style='border: 1px solid #dddddd'>watson_machine_learning</td><td style='border: 1px solid #dddddd'>2024-08-06 13:25:37.046000+00:00</td><td style='border: 1px solid #dddddd'>c3601bc6-22c9-45a1-b016-c29cf0bf64b4</td></tr><tr><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>IAE7</td><td style='border: 1px solid #dddddd'>custom_machine_learning</td><td style='border: 1px solid #dddddd'>2024-07-02 09:08:06.273000+00:00</td><td style='border: 1px solid #dddddd'>184e73a2-7fd8-4f3f-b994-bb648f6eb8ec</td></tr><tr><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>IAE6</td><td style='border: 1px solid #dddddd'>custom_machine_learning</td><td style='border: 1px solid #dddddd'>2024-07-02 09:04:22.643000+00:00</td><td style='border: 1px solid #dddddd'>644befcd-6d36-4f4d-a30a-cd51a28b63fe</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: First 10 records were displayed.\n"
     ]
    }
   ],
   "source": [
    "wos_client.service_providers.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0272beea-9837-4bc9-9798-18324240e250"
   },
   "source": [
    "## Create an integration to IBM OpenPages\n",
    "\n",
    "To push metrics from Watson OpenScale to IBM OpenPages, a connection must be established between the two services."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "id": "077c4762-6e89-444b-bc2e-9d6b90b7c296"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'entity': {'credentials': {'secret_id': '2e612f85-588a-4e14-ac61-1f0be9db9945'}, 'description': 'Integration with OpenPages', 'name': 'OpenPages Connection', 'status': {'state': 'active'}, 'type': 'open_pages'}, 'metadata': {'created_at': '2024-08-08T00:28:12.866Z', 'created_by': 'cpadmin', 'crn': 'crn:v1:bluemix:public:aiopenscale:us-south:a/na:00000000-0000-0000-0000-000000000000:integrated_system:cf7e2e39-78c6-4711-8d76-89fa08acc9a0', 'id': 'cf7e2e39-78c6-4711-8d76-89fa08acc9a0', 'url': '/v2/integrated_systems/cf7e2e39-78c6-4711-8d76-89fa08acc9a0'}}\n",
      "cf7e2e39-78c6-4711-8d76-89fa08acc9a0\n"
     ]
    }
   ],
   "source": [
    "headers = {}\n",
    "headers[\"Content-Type\"] = \"application/json\"\n",
    "headers[\"Authorization\"] = \"Bearer {}\".format(generate_access_token())\n",
    "\n",
    "INTEGRATED_SYSTEMS_URL = WOS_CREDENTIALS[\"url\"] + \"/openscale/{0}/v2/integrated_systems\".format(WOS_GUID)\n",
    "\n",
    "payload = {\n",
    "    \"name\": \"OpenPages Connection\",\n",
    "    \"type\": \"open_pages\",\n",
    "    \"description\": \"Integration with OpenPages\",\n",
    "    \"credentials\": OPENPAGES_CREDENTIALS\n",
    "}\n",
    "\n",
    "response = requests.post(INTEGRATED_SYSTEMS_URL, json=payload, headers=headers, verify=False)\n",
    "json_data = response.json()\n",
    "print(json_data)\n",
    "\n",
    "if \"metadata\" in json_data and \"id\" in json_data[\"metadata\"]:\n",
    "    integrated_system_id = json_data[\"metadata\"][\"id\"]\n",
    "    print(integrated_system_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7ebeaba2-1ef8-441b-8a70-cb8bf8c73865"
   },
   "source": [
    "## Subscriptions\n",
    "### Remove existing PreProd and Challenger credit risk subscriptions\n",
    "This code removes previous subscriptions with name `German Credit Risk Model - PreProd` and `German Credit Risk Model - Challenger` to refresh the monitors with the new model and new data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "id": "987e9389-4cb2-4817-97e6-d50fe7a241ed"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Deleted existing subscription for German Credit Risk Model - Challenger\n",
      "Deleted existing subscription for German Credit Risk Model - PreProd\n"
     ]
    }
   ],
   "source": [
    "subscriptions = wos_client.subscriptions.list().result.subscriptions\n",
    "for subscription in subscriptions:\n",
    "    sub_name = subscription.entity.asset.name\n",
    "    if sub_name == PRE_PROD_MODEL_NAME or sub_name == PRE_PROD_CHALLENGER_MODEL_NAME:\n",
    "        wos_client.subscriptions.delete(subscription.metadata.id)\n",
    "        print('Deleted existing subscription for', sub_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "id": "32b7805b67ea46af83e2e0cb55b3f677"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>Subscriptions</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>asset_id</th><th style='border: 1px solid #dddddd'>asset_type</th><th style='border: 1px solid #dddddd'>asset_name</th><th style='border: 1px solid #dddddd'>data_mart_id</th><th style='border: 1px solid #dddddd'>deployment_id</th><th style='border: 1px solid #dddddd'>deployment_name</th><th style='border: 1px solid #dddddd'>service_provider_id</th><th style='border: 1px solid #dddddd'>status</th><th style='border: 1px solid #dddddd'>created_at</th><th style='border: 1px solid #dddddd'>id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>f1b4fe35-1a75-4dca-a9b3-bc5dd7ed43db</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>German Credit Risk Model - PreProd</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>42b13344-baa1-45a0-89ca-ef662387f691</td><td style='border: 1px solid #dddddd'>German Credit Risk Model - PreProd</td><td style='border: 1px solid #dddddd'>81655d73-6ea3-44ae-9876-86df1fa6390d</td><td style='border: 1px solid #dddddd'>pending_delete</td><td style='border: 1px solid #dddddd'>2024-08-07 06:07:24.153000+00:00</td><td style='border: 1px solid #dddddd'>61003e50-20d6-4ce8-a379-013a03c44229</td></tr><tr><td style='border: 1px solid #dddddd'>67e11064-81b2-4ea8-addf-b39c218299b4</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>German Credit Risk Model - Challenger</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>978e26e1-509c-4e03-b20a-baacfbbe222b</td><td style='border: 1px solid #dddddd'>German Credit Risk Model - Challenger</td><td style='border: 1px solid #dddddd'>81655d73-6ea3-44ae-9876-86df1fa6390d</td><td style='border: 1px solid #dddddd'>pending_delete</td><td style='border: 1px solid #dddddd'>2024-08-07 06:07:38.473000+00:00</td><td style='border: 1px solid #dddddd'>c0174a6c-d8e6-458b-8549-cd03d7df486e</td></tr><tr><td style='border: 1px solid #dddddd'>1b9f22c5-ac03-4017-b40b-27969e429379</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>Scikit German Risk Model WML V4 - zLinux</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>edc1e06c-7bf9-4522-8893-05a919a35ca0</td><td style='border: 1px solid #dddddd'>Scikit German Risk Deployment WML V4 - zLinux</td><td style='border: 1px solid #dddddd'>cf31400f-925c-4180-b4b2-77e53440d9a7</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-08-07 09:57:28.433000+00:00</td><td style='border: 1px solid #dddddd'>ad19393e-2078-4316-ac19-7de0ddfd7937</td></tr><tr><td style='border: 1px solid #dddddd'>df904b8a-2884-4318-82df-35cb430f8587</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>Scikit German Risk Model - for training data - zLinux</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>17e4097d-c4b5-41fe-bba5-3e12af95041e</td><td style='border: 1px solid #dddddd'>Scikit German Risk Model - for training data - zLinux</td><td style='border: 1px solid #dddddd'>1b7f6643-f6ac-4754-9f39-e8dbe5b09232</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-08-07 08:51:02.441000+00:00</td><td style='border: 1px solid #dddddd'>bcbbcc1f-7e1c-4746-8055-128ce8d805ff</td></tr><tr><td style='border: 1px solid #dddddd'>20516963-37ff-4769-a41f-b509827f3076</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>Text Binary Classifier</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>18c8f8d4-cfbf-4fbb-a51a-5ac59d2e9e20</td><td style='border: 1px solid #dddddd'>Text Binary Classifier deployment</td><td style='border: 1px solid #dddddd'>b9fd3cb5-c7a4-4b5c-a90c-684676276352</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-08-07 06:51:41.900000+00:00</td><td style='border: 1px solid #dddddd'>d673e127-dd69-4b05-a204-5b90332d7e72</td></tr><tr><td style='border: 1px solid #dddddd'>a4a1a71d-2a44-43f8-b92a-ba3c80dc36fd</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>Credit Risk python Fn Model</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>90b19695-de03-41b9-b820-2d3e11c2f662</td><td style='border: 1px solid #dddddd'>dep_Credit Risk python Fn Model</td><td style='border: 1px solid #dddddd'>e413a76d-802c-44f3-8c55-300c5aa2659b</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-08-06 13:32:53.985000+00:00</td><td style='border: 1px solid #dddddd'>77c99979-8408-40b7-ab3a-e3c0a8a80aab</td></tr><tr><td style='border: 1px solid #dddddd'>eed8b225-19e5-4460-9b6f-7271dc1e3ff2</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>Adult Census Income Classifier Model</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>038b8c68-6615-4c0d-8d46-15783d6ab9c8</td><td style='border: 1px solid #dddddd'>Adult Census Income Classifier Deployment</td><td style='border: 1px solid #dddddd'>c3601bc6-22c9-45a1-b016-c29cf0bf64b4</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-08-06 13:26:09.549000+00:00</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td></tr><tr><td style='border: 1px solid #dddddd'>438ca544-9bd1-48c2-8e8d-3de4ef4ca79b</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>WML_IAE4</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>78a0af9e-1014-4fb1-b22a-5e11f4fd70e7</td><td style='border: 1px solid #dddddd'>WML_IAE4</td><td style='border: 1px solid #dddddd'>d90c6bf2-49c6-4179-9876-8b85b0247d95</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-02 07:03:31.504000+00:00</td><td style='border: 1px solid #dddddd'>e34b9b87-b6e1-4c53-b92e-cb80dea042be</td></tr><tr><td style='border: 1px solid #dddddd'>592b902d-3dc9-4e56-8bcb-86cbf1a6d8a9</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>gcr - P2 XGB Classifier - Model</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>2b976af0-e4ab-4859-af7d-2f2287d864ad</td><td style='border: 1px solid #dddddd'>gcr model</td><td style='border: 1px solid #dddddd'>4d2f2fb2-6b64-4d58-8f13-257166e468e9</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-17 07:11:44.727000+00:00</td><td style='border: 1px solid #dddddd'>e2df4ec7-6c75-416f-a444-8d21389f7513</td></tr><tr><td style='border: 1px solid #dddddd'>e3ac9fc3-bccf-4a4e-b37b-490bfb93dd81</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>GCR AutoAI - P2 XGB Classifier - Model</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>6399e6e8-df5a-4370-9af4-34b2f2e76bc6</td><td style='border: 1px solid #dddddd'>GCR Auto AI</td><td style='border: 1px solid #dddddd'>a7ca157a-de07-457a-8c4c-b1a2e998699c</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-03 10:37:20.487000+00:00</td><td style='border: 1px solid #dddddd'>ce36911c-75f6-4f99-b4d2-b71e5d55a802</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: First 10 records were displayed.\n"
     ]
    }
   ],
   "source": [
    "wos_client.subscriptions.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "id": "3b900ee8ed16456e82db4c5d45ef88fe"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: Software specification spark-mllib_3.3 is deprecated. Use spark-mllib_3.4 software specification instead when saving a spark model. For details, see https://www.ibm.com/support/producthub/icpdata/docs/content/SSQNUZ_latest/wsj/wmls/wmls-deploy-python-types.html.\n"
     ]
    }
   ],
   "source": [
    "# Create preprod subscription\n",
    "pre_prod_deployment = wml_client.deployments.get_details(deployment_uid=pre_prod_deployment_uid)\n",
    "pre_prod_asset_details=wos_client.service_providers.get_deployment_asset(data_mart_id=data_mart_id,service_provider_id=service_provider_id,deployment_id=pre_prod_deployment_uid,deployment_space_id=space_id)\n",
    "pre_prod_asset = Asset(\n",
    "    asset_id=pre_prod_asset_details[\"entity\"][\"asset\"][\"asset_id\"],\n",
    "    name=pre_prod_asset_details[\"entity\"][\"asset\"][\"name\"],\n",
    "    url=pre_prod_asset_details[\"entity\"][\"asset\"][\"url\"],\n",
    "    asset_type=AssetTypes.MODEL,\n",
    "    input_data_type=InputDataType.STRUCTURED,\n",
    "    problem_type=ProblemType.BINARY_CLASSIFICATION,\n",
    "    model_type='mllib_3.3'\n",
    ")\n",
    "pre_prod_asset_deployment = AssetDeploymentRequest(\n",
    "    deployment_id=pre_prod_deployment_uid,\n",
    "    name=PRE_PROD_DEPLOYMENT_NAME,\n",
    "    deployment_type=DeploymentTypes.ONLINE,\n",
    "    url=pre_prod_deployment[\"entity\"][\"status\"][\"online_url\"][\"url\"]\n",
    ")\n",
    "\n",
    "pre_prod_asset_properties_request = AssetPropertiesRequest(\n",
    "    label_column=\"Risk\",\n",
    "    probability_fields=[\"probability\"],\n",
    "    prediction_field=\"predictedLabel\",\n",
    "    feature_fields=[\"CheckingStatus\",\"LoanDuration\",\"CreditHistory\",\"LoanPurpose\",\"LoanAmount\",\"ExistingSavings\",\"EmploymentDuration\",\"InstallmentPercent\",\"Sex\",\"OthersOnLoan\",\"CurrentResidenceDuration\",\"OwnsProperty\",\"Age\",\"InstallmentPlans\",\"Housing\",\"ExistingCreditsCount\",\"Job\",\"Dependents\",\"Telephone\",\"ForeignWorker\"],\n",
    "    categorical_fields=[\"CheckingStatus\",\"CreditHistory\",\"LoanPurpose\",\"ExistingSavings\",\"EmploymentDuration\",\"Sex\",\"OthersOnLoan\",\"OwnsProperty\",\"InstallmentPlans\",\"Housing\",\"Job\",\"Telephone\",\"ForeignWorker\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "id": "d3902de0ac9c4707b975623eac635ac3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'metadata': {'guid': '42b79776-470c-4d05-9115-76c4f5fffeab',\n",
       "  'created_at': '2024-08-08T00:21:34.088Z',\n",
       "  'modified_at': '2024-08-08T00:21:34.088Z'},\n",
       " 'entity': {'name': 'German Credit Risk Model - PreProd',\n",
       "  'type': 'online',\n",
       "  'description': 'German Credit Risk Model - PreProd',\n",
       "  'scoring_endpoint': {'url': 'https://internal-nginx-svc:12443/ml/v4/deployments/42b79776-470c-4d05-9115-76c4f5fffeab/predictions'},\n",
       "  'asset': {'asset_id': 'fe7ed7dd-2856-42d6-83ab-51c13079f78d',\n",
       "   'url': 'https://internal-nginx-svc:12443/ml/v4/models/fe7ed7dd-2856-42d6-83ab-51c13079f78d?space_id=498630dc-3e0b-4470-b44d-583f2a393c0c&version=2020-06-12',\n",
       "   'name': 'German Credit Risk Model - PreProd',\n",
       "   'asset_type': 'model',\n",
       "   'created_at': '2024-08-08T00:21:19.987Z',\n",
       "   'modified_at': '2024-08-08T00:21:28.415Z'},\n",
       "  'asset_properties': {'model_type': 'mllib_3.3',\n",
       "   'runtime_environment': 'spark-3.3.0',\n",
       "   'label_column': 'Risk',\n",
       "   'input_data_schema': {'type': 'struct',\n",
       "    'id': '1',\n",
       "    'fields': [{'name': 'CheckingStatus',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'LoanDuration',\n",
       "      'type': 'integer',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'CreditHistory',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'LoanPurpose',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'LoanAmount',\n",
       "      'type': 'integer',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'ExistingSavings',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'EmploymentDuration',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'InstallmentPercent',\n",
       "      'type': 'integer',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'Sex', 'type': 'string', 'nullable': True, 'metadata': {}},\n",
       "     {'name': 'OthersOnLoan',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'CurrentResidenceDuration',\n",
       "      'type': 'integer',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'OwnsProperty',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'Age', 'type': 'integer', 'nullable': True, 'metadata': {}},\n",
       "     {'name': 'InstallmentPlans',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'Housing', 'type': 'string', 'nullable': True, 'metadata': {}},\n",
       "     {'name': 'ExistingCreditsCount',\n",
       "      'type': 'integer',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'Job', 'type': 'string', 'nullable': True, 'metadata': {}},\n",
       "     {'name': 'Dependents',\n",
       "      'type': 'integer',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'Telephone', 'type': 'string', 'nullable': True, 'metadata': {}},\n",
       "     {'name': 'ForeignWorker',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}}]},\n",
       "   'training_data_schema': {'type': 'struct',\n",
       "    'id': '1',\n",
       "    'fields': [{'name': 'CheckingStatus',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'LoanDuration',\n",
       "      'type': 'integer',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'CreditHistory',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'LoanPurpose',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'LoanAmount',\n",
       "      'type': 'integer',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'ExistingSavings',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'EmploymentDuration',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'InstallmentPercent',\n",
       "      'type': 'integer',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'Sex', 'type': 'string', 'nullable': True, 'metadata': {}},\n",
       "     {'name': 'OthersOnLoan',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'CurrentResidenceDuration',\n",
       "      'type': 'integer',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'OwnsProperty',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'Age', 'type': 'integer', 'nullable': True, 'metadata': {}},\n",
       "     {'name': 'InstallmentPlans',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'Housing', 'type': 'string', 'nullable': True, 'metadata': {}},\n",
       "     {'name': 'ExistingCreditsCount',\n",
       "      'type': 'integer',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'Job', 'type': 'string', 'nullable': True, 'metadata': {}},\n",
       "     {'name': 'Dependents',\n",
       "      'type': 'integer',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'Telephone', 'type': 'string', 'nullable': True, 'metadata': {}},\n",
       "     {'name': 'ForeignWorker',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {}},\n",
       "     {'name': 'Risk',\n",
       "      'type': 'string',\n",
       "      'nullable': True,\n",
       "      'metadata': {'modeling_role': 'target'}}]}}}}"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pre_prod_asset_details"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "id": "67443902-29ae-4f39-a0af-369e8c2b7ab1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "06b4e163-86b5-493d-8e28-8606fb296dca\n"
     ]
    }
   ],
   "source": [
    "pre_prod_subscription = wos_client.subscriptions.add(\n",
    "        data_mart_id=data_mart_id,\n",
    "        service_provider_id=service_provider_id,\n",
    "        asset=pre_prod_asset,\n",
    "        deployment=pre_prod_asset_deployment,\n",
    "        asset_properties=pre_prod_asset_properties_request).result\n",
    "pre_prod_subscription_id = pre_prod_subscription.metadata.id\n",
    "print(pre_prod_subscription_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "id": "92e408de-7e75-4df5-9ea9-20c43253bfc5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: Software specification spark-mllib_3.3 is deprecated. Use spark-mllib_3.4 software specification instead when saving a spark model. For details, see https://www.ibm.com/support/producthub/icpdata/docs/content/SSQNUZ_latest/wsj/wmls/wmls-deploy-python-types.html.\n"
     ]
    }
   ],
   "source": [
    "# Create challenger subscription\n",
    "challenger_deployment = wml_client.deployments.get_details(deployment_uid=challenger_deployment_uid)\n",
    "challenger_asset_details=wos_client.service_providers.get_deployment_asset(data_mart_id=data_mart_id,service_provider_id=service_provider_id,deployment_id=challenger_deployment_uid,deployment_space_id=space_id)\n",
    "challenger_asset = Asset(\n",
    "    asset_id=challenger_asset_details[\"entity\"][\"asset\"][\"asset_id\"],\n",
    "    name=challenger_asset_details[\"entity\"][\"asset\"][\"name\"],\n",
    "    url=challenger_asset_details[\"entity\"][\"asset\"][\"url\"],\n",
    "    asset_type=AssetTypes.MODEL,\n",
    "    input_data_type=InputDataType.STRUCTURED,\n",
    "    problem_type=ProblemType.BINARY_CLASSIFICATION,\n",
    "    model_type='scikit-learn_0.23'\n",
    ")\n",
    "challenger_asset_deployment = AssetDeploymentRequest(\n",
    "    deployment_id=challenger_deployment_uid,\n",
    "    name=PRE_PROD_CHALLENGER_DEPLOYMENT_NAME,\n",
    "    deployment_type=DeploymentTypes.ONLINE,\n",
    "    url=challenger_deployment[\"entity\"][\"status\"][\"online_url\"][\"url\"]\n",
    ")\n",
    "\n",
    "challenger_asset_properties_request = AssetPropertiesRequest(\n",
    "    label_column=\"Risk\",\n",
    "    probability_fields=[\"probability\"],\n",
    "    prediction_field=\"prediction\",\n",
    "    feature_fields=[\"CheckingStatus\",\"LoanDuration\",\"CreditHistory\",\"LoanPurpose\",\"LoanAmount\",\"ExistingSavings\",\"EmploymentDuration\",\"InstallmentPercent\",\"Sex\",\"OthersOnLoan\",\"CurrentResidenceDuration\",\"OwnsProperty\",\"Age\",\"InstallmentPlans\",\"Housing\",\"ExistingCreditsCount\",\"Job\",\"Dependents\",\"Telephone\",\"ForeignWorker\"],\n",
    "    categorical_fields=[\"CheckingStatus\",\"CreditHistory\",\"LoanPurpose\",\"ExistingSavings\",\"EmploymentDuration\",\"Sex\",\"OthersOnLoan\",\"OwnsProperty\",\"InstallmentPlans\",\"Housing\",\"Job\",\"Telephone\",\"ForeignWorker\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "id": "951027ef14c94d659d39fdc571d3ddbc"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7\n"
     ]
    }
   ],
   "source": [
    "challenger_subscription = wos_client.subscriptions.add(\n",
    "        data_mart_id=data_mart_id,\n",
    "        service_provider_id=service_provider_id,\n",
    "        asset=challenger_asset,\n",
    "        deployment=challenger_asset_deployment,\n",
    "        asset_properties=challenger_asset_properties_request).result\n",
    "challenger_subscription_id = challenger_subscription.metadata.id\n",
    "print(challenger_subscription_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "id": "dbd05839-fac8-4f66-8800-d36ec5d31ac7"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>Subscriptions</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>asset_id</th><th style='border: 1px solid #dddddd'>asset_type</th><th style='border: 1px solid #dddddd'>asset_name</th><th style='border: 1px solid #dddddd'>data_mart_id</th><th style='border: 1px solid #dddddd'>deployment_id</th><th style='border: 1px solid #dddddd'>deployment_name</th><th style='border: 1px solid #dddddd'>service_provider_id</th><th style='border: 1px solid #dddddd'>status</th><th style='border: 1px solid #dddddd'>created_at</th><th style='border: 1px solid #dddddd'>id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>fe7ed7dd-2856-42d6-83ab-51c13079f78d</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>German Credit Risk Model - PreProd</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>42b79776-470c-4d05-9115-76c4f5fffeab</td><td style='border: 1px solid #dddddd'>German Credit Risk Model - Challenger</td><td style='border: 1px solid #dddddd'>33397d85-dcae-4703-907d-4668932f7df0</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-08-08 00:28:30.628000+00:00</td><td style='border: 1px solid #dddddd'>a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7</td></tr><tr><td style='border: 1px solid #dddddd'>fe7ed7dd-2856-42d6-83ab-51c13079f78d</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>German Credit Risk Model - PreProd</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>42b79776-470c-4d05-9115-76c4f5fffeab</td><td style='border: 1px solid #dddddd'>German Credit Risk Model - PreProd</td><td style='border: 1px solid #dddddd'>33397d85-dcae-4703-907d-4668932f7df0</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-08-08 00:28:25.288000+00:00</td><td style='border: 1px solid #dddddd'>06b4e163-86b5-493d-8e28-8606fb296dca</td></tr><tr><td style='border: 1px solid #dddddd'>1b9f22c5-ac03-4017-b40b-27969e429379</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>Scikit German Risk Model WML V4 - zLinux</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>edc1e06c-7bf9-4522-8893-05a919a35ca0</td><td style='border: 1px solid #dddddd'>Scikit German Risk Deployment WML V4 - zLinux</td><td style='border: 1px solid #dddddd'>cf31400f-925c-4180-b4b2-77e53440d9a7</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-08-07 09:57:28.433000+00:00</td><td style='border: 1px solid #dddddd'>ad19393e-2078-4316-ac19-7de0ddfd7937</td></tr><tr><td style='border: 1px solid #dddddd'>df904b8a-2884-4318-82df-35cb430f8587</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>Scikit German Risk Model - for training data - zLinux</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>17e4097d-c4b5-41fe-bba5-3e12af95041e</td><td style='border: 1px solid #dddddd'>Scikit German Risk Model - for training data - zLinux</td><td style='border: 1px solid #dddddd'>1b7f6643-f6ac-4754-9f39-e8dbe5b09232</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-08-07 08:51:02.441000+00:00</td><td style='border: 1px solid #dddddd'>bcbbcc1f-7e1c-4746-8055-128ce8d805ff</td></tr><tr><td style='border: 1px solid #dddddd'>20516963-37ff-4769-a41f-b509827f3076</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>Text Binary Classifier</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>18c8f8d4-cfbf-4fbb-a51a-5ac59d2e9e20</td><td style='border: 1px solid #dddddd'>Text Binary Classifier deployment</td><td style='border: 1px solid #dddddd'>b9fd3cb5-c7a4-4b5c-a90c-684676276352</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-08-07 06:51:41.900000+00:00</td><td style='border: 1px solid #dddddd'>d673e127-dd69-4b05-a204-5b90332d7e72</td></tr><tr><td style='border: 1px solid #dddddd'>a4a1a71d-2a44-43f8-b92a-ba3c80dc36fd</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>Credit Risk python Fn Model</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>90b19695-de03-41b9-b820-2d3e11c2f662</td><td style='border: 1px solid #dddddd'>dep_Credit Risk python Fn Model</td><td style='border: 1px solid #dddddd'>e413a76d-802c-44f3-8c55-300c5aa2659b</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-08-06 13:32:53.985000+00:00</td><td style='border: 1px solid #dddddd'>77c99979-8408-40b7-ab3a-e3c0a8a80aab</td></tr><tr><td style='border: 1px solid #dddddd'>eed8b225-19e5-4460-9b6f-7271dc1e3ff2</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>Adult Census Income Classifier Model</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>038b8c68-6615-4c0d-8d46-15783d6ab9c8</td><td style='border: 1px solid #dddddd'>Adult Census Income Classifier Deployment</td><td style='border: 1px solid #dddddd'>c3601bc6-22c9-45a1-b016-c29cf0bf64b4</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-08-06 13:26:09.549000+00:00</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td></tr><tr><td style='border: 1px solid #dddddd'>438ca544-9bd1-48c2-8e8d-3de4ef4ca79b</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>WML_IAE4</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>78a0af9e-1014-4fb1-b22a-5e11f4fd70e7</td><td style='border: 1px solid #dddddd'>WML_IAE4</td><td style='border: 1px solid #dddddd'>d90c6bf2-49c6-4179-9876-8b85b0247d95</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-02 07:03:31.504000+00:00</td><td style='border: 1px solid #dddddd'>e34b9b87-b6e1-4c53-b92e-cb80dea042be</td></tr><tr><td style='border: 1px solid #dddddd'>592b902d-3dc9-4e56-8bcb-86cbf1a6d8a9</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>gcr - P2 XGB Classifier - Model</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>2b976af0-e4ab-4859-af7d-2f2287d864ad</td><td style='border: 1px solid #dddddd'>gcr model</td><td style='border: 1px solid #dddddd'>4d2f2fb2-6b64-4d58-8f13-257166e468e9</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-17 07:11:44.727000+00:00</td><td style='border: 1px solid #dddddd'>e2df4ec7-6c75-416f-a444-8d21389f7513</td></tr><tr><td style='border: 1px solid #dddddd'>e3ac9fc3-bccf-4a4e-b37b-490bfb93dd81</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>GCR AutoAI - P2 XGB Classifier - Model</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>6399e6e8-df5a-4370-9af4-34b2f2e76bc6</td><td style='border: 1px solid #dddddd'>GCR Auto AI</td><td style='border: 1px solid #dddddd'>a7ca157a-de07-457a-8c4c-b1a2e998699c</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-03 10:37:20.487000+00:00</td><td style='border: 1px solid #dddddd'>ce36911c-75f6-4f99-b4d2-b71e5d55a802</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: First 10 records were displayed.\n"
     ]
    }
   ],
   "source": [
    "wos_client.subscriptions.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "id": "b123565b-aa22-4e18-b601-25817ebff9f0"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'06b4e163-86b5-493d-8e28-8606fb296dca'"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pre_prod_subscription_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "id": "d51b3056-6e8e-4914-a98f-958a53c00210"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7'"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "challenger_subscription_id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5d16d0e9-9d15-47d3-9001-e7e25f8d8420"
   },
   "source": [
    "## Patch the training data reference in both the preprod & challenger subscription"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "id": "c5dd05f6-5b6a-424c-8596-5f0d1fca403c"
   },
   "outputs": [],
   "source": [
    "headers = {}\n",
    "headers[\"Content-Type\"] = \"application/json\"\n",
    "headers[\"Authorization\"] = \"Bearer {}\".format(generate_access_token())\n",
    "\n",
    "training_data_reference = {\n",
    "  \"connection\": {\n",
    "    \"connection_string\": \"jdbc:db2://dashdb-txn-sbox-yp-dal09-03.services.dal.bluemix.net:50000/BLUDB:retrieveMessagesFromServerOnGetMessage=true;\",\n",
    "    \"database_name\": \"BLUDB\",\n",
    "    \"hostname\": \"dashdb-txn-sbox-yp-dal09-03.services.dal.bluemix.net\",\n",
    "    \"password\": \"khhz72v+6mcwwkfv\",\n",
    "    \"username\": \"cmb91569\"\n",
    "  },\n",
    "  \"location\": {\n",
    "    \"schema_name\": \"CMB91569\",\n",
    "    \"table_name\": \"CREDIT_RISK_TRAIN_DATA\"\n",
    "  },\n",
    "  \"name\": \"German credit risk training data\",\n",
    "  \"type\": \"db2\"\n",
    "}\n",
    "\n",
    "payload = [\n",
    " {\n",
    "   \"op\": \"replace\",\n",
    "   \"path\": \"/asset_properties/training_data_reference\",\n",
    "   \"value\": training_data_reference\n",
    " }\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "id": "5f8350cd073a41078c1d49c0afcf06df"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'metadata': {'id': '06b4e163-86b5-493d-8e28-8606fb296dca',\n",
       "  'crn': 'crn:v1:bluemix:public:aiopenscale:us-south:a/na:00000000-0000-0000-0000-000000000000:subscription:06b4e163-86b5-493d-8e28-8606fb296dca',\n",
       "  'url': '/v2/subscriptions/06b4e163-86b5-493d-8e28-8606fb296dca',\n",
       "  'created_at': '2024-08-08T00:28:25.288000Z',\n",
       "  'created_by': 'cpadmin',\n",
       "  'modified_at': '2024-08-08T01:17:25.916000Z',\n",
       "  'modified_by': 'cpadmin'},\n",
       " 'entity': {'data_mart_id': '00000000-0000-0000-0000-000000000000',\n",
       "  'service_provider_id': '33397d85-dcae-4703-907d-4668932f7df0',\n",
       "  'asset': {'asset_id': 'fe7ed7dd-2856-42d6-83ab-51c13079f78d',\n",
       "   'url': 'https://internal-nginx-svc:12443/ml/v4/models/fe7ed7dd-2856-42d6-83ab-51c13079f78d?space_id=498630dc-3e0b-4470-b44d-583f2a393c0c&version=2020-06-12',\n",
       "   'name': 'German Credit Risk Model - PreProd',\n",
       "   'asset_type': 'model',\n",
       "   'problem_type': 'binary',\n",
       "   'model_type': 'mllib_3.3',\n",
       "   'input_data_type': 'structured'},\n",
       "  'asset_properties': {'training_data_reference': {'secret_id': '96bc75a0-ac93-44ac-bcf7-f3542a017164'},\n",
       "   'training_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'modeling_role': 'target'},\n",
       "      'name': 'Risk',\n",
       "      'nullable': True,\n",
       "      'type': 'string'}]},\n",
       "   'input_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'}]},\n",
       "   'output_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 128},\n",
       "       'modeling_role': 'record-id',\n",
       "       'primary_key': True},\n",
       "      'name': 'scoring_id',\n",
       "      'nullable': False,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'record-timestamp'},\n",
       "      'name': 'scoring_timestamp',\n",
       "      'nullable': False,\n",
       "      'type': 'timestamp'},\n",
       "     {'metadata': {},\n",
       "      'name': 'deployment_id',\n",
       "      'nullable': False,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {},\n",
       "      'name': 'asset_revision',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {},\n",
       "      'name': 'CheckingStatus_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'CreditHistory_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'EmploymentDuration_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'ExistingSavings_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'ForeignWorker_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'Housing_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'InstallmentPlans_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {}, 'name': 'Job_IX', 'nullable': True, 'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'LoanPurpose_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'OthersOnLoan_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'OwnsProperty_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {}, 'name': 'Sex_IX', 'nullable': True, 'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'Telephone_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'features',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {},\n",
       "      'name': 'rawPrediction',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {'modeling_role': 'probability'},\n",
       "      'name': 'probability',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {},\n",
       "      'name': 'prediction',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'modeling_role': 'prediction'},\n",
       "      'name': 'predictedLabel',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'prediction-probability'},\n",
       "      'name': 'prediction_probability',\n",
       "      'nullable': True,\n",
       "      'type': 'double'}]},\n",
       "   'label_column': 'Risk',\n",
       "   'prediction_field': 'predictedLabel',\n",
       "   'feature_fields': ['CheckingStatus',\n",
       "    'LoanDuration',\n",
       "    'CreditHistory',\n",
       "    'LoanPurpose',\n",
       "    'LoanAmount',\n",
       "    'ExistingSavings',\n",
       "    'EmploymentDuration',\n",
       "    'InstallmentPercent',\n",
       "    'Sex',\n",
       "    'OthersOnLoan',\n",
       "    'CurrentResidenceDuration',\n",
       "    'OwnsProperty',\n",
       "    'Age',\n",
       "    'InstallmentPlans',\n",
       "    'Housing',\n",
       "    'ExistingCreditsCount',\n",
       "    'Job',\n",
       "    'Dependents',\n",
       "    'Telephone',\n",
       "    'ForeignWorker'],\n",
       "   'categorical_fields': ['CheckingStatus',\n",
       "    'CreditHistory',\n",
       "    'LoanPurpose',\n",
       "    'ExistingSavings',\n",
       "    'EmploymentDuration',\n",
       "    'Sex',\n",
       "    'OthersOnLoan',\n",
       "    'OwnsProperty',\n",
       "    'InstallmentPlans',\n",
       "    'Housing',\n",
       "    'Job',\n",
       "    'Telephone',\n",
       "    'ForeignWorker'],\n",
       "   'probability_fields': ['probability']},\n",
       "  'deployment': {'deployment_id': '42b79776-470c-4d05-9115-76c4f5fffeab',\n",
       "   'url': 'https://cpd-cpd-instance.apps.wos415nfs2672.cp.fyre.ibm.com/ml/v4/deployments/42b79776-470c-4d05-9115-76c4f5fffeab/predictions',\n",
       "   'name': 'German Credit Risk Model - PreProd',\n",
       "   'deployment_type': 'online'},\n",
       "  'status': {'state': 'active'}}}"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = wos_client.subscriptions.update(pre_prod_subscription_id, payload)\n",
    "response.result.to_dict()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "id": "a90a8863-d279-48af-8a91-272c6c8ea102"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'metadata': {'id': 'a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7',\n",
       "  'crn': 'crn:v1:bluemix:public:aiopenscale:us-south:a/na:00000000-0000-0000-0000-000000000000:subscription:a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7',\n",
       "  'url': '/v2/subscriptions/a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7',\n",
       "  'created_at': '2024-08-08T00:28:30.628000Z',\n",
       "  'created_by': 'cpadmin',\n",
       "  'modified_at': '2024-08-08T01:17:29.858000Z',\n",
       "  'modified_by': 'cpadmin'},\n",
       " 'entity': {'data_mart_id': '00000000-0000-0000-0000-000000000000',\n",
       "  'service_provider_id': '33397d85-dcae-4703-907d-4668932f7df0',\n",
       "  'asset': {'asset_id': 'fe7ed7dd-2856-42d6-83ab-51c13079f78d',\n",
       "   'url': 'https://internal-nginx-svc:12443/ml/v4/models/fe7ed7dd-2856-42d6-83ab-51c13079f78d?space_id=498630dc-3e0b-4470-b44d-583f2a393c0c&version=2020-06-12',\n",
       "   'name': 'German Credit Risk Model - PreProd',\n",
       "   'asset_type': 'model',\n",
       "   'problem_type': 'binary',\n",
       "   'model_type': 'scikit-learn_0.23',\n",
       "   'input_data_type': 'structured'},\n",
       "  'asset_properties': {'training_data_reference': {'secret_id': '9149333f-fb01-4260-8fb2-b261fc739704'},\n",
       "   'training_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'target'},\n",
       "      'name': 'Risk',\n",
       "      'nullable': True,\n",
       "      'type': 'double'}]},\n",
       "   'input_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'}]},\n",
       "   'output_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 128},\n",
       "       'modeling_role': 'record-id',\n",
       "       'primary_key': True},\n",
       "      'name': 'scoring_id',\n",
       "      'nullable': False,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'record-timestamp'},\n",
       "      'name': 'scoring_timestamp',\n",
       "      'nullable': False,\n",
       "      'type': 'timestamp'},\n",
       "     {'metadata': {},\n",
       "      'name': 'deployment_id',\n",
       "      'nullable': False,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {},\n",
       "      'name': 'asset_revision',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {},\n",
       "      'name': 'CheckingStatus_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'CreditHistory_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'EmploymentDuration_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'ExistingSavings_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'ForeignWorker_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'Housing_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'InstallmentPlans_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {}, 'name': 'Job_IX', 'nullable': True, 'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'LoanPurpose_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'OthersOnLoan_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'OwnsProperty_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {}, 'name': 'Sex_IX', 'nullable': True, 'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'Telephone_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'features',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {},\n",
       "      'name': 'rawPrediction',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {'modeling_role': 'probability'},\n",
       "      'name': 'probability',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {'modeling_role': 'prediction'},\n",
       "      'name': 'prediction',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64}},\n",
       "      'name': 'predictedLabel',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'prediction-probability'},\n",
       "      'name': 'prediction_probability',\n",
       "      'nullable': True,\n",
       "      'type': 'double'}]},\n",
       "   'label_column': 'Risk',\n",
       "   'prediction_field': 'prediction',\n",
       "   'feature_fields': ['CheckingStatus',\n",
       "    'LoanDuration',\n",
       "    'CreditHistory',\n",
       "    'LoanPurpose',\n",
       "    'LoanAmount',\n",
       "    'ExistingSavings',\n",
       "    'EmploymentDuration',\n",
       "    'InstallmentPercent',\n",
       "    'Sex',\n",
       "    'OthersOnLoan',\n",
       "    'CurrentResidenceDuration',\n",
       "    'OwnsProperty',\n",
       "    'Age',\n",
       "    'InstallmentPlans',\n",
       "    'Housing',\n",
       "    'ExistingCreditsCount',\n",
       "    'Job',\n",
       "    'Dependents',\n",
       "    'Telephone',\n",
       "    'ForeignWorker'],\n",
       "   'categorical_fields': ['CheckingStatus',\n",
       "    'CreditHistory',\n",
       "    'LoanPurpose',\n",
       "    'ExistingSavings',\n",
       "    'EmploymentDuration',\n",
       "    'Sex',\n",
       "    'OthersOnLoan',\n",
       "    'OwnsProperty',\n",
       "    'InstallmentPlans',\n",
       "    'Housing',\n",
       "    'Job',\n",
       "    'Telephone',\n",
       "    'ForeignWorker'],\n",
       "   'probability_fields': ['probability']},\n",
       "  'deployment': {'deployment_id': '42b79776-470c-4d05-9115-76c4f5fffeab',\n",
       "   'url': 'https://cpd-cpd-instance.apps.wos415nfs2672.cp.fyre.ibm.com/ml/v4/deployments/42b79776-470c-4d05-9115-76c4f5fffeab/predictions',\n",
       "   'name': 'German Credit Risk Model - Challenger',\n",
       "   'deployment_type': 'online'},\n",
       "  'status': {'state': 'active'}}}"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = wos_client.subscriptions.update(challenger_subscription_id, payload)\n",
    "response.result.to_dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "75289b15-faeb-4dfe-a420-1bd3e7fb60e1"
   },
   "source": [
    "### Score the model so we can configure monitors\n",
    "Now that the WML service has been bound and the subscription has been created, we need to send a request to the model before we configure OpenScale. This allows OpenScale to create a payload log in the datamart with the correct schema, so it can capture data coming into and out of the model. First, the code gets the model deployment's endpoint URL, and then sends a few records for predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "id": "41491ac0-c360-42ec-9f55-4cd0c5e32407"
   },
   "outputs": [],
   "source": [
    "fields = [\"CheckingStatus\",\"LoanDuration\",\"CreditHistory\",\"LoanPurpose\",\"LoanAmount\",\"ExistingSavings\",\"EmploymentDuration\",\"InstallmentPercent\",\"Sex\",\"OthersOnLoan\",\"CurrentResidenceDuration\",\"OwnsProperty\",\"Age\",\"InstallmentPlans\",\"Housing\",\"ExistingCreditsCount\",\"Job\",\"Dependents\",\"Telephone\",\"ForeignWorker\"]\n",
    "values = [\n",
    "  [\"no_checking\",13,\"credits_paid_to_date\",\"car_new\",1343,\"100_to_500\",\"1_to_4\",2,\"female\",\"none\",3,\"savings_insurance\",46,\"none\",\"own\",2,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"no_checking\",24,\"prior_payments_delayed\",\"furniture\",4567,\"500_to_1000\",\"1_to_4\",4,\"male\",\"none\",4,\"savings_insurance\",36,\"none\",\"free\",2,\"management_self-employed\",1,\"none\",\"yes\"],\n",
    "  [\"0_to_200\",26,\"all_credits_paid_back\",\"car_new\",863,\"less_100\",\"less_1\",2,\"female\",\"co-applicant\",2,\"real_estate\",38,\"none\",\"own\",1,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"0_to_200\",14,\"no_credits\",\"car_new\",2368,\"less_100\",\"1_to_4\",3,\"female\",\"none\",3,\"real_estate\",29,\"none\",\"own\",1,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"0_to_200\",4,\"no_credits\",\"car_new\",250,\"less_100\",\"unemployed\",2,\"female\",\"none\",3,\"real_estate\",23,\"none\",\"rent\",1,\"management_self-employed\",1,\"none\",\"yes\"],\n",
    "  [\"no_checking\",17,\"credits_paid_to_date\",\"car_new\",832,\"100_to_500\",\"1_to_4\",2,\"male\",\"none\",2,\"real_estate\",42,\"none\",\"own\",1,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"no_checking\",33,\"outstanding_credit\",\"appliances\",5696,\"unknown\",\"greater_7\",4,\"male\",\"co-applicant\",4,\"unknown\",54,\"none\",\"free\",2,\"skilled\",1,\"yes\",\"yes\"],\n",
    "  [\"0_to_200\",13,\"prior_payments_delayed\",\"retraining\",1375,\"100_to_500\",\"4_to_7\",3,\"male\",\"none\",3,\"real_estate\",37,\"none\",\"own\",2,\"management_self-employed\",1,\"none\",\"yes\"]\n",
    "]\n",
    "\n",
    "payload_scoring = {\"fields\": fields,\"values\": values}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "id": "eacd013d-8e52-430e-9c9f-3fb42a4add4d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Single record Scoring result: \n",
      " fields: ['CheckingStatus', 'LoanDuration', 'CreditHistory', 'LoanPurpose', 'LoanAmount', 'ExistingSavings', 'EmploymentDuration', 'InstallmentPercent', 'Sex', 'OthersOnLoan', 'CurrentResidenceDuration', 'OwnsProperty', 'Age', 'InstallmentPlans', 'Housing', 'ExistingCreditsCount', 'Job', 'Dependents', 'Telephone', 'ForeignWorker', 'CheckingStatus_IX', 'CreditHistory_IX', 'EmploymentDuration_IX', 'ExistingSavings_IX', 'ForeignWorker_IX', 'Housing_IX', 'InstallmentPlans_IX', 'Job_IX', 'LoanPurpose_IX', 'OthersOnLoan_IX', 'OwnsProperty_IX', 'Sex_IX', 'Telephone_IX', 'features', 'rawPrediction', 'probability', 'prediction', 'predictedLabel'] \n",
      " values:  ['no_checking', 13, 'credits_paid_to_date', 'car_new', 1343, '100_to_500', '1_to_4', 2, 'female', 'none', 3, 'savings_insurance', 46, 'none', 'own', 2, 'skilled', 1, 'none', 'yes', 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, [21, [1, 3, 5, 13, 14, 15, 16, 17, 18, 19, 20], [1.0, 1.0, 1.0, 13.0, 1343.0, 2.0, 3.0, 13.0, 46.0, 2.0, 1.0]], [0.37923501094948475, -0.37923501094948475], [0.6810214668555962, 0.31897853314440383], 0.0, 'No Risk']\n"
     ]
    }
   ],
   "source": [
    "payload = {\n",
    "    wml_client.deployments.ScoringMetaNames.INPUT_DATA: [payload_scoring]\n",
    "}\n",
    "scoring_response = wml_client.deployments.score(pre_prod_deployment_uid, payload)\n",
    "\n",
    "print('Single record Scoring result:', '\\n fields:', scoring_response['predictions'][0]['fields'], '\\n values: ', scoring_response['predictions'][0]['values'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "id": "1e48deef-7631-4ee1-8ec4-d15f10313c59"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Payload data set id:  037c48fa-9023-4544-abe3-bcf6e8f83b00\n",
      "Number of records in the payload logging table: 8\n"
     ]
    }
   ],
   "source": [
    "time.sleep(10)\n",
    "pre_prod_pl_data_set_id = None\n",
    "pre_prod_pl_data_set_id = wos_client.data_sets.list(type=DataSetTypes.PAYLOAD_LOGGING, \n",
    "                                                target_target_id=pre_prod_subscription_id, \n",
    "                                                target_target_type=TargetTypes.SUBSCRIPTION).result.data_sets[0].metadata.id\n",
    "if pre_prod_pl_data_set_id is None:\n",
    "    print(\"Payload data set not found. Please check subscription status.\")\n",
    "else:\n",
    "    print(\"Payload data set id: \", pre_prod_pl_data_set_id)\n",
    "    pl_records_count = wos_client.data_sets.get_records_count(pre_prod_pl_data_set_id)\n",
    "    print(\"Number of records in the payload logging table: {}\".format(pl_records_count))\n",
    "    if pl_records_count == 0:\n",
    "        print(\"Automatic payload logging did not happen, performing explicit payload logging.\")\n",
    "        wos_client.data_sets.store_records(data_set_id=pre_prod_pl_data_set_id, request_body=[PayloadRecord(\n",
    "                       scoring_id=\"mrm_\"+str(uuid.uuid4()),\n",
    "                       request=payload_scoring,\n",
    "                       response=scoring_response['predictions'][0]\n",
    "                   )])\n",
    "        time.sleep(5)\n",
    "        pl_records_count = wos_client.data_sets.get_records_count(pre_prod_pl_data_set_id)\n",
    "        print(\"Number of records in the payload logging table: {}\".format(pl_records_count))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "id": "b3360e3b-a588-4b14-9dff-ab766209c501"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Single record Scoring result: \n",
      " fields: ['CheckingStatus', 'LoanDuration', 'CreditHistory', 'LoanPurpose', 'LoanAmount', 'ExistingSavings', 'EmploymentDuration', 'InstallmentPercent', 'Sex', 'OthersOnLoan', 'CurrentResidenceDuration', 'OwnsProperty', 'Age', 'InstallmentPlans', 'Housing', 'ExistingCreditsCount', 'Job', 'Dependents', 'Telephone', 'ForeignWorker', 'CheckingStatus_IX', 'CreditHistory_IX', 'EmploymentDuration_IX', 'ExistingSavings_IX', 'ForeignWorker_IX', 'Housing_IX', 'InstallmentPlans_IX', 'Job_IX', 'LoanPurpose_IX', 'OthersOnLoan_IX', 'OwnsProperty_IX', 'Sex_IX', 'Telephone_IX', 'features', 'rawPrediction', 'probability', 'prediction', 'predictedLabel'] \n",
      " values:  ['no_checking', 13, 'credits_paid_to_date', 'car_new', 1343, '100_to_500', '1_to_4', 2, 'female', 'none', 3, 'savings_insurance', 46, 'none', 'own', 2, 'skilled', 1, 'none', 'yes', 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, [21, [1, 3, 5, 13, 14, 15, 16, 17, 18, 19, 20], [1.0, 1.0, 1.0, 13.0, 1343.0, 2.0, 3.0, 13.0, 46.0, 2.0, 1.0]], [0.37923501094948475, -0.37923501094948475], [0.6810214668555962, 0.31897853314440383], 0.0, 'No Risk']\n"
     ]
    }
   ],
   "source": [
    "payload = {\n",
    "    wml_client.deployments.ScoringMetaNames.INPUT_DATA: [payload_scoring]\n",
    "}\n",
    "scoring_response = wml_client.deployments.score(challenger_deployment_uid, payload)\n",
    "\n",
    "print('Single record Scoring result:', '\\n fields:', scoring_response['predictions'][0]['fields'], '\\n values: ', scoring_response['predictions'][0]['values'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "id": "1f7b180e-46e0-4e14-90de-0b4f9a701a02"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Payload data set id:  2f7ba47c-cff2-4897-a27e-55b8d60eaafa\n",
      "Number of records in the payload logging table: 16\n"
     ]
    }
   ],
   "source": [
    "from ibm_watson_openscale.supporting_classes import *\n",
    "time.sleep(10)\n",
    "challenger_pl_data_set_id = None\n",
    "challenger_pl_data_set_id = wos_client.data_sets.list(type=DataSetTypes.PAYLOAD_LOGGING, \n",
    "                                                target_target_id=challenger_subscription_id, \n",
    "                                                target_target_type=TargetTypes.SUBSCRIPTION).result.data_sets[0].metadata.id\n",
    "if challenger_pl_data_set_id is None:\n",
    "    print(\"Payload data set not found. Please check subscription status.\")\n",
    "else:\n",
    "    print(\"Payload data set id: \", challenger_pl_data_set_id)\n",
    "    pl_records_count = wos_client.data_sets.get_records_count(challenger_pl_data_set_id)\n",
    "    print(\"Number of records in the payload logging table: {}\".format(pl_records_count))\n",
    "    if pl_records_count == 0:\n",
    "        print(\"Automatic payload logging did not happen, performing explicit payload logging.\")\n",
    "        wos_client.data_sets.store_records(data_set_id=challenger_pl_data_set_id, request_body=[PayloadRecord(\n",
    "                       scoring_id=\"mrm_\"+str(uuid.uuid4()),\n",
    "                       request=payload_scoring,\n",
    "                       response=scoring_response['predictions'][0]\n",
    "                   )])\n",
    "        time.sleep(5)\n",
    "        pl_records_count = wos_client.data_sets.get_records_count(challenger_pl_data_set_id)\n",
    "        print(\"Number of records in the payload logging table: {}\".format(pl_records_count))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9f743917-8bd6-4b0b-a7a4-22e323f9ca0f"
   },
   "source": [
    "# Quality monitoring\n",
    "\n",
    "## Enable quality monitoring\n",
    "The code below waits ten seconds to allow the payload logging table to be set up before it begins enabling monitors. First, it turns on the quality (accuracy) monitor and sets an alert threshold of 80%. OpenScale will show an alert on the dashboard if the model accuracy measurement (area under the curve, in the case of a binary classifier) falls below this threshold.\n",
    "\n",
    "The second paramater supplied, min_records, specifies the minimum number of feedback records OpenScale needs before it calculates a new measurement. The quality monitor runs hourly, but the accuracy reading in the dashboard will not change until an additional 50 feedback records have been added, via the user interface, the Python client, or the supplied feedback endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "id": "951aeb7a-ea55-4391-9406-a3008b6078f5",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "===================================================================================\n",
      "\n",
      " Waiting for end of monitor instance creation 0392d9f8-d375-49ef-b754-e3c6584070c7 \n",
      "\n",
      "===================================================================================\n",
      "\n",
      "\n",
      "\n",
      "preparing\n",
      "active\n",
      "\n",
      "---------------------------------------\n",
      " Monitor instance successfully created \n",
      "---------------------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "time.sleep(10)\n",
    "target = Target(\n",
    "        target_type=TargetTypes.SUBSCRIPTION,\n",
    "        target_id=pre_prod_subscription_id\n",
    ")\n",
    "parameters = {\n",
    "    \"min_feedback_data_size\": 100,\n",
    "    \"threshold\": 0.8\n",
    "}\n",
    "pre_prod_quality_monitor_details = wos_client.monitor_instances.create(\n",
    "    data_mart_id=data_mart_id,\n",
    "    background_mode=False,\n",
    "    monitor_definition_id=wos_client.monitor_definitions.MONITORS.QUALITY.ID,\n",
    "    target=target,\n",
    "    parameters=parameters\n",
    ").result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "id": "d285aae9-f0c4-4de0-abad-beebe4472f56"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "===================================================================================\n",
      "\n",
      " Waiting for end of monitor instance creation c0c3274d-dc61-4384-84e3-5d4b3b258379 \n",
      "\n",
      "===================================================================================\n",
      "\n",
      "\n",
      "\n",
      "preparing\n",
      "active\n",
      "\n",
      "---------------------------------------\n",
      " Monitor instance successfully created \n",
      "---------------------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "time.sleep(10)\n",
    "target = Target(\n",
    "        target_type=TargetTypes.SUBSCRIPTION,\n",
    "        target_id=challenger_subscription_id\n",
    ")\n",
    "parameters = {\n",
    "    \"min_feedback_data_size\": 100,\n",
    "    \"threshold\": 0.8\n",
    "}\n",
    "challenger_quality_monitor_details = wos_client.monitor_instances.create(\n",
    "    data_mart_id=data_mart_id,\n",
    "    background_mode=False,\n",
    "    monitor_definition_id=wos_client.monitor_definitions.MONITORS.QUALITY.ID,\n",
    "    target=target,\n",
    "    parameters=parameters\n",
    ").result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "13e15202-a809-4dea-81ef-29307bdac80e"
   },
   "source": [
    "# Fairness, drift monitoring and explanations \n",
    "\n",
    "## Fairness configuration\n",
    "The code below configures fairness monitoring for our model. It turns on monitoring for two features, Sex and Age. In each case, we must specify:\n",
    "\n",
    "Which model feature to monitor\n",
    "One or more majority groups, which are values of that feature that we expect to receive a higher percentage of favorable outcomes\n",
    "One or more minority groups, which are values of that feature that we expect to receive a higher percentage of unfavorable outcomes\n",
    "The threshold at which we would like OpenScale to display an alert if the fairness measurement falls below (in this case, 80%)\n",
    "Additionally, we must specify which outcomes from the model are favourable outcomes, and which are unfavourable. We must also provide the number of records OpenScale will use to calculate the fairness score. In this case, OpenScale's fairness monitor will run hourly, but will not calculate a new fairness rating until at least 100 records have been added. Finally, to calculate fairness, OpenScale must perform some calculations on the training data, so we provide the dataframe containing the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "id": "dc3be33f-48e0-4be9-b52b-660f2a0438ca"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "===================================================================================\n",
      "\n",
      " Waiting for end of monitor instance creation 0654edac-ba0f-4167-87d6-9875b22e773c \n",
      "\n",
      "===================================================================================\n",
      "\n",
      "\n",
      "\n",
      "active\n",
      "\n",
      "---------------------------------------\n",
      " Monitor instance successfully created \n",
      "---------------------------------------\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'0654edac-ba0f-4167-87d6-9875b22e773c'"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "target = Target(\n",
    "    target_type=TargetTypes.SUBSCRIPTION,\n",
    "    target_id=pre_prod_subscription_id\n",
    "\n",
    ")\n",
    "parameters = {\n",
    "    \"features\": [\n",
    "        {\"feature\": \"Sex\",\n",
    "         \"majority\": ['male'],\n",
    "         \"minority\": ['female']\n",
    "         },\n",
    "        {\"feature\": \"Age\",\n",
    "         \"majority\": [[26, 75]],\n",
    "         \"minority\": [[18, 25]]\n",
    "         }\n",
    "    ],\n",
    "    \"favourable_class\": [\"No Risk\"],\n",
    "    \"unfavourable_class\": [\"Risk\"],\n",
    "    \"min_records\": 100\n",
    "}\n",
    "thresholds = [{\n",
    "    \"metric_id\": \"fairness_value\",\n",
    "    \"specific_values\": [{\n",
    "            \"applies_to\": [{\n",
    "                \"key\": \"feature\",\n",
    "                \"type\": \"tag\",\n",
    "                \"value\": \"Age\"\n",
    "            }],\n",
    "            \"value\": 95\n",
    "        },\n",
    "        {\n",
    "            \"applies_to\": [{\n",
    "                \"key\": \"feature\",\n",
    "                \"type\": \"tag\",\n",
    "                \"value\": \"Sex\"\n",
    "            }],\n",
    "            \"value\": 95\n",
    "        }\n",
    "    ],\n",
    "    \"type\": \"lower_limit\",\n",
    "    \"value\": 80.0\n",
    "}]\n",
    "\n",
    "pre_prod_fairness_monitor_details = wos_client.monitor_instances.create(\n",
    "    data_mart_id=data_mart_id,\n",
    "    background_mode=False,\n",
    "    monitor_definition_id=wos_client.monitor_definitions.MONITORS.FAIRNESS.ID,\n",
    "    target=target,\n",
    "    parameters=parameters,\n",
    "    thresholds=thresholds).result\n",
    "pre_prod_fairness_monitor_details.metadata.id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "id": "0f7fef90-5b12-42dd-971f-602b1808054d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "===================================================================================\n",
      "\n",
      " Waiting for end of monitor instance creation 46eba4d5-8367-406d-9ca9-db53acd7e409 \n",
      "\n",
      "===================================================================================\n",
      "\n",
      "\n",
      "\n",
      "preparing\n",
      "active\n",
      "\n",
      "---------------------------------------\n",
      " Monitor instance successfully created \n",
      "---------------------------------------\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'46eba4d5-8367-406d-9ca9-db53acd7e409'"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "target = Target(\n",
    "    target_type=TargetTypes.SUBSCRIPTION,\n",
    "    target_id=challenger_subscription_id\n",
    "\n",
    ")\n",
    "parameters = {\n",
    "    \"features\": [\n",
    "        {\"feature\": \"Sex\",\n",
    "         \"majority\": ['male'],\n",
    "         \"minority\": ['female']\n",
    "         },\n",
    "        {\"feature\": \"Age\",\n",
    "         \"majority\": [[26, 75]],\n",
    "         \"minority\": [[18, 25]]\n",
    "         }\n",
    "    ],\n",
    "    \"favourable_class\": [\"No Risk\"],\n",
    "    \"unfavourable_class\": [\"Risk\"],\n",
    "    \"min_records\": 100\n",
    "}\n",
    "thresholds = [{\n",
    "    \"metric_id\": \"fairness_value\",\n",
    "    \"specific_values\": [{\n",
    "            \"applies_to\": [{\n",
    "                \"key\": \"feature\",\n",
    "                \"type\": \"tag\",\n",
    "                \"value\": \"Age\"\n",
    "            }],\n",
    "            \"value\": 95\n",
    "        },\n",
    "        {\n",
    "            \"applies_to\": [{\n",
    "                \"key\": \"feature\",\n",
    "                \"type\": \"tag\",\n",
    "                \"value\": \"Sex\"\n",
    "            }],\n",
    "            \"value\": 95\n",
    "        }\n",
    "    ],\n",
    "    \"type\": \"lower_limit\",\n",
    "    \"value\": 80.0\n",
    "}]\n",
    "\n",
    "challenger_fairness_monitor_details = wos_client.monitor_instances.create(\n",
    "    data_mart_id=data_mart_id,\n",
    "    background_mode=False,\n",
    "    monitor_definition_id=wos_client.monitor_definitions.MONITORS.FAIRNESS.ID,\n",
    "    target=target,\n",
    "    parameters=parameters,\n",
    "    thresholds=thresholds).result\n",
    "challenger_fairness_monitor_details.metadata.id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6626044b-78a6-4788-beb3-93b83c1adb9b"
   },
   "source": [
    "## Drift configuration\n",
    "\n",
    "Enable the drift configuration for both the subscription created with a threshold of 10% and minimal sample as 100 records."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "id": "2d39d4ac-31eb-4855-88bb-afb23c7f4c39"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "===================================================================================\n",
      "\n",
      " Waiting for end of monitor instance creation 46eba4d5-8367-406d-9ca9-db53acd7e409 \n",
      "\n",
      "===================================================================================\n",
      "\n",
      "\n",
      "\n",
      "preparing\n",
      "active\n",
      "\n",
      "---------------------------------------\n",
      " Monitor instance successfully created \n",
      "---------------------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "monitor_instances = wos_client.monitor_instances.list().result.monitor_instances\n",
    "for monitor_instance in monitor_instances:\n",
    "    monitor_def_id=monitor_instance.entity.monitor_definition_id\n",
    "    if monitor_def_id == wos_client.monitor_definitions.MONITORS.DRIFT.ID and monitor_instance.entity.target.target_id == pre_prod_subscription_id:\n",
    "        wos_client.monitor_instances.delete(monitor_instance.metadata.id)\n",
    "        print('Deleted existing drift monitor instance with id: ', monitor_instance.metadata.id)\n",
    "\n",
    "\n",
    "target = Target(\n",
    "    target_type=TargetTypes.SUBSCRIPTION,\n",
    "    target_id=pre_prod_subscription_id\n",
    "\n",
    ")\n",
    "parameters = {\n",
    "    \"min_samples\": 100,\n",
    "    \"train_drift_model\": True,\n",
    "    \"enable_model_drift\": False,\n",
    "    \"enable_data_drift\": True\n",
    "}\n",
    "\n",
    "pre_prod_drift_monitor_details = wos_client.monitor_instances.create(\n",
    "    data_mart_id=data_mart_id,\n",
    "    background_mode=False,\n",
    "    monitor_definition_id=wos_client.monitor_definitions.MONITORS.DRIFT.ID,\n",
    "    target=target,\n",
    "    parameters=parameters\n",
    ").result\n",
    "\n",
    "pre_prod_drift_monitor_details.metadata.id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "id": "3351a278-ea8e-4767-b266-d69aca03028b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "===================================================================================\n",
      "\n",
      " Waiting for end of monitor instance creation ce9a3bc2-bd0e-4c03-b2b1-31dbffa37a6c \n",
      "\n",
      "===================================================================================\n",
      "\n",
      "\n",
      "\n",
      "preparing\n",
      "active\n",
      "\n",
      "---------------------------------------\n",
      " Monitor instance successfully created \n",
      "---------------------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "monitor_instances = wos_client.monitor_instances.list().result.monitor_instances\n",
    "for monitor_instance in monitor_instances:\n",
    "    monitor_def_id=monitor_instance.entity.monitor_definition_id\n",
    "    if monitor_def_id == wos_client.monitor_definitions.MONITORS.DRIFT.ID and monitor_instance.entity.target.target_id == challenger_subscription_id:\n",
    "        wos_client.monitor_instances.delete(monitor_instance.metadata.id)\n",
    "        print('Deleted existing drift monitor instance with id: ', monitor_instance.metadata.id)\n",
    "\n",
    "\n",
    "target = Target(\n",
    "    target_type=TargetTypes.SUBSCRIPTION,\n",
    "    target_id=challenger_subscription_id\n",
    "\n",
    ")\n",
    "parameters = {\n",
    "    \"min_samples\": 100,\n",
    "    \"train_drift_model\": True,\n",
    "    \"enable_model_drift\": False,\n",
    "    \"enable_data_drift\": True\n",
    "}\n",
    "\n",
    "challenger_drift_monitor_details = wos_client.monitor_instances.create(\n",
    "    data_mart_id=data_mart_id,\n",
    "    background_mode=False,\n",
    "    monitor_definition_id=wos_client.monitor_definitions.MONITORS.DRIFT.ID,\n",
    "    target=target,\n",
    "    parameters=parameters\n",
    ").result\n",
    "\n",
    "challenger_drift_monitor_details.metadata.id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Drift V2 Configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "===================================================================================\n",
      "\n",
      " Waiting for end of monitor instance creation 9c8ea6c0-9513-458a-a3ed-e0b04f74cbbc \n",
      "\n",
      "===================================================================================\n",
      "\n",
      "\n",
      "\n",
      "preparing\n",
      "active\n",
      "\n",
      "---------------------------------------\n",
      " Monitor instance successfully created \n",
      "---------------------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from random import choices\n",
    "\n",
    "monitor_instances = wos_client.monitor_instances.list().result.monitor_instances\n",
    "for monitor_instance in monitor_instances:\n",
    "    monitor_def_id=monitor_instance.entity.monitor_definition_id\n",
    "    if monitor_def_id == wos_client.monitor_definitions.MONITORS.DRIFT_V2.ID and monitor_instance.entity.target.target_id == pre_prod_subscription_id:\n",
    "        wos_client.monitor_instances.delete(monitor_instance.metadata.id)\n",
    "        print('Deleted existing drift monitor instance with id: ', monitor_instance.metadata.id)\n",
    "\n",
    "target = Target(\n",
    "    target_type=TargetTypes.SUBSCRIPTION,\n",
    "    target_id=pre_prod_subscription_id\n",
    ")\n",
    "parameters = {\n",
    "        \"min_samples\": 10,\n",
    "        \"max_samples\": 1000,\n",
    "        \"train_archive\": True,\n",
    "        \"features\": {\n",
    "            \"fields\": fields,\n",
    "        },\n",
    "        \"most_important_features\": { \n",
    "            \"fields\": choices(fields, k=5)\n",
    "        },\n",
    "}\n",
    "pre_prod_drift_v2_monitor_details = wos_client.monitor_instances.create(\n",
    "    data_mart_id=data_mart_id,\n",
    "    background_mode=False,\n",
    "    monitor_definition_id=wos_client.monitor_definitions.MONITORS.DRIFT_V2.ID,\n",
    "    target=target,\n",
    "    parameters=parameters\n",
    ").result\n",
    "\n",
    "pre_prod_drift_v2_monitor_details.metadata.id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "===================================================================================\n",
      "\n",
      " Waiting for end of monitor instance creation 0f087b1e-48fc-4a25-bdfd-25f90e2b1dd4 \n",
      "\n",
      "===================================================================================\n",
      "\n",
      "\n",
      "\n",
      "preparing\n",
      "active\n",
      "\n",
      "---------------------------------------\n",
      " Monitor instance successfully created \n",
      "---------------------------------------\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'0f087b1e-48fc-4a25-bdfd-25f90e2b1dd4'"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from random import choices\n",
    "\n",
    "monitor_instances = wos_client.monitor_instances.list().result.monitor_instances\n",
    "for monitor_instance in monitor_instances:\n",
    "    monitor_def_id=monitor_instance.entity.monitor_definition_id\n",
    "    if monitor_def_id == wos_client.monitor_definitions.MONITORS.DRIFT_V2.ID and monitor_instance.entity.target.target_id == challenger_subscription_id:\n",
    "        wos_client.monitor_instances.delete(monitor_instance.metadata.id)\n",
    "        print('Deleted existing drift monitor instance with id: ', monitor_instance.metadata.id)\n",
    "\n",
    "target = Target(\n",
    "    target_type=TargetTypes.SUBSCRIPTION,\n",
    "    target_id=challenger_subscription_id\n",
    ")\n",
    "parameters = {\n",
    "        \"min_samples\": 10,\n",
    "        \"max_samples\": 1000,\n",
    "        \"train_archive\": True,\n",
    "        \"features\": {\n",
    "            \"fields\": fields,\n",
    "        },\n",
    "        \"most_important_features\": { \n",
    "            \"fields\": choices(fields, k=5)\n",
    "        },\n",
    "}\n",
    "challenger_drift_v2_monitor_details = wos_client.monitor_instances.create(\n",
    "    data_mart_id=data_mart_id,\n",
    "    background_mode=False,\n",
    "    monitor_definition_id=wos_client.monitor_definitions.MONITORS.DRIFT_V2.ID,\n",
    "    target=target,\n",
    "    parameters=parameters\n",
    ").result\n",
    "\n",
    "challenger_drift_v2_monitor_details.metadata.id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cd722c63-9ac7-47f6-8929-a614056c997f"
   },
   "source": [
    "## Configure Explainability\n",
    "Finally, we provide OpenScale with the training data to enable and configure the explainability features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "id": "2c008c5b-70ff-4dbe-92b1-06e4781e5df1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "===================================================================================\n",
      "\n",
      " Waiting for end of monitor instance creation 6b2e8d86-3d05-42f6-b32f-6e2a5ce3a294 \n",
      "\n",
      "===================================================================================\n",
      "\n",
      "\n",
      "\n",
      "preparing\n",
      "active\n",
      "\n",
      "---------------------------------------\n",
      " Monitor instance successfully created \n",
      "---------------------------------------\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'6b2e8d86-3d05-42f6-b32f-6e2a5ce3a294'"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "target = Target(\n",
    "    target_type=TargetTypes.SUBSCRIPTION,\n",
    "    target_id=pre_prod_subscription_id\n",
    ")\n",
    "parameters = {\n",
    "    \"enabled\": True\n",
    "}\n",
    "pre_prod_explain_monitor_details = wos_client.monitor_instances.create(\n",
    "    data_mart_id=data_mart_id,\n",
    "    background_mode=False,\n",
    "    monitor_definition_id=wos_client.monitor_definitions.MONITORS.EXPLAINABILITY.ID,\n",
    "    target=target,\n",
    "    parameters=parameters\n",
    ").result\n",
    "\n",
    "pre_prod_explain_monitor_details.metadata.id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "id": "58f069b8adbd4632b03f6718715c0cde"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "===================================================================================\n",
      "\n",
      " Waiting for end of monitor instance creation 2602759f-4180-4fac-93d7-543b3b5aa71c \n",
      "\n",
      "===================================================================================\n",
      "\n",
      "\n",
      "\n",
      "preparing\n",
      "active\n",
      "\n",
      "---------------------------------------\n",
      " Monitor instance successfully created \n",
      "---------------------------------------\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'2602759f-4180-4fac-93d7-543b3b5aa71c'"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "target = Target(\n",
    "    target_type=TargetTypes.SUBSCRIPTION,\n",
    "    target_id=challenger_subscription_id\n",
    ")\n",
    "parameters = {\n",
    "    \"enabled\": True\n",
    "}\n",
    "challenger_explain_monitor_details = wos_client.monitor_instances.create(\n",
    "    data_mart_id=data_mart_id,\n",
    "    background_mode=False,\n",
    "    monitor_definition_id=wos_client.monitor_definitions.MONITORS.EXPLAINABILITY.ID,\n",
    "    target=target,\n",
    "    parameters=parameters\n",
    ").result\n",
    "\n",
    "challenger_explain_monitor_details.metadata.id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "e13c1f8b-ba92-4775-9acb-781f8796436b"
   },
   "source": [
    "## Enable model risk management (MRM) \n",
    "\n",
    "We enable the MRM configuration for both the subscriptions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "id": "c43ca07fbc8a445d86da2d96b3ec4ca8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "===================================================================================\n",
      "\n",
      " Waiting for end of monitor instance creation 36867a3c-9170-4b43-8218-041f48822df9 \n",
      "\n",
      "===================================================================================\n",
      "\n",
      "\n",
      "\n",
      "active\n",
      "\n",
      "---------------------------------------\n",
      " Monitor instance successfully created \n",
      "---------------------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "target = Target(\n",
    "    target_type=TargetTypes.SUBSCRIPTION,\n",
    "    target_id=pre_prod_subscription_id\n",
    ")\n",
    "parameters = {\n",
    "}\n",
    "pre_prod_mrm_monitor_details = wos_client.monitor_instances.create(\n",
    "    data_mart_id=data_mart_id,\n",
    "    background_mode=False,\n",
    "    monitor_definition_id='mrm',\n",
    "    target=target,\n",
    "    parameters=parameters\n",
    ").result\n",
    "\n",
    "pre_prod_mrm_instance_id = pre_prod_mrm_monitor_details.metadata.id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "id": "fa0b8ff607804d24bd21a27b7a7e3e3b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "===================================================================================\n",
      "\n",
      " Waiting for end of monitor instance creation 6a04baa9-baab-49e0-8864-566afa17d24f \n",
      "\n",
      "===================================================================================\n",
      "\n",
      "\n",
      "\n",
      "active\n",
      "\n",
      "---------------------------------------\n",
      " Monitor instance successfully created \n",
      "---------------------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "target = Target(\n",
    "    target_type=TargetTypes.SUBSCRIPTION,\n",
    "    target_id=challenger_subscription_id\n",
    ")\n",
    "parameters = {\n",
    "}\n",
    "challenger_mrm_monitor_details = wos_client.monitor_instances.create(\n",
    "    data_mart_id=data_mart_id,\n",
    "    background_mode=False,\n",
    "    monitor_definition_id='mrm',\n",
    "    target=target,\n",
    "    parameters=parameters\n",
    ").result\n",
    "\n",
    "challenger_mrm_instance_id = challenger_mrm_monitor_details.metadata.id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "id": "e98894c5dda44926b5c6aff1646b1005"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>Monitor instances</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>data_mart_id</th><th style='border: 1px solid #dddddd'>status</th><th style='border: 1px solid #dddddd'>target_id</th><th style='border: 1px solid #dddddd'>target_type</th><th style='border: 1px solid #dddddd'>monitor_definition_id</th><th style='border: 1px solid #dddddd'>created_at</th><th style='border: 1px solid #dddddd'>id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>2024-08-08 00:39:50.234000+00:00</td><td style='border: 1px solid #dddddd'>6a04baa9-baab-49e0-8864-566afa17d24f</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>06b4e163-86b5-493d-8e28-8606fb296dca</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>2024-08-08 00:39:42.325000+00:00</td><td style='border: 1px solid #dddddd'>36867a3c-9170-4b43-8218-041f48822df9</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>ce36911c-75f6-4f99-b4d2-b71e5d55a802</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>model_health</td><td style='border: 1px solid #dddddd'>2024-07-03 10:37:23.321000+00:00</td><td style='border: 1px solid #dddddd'>097c7a99-a0ac-4a5e-a415-ed733899ed3e</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>error</td><td style='border: 1px solid #dddddd'>a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>explainability</td><td style='border: 1px solid #dddddd'>2024-08-08 00:38:42.660000+00:00</td><td style='border: 1px solid #dddddd'>2602759f-4180-4fac-93d7-543b3b5aa71c</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>c385a4f8-2ef0-441d-baeb-b3f1a11cb3b0</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>model_health</td><td style='border: 1px solid #dddddd'>2024-06-14 05:38:07.626000+00:00</td><td style='border: 1px solid #dddddd'>15ee2b53-f250-4004-a391-b8d65dcdfcff</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>error</td><td style='border: 1px solid #dddddd'>06b4e163-86b5-493d-8e28-8606fb296dca</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>explainability</td><td style='border: 1px solid #dddddd'>2024-08-08 00:37:39.125000+00:00</td><td style='border: 1px solid #dddddd'>6b2e8d86-3d05-42f6-b32f-6e2a5ce3a294</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>ce36911c-75f6-4f99-b4d2-b71e5d55a802</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>2024-07-03 10:37:23.122000+00:00</td><td style='border: 1px solid #dddddd'>c3b21f0e-46d6-43bf-bb76-917587d52557</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>1b8ebb3c-e69a-4cc1-9a0a-57424546cab3</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>2024-07-03 04:37:00.338000+00:00</td><td style='border: 1px solid #dddddd'>c807c46a-bdd4-493b-9967-47985673990b</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>3fc64ae7-b852-4951-bc65-4e450b473952</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>2024-06-26 04:34:21.284000+00:00</td><td style='border: 1px solid #dddddd'>80265b34-0089-46b3-8191-fad7e4557ea4</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>e96278a6-7190-48f3-b8bd-945fa48cfe50</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>2024-07-02 15:33:20.161000+00:00</td><td style='border: 1px solid #dddddd'>78f86067-1da9-4f8d-a633-b0ab09dbe020</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: First 10 records were displayed.\n"
     ]
    }
   ],
   "source": [
    "wos_client.monitor_instances.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9b13ca29-fcec-47a1-92e1-3c88c8e0f3f7"
   },
   "source": [
    "## Patch the integration reference in the pre-prod subscription"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "id": "fc4d655a-b7f4-4a93-9f59-84af44740626"
   },
   "outputs": [],
   "source": [
    "payload = [\n",
    "  {\n",
    "    \"op\": \"add\",\n",
    "    \"path\": \"/integration_reference\",\n",
    "    \"value\": {\n",
    "        \"integrated_system_id\": integrated_system_id,\n",
    "        \"external_id\": openpages_model_id\n",
    "    }\n",
    "  }\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "id": "16c821db7c9740939c4a4d55f22a1097"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'metadata': {'id': '06b4e163-86b5-493d-8e28-8606fb296dca',\n",
       "  'crn': 'crn:v1:bluemix:public:aiopenscale:us-south:a/na:00000000-0000-0000-0000-000000000000:subscription:06b4e163-86b5-493d-8e28-8606fb296dca',\n",
       "  'url': '/v2/subscriptions/06b4e163-86b5-493d-8e28-8606fb296dca',\n",
       "  'created_at': '2024-08-08T00:28:25.288000Z',\n",
       "  'created_by': 'cpadmin',\n",
       "  'modified_at': '2024-08-08T01:21:27.170000Z',\n",
       "  'modified_by': 'cpadmin'},\n",
       " 'entity': {'data_mart_id': '00000000-0000-0000-0000-000000000000',\n",
       "  'service_provider_id': '33397d85-dcae-4703-907d-4668932f7df0',\n",
       "  'asset': {'asset_id': 'fe7ed7dd-2856-42d6-83ab-51c13079f78d',\n",
       "   'url': 'https://internal-nginx-svc:12443/ml/v4/models/fe7ed7dd-2856-42d6-83ab-51c13079f78d?space_id=498630dc-3e0b-4470-b44d-583f2a393c0c&version=2020-06-12',\n",
       "   'name': 'German Credit Risk Model - PreProd',\n",
       "   'asset_type': 'model',\n",
       "   'problem_type': 'binary',\n",
       "   'model_type': 'mllib_3.3',\n",
       "   'input_data_type': 'structured'},\n",
       "  'asset_properties': {'training_data_reference': {'secret_id': '96bc75a0-ac93-44ac-bcf7-f3542a017164'},\n",
       "   'training_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'modeling_role': 'target'},\n",
       "      'name': 'Risk',\n",
       "      'nullable': True,\n",
       "      'type': 'string'}]},\n",
       "   'input_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'}]},\n",
       "   'output_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 128},\n",
       "       'modeling_role': 'record-id',\n",
       "       'primary_key': True},\n",
       "      'name': 'scoring_id',\n",
       "      'nullable': False,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'record-timestamp'},\n",
       "      'name': 'scoring_timestamp',\n",
       "      'nullable': False,\n",
       "      'type': 'timestamp'},\n",
       "     {'metadata': {},\n",
       "      'name': 'deployment_id',\n",
       "      'nullable': False,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {},\n",
       "      'name': 'asset_revision',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {},\n",
       "      'name': 'CheckingStatus_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'CreditHistory_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'EmploymentDuration_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'ExistingSavings_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'ForeignWorker_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'Housing_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'InstallmentPlans_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {}, 'name': 'Job_IX', 'nullable': True, 'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'LoanPurpose_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'OthersOnLoan_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'OwnsProperty_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {}, 'name': 'Sex_IX', 'nullable': True, 'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'Telephone_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'features',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {},\n",
       "      'name': 'rawPrediction',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {'modeling_role': 'probability'},\n",
       "      'name': 'probability',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {},\n",
       "      'name': 'prediction',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'modeling_role': 'prediction'},\n",
       "      'name': 'predictedLabel',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'prediction-probability'},\n",
       "      'name': 'prediction_probability',\n",
       "      'nullable': True,\n",
       "      'type': 'double'}]},\n",
       "   'label_column': 'Risk',\n",
       "   'prediction_field': 'predictedLabel',\n",
       "   'feature_fields': ['CheckingStatus',\n",
       "    'LoanDuration',\n",
       "    'CreditHistory',\n",
       "    'LoanPurpose',\n",
       "    'LoanAmount',\n",
       "    'ExistingSavings',\n",
       "    'EmploymentDuration',\n",
       "    'InstallmentPercent',\n",
       "    'Sex',\n",
       "    'OthersOnLoan',\n",
       "    'CurrentResidenceDuration',\n",
       "    'OwnsProperty',\n",
       "    'Age',\n",
       "    'InstallmentPlans',\n",
       "    'Housing',\n",
       "    'ExistingCreditsCount',\n",
       "    'Job',\n",
       "    'Dependents',\n",
       "    'Telephone',\n",
       "    'ForeignWorker'],\n",
       "   'categorical_fields': ['CheckingStatus',\n",
       "    'CreditHistory',\n",
       "    'LoanPurpose',\n",
       "    'ExistingSavings',\n",
       "    'EmploymentDuration',\n",
       "    'Sex',\n",
       "    'OthersOnLoan',\n",
       "    'OwnsProperty',\n",
       "    'InstallmentPlans',\n",
       "    'Housing',\n",
       "    'Job',\n",
       "    'Telephone',\n",
       "    'ForeignWorker'],\n",
       "   'probability_fields': ['probability']},\n",
       "  'deployment': {'deployment_id': '42b79776-470c-4d05-9115-76c4f5fffeab',\n",
       "   'url': 'https://cpd-cpd-instance.apps.wos415nfs2672.cp.fyre.ibm.com/ml/v4/deployments/42b79776-470c-4d05-9115-76c4f5fffeab/predictions',\n",
       "   'name': 'German Credit Risk Model - PreProd',\n",
       "   'deployment_type': 'online'},\n",
       "  'integration_reference': {'integrated_system_id': 'cf7e2e39-78c6-4711-8d76-89fa08acc9a0',\n",
       "   'external_id': '5066'},\n",
       "  'status': {'state': 'active'}}}"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = wos_client.subscriptions.update(pre_prod_subscription_id, payload)\n",
    "response.result.to_dict()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "id": "b89eaaf88342447c811f624880a90fbc",
    "tags": [
     "parameters"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'metadata': {'id': 'a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7',\n",
       "  'crn': 'crn:v1:bluemix:public:aiopenscale:us-south:a/na:00000000-0000-0000-0000-000000000000:subscription:a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7',\n",
       "  'url': '/v2/subscriptions/a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7',\n",
       "  'created_at': '2024-08-08T00:28:30.628000Z',\n",
       "  'created_by': 'cpadmin',\n",
       "  'modified_at': '2024-08-08T01:21:30.687000Z',\n",
       "  'modified_by': 'cpadmin'},\n",
       " 'entity': {'data_mart_id': '00000000-0000-0000-0000-000000000000',\n",
       "  'service_provider_id': '33397d85-dcae-4703-907d-4668932f7df0',\n",
       "  'asset': {'asset_id': 'fe7ed7dd-2856-42d6-83ab-51c13079f78d',\n",
       "   'url': 'https://internal-nginx-svc:12443/ml/v4/models/fe7ed7dd-2856-42d6-83ab-51c13079f78d?space_id=498630dc-3e0b-4470-b44d-583f2a393c0c&version=2020-06-12',\n",
       "   'name': 'German Credit Risk Model - PreProd',\n",
       "   'asset_type': 'model',\n",
       "   'problem_type': 'binary',\n",
       "   'model_type': 'scikit-learn_0.23',\n",
       "   'input_data_type': 'structured'},\n",
       "  'asset_properties': {'training_data_reference': {'secret_id': '9149333f-fb01-4260-8fb2-b261fc739704'},\n",
       "   'training_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'target'},\n",
       "      'name': 'Risk',\n",
       "      'nullable': True,\n",
       "      'type': 'double'}]},\n",
       "   'input_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'}]},\n",
       "   'output_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 128},\n",
       "       'modeling_role': 'record-id',\n",
       "       'primary_key': True},\n",
       "      'name': 'scoring_id',\n",
       "      'nullable': False,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'record-timestamp'},\n",
       "      'name': 'scoring_timestamp',\n",
       "      'nullable': False,\n",
       "      'type': 'timestamp'},\n",
       "     {'metadata': {},\n",
       "      'name': 'deployment_id',\n",
       "      'nullable': False,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {},\n",
       "      'name': 'asset_revision',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {},\n",
       "      'name': 'CheckingStatus_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'CreditHistory_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'EmploymentDuration_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'ExistingSavings_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'ForeignWorker_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'Housing_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'InstallmentPlans_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {}, 'name': 'Job_IX', 'nullable': True, 'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'LoanPurpose_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'OthersOnLoan_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'OwnsProperty_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {}, 'name': 'Sex_IX', 'nullable': True, 'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'Telephone_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'features',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {},\n",
       "      'name': 'rawPrediction',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {'modeling_role': 'probability'},\n",
       "      'name': 'probability',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {'modeling_role': 'prediction'},\n",
       "      'name': 'prediction',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64}},\n",
       "      'name': 'predictedLabel',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'prediction-probability'},\n",
       "      'name': 'prediction_probability',\n",
       "      'nullable': True,\n",
       "      'type': 'double'}]},\n",
       "   'label_column': 'Risk',\n",
       "   'prediction_field': 'prediction',\n",
       "   'feature_fields': ['CheckingStatus',\n",
       "    'LoanDuration',\n",
       "    'CreditHistory',\n",
       "    'LoanPurpose',\n",
       "    'LoanAmount',\n",
       "    'ExistingSavings',\n",
       "    'EmploymentDuration',\n",
       "    'InstallmentPercent',\n",
       "    'Sex',\n",
       "    'OthersOnLoan',\n",
       "    'CurrentResidenceDuration',\n",
       "    'OwnsProperty',\n",
       "    'Age',\n",
       "    'InstallmentPlans',\n",
       "    'Housing',\n",
       "    'ExistingCreditsCount',\n",
       "    'Job',\n",
       "    'Dependents',\n",
       "    'Telephone',\n",
       "    'ForeignWorker'],\n",
       "   'categorical_fields': ['CheckingStatus',\n",
       "    'CreditHistory',\n",
       "    'LoanPurpose',\n",
       "    'ExistingSavings',\n",
       "    'EmploymentDuration',\n",
       "    'Sex',\n",
       "    'OthersOnLoan',\n",
       "    'OwnsProperty',\n",
       "    'InstallmentPlans',\n",
       "    'Housing',\n",
       "    'Job',\n",
       "    'Telephone',\n",
       "    'ForeignWorker'],\n",
       "   'probability_fields': ['probability']},\n",
       "  'deployment': {'deployment_id': '42b79776-470c-4d05-9115-76c4f5fffeab',\n",
       "   'url': 'https://cpd-cpd-instance.apps.wos415nfs2672.cp.fyre.ibm.com/ml/v4/deployments/42b79776-470c-4d05-9115-76c4f5fffeab/predictions',\n",
       "   'name': 'German Credit Risk Model - Challenger',\n",
       "   'deployment_type': 'online'},\n",
       "  'integration_reference': {'integrated_system_id': 'cf7e2e39-78c6-4711-8d76-89fa08acc9a0',\n",
       "   'external_id': '5066'},\n",
       "  'status': {'state': 'active'}}}"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = wos_client.subscriptions.update(challenger_subscription_id, payload)\n",
    "response.result.to_dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6a050eb6-b070-4789-af86-0602cb1af95a"
   },
   "source": [
    "## Create test data sets from the training data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "id": "76a4364f-caf8-46fc-8b21-b5c8d99f71d4"
   },
   "outputs": [],
   "source": [
    "test_data_1 = pd_data[1:201]\n",
    "test_data_1.to_csv(\"german_credit_risk_test_data_1.csv\", encoding=\"utf-8\", index=False)\n",
    "test_data_2 = pd_data[201:401]\n",
    "test_data_2.to_csv(\"german_credit_risk_test_data_2.csv\", encoding=\"utf-8\", index=False)\n",
    "test_data_3 = pd_data[401:601]\n",
    "test_data_3.to_csv(\"german_credit_risk_test_data_3.csv\", encoding=\"utf-8\", index=False)\n",
    "test_data_4 = pd_data[601:801]\n",
    "test_data_4.to_csv(\"german_credit_risk_test_data_4.csv\", encoding=\"utf-8\", index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9a859d02-9000-4640-875e-75e21f631919"
   },
   "source": [
    "## Function to upload, evaluate and check the status of the evaluation\n",
    "\n",
    "This function will upload the test data CSV and trigger the risk evaluation. It will iterate and check the status of the evaluation until its finished with a finite wait duration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "id": "bfccd770-08e0-492e-87c2-b3b0cc73ad58"
   },
   "outputs": [],
   "source": [
    "def upload_and_evaluate(file_name, mrm_instance_id):\n",
    "    \n",
    "    print(\"Running upload and evaluate for {}\".format(file_name))\n",
    "    import json\n",
    "    import time\n",
    "    from datetime import datetime\n",
    "\n",
    "    status = None\n",
    "    monitoring_run_id = None\n",
    "    GET_UPLOAD_AND_EVALUATION_STATUS_RETRIES = 32\n",
    "    GET_UPLOAD_AND_EVALUATION_STATUS_INTERVAL = 10\n",
    "    \n",
    "    if file_name is not None:\n",
    "        \n",
    "        response = wos_client.monitor_instances.mrm.evaluate_risk(\n",
    "                        monitor_instance_id = mrm_instance_id,\n",
    "                        test_data_set_name = file_name,\n",
    "                        feedback_data_path = file_name,\n",
    "                        test_data_path=file_name)\n",
    "    \n",
    "        if response.status_code != 202:\n",
    "            print(\"Upload and evalaute for {0} failed with error : {1}\".format(file_name, response.status_code))\n",
    "            return\n",
    "        \n",
    "        for i in range(GET_UPLOAD_AND_EVALUATION_STATUS_RETRIES):\n",
    "        \n",
    "            response = wos_client.monitor_instances.mrm.get_risk_evaluation(\n",
    "                            monitor_instance_id = mrm_instance_id)\n",
    "            \n",
    "            if response.status_code != 200:\n",
    "                print(\"Getting status of upload and evalaute for {0} failed with error : {1}\".format(file_name, response.status_code))\n",
    "                return\n",
    "\n",
    "            response = response.result._to_dict()\n",
    "\n",
    "            if \"metadata\" in response and \"id\" in response[\"metadata\"]:\n",
    "                monitoring_run_id = response[\"metadata\"][\"id\"]\n",
    "            if \"entity\" in response and \"status\" in response[\"entity\"]:\n",
    "                status = response[\"entity\"][\"status\"][\"state\"]\n",
    "            \n",
    "            if status is not None:\n",
    "                print(datetime.utcnow().strftime('%H:%M:%S'), status.lower())\n",
    "                if status.lower() in [\"finished\", \"completed\"]:\n",
    "                    break\n",
    "                elif \"error\" in status.lower():\n",
    "                    print(response)\n",
    "                    break\n",
    "\n",
    "            time.sleep(GET_UPLOAD_AND_EVALUATION_STATUS_INTERVAL)\n",
    "\n",
    "    return status, monitoring_run_id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "85dfeacb-52c8-4380-bfe7-08d6d28eff56"
   },
   "source": [
    "## Perform Risk Evaluations\n",
    "\n",
    "We now start performing evaluations of smaller data sets against both the PreProd and Challenger subscriptions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "id": "cc2fe2df-19f4-41f0-9a80-e746e397ed8a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running upload and evaluate for german_credit_risk_test_data_1.csv\n",
      "00:41:09 upload_in_progress\n",
      "00:41:20 upload_in_progress\n",
      "00:41:31 running\n",
      "00:41:42 running\n",
      "00:41:56 running\n",
      "00:42:07 running\n",
      "00:42:18 running\n",
      "00:42:29 running\n",
      "00:42:40 finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "('finished', 'b97e7700-22b1-4d77-978b-4afa24d49abd')"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "upload_and_evaluate(\"german_credit_risk_test_data_1.csv\", pre_prod_mrm_instance_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "id": "71cefcf8-2c60-45a9-87ef-d23279fc06b3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running upload and evaluate for german_credit_risk_test_data_2.csv\n",
      "00:42:47 upload_in_progress\n",
      "00:42:58 running\n",
      "00:43:09 running\n",
      "00:43:20 running\n",
      "00:43:31 finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "('finished', '1513f272-2cb7-4d62-bd0b-098a3106dcab')"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "upload_and_evaluate(\"german_credit_risk_test_data_2.csv\", pre_prod_mrm_instance_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "id": "5cbc1cb3-08cb-4b04-923a-3ebaec929438"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running upload and evaluate for german_credit_risk_test_data_3.csv\n",
      "00:43:34 upload_in_progress\n",
      "00:43:45 running\n",
      "00:43:56 running\n",
      "00:44:07 running\n",
      "00:44:18 finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "('finished', 'cb1905a7-4103-4116-8507-874bdc3f094f')"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "upload_and_evaluate(\"german_credit_risk_test_data_3.csv\", pre_prod_mrm_instance_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "id": "7e4d0ee0-4452-49b8-a7f1-cce3ac41cb8f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running upload and evaluate for german_credit_risk_test_data_4.csv\n",
      "00:44:21 upload_in_progress\n",
      "00:44:32 running\n",
      "00:44:43 running\n",
      "00:44:54 running\n",
      "00:45:05 finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "('finished', '55c28302-3bbf-4a0e-a106-2f632d9dad26')"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "upload_and_evaluate(\"german_credit_risk_test_data_4.csv\", pre_prod_mrm_instance_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "id": "6c44912b-05c9-4e3b-ad3f-f56bb982f88d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running upload and evaluate for german_credit_risk_test_data_1.csv\n",
      "00:45:08 upload_in_progress\n",
      "00:45:19 upload_in_progress\n",
      "00:45:30 finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "('finished', 'ab26bdb4-eb7f-4d0d-ad7d-6a99e43dc261')"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "upload_and_evaluate(\"german_credit_risk_test_data_1.csv\", challenger_mrm_instance_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "id": "217941be-48cb-4be7-8b35-3ed2f9bf6f01"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running upload and evaluate for german_credit_risk_test_data_2.csv\n",
      "00:45:33 upload_in_progress\n",
      "00:45:44 finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "('finished', '06f41e46-200b-40b5-8ccb-2a149ff17b0c')"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "upload_and_evaluate(\"german_credit_risk_test_data_2.csv\", challenger_mrm_instance_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "id": "fa2f44a1-815a-428f-96ab-180f255eda64"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running upload and evaluate for german_credit_risk_test_data_3.csv\n",
      "00:45:47 upload_in_progress\n",
      "00:45:58 upload_in_progress\n",
      "00:46:09 finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "('finished', '533e43b7-e921-4cb3-abc4-913b03936214')"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "upload_and_evaluate(\"german_credit_risk_test_data_3.csv\", challenger_mrm_instance_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "id": "2e03dd93-dfa2-4095-a1d8-0eb127cceeb6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running upload and evaluate for german_credit_risk_test_data_4.csv\n",
      "00:46:12 upload_in_progress\n",
      "00:46:23 upload_in_progress\n",
      "00:46:34 finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "('finished', 'cab790a9-8c34-4d6c-9cf8-08b8adfd8cbd')"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "upload_and_evaluate(\"german_credit_risk_test_data_4.csv\", challenger_mrm_instance_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "426f03ef-de4a-4bc4-9756-744349f5b660"
   },
   "source": [
    "## Explore the Model Risk Management UI\n",
    "\n",
    "Here is a quick recap of what we have done so far.\n",
    "\n",
    "1. We've deployed two Credit Risk Model to a WML instance that is designated as Pre-Production\n",
    "2. We've created subscriptions of these two model deployments in OpenScale\n",
    "3. Configured all monitors supported by OpenScale for these subscriptions\n",
    "4. We've performed a few risk evaluations against both these susbscription with the same set of test data\n",
    "\n",
    "Now, please explore the Model Risk Management UI to visualize the results, compare the performance of models, download the evaluation report as PDF. For more information, refer to the Beta Guide section \"Work in Watson OpenScale.\"\n",
    "\n",
    "Link to OpenScale : https://CLUSTER_URL/aiopenscale/insights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "215eb279-37e0-4834-b34c-846b33c2700e"
   },
   "source": [
    "# Promote pre-production model to production \n",
    "\n",
    "After you have reviewed the evaluation results of the PreProd Vs Challenger and if you make the decision to promote the Challenger model to Production, the first thing you need to do is to deploy the model into a WML instance that is designated as Production instance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "98667af1-fea5-4a7f-a547-179582280646"
   },
   "source": [
    "## Deploy model to production WML instance "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "id": "e4081ad4-010c-478e-b5b2-f762da9869c0"
   },
   "outputs": [],
   "source": [
    "PROD_MODEL_NAME=\"German Credit Risk Model - Prod\"\n",
    "PROD_DEPLOYMENT_NAME=\"German Credit Risk Model - Prod\"\n",
    "\n",
    "PROD_SPACE_NAME=\"prod\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "id": "4d07ab26-cf94-41f1-b5bb-c8272a5e6fe7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: 'limit' is not provided. Only first 50 records will be displayed if the number of records exceed 50\n",
      "------------------------------------  -----------------------------------------------------------------------  ------------------------\n",
      "ID                                    NAME                                                                     CREATED\n",
      "498630dc-3e0b-4470-b44d-583f2a393c0c  wml_preprod_space                                                        2024-08-08T00:19:18.705Z\n",
      "00969846-29b6-446a-959e-587df21992e0  prod                                                                     2024-08-07T06:33:55.297Z\n",
      "06873ce9-baf3-46a6-86c4-54c1e3d84429  tutorial_space                                                           2024-08-06T13:29:18.489Z\n",
      "e8b87647-a0e9-4932-920f-1e8f1d1f383d  tutorial-space                                                           2024-08-06T12:55:07.962Z\n",
      "832d6bbe-a2ea-451f-8d94-853da7c52b00  FrauDScore                                                               2024-08-05T07:48:21.490Z\n",
      "8d3a3e65-4875-4a5e-a6bd-617d2deeb9ac  Neelima                                                                  2024-08-05T05:42:10.490Z\n",
      "3a051b57-d5f6-4290-a2ac-5f66a6ed6d65  openscale-express-path-preprod-00000000-0000-0000-0000-1722503677398532  2024-08-01T09:56:04.456Z\n",
      "3d10da0f-497d-43d6-a8cd-71d15b833511  openscale-express-path-00000000-0000-0000-0000-1722503677398532          2024-08-01T09:55:43.160Z\n",
      "45683c18-10ea-4b25-8888-cf3c12646b8d  hk_pre_prod                                                              2024-07-27T12:34:07.223Z\n",
      "7f4afba9-1a6b-4f27-9a0a-507182adf269  goSales deployment space                                                 2024-07-09T14:42:26.982Z\n",
      "47dab2f7-5fe2-49bb-bf69-2cad9c7c5aa8  Notebook demo space                                                      2024-07-09T07:44:12.138Z\n",
      "a4bc50f8-8acf-47b5-b82d-103f311a5fa4  GCR Auto AI prod space                                                   2024-07-04T06:40:01.094Z\n",
      "4a0ad576-aad5-43ee-9457-11b4d2f39cf6  GCR AutoAI space Demo                                                    2024-07-03T14:26:17.440Z\n",
      "a8233d23-2f8b-40fc-b9a2-45aec8b53b83  openscale-express-path-preprod-00000000-0000-0000-0000-1719914010257861  2024-07-02T09:59:03.652Z\n",
      "3e3e2ae1-23d9-468d-abff-02c127d052ef  openscale-express-path-00000000-0000-0000-0000-1719914010257861          2024-07-02T09:58:32.157Z\n",
      "dcb2cb00-cb6f-47fe-8543-cffd5df0786a  notebooks CPD                                                            2024-07-01T17:10:55.025Z\n",
      "c3905865-4e73-43d1-a18f-24cf3a8b9163  wml_preprod                                                              2024-06-29T17:49:53.207Z\n",
      "7e5a8be6-9103-4c22-9c43-b66f3d8364de  poojitha_notebooks_space                                                 2024-06-29T13:53:46.645Z\n",
      "16ccd855-46bd-43ed-8219-5f00ac565d08  shreya-space                                                             2024-06-26T04:29:17.302Z\n",
      "bc3b9797-c509-4fb4-a424-f67b1e2ed4be  QUALITY_WMLV4_PREPROD                                                    2024-06-23T12:23:04.790Z\n",
      "e396e187-2977-47b4-ade3-1539f9f10adc  QUALITY_WMLV4_PROD                                                       2024-06-23T12:22:54.422Z\n",
      "40c4d032-0339-4da6-bfec-4bdb096c9650  shreya                                                                   2024-06-20T10:54:20.088Z\n",
      "088c142e-f35e-4e48-a30c-ad55a6edeecc  notebooks 5.0                                                            2024-06-13T04:42:07.336Z\n",
      "b9b3d3b4-6e26-4e16-807d-e8bf5e7d6984  MRM_WMLV4_PREPROD                                                        2024-06-12T15:49:26.571Z\n",
      "d22e2b6b-917c-4427-a40c-1a439352a742  MRM_WMLV4_PROD                                                           2024-06-12T15:49:16.185Z\n",
      "ce15e0f6-be30-4349-af47-35ae15983bf1  openscale-express-path-preprod-00000000-0000-0000-0000-000000000000      2024-06-04T05:18:51.988Z\n",
      "6264dc0e-087a-4dea-bcbc-6bd872b510fb  openscale-express-path-00000000-0000-0000-0000-000000000000              2024-06-04T05:18:30.811Z\n",
      "------------------------------------  -----------------------------------------------------------------------  ------------------------\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'SUCCESS'"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wml_client.spaces.list()\n",
    "\n",
    "# Find and set the default space\n",
    "space_name=PROD_SPACE_NAME\n",
    "spaces = wml_client.spaces.get_details()['resources']\n",
    "space_id = None\n",
    "for space in spaces:\n",
    "    if space['entity']['name'] == space_name:\n",
    "        space_id = space[\"metadata\"][\"id\"]\n",
    "if space_id is None:\n",
    "    space_id = wml_client.spaces.store(\n",
    "        meta_props={wml_client.spaces.ConfigurationMetaNames.NAME: space_name})[\"metadata\"][\"id\"]\n",
    "wml_client.set.default_space(space_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "id": "24dc7643-4d5e-4003-ad6c-4b3ed347f156"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of records for training: 4485\n",
      "Number of records for evaluation: 515\n",
      "Accuracy = 0.726321\n",
      "Deleting deployment id 2a85d08a-ea0b-4778-a76d-49099810f8dc\n",
      "Deleting model id 38f4cc7c-7231-4703-95bc-af7601819ddd\n",
      "--  ----  -------  ----  ----------  ----------------\n",
      "ID  NAME  CREATED  TYPE  SPEC_STATE  SPEC_REPLACEMENT\n",
      "--  ----  -------  ----  ----------  ----------------\n",
      "Note: Warnings!! :  Software specification spark-mllib_3.3 specified for the wml_model is deprecated and will be removed in the future. We recommend you use spark-mllib_3.4 instead. For details see Supported Frameworks https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=specifications-supported-deployment-frameworks\n",
      "{'entity': {'hybrid_pipeline_software_specs': [], 'label_column': 'Risk', 'pipeline': {'id': 'fc900e5c-12c5-4125-ab94-0f99230141ea'}, 'schemas': {'input': [{'fields': [{'metadata': {}, 'name': 'CheckingStatus', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanDuration', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'CreditHistory', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanPurpose', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanAmount', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'ExistingSavings', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'EmploymentDuration', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'InstallmentPercent', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Sex', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'OthersOnLoan', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'CurrentResidenceDuration', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'OwnsProperty', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Age', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'InstallmentPlans', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Housing', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'ExistingCreditsCount', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Job', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Dependents', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Telephone', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'ForeignWorker', 'nullable': True, 'type': 'string'}], 'id': '1', 'type': 'struct'}], 'output': []}, 'software_spec': {'id': 'd11f2434-4fc7-58b7-8a62-755da64fdaf8', 'name': 'spark-mllib_3.3'}, 'training_data_references': [{'id': '1', 'location': {}, 'schema': {'fields': [{'metadata': {}, 'name': 'CheckingStatus', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanDuration', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'CreditHistory', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanPurpose', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'LoanAmount', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'ExistingSavings', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'EmploymentDuration', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'InstallmentPercent', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Sex', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'OthersOnLoan', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'CurrentResidenceDuration', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'OwnsProperty', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Age', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'InstallmentPlans', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Housing', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'ExistingCreditsCount', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Job', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'Dependents', 'nullable': True, 'type': 'integer'}, {'metadata': {}, 'name': 'Telephone', 'nullable': True, 'type': 'string'}, {'metadata': {}, 'name': 'ForeignWorker', 'nullable': True, 'type': 'string'}, {'metadata': {'modeling_role': 'target'}, 'name': 'Risk', 'nullable': True, 'type': 'string'}], 'id': '1', 'type': 'struct'}, 'type': 'fs'}], 'type': 'mllib_3.3'}, 'metadata': {'created_at': '2024-08-08T00:47:00.843Z', 'description': 'German Credit Risk Model - Prod', 'id': '8342537b-4586-4a4b-94b5-c218bbdca9d9', 'modified_at': '2024-08-08T00:47:09.011Z', 'name': 'German Credit Risk Model - Prod', 'owner': '1000331001', 'resource_key': 'd1e3cd06-ac78-49ae-a7a6-eb08f9ed40b6', 'space_id': '00969846-29b6-446a-959e-587df21992e0'}, 'system': {'warnings': [{'message': 'Software specification spark-mllib_3.3 specified for the wml_model is deprecated and will be removed in the future. We recommend you use spark-mllib_3.4 instead. For details see Supported Frameworks https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=specifications-supported-deployment-frameworks'}]}}\n",
      "------------------------------------  -------------------------------  ------------------------  ---------  ----------  ----------------\n",
      "ID                                    NAME                             CREATED                   TYPE       SPEC_STATE  SPEC_REPLACEMENT\n",
      "8342537b-4586-4a4b-94b5-c218bbdca9d9  German Credit Risk Model - Prod  2024-08-08T00:47:00.002Z  mllib_3.3  deprecated  spark-mllib_3.4\n",
      "------------------------------------  -------------------------------  ------------------------  ---------  ----------  ----------------\n",
      "Deploying model...\n",
      "\n",
      "\n",
      "#######################################################################################\n",
      "\n",
      "Synchronous deployment creation for uid: '8342537b-4586-4a4b-94b5-c218bbdca9d9' started\n",
      "\n",
      "#######################################################################################\n",
      "\n",
      "\n",
      "initializing\n",
      "Note: Software specification spark-mllib_3.3 is deprecated. Use spark-mllib_3.4 software specification instead when saving a spark model. For details, see https://www.ibm.com/support/producthub/icpdata/docs/content/SSQNUZ_latest/wsj/wmls/wmls-deploy-python-types.html.\n",
      "\n",
      "ready\n",
      "\n",
      "\n",
      "------------------------------------------------------------------------------------------------\n",
      "Successfully finished deployment creation, deployment_uid='17fcc5ed-2ec1-4bba-a7d2-f0bf3a005cd7'\n",
      "------------------------------------------------------------------------------------------------\n",
      "\n",
      "\n",
      "Model id: 8342537b-4586-4a4b-94b5-c218bbdca9d9\n",
      "Deployment id: 17fcc5ed-2ec1-4bba-a7d2-f0bf3a005cd7\n",
      "Single record scoring result: \n",
      " fields: ['CheckingStatus', 'LoanDuration', 'CreditHistory', 'LoanPurpose', 'LoanAmount', 'ExistingSavings', 'EmploymentDuration', 'InstallmentPercent', 'Sex', 'OthersOnLoan', 'CurrentResidenceDuration', 'OwnsProperty', 'Age', 'InstallmentPlans', 'Housing', 'ExistingCreditsCount', 'Job', 'Dependents', 'Telephone', 'ForeignWorker', 'CheckingStatus_IX', 'CreditHistory_IX', 'EmploymentDuration_IX', 'ExistingSavings_IX', 'ForeignWorker_IX', 'Housing_IX', 'InstallmentPlans_IX', 'Job_IX', 'LoanPurpose_IX', 'OthersOnLoan_IX', 'OwnsProperty_IX', 'Sex_IX', 'Telephone_IX', 'features', 'rawPrediction', 'probability', 'prediction', 'predictedLabel'] \n",
      " values:  ['no_checking', 13, 'credits_paid_to_date', 'car_new', 1343, '100_to_500', '1_to_4', 2, 'female', 'none', 3, 'savings_insurance', 46, 'none', 'own', 2, 'skilled', 1, 'none', 'yes', 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, [21, [1, 3, 5, 13, 14, 15, 16, 17, 18, 19, 20], [1.0, 1.0, 1.0, 13.0, 1343.0, 2.0, 3.0, 13.0, 46.0, 2.0, 1.0]], [0.37923501094948475, -0.37923501094948475], [0.6810214668555962, 0.31897853314440383], 0.0, 'No Risk']\n"
     ]
    }
   ],
   "source": [
    "import numpy \n",
    "numpy.version.version\n",
    "\n",
    "import pandas as pd\n",
    "import json\n",
    "\n",
    "from pyspark import SparkContext, SQLContext\n",
    "from pyspark.ml import Pipeline\n",
    "from pyspark.ml.classification import RandomForestClassifier,GBTClassifier\n",
    "from pyspark.ml.evaluation import BinaryClassificationEvaluator\n",
    "from pyspark.ml.feature import StringIndexer, VectorAssembler, IndexToString\n",
    "from pyspark.sql.types import StructType, DoubleType, StringType, ArrayType\n",
    "\n",
    "from pyspark.sql import SparkSession\n",
    "from pyspark import SparkFiles\n",
    "\n",
    "spark = SparkSession.builder.getOrCreate()\n",
    "pd_data = pd.read_csv(\"german_credit_data_biased_training.csv\", sep=\",\", header=0)\n",
    "spark_df = spark.read.csv(path=\"german_credit_data_biased_training.csv\", sep=\",\", header=True, inferSchema=True)\n",
    "spark_df.head()\n",
    "\n",
    "(train_data, test_data) = spark_df.randomSplit([0.9, 0.1], 24)\n",
    "print(\"Number of records for training: \" + str(train_data.count()))\n",
    "print(\"Number of records for evaluation: \" + str(test_data.count()))\n",
    "\n",
    "si_CheckingStatus = StringIndexer(inputCol='CheckingStatus', outputCol='CheckingStatus_IX')\n",
    "si_CreditHistory = StringIndexer(inputCol='CreditHistory', outputCol='CreditHistory_IX')\n",
    "si_LoanPurpose = StringIndexer(inputCol='LoanPurpose', outputCol='LoanPurpose_IX')\n",
    "si_ExistingSavings = StringIndexer(inputCol='ExistingSavings', outputCol='ExistingSavings_IX')\n",
    "si_EmploymentDuration = StringIndexer(inputCol='EmploymentDuration', outputCol='EmploymentDuration_IX')\n",
    "si_Sex = StringIndexer(inputCol='Sex', outputCol='Sex_IX')\n",
    "si_OthersOnLoan = StringIndexer(inputCol='OthersOnLoan', outputCol='OthersOnLoan_IX')\n",
    "si_OwnsProperty = StringIndexer(inputCol='OwnsProperty', outputCol='OwnsProperty_IX')\n",
    "si_InstallmentPlans = StringIndexer(inputCol='InstallmentPlans', outputCol='InstallmentPlans_IX')\n",
    "si_Housing = StringIndexer(inputCol='Housing', outputCol='Housing_IX')\n",
    "si_Job = StringIndexer(inputCol='Job', outputCol='Job_IX')\n",
    "si_Telephone = StringIndexer(inputCol='Telephone', outputCol='Telephone_IX')\n",
    "si_ForeignWorker = StringIndexer(inputCol='ForeignWorker', outputCol='ForeignWorker_IX')\n",
    "si_Label = StringIndexer(inputCol=\"Risk\", outputCol=\"label\").fit(spark_df)\n",
    "label_converter = IndexToString(inputCol=\"prediction\", outputCol=\"predictedLabel\", labels=si_Label.labels)\n",
    "\n",
    "va_features = VectorAssembler(\n",
    "inputCols=[\"CheckingStatus_IX\", \"CreditHistory_IX\", \"LoanPurpose_IX\", \"ExistingSavings_IX\",\n",
    "           \"EmploymentDuration_IX\", \"Sex_IX\", \"OthersOnLoan_IX\", \"OwnsProperty_IX\", \"InstallmentPlans_IX\",\n",
    "           \"Housing_IX\", \"Job_IX\", \"Telephone_IX\", \"ForeignWorker_IX\", \"LoanDuration\", \"LoanAmount\",\n",
    "           \"InstallmentPercent\", \"CurrentResidenceDuration\", \"LoanDuration\", \"Age\", \"ExistingCreditsCount\",\n",
    "           \"Dependents\"], outputCol=\"features\")\n",
    "\n",
    "classifier=GBTClassifier(featuresCol=\"features\")\n",
    "\n",
    "pipeline = Pipeline(\n",
    "stages=[si_CheckingStatus, si_CreditHistory, si_EmploymentDuration, si_ExistingSavings, si_ForeignWorker,\n",
    "        si_Housing, si_InstallmentPlans, si_Job, si_LoanPurpose, si_OthersOnLoan,\n",
    "        si_OwnsProperty, si_Sex, si_Telephone, si_Label, va_features, classifier, label_converter])\n",
    "\n",
    "model = pipeline.fit(train_data)\n",
    "predictions = model.transform(test_data)\n",
    "evaluator = BinaryClassificationEvaluator(rawPredictionCol=\"prediction\")\n",
    "auc = evaluator.evaluate(predictions)\n",
    "\n",
    "print(\"Accuracy = %g\" % auc)\n",
    "\n",
    "# Remove existing model and deployment\n",
    "MODEL_NAME=PROD_MODEL_NAME\n",
    "DEPLOYMENT_NAME=PROD_DEPLOYMENT_NAME\n",
    "\n",
    "deployment_details = wml_client.deployments.get_details()\n",
    "for deployment in deployment_details['resources']:\n",
    "    deployment_id = wml_client.deployments.get_id(deployment)\n",
    "    model_id = deployment['entity']['asset']['id']\n",
    "    if deployment['entity']['name'] == DEPLOYMENT_NAME:\n",
    "        print('Deleting deployment id', deployment_id)\n",
    "        wml_client.deployments.delete(deployment_id)\n",
    "        print('Deleting model id', model_id)\n",
    "        wml_client.repository.delete(model_id)\n",
    "wml_client.repository.list_models()\n",
    "\n",
    "# Save Model\n",
    "software_spec_uid = wml_client.software_specifications.get_id_by_name(\"spark-mllib_3.3\")\n",
    "model_props_rf = {\n",
    "    wml_client.repository.ModelMetaNames.NAME: MODEL_NAME,\n",
    "    wml_client.repository.ModelMetaNames.DESCRIPTION: MODEL_NAME,\n",
    "    wml_client.repository.ModelMetaNames.SOFTWARE_SPEC_UID: software_spec_uid,\n",
    "    wml_client.repository.ModelMetaNames.TYPE: 'mllib_3.3'\n",
    "}\n",
    "published_model_details = wml_client.repository.store_model(model=model, meta_props=model_props_rf, training_data=train_data, pipeline=pipeline)\n",
    "print(published_model_details)\n",
    "\n",
    "# List models in the repository\n",
    "wml_client.repository.list_models()\n",
    "\n",
    "# Get the model UID\n",
    "prod_model_uid = wml_client.repository.get_model_id(published_model_details)\n",
    "prod_model_uid\n",
    "\n",
    "\n",
    "# Deploy model\n",
    "wml_deployments = wml_client.deployments.get_details()\n",
    "prod_deployment_uid = None\n",
    "for deployment in wml_deployments['resources']:\n",
    "    if DEPLOYMENT_NAME == deployment['entity']['name']:\n",
    "        prod_deployment_uid = wml_client.deployments.get_id(deployment)\n",
    "        break\n",
    "\n",
    "if prod_deployment_uid is None:\n",
    "    print(\"Deploying model...\")\n",
    "    meta_props = {\n",
    "        wml_client.deployments.ConfigurationMetaNames.NAME: DEPLOYMENT_NAME,\n",
    "        wml_client.deployments.ConfigurationMetaNames.DESCRIPTION: DEPLOYMENT_NAME,\n",
    "        wml_client.deployments.ConfigurationMetaNames.ONLINE: {}\n",
    "    }\n",
    "    deployment = wml_client.deployments.create(artifact_uid=prod_model_uid, name=DEPLOYMENT_NAME, meta_props=meta_props)\n",
    "    prod_deployment_uid = wml_client.deployments.get_id(deployment)\n",
    "\n",
    "print(\"Model id: {}\".format(prod_model_uid))\n",
    "print(\"Deployment id: {}\".format(prod_deployment_uid))\n",
    "\n",
    "prod_deployment_uid=wml_client.deployments.get_id(deployment)\n",
    "prod_deployment_uid\n",
    "\n",
    "fields = [\"CheckingStatus\",\"LoanDuration\",\"CreditHistory\",\"LoanPurpose\",\"LoanAmount\",\"ExistingSavings\",\"EmploymentDuration\",\"InstallmentPercent\",\"Sex\",\"OthersOnLoan\",\"CurrentResidenceDuration\",\"OwnsProperty\",\"Age\",\"InstallmentPlans\",\"Housing\",\"ExistingCreditsCount\",\"Job\",\"Dependents\",\"Telephone\",\"ForeignWorker\"]\n",
    "values = [\n",
    "  [\"no_checking\",13,\"credits_paid_to_date\",\"car_new\",1343,\"100_to_500\",\"1_to_4\",2,\"female\",\"none\",3,\"savings_insurance\",46,\"none\",\"own\",2,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"no_checking\",24,\"prior_payments_delayed\",\"furniture\",4567,\"500_to_1000\",\"1_to_4\",4,\"male\",\"none\",4,\"savings_insurance\",36,\"none\",\"free\",2,\"management_self-employed\",1,\"none\",\"yes\"],\n",
    "  [\"0_to_200\",26,\"all_credits_paid_back\",\"car_new\",863,\"less_100\",\"less_1\",2,\"female\",\"co-applicant\",2,\"real_estate\",38,\"none\",\"own\",1,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"0_to_200\",14,\"no_credits\",\"car_new\",2368,\"less_100\",\"1_to_4\",3,\"female\",\"none\",3,\"real_estate\",29,\"none\",\"own\",1,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"0_to_200\",4,\"no_credits\",\"car_new\",250,\"less_100\",\"unemployed\",2,\"female\",\"none\",3,\"real_estate\",23,\"none\",\"rent\",1,\"management_self-employed\",1,\"none\",\"yes\"],\n",
    "  [\"no_checking\",17,\"credits_paid_to_date\",\"car_new\",832,\"100_to_500\",\"1_to_4\",2,\"male\",\"none\",2,\"real_estate\",42,\"none\",\"own\",1,\"skilled\",1,\"none\",\"yes\"],\n",
    "  [\"no_checking\",33,\"outstanding_credit\",\"appliances\",5696,\"unknown\",\"greater_7\",4,\"male\",\"co-applicant\",4,\"unknown\",54,\"none\",\"free\",2,\"skilled\",1,\"yes\",\"yes\"],\n",
    "  [\"0_to_200\",13,\"prior_payments_delayed\",\"retraining\",1375,\"100_to_500\",\"4_to_7\",3,\"male\",\"none\",3,\"real_estate\",37,\"none\",\"own\",2,\"management_self-employed\",1,\"none\",\"yes\"]\n",
    "]\n",
    "\n",
    "payload_scoring = {\"fields\": fields,\"values\": values}\n",
    "payload = {\n",
    "    wml_client.deployments.ScoringMetaNames.INPUT_DATA: [payload_scoring]\n",
    "}\n",
    "scoring_response = wml_client.deployments.score(prod_deployment_uid, payload)\n",
    "\n",
    "print('Single record scoring result:', '\\n fields:', scoring_response['predictions'][0]['fields'], '\\n values: ', scoring_response['predictions'][0]['values'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "id": "f6896d25dd824acb9799523b6ed6b651"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: Software specification spark-mllib_3.3 is deprecated. Use spark-mllib_3.4 software specification instead when saving a spark model. For details, see https://www.ibm.com/support/producthub/icpdata/docs/content/SSQNUZ_latest/wsj/wmls/wmls-deploy-python-types.html.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'entity': {'asset': {'id': '8342537b-4586-4a4b-94b5-c218bbdca9d9'},\n",
       "  'custom': {},\n",
       "  'deployed_asset_type': 'model',\n",
       "  'description': 'German Credit Risk Model - Prod',\n",
       "  'hardware_spec': {'id': 'e7ed1d6c-2e89-42d7-aed5-863b972c1d2b',\n",
       "   'name': 'S',\n",
       "   'num_nodes': 1},\n",
       "  'name': 'German Credit Risk Model - Prod',\n",
       "  'online': {},\n",
       "  'space_id': '00969846-29b6-446a-959e-587df21992e0',\n",
       "  'status': {'inference': [{'url': 'https://cpd-cpd-instance.apps.wos415nfs2672.cp.fyre.ibm.com/ml/v4/deployments/17fcc5ed-2ec1-4bba-a7d2-f0bf3a005cd7/predictions'}],\n",
       "   'online_url': {'url': 'https://cpd-cpd-instance.apps.wos415nfs2672.cp.fyre.ibm.com/ml/v4/deployments/17fcc5ed-2ec1-4bba-a7d2-f0bf3a005cd7/predictions'},\n",
       "   'serving_urls': ['https://cpd-cpd-instance.apps.wos415nfs2672.cp.fyre.ibm.com/ml/v4/deployments/17fcc5ed-2ec1-4bba-a7d2-f0bf3a005cd7/predictions'],\n",
       "   'state': 'ready'}},\n",
       " 'metadata': {'created_at': '2024-08-08T00:47:14.676Z',\n",
       "  'description': 'German Credit Risk Model - Prod',\n",
       "  'id': '17fcc5ed-2ec1-4bba-a7d2-f0bf3a005cd7',\n",
       "  'modified_at': '2024-08-08T00:47:14.676Z',\n",
       "  'name': 'German Credit Risk Model - Prod',\n",
       "  'owner': '1000331001',\n",
       "  'space_id': '00969846-29b6-446a-959e-587df21992e0'},\n",
       " 'system': {'warnings': [{'id': 'Deprecated',\n",
       "    'message': 'Software specification spark-mllib_3.3 is deprecated. Use spark-mllib_3.4 software specification instead when saving a spark model. For details, see https://www.ibm.com/support/producthub/icpdata/docs/content/SSQNUZ_latest/wsj/wmls/wmls-deploy-python-types.html.'},\n",
       "   {'id': 'Deprecated',\n",
       "    'message': 'online_url is deprecated and will be removed in a future release. Use serving_urls instead.'}]}}"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wml_client.deployments.get_details(prod_deployment_uid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "15003742-8f20-4747-82c9-20fdd6279d86"
   },
   "source": [
    "## Bind WML machine learning instance as Prod\n",
    "\n",
    "Watson OpenScale needs to be bound to the Watson Machine Learning instance to capture payload data into and out of the model. If a binding with name \"WML Prod\" already exists, this code will delete that binding a create a new one.\n",
    "\n",
    "**Note**: Binding with name `WML Prod` is assumed to be only created by this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "id": "164bd39becbd45d286cbc17a6b999ca0"
   },
   "outputs": [],
   "source": [
    "SERVICE_PROVIDER_NAME = 'WML Prod'\n",
    "SERVICE_PROVIDER_DESCRIPTION = 'Prod service provider created by MRM tutorial'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "id": "28366db5-b7b4-4cc8-aae1-1205bfaa15f1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Deleted existing service_provider for WML instance: f4931b54-2103-40d9-b564-9ba3b1982fdb\n"
     ]
    }
   ],
   "source": [
    "service_providers = wos_client.service_providers.list().result.service_providers\n",
    "for service_provider in service_providers:\n",
    "    service_instance_name = service_provider.entity.name\n",
    "    if service_instance_name == SERVICE_PROVIDER_NAME:\n",
    "        service_provider_id = service_provider.metadata.id\n",
    "        wos_client.service_providers.delete(service_provider_id)\n",
    "        print(\"Deleted existing service_provider for WML instance: {}\".format(service_provider_id))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "id": "d011bc288a184df987ffb5ecc782adba"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "=================================================================================\n",
      "\n",
      " Waiting for end of adding service provider 4c6d2d1f-79c9-4a5c-a8ac-e1a0fbc35ab2 \n",
      "\n",
      "=================================================================================\n",
      "\n",
      "\n",
      "\n",
      "active\n",
      "\n",
      "-----------------------------------------------\n",
      " Successfully finished adding service provider \n",
      "-----------------------------------------------\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'4c6d2d1f-79c9-4a5c-a8ac-e1a0fbc35ab2'"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "added_service_provider_result = wos_client.service_providers.add(\n",
    "        name=SERVICE_PROVIDER_NAME,\n",
    "        description=SERVICE_PROVIDER_DESCRIPTION,\n",
    "        service_type=ServiceTypes.WATSON_MACHINE_LEARNING,\n",
    "        deployment_space_id = space_id,\n",
    "        operational_space_id = \"production\",\n",
    "        credentials=WMLCredentialsCP4D(),\n",
    "        background_mode=False\n",
    "    ).result\n",
    "service_provider_id = added_service_provider_result.metadata.id\n",
    "service_provider_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "id": "d351eda7-2aed-48b3-95ce-2763e6c71688"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'metadata': {'id': '4c6d2d1f-79c9-4a5c-a8ac-e1a0fbc35ab2',\n",
       "  'crn': 'crn:v1:bluemix:public:aiopenscale:us-south:a/na:00000000-0000-0000-0000-000000000000:service_provider:4c6d2d1f-79c9-4a5c-a8ac-e1a0fbc35ab2',\n",
       "  'url': '/v2/service_providers/4c6d2d1f-79c9-4a5c-a8ac-e1a0fbc35ab2',\n",
       "  'created_at': '2024-08-08T00:47:26.285000Z',\n",
       "  'created_by': 'cpadmin'},\n",
       " 'entity': {'name': 'WML Prod',\n",
       "  'service_type': 'watson_machine_learning',\n",
       "  'instance_id': '99999999-9999-9999-9999-999999999999',\n",
       "  'credentials': {'secret_id': 'aac7b217-fea3-4a80-b6ed-027e72e96f24'},\n",
       "  'operational_space_id': 'production',\n",
       "  'deployment_space_id': '00969846-29b6-446a-959e-587df21992e0',\n",
       "  'status': {'state': 'active'}}}"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "added_service_provider_result.to_dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "37ec5e99-a8d0-43c6-8c3d-7570023a71cf"
   },
   "source": [
    "## Remove existing prod subscription\n",
    "\n",
    "This code removes previous subscription that matches the name `German Credit Risk Model - Prod` as it is expected this subscription is created only via this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "id": "c77139ad-8952-42fe-bfcc-c91bc76438bf"
   },
   "outputs": [],
   "source": [
    "subscriptions = wos_client.subscriptions.list().result.subscriptions\n",
    "for subscription in subscriptions:\n",
    "    sub_name = subscription.entity.asset.name\n",
    "    if sub_name == PROD_MODEL_NAME:\n",
    "        wos_client.subscriptions.delete(subscription.metadata.id)\n",
    "        print('Deleted existing subscription for', sub_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1be0d17e-11eb-4a06-b376-a1a03827ea7a"
   },
   "source": [
    "# Import configuration settings from pre-prod model\n",
    "\n",
    "With MRM we provide a important feature that lets you copy the configuration settings of your pre-production subscription to the production subscription. To try this out\n",
    "\n",
    "1. Navigate to Model Monitors view in Insights dashboard of OpenScale\n",
    "2. Click on the Add to dashboard\n",
    "3. Select the production model deployment from WML production machine learning provider and click on Configure\n",
    "4. In Selections saved dialog, click on Configure monitors\n",
    "4. In Model Governance, click on Begin and choose the same OpenPages model that was linked to the pre-production model and click on Save\n",
    "5. You will be presented with an dialog about the OpenPages model being associated with another subscription and if you'd like to import settings.\n",
    "6. Click on Import and confirm\n",
    "\n",
    "All the configuration settings are now copied into the production subscription\n",
    "\n",
    "\n",
    "<b>Note: The next set of cells should be executed only after finishing the import settings from the OpenScale dashboard</b>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bf9715aa-cbca-4e41-9078-5d3a9525ebb7"
   },
   "source": [
    "## Score the production model so that we can trigger monitors\n",
    "\n",
    "Now that the production subscription is configured by copying the configuration, there would be schedules created for each of the monitors to run on a scheduled basis. \n",
    "Quality, Fairness and Mrm will run hourly. Drift will run once in three hours.\n",
    "\n",
    "For this demo purpose, we will trigger the monitors on-demand so that we can see the model summary dashboard without having to wait the entire hour. \n",
    "To do that lets first push some records in the Payload Logging table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "id": "041e607a-563a-47df-9316-c8898c4d56bd"
   },
   "outputs": [],
   "source": [
    "df = pd_data.sample(n=400)\n",
    "df = df.drop(['Risk'], axis=1)\n",
    "fields = df.columns.tolist()\n",
    "values = df.values.tolist()\n",
    "\n",
    "payload_scoring = {\"fields\": fields,\"values\": values}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "id": "e22a78d1-4d47-4327-b862-f8c113dc561c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Single record scoring result: \n",
      " fields: ['CheckingStatus', 'LoanDuration', 'CreditHistory', 'LoanPurpose', 'LoanAmount', 'ExistingSavings', 'EmploymentDuration', 'InstallmentPercent', 'Sex', 'OthersOnLoan', 'CurrentResidenceDuration', 'OwnsProperty', 'Age', 'InstallmentPlans', 'Housing', 'ExistingCreditsCount', 'Job', 'Dependents', 'Telephone', 'ForeignWorker', 'CheckingStatus_IX', 'CreditHistory_IX', 'EmploymentDuration_IX', 'ExistingSavings_IX', 'ForeignWorker_IX', 'Housing_IX', 'InstallmentPlans_IX', 'Job_IX', 'LoanPurpose_IX', 'OthersOnLoan_IX', 'OwnsProperty_IX', 'Sex_IX', 'Telephone_IX', 'features', 'rawPrediction', 'probability', 'prediction', 'predictedLabel'] \n",
      " values:  ['0_to_200', 21, 'credits_paid_to_date', 'furniture', 2664, 'less_100', '1_to_4', 2, 'male', 'none', 3, 'car_other', 40, 'none', 'rent', 1, 'skilled', 1, 'none', 'yes', 2.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, [2.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 21.0, 2664.0, 2.0, 3.0, 21.0, 40.0, 1.0, 1.0], [0.2083275028547925, -0.2083275028547925], [0.6026825456922584, 0.39731745430774157], 0.0, 'No Risk']\n"
     ]
    }
   ],
   "source": [
    "payload = {\n",
    "    wml_client.deployments.ScoringMetaNames.INPUT_DATA: [payload_scoring]\n",
    "}\n",
    "scoring_response = wml_client.deployments.score(prod_deployment_uid, payload)\n",
    "\n",
    "print('Single record scoring result:', '\\n fields:', scoring_response['predictions'][0]['fields'], '\\n values: ', scoring_response['predictions'][0]['values'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "id": "75cd2a27-8dc7-4f2b-9d99-6881c810b876"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7\n"
     ]
    }
   ],
   "source": [
    "subscriptions = wos_client.subscriptions.list().result.subscriptions\n",
    "for subscription in subscriptions:\n",
    "    sub_name = subscription.entity.asset.name\n",
    "    if sub_name == PROD_MODEL_NAME:\n",
    "        prod_subscription = subscription\n",
    "prod_subscription_id = prod_subscription.metadata.id\n",
    "print(prod_subscription_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "id": "f793e126be92449096066d686c5746b6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'metadata': {'id': 'a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7',\n",
       "  'crn': 'crn:v1:bluemix:public:aiopenscale:us-south:a/na:00000000-0000-0000-0000-000000000000:subscription:a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7',\n",
       "  'url': '/v2/subscriptions/a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7',\n",
       "  'created_at': '2024-08-08T00:28:30.628000Z',\n",
       "  'created_by': 'cpadmin',\n",
       "  'modified_at': '2024-08-08T00:28:32.262000Z',\n",
       "  'modified_by': 'cpadmin'},\n",
       " 'entity': {'data_mart_id': '00000000-0000-0000-0000-000000000000',\n",
       "  'service_provider_id': '33397d85-dcae-4703-907d-4668932f7df0',\n",
       "  'asset': {'asset_id': 'fe7ed7dd-2856-42d6-83ab-51c13079f78d',\n",
       "   'url': 'https://internal-nginx-svc:12443/ml/v4/models/fe7ed7dd-2856-42d6-83ab-51c13079f78d?space_id=498630dc-3e0b-4470-b44d-583f2a393c0c&version=2020-06-12',\n",
       "   'name': 'German Credit Risk Model - PreProd',\n",
       "   'asset_type': 'model',\n",
       "   'problem_type': 'binary',\n",
       "   'model_type': 'scikit-learn_0.23',\n",
       "   'input_data_type': 'structured'},\n",
       "  'asset_properties': {'training_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'target'},\n",
       "      'name': 'Risk',\n",
       "      'nullable': True,\n",
       "      'type': 'double'}]},\n",
       "   'input_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'}]},\n",
       "   'output_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 128},\n",
       "       'modeling_role': 'record-id',\n",
       "       'primary_key': True},\n",
       "      'name': 'scoring_id',\n",
       "      'nullable': False,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'record-timestamp'},\n",
       "      'name': 'scoring_timestamp',\n",
       "      'nullable': False,\n",
       "      'type': 'timestamp'},\n",
       "     {'metadata': {},\n",
       "      'name': 'deployment_id',\n",
       "      'nullable': False,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {},\n",
       "      'name': 'asset_revision',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CheckingStatus',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'CreditHistory',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'LoanPurpose',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'LoanAmount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ExistingSavings',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'EmploymentDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPercent',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Sex',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OthersOnLoan',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'CurrentResidenceDuration',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'OwnsProperty',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Age',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'InstallmentPlans',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Housing',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'ExistingCreditsCount',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Job',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'Dependents',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Telephone',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'ForeignWorker',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {},\n",
       "      'name': 'CheckingStatus_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'CreditHistory_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'EmploymentDuration_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'ExistingSavings_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'ForeignWorker_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'Housing_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'InstallmentPlans_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {}, 'name': 'Job_IX', 'nullable': True, 'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'LoanPurpose_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'OthersOnLoan_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'OwnsProperty_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {}, 'name': 'Sex_IX', 'nullable': True, 'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'Telephone_IX',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {},\n",
       "      'name': 'features',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {},\n",
       "      'name': 'rawPrediction',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {'modeling_role': 'probability'},\n",
       "      'name': 'probability',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}},\n",
       "     {'metadata': {'modeling_role': 'prediction'},\n",
       "      'name': 'prediction',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64}},\n",
       "      'name': 'predictedLabel',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'prediction-probability'},\n",
       "      'name': 'prediction_probability',\n",
       "      'nullable': True,\n",
       "      'type': 'double'}]},\n",
       "   'label_column': 'Risk',\n",
       "   'prediction_field': 'prediction',\n",
       "   'feature_fields': ['CheckingStatus',\n",
       "    'LoanDuration',\n",
       "    'CreditHistory',\n",
       "    'LoanPurpose',\n",
       "    'LoanAmount',\n",
       "    'ExistingSavings',\n",
       "    'EmploymentDuration',\n",
       "    'InstallmentPercent',\n",
       "    'Sex',\n",
       "    'OthersOnLoan',\n",
       "    'CurrentResidenceDuration',\n",
       "    'OwnsProperty',\n",
       "    'Age',\n",
       "    'InstallmentPlans',\n",
       "    'Housing',\n",
       "    'ExistingCreditsCount',\n",
       "    'Job',\n",
       "    'Dependents',\n",
       "    'Telephone',\n",
       "    'ForeignWorker'],\n",
       "   'categorical_fields': ['CheckingStatus',\n",
       "    'CreditHistory',\n",
       "    'LoanPurpose',\n",
       "    'ExistingSavings',\n",
       "    'EmploymentDuration',\n",
       "    'Sex',\n",
       "    'OthersOnLoan',\n",
       "    'OwnsProperty',\n",
       "    'InstallmentPlans',\n",
       "    'Housing',\n",
       "    'Job',\n",
       "    'Telephone',\n",
       "    'ForeignWorker'],\n",
       "   'probability_fields': ['probability']},\n",
       "  'deployment': {'deployment_id': '42b79776-470c-4d05-9115-76c4f5fffeab',\n",
       "   'url': 'https://cpd-cpd-instance.apps.wos415nfs2672.cp.fyre.ibm.com/ml/v4/deployments/42b79776-470c-4d05-9115-76c4f5fffeab/predictions',\n",
       "   'name': 'German Credit Risk Model - Challenger',\n",
       "   'deployment_type': 'online'},\n",
       "  'status': {'state': 'active'}}}"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "res = wos_client.subscriptions.get(prod_subscription_id).result.to_dict()\n",
    "res"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "id": "04f0c84a-6e72-474a-8e0b-ce4c658809a1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Payload data set id:  2f7ba47c-cff2-4897-a27e-55b8d60eaafa\n",
      "Number of records in the payload logging table: 816\n"
     ]
    }
   ],
   "source": [
    "time.sleep(10)\n",
    "prod_pl_data_set_id = None\n",
    "prod_pl_data_set_id = wos_client.data_sets.list(type=DataSetTypes.PAYLOAD_LOGGING, \n",
    "                                                target_target_id=prod_subscription_id, \n",
    "                                                target_target_type=TargetTypes.SUBSCRIPTION).result.data_sets[0].metadata.id\n",
    "if prod_pl_data_set_id is None:\n",
    "    print(\"Payload data set not found. Please check subscription status.\")\n",
    "else:\n",
    "    print(\"Payload data set id: \", prod_pl_data_set_id)\n",
    "    pl_records_count = wos_client.data_sets.get_records_count(prod_pl_data_set_id)\n",
    "    print(\"Number of records in the payload logging table: {}\".format(pl_records_count))\n",
    "    if pl_records_count == 0:\n",
    "        print(\"Automatic payload logging did not happen, performing explicit payload logging.\")\n",
    "        wos_client.data_sets.store_records(data_set_id=prod_pl_data_set_id, request_body=[PayloadRecord(\n",
    "                       scoring_id='mrm_'+str(uuid.uuid4()),\n",
    "                       request=payload_scoring,\n",
    "                       response=scoring_response['predictions'][0]\n",
    "                   )])\n",
    "        time.sleep(5)\n",
    "        pl_records_count = wos_client.data_sets.get_records_count(prod_pl_data_set_id)\n",
    "        print(\"Number of records in the payload logging table: {}\".format(pl_records_count))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bbb399a4-c647-4e78-8181-934b6646931c"
   },
   "source": [
    "## Fetch all monitor instances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "id": "0852a68b-67b3-4d14-b7b1-c06192d09923"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Quality monitor instance id - c0c3274d-dc61-4384-84e3-5d4b3b258379\n",
      "Drift monitor instance id - ce9a3bc2-bd0e-4c03-b2b1-31dbffa37a6c\n",
      "Drift v2 monitor instance id - 0f087b1e-48fc-4a25-bdfd-25f90e2b1dd4\n",
      "Fairness monitor instance id - 46eba4d5-8367-406d-9ca9-db53acd7e409\n",
      "MRM monitor instance id - 6a04baa9-baab-49e0-8864-566afa17d24f\n"
     ]
    }
   ],
   "source": [
    "monitor_instances = wos_client.monitor_instances.list(target_target_id=prod_subscription_id).result.monitor_instances\n",
    "for monitor_instance in monitor_instances:\n",
    "    monitor_def_id=monitor_instance.entity.monitor_definition_id\n",
    "    if monitor_def_id == wos_client.monitor_definitions.MONITORS.QUALITY.ID:\n",
    "        quality_monitor_instance_id = monitor_instance.metadata.id\n",
    "    if monitor_def_id == wos_client.monitor_definitions.MONITORS.DRIFT.ID:\n",
    "        drift_monitor_instance_id = monitor_instance.metadata.id\n",
    "    if monitor_def_id == wos_client.monitor_definitions.MONITORS.DRIFT_V2.ID:\n",
    "        drift_v2_monitor_instance_id = monitor_instance.metadata.id\n",
    "    if monitor_def_id == wos_client.monitor_definitions.MONITORS.FAIRNESS.ID:\n",
    "        fairness_monitor_instance_id = monitor_instance.metadata.id\n",
    "    if monitor_def_id == 'mrm':\n",
    "        mrm_monitor_instance_id = monitor_instance.metadata.id\n",
    "                    \n",
    "print(\"Quality monitor instance id - {0}\".format(quality_monitor_instance_id))\n",
    "print(\"Drift monitor instance id - {0}\".format(drift_monitor_instance_id))\n",
    "print(\"Drift v2 monitor instance id - {0}\".format(drift_v2_monitor_instance_id))\n",
    "print(\"Fairness monitor instance id - {0}\".format(fairness_monitor_instance_id))\n",
    "print(\"MRM monitor instance id - {0}\".format(mrm_monitor_instance_id))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3ea2b1bc-d181-4ec1-95ea-37e64d3e586f"
   },
   "source": [
    "## Run on-demand Quality"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f6c54f32bc9e44899f84e36b5f0b5ebd"
   },
   "source": [
    "### Feedback logging\n",
    "\n",
    "The code below downloads and stores enough feedback data to meet the minimum threshold so that OpenScale can calculate a new accuracy measurement. It then kicks off the accuracy monitor. The monitors run hourly, or can be initiated via the Python API, the REST API, or the graphical user interface.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "id": "07861f904230444a85225b0814103bfb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "64ee830d-acab-4fce-bc0a-7a6c4fa93d48\n"
     ]
    }
   ],
   "source": [
    "feedback_dataset_id = None\n",
    "feedback_dataset = wos_client.data_sets.list(type=DataSetTypes.FEEDBACK, \n",
    "                                                target_target_id=prod_subscription_id, \n",
    "                                                target_target_type=TargetTypes.SUBSCRIPTION).result\n",
    "feedback_dataset_id = feedback_dataset.data_sets[0].metadata.id\n",
    "if feedback_dataset_id is None:\n",
    "    print(\"Feedback data set not found. Please check quality monitor status.\")\n",
    "print(feedback_dataset_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "id": "394b4292b27d42a58fb27034b6170bdd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2024-08-08 06:29:32--  https://raw.githubusercontent.com/pmservice/ai-openscale-tutorials/master/assets/historical_data/german_credit_risk/wml/additional_feedback_data_v2.json\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.111.133, 185.199.109.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 50890 (50K) [text/plain]\n",
      "Saving to: ‘additional_feedback_data_v2.json’\n",
      "\n",
      "additional_feedback 100%[===================>]  49.70K  --.-KB/s    in 0.01s   \n",
      "\n",
      "2024-08-08 06:29:32 (3.58 MB/s) - ‘additional_feedback_data_v2.json’ saved [50890/50890]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!rm additional_feedback_data_v2.json\n",
    "!wget https://raw.githubusercontent.com/pmservice/ai-openscale-tutorials/master/assets/historical_data/german_credit_risk/wml/additional_feedback_data_v2.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "id": "077e9013ba414ad8b4c15bfa3413cb22"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "==========================================================================================\n",
      "\n",
      " Waiting for end of storing records with request id: 9cc5ec04-fe15-4d85-87a6-083b13796ad1 \n",
      "\n",
      "==========================================================================================\n",
      "\n",
      "\n",
      "\n",
      "active\n",
      "\n",
      "---------------------------------------\n",
      " Successfully finished storing records \n",
      "---------------------------------------\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<ibm_watson_openscale.base_classes.watson_open_scale_v2.Status at 0x14a8a74c0>"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with open('additional_feedback_data_v2.json') as feedback_file:\n",
    "    additional_feedback_data = json.load(feedback_file)\n",
    "wos_client.data_sets.store_records(feedback_dataset_id, request_body=additional_feedback_data, background_mode=False).result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "id": "10b0d41eaba14fc883405ab8e0393a5f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of records in the feedback table:  898\n"
     ]
    }
   ],
   "source": [
    "time.sleep(5)\n",
    "feedback_records_count = wos_client.data_sets.get_records_count(data_set_id=feedback_dataset_id)\n",
    "print('Number of records in the feedback table: ', feedback_records_count)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "id": "daa085fd-1821-485b-8681-d0751192d42d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "========================================================================\n",
      "\n",
      " Waiting for end of monitoring run 71790afc-05d2-4f4b-be08-f0afc85da02d \n",
      "\n",
      "========================================================================\n",
      "\n",
      "\n",
      "\n",
      "finished\n",
      "\n",
      "---------------------------\n",
      " Successfully finished run \n",
      "---------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "if quality_monitor_instance_id is not None:\n",
    "    run_details = wos_client.monitor_instances.run(monitor_instance_id=quality_monitor_instance_id, background_mode=False).result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "id": "0e25e9071bae47b6b8d96c0c3a74055c"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>c0c3274d-dc61-4384-84e3-5d4b3b258379 Monitor Runs Metrics from: 2024-08-01 06:29:51.788024  till: 2024-08-08 06:29:51.788034</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>ts</th><th style='border: 1px solid #dddddd'>id</th><th style='border: 1px solid #dddddd'>measurement_id</th><th style='border: 1px solid #dddddd'>value</th><th style='border: 1px solid #dddddd'>lower_limit</th><th style='border: 1px solid #dddddd'>upper_limit</th><th style='border: 1px solid #dddddd'>tags</th><th style='border: 1px solid #dddddd'>monitor_definition_id</th><th style='border: 1px solid #dddddd'>monitor_instance_id</th><th style='border: 1px solid #dddddd'>run_id</th><th style='border: 1px solid #dddddd'>target_type</th><th style='border: 1px solid #dddddd'>target_id</th>\n",
       "                \n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "wos_client.monitor_instances.show_metrics(monitor_instance_id=quality_monitor_instance_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "df8145d0-f3b9-4fbb-8605-7608c84731be"
   },
   "source": [
    "## Run on-demand Drift"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "id": "1d167f82-54e9-491e-959a-da4689fb8dff"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "========================================================================\n",
      "\n",
      " Waiting for end of monitoring run 52821088-b20b-4756-b67e-a28f0773db97 \n",
      "\n",
      "========================================================================\n",
      "\n",
      "\n",
      "\n",
      "finished\n",
      "\n",
      "---------------------------\n",
      " Successfully finished run \n",
      "---------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "if drift_monitor_instance_id is not None:\n",
    "    run_details = wos_client.monitor_instances.run(monitor_instance_id=drift_monitor_instance_id, background_mode=False).result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "id": "b9ab6682-e76f-4507-ae4d-0c420c352ce0"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>505f727e-fa36-473e-91a5-e7f21cbabfc7 Monitor Runs Metrics from: 2024-07-31 16:07:51.953741  till: 2024-08-07 16:07:51.953771</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>ts</th><th style='border: 1px solid #dddddd'>id</th><th style='border: 1px solid #dddddd'>measurement_id</th><th style='border: 1px solid #dddddd'>value</th><th style='border: 1px solid #dddddd'>lower_limit</th><th style='border: 1px solid #dddddd'>upper_limit</th><th style='border: 1px solid #dddddd'>tags</th><th style='border: 1px solid #dddddd'>monitor_definition_id</th><th style='border: 1px solid #dddddd'>monitor_instance_id</th><th style='border: 1px solid #dddddd'>run_id</th><th style='border: 1px solid #dddddd'>target_type</th><th style='border: 1px solid #dddddd'>target_id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>2024-08-07 09:59:46.400114+00:00</td><td style='border: 1px solid #dddddd'>data_drift_magnitude</td><td style='border: 1px solid #dddddd'>2080115b-d1d8-4bc5-aa20-56251ca634e5</td><td style='border: 1px solid #dddddd'>0.0673076923076923</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>0.1</td><td style='border: 1px solid #dddddd'>[]</td><td style='border: 1px solid #dddddd'>drift</td><td style='border: 1px solid #dddddd'>505f727e-fa36-473e-91a5-e7f21cbabfc7</td><td style='border: 1px solid #dddddd'>bdbf70f7-e301-497c-a135-c49a4dec9d0b</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>ad19393e-2078-4316-ac19-7de0ddfd7937</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "wos_client.monitor_instances.show_metrics(monitor_instance_id=drift_monitor_instance_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run on-demand Drift V2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "========================================================================\n",
      "\n",
      " Waiting for end of monitoring run 9c8ea6c0-9513-458a-a3ed-e0b04f74cbbc \n",
      "\n",
      "========================================================================\n",
      "\n",
      "\n",
      "\n",
      "finished\n",
      "\n",
      "---------------------------\n",
      " Successfully finished run \n",
      "---------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "if drift_v2_monitor_instance_id is not None:\n",
    "    run_details = wos_client.monitor_instances.run(monitor_instance_id=drift_v2_monitor_instance_id, background_mode=False).result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>ddf7db84-b9a6-4436-93c4-e386a5adb8d7 Monitor Runs Metrics from: 2024-07-30 18:38:28.557913  till: 2024-08-06 18:38:28.557941</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>ts</th><th style='border: 1px solid #dddddd'>id</th><th style='border: 1px solid #dddddd'>measurement_id</th><th style='border: 1px solid #dddddd'>value</th><th style='border: 1px solid #dddddd'>lower_limit</th><th style='border: 1px solid #dddddd'>upper_limit</th><th style='border: 1px solid #dddddd'>tags</th><th style='border: 1px solid #dddddd'>monitor_definition_id</th><th style='border: 1px solid #dddddd'>monitor_instance_id</th><th style='border: 1px solid #dddddd'>run_id</th><th style='border: 1px solid #dddddd'>target_type</th><th style='border: 1px solid #dddddd'>target_id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>2024-08-06 13:03:30.703769+00:00</td><td style='border: 1px solid #dddddd'>records_processed</td><td style='border: 1px solid #dddddd'>79b5fec3-52e9-4ccb-8d28-a48adcd894e2</td><td style='border: 1px solid #dddddd'>408.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['algorithm_used:total_variation', 'computed_on:payload', 'field_type:class', 'field_name:Class Probability for No Risk']</td><td style='border: 1px solid #dddddd'>drift_v2</td><td style='border: 1px solid #dddddd'>ddf7db84-b9a6-4436-93c4-e386a5adb8d7</td><td style='border: 1px solid #dddddd'>ed70ede4-5f54-491c-86c1-9845d7530447</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>e4098249-ffec-436c-b76d-0b2ffeec5bf4</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-06 13:03:30.703769+00:00</td><td style='border: 1px solid #dddddd'>confidence_drift_score</td><td style='border: 1px solid #dddddd'>79b5fec3-52e9-4ccb-8d28-a48adcd894e2</td><td style='border: 1px solid #dddddd'>0.1534</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>0.05</td><td style='border: 1px solid #dddddd'>['algorithm_used:total_variation', 'computed_on:payload', 'field_type:class', 'field_name:Class Probability for No Risk']</td><td style='border: 1px solid #dddddd'>drift_v2</td><td style='border: 1px solid #dddddd'>ddf7db84-b9a6-4436-93c4-e386a5adb8d7</td><td style='border: 1px solid #dddddd'>ed70ede4-5f54-491c-86c1-9845d7530447</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>e4098249-ffec-436c-b76d-0b2ffeec5bf4</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-06 13:03:30.703769+00:00</td><td style='border: 1px solid #dddddd'>records_processed</td><td style='border: 1px solid #dddddd'>79b5fec3-52e9-4ccb-8d28-a48adcd894e2</td><td style='border: 1px solid #dddddd'>408.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['algorithm_used:overlap_coefficient', 'computed_on:payload', 'field_type:class', 'field_name:Class Probability for No Risk']</td><td style='border: 1px solid #dddddd'>drift_v2</td><td style='border: 1px solid #dddddd'>ddf7db84-b9a6-4436-93c4-e386a5adb8d7</td><td style='border: 1px solid #dddddd'>ed70ede4-5f54-491c-86c1-9845d7530447</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>e4098249-ffec-436c-b76d-0b2ffeec5bf4</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-06 13:03:30.703769+00:00</td><td style='border: 1px solid #dddddd'>confidence_drift_score</td><td style='border: 1px solid #dddddd'>79b5fec3-52e9-4ccb-8d28-a48adcd894e2</td><td style='border: 1px solid #dddddd'>0.1552</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>0.05</td><td style='border: 1px solid #dddddd'>['algorithm_used:overlap_coefficient', 'computed_on:payload', 'field_type:class', 'field_name:Class Probability for No Risk']</td><td style='border: 1px solid #dddddd'>drift_v2</td><td style='border: 1px solid #dddddd'>ddf7db84-b9a6-4436-93c4-e386a5adb8d7</td><td style='border: 1px solid #dddddd'>ed70ede4-5f54-491c-86c1-9845d7530447</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>e4098249-ffec-436c-b76d-0b2ffeec5bf4</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-06 13:03:38.611350+00:00</td><td style='border: 1px solid #dddddd'>records_processed</td><td style='border: 1px solid #dddddd'>7e38094c-c3bf-418a-beb7-7033e7e8568a</td><td style='border: 1px solid #dddddd'>408.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['algorithm_used:total_variation', 'computed_on:payload', 'field_type:class', 'field_name:Class Probability for Risk']</td><td style='border: 1px solid #dddddd'>drift_v2</td><td style='border: 1px solid #dddddd'>ddf7db84-b9a6-4436-93c4-e386a5adb8d7</td><td style='border: 1px solid #dddddd'>ed70ede4-5f54-491c-86c1-9845d7530447</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>e4098249-ffec-436c-b76d-0b2ffeec5bf4</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-06 13:03:38.611350+00:00</td><td style='border: 1px solid #dddddd'>confidence_drift_score</td><td style='border: 1px solid #dddddd'>7e38094c-c3bf-418a-beb7-7033e7e8568a</td><td style='border: 1px solid #dddddd'>0.1583</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>0.05</td><td style='border: 1px solid #dddddd'>['algorithm_used:total_variation', 'computed_on:payload', 'field_type:class', 'field_name:Class Probability for Risk']</td><td style='border: 1px solid #dddddd'>drift_v2</td><td style='border: 1px solid #dddddd'>ddf7db84-b9a6-4436-93c4-e386a5adb8d7</td><td style='border: 1px solid #dddddd'>ed70ede4-5f54-491c-86c1-9845d7530447</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>e4098249-ffec-436c-b76d-0b2ffeec5bf4</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-06 13:03:38.611350+00:00</td><td style='border: 1px solid #dddddd'>records_processed</td><td style='border: 1px solid #dddddd'>7e38094c-c3bf-418a-beb7-7033e7e8568a</td><td style='border: 1px solid #dddddd'>408.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['algorithm_used:overlap_coefficient', 'computed_on:payload', 'field_type:class', 'field_name:Class Probability for Risk']</td><td style='border: 1px solid #dddddd'>drift_v2</td><td style='border: 1px solid #dddddd'>ddf7db84-b9a6-4436-93c4-e386a5adb8d7</td><td style='border: 1px solid #dddddd'>ed70ede4-5f54-491c-86c1-9845d7530447</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>e4098249-ffec-436c-b76d-0b2ffeec5bf4</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-06 13:03:38.611350+00:00</td><td style='border: 1px solid #dddddd'>confidence_drift_score</td><td style='border: 1px solid #dddddd'>7e38094c-c3bf-418a-beb7-7033e7e8568a</td><td style='border: 1px solid #dddddd'>0.1598</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>0.05</td><td style='border: 1px solid #dddddd'>['algorithm_used:overlap_coefficient', 'computed_on:payload', 'field_type:class', 'field_name:Class Probability for Risk']</td><td style='border: 1px solid #dddddd'>drift_v2</td><td style='border: 1px solid #dddddd'>ddf7db84-b9a6-4436-93c4-e386a5adb8d7</td><td style='border: 1px solid #dddddd'>ed70ede4-5f54-491c-86c1-9845d7530447</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>e4098249-ffec-436c-b76d-0b2ffeec5bf4</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-06 13:03:38.821078+00:00</td><td style='border: 1px solid #dddddd'>records_processed</td><td style='border: 1px solid #dddddd'>10619c86-1908-48ea-9301-0df6fdffad98</td><td style='border: 1px solid #dddddd'>408.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['algorithm_used:jensen_shannon', 'computed_on:payload', 'field_type:class', 'field_name:predictedLabel']</td><td style='border: 1px solid #dddddd'>drift_v2</td><td style='border: 1px solid #dddddd'>ddf7db84-b9a6-4436-93c4-e386a5adb8d7</td><td style='border: 1px solid #dddddd'>ed70ede4-5f54-491c-86c1-9845d7530447</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>e4098249-ffec-436c-b76d-0b2ffeec5bf4</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-06 13:03:38.821078+00:00</td><td style='border: 1px solid #dddddd'>prediction_drift_score</td><td style='border: 1px solid #dddddd'>10619c86-1908-48ea-9301-0df6fdffad98</td><td style='border: 1px solid #dddddd'>0.0235</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>0.05</td><td style='border: 1px solid #dddddd'>['algorithm_used:jensen_shannon', 'computed_on:payload', 'field_type:class', 'field_name:predictedLabel']</td><td style='border: 1px solid #dddddd'>drift_v2</td><td style='border: 1px solid #dddddd'>ddf7db84-b9a6-4436-93c4-e386a5adb8d7</td><td style='border: 1px solid #dddddd'>ed70ede4-5f54-491c-86c1-9845d7530447</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>e4098249-ffec-436c-b76d-0b2ffeec5bf4</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: First 10 records were displayed.\n"
     ]
    }
   ],
   "source": [
    "wos_client.monitor_instances.show_metrics(monitor_instance_id=drift_v2_monitor_instance_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3cdf7ea4-5854-4a83-8680-f1effda282d2"
   },
   "source": [
    "## Run on-demand Fairness"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "id": "1bebebf0-f118-4eab-8273-0935ef82dccc"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "========================================================================\n",
      "\n",
      " Waiting for end of monitoring run 46eba4d5-8367-406d-9ca9-db53acd7e409 \n",
      "\n",
      "========================================================================\n",
      "\n",
      "\n",
      "\n",
      "running......................\n",
      "finished\n",
      "\n",
      "---------------------------\n",
      " Successfully finished run \n",
      "---------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "if fairness_monitor_instance_id is not None:\n",
    "    run_details = wos_client.monitor_instances.run(monitor_instance_id=fairness_monitor_instance_id, background_mode=False).result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "id": "12629b95-2a2d-4aca-9227-7b798e82fcc5"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>505f727e-fa36-473e-91a5-e7f21cbabfc7 Monitor Runs Metrics from: 2024-07-31 16:07:51.953741  till: 2024-08-07 16:07:51.953771</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>ts</th><th style='border: 1px solid #dddddd'>id</th><th style='border: 1px solid #dddddd'>measurement_id</th><th style='border: 1px solid #dddddd'>value</th><th style='border: 1px solid #dddddd'>lower_limit</th><th style='border: 1px solid #dddddd'>upper_limit</th><th style='border: 1px solid #dddddd'>tags</th><th style='border: 1px solid #dddddd'>monitor_definition_id</th><th style='border: 1px solid #dddddd'>monitor_instance_id</th><th style='border: 1px solid #dddddd'>run_id</th><th style='border: 1px solid #dddddd'>target_type</th><th style='border: 1px solid #dddddd'>target_id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>2024-08-07 09:59:46.400114+00:00</td><td style='border: 1px solid #dddddd'>data_drift_magnitude</td><td style='border: 1px solid #dddddd'>2080115b-d1d8-4bc5-aa20-56251ca634e5</td><td style='border: 1px solid #dddddd'>0.0673076923076923</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>0.1</td><td style='border: 1px solid #dddddd'>[]</td><td style='border: 1px solid #dddddd'>drift</td><td style='border: 1px solid #dddddd'>505f727e-fa36-473e-91a5-e7f21cbabfc7</td><td style='border: 1px solid #dddddd'>bdbf70f7-e301-497c-a135-c49a4dec9d0b</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>ad19393e-2078-4316-ac19-7de0ddfd7937</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "wos_client.monitor_instances.show_metrics(monitor_instance_id=fairness_monitor_instance_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "40354858-c9b1-4d10-a1f7-225a7f1c5afa"
   },
   "source": [
    "## Run on-demand MRM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "id": "d9812595-255b-4046-842e-53488dbf7b31"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "========================================================================\n",
      "\n",
      " Waiting for end of monitoring run 975c4f7d-42dd-4f94-ba48-e7abba54a61e \n",
      "\n",
      "========================================================================\n",
      "\n",
      "\n",
      "\n",
      "running......................\n",
      "finished\n",
      "\n",
      "---------------------------\n",
      " Successfully finished run \n",
      "---------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "if mrm_monitor_instance_id is not None:\n",
    "    run_details = wos_client.monitor_instances.run(monitor_instance_id=mrm_monitor_instance_id, triggered_by='user', background_mode=False).result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "id": "3c3f5422-2c19-4fdc-bd80-046b5ed35833"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>6a04baa9-baab-49e0-8864-566afa17d24f Monitor Runs Metrics from: 2024-08-01 06:34:54.861930  till: 2024-08-08 06:34:54.861940</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>ts</th><th style='border: 1px solid #dddddd'>id</th><th style='border: 1px solid #dddddd'>measurement_id</th><th style='border: 1px solid #dddddd'>value</th><th style='border: 1px solid #dddddd'>lower_limit</th><th style='border: 1px solid #dddddd'>upper_limit</th><th style='border: 1px solid #dddddd'>tags</th><th style='border: 1px solid #dddddd'>monitor_definition_id</th><th style='border: 1px solid #dddddd'>monitor_instance_id</th><th style='border: 1px solid #dddddd'>run_id</th><th style='border: 1px solid #dddddd'>target_type</th><th style='border: 1px solid #dddddd'>target_id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>2024-08-08 01:02:45.254000+00:00</td><td style='border: 1px solid #dddddd'>tests_passed</td><td style='border: 1px solid #dddddd'>be483ae2-da00-4347-a204-88083fbd34fd</td><td style='border: 1px solid #dddddd'>1.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>[]</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>6a04baa9-baab-49e0-8864-566afa17d24f</td><td style='border: 1px solid #dddddd'>975c4f7d-42dd-4f94-ba48-e7abba54a61e</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-08 01:02:45.254000+00:00</td><td style='border: 1px solid #dddddd'>tests_run</td><td style='border: 1px solid #dddddd'>be483ae2-da00-4347-a204-88083fbd34fd</td><td style='border: 1px solid #dddddd'>1.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>[]</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>6a04baa9-baab-49e0-8864-566afa17d24f</td><td style='border: 1px solid #dddddd'>975c4f7d-42dd-4f94-ba48-e7abba54a61e</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-08 01:02:45.254000+00:00</td><td style='border: 1px solid #dddddd'>tests_skipped</td><td style='border: 1px solid #dddddd'>be483ae2-da00-4347-a204-88083fbd34fd</td><td style='border: 1px solid #dddddd'>3.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>[]</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>6a04baa9-baab-49e0-8864-566afa17d24f</td><td style='border: 1px solid #dddddd'>975c4f7d-42dd-4f94-ba48-e7abba54a61e</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-08 01:02:45.254000+00:00</td><td style='border: 1px solid #dddddd'>tests_failed</td><td style='border: 1px solid #dddddd'>be483ae2-da00-4347-a204-88083fbd34fd</td><td style='border: 1px solid #dddddd'>0.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>[]</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>6a04baa9-baab-49e0-8864-566afa17d24f</td><td style='border: 1px solid #dddddd'>975c4f7d-42dd-4f94-ba48-e7abba54a61e</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-08 00:46:23.119000+00:00</td><td style='border: 1px solid #dddddd'>tests_passed</td><td style='border: 1px solid #dddddd'>432cf99d-e6f5-4a3d-bfe4-6ccc50cd1cf7</td><td style='border: 1px solid #dddddd'>0.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['test_data_set_name:german_credit_risk_test_data_4.csv']</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>6a04baa9-baab-49e0-8864-566afa17d24f</td><td style='border: 1px solid #dddddd'>cab790a9-8c34-4d6c-9cf8-08b8adfd8cbd</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-08 00:46:23.119000+00:00</td><td style='border: 1px solid #dddddd'>tests_run</td><td style='border: 1px solid #dddddd'>432cf99d-e6f5-4a3d-bfe4-6ccc50cd1cf7</td><td style='border: 1px solid #dddddd'>1.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['test_data_set_name:german_credit_risk_test_data_4.csv']</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>6a04baa9-baab-49e0-8864-566afa17d24f</td><td style='border: 1px solid #dddddd'>cab790a9-8c34-4d6c-9cf8-08b8adfd8cbd</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-08 00:46:23.119000+00:00</td><td style='border: 1px solid #dddddd'>tests_skipped</td><td style='border: 1px solid #dddddd'>432cf99d-e6f5-4a3d-bfe4-6ccc50cd1cf7</td><td style='border: 1px solid #dddddd'>3.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['test_data_set_name:german_credit_risk_test_data_4.csv']</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>6a04baa9-baab-49e0-8864-566afa17d24f</td><td style='border: 1px solid #dddddd'>cab790a9-8c34-4d6c-9cf8-08b8adfd8cbd</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-08 00:46:23.119000+00:00</td><td style='border: 1px solid #dddddd'>tests_failed</td><td style='border: 1px solid #dddddd'>432cf99d-e6f5-4a3d-bfe4-6ccc50cd1cf7</td><td style='border: 1px solid #dddddd'>1.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['test_data_set_name:german_credit_risk_test_data_4.csv']</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>6a04baa9-baab-49e0-8864-566afa17d24f</td><td style='border: 1px solid #dddddd'>cab790a9-8c34-4d6c-9cf8-08b8adfd8cbd</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-08 00:45:57.944000+00:00</td><td style='border: 1px solid #dddddd'>tests_passed</td><td style='border: 1px solid #dddddd'>0050100d-8842-4254-a4dc-9d85a0c616ed</td><td style='border: 1px solid #dddddd'>0.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['test_data_set_name:german_credit_risk_test_data_3.csv']</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>6a04baa9-baab-49e0-8864-566afa17d24f</td><td style='border: 1px solid #dddddd'>533e43b7-e921-4cb3-abc4-913b03936214</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-08 00:45:57.944000+00:00</td><td style='border: 1px solid #dddddd'>tests_run</td><td style='border: 1px solid #dddddd'>0050100d-8842-4254-a4dc-9d85a0c616ed</td><td style='border: 1px solid #dddddd'>1.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['test_data_set_name:german_credit_risk_test_data_3.csv']</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>6a04baa9-baab-49e0-8864-566afa17d24f</td><td style='border: 1px solid #dddddd'>533e43b7-e921-4cb3-abc4-913b03936214</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>a9a2e1e9-181e-45bd-9fec-18eea1ddd3e7</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: First 10 records were displayed.\n"
     ]
    }
   ],
   "source": [
    "wos_client.monitor_instances.show_metrics(monitor_instance_id=mrm_monitor_instance_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "69ff86e6-b02a-4963-941f-cfd41cd179db"
   },
   "source": [
    "## Refresh the model summary of the production subscription in the OpenScale dashboard\n",
    "\n",
    "This brings us to the end of this demo exercise. Thank you for trying it out."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}