Cloud Pak for Data/WML/notebooks/indirect_bias/OpenScale Indirect Bias and active debias API usage - CP4D.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "id": "75feab07-55ef-4436-ab6c-48d5c2d133e4",
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "source": [
    "<img src=\"https://github.com/pmservice/ai-openscale-tutorials/raw/master/notebooks/images/banner.png\" align=\"left\" alt=\"banner\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b18c0eec7d1c46f6b8fe4c779c478b7b"
   },
   "source": [
    "# Working with Watson Machine Learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fc3ebd630e524b3e812e2d17d603e5c9"
   },
   "source": [
    "This notebook should be run using with **Default Spark 3.3 & Python 3.10** runtime environment. **If you are viewing this in Watson Studio and do not see Python 3.10.x in the upper right corner of your screen, please update the runtime now.** It requires service credentials for the following services:\n",
    "  * Watson OpenScale\n",
    "  * Watson Machine Learning \n",
    "  * DB2\n",
    "\n",
    "  \n",
    "The notebook will train, create and deploy a model, configure OpenScale to monitor that deployment, and inject seven days' worth of historical records and measurements for viewing in the OpenScale Insights dashboard."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "262ffb75c64b4523bfda24ea95d4b8f5"
   },
   "source": [
    "# Setup <a name=\"setup\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3b7c0a42ac1e43139057e5d78abfe075"
   },
   "source": [
    "## Package installation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "827d92b4b8e1491e81d23153816f555c"
   },
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "dc441429a8ad4aada3fa29d81e98e5cf"
   },
   "outputs": [],
   "source": [
    "!pip install --upgrade pyspark==3.0.3 --no-cache | tail -n 1\n",
    "\n",
    "#If you are running this notebook in non IBM Watson Studio env then uncomment the below pip statements and run it\n",
    "#!pip install --upgrade pandas==1.2.3 --no-cache | tail -n 1\n",
    "#!pip install --upgrade requests==2.23 --no-cache | tail -n 1\n",
    "#!pip install numpy==1.20.1 --no-cache | tail -n 1\n",
    "#!pip install SciPy --no-cache | tail -n 1\n",
    "#!pip install lime --no-cache | tail -n 1\n",
    "\n",
    "!pip install --upgrade ibm-watson-machine-learning --user | tail -n 1\n",
    "!pip install --upgrade ibm-watson-openscale --no-cache | tail -n 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "44f0e2adc53c4f04ba9647b031d88cb9"
   },
   "source": [
    "### Action: restart the kernel!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f8cdd3fa70d94b9c8e3bea9fc4c65e9f"
   },
   "source": [
    "## Configure credentials"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "64b031a79e2f497d8d63ca093d9bf0ad"
   },
   "source": [
    "- WOS_CREDENTIALS (CP4D)\n",
    "- WML_CREDENTIALS (CP4D)\n",
    "- DATABASE_CREDENTIALS (DB2 on CP4D or Cloud Object Storage (COS))\n",
    "- SCHEMA_NAME"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "c470852b2bf14f7caeef5d025c8d64c7"
   },
   "outputs": [],
   "source": [
    "#masked\n",
    "WOS_CREDENTIALS = {\n",
    "    \"url\": \"Cluster host name\",\n",
    "    \"username\": \"XX\",\n",
    "    \"password\": \"XX\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "dd83a4b858414f22895adb44c3234185"
   },
   "outputs": [],
   "source": [
    "#masked\n",
    "WML_CREDENTIALS = {\n",
    "                   \"url\": \"Cluster host name\",\n",
    "                   \"username\": \"XX\",\n",
    "                   \"password\" : \"XX\",\n",
    "                   \"instance_id\": \"wml_local\",\n",
    "                   \"version\" : \"3.5\" #If your env is CP4D 4.0 then specify \"4.0\" instead of \"3.5\" \n",
    "                  }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "8604bf1ad39c4e6981754957eec367d4"
   },
   "outputs": [],
   "source": [
    "#masked\n",
    "#IBM DB2 database connection format example\n",
    "DATABASE_CREDENTIALS = {\n",
    "    \"hostname\":\"9.999.999.99\",\n",
    "    \"username\":\"XX\",\n",
    "    \"password\":\"XX\",\n",
    "    \"database\":\"SAMPLE\",\n",
    "    \"port\":\"50000\"\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "24b4c90ba7cb4f3e89a95faa26555865"
   },
   "source": [
    "### Action: put created schema name below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "ede68cb2c2564377984fde01b0a8b2d7"
   },
   "outputs": [],
   "source": [
    "SCHEMA_NAME = 'AIOSFASTPATHICP-00000000-0000-0000-0000-000000000000'\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "d85650ef53984d799a58b8ce79160674"
   },
   "source": [
    "## Save training data to Cloud Object Storage\n",
    "\n",
    "### Cloud object storage details¶\n",
    "\n",
    "In next cells, you will need to paste some credentials to Cloud Object Storage. If you haven't worked with COS yet please visit getting started with COS tutorial. You can find COS_API_KEY_ID and COS_RESOURCE_CRN variables in Service Credentials in menu of your COS instance. Used COS Service Credentials must be created with Role parameter set as Writer. Later training data file will be loaded to the bucket of your instance and used as training refecence in subsription. COS_ENDPOINT variable can be found in Endpoint field of the menu."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "9cb6e02e0589449b9d2a9813d44dec07"
   },
   "outputs": [],
   "source": [
    "IAM_URL=\"https://iam.ng.bluemix.net/oidc/token\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "dbac5a57926d4f7eb3e74564cc6a95af"
   },
   "outputs": [],
   "source": [
    "# masked\n",
    "COS_API_KEY_ID = \"***\"\n",
    "COS_RESOURCE_CRN = \"***\" # eg \"crn:v1:bluemix:public:cloud-object-storage:global:a/3bf0d9003abfb5d29761c3e97696b71c:d6f04d83-6c4f-4a62-a165-696756d63903::\"\n",
    "COS_ENDPOINT = \"https://s3.us.cloud-object-storage.appdomain.cloud\" # Current list avaiable at https://control.cloud-object-storage.cloud.ibm.com/v2/endpoints\n",
    "BUCKET_NAME = \"testcasebucket\"\n",
    "FILE_NAME = \"Indirect_bias_AdultCensusdata.csv\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "056e2d289da0487c83e16983cde997ec"
   },
   "source": [
    "# Load and explore data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "a74cc3aca27d406d88a25ab74a373157"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2024-08-06 18:54:16--  https://raw.githubusercontent.com/IBM/watson-openscale-samples/main/IBM%20Cloud/WML/assets/data/adult_census/Indirect_bias_AdultCensusdata.csv\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.109.133, 185.199.111.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 3551145 (3.4M) [text/plain]\n",
      "Saving to: ‘Indirect_bias_AdultCensusdata.csv’\n",
      "\n",
      "Indirect_bias_Adult 100%[===================>]   3.39M  10.4MB/s    in 0.3s    \n",
      "\n",
      "2024-08-06 18:54:16 (10.4 MB/s) - ‘Indirect_bias_AdultCensusdata.csv’ saved [3551145/3551145]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!rm Indirect_bias_AdultCensusdata.csv\n",
    "!wget https://raw.githubusercontent.com/IBM/watson-openscale-samples/main/IBM%20Cloud/WML/assets/data/adult_census/Indirect_bias_AdultCensusdata.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b2235027af5a4d329972a869fbcfd2f8"
   },
   "source": [
    "## Explore data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "b6eec247a77146d88e6476fc5ad49b03"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "24/08/06 18:54:18 WARN Utils: Your hostname, Nelwins-MacBook-Pro.local resolves to a loopback address: 127.0.0.1; using 192.168.0.103 instead (on interface en0)\n",
      "24/08/06 18:54:18 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address\n",
      "Setting default log level to \"WARN\".\n",
      "To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).\n",
      "24/08/06 18:54:18 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable\n",
      "24/08/06 18:54:18 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Row(age=39, workclass='State-gov', fnlwgt=77516, education='Bachelors', education-num=13, Marital='Never-married', occupation='Adm-clerical', relationship='Not-in-family', race='White', sex='Male', capitalgain=2174, loss=0, hoursper=40, citizen_status='United-States', label='<=50K')"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from pyspark.sql import SparkSession\n",
    "import json\n",
    "\n",
    "spark = SparkSession.builder.getOrCreate()\n",
    "df_data = spark.read.csv(path=\"Indirect_bias_AdultCensusdata.csv\", sep=\",\", header=True, inferSchema=True) \n",
    "df_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "a09d9e346bce47a98a7eccc833112128"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of records: 32561\n"
     ]
    }
   ],
   "source": [
    "print(\"Number of records: \" + str(df_data.count()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9d55078678a94079b67b183e549fc534"
   },
   "source": [
    "# Create a model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "88357d1f249f427491163b19f6acc9ab"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root\n",
      " |-- workclass: string (nullable = true)\n",
      " |-- fnlwgt: integer (nullable = true)\n",
      " |-- education: string (nullable = true)\n",
      " |-- education-num: integer (nullable = true)\n",
      " |-- Marital: string (nullable = true)\n",
      " |-- occupation: string (nullable = true)\n",
      " |-- relationship: string (nullable = true)\n",
      " |-- capitalgain: integer (nullable = true)\n",
      " |-- loss: integer (nullable = true)\n",
      " |-- hoursper: integer (nullable = true)\n",
      " |-- citizen_status: string (nullable = true)\n",
      " |-- label: string (nullable = true)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# spark_df = sqlCtx.createDataFrame(df_data)\n",
    "spark_df = df_data\n",
    "# Remove protected attributes from training data\n",
    "protected_attributes = [\"race\", \"age\", \"sex\"]\n",
    "for attr in protected_attributes:\n",
    "    spark_df = spark_df.drop(attr)\n",
    "columns = spark_df.columns\n",
    "model_name = \"Adult Census Income Classifier Model\"\n",
    "deployment_name = \"Adult Census Income Classifier Deployment\"\n",
    "\n",
    "spark_df.printSchema()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "82155d92dee14d8596d23529d454c3e9"
   },
   "outputs": [],
   "source": [
    "from pyspark.ml.feature import OneHotEncoder, StringIndexer, IndexToString, VectorAssembler\n",
    "from pyspark.ml import Pipeline, Model\n",
    "\n",
    "cat_features = ['workclass', 'education', 'Marital', 'occupation', 'relationship', 'citizen_status'] \n",
    "num_features = [\"fnlwgt\", \"education-num\", \"capitalgain\", \"loss\", \"hoursper\"]\n",
    "stages=[]\n",
    "\n",
    "for feature in cat_features:\n",
    "    string_indexer = StringIndexer(inputCol = feature, outputCol = feature + '_IX').setHandleInvalid(\"keep\")\n",
    "    encoder = OneHotEncoder(inputCols=[string_indexer.getOutputCol()], outputCols=[feature + \"classVec\"])\n",
    "    stages += [string_indexer, encoder]\n",
    "\n",
    "si_Label = StringIndexer(inputCol=\"label\", outputCol=\"encoded_label\").fit(spark_df)\n",
    "label_converter = IndexToString(inputCol=\"prediction\", outputCol=\"predictedLabel\", labels=si_Label.labels)\n",
    "stages.append(si_Label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "cadc3aae47ca464abf9c956375b1ec72"
   },
   "outputs": [],
   "source": [
    "assembler_inputs = [c + \"classVec\" for c in cat_features] + num_features\n",
    "va_features = VectorAssembler(inputCols=assembler_inputs, outputCol=\"features\")\n",
    "stages.append(va_features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "a3d7ff9894674a94bd713d2e9af4cd55"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of records for training: 26056\n",
      "Number of records for evaluation: 6505\n"
     ]
    }
   ],
   "source": [
    "(train_data, test_data) = spark_df.randomSplit([0.8, 0.2], 24)\n",
    "print(\"Number of records for training: \" + str(train_data.count()))\n",
    "print(\"Number of records for evaluation: \" + str(test_data.count()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "720b9ee7712b450c8b3ecdcd4ab58893"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['workclass',\n",
       " 'fnlwgt',\n",
       " 'education',\n",
       " 'education-num',\n",
       " 'Marital',\n",
       " 'occupation',\n",
       " 'relationship',\n",
       " 'capitalgain',\n",
       " 'loss',\n",
       " 'hoursper',\n",
       " 'citizen_status',\n",
       " 'label']"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "04cf8f5bdf08438f8992890b1bc70442"
   },
   "outputs": [],
   "source": [
    "from pyspark.ml.classification import GBTClassifier, DecisionTreeClassifier, RandomForestClassifier\n",
    "classifier = RandomForestClassifier(labelCol=\"encoded_label\", featuresCol=\"features\")\n",
    "stages.append(classifier)\n",
    "stages.append(label_converter)\n",
    "pipeline = Pipeline(stages=stages)\n",
    "model = pipeline.fit(train_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "id": "30adcf7239c54415a6e74250a6f80903"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root\n",
      " |-- workclass: string (nullable = true)\n",
      " |-- fnlwgt: integer (nullable = true)\n",
      " |-- education: string (nullable = true)\n",
      " |-- education-num: integer (nullable = true)\n",
      " |-- Marital: string (nullable = true)\n",
      " |-- occupation: string (nullable = true)\n",
      " |-- relationship: string (nullable = true)\n",
      " |-- capitalgain: integer (nullable = true)\n",
      " |-- loss: integer (nullable = true)\n",
      " |-- hoursper: integer (nullable = true)\n",
      " |-- citizen_status: string (nullable = true)\n",
      " |-- label: string (nullable = true)\n",
      " |-- workclass_IX: double (nullable = false)\n",
      " |-- workclassclassVec: vector (nullable = true)\n",
      " |-- education_IX: double (nullable = false)\n",
      " |-- educationclassVec: vector (nullable = true)\n",
      " |-- Marital_IX: double (nullable = false)\n",
      " |-- MaritalclassVec: vector (nullable = true)\n",
      " |-- occupation_IX: double (nullable = false)\n",
      " |-- occupationclassVec: vector (nullable = true)\n",
      " |-- relationship_IX: double (nullable = false)\n",
      " |-- relationshipclassVec: vector (nullable = true)\n",
      " |-- citizen_status_IX: double (nullable = false)\n",
      " |-- citizen_statusclassVec: vector (nullable = true)\n",
      " |-- encoded_label: double (nullable = false)\n",
      " |-- features: vector (nullable = true)\n",
      " |-- rawPrediction: vector (nullable = true)\n",
      " |-- probability: vector (nullable = true)\n",
      " |-- prediction: double (nullable = false)\n",
      " |-- predictedLabel: string (nullable = true)\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "24/08/06 18:54:27 WARN SparkStringUtils: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.sql.debug.maxToStringFields'.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Row(workclass='?', fnlwgt=23780, education='Masters', education-num=14, Marital='Married-spouse-absent', occupation='?', relationship='Other-relative', capitalgain=0, loss=0, hoursper=40, citizen_status='United-States', label='<=50K', workclass_IX=3.0, workclassclassVec=SparseVector(9, {3: 1.0}), education_IX=3.0, educationclassVec=SparseVector(16, {3: 1.0}), Marital_IX=5.0, MaritalclassVec=SparseVector(7, {5: 1.0}), occupation_IX=7.0, occupationclassVec=SparseVector(15, {7: 1.0}), relationship_IX=5.0, relationshipclassVec=SparseVector(6, {5: 1.0}), citizen_status_IX=0.0, citizen_statusclassVec=SparseVector(42, {0: 1.0}), encoded_label=0.0, features=SparseVector(100, {3: 1.0, 12: 1.0, 30: 1.0, 39: 1.0, 52: 1.0, 53: 1.0, 95: 23780.0, 96: 14.0, 99: 40.0}), rawPrediction=DenseVector([15.3339, 4.6661]), probability=DenseVector([0.7667, 0.2333]), prediction=0.0, predictedLabel='<=50K')"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictions = model.transform(test_data)\n",
    "predictions.printSchema()\n",
    "predictions.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "efa502d57f284612b95903f27157992f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy = 0.891208\n"
     ]
    }
   ],
   "source": [
    "from pyspark.ml.evaluation import BinaryClassificationEvaluator\n",
    "evaluatorDT = BinaryClassificationEvaluator(labelCol=\"encoded_label\", rawPredictionCol=\"rawPrediction\")\n",
    "accuracy = evaluatorDT.evaluate(predictions)\n",
    "\n",
    "print(\"Accuracy = %g\" % accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bbb45c2f139940b28e3f9dfff4e99576"
   },
   "source": [
    "# Save and deploy the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "id": "8c46867b3f484ba1a396f364742552de"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "24/08/06 18:54:34 WARN GarbageCollectionMetrics: To enable non-built-in garbage collector(s) List(G1 Concurrent GC), users should configure it(them) to spark.eventLog.gcMetrics.youngGenerationGarbageCollectors or spark.eventLog.gcMetrics.oldGenerationGarbageCollectors\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'1.0.360'"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import json\n",
    "from ibm_watson_machine_learning import APIClient\n",
    "\n",
    "wml_client = APIClient(WML_CREDENTIALS)\n",
    "wml_client.version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "5d6754453fdb4e50836da3e9ed32df17"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------------------------------------  -------------------------------------------------------------------  ------------------------\n",
      "ID                                    NAME                                                                 CREATED\n",
      "7e5a8be6-9103-4c22-9c43-b66f3d8364de  poojitha_notebooks_space                                             2024-06-29T13:53:46.645Z\n",
      "16ccd855-46bd-43ed-8219-5f00ac565d08  shreya-space                                                         2024-06-26T04:29:17.302Z\n",
      "bc3b9797-c509-4fb4-a424-f67b1e2ed4be  QUALITY_WMLV4_PREPROD                                                2024-06-23T12:23:04.790Z\n",
      "e396e187-2977-47b4-ade3-1539f9f10adc  QUALITY_WMLV4_PROD                                                   2024-06-23T12:22:54.422Z\n",
      "40c4d032-0339-4da6-bfec-4bdb096c9650  shreya                                                               2024-06-20T10:54:20.088Z\n",
      "088c142e-f35e-4e48-a30c-ad55a6edeecc  notebooks 5.0                                                        2024-06-13T04:42:07.336Z\n",
      "b9b3d3b4-6e26-4e16-807d-e8bf5e7d6984  MRM_WMLV4_PREPROD                                                    2024-06-12T15:49:26.571Z\n",
      "d22e2b6b-917c-4427-a40c-1a439352a742  MRM_WMLV4_PROD                                                       2024-06-12T15:49:16.185Z\n",
      "ce15e0f6-be30-4349-af47-35ae15983bf1  openscale-express-path-preprod-00000000-0000-0000-0000-000000000000  2024-06-04T05:18:51.988Z\n",
      "6264dc0e-087a-4dea-bcbc-6bd872b510fb  openscale-express-path-00000000-0000-0000-0000-000000000000          2024-06-04T05:18:30.811Z\n",
      "------------------------------------  -------------------------------------------------------------------  ------------------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>NAME</th>\n",
       "      <th>CREATED</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7e5a8be6-9103-4c22-9c43-b66f3d8364de</td>\n",
       "      <td>poojitha_notebooks_space</td>\n",
       "      <td>2024-06-29T13:53:46.645Z</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>16ccd855-46bd-43ed-8219-5f00ac565d08</td>\n",
       "      <td>shreya-space</td>\n",
       "      <td>2024-06-26T04:29:17.302Z</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>bc3b9797-c509-4fb4-a424-f67b1e2ed4be</td>\n",
       "      <td>QUALITY_WMLV4_PREPROD</td>\n",
       "      <td>2024-06-23T12:23:04.790Z</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>e396e187-2977-47b4-ade3-1539f9f10adc</td>\n",
       "      <td>QUALITY_WMLV4_PROD</td>\n",
       "      <td>2024-06-23T12:22:54.422Z</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>40c4d032-0339-4da6-bfec-4bdb096c9650</td>\n",
       "      <td>shreya</td>\n",
       "      <td>2024-06-20T10:54:20.088Z</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>088c142e-f35e-4e48-a30c-ad55a6edeecc</td>\n",
       "      <td>notebooks 5.0</td>\n",
       "      <td>2024-06-13T04:42:07.336Z</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>b9b3d3b4-6e26-4e16-807d-e8bf5e7d6984</td>\n",
       "      <td>MRM_WMLV4_PREPROD</td>\n",
       "      <td>2024-06-12T15:49:26.571Z</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>d22e2b6b-917c-4427-a40c-1a439352a742</td>\n",
       "      <td>MRM_WMLV4_PROD</td>\n",
       "      <td>2024-06-12T15:49:16.185Z</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>ce15e0f6-be30-4349-af47-35ae15983bf1</td>\n",
       "      <td>openscale-express-path-preprod-00000000-0000-0...</td>\n",
       "      <td>2024-06-04T05:18:51.988Z</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>6264dc0e-087a-4dea-bcbc-6bd872b510fb</td>\n",
       "      <td>openscale-express-path-00000000-0000-0000-0000...</td>\n",
       "      <td>2024-06-04T05:18:30.811Z</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     ID  \\\n",
       "0  7e5a8be6-9103-4c22-9c43-b66f3d8364de   \n",
       "1  16ccd855-46bd-43ed-8219-5f00ac565d08   \n",
       "2  bc3b9797-c509-4fb4-a424-f67b1e2ed4be   \n",
       "3  e396e187-2977-47b4-ade3-1539f9f10adc   \n",
       "4  40c4d032-0339-4da6-bfec-4bdb096c9650   \n",
       "5  088c142e-f35e-4e48-a30c-ad55a6edeecc   \n",
       "6  b9b3d3b4-6e26-4e16-807d-e8bf5e7d6984   \n",
       "7  d22e2b6b-917c-4427-a40c-1a439352a742   \n",
       "8  ce15e0f6-be30-4349-af47-35ae15983bf1   \n",
       "9  6264dc0e-087a-4dea-bcbc-6bd872b510fb   \n",
       "\n",
       "                                                NAME                   CREATED  \n",
       "0                           poojitha_notebooks_space  2024-06-29T13:53:46.645Z  \n",
       "1                                       shreya-space  2024-06-26T04:29:17.302Z  \n",
       "2                              QUALITY_WMLV4_PREPROD  2024-06-23T12:23:04.790Z  \n",
       "3                                 QUALITY_WMLV4_PROD  2024-06-23T12:22:54.422Z  \n",
       "4                                             shreya  2024-06-20T10:54:20.088Z  \n",
       "5                                     notebooks 5.0   2024-06-13T04:42:07.336Z  \n",
       "6                                  MRM_WMLV4_PREPROD  2024-06-12T15:49:26.571Z  \n",
       "7                                     MRM_WMLV4_PROD  2024-06-12T15:49:16.185Z  \n",
       "8  openscale-express-path-preprod-00000000-0000-0...  2024-06-04T05:18:51.988Z  \n",
       "9  openscale-express-path-00000000-0000-0000-0000...  2024-06-04T05:18:30.811Z  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wml_client.spaces.list(limit=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f1cd9533f53a414f86f748e34275cfce"
   },
   "source": [
    "## Find the space that you would like to associate the model that is created and deployed as part of the notebook, and specify it in the next cell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "id": "3393075295af4022a7f060b5a85a5b6e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'SUCCESS'"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "WML_SPACE_ID='***' # use space id here\n",
    "wml_client.set.default_space(WML_SPACE_ID)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "7f4015490dfc48c7a7b74c7ed9627284"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Deleting deployment id 1bc4d5d7-26c6-4d9e-8cc8-c128e8bf1983\n",
      "Deleting model id 62b958dd-8bd9-42b5-9a63-33a58ea0a26c\n",
      "------------------------------------  -------------------------------  ------------------------  ---------  ----------  ----------------\n",
      "ID                                    NAME                             CREATED                   TYPE       SPEC_STATE  SPEC_REPLACEMENT\n",
      "773f34d7-920b-4cb1-a0b6-043d71d3a8ec  Spark German Risk Model - Final  2024-08-06T12:55:37.002Z  mllib_3.3  deprecated  spark-mllib_3.4\n",
      "------------------------------------  -------------------------------  ------------------------  ---------  ----------  ----------------\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>NAME</th>\n",
       "      <th>CREATED</th>\n",
       "      <th>TYPE</th>\n",
       "      <th>SPEC_STATE</th>\n",
       "      <th>SPEC_REPLACEMENT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>773f34d7-920b-4cb1-a0b6-043d71d3a8ec</td>\n",
       "      <td>Spark German Risk Model - Final</td>\n",
       "      <td>2024-08-06T12:55:37.002Z</td>\n",
       "      <td>mllib_3.3</td>\n",
       "      <td>deprecated</td>\n",
       "      <td>spark-mllib_3.4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     ID                             NAME  \\\n",
       "0  773f34d7-920b-4cb1-a0b6-043d71d3a8ec  Spark German Risk Model - Final   \n",
       "\n",
       "                    CREATED       TYPE  SPEC_STATE SPEC_REPLACEMENT  \n",
       "0  2024-08-06T12:55:37.002Z  mllib_3.3  deprecated  spark-mllib_3.4  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "deployments_list = wml_client.deployments.get_details()\n",
    "for deployment in deployments_list[\"resources\"]:\n",
    "    model_id = deployment[\"entity\"][\"asset\"][\"id\"]\n",
    "    deployment_id = deployment[\"metadata\"][\"id\"]\n",
    "    if deployment[\"metadata\"][\"name\"] == deployment_name:\n",
    "        print(\"Deleting deployment id\", deployment_id)\n",
    "        wml_client.deployments.delete(deployment_id)\n",
    "        print(\"Deleting model id\", model_id)\n",
    "        wml_client.repository.delete(model_id)\n",
    "wml_client.repository.list_models()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "id": "e9df60c1cc314ccb887411f0d3253a92"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating connections...\n",
      "SUCCESS\n"
     ]
    }
   ],
   "source": [
    "datasource_type = wml_client.connections.get_datasource_type_uid_by_name('bluemixcloudobjectstorage')\n",
    "conn_meta_props= {\n",
    "    wml_client.connections.ConfigurationMetaNames.NAME: \"Connection My COS \",\n",
    "    wml_client.connections.ConfigurationMetaNames.DATASOURCE_TYPE: datasource_type,\n",
    "    wml_client.connections.ConfigurationMetaNames.DESCRIPTION: \"Connection to my COS\",\n",
    "    wml_client.connections.ConfigurationMetaNames.PROPERTIES: {\n",
    "        'bucket': BUCKET_NAME,\n",
    "        'api_key': COS_API_KEY_ID,\n",
    "        'resource_instance_id': COS_RESOURCE_CRN,\n",
    "        'iam_url': \"https://iam.ng.bluemix.net/oidc/token\",\n",
    "        'url': COS_ENDPOINT\n",
    "    }\n",
    "}\n",
    "\n",
    "conn_details = wml_client.connections.create(meta_props=conn_meta_props)\n",
    "connection_id = wml_client.connections.get_uid(conn_details)\n",
    "\n",
    "training_data_references = [\n",
    "    {\n",
    "        \"id\": \"German Credit Risk\", \n",
    "        \"type\": \"connection_asset\",\n",
    "        \"connection\": {\n",
    "            \"id\": connection_id,\n",
    "            \"href\": \"/v2/connections/\" + connection_id + \"?space_id=\" + WML_SPACE_ID\n",
    "\n",
    "        },\n",
    "        \"location\": {\n",
    "            \"bucket\": BUCKET_NAME,\n",
    "            \"file_name\": FILE_NAME\n",
    "        }\n",
    "    }    \n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "id": "cf0293f9272c4623a4c39f752b6554cf"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Software Specification ID: d11f2434-4fc7-58b7-8a62-755da64fdaf8\n"
     ]
    }
   ],
   "source": [
    "software_spec_uid = wml_client.software_specifications.get_id_by_name(\"spark-mllib_3.3\")\n",
    "print(\"Software Specification ID: {}\".format(software_spec_uid))\n",
    "model_props = {\n",
    "        wml_client._models.ConfigurationMetaNames.NAME:\"{}\".format(model_name),\n",
    "        wml_client._models.ConfigurationMetaNames.TYPE: \"mllib_3.3\",\n",
    "        wml_client._models.ConfigurationMetaNames.SOFTWARE_SPEC_UID: software_spec_uid,\n",
    "        wml_client._models.ConfigurationMetaNames.LABEL_FIELD: \"label\",\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "id": "e7d3327cef8c43c18ff8191f930f887d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Storing model ...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                \r"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: Warnings!! :  Software specification spark-mllib_3.3 specified for the wml_model is deprecated and will be removed in the future. We recommend you use spark-mllib_3.4 instead. For details see Supported Frameworks https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=specifications-supported-deployment-frameworks\n",
      "Done\n",
      "Model ID: eed8b225-19e5-4460-9b6f-7271dc1e3ff2\n"
     ]
    }
   ],
   "source": [
    "print(\"Storing model ...\")\n",
    "published_model_details = wml_client.repository.store_model(\n",
    "    model=model, \n",
    "    meta_props=model_props, \n",
    "    training_data=train_data, \n",
    "    pipeline=pipeline)\n",
    "\n",
    "model_uid = wml_client.repository.get_model_id(published_model_details)\n",
    "print(\"Done\")\n",
    "print(\"Model ID: {}\".format(model_uid))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "id": "06c2bb55cbf04f59b092c8d773e7952b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'entity': {'hybrid_pipeline_software_specs': [],\n",
       "  'label_column': 'label',\n",
       "  'pipeline': {'id': '5174fc07-5a11-4d7e-ab19-8228b0c2c0ef'},\n",
       "  'schemas': {'input': [{'fields': [{'metadata': {},\n",
       "       'name': 'workclass',\n",
       "       'nullable': True,\n",
       "       'type': 'string'},\n",
       "      {'metadata': {}, 'name': 'fnlwgt', 'nullable': True, 'type': 'integer'},\n",
       "      {'metadata': {},\n",
       "       'name': 'education',\n",
       "       'nullable': True,\n",
       "       'type': 'string'},\n",
       "      {'metadata': {},\n",
       "       'name': 'education-num',\n",
       "       'nullable': True,\n",
       "       'type': 'integer'},\n",
       "      {'metadata': {}, 'name': 'Marital', 'nullable': True, 'type': 'string'},\n",
       "      {'metadata': {},\n",
       "       'name': 'occupation',\n",
       "       'nullable': True,\n",
       "       'type': 'string'},\n",
       "      {'metadata': {},\n",
       "       'name': 'relationship',\n",
       "       'nullable': True,\n",
       "       'type': 'string'},\n",
       "      {'metadata': {},\n",
       "       'name': 'capitalgain',\n",
       "       'nullable': True,\n",
       "       'type': 'integer'},\n",
       "      {'metadata': {}, 'name': 'loss', 'nullable': True, 'type': 'integer'},\n",
       "      {'metadata': {},\n",
       "       'name': 'hoursper',\n",
       "       'nullable': True,\n",
       "       'type': 'integer'},\n",
       "      {'metadata': {},\n",
       "       'name': 'citizen_status',\n",
       "       'nullable': True,\n",
       "       'type': 'string'}],\n",
       "     'id': '1',\n",
       "     'type': 'struct'}],\n",
       "   'output': []},\n",
       "  'software_spec': {'id': 'd11f2434-4fc7-58b7-8a62-755da64fdaf8',\n",
       "   'name': 'spark-mllib_3.3'},\n",
       "  'training_data_references': [{'id': '1',\n",
       "    'location': {},\n",
       "    'schema': {'fields': [{'metadata': {},\n",
       "       'name': 'workclass',\n",
       "       'nullable': True,\n",
       "       'type': 'string'},\n",
       "      {'metadata': {}, 'name': 'fnlwgt', 'nullable': True, 'type': 'integer'},\n",
       "      {'metadata': {},\n",
       "       'name': 'education',\n",
       "       'nullable': True,\n",
       "       'type': 'string'},\n",
       "      {'metadata': {},\n",
       "       'name': 'education-num',\n",
       "       'nullable': True,\n",
       "       'type': 'integer'},\n",
       "      {'metadata': {}, 'name': 'Marital', 'nullable': True, 'type': 'string'},\n",
       "      {'metadata': {},\n",
       "       'name': 'occupation',\n",
       "       'nullable': True,\n",
       "       'type': 'string'},\n",
       "      {'metadata': {},\n",
       "       'name': 'relationship',\n",
       "       'nullable': True,\n",
       "       'type': 'string'},\n",
       "      {'metadata': {},\n",
       "       'name': 'capitalgain',\n",
       "       'nullable': True,\n",
       "       'type': 'integer'},\n",
       "      {'metadata': {}, 'name': 'loss', 'nullable': True, 'type': 'integer'},\n",
       "      {'metadata': {},\n",
       "       'name': 'hoursper',\n",
       "       'nullable': True,\n",
       "       'type': 'integer'},\n",
       "      {'metadata': {},\n",
       "       'name': 'citizen_status',\n",
       "       'nullable': True,\n",
       "       'type': 'string'},\n",
       "      {'metadata': {'modeling_role': 'target'},\n",
       "       'name': 'label',\n",
       "       'nullable': True,\n",
       "       'type': 'string'}],\n",
       "     'id': '1',\n",
       "     'type': 'struct'},\n",
       "    'type': 'fs'}],\n",
       "  'type': 'mllib_3.3'},\n",
       " 'metadata': {'created_at': '2024-08-06T13:25:05.830Z',\n",
       "  'id': 'eed8b225-19e5-4460-9b6f-7271dc1e3ff2',\n",
       "  'modified_at': '2024-08-06T13:25:13.439Z',\n",
       "  'name': 'Adult Census Income Classifier Model',\n",
       "  'owner': '1000331001',\n",
       "  'resource_key': 'e73c764a-4f79-48ed-8543-618c6caadfd1',\n",
       "  'space_id': 'e8b87647-a0e9-4932-920f-1e8f1d1f383d'},\n",
       " 'system': {'warnings': [{'message': 'Software specification spark-mllib_3.3 specified for the wml_model is deprecated and will be removed in the future. We recommend you use spark-mllib_3.4 instead. For details see Supported Frameworks https://www.ibm.com/docs/en/cloud-paks/cp-data/4.0?topic=specifications-supported-deployment-frameworks'}]}}"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "published_model_details"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f5483e2532fe43c1b931b1e18aa246d0"
   },
   "source": [
    "## Create a model deployment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "id": "cdc6ce1dcab84ca2803daa6444eaea6f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "#######################################################################################\n",
      "\n",
      "Synchronous deployment creation for uid: 'eed8b225-19e5-4460-9b6f-7271dc1e3ff2' started\n",
      "\n",
      "#######################################################################################\n",
      "\n",
      "\n",
      "initializing\n",
      "Note: Software specification spark-mllib_3.3 is deprecated. Use spark-mllib_3.4 software specification instead when saving a spark model. For details, see https://www.ibm.com/support/producthub/icpdata/docs/content/SSQNUZ_latest/wsj/wmls/wmls-deploy-python-types.html.\n",
      ".\n",
      "ready\n",
      "\n",
      "\n",
      "------------------------------------------------------------------------------------------------\n",
      "Successfully finished deployment creation, deployment_uid='038b8c68-6615-4c0d-8d46-15783d6ab9c8'\n",
      "------------------------------------------------------------------------------------------------\n",
      "\n",
      "\n",
      "Scoring URL:https://cpd-cpd-instance.apps.wos415nfs2672.cp.fyre.ibm.com/ml/v4/deployments/038b8c68-6615-4c0d-8d46-15783d6ab9c8/predictions\n",
      "Model id: eed8b225-19e5-4460-9b6f-7271dc1e3ff2\n",
      "Deployment id: 038b8c68-6615-4c0d-8d46-15783d6ab9c8\n"
     ]
    }
   ],
   "source": [
    "deployment_details = wml_client.deployments.create(\n",
    "    model_uid, \n",
    "    meta_props={\n",
    "        wml_client.deployments.ConfigurationMetaNames.NAME: \"{}\".format(deployment_name),\n",
    "        wml_client.deployments.ConfigurationMetaNames.ONLINE: {}\n",
    "    }\n",
    ")\n",
    "scoring_url = wml_client.deployments.get_scoring_href(deployment_details)\n",
    "deployment_uid=wml_client.deployments.get_id(deployment_details)\n",
    "\n",
    "print(\"Scoring URL:\" + scoring_url)\n",
    "print(\"Model id: {}\".format(model_uid))\n",
    "print(\"Deployment id: {}\".format(deployment_uid))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0fa3b287605044538f3c1b3ee632eb9b"
   },
   "source": [
    "# Construct the scoring payload"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "id": "eb1e48ae98514c14a6dc225eba1acc9f"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>fnlwgt</th>\n",
       "      <th>education</th>\n",
       "      <th>education-num</th>\n",
       "      <th>Marital</th>\n",
       "      <th>occupation</th>\n",
       "      <th>relationship</th>\n",
       "      <th>race</th>\n",
       "      <th>sex</th>\n",
       "      <th>capitalgain</th>\n",
       "      <th>loss</th>\n",
       "      <th>hoursper</th>\n",
       "      <th>citizen_status</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>39</td>\n",
       "      <td>State-gov</td>\n",
       "      <td>77516</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Adm-clerical</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>2174</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50</td>\n",
       "      <td>Self-emp-not-inc</td>\n",
       "      <td>83311</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Exec-managerial</td>\n",
       "      <td>Husband</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>38</td>\n",
       "      <td>Private</td>\n",
       "      <td>215646</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>9</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>53</td>\n",
       "      <td>Private</td>\n",
       "      <td>234721</td>\n",
       "      <td>11th</td>\n",
       "      <td>7</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Husband</td>\n",
       "      <td>Black</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28</td>\n",
       "      <td>Private</td>\n",
       "      <td>338409</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Prof-specialty</td>\n",
       "      <td>Wife</td>\n",
       "      <td>Black</td>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>Cuba</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age         workclass  fnlwgt  education  education-num  \\\n",
       "0   39         State-gov   77516  Bachelors             13   \n",
       "1   50  Self-emp-not-inc   83311  Bachelors             13   \n",
       "2   38           Private  215646    HS-grad              9   \n",
       "3   53           Private  234721       11th              7   \n",
       "4   28           Private  338409  Bachelors             13   \n",
       "\n",
       "              Marital         occupation   relationship   race     sex  \\\n",
       "0       Never-married       Adm-clerical  Not-in-family  White    Male   \n",
       "1  Married-civ-spouse    Exec-managerial        Husband  White    Male   \n",
       "2            Divorced  Handlers-cleaners  Not-in-family  White    Male   \n",
       "3  Married-civ-spouse  Handlers-cleaners        Husband  Black    Male   \n",
       "4  Married-civ-spouse     Prof-specialty           Wife  Black  Female   \n",
       "\n",
       "   capitalgain  loss  hoursper citizen_status  label  \n",
       "0         2174     0        40  United-States  <=50K  \n",
       "1            0     0        13  United-States  <=50K  \n",
       "2            0     0        40  United-States  <=50K  \n",
       "3            0     0        40  United-States  <=50K  \n",
       "4            0     0        40           Cuba  <=50K  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"Indirect_bias_AdultCensusdata.csv\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "a8aaab665f9947b594380d4892c237a7"
   },
   "source": [
    "## Remove the sensitive attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "id": "dcd4da086e604f92b7bedd6f1283ba4e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['label', 'race', 'age', 'sex']"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cols_to_remove = ['label']\n",
    "cols_to_remove.extend(protected_attributes)\n",
    "cols_to_remove"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "546d7ce9df5c404f84c2a556c3679e4e"
   },
   "source": [
    "## Create the meta data frame capturing the sensitive data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "id": "37aa35f9cdfb466085acbb7eac3e56ec"
   },
   "outputs": [],
   "source": [
    "meta_df = df[protected_attributes].copy()\n",
    "meta_fields = meta_df.columns.tolist()\n",
    "meta_values = meta_df[meta_fields].values.tolist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ee69bf1da4a04865a6db14f13c46908d"
   },
   "source": [
    "## Construct the scoring payload comprising the meta fields"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "id": "3f50af2cb0f04203998c93f6caade4fb"
   },
   "outputs": [],
   "source": [
    "def get_scoring_payload(no_of_records_to_score = 1):\n",
    "    meta_payload = {\n",
    "        \"fields\": meta_fields,\n",
    "        \"values\": meta_values[:no_of_records_to_score]\n",
    "    }\n",
    "\n",
    "    for col in cols_to_remove:\n",
    "        if col in df.columns:\n",
    "            del df[col] \n",
    "\n",
    "    fields = df.columns.tolist()\n",
    "    values = df[fields].values.tolist()\n",
    "\n",
    "    payload_scoring = {\"input_data\": [{\"fields\": fields, \"values\": values[:no_of_records_to_score],\"meta\": meta_payload}]}  \n",
    "    return payload_scoring"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "90f239341ee04e13acb5455607cd4f25"
   },
   "source": [
    "## Method to perform scoring"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "id": "ad3991e04720434c9bcd42c7acf44b59"
   },
   "outputs": [],
   "source": [
    "def sample_scoring(no_of_records_to_score = 1):\n",
    "    records_list=[]\n",
    "    payload_scoring = get_scoring_payload(no_of_records_to_score)\n",
    "    scoring_response = wml_client.deployments.score(deployment_uid, payload_scoring)\n",
    "    print('Single record scoring result:', '\\n fields:', scoring_response['predictions'][0]['fields'], '\\n values: ', scoring_response['predictions'][0]['values'][0])\n",
    "    print(json.dumps(scoring_response, indent=None))\n",
    "    return payload_scoring, scoring_response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "id": "fccba0eaa7794d6c9bb5990c435aa563"
   },
   "outputs": [],
   "source": [
    "from ibm_watson_openscale.supporting_classes.payload_record import PayloadRecord\n",
    "def payload_logging(no_of_records_to_score = 1):\n",
    "    records_list=[]\n",
    "    payload_scoring = get_scoring_payload(no_of_records_to_score)\n",
    "    \n",
    "    \n",
    "    scoring_response = wml_client.deployments.score(deployment_uid, payload_scoring)\n",
    "    time.sleep(5)\n",
    "    pl_records_count = wos_client.data_sets.get_records_count(payload_data_set_id)\n",
    "    print(\"Number of records in the payload logging table: {}\".format(pl_records_count))\n",
    "    if pl_records_count == 0:\n",
    "        print(\"Payload logging did not happen, performing explicit payload logging.\")\n",
    "    \n",
    "        #manual PL logging if automated logging does not work\n",
    "        score_input=payload_scoring['input_data'][0]\n",
    "        score_response=scoring_response['predictions'][0]\n",
    "        pl_record = PayloadRecord(request=score_input, response=score_response, response_time=int(460))\n",
    "        records_list.append(pl_record)\n",
    "        wos_client.data_sets.store_records(data_set_id = payload_data_set_id, request_body=records_list)\n",
    "        \n",
    "        \n",
    "        time.sleep(5)\n",
    "        pl_records_count = wos_client.data_sets.get_records_count(payload_data_set_id)\n",
    "        print(\"Number of records in the payload logging table: {}\".format(pl_records_count))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bd59a8e3d0bb431eb3be9252b1f3772c"
   },
   "source": [
    "## Score the model and print the scoring response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "id": "9b1853a125a340e99c622b9f71167c9c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Single record scoring result: \n",
      " fields: ['workclass', 'fnlwgt', 'education', 'education-num', 'Marital', 'occupation', 'relationship', 'capitalgain', 'loss', 'hoursper', 'citizen_status', 'workclass_IX', 'workclassclassVec', 'education_IX', 'educationclassVec', 'Marital_IX', 'MaritalclassVec', 'occupation_IX', 'occupationclassVec', 'relationship_IX', 'relationshipclassVec', 'citizen_status_IX', 'citizen_statusclassVec', 'features', 'rawPrediction', 'probability', 'prediction', 'predictedLabel'] \n",
      " values:  ['State-gov', 77516, 'Bachelors', 13, 'Never-married', 'Adm-clerical', 'Not-in-family', 2174, 0, 40, 'United-States', 4.0, [9, [4], [1.0]], 2.0, [16, [2], [1.0]], 1.0, [7, [1], [1.0]], 3.0, [15, [3], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [4, 11, 26, 35, 48, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 77516.0, 13.0, 2174.0, 40.0]], [17.120135477812976, 2.879864522187024], [0.8560067738906488, 0.1439932261093512], 0.0, '<=50K']\n",
      "{\"predictions\": [{\"fields\": [\"workclass\", \"fnlwgt\", \"education\", \"education-num\", \"Marital\", \"occupation\", \"relationship\", \"capitalgain\", \"loss\", \"hoursper\", \"citizen_status\", \"workclass_IX\", \"workclassclassVec\", \"education_IX\", \"educationclassVec\", \"Marital_IX\", \"MaritalclassVec\", \"occupation_IX\", \"occupationclassVec\", \"relationship_IX\", \"relationshipclassVec\", \"citizen_status_IX\", \"citizen_statusclassVec\", \"features\", \"rawPrediction\", \"probability\", \"prediction\", \"predictedLabel\"], \"values\": [[\"State-gov\", 77516, \"Bachelors\", 13, \"Never-married\", \"Adm-clerical\", \"Not-in-family\", 2174, 0, 40, \"United-States\", 4.0, [9, [4], [1.0]], 2.0, [16, [2], [1.0]], 1.0, [7, [1], [1.0]], 3.0, [15, [3], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [4, 11, 26, 35, 48, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 77516.0, 13.0, 2174.0, 40.0]], [17.120135477812976, 2.879864522187024], [0.8560067738906488, 0.1439932261093512], 0.0, \"<=50K\"]]}]}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "({'input_data': [{'fields': ['workclass',\n",
       "     'fnlwgt',\n",
       "     'education',\n",
       "     'education-num',\n",
       "     'Marital',\n",
       "     'occupation',\n",
       "     'relationship',\n",
       "     'capitalgain',\n",
       "     'loss',\n",
       "     'hoursper',\n",
       "     'citizen_status'],\n",
       "    'values': [['State-gov',\n",
       "      77516,\n",
       "      'Bachelors',\n",
       "      13,\n",
       "      'Never-married',\n",
       "      'Adm-clerical',\n",
       "      'Not-in-family',\n",
       "      2174,\n",
       "      0,\n",
       "      40,\n",
       "      'United-States']],\n",
       "    'meta': {'fields': ['race', 'age', 'sex'],\n",
       "     'values': [['White', 39, 'Male']]}}]},\n",
       " {'predictions': [{'fields': ['workclass',\n",
       "     'fnlwgt',\n",
       "     'education',\n",
       "     'education-num',\n",
       "     'Marital',\n",
       "     'occupation',\n",
       "     'relationship',\n",
       "     'capitalgain',\n",
       "     'loss',\n",
       "     'hoursper',\n",
       "     'citizen_status',\n",
       "     'workclass_IX',\n",
       "     'workclassclassVec',\n",
       "     'education_IX',\n",
       "     'educationclassVec',\n",
       "     'Marital_IX',\n",
       "     'MaritalclassVec',\n",
       "     'occupation_IX',\n",
       "     'occupationclassVec',\n",
       "     'relationship_IX',\n",
       "     'relationshipclassVec',\n",
       "     'citizen_status_IX',\n",
       "     'citizen_statusclassVec',\n",
       "     'features',\n",
       "     'rawPrediction',\n",
       "     'probability',\n",
       "     'prediction',\n",
       "     'predictedLabel'],\n",
       "    'values': [['State-gov',\n",
       "      77516,\n",
       "      'Bachelors',\n",
       "      13,\n",
       "      'Never-married',\n",
       "      'Adm-clerical',\n",
       "      'Not-in-family',\n",
       "      2174,\n",
       "      0,\n",
       "      40,\n",
       "      'United-States',\n",
       "      4.0,\n",
       "      [9, [4], [1.0]],\n",
       "      2.0,\n",
       "      [16, [2], [1.0]],\n",
       "      1.0,\n",
       "      [7, [1], [1.0]],\n",
       "      3.0,\n",
       "      [15, [3], [1.0]],\n",
       "      1.0,\n",
       "      [6, [1], [1.0]],\n",
       "      0.0,\n",
       "      [42, [0], [1.0]],\n",
       "      [100,\n",
       "       [4, 11, 26, 35, 48, 53, 95, 96, 97, 99],\n",
       "       [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 77516.0, 13.0, 2174.0, 40.0]],\n",
       "      [17.120135477812976, 2.879864522187024],\n",
       "      [0.8560067738906488, 0.1439932261093512],\n",
       "      0.0,\n",
       "      '<=50K']]}]})"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_scoring(no_of_records_to_score = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7056440604434d4f8b9429befee9bdce"
   },
   "source": [
    "# Configure OpenScale \n",
    "\n",
    "The notebook will now import the necessary libraries and set up a Python OpenScale client."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "id": "e577b95129004af19aeccbb4ae4a7862"
   },
   "outputs": [],
   "source": [
    "from ibm_watson_openscale import APIClient\n",
    "from ibm_watson_openscale.utils import *\n",
    "from ibm_watson_openscale.supporting_classes import *\n",
    "from ibm_watson_openscale.supporting_classes.enums import *\n",
    "from ibm_cloud_sdk_core.authenticators import CloudPakForDataAuthenticator\n",
    "\n",
    "import json\n",
    "import requests\n",
    "import base64\n",
    "from requests.auth import HTTPBasicAuth\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3043544c667e49d38cd70855d8ccb5ad"
   },
   "source": [
    "## Get a instance of the OpenScale SDK client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "id": "d3d5b4df0a9f4d598a6cbd595700ae53"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'3.0.39'"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "authenticator = CloudPakForDataAuthenticator(\n",
    "        url=WOS_CREDENTIALS['url'],\n",
    "        username=WOS_CREDENTIALS['username'],\n",
    "        password=WOS_CREDENTIALS['password'],\n",
    "        disable_ssl_verification=True\n",
    "    )\n",
    "\n",
    "wos_client = APIClient(service_url=WOS_CREDENTIALS['url'],authenticator=authenticator)\n",
    "wos_client.version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cbebb627856a43cbb607cf844568a9f3"
   },
   "source": [
    "## Create datamart"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5dbc59b8112144b8a06e6c41a8301f0a"
   },
   "source": [
    "### Set up datamart\n",
    "\n",
    "Watson OpenScale uses a database to store payload logs and calculated metrics. If database credentials were supplied, the datamart will be created there unless there is an existing datamart and the KEEP_MY_INTERNAL_POSTGRES variable is set to True. If an OpenScale datamart exists in Db2 or PostgreSQL, the existing datamart will be used and no data will be overwritten.\n",
    "\n",
    "Prior instances of the model will be removed from OpenScale monitoring."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "id": "ca368ed3793443da944e24b8b5e0d281"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>Data Marts</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>name</th><th style='border: 1px solid #dddddd'>description</th><th style='border: 1px solid #dddddd'>internal_database</th><th style='border: 1px solid #dddddd'>status</th><th style='border: 1px solid #dddddd'>created_at</th><th style='border: 1px solid #dddddd'>id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>AIOSFASTPATHICP-00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>Data Mart created by OpenScale ExpressPath</td><td style='border: 1px solid #dddddd'>False</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-06-04 05:19:03.698000+00:00</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "wos_client.data_marts.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "id": "2966db92a2994a958136635144f05572"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using existing datamart 00000000-0000-0000-0000-000000000000\n"
     ]
    }
   ],
   "source": [
    "data_marts = wos_client.data_marts.list().result.data_marts\n",
    "if len(data_marts) == 0:\n",
    "    if DB_CREDENTIALS is not None:\n",
    "        if SCHEMA_NAME is None: \n",
    "            print(\"Please specify the SCHEMA_NAME and rerun the cell\")\n",
    "\n",
    "        print('Setting up external datamart')\n",
    "        added_data_mart_result = wos_client.data_marts.add(\n",
    "                background_mode=False,\n",
    "                name=\"WOS Data Mart\",\n",
    "                description=\"Data Mart created by WOS tutorial notebook\",\n",
    "                database_configuration=DatabaseConfigurationRequest(\n",
    "                  database_type=DatabaseType.DB2,\n",
    "                    credentials=PrimaryStorageCredentialsLong(\n",
    "                        hostname=DATABASE_CREDENTIALS['hostname'],\n",
    "                        username=DATABASE_CREDENTIALS['username'],\n",
    "                        password=DATABASE_CREDENTIALS['password'],\n",
    "                        db=DATABASE_CREDENTIALS['database'],\n",
    "                        port=DATABASE_CREDENTIALS['port'],\n",
    "                        ssl=DATABASE_CREDENTIALS['ssl'],\n",
    "                        sslmode=DATABASE_CREDENTIALS['sslmode'],\n",
    "                        certificate_base64=DATABASE_CREDENTIALS['certificate_base64']\n",
    "                    ),\n",
    "                    location=LocationSchemaName(\n",
    "                        schema_name= SCHEMA_NAME\n",
    "                    )\n",
    "                )\n",
    "             ).result\n",
    "    else:\n",
    "        print('Setting up internal datamart')\n",
    "        added_data_mart_result = wos_client.data_marts.add(\n",
    "                background_mode=False,\n",
    "                name=\"WOS Data Mart\",\n",
    "                description=\"Data Mart created by WOS tutorial notebook\", \n",
    "                internal_database = True).result\n",
    "        \n",
    "    data_mart_id = added_data_mart_result.metadata.id\n",
    "    \n",
    "else:\n",
    "    data_mart_id=data_marts[0].metadata.id\n",
    "    print('Using existing datamart {}'.format(data_mart_id))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "id": "5862cdb82a534375b7e290cee338774e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'metadata': {'id': '00000000-0000-0000-0000-000000000000',\n",
       "  'crn': 'crn:v1:bluemix:public:aiopenscale:us-south:a/na:00000000-0000-0000-0000-000000000000:data_mart:00000000-0000-0000-0000-000000000000',\n",
       "  'url': '/v2/data_marts/00000000-0000-0000-0000-000000000000',\n",
       "  'created_at': '2024-06-04T05:19:03.698000Z',\n",
       "  'created_by': 'cpadmin',\n",
       "  'modified_at': '2024-06-04T05:19:11.402000Z',\n",
       "  'modified_by': 'cpadmin'},\n",
       " 'entity': {'name': 'AIOSFASTPATHICP-00000000-0000-0000-0000-000000000000',\n",
       "  'description': 'Data Mart created by OpenScale ExpressPath',\n",
       "  'service_instance_crn': 'N/A',\n",
       "  'internal_database': False,\n",
       "  'database_configuration': {'database_type': 'db2',\n",
       "   'credentials': {'secret_id': '3bd1e5c0-6987-4719-a9f8-7dc94bfac74d'},\n",
       "   'location': {'schema_name': 'AIOSFASTPATHICP-00000000-0000-0000-0000-000000000000'}},\n",
       "  'status': {'state': 'active'}}}"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_mart_details = wos_client.data_marts.list().result.data_marts[0]\n",
    "data_mart_details.to_dict()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "id": "b61551bb00fa430b8595748c93a2dfb9"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>Service Providers</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>instance_id</th><th style='border: 1px solid #dddddd'>status</th><th style='border: 1px solid #dddddd'>name</th><th style='border: 1px solid #dddddd'>service_type</th><th style='border: 1px solid #dddddd'>created_at</th><th style='border: 1px solid #dddddd'>id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>IAE7</td><td style='border: 1px solid #dddddd'>custom_machine_learning</td><td style='border: 1px solid #dddddd'>2024-07-02 09:08:06.273000+00:00</td><td style='border: 1px solid #dddddd'>184e73a2-7fd8-4f3f-b994-bb648f6eb8ec</td></tr><tr><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>IAE6</td><td style='border: 1px solid #dddddd'>custom_machine_learning</td><td style='border: 1px solid #dddddd'>2024-07-02 09:04:22.643000+00:00</td><td style='border: 1px solid #dddddd'>644befcd-6d36-4f4d-a30a-cd51a28b63fe</td></tr><tr><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>WML_IAE5</td><td style='border: 1px solid #dddddd'>custom_machine_learning</td><td style='border: 1px solid #dddddd'>2024-07-02 08:43:08.723000+00:00</td><td style='border: 1px solid #dddddd'>e986a0d7-8187-4fed-ab2a-c614a9683cae</td></tr><tr><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>WML_IAE4</td><td style='border: 1px solid #dddddd'>custom_machine_learning</td><td style='border: 1px solid #dddddd'>2024-07-02 07:00:41.816000+00:00</td><td style='border: 1px solid #dddddd'>d90c6bf2-49c6-4179-9876-8b85b0247d95</td></tr><tr><td style='border: 1px solid #dddddd'>99999999-9999-9999-9999-999999999999</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>Image Multiclass Watson Machine Learning V2_test</td><td style='border: 1px solid #dddddd'>watson_machine_learning</td><td style='border: 1px solid #dddddd'>2024-07-01 17:17:14.696000+00:00</td><td style='border: 1px solid #dddddd'>a7ca157a-de07-457a-8c4c-b1a2e998699c</td></tr><tr><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>WML_remote_spark_jdbc</td><td style='border: 1px solid #dddddd'>custom_machine_learning</td><td style='border: 1px solid #dddddd'>2024-07-01 09:40:35.190000+00:00</td><td style='border: 1px solid #dddddd'>26b2af15-e396-4295-81d2-6912ab93912b</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>WML_IAE3</td><td style='border: 1px solid #dddddd'>watson_machine_learning</td><td style='border: 1px solid #dddddd'>2024-06-30 11:22:00.507000+00:00</td><td style='border: 1px solid #dddddd'>d7ad1164-a387-462d-b495-dd25d6241404</td></tr><tr><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>OpenScale Headless Service Provider</td><td style='border: 1px solid #dddddd'>custom_machine_learning</td><td style='border: 1px solid #dddddd'>2024-06-29 03:21:50.385000+00:00</td><td style='border: 1px solid #dddddd'>e67332c4-0f88-4a8f-9c41-396d28c07448</td></tr><tr><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>IAE3</td><td style='border: 1px solid #dddddd'>custom_machine_learning</td><td style='border: 1px solid #dddddd'>2024-06-28 05:11:53.191000+00:00</td><td style='border: 1px solid #dddddd'>8ac485c9-66c8-4a9c-9786-57d2940f26e8</td></tr><tr><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>WML_IAE2</td><td style='border: 1px solid #dddddd'>custom_machine_learning</td><td style='border: 1px solid #dddddd'>2024-06-27 13:36:27.009000+00:00</td><td style='border: 1px solid #dddddd'>8b65752d-0dc9-4715-a2a2-96ee19fb7ded</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: First 10 records were displayed.\n"
     ]
    }
   ],
   "source": [
    "wos_client.service_providers.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b1ecf806072240cd89af4324cb844744"
   },
   "source": [
    "## Remove existing service provider connected with used WML instance.\n",
    "\n",
    "Multiple service providers for the same engine instance are avaiable in Watson OpenScale. To avoid multiple service providers of used WML instance in the tutorial notebook the following code deletes existing service provder(s) and then adds new one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "id": "34534f784ac845438b54c6947d227ccb"
   },
   "outputs": [],
   "source": [
    "SERVICE_PROVIDER_NAME = \"Watson Machine Learning - Indirect Bias Demo\"\n",
    "SERVICE_PROVIDER_DESCRIPTION = \"Added by tutorial WOS notebook to showcase Indirect Bias functionality.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "id": "ebd41ed59d0a44b3837152bdd67e10fc"
   },
   "outputs": [],
   "source": [
    "service_providers = wos_client.service_providers.list().result.service_providers\n",
    "for service_provider in service_providers:\n",
    "    service_instance_name = service_provider.entity.name\n",
    "    if service_instance_name == SERVICE_PROVIDER_NAME:\n",
    "        service_provider_id = service_provider.metadata.id\n",
    "        wos_client.service_providers.delete(service_provider_id)\n",
    "        print(\"Deleted existing service_provider for WML instance: {}\".format(service_provider_id))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "48cf5770290744c28274669a5af0103b"
   },
   "source": [
    "## Add service provider\n",
    "\n",
    "Watson OpenScale needs to be bound to the Watson Machine Learning instance to capture payload data into and out of the model.\n",
    "Note: You can bind more than one engine instance if needed by calling wos_client.service_providers.add method. Next, you can refer to particular service provider using service_provider_id."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "id": "2d260d321cad475682aea0857c9b1054"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "=================================================================================\n",
      "\n",
      " Waiting for end of adding service provider c3601bc6-22c9-45a1-b016-c29cf0bf64b4 \n",
      "\n",
      "=================================================================================\n",
      "\n",
      "\n",
      "\n",
      "active\n",
      "\n",
      "-----------------------------------------------\n",
      " Successfully finished adding service provider \n",
      "-----------------------------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "added_service_provider_result = wos_client.service_providers.add(\n",
    "        name=SERVICE_PROVIDER_NAME,\n",
    "        description=SERVICE_PROVIDER_DESCRIPTION,\n",
    "        service_type=ServiceTypes.WATSON_MACHINE_LEARNING,\n",
    "        deployment_space_id = WML_SPACE_ID,\n",
    "        operational_space_id = \"production\",\n",
    "        credentials=WMLCredentialsCP4D(),\n",
    "        background_mode=False\n",
    "    ).result\n",
    "service_provider_id = added_service_provider_result.metadata.id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "id": "eae202864f724fb9880ed31fadf75ac0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"metadata\": {\n",
      "    \"id\": \"c3601bc6-22c9-45a1-b016-c29cf0bf64b4\",\n",
      "    \"crn\": \"crn:v1:bluemix:public:aiopenscale:us-south:a/na:00000000-0000-0000-0000-000000000000:service_provider:c3601bc6-22c9-45a1-b016-c29cf0bf64b4\",\n",
      "    \"url\": \"/v2/service_providers/c3601bc6-22c9-45a1-b016-c29cf0bf64b4\",\n",
      "    \"created_at\": \"2024-08-06T13:25:37.046000Z\",\n",
      "    \"created_by\": \"cpadmin\"\n",
      "  },\n",
      "  \"entity\": {\n",
      "    \"name\": \"Watson Machine Learning - Indirect Bias Demo-2\",\n",
      "    \"service_type\": \"watson_machine_learning\",\n",
      "    \"instance_id\": \"99999999-9999-9999-9999-999999999999\",\n",
      "    \"credentials\": {\n",
      "      \"secret_id\": \"b5bc1437-c72a-4efc-b06c-8e59934ed321\"\n",
      "    },\n",
      "    \"operational_space_id\": \"production\",\n",
      "    \"deployment_space_id\": \"e8b87647-a0e9-4932-920f-1e8f1d1f383d\",\n",
      "    \"status\": {\n",
      "      \"state\": \"active\"\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print(wos_client.service_providers.get(service_provider_id).result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "id": "4ced8049e546405e81129ada3cbf73d1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'metadata': {'guid': '038b8c68-6615-4c0d-8d46-15783d6ab9c8',\n",
       "  'created_at': '2024-08-06T13:25:16.409Z',\n",
       "  'modified_at': '2024-08-06T13:25:16.409Z'},\n",
       " 'entity': {'name': 'Adult Census Income Classifier Deployment',\n",
       "  'type': 'online',\n",
       "  'scoring_endpoint': {'url': 'https://internal-nginx-svc:12443/ml/v4/deployments/038b8c68-6615-4c0d-8d46-15783d6ab9c8/predictions'},\n",
       "  'asset': {},\n",
       "  'asset_properties': {}}}"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "asset_deployment_details = wos_client.service_providers.list_assets(data_mart_id=data_mart_id, service_provider_id=service_provider_id, deployment_id=deployment_uid,deployment_space_id = WML_SPACE_ID).result['resources'][0]\n",
    "asset_deployment_details"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "id": "5ed2cbb8cfbd40518178d204be799ecb"
   },
   "outputs": [],
   "source": [
    "model_asset_details_from_deployment=wos_client.service_providers.get_deployment_asset(data_mart_id=data_mart_id,service_provider_id=service_provider_id,deployment_id=deployment_uid,deployment_space_id=WML_SPACE_ID)\n",
    "#model_asset_details_from_deployment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "09f4993d2caa48269b9832c7e9b145d6"
   },
   "source": [
    "## Subscriptions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2e59906593c44bda97f79a9c35e728f6"
   },
   "source": [
    "Remove existing credit risk subscriptions\n",
    "\n",
    "This code removes previous subscriptions to the model to refresh the monitors with the new model and new data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "id": "6b97d781684b401095188860bd0a3c96"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>Subscriptions</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>asset_id</th><th style='border: 1px solid #dddddd'>asset_type</th><th style='border: 1px solid #dddddd'>asset_name</th><th style='border: 1px solid #dddddd'>data_mart_id</th><th style='border: 1px solid #dddddd'>deployment_id</th><th style='border: 1px solid #dddddd'>deployment_name</th><th style='border: 1px solid #dddddd'>service_provider_id</th><th style='border: 1px solid #dddddd'>status</th><th style='border: 1px solid #dddddd'>created_at</th><th style='border: 1px solid #dddddd'>id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>438ca544-9bd1-48c2-8e8d-3de4ef4ca79b</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>WML_IAE4</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>78a0af9e-1014-4fb1-b22a-5e11f4fd70e7</td><td style='border: 1px solid #dddddd'>WML_IAE4</td><td style='border: 1px solid #dddddd'>d90c6bf2-49c6-4179-9876-8b85b0247d95</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-02 07:03:31.504000+00:00</td><td style='border: 1px solid #dddddd'>e34b9b87-b6e1-4c53-b92e-cb80dea042be</td></tr><tr><td style='border: 1px solid #dddddd'>592b902d-3dc9-4e56-8bcb-86cbf1a6d8a9</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>gcr - P2 XGB Classifier - Model</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>2b976af0-e4ab-4859-af7d-2f2287d864ad</td><td style='border: 1px solid #dddddd'>gcr model</td><td style='border: 1px solid #dddddd'>4d2f2fb2-6b64-4d58-8f13-257166e468e9</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-17 07:11:44.727000+00:00</td><td style='border: 1px solid #dddddd'>e2df4ec7-6c75-416f-a444-8d21389f7513</td></tr><tr><td style='border: 1px solid #dddddd'>e3ac9fc3-bccf-4a4e-b37b-490bfb93dd81</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>GCR AutoAI - P2 XGB Classifier - Model</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>6399e6e8-df5a-4370-9af4-34b2f2e76bc6</td><td style='border: 1px solid #dddddd'>GCR Auto AI</td><td style='border: 1px solid #dddddd'>a7ca157a-de07-457a-8c4c-b1a2e998699c</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-03 10:37:20.487000+00:00</td><td style='border: 1px solid #dddddd'>ce36911c-75f6-4f99-b4d2-b71e5d55a802</td></tr><tr><td style='border: 1px solid #dddddd'>592b902d-3dc9-4e56-8bcb-86cbf1a6d8a9</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>gcr - P2 XGB Classifier - Model</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>2b976af0-e4ab-4859-af7d-2f2287d864ad</td><td style='border: 1px solid #dddddd'>gcr model</td><td style='border: 1px solid #dddddd'>4d2f2fb2-6b64-4d58-8f13-257166e468e9</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-02 15:33:17.454000+00:00</td><td style='border: 1px solid #dddddd'>e96278a6-7190-48f3-b8bd-945fa48cfe50</td></tr><tr><td style='border: 1px solid #dddddd'>327d8aea-ecfc-4990-9bb9-601a1695094d</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>GCR AutoAI - P2 XGB Classifier - Model</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>755c3e75-24b5-4839-8a8f-3f85c07a40c9</td><td style='border: 1px solid #dddddd'>GCR demo</td><td style='border: 1px solid #dddddd'>a7ca157a-de07-457a-8c4c-b1a2e998699c</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-02 12:03:14.224000+00:00</td><td style='border: 1px solid #dddddd'>a0f86241-8bfc-4322-895a-2597512e1653</td></tr><tr><td style='border: 1px solid #dddddd'>b21904ef-7478-4dae-b93f-c120e95c9200</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>My SDK Batch Subscription-db2</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>a10a121b-2394-4fe7-9a2d-f0520abd212c</td><td style='border: 1px solid #dddddd'>My SDK Batch Subscription-db2</td><td style='border: 1px solid #dddddd'>644befcd-6d36-4f4d-a30a-cd51a28b63fe</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-02 09:04:38.883000+00:00</td><td style='border: 1px solid #dddddd'>70c9c394-2530-4c0f-b0b2-6b81e44612d0</td></tr><tr><td style='border: 1px solid #dddddd'>7bf1d492-3275-405d-9b61-da4e2075e746</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>My SDK Batch Subscription-db2</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>a302d0d4-3f38-4ded-8688-3e4b2e9bb55c</td><td style='border: 1px solid #dddddd'>My SDK Batch Subscription-db2</td><td style='border: 1px solid #dddddd'>e986a0d7-8187-4fed-ab2a-c614a9683cae</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-02 08:43:25.526000+00:00</td><td style='border: 1px solid #dddddd'>f4178567-4f9a-4b0e-b255-2babdb8b5f38</td></tr><tr><td style='border: 1px solid #dddddd'>405fe789-e294-42ce-9989-774d484205c3</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>WML_remote_spark_jdbc</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>ea8d064d-2eb9-48a0-97af-f385dd358843</td><td style='border: 1px solid #dddddd'>WML_remote_spark_jdbc</td><td style='border: 1px solid #dddddd'>26b2af15-e396-4295-81d2-6912ab93912b</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-01 09:41:24.463000+00:00</td><td style='border: 1px solid #dddddd'>3046b546-e422-4737-9123-a26e025d3cef</td></tr><tr><td style='border: 1px solid #dddddd'>d1869f46-b9a0-4e99-bfbe-491074a4e40d</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>MNIST Model</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>211b2ebc-ec74-4090-a526-e896d5ed6166</td><td style='border: 1px solid #dddddd'>MNIST Model deployment</td><td style='border: 1px solid #dddddd'>a7ca157a-de07-457a-8c4c-b1a2e998699c</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-07-01 17:17:44.331000+00:00</td><td style='border: 1px solid #dddddd'>1b8ebb3c-e69a-4cc1-9a0a-57424546cab3</td></tr><tr><td style='border: 1px solid #dddddd'>592b902d-3dc9-4e56-8bcb-86cbf1a6d8a9</td><td style='border: 1px solid #dddddd'>model</td><td style='border: 1px solid #dddddd'>gcr - P2 XGB Classifier - Model</td><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>2b976af0-e4ab-4859-af7d-2f2287d864ad</td><td style='border: 1px solid #dddddd'>gcr model</td><td style='border: 1px solid #dddddd'>4d2f2fb2-6b64-4d58-8f13-257166e468e9</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>2024-06-30 12:02:35.155000+00:00</td><td style='border: 1px solid #dddddd'>7585a078-c1b6-4a77-9ab3-87be9fb6026c</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: First 10 records were displayed.\n"
     ]
    }
   ],
   "source": [
    "wos_client.subscriptions.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b0d085c8a875458b834992abd3577219"
   },
   "source": [
    "## Remove the existing subscription"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "id": "f5ef621a07cb4e468fca6cc62d22595e"
   },
   "outputs": [],
   "source": [
    "subscriptions = wos_client.subscriptions.list().result.subscriptions\n",
    "for subscription in subscriptions:\n",
    "    sub_model_id = subscription.entity.asset.asset_id\n",
    "    if sub_model_id == model_uid:\n",
    "        wos_client.subscriptions.delete(subscription.metadata.id)\n",
    "        print('Deleted existing subscription for model', sub_model_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "d117477d751b4d6380d2feceac14450e"
   },
   "source": [
    "This code creates the model subscription in OpenScale using the Python client API. Note that we need to provide the model unique identifier, and some information about the model itself."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "id": "e3d6cdba01c140a090a4e3e27a12bdcf"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['workclass',\n",
       " 'education',\n",
       " 'Marital',\n",
       " 'occupation',\n",
       " 'relationship',\n",
       " 'citizen_status',\n",
       " 'fnlwgt',\n",
       " 'education-num',\n",
       " 'capitalgain',\n",
       " 'loss',\n",
       " 'hoursper']"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_columns = cat_features + num_features\n",
    "feature_columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "id": "b7e200c3388d42c087f937ca745938c9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "subscription_id: c274e76c-6bb9-43ce-ba64-127a3db95ede\n"
     ]
    }
   ],
   "source": [
    "subscription_details = wos_client.subscriptions.add(\n",
    "        data_mart_id=data_mart_id,\n",
    "        service_provider_id=service_provider_id,\n",
    "        asset=Asset(\n",
    "            asset_id=model_asset_details_from_deployment[\"entity\"][\"asset\"][\"asset_id\"],\n",
    "            name=model_asset_details_from_deployment[\"entity\"][\"asset\"][\"name\"],\n",
    "            url=model_asset_details_from_deployment[\"entity\"][\"asset\"][\"url\"],\n",
    "            asset_type=AssetTypes.MODEL,\n",
    "            input_data_type=InputDataType.STRUCTURED,\n",
    "            problem_type=ProblemType.BINARY_CLASSIFICATION\n",
    "        ),\n",
    "        deployment=AssetDeploymentRequest(\n",
    "            deployment_id=asset_deployment_details['metadata']['guid'],\n",
    "            name=asset_deployment_details['entity']['name'],\n",
    "            deployment_type= DeploymentTypes.ONLINE,\n",
    "            url=asset_deployment_details['entity']['scoring_endpoint']['url']\n",
    "        ),\n",
    "        asset_properties=AssetPropertiesRequest(\n",
    "            label_column=\"label\",\n",
    "            probability_fields=[\"probability\"],\n",
    "            prediction_field=\"predictedLabel\",\n",
    "            feature_fields = feature_columns,\n",
    "            categorical_fields = cat_features,\n",
    "            training_data_reference=TrainingDataReference(type=\"cos\",\n",
    "                                                          location=COSTrainingDataReferenceLocation(bucket = BUCKET_NAME,\n",
    "                                                                                                    file_name = FILE_NAME),\n",
    "                                                          connection=COSTrainingDataReferenceConnection.from_dict({\n",
    "                                                                        \"resource_instance_id\": COS_RESOURCE_CRN,\n",
    "                                                                        \"url\": COS_ENDPOINT,\n",
    "                                                                        \"api_key\": COS_API_KEY_ID,\n",
    "                                                                        \"iam_url\": IAM_URL})),\n",
    "            training_data_schema=SparkStruct.from_dict(model_asset_details_from_deployment[\"entity\"][\"asset_properties\"][\"training_data_schema\"])\n",
    "        )\n",
    "    ).result\n",
    "subscription_id = subscription_details.metadata.id\n",
    "print('subscription_id: ' + subscription_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "id": "9dee41fc8ae2407483f7d08f7ecd9663"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Payload data set id: 4b2d90a6-b7b2-49b8-9fa8-ee01e03d7d13\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "time.sleep(5)\n",
    "payload_data_set_id = None\n",
    "payload_data_set_id = wos_client.data_sets.list(type=DataSetTypes.PAYLOAD_LOGGING, \n",
    "                                                target_target_id=subscription_id, \n",
    "                                                target_target_type=TargetTypes.SUBSCRIPTION).result.data_sets[0].metadata.id\n",
    "if payload_data_set_id is None:\n",
    "    print(\"Payload data set not found. Please check subscription status.\")\n",
    "else:\n",
    "    print(\"Payload data set id:\", payload_data_set_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "id": "2e2d47b3928c4f6c87fe3379168f2301"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>Data sets</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>data_mart_id</th><th style='border: 1px solid #dddddd'>status</th><th style='border: 1px solid #dddddd'>target_id</th><th style='border: 1px solid #dddddd'>target_type</th><th style='border: 1px solid #dddddd'>type</th><th style='border: 1px solid #dddddd'>created_at</th><th style='border: 1px solid #dddddd'>id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>payload_logging_error</td><td style='border: 1px solid #dddddd'>2024-08-06 13:26:11.281000+00:00</td><td style='border: 1px solid #dddddd'>857c4ab8-1dc5-455f-8603-5de875cfba52</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>model_health</td><td style='border: 1px solid #dddddd'>2024-08-06 13:26:11.262000+00:00</td><td style='border: 1px solid #dddddd'>16f42ee3-0e97-4e29-9056-25fe654c96d0</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>manual_labeling</td><td style='border: 1px solid #dddddd'>2024-08-06 13:26:11.076000+00:00</td><td style='border: 1px solid #dddddd'>0650fc95-c136-4c31-883c-2dd5e3307d7e</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>payload_logging</td><td style='border: 1px solid #dddddd'>2024-08-06 13:26:10.877000+00:00</td><td style='border: 1px solid #dddddd'>4b2d90a6-b7b2-49b8-9fa8-ee01e03d7d13</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>e34b9b87-b6e1-4c53-b92e-cb80dea042be</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>payload_logging</td><td style='border: 1px solid #dddddd'>2024-07-02 07:03:32.634000+00:00</td><td style='border: 1px solid #dddddd'>0063481a-26eb-4934-85a7-4ebc71a2fea0</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>e96278a6-7190-48f3-b8bd-945fa48cfe50</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>model_health</td><td style='border: 1px solid #dddddd'>2024-07-02 15:33:20.908000+00:00</td><td style='border: 1px solid #dddddd'>0189e173-0d7f-41da-baaa-20f022f362f8</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>7c0c6db0-0c7f-415b-85f1-cae28daded5b</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>model_health</td><td style='border: 1px solid #dddddd'>2024-06-14 03:16:37.811000+00:00</td><td style='border: 1px solid #dddddd'>e56a3698-37d4-49cd-87fd-34b9be83b343</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>e2df4ec7-6c75-416f-a444-8d21389f7513</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>payload_logging</td><td style='border: 1px solid #dddddd'>2024-07-17 07:11:46.741000+00:00</td><td style='border: 1px solid #dddddd'>9d6c26da-0e8a-4ec1-a077-30595e96797c</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>e2df4ec7-6c75-416f-a444-8d21389f7513</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>explanations</td><td style='border: 1px solid #dddddd'>2024-07-17 12:14:38.155000+00:00</td><td style='border: 1px solid #dddddd'>884d6811-c36a-4756-bc7b-45c7f0220a44</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>e2df4ec7-6c75-416f-a444-8d21389f7513</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>payload_logging_error</td><td style='border: 1px solid #dddddd'>2024-07-17 07:11:47.156000+00:00</td><td style='border: 1px solid #dddddd'>41ef8863-2f81-42d4-97e2-814a4cf31a2f</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: First 10 records were displayed.\n"
     ]
    }
   ],
   "source": [
    "wos_client.data_sets.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "id": "5ba479997dca4e2899796ba7c050fed5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'metadata': {'id': 'c274e76c-6bb9-43ce-ba64-127a3db95ede',\n",
       "  'crn': 'crn:v1:bluemix:public:aiopenscale:us-south:a/na:00000000-0000-0000-0000-000000000000:subscription:c274e76c-6bb9-43ce-ba64-127a3db95ede',\n",
       "  'url': '/v2/subscriptions/c274e76c-6bb9-43ce-ba64-127a3db95ede',\n",
       "  'created_at': '2024-08-06T13:26:09.549000Z',\n",
       "  'created_by': 'cpadmin',\n",
       "  'modified_at': '2024-08-06T13:26:11.503000Z',\n",
       "  'modified_by': 'cpadmin'},\n",
       " 'entity': {'data_mart_id': '00000000-0000-0000-0000-000000000000',\n",
       "  'service_provider_id': 'c3601bc6-22c9-45a1-b016-c29cf0bf64b4',\n",
       "  'asset': {'asset_id': 'eed8b225-19e5-4460-9b6f-7271dc1e3ff2',\n",
       "   'url': 'https://internal-nginx-svc:12443/ml/v4/models/eed8b225-19e5-4460-9b6f-7271dc1e3ff2?space_id=e8b87647-a0e9-4932-920f-1e8f1d1f383d&version=2020-06-12',\n",
       "   'name': 'Adult Census Income Classifier Model',\n",
       "   'asset_type': 'model',\n",
       "   'problem_type': 'binary',\n",
       "   'input_data_type': 'structured'},\n",
       "  'asset_properties': {'training_data_reference': {'secret_id': 'f4dd80ca-40a2-4c0e-89c4-2ce3901eab68'},\n",
       "   'training_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'workclass',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'fnlwgt',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'measure': 'discrete', 'modeling_role': 'feature'},\n",
       "      'name': 'education',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'education-num',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'measure': 'discrete', 'modeling_role': 'feature'},\n",
       "      'name': 'Marital',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'measure': 'discrete', 'modeling_role': 'feature'},\n",
       "      'name': 'occupation',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'measure': 'discrete', 'modeling_role': 'feature'},\n",
       "      'name': 'relationship',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'capitalgain',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'loss',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'hoursper',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'measure': 'discrete', 'modeling_role': 'feature'},\n",
       "      'name': 'citizen_status',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'target'},\n",
       "      'name': 'label',\n",
       "      'nullable': True,\n",
       "      'type': 'string'}],\n",
       "    'id': '1'},\n",
       "   'output_data_schema': {'type': 'struct',\n",
       "    'fields': [{'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'workclass',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'fnlwgt',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'education',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'education-num',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'Marital',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'occupation',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'relationship',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'capitalgain',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'loss',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'modeling_role': 'feature'},\n",
       "      'name': 'hoursper',\n",
       "      'nullable': True,\n",
       "      'type': 'integer'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 64},\n",
       "       'measure': 'discrete',\n",
       "       'modeling_role': 'feature'},\n",
       "      'name': 'citizen_status',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'prediction-probability'},\n",
       "      'name': 'prediction_probability',\n",
       "      'nullable': True,\n",
       "      'type': 'double'},\n",
       "     {'metadata': {'columnInfo': {'columnLength': 128},\n",
       "       'modeling_role': 'record-id',\n",
       "       'primary_key': True},\n",
       "      'name': 'scoring_id',\n",
       "      'nullable': False,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'record-timestamp'},\n",
       "      'name': 'scoring_timestamp',\n",
       "      'nullable': False,\n",
       "      'type': 'timestamp'},\n",
       "     {'metadata': {},\n",
       "      'name': 'deployment_id',\n",
       "      'nullable': False,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {},\n",
       "      'name': 'asset_revision',\n",
       "      'nullable': True,\n",
       "      'type': 'string'},\n",
       "     {'metadata': {'modeling_role': 'probability'},\n",
       "      'name': 'ProbabilityVector',\n",
       "      'nullable': True,\n",
       "      'type': {'containsNull': True,\n",
       "       'elementType': 'double',\n",
       "       'type': 'array'}}]},\n",
       "   'label_column': 'label',\n",
       "   'prediction_field': 'predictedLabel',\n",
       "   'feature_fields': ['workclass',\n",
       "    'education',\n",
       "    'Marital',\n",
       "    'occupation',\n",
       "    'relationship',\n",
       "    'citizen_status',\n",
       "    'fnlwgt',\n",
       "    'education-num',\n",
       "    'capitalgain',\n",
       "    'loss',\n",
       "    'hoursper'],\n",
       "   'categorical_fields': ['workclass',\n",
       "    'education',\n",
       "    'Marital',\n",
       "    'occupation',\n",
       "    'relationship',\n",
       "    'citizen_status'],\n",
       "   'probability_fields': ['probability']},\n",
       "  'deployment': {'deployment_id': '038b8c68-6615-4c0d-8d46-15783d6ab9c8',\n",
       "   'url': 'https://internal-nginx-svc:12443/ml/v4/deployments/038b8c68-6615-4c0d-8d46-15783d6ab9c8/predictions',\n",
       "   'name': 'Adult Census Income Classifier Deployment',\n",
       "   'deployment_type': 'online'},\n",
       "  'status': {'state': 'active'}}}"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wos_client.subscriptions.get(subscription_id).result.to_dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2d75c57ab8914d70acdc3ed523675b1b"
   },
   "source": [
    "# Score the model so we can configure monitors\n",
    "\n",
    "Now that the WML service has been bound and the subscription has been created, we need to send a request to the model before we configure OpenScale. This allows OpenScale to create a payload log in the datamart with the correct schema, so it can capture data coming into and out of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "id": "14ef8cf50f5748589f34ab3c15fe5d2f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of records in the payload logging table: 1000\n"
     ]
    }
   ],
   "source": [
    "payload_logging(no_of_records_to_score = 1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "id": "933b5220cd9d4c5681f54308ce266ade"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of records in the payload logging table: 1000\n"
     ]
    }
   ],
   "source": [
    "time.sleep(5)\n",
    "pl_records_count = wos_client.data_sets.get_records_count(payload_data_set_id)\n",
    "print(\"Number of records in the payload logging table: {}\".format(pl_records_count))\n",
    "if pl_records_count == 0:\n",
    "    raise Exception(\"Payload logging did not happen!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "09c318b2e5c54b839eb6f3ac8b247ca4"
   },
   "source": [
    "## Fairness configuration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3d73b84061cc494a8ea5a19e897b981f"
   },
   "source": [
    "The code below configures fairness monitoring for our model. It turns on monitoring for two features, sex and age. In each case, we must specify:\n",
    "    \n",
    "Which model feature to monitor One or more majority groups, which are values of that feature that we expect to receive a higher percentage of favorable outcomes One or more minority groups, which are values of that feature that we expect to receive a higher percentage of unfavorable outcomes The threshold at which we would like OpenScale to display an alert if the fairness measurement falls below (in this case, 80%) Additionally, we must specify which outcomes from the model are favourable outcomes, and which are unfavourable. We must also provide the number of records OpenScale will use to calculate the fairness score. In this case, OpenScale's fairness monitor will run hourly, but will not calculate a new fairness rating until at least 100 records have been added. Finally, to calculate fairness, OpenScale must perform some calculations on the training data, so we provide the dataframe containing the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f59d4d82ca7d4aa7978e7a777416c281"
   },
   "source": [
    "### Create Fairness Monitor Instance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "id": "1864c05c176f4a718e36059628d7f0ed"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "===================================================================================\n",
      "\n",
      " Waiting for end of monitor instance creation 841a81b1-b4cf-4e98-87de-1d374d43df63 \n",
      "\n",
      "===================================================================================\n",
      "\n",
      "\n",
      "\n",
      "active\n",
      "\n",
      "---------------------------------------\n",
      " Monitor instance successfully created \n",
      "---------------------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "target = Target(\n",
    "    target_type=TargetTypes.SUBSCRIPTION,\n",
    "    target_id=subscription_id\n",
    ")\n",
    "parameters = {\n",
    "    \"features\": [\n",
    "        {\n",
    "            \"feature\": \"sex\",\n",
    "            \"majority\": [\"Male\"],\n",
    "            \"minority\": [\"Female\"]\n",
    "        },\n",
    "        {\n",
    "            \"feature\": \"age\",\n",
    "            \"majority\": [[41,75]],\n",
    "            \"minority\": [[18,33]]\n",
    "        }\n",
    "    ],\n",
    "    \"favourable_class\": [\">50K\"],\n",
    "    \"unfavourable_class\": [\"<=50K\"],\n",
    "    \"min_records\": 1000\n",
    "}\n",
    "thresholds = [\n",
    "    {\n",
    "        \"metric_id\": \"fairness_value\",\n",
    "        \"specific_values\": [\n",
    "            {\n",
    "                \"applies_to\": [\n",
    "                    {\n",
    "                        \"type\": \"tag\",\n",
    "                        \"value\": \"sex\",\n",
    "                        \"key\": \"feature\"\n",
    "                    }\n",
    "                ],\n",
    "                \"value\": 80\n",
    "            },\n",
    "            {\n",
    "                \"applies_to\": [\n",
    "                    {\n",
    "                        \"type\": \"tag\",\n",
    "                        \"value\": \"age\",\n",
    "                        \"key\": \"feature\"\n",
    "                    }\n",
    "                ],\n",
    "                \"value\": 80\n",
    "            }\n",
    "        ],\n",
    "        \"type\": \"lower_limit\",\n",
    "        \"value\": 80\n",
    "    }\n",
    "]\n",
    "fairness_monitor_details = wos_client.monitor_instances.create(\n",
    "    data_mart_id=data_mart_id,\n",
    "    background_mode=False,\n",
    "    monitor_definition_id=wos_client.monitor_definitions.MONITORS.FAIRNESS.ID,\n",
    "    target=target,\n",
    "    parameters=parameters,\n",
    "    thresholds=thresholds\n",
    ").result\n",
    "fairness_monitor_instance_id =fairness_monitor_details.metadata.id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ae999234ea39453e82c74169ac7d74a8"
   },
   "source": [
    "### Get Fairness Monitor Instance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "id": "812cdeea78af4ca8ba38ea5f07efd990"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>Monitor instances</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>data_mart_id</th><th style='border: 1px solid #dddddd'>status</th><th style='border: 1px solid #dddddd'>target_id</th><th style='border: 1px solid #dddddd'>target_type</th><th style='border: 1px solid #dddddd'>monitor_definition_id</th><th style='border: 1px solid #dddddd'>created_at</th><th style='border: 1px solid #dddddd'>id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>fairness</td><td style='border: 1px solid #dddddd'>2024-08-06 13:26:36.564000+00:00</td><td style='border: 1px solid #dddddd'>841a81b1-b4cf-4e98-87de-1d374d43df63</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>model_health</td><td style='border: 1px solid #dddddd'>2024-08-06 13:26:10.720000+00:00</td><td style='border: 1px solid #dddddd'>0f6bfa05-d2f6-4286-8714-827d711d22aa</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>performance</td><td style='border: 1px solid #dddddd'>2024-08-06 13:26:11.762000+00:00</td><td style='border: 1px solid #dddddd'>da29a599-b38d-4f70-a562-43e371fc4e87</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>7c0c6db0-0c7f-415b-85f1-cae28daded5b</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>fairness</td><td style='border: 1px solid #dddddd'>2024-06-14 03:25:06.580000+00:00</td><td style='border: 1px solid #dddddd'>a7160110-6133-4635-8534-6478e96e1fbf</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>9acf1d51-2958-44f8-ade9-d6d7663784ef</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>model_health</td><td style='border: 1px solid #dddddd'>2024-06-20 13:24:56.801000+00:00</td><td style='border: 1px solid #dddddd'>309f5342-a9bd-4bbb-ab22-a109c8deb35c</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>cb83c8bf-9e3b-42bf-9447-c4fb2706b56e</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>model_health</td><td style='border: 1px solid #dddddd'>2024-06-29 03:22:11.907000+00:00</td><td style='border: 1px solid #dddddd'>a1e95409-191f-4382-9128-77fe7f9349db</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>cc0a7062-56ef-4feb-91aa-33764217ad77</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>model_health</td><td style='border: 1px solid #dddddd'>2024-06-20 14:21:30.390000+00:00</td><td style='border: 1px solid #dddddd'>7dcdf7ac-7cba-4a07-914f-b6a58b499d6a</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>1b8ebb3c-e69a-4cc1-9a0a-57424546cab3</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>model_health</td><td style='border: 1px solid #dddddd'>2024-07-01 17:17:45.382000+00:00</td><td style='border: 1px solid #dddddd'>1d0924c1-7f1a-419c-b2bc-bff1f9a5d5d0</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>7c0c6db0-0c7f-415b-85f1-cae28daded5b</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>mrm</td><td style='border: 1px solid #dddddd'>2024-06-14 03:16:37.493000+00:00</td><td style='border: 1px solid #dddddd'>87fce793-9360-4782-8b02-1f83bd4b8b2f</td></tr><tr><td style='border: 1px solid #dddddd'>00000000-0000-0000-0000-000000000000</td><td style='border: 1px solid #dddddd'>active</td><td style='border: 1px solid #dddddd'>7c0c6db0-0c7f-415b-85f1-cae28daded5b</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>model_health</td><td style='border: 1px solid #dddddd'>2024-06-14 03:16:37.532000+00:00</td><td style='border: 1px solid #dddddd'>bf46efe7-3db2-4627-92db-e4119cb9bbe0</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: First 10 records were displayed.\n"
     ]
    }
   ],
   "source": [
    "wos_client.monitor_instances.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "03e3bf32570d4bb5816b860d58db555d"
   },
   "source": [
    "### Get run details\n",
    "In case of production subscription, initial monitoring run is triggered internally. Checking its status"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "id": "6af6098a23384c93b88f113f6b5024b7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "run_status:  finished\n"
     ]
    }
   ],
   "source": [
    "runs = wos_client.monitor_instances.list_runs(fairness_monitor_instance_id, limit=1).result.to_dict()\n",
    "fairness_monitoring_run_id = runs[\"runs\"][0][\"metadata\"][\"id\"]\n",
    "run_status = None\n",
    "while(run_status not in [\"finished\", \"error\"]):\n",
    "    run_details = wos_client.monitor_instances.get_run_details(fairness_monitor_instance_id, fairness_monitoring_run_id).result.to_dict()\n",
    "    run_status = run_details[\"entity\"][\"status\"][\"state\"]\n",
    "    print('run_status: ', run_status)\n",
    "    if run_status in [\"finished\", \"error\"]:\n",
    "        break\n",
    "    time.sleep(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "13e80670e3474f5581ef4de516e19d70"
   },
   "source": [
    "### Fairness run output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "id": "8b9b909ce8104fc4ad51ed3b7f388c5e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'metadata': {'id': '6c53aa46-439a-4008-bbfc-165132dff3ff',\n",
       "  'crn': 'crn:v1:bluemix:public:aiopenscale:us-south:a/na:00000000-0000-0000-0000-000000000000:run:6c53aa46-439a-4008-bbfc-165132dff3ff',\n",
       "  'url': '/v2/monitor_instances/841a81b1-b4cf-4e98-87de-1d374d43df63/runs/6c53aa46-439a-4008-bbfc-165132dff3ff',\n",
       "  'created_at': '2024-08-06T13:26:37.927000Z',\n",
       "  'created_by': 'internal-service'},\n",
       " 'entity': {'triggered_by': 'user',\n",
       "  'parameters': {'is_group_bias_completed': True,\n",
       "   'measurement_id': '0ac72f74-aa61-4412-84fd-718b5e746fa6',\n",
       "   'total_records_processed': 1000},\n",
       "  'status': {'state': 'finished',\n",
       "   'queued_at': '2024-08-06T13:26:37.919000Z',\n",
       "   'started_at': '2024-08-06T13:26:38.779000Z',\n",
       "   'updated_at': '2024-08-06T13:26:40.985000Z',\n",
       "   'completed_at': '2024-08-06T13:26:40.933000Z',\n",
       "   'message': 'bias run is successful.',\n",
       "   'operators': []}}}"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wos_client.monitor_instances.get_run_details(fairness_monitor_instance_id, fairness_monitoring_run_id).result.to_dict()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "id": "cd87098c263343c88302caf6c6a7fe67"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>841a81b1-b4cf-4e98-87de-1d374d43df63 Monitor Runs Metrics from: 2024-07-30 18:56:45.276171  till: 2024-08-06 18:56:45.276177</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>ts</th><th style='border: 1px solid #dddddd'>id</th><th style='border: 1px solid #dddddd'>measurement_id</th><th style='border: 1px solid #dddddd'>value</th><th style='border: 1px solid #dddddd'>lower_limit</th><th style='border: 1px solid #dddddd'>upper_limit</th><th style='border: 1px solid #dddddd'>tags</th><th style='border: 1px solid #dddddd'>monitor_definition_id</th><th style='border: 1px solid #dddddd'>monitor_instance_id</th><th style='border: 1px solid #dddddd'>run_id</th><th style='border: 1px solid #dddddd'>target_type</th><th style='border: 1px solid #dddddd'>target_id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>2024-08-06 13:26:40.688183+00:00</td><td style='border: 1px solid #dddddd'>fairness_value</td><td style='border: 1px solid #dddddd'>0ac72f74-aa61-4412-84fd-718b5e746fa6</td><td style='border: 1px solid #dddddd'>18.377</td><td style='border: 1px solid #dddddd'>80.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['feature:sex', 'fairness_metric_type:fairness', 'feature_value:Female']</td><td style='border: 1px solid #dddddd'>fairness</td><td style='border: 1px solid #dddddd'>841a81b1-b4cf-4e98-87de-1d374d43df63</td><td style='border: 1px solid #dddddd'>6c53aa46-439a-4008-bbfc-165132dff3ff</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-06 13:26:40.688183+00:00</td><td style='border: 1px solid #dddddd'>fairness_value</td><td style='border: 1px solid #dddddd'>0ac72f74-aa61-4412-84fd-718b5e746fa6</td><td style='border: 1px solid #dddddd'>38.577</td><td style='border: 1px solid #dddddd'>80.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['feature:age', 'fairness_metric_type:fairness', 'feature_value:18-33']</td><td style='border: 1px solid #dddddd'>fairness</td><td style='border: 1px solid #dddddd'>841a81b1-b4cf-4e98-87de-1d374d43df63</td><td style='border: 1px solid #dddddd'>6c53aa46-439a-4008-bbfc-165132dff3ff</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "wos_client.monitor_instances.show_metrics(monitor_instance_id=fairness_monitor_instance_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "id": "9a98378e36cc46388d1917d0b2440d98"
   },
   "outputs": [],
   "source": [
    "FAIRNESS_DASHBOARD_URL = WOS_CREDENTIALS[\"url\"] + \"/aiopenscale/insights/{0}/fairness/age?features=fairnessv2,indirect_bias,v2transaction\".format(deployment_uid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "id": "f957d3d9312a42bb8a446bf771490299"
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "#### Link to IBM Watson OpenScale Fairness Dashboard: https://cpd-cpd-instance.apps.wos415nfs2672.cp.fyre.ibm.com/aiopenscale/insights/038b8c68-6615-4c0d-8d46-15783d6ab9c8/fairness/age?features=fairnessv2,indirect_bias,v2transaction"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import Markdown as md\n",
    "md(\"#### Link to IBM Watson OpenScale Fairness Dashboard: {}\".format(FAIRNESS_DASHBOARD_URL))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "03db2a8db59c417b89348fa82c665260"
   },
   "source": [
    "### Run on-demand Fairness\n",
    "If you would like to peform an on-demand fairness check, then we need to score a fresh set of data with meta-fields, so that they would be used for indirect bias checking. So the below two cells will score and make sure these records are reached to payload logging table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "id": "81d3d0ba7c344c0182a80525ba73bd15"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of records in the payload logging table: 2000\n"
     ]
    }
   ],
   "source": [
    "payload_logging(no_of_records_to_score = 1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "id": "a75a71a4d0ee44a7833a1fa0d39edbbe"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of records in the payload logging table: 2000\n"
     ]
    }
   ],
   "source": [
    "time.sleep(5)\n",
    "pl_records_count = wos_client.data_sets.get_records_count(payload_data_set_id)\n",
    "print(\"Number of records in the payload logging table: {}\".format(pl_records_count))\n",
    "if pl_records_count == 0:\n",
    "    raise Exception(\"Payload logging did not happen!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f02860d8d5564f25824b13d0c058cead"
   },
   "source": [
    "### Trigger fairness monitoring run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "id": "ff4b16a95858418aa085052cfabc350a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "========================================================================\n",
      "\n",
      " Waiting for end of monitoring run 1e2008e1-3cf1-4876-ae20-a10401ba8ae1 \n",
      "\n",
      "========================================================================\n",
      "\n",
      "\n",
      "\n",
      "finished\n",
      "\n",
      "---------------------------\n",
      " Successfully finished run \n",
      "---------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "run_details = wos_client.monitor_instances.run(monitor_instance_id=fairness_monitor_instance_id, background_mode=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "d272b783751e44fe83a5a179bfa4d8ad"
   },
   "source": [
    "### Check for its status"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "id": "d8677dd14e5e42eb84da876e86eb15ee"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<HTML>\n",
       "        <body>\n",
       "            <h3>841a81b1-b4cf-4e98-87de-1d374d43df63 Monitor Runs Metrics from: 2024-07-30 18:57:07.688046  till: 2024-08-06 18:57:07.688059</h3>\n",
       "            <table style='border: 1px solid #dddddd; font-family: Courier'>\n",
       "                <th style='border: 1px solid #dddddd'>ts</th><th style='border: 1px solid #dddddd'>id</th><th style='border: 1px solid #dddddd'>measurement_id</th><th style='border: 1px solid #dddddd'>value</th><th style='border: 1px solid #dddddd'>lower_limit</th><th style='border: 1px solid #dddddd'>upper_limit</th><th style='border: 1px solid #dddddd'>tags</th><th style='border: 1px solid #dddddd'>monitor_definition_id</th><th style='border: 1px solid #dddddd'>monitor_instance_id</th><th style='border: 1px solid #dddddd'>run_id</th><th style='border: 1px solid #dddddd'>target_type</th><th style='border: 1px solid #dddddd'>target_id</th>\n",
       "                <tr><td style='border: 1px solid #dddddd'>2024-08-06 13:27:03.983373+00:00</td><td style='border: 1px solid #dddddd'>fairness_value</td><td style='border: 1px solid #dddddd'>5826be95-e692-4c13-ab6e-274f08b33563</td><td style='border: 1px solid #dddddd'>18.377</td><td style='border: 1px solid #dddddd'>80.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['feature:sex', 'fairness_metric_type:fairness', 'feature_value:Female']</td><td style='border: 1px solid #dddddd'>fairness</td><td style='border: 1px solid #dddddd'>841a81b1-b4cf-4e98-87de-1d374d43df63</td><td style='border: 1px solid #dddddd'>1e2008e1-3cf1-4876-ae20-a10401ba8ae1</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-06 13:27:03.983373+00:00</td><td style='border: 1px solid #dddddd'>fairness_value</td><td style='border: 1px solid #dddddd'>5826be95-e692-4c13-ab6e-274f08b33563</td><td style='border: 1px solid #dddddd'>38.577</td><td style='border: 1px solid #dddddd'>80.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['feature:age', 'fairness_metric_type:fairness', 'feature_value:18-33']</td><td style='border: 1px solid #dddddd'>fairness</td><td style='border: 1px solid #dddddd'>841a81b1-b4cf-4e98-87de-1d374d43df63</td><td style='border: 1px solid #dddddd'>1e2008e1-3cf1-4876-ae20-a10401ba8ae1</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-06 13:26:40.688183+00:00</td><td style='border: 1px solid #dddddd'>fairness_value</td><td style='border: 1px solid #dddddd'>0ac72f74-aa61-4412-84fd-718b5e746fa6</td><td style='border: 1px solid #dddddd'>18.377</td><td style='border: 1px solid #dddddd'>80.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['feature:sex', 'fairness_metric_type:fairness', 'feature_value:Female']</td><td style='border: 1px solid #dddddd'>fairness</td><td style='border: 1px solid #dddddd'>841a81b1-b4cf-4e98-87de-1d374d43df63</td><td style='border: 1px solid #dddddd'>6c53aa46-439a-4008-bbfc-165132dff3ff</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td></tr><tr><td style='border: 1px solid #dddddd'>2024-08-06 13:26:40.688183+00:00</td><td style='border: 1px solid #dddddd'>fairness_value</td><td style='border: 1px solid #dddddd'>0ac72f74-aa61-4412-84fd-718b5e746fa6</td><td style='border: 1px solid #dddddd'>38.577</td><td style='border: 1px solid #dddddd'>80.0</td><td style='border: 1px solid #dddddd'>None</td><td style='border: 1px solid #dddddd'>['feature:age', 'fairness_metric_type:fairness', 'feature_value:18-33']</td><td style='border: 1px solid #dddddd'>fairness</td><td style='border: 1px solid #dddddd'>841a81b1-b4cf-4e98-87de-1d374d43df63</td><td style='border: 1px solid #dddddd'>6c53aa46-439a-4008-bbfc-165132dff3ff</td><td style='border: 1px solid #dddddd'>subscription</td><td style='border: 1px solid #dddddd'>c274e76c-6bb9-43ce-ba64-127a3db95ede</td></tr>\n",
       "            </table>\n",
       "        </body>\n",
       "        </HTML>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "wos_client.monitor_instances.show_metrics(monitor_instance_id=fairness_monitor_instance_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "id": "34bb889585c04e9ca245ff091d58d921"
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "#### To view the latest evaluation of the fairness check, please visit IBM Watson OpenScale Fairness Dashboard: https://cpd-cpd-instance.apps.wos415nfs2672.cp.fyre.ibm.com/aiopenscale/insights/038b8c68-6615-4c0d-8d46-15783d6ab9c8/fairness/age?features=fairnessv2,indirect_bias,v2transaction"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import Markdown as md\n",
    "md(\"#### To view the latest evaluation of the fairness check, please visit IBM Watson OpenScale Fairness Dashboard: {}\".format(FAIRNESS_DASHBOARD_URL))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "958a4549555641c58bd9fbd0058d24cd"
   },
   "source": [
    "# Active debiasing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "id": "4f0591e8da8b48ea9845950fd6423685"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Single record scoring result: \n",
      " fields: ['workclass', 'fnlwgt', 'education', 'education-num', 'Marital', 'occupation', 'relationship', 'capitalgain', 'loss', 'hoursper', 'citizen_status', 'workclass_IX', 'workclassclassVec', 'education_IX', 'educationclassVec', 'Marital_IX', 'MaritalclassVec', 'occupation_IX', 'occupationclassVec', 'relationship_IX', 'relationshipclassVec', 'citizen_status_IX', 'citizen_statusclassVec', 'features', 'rawPrediction', 'probability', 'prediction', 'predictedLabel'] \n",
      " values:  ['State-gov', 77516, 'Bachelors', 13, 'Never-married', 'Adm-clerical', 'Not-in-family', 2174, 0, 40, 'United-States', 4.0, [9, [4], [1.0]], 2.0, [16, [2], [1.0]], 1.0, [7, [1], [1.0]], 3.0, [15, [3], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [4, 11, 26, 35, 48, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 77516.0, 13.0, 2174.0, 40.0]], [17.120135477812976, 2.879864522187024], [0.8560067738906488, 0.1439932261093512], 0.0, '<=50K']\n",
      "{\"predictions\": [{\"fields\": [\"workclass\", \"fnlwgt\", \"education\", \"education-num\", \"Marital\", \"occupation\", \"relationship\", \"capitalgain\", \"loss\", \"hoursper\", \"citizen_status\", \"workclass_IX\", \"workclassclassVec\", \"education_IX\", \"educationclassVec\", \"Marital_IX\", \"MaritalclassVec\", \"occupation_IX\", \"occupationclassVec\", \"relationship_IX\", \"relationshipclassVec\", \"citizen_status_IX\", \"citizen_statusclassVec\", \"features\", \"rawPrediction\", \"probability\", \"prediction\", \"predictedLabel\"], \"values\": [[\"State-gov\", 77516, \"Bachelors\", 13, \"Never-married\", \"Adm-clerical\", \"Not-in-family\", 2174, 0, 40, \"United-States\", 4.0, [9, [4], [1.0]], 2.0, [16, [2], [1.0]], 1.0, [7, [1], [1.0]], 3.0, [15, [3], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [4, 11, 26, 35, 48, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 77516.0, 13.0, 2174.0, 40.0]], [17.120135477812976, 2.879864522187024], [0.8560067738906488, 0.1439932261093512], 0.0, \"<=50K\"], [\"Self-emp-not-inc\", 83311, \"Bachelors\", 13, \"Married-civ-spouse\", \"Exec-managerial\", \"Husband\", 0, 0, 13, \"United-States\", 1.0, [9, [1], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 2.0, [15, [2], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 11, 25, 34, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 83311.0, 13.0, 13.0]], [9.8212518471239, 10.178748152876102], [0.491062592356195, 0.5089374076438051], 1.0, \">50K\"], [\"Private\", 215646, \"HS-grad\", 9, \"Divorced\", \"Handlers-cleaners\", \"Not-in-family\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 2.0, [7, [2], [1.0]], 9.0, [15, [9], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 27, 41, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 215646.0, 9.0, 40.0]], [17.92490871282145, 2.0750912871785534], [0.8962454356410724, 0.10375456435892767], 0.0, \"<=50K\"], [\"Private\", 234721, \"11th\", 7, \"Married-civ-spouse\", \"Handlers-cleaners\", \"Husband\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 5.0, [16, [5], [1.0]], 0.0, [7, [0], [1.0]], 9.0, [15, [9], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 14, 25, 41, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 234721.0, 7.0, 40.0]], [14.821917527787436, 5.178082472212567], [0.7410958763893717, 0.2589041236106283], 0.0, \"<=50K\"], [\"Private\", 338409, \"Bachelors\", 13, \"Married-civ-spouse\", \"Prof-specialty\", \"Wife\", 0, 0, 40, \"Cuba\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 4.0, [6, [4], [1.0]], 9.0, [42, [9], [1.0]], [100, [0, 11, 25, 32, 51, 62, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 338409.0, 13.0, 40.0]], [10.943969220583075, 9.056030779416925], [0.5471984610291537, 0.4528015389708463], 0.0, \"<=50K\"], [\"Private\", 284582, \"Masters\", 14, \"Married-civ-spouse\", \"Exec-managerial\", \"Wife\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 3.0, [16, [3], [1.0]], 0.0, [7, [0], [1.0]], 2.0, [15, [2], [1.0]], 4.0, [6, [4], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 12, 25, 34, 51, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 284582.0, 14.0, 40.0]], [9.332511731946578, 10.667488268053424], [0.4666255865973289, 0.5333744134026712], 1.0, \">50K\"], [\"Private\", 160187, \"9th\", 5, \"Married-spouse-absent\", \"Other-service\", \"Not-in-family\", 0, 0, 16, \"Jamaica\", 0.0, [9, [0], [1.0]], 10.0, [16, [10], [1.0]], 5.0, [7, [5], [1.0]], 5.0, [15, [5], [1.0]], 1.0, [6, [1], [1.0]], 13.0, [42, [13], [1.0]], [100, [0, 19, 30, 37, 48, 66, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 160187.0, 5.0, 16.0]], [18.01799291677759, 1.9820070832224141], [0.9008996458388794, 0.09910035416112069], 0.0, \"<=50K\"], [\"Self-emp-not-inc\", 209642, \"HS-grad\", 9, \"Married-civ-spouse\", \"Exec-managerial\", \"Husband\", 0, 0, 45, \"United-States\", 1.0, [9, [1], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 2.0, [15, [2], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 9, 25, 34, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 209642.0, 9.0, 45.0]], [13.490237134025236, 6.509762865974766], [0.6745118567012618, 0.3254881432987383], 0.0, \"<=50K\"], [\"Private\", 45781, \"Masters\", 14, \"Never-married\", \"Prof-specialty\", \"Not-in-family\", 14084, 0, 50, \"United-States\", 0.0, [9, [0], [1.0]], 3.0, [16, [3], [1.0]], 1.0, [7, [1], [1.0]], 0.0, [15, [0], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 12, 26, 32, 48, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 45781.0, 14.0, 14084.0, 50.0]], [10.667458673807754, 9.332541326192247], [0.5333729336903877, 0.46662706630961237], 0.0, \"<=50K\"], [\"Private\", 159449, \"Bachelors\", 13, \"Married-civ-spouse\", \"Exec-managerial\", \"Husband\", 5178, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 2.0, [15, [2], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 25, 34, 47, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 159449.0, 13.0, 5178.0, 40.0]], [6.325401479699091, 13.674598520300911], [0.3162700739849545, 0.6837299260150455], 1.0, \">50K\"], [\"Private\", 280464, \"Some-college\", 10, \"Married-civ-spouse\", \"Exec-managerial\", \"Husband\", 0, 0, 80, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 2.0, [15, [2], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 25, 34, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 280464.0, 10.0, 80.0]], [13.041040567958191, 6.958959432041812], [0.6520520283979094, 0.34794797160209057], 0.0, \"<=50K\"], [\"State-gov\", 141297, \"Bachelors\", 13, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 0, 40, \"India\", 4.0, [9, [4], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 8.0, [42, [8], [1.0]], [100, [4, 11, 25, 32, 47, 61, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 141297.0, 13.0, 40.0]], [9.714568381317775, 10.285431618682226], [0.48572841906588876, 0.5142715809341113], 1.0, \">50K\"], [\"Private\", 122272, \"Bachelors\", 13, \"Never-married\", \"Adm-clerical\", \"Own-child\", 0, 0, 30, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 1.0, [7, [1], [1.0]], 3.0, [15, [3], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 26, 35, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 122272.0, 13.0, 30.0]], [17.414370823371367, 2.5856291766286357], [0.8707185411685682, 0.12928145883143177], 0.0, \"<=50K\"], [\"Private\", 205019, \"Assoc-acdm\", 12, \"Never-married\", \"Sales\", \"Not-in-family\", 0, 0, 50, \"United-States\", 0.0, [9, [0], [1.0]], 6.0, [16, [6], [1.0]], 1.0, [7, [1], [1.0]], 4.0, [15, [4], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 15, 26, 36, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 205019.0, 12.0, 50.0]], [17.24813712394819, 2.751862876051808], [0.8624068561974095, 0.1375931438025904], 0.0, \"<=50K\"], [\"Private\", 121772, \"Assoc-voc\", 11, \"Married-civ-spouse\", \"Craft-repair\", \"Husband\", 0, 0, 40, \"?\", 0.0, [9, [0], [1.0]], 4.0, [16, [4], [1.0]], 0.0, [7, [0], [1.0]], 1.0, [15, [1], [1.0]], 0.0, [6, [0], [1.0]], 2.0, [42, [2], [1.0]], [100, [0, 13, 25, 33, 47, 55, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 121772.0, 11.0, 40.0]], [14.788410539811604, 5.2115894601883985], [0.73942052699058, 0.26057947300941986], 0.0, \"<=50K\"], [\"Private\", 245487, \"7th-8th\", 4, \"Married-civ-spouse\", \"Transport-moving\", \"Husband\", 0, 0, 45, \"Mexico\", 0.0, [9, [0], [1.0]], 8.0, [16, [8], [1.0]], 0.0, [7, [0], [1.0]], 8.0, [15, [8], [1.0]], 0.0, [6, [0], [1.0]], 1.0, [42, [1], [1.0]], [100, [0, 17, 25, 40, 47, 54, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 245487.0, 4.0, 45.0]], [14.927868897876566, 5.072131102123436], [0.7463934448938283, 0.2536065551061718], 0.0, \"<=50K\"], [\"Self-emp-not-inc\", 176756, \"HS-grad\", 9, \"Never-married\", \"Farming-fishing\", \"Own-child\", 0, 0, 35, \"United-States\", 1.0, [9, [1], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 10.0, [15, [10], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 9, 26, 42, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 176756.0, 9.0, 35.0]], [18.39222740348624, 1.6077725965137608], [0.9196113701743119, 0.08038862982568803], 0.0, \"<=50K\"], [\"Private\", 186824, \"HS-grad\", 9, \"Never-married\", \"Machine-op-inspct\", \"Unmarried\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 6.0, [15, [6], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 38, 50, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 186824.0, 9.0, 40.0]], [17.83501937594976, 2.1649806240502394], [0.8917509687974879, 0.10824903120251197], 0.0, \"<=50K\"], [\"Private\", 28887, \"11th\", 7, \"Married-civ-spouse\", \"Sales\", \"Husband\", 0, 0, 50, \"United-States\", 0.0, [9, [0], [1.0]], 5.0, [16, [5], [1.0]], 0.0, [7, [0], [1.0]], 4.0, [15, [4], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 14, 25, 36, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 28887.0, 7.0, 50.0]], [14.481725048863133, 5.518274951136867], [0.7240862524431566, 0.27591374755684334], 0.0, \"<=50K\"], [\"Self-emp-not-inc\", 292175, \"Masters\", 14, \"Divorced\", \"Exec-managerial\", \"Unmarried\", 0, 0, 45, \"United-States\", 1.0, [9, [1], [1.0]], 3.0, [16, [3], [1.0]], 2.0, [7, [2], [1.0]], 2.0, [15, [2], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 12, 27, 34, 50, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 292175.0, 14.0, 45.0]], [13.269670357201049, 6.730329642798952], [0.6634835178600524, 0.3365164821399476], 0.0, \"<=50K\"], [\"Private\", 193524, \"Doctorate\", 16, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 0, 60, \"United-States\", 0.0, [9, [0], [1.0]], 12.0, [16, [12], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 21, 25, 32, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 193524.0, 16.0, 60.0]], [7.583618850581144, 12.416381149418854], [0.37918094252905726, 0.6208190574709428], 1.0, \">50K\"], [\"Private\", 302146, \"HS-grad\", 9, \"Separated\", \"Other-service\", \"Unmarried\", 0, 0, 20, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 3.0, [7, [3], [1.0]], 5.0, [15, [5], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 28, 37, 50, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 302146.0, 9.0, 20.0]], [17.91518952856918, 2.0848104714308224], [0.895759476428459, 0.10424052357154112], 0.0, \"<=50K\"], [\"Federal-gov\", 76845, \"9th\", 5, \"Married-civ-spouse\", \"Farming-fishing\", \"Husband\", 0, 0, 40, \"United-States\", 6.0, [9, [6], [1.0]], 10.0, [16, [10], [1.0]], 0.0, [7, [0], [1.0]], 10.0, [15, [10], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [6, 19, 25, 42, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 76845.0, 5.0, 40.0]], [14.703872315760908, 5.296127684239095], [0.7351936157880452, 0.2648063842119547], 0.0, \"<=50K\"], [\"Private\", 117037, \"11th\", 7, \"Married-civ-spouse\", \"Transport-moving\", \"Husband\", 0, 2042, 40, \"United-States\", 0.0, [9, [0], [1.0]], 5.0, [16, [5], [1.0]], 0.0, [7, [0], [1.0]], 8.0, [15, [8], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 14, 25, 40, 47, 53, 95, 96, 98, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 117037.0, 7.0, 2042.0, 40.0]], [12.576898261561475, 7.423101738438529], [0.6288449130780737, 0.3711550869219264], 0.0, \"<=50K\"], [\"Private\", 109015, \"HS-grad\", 9, \"Divorced\", \"Tech-support\", \"Unmarried\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 2.0, [7, [2], [1.0]], 11.0, [15, [11], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 27, 43, 50, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 109015.0, 9.0, 40.0]], [17.83061313591393, 2.16938686408607], [0.8915306567956964, 0.1084693432043035], 0.0, \"<=50K\"], [\"Local-gov\", 216851, \"Bachelors\", 13, \"Married-civ-spouse\", \"Tech-support\", \"Husband\", 0, 0, 40, \"United-States\", 2.0, [9, [2], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 11.0, [15, [11], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [2, 11, 25, 43, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 216851.0, 13.0, 40.0]], [9.852191124757184, 10.147808875242816], [0.4926095562378592, 0.5073904437621408], 1.0, \">50K\"], [\"Private\", 168294, \"HS-grad\", 9, \"Never-married\", \"Craft-repair\", \"Own-child\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 1.0, [15, [1], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 33, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 168294.0, 9.0, 40.0]], [18.448141733138897, 1.551858266861105], [0.9224070866569448, 0.07759291334305525], 0.0, \"<=50K\"], [\"?\", 180211, \"Some-college\", 10, \"Married-civ-spouse\", \"?\", \"Husband\", 0, 0, 60, \"South\", 3.0, [9, [3], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 7.0, [15, [7], [1.0]], 0.0, [6, [0], [1.0]], 14.0, [42, [14], [1.0]], [100, [3, 10, 25, 39, 47, 67, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 180211.0, 10.0, 60.0]], [14.651355785395486, 5.348644214604515], [0.7325677892697743, 0.2674322107302257], 0.0, \"<=50K\"], [\"Private\", 367260, \"HS-grad\", 9, \"Divorced\", \"Exec-managerial\", \"Not-in-family\", 0, 0, 80, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 2.0, [7, [2], [1.0]], 2.0, [15, [2], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 27, 34, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 367260.0, 9.0, 80.0]], [16.82954804449254, 3.170451955507462], [0.8414774022246269, 0.15852259777537306], 0.0, \"<=50K\"], [\"Private\", 193366, \"HS-grad\", 9, \"Married-civ-spouse\", \"Craft-repair\", \"Husband\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 1.0, [15, [1], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 25, 33, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 193366.0, 9.0, 40.0]], [14.794549491868143, 5.205450508131859], [0.7397274745934072, 0.26027252540659296], 0.0, \"<=50K\"], [\"Local-gov\", 190709, \"Assoc-acdm\", 12, \"Never-married\", \"Protective-serv\", \"Not-in-family\", 0, 0, 52, \"United-States\", 2.0, [9, [2], [1.0]], 6.0, [16, [6], [1.0]], 1.0, [7, [1], [1.0]], 12.0, [15, [12], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [2, 15, 26, 44, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 190709.0, 12.0, 52.0]], [17.07065387243099, 2.929346127569008], [0.8535326936215496, 0.1464673063784504], 0.0, \"<=50K\"], [\"Private\", 266015, \"Some-college\", 10, \"Never-married\", \"Sales\", \"Own-child\", 0, 0, 44, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 4.0, [15, [4], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 26, 36, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 266015.0, 10.0, 44.0]], [18.121437768995353, 1.8785622310046466], [0.9060718884497676, 0.09392811155023233], 0.0, \"<=50K\"], [\"Private\", 386940, \"Bachelors\", 13, \"Divorced\", \"Exec-managerial\", \"Own-child\", 0, 1408, 40, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 2.0, [7, [2], [1.0]], 2.0, [15, [2], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 27, 34, 49, 53, 95, 96, 98, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 386940.0, 13.0, 1408.0, 40.0]], [15.197045287586166, 4.802954712413834], [0.7598522643793083, 0.24014773562069172], 0.0, \"<=50K\"], [\"Federal-gov\", 59951, \"Some-college\", 10, \"Married-civ-spouse\", \"Adm-clerical\", \"Own-child\", 0, 0, 40, \"United-States\", 6.0, [9, [6], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 3.0, [15, [3], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [6, 10, 25, 35, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 59951.0, 10.0, 40.0]], [16.757028863642468, 3.2429711363575344], [0.8378514431821232, 0.16214855681787668], 0.0, \"<=50K\"], [\"State-gov\", 311512, \"Some-college\", 10, \"Married-civ-spouse\", \"Other-service\", \"Husband\", 0, 0, 15, \"United-States\", 4.0, [9, [4], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 5.0, [15, [5], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [4, 10, 25, 37, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 311512.0, 10.0, 15.0]], [15.783038004298891, 4.216961995701109], [0.7891519002149445, 0.21084809978505542], 0.0, \"<=50K\"], [\"Private\", 242406, \"11th\", 7, \"Never-married\", \"Machine-op-inspct\", \"Unmarried\", 0, 0, 40, \"Puerto-Rico\", 0.0, [9, [0], [1.0]], 5.0, [16, [5], [1.0]], 1.0, [7, [1], [1.0]], 6.0, [15, [6], [1.0]], 3.0, [6, [3], [1.0]], 6.0, [42, [6], [1.0]], [100, [0, 14, 26, 38, 50, 59, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 242406.0, 7.0, 40.0]], [17.966763428218073, 2.033236571781929], [0.8983381714109037, 0.10166182858909645], 0.0, \"<=50K\"], [\"Private\", 197200, \"Some-college\", 10, \"Never-married\", \"Machine-op-inspct\", \"Own-child\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 6.0, [15, [6], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 26, 38, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 197200.0, 10.0, 40.0]], [18.432617713253986, 1.5673822867460143], [0.9216308856626994, 0.07836911433730072], 0.0, \"<=50K\"], [\"Private\", 544091, \"HS-grad\", 9, \"Married-AF-spouse\", \"Adm-clerical\", \"Wife\", 0, 0, 25, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 6.0, [7, [6], [1.0]], 3.0, [15, [3], [1.0]], 4.0, [6, [4], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 31, 35, 51, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 544091.0, 9.0, 25.0]], [15.898512905830845, 4.1014870941691495], [0.7949256452915425, 0.20507435470845756], 0.0, \"<=50K\"], [\"Private\", 84154, \"Some-college\", 10, \"Married-civ-spouse\", \"Sales\", \"Husband\", 0, 0, 38, \"?\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 4.0, [15, [4], [1.0]], 0.0, [6, [0], [1.0]], 2.0, [42, [2], [1.0]], [100, [0, 10, 25, 36, 47, 55, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 84154.0, 10.0, 38.0]], [14.703872315760908, 5.296127684239095], [0.7351936157880452, 0.2648063842119547], 0.0, \"<=50K\"], [\"Self-emp-not-inc\", 265477, \"Assoc-acdm\", 12, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 0, 40, \"United-States\", 1.0, [9, [1], [1.0]], 6.0, [16, [6], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 15, 25, 32, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 265477.0, 12.0, 40.0]], [13.283475449467115, 6.7165245505328866], [0.6641737724733557, 0.3358262275266443], 0.0, \"<=50K\"], [\"Private\", 507875, \"9th\", 5, \"Married-civ-spouse\", \"Machine-op-inspct\", \"Husband\", 0, 0, 43, \"United-States\", 0.0, [9, [0], [1.0]], 10.0, [16, [10], [1.0]], 0.0, [7, [0], [1.0]], 6.0, [15, [6], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 19, 25, 38, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 507875.0, 5.0, 43.0]], [14.880214722667738, 5.119785277332264], [0.7440107361333869, 0.2559892638666132], 0.0, \"<=50K\"], [\"Self-emp-not-inc\", 88506, \"Bachelors\", 13, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 0, 40, \"United-States\", 1.0, [9, [1], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 11, 25, 32, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 88506.0, 13.0, 40.0]], [9.475041337482137, 10.524958662517864], [0.47375206687410687, 0.5262479331258932], 1.0, \">50K\"], [\"Private\", 172987, \"Bachelors\", 13, \"Married-civ-spouse\", \"Tech-support\", \"Husband\", 0, 0, 50, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 11.0, [15, [11], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 25, 43, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 172987.0, 13.0, 50.0]], [9.592696706190766, 10.407303293809235], [0.4796348353095383, 0.5203651646904618], 1.0, \">50K\"], [\"Private\", 94638, \"HS-grad\", 9, \"Separated\", \"Adm-clerical\", \"Unmarried\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 3.0, [7, [3], [1.0]], 3.0, [15, [3], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 28, 35, 50, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 94638.0, 9.0, 40.0]], [17.637275609553566, 2.362724390446435], [0.8818637804776783, 0.11813621952232174], 0.0, \"<=50K\"], [\"Private\", 289980, \"HS-grad\", 9, \"Never-married\", \"Handlers-cleaners\", \"Not-in-family\", 0, 0, 35, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 9.0, [15, [9], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 41, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 289980.0, 9.0, 35.0]], [17.969546059900928, 2.030453940099076], [0.8984773029950462, 0.10152269700495378], 0.0, \"<=50K\"], [\"Federal-gov\", 337895, \"Bachelors\", 13, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 0, 40, \"United-States\", 6.0, [9, [6], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [6, 11, 25, 32, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 337895.0, 13.0, 40.0]], [9.249446587687224, 10.750553412312778], [0.4624723293843612, 0.5375276706156389], 1.0, \">50K\"], [\"Private\", 144361, \"HS-grad\", 9, \"Married-civ-spouse\", \"Machine-op-inspct\", \"Husband\", 0, 0, 38, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 6.0, [15, [6], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 25, 38, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 144361.0, 9.0, 38.0]], [14.971577815640618, 5.028422184359383], [0.7485788907820309, 0.2514211092179691], 0.0, \"<=50K\"], [\"Private\", 128354, \"Masters\", 14, \"Divorced\", \"Exec-managerial\", \"Unmarried\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 3.0, [16, [3], [1.0]], 2.0, [7, [2], [1.0]], 2.0, [15, [2], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 12, 27, 34, 50, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 128354.0, 14.0, 40.0]], [13.771403600220694, 6.228596399779305], [0.6885701800110346, 0.31142981998896524], 0.0, \"<=50K\"], [\"State-gov\", 101603, \"Assoc-voc\", 11, \"Married-civ-spouse\", \"Craft-repair\", \"Husband\", 0, 0, 40, \"United-States\", 4.0, [9, [4], [1.0]], 4.0, [16, [4], [1.0]], 0.0, [7, [0], [1.0]], 1.0, [15, [1], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [4, 13, 25, 33, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 101603.0, 11.0, 40.0]], [14.788410539811604, 5.2115894601883985], [0.73942052699058, 0.26057947300941986], 0.0, \"<=50K\"], [\"Private\", 271466, \"Assoc-voc\", 11, \"Never-married\", \"Prof-specialty\", \"Not-in-family\", 0, 0, 43, \"United-States\", 0.0, [9, [0], [1.0]], 4.0, [16, [4], [1.0]], 1.0, [7, [1], [1.0]], 0.0, [15, [0], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 13, 26, 32, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 271466.0, 11.0, 43.0]], [17.283166621221405, 2.7168333787785963], [0.8641583310610702, 0.13584166893892982], 0.0, \"<=50K\"], [\"Private\", 32275, \"Some-college\", 10, \"Married-civ-spouse\", \"Exec-managerial\", \"Wife\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 2.0, [15, [2], [1.0]], 4.0, [6, [4], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 25, 34, 51, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 32275.0, 10.0, 40.0]], [13.59503727639914, 6.404962723600858], [0.6797518638199571, 0.32024813618004294], 0.0, \"<=50K\"], [\"Private\", 226956, \"HS-grad\", 9, \"Never-married\", \"Other-service\", \"Own-child\", 0, 0, 30, \"?\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 5.0, [15, [5], [1.0]], 2.0, [6, [2], [1.0]], 2.0, [42, [2], [1.0]], [100, [0, 9, 26, 37, 49, 55, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 226956.0, 9.0, 30.0]], [18.683542460762546, 1.3164575392374505], [0.9341771230381274, 0.06582287696187253], 0.0, \"<=50K\"], [\"Private\", 51835, \"Prof-school\", 15, \"Married-civ-spouse\", \"Prof-specialty\", \"Wife\", 0, 1902, 60, \"Honduras\", 0.0, [9, [0], [1.0]], 9.0, [16, [9], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 4.0, [6, [4], [1.0]], 38.0, [42, [38], [1.0]], [100, [0, 18, 25, 32, 51, 91, 95, 96, 98, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 51835.0, 15.0, 1902.0, 60.0]], [9.049989640092301, 10.9500103599077], [0.45249948200461504, 0.547500517995385], 1.0, \">50K\"], [\"Federal-gov\", 251585, \"Bachelors\", 13, \"Divorced\", \"Exec-managerial\", \"Not-in-family\", 0, 0, 55, \"United-States\", 6.0, [9, [6], [1.0]], 2.0, [16, [2], [1.0]], 2.0, [7, [2], [1.0]], 2.0, [15, [2], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [6, 11, 27, 34, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 251585.0, 13.0, 55.0]], [14.152079659267418, 5.847920340732583], [0.7076039829633709, 0.29239601703662915], 0.0, \"<=50K\"], [\"Self-emp-inc\", 109832, \"HS-grad\", 9, \"Divorced\", \"Exec-managerial\", \"Not-in-family\", 0, 0, 60, \"United-States\", 5.0, [9, [5], [1.0]], 0.0, [16, [0], [1.0]], 2.0, [7, [2], [1.0]], 2.0, [15, [2], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [5, 9, 27, 34, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 109832.0, 9.0, 60.0]], [16.65083305147959, 3.349166948520413], [0.8325416525739794, 0.16745834742602064], 0.0, \"<=50K\"], [\"Private\", 237993, \"Some-college\", 10, \"Married-civ-spouse\", \"Tech-support\", \"Husband\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 11.0, [15, [11], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 25, 43, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 237993.0, 10.0, 40.0]], [14.703872315760908, 5.296127684239095], [0.7351936157880452, 0.2648063842119547], 0.0, \"<=50K\"], [\"Private\", 216666, \"5th-6th\", 3, \"Married-civ-spouse\", \"Machine-op-inspct\", \"Husband\", 0, 0, 40, \"Mexico\", 0.0, [9, [0], [1.0]], 13.0, [16, [13], [1.0]], 0.0, [7, [0], [1.0]], 6.0, [15, [6], [1.0]], 0.0, [6, [0], [1.0]], 1.0, [42, [1], [1.0]], [100, [0, 22, 25, 38, 47, 54, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 216666.0, 3.0, 40.0]], [15.296441329493577, 4.703558670506424], [0.7648220664746789, 0.23517793352532118], 0.0, \"<=50K\"], [\"Private\", 56352, \"Assoc-voc\", 11, \"Married-civ-spouse\", \"Other-service\", \"Husband\", 0, 0, 40, \"Puerto-Rico\", 0.0, [9, [0], [1.0]], 4.0, [16, [4], [1.0]], 0.0, [7, [0], [1.0]], 5.0, [15, [5], [1.0]], 0.0, [6, [0], [1.0]], 6.0, [42, [6], [1.0]], [100, [0, 13, 25, 37, 47, 59, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 56352.0, 11.0, 40.0]], [15.783038004298891, 4.216961995701109], [0.7891519002149445, 0.21084809978505542], 0.0, \"<=50K\"], [\"Private\", 147372, \"HS-grad\", 9, \"Married-civ-spouse\", \"Adm-clerical\", \"Husband\", 0, 0, 48, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 3.0, [15, [3], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 25, 35, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 147372.0, 9.0, 48.0]], [14.298903341997919, 5.701096658002083], [0.714945167099896, 0.28505483290010414], 0.0, \"<=50K\"], [\"Private\", 188146, \"HS-grad\", 9, \"Married-civ-spouse\", \"Machine-op-inspct\", \"Husband\", 5013, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 6.0, [15, [6], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 25, 38, 47, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 188146.0, 9.0, 5013.0, 40.0]], [10.468216656730121, 9.531783343269879], [0.523410832836506, 0.476589167163494], 0.0, \"<=50K\"], [\"Private\", 59496, \"Bachelors\", 13, \"Married-civ-spouse\", \"Sales\", \"Husband\", 2407, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 4.0, [15, [4], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 25, 36, 47, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 59496.0, 13.0, 2407.0, 40.0]], [10.026121215324485, 9.973878784675517], [0.5013060607662243, 0.4986939392337758], 0.0, \"<=50K\"], [\"?\", 293936, \"7th-8th\", 4, \"Married-spouse-absent\", \"?\", \"Not-in-family\", 0, 0, 40, \"?\", 3.0, [9, [3], [1.0]], 8.0, [16, [8], [1.0]], 5.0, [7, [5], [1.0]], 7.0, [15, [7], [1.0]], 1.0, [6, [1], [1.0]], 2.0, [42, [2], [1.0]], [100, [3, 17, 30, 39, 48, 55, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 293936.0, 4.0, 40.0]], [17.79182328396887, 2.2081767160311303], [0.8895911641984435, 0.11040883580155651], 0.0, \"<=50K\"], [\"Private\", 149640, \"HS-grad\", 9, \"Married-civ-spouse\", \"Transport-moving\", \"Husband\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 8.0, [15, [8], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 25, 40, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 149640.0, 9.0, 40.0]], [14.896997953219348, 5.103002046780652], [0.7448498976609674, 0.2551501023390326], 0.0, \"<=50K\"], [\"Private\", 116632, \"Doctorate\", 16, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 0, 45, \"United-States\", 0.0, [9, [0], [1.0]], 12.0, [16, [12], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 21, 25, 32, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 116632.0, 16.0, 45.0]], [7.583618850581144, 12.416381149418854], [0.37918094252905726, 0.6208190574709428], 1.0, \">50K\"], [\"Private\", 105598, \"Some-college\", 10, \"Divorced\", \"Tech-support\", \"Not-in-family\", 0, 0, 58, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 2.0, [7, [2], [1.0]], 11.0, [15, [11], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 27, 43, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 105598.0, 10.0, 58.0]], [17.10439198864946, 2.895608011350541], [0.855219599432473, 0.14478040056752706], 0.0, \"<=50K\"], [\"Private\", 155537, \"HS-grad\", 9, \"Married-civ-spouse\", \"Craft-repair\", \"Husband\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 1.0, [15, [1], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 25, 33, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 155537.0, 9.0, 40.0]], [14.794549491868143, 5.205450508131859], [0.7397274745934072, 0.26027252540659296], 0.0, \"<=50K\"], [\"Private\", 183175, \"Some-college\", 10, \"Divorced\", \"Adm-clerical\", \"Not-in-family\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 2.0, [7, [2], [1.0]], 3.0, [15, [3], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 27, 35, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 183175.0, 10.0, 40.0]], [17.997860843571935, 2.002139156428068], [0.8998930421785966, 0.10010695782140339], 0.0, \"<=50K\"], [\"Private\", 169846, \"HS-grad\", 9, \"Married-civ-spouse\", \"Adm-clerical\", \"Wife\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 3.0, [15, [3], [1.0]], 4.0, [6, [4], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 25, 35, 51, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 169846.0, 9.0, 40.0]], [14.71731272501981, 5.282687274980188], [0.7358656362509906, 0.26413436374900945], 0.0, \"<=50K\"], [\"Self-emp-inc\", 191681, \"Some-college\", 10, \"Married-civ-spouse\", \"Exec-managerial\", \"Husband\", 0, 0, 50, \"United-States\", 5.0, [9, [5], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 2.0, [15, [2], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [5, 10, 25, 34, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 191681.0, 10.0, 50.0]], [12.759735982193934, 7.240264017806069], [0.6379867991096966, 0.3620132008903034], 0.0, \"<=50K\"], [\"?\", 200681, \"Some-college\", 10, \"Never-married\", \"?\", \"Own-child\", 0, 0, 40, \"United-States\", 3.0, [9, [3], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 7.0, [15, [7], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [3, 10, 26, 39, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 200681.0, 10.0, 40.0]], [18.619880460546188, 1.3801195394538104], [0.9309940230273094, 0.06900597697269052], 0.0, \"<=50K\"], [\"Private\", 101509, \"Some-college\", 10, \"Never-married\", \"Prof-specialty\", \"Own-child\", 0, 0, 32, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 0.0, [15, [0], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 26, 32, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 101509.0, 10.0, 32.0]], [18.530156368921105, 1.469843631078895], [0.9265078184460552, 0.07349218155394474], 0.0, \"<=50K\"], [\"Private\", 309974, \"Bachelors\", 13, \"Separated\", \"Sales\", \"Own-child\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 3.0, [7, [3], [1.0]], 4.0, [15, [4], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 28, 36, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 309974.0, 13.0, 40.0]], [15.68555043598227, 4.314449564017729], [0.7842775217991135, 0.21572247820088647], 0.0, \"<=50K\"], [\"Self-emp-not-inc\", 162298, \"Bachelors\", 13, \"Married-civ-spouse\", \"Sales\", \"Husband\", 0, 0, 70, \"United-States\", 1.0, [9, [1], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 4.0, [15, [4], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 11, 25, 36, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 162298.0, 13.0, 70.0]], [10.113431115286135, 9.886568884713867], [0.5056715557643068, 0.49432844423569333], 0.0, \"<=50K\"], [\"Private\", 211678, \"Some-college\", 10, \"Never-married\", \"Machine-op-inspct\", \"Not-in-family\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 6.0, [15, [6], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 26, 38, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 211678.0, 10.0, 40.0]], [17.89395094938083, 2.1060490506191707], [0.8946975474690415, 0.10530245253095853], 0.0, \"<=50K\"], [\"Private\", 124744, \"Some-college\", 10, \"Married-civ-spouse\", \"Prof-specialty\", \"Other-relative\", 0, 0, 20, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 5.0, [6, [5], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 25, 32, 52, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 124744.0, 10.0, 20.0]], [15.72125306890868, 4.27874693109132], [0.786062653445434, 0.21393734655456603], 0.0, \"<=50K\"], [\"Private\", 213921, \"HS-grad\", 9, \"Never-married\", \"Other-service\", \"Own-child\", 0, 0, 40, \"Mexico\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 5.0, [15, [5], [1.0]], 2.0, [6, [2], [1.0]], 1.0, [42, [1], [1.0]], [100, [0, 9, 26, 37, 49, 54, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 213921.0, 9.0, 40.0]], [18.67340243179104, 1.3265975682089617], [0.933670121589552, 0.06632987841044809], 0.0, \"<=50K\"], [\"Private\", 32214, \"Assoc-acdm\", 12, \"Married-civ-spouse\", \"Adm-clerical\", \"Husband\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 6.0, [16, [6], [1.0]], 0.0, [7, [0], [1.0]], 3.0, [15, [3], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 15, 25, 35, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 32214.0, 12.0, 40.0]], [14.795518962068877, 5.204481037931126], [0.7397759481034437, 0.2602240518965563], 0.0, \"<=50K\"], [\"?\", 212759, \"10th\", 6, \"Married-civ-spouse\", \"?\", \"Husband\", 0, 0, 2, \"United-States\", 3.0, [9, [3], [1.0]], 7.0, [16, [7], [1.0]], 0.0, [7, [0], [1.0]], 7.0, [15, [7], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [3, 16, 25, 39, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 212759.0, 6.0, 2.0]], [14.947049392705768, 5.052950607294234], [0.7473524696352885, 0.25264753036471166], 0.0, \"<=50K\"], [\"Private\", 309634, \"11th\", 7, \"Never-married\", \"Other-service\", \"Own-child\", 0, 0, 22, \"United-States\", 0.0, [9, [0], [1.0]], 5.0, [16, [5], [1.0]], 1.0, [7, [1], [1.0]], 5.0, [15, [5], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 14, 26, 37, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 309634.0, 7.0, 22.0]], [18.80514648405935, 1.1948535159406517], [0.9402573242029675, 0.05974267579703259], 0.0, \"<=50K\"], [\"Local-gov\", 125927, \"7th-8th\", 4, \"Married-civ-spouse\", \"Farming-fishing\", \"Husband\", 0, 0, 40, \"United-States\", 2.0, [9, [2], [1.0]], 8.0, [16, [8], [1.0]], 0.0, [7, [0], [1.0]], 10.0, [15, [10], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [2, 17, 25, 42, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 125927.0, 4.0, 40.0]], [15.03899737436804, 4.96100262563196], [0.7519498687184021, 0.248050131281598], 0.0, \"<=50K\"], [\"Private\", 446839, \"HS-grad\", 9, \"Never-married\", \"Sales\", \"Not-in-family\", 0, 0, 30, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 4.0, [15, [4], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 36, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 446839.0, 9.0, 30.0]], [17.908243227769987, 2.091756772230013], [0.8954121613884993, 0.10458783861150064], 0.0, \"<=50K\"], [\"Private\", 276515, \"Bachelors\", 13, \"Married-civ-spouse\", \"Other-service\", \"Husband\", 0, 0, 40, \"Cuba\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 5.0, [15, [5], [1.0]], 0.0, [6, [0], [1.0]], 9.0, [42, [9], [1.0]], [100, [0, 11, 25, 37, 47, 62, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 276515.0, 13.0, 40.0]], [11.31090263420991, 8.68909736579009], [0.5655451317104955, 0.4344548682895045], 0.0, \"<=50K\"], [\"Private\", 51618, \"HS-grad\", 9, \"Married-civ-spouse\", \"Other-service\", \"Wife\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 5.0, [15, [5], [1.0]], 4.0, [6, [4], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 25, 37, 51, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 51618.0, 9.0, 40.0]], [16.0620963224871, 3.937903677512894], [0.8031048161243554, 0.19689518387564478], 0.0, \"<=50K\"], [\"Private\", 159937, \"HS-grad\", 9, \"Married-civ-spouse\", \"Sales\", \"Husband\", 0, 0, 48, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 4.0, [15, [4], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 25, 36, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 159937.0, 9.0, 48.0]], [14.49885588455786, 5.501144115442139], [0.724942794227893, 0.27505720577210696], 0.0, \"<=50K\"], [\"Private\", 343591, \"HS-grad\", 9, \"Divorced\", \"Craft-repair\", \"Not-in-family\", 14344, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 2.0, [7, [2], [1.0]], 1.0, [15, [1], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 27, 33, 48, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 343591.0, 9.0, 14344.0, 40.0]], [12.283542013128882, 7.716457986871117], [0.614177100656444, 0.38582289934355585], 0.0, \"<=50K\"], [\"Private\", 346253, \"HS-grad\", 9, \"Divorced\", \"Sales\", \"Own-child\", 0, 0, 35, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 2.0, [7, [2], [1.0]], 4.0, [15, [4], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 27, 36, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 346253.0, 9.0, 35.0]], [18.443735493103063, 1.5562645068969352], [0.9221867746551533, 0.07781322534484678], 0.0, \"<=50K\"], [\"Local-gov\", 268234, \"HS-grad\", 9, \"Married-civ-spouse\", \"Protective-serv\", \"Husband\", 0, 0, 40, \"United-States\", 2.0, [9, [2], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 12.0, [15, [12], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [2, 9, 25, 44, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 268234.0, 9.0, 40.0]], [14.770957661355512, 5.22904233864449], [0.7385478830677756, 0.2614521169322245], 0.0, \"<=50K\"], [\"Private\", 202051, \"Masters\", 14, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 0, 50, \"United-States\", 0.0, [9, [0], [1.0]], 3.0, [16, [3], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 12, 25, 32, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 202051.0, 14.0, 50.0]], [7.958711823192558, 12.041288176807441], [0.3979355911596279, 0.602064408840372], 1.0, \">50K\"], [\"Private\", 54334, \"9th\", 5, \"Never-married\", \"Sales\", \"Not-in-family\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 10.0, [16, [10], [1.0]], 1.0, [7, [1], [1.0]], 4.0, [15, [4], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 19, 26, 36, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 54334.0, 5.0, 40.0]], [17.89271920788508, 2.1072807921149224], [0.894635960394254, 0.10536403960574611], 0.0, \"<=50K\"], [\"Federal-gov\", 410867, \"Doctorate\", 16, \"Never-married\", \"Prof-specialty\", \"Not-in-family\", 0, 0, 50, \"United-States\", 6.0, [9, [6], [1.0]], 12.0, [16, [12], [1.0]], 1.0, [7, [1], [1.0]], 0.0, [15, [0], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [6, 21, 26, 32, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 410867.0, 16.0, 50.0]], [13.622773779327748, 6.377226220672254], [0.6811386889663874, 0.3188613110336127], 0.0, \"<=50K\"], [\"Private\", 249977, \"Assoc-voc\", 11, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 4.0, [16, [4], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 13, 25, 32, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 249977.0, 11.0, 40.0]], [13.283475449467115, 6.7165245505328866], [0.6641737724733557, 0.3358262275266443], 0.0, \"<=50K\"], [\"Private\", 286730, \"Some-college\", 10, \"Divorced\", \"Craft-repair\", \"Unmarried\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 2.0, [7, [2], [1.0]], 1.0, [15, [1], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 27, 33, 50, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 286730.0, 10.0, 40.0]], [17.81508911602902, 2.184910883970979], [0.8907544558014511, 0.10924554419854896], 0.0, \"<=50K\"], [\"Private\", 212563, \"Some-college\", 10, \"Divorced\", \"Machine-op-inspct\", \"Unmarried\", 0, 0, 25, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 2.0, [7, [2], [1.0]], 6.0, [15, [6], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 27, 38, 50, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 212563.0, 10.0, 25.0]], [17.81508911602902, 2.184910883970979], [0.8907544558014511, 0.10924554419854896], 0.0, \"<=50K\"], [\"Private\", 117747, \"HS-grad\", 9, \"Married-civ-spouse\", \"Sales\", \"Wife\", 0, 1573, 35, \"?\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 4.0, [15, [4], [1.0]], 4.0, [6, [4], [1.0]], 2.0, [42, [2], [1.0]], [100, [0, 9, 25, 36, 51, 55, 95, 96, 98, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 117747.0, 9.0, 1573.0, 35.0]], [15.063498329910376, 4.936501670089623], [0.7531749164955188, 0.24682508350448112], 0.0, \"<=50K\"], [\"Local-gov\", 226296, \"Bachelors\", 13, \"Married-civ-spouse\", \"Protective-serv\", \"Husband\", 0, 0, 40, \"United-States\", 2.0, [9, [2], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 12.0, [15, [12], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [2, 11, 25, 44, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 226296.0, 13.0, 40.0]], [10.086725999691062, 9.91327400030894], [0.5043362999845531, 0.495663700015447], 0.0, \"<=50K\"], [\"Local-gov\", 115585, \"Some-college\", 10, \"Never-married\", \"Handlers-cleaners\", \"Not-in-family\", 0, 0, 50, \"United-States\", 2.0, [9, [2], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 9.0, [15, [9], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [2, 10, 26, 41, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 115585.0, 10.0, 50.0]], [17.141023385283642, 2.8589766147163598], [0.8570511692641821, 0.142948830735818], 0.0, \"<=50K\"], [\"Self-emp-not-inc\", 191277, \"Doctorate\", 16, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 1902, 60, \"United-States\", 1.0, [9, [1], [1.0]], 12.0, [16, [12], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 21, 25, 32, 47, 53, 95, 96, 98, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 191277.0, 16.0, 1902.0, 60.0]], [7.107954229615181, 12.892045770384815], [0.35539771148075916, 0.6446022885192408], 1.0, \">50K\"], [\"Private\", 202683, \"Some-college\", 10, \"Married-civ-spouse\", \"Sales\", \"Husband\", 0, 0, 48, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 4.0, [15, [4], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 25, 36, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 202683.0, 10.0, 48.0]], [14.408178708450626, 5.591821291549375], [0.7204089354225313, 0.27959106457746874], 0.0, \"<=50K\"], [\"Private\", 171095, \"Assoc-acdm\", 12, \"Divorced\", \"Exec-managerial\", \"Unmarried\", 0, 0, 40, \"England\", 0.0, [9, [0], [1.0]], 6.0, [16, [6], [1.0]], 2.0, [7, [2], [1.0]], 2.0, [15, [2], [1.0]], 3.0, [6, [3], [1.0]], 11.0, [42, [11], [1.0]], [100, [0, 15, 27, 34, 50, 64, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 171095.0, 12.0, 40.0]], [17.421024527427214, 2.5789754725727874], [0.8710512263713607, 0.12894877362863938], 0.0, \"<=50K\"], [\"Federal-gov\", 249409, \"HS-grad\", 9, \"Never-married\", \"Other-service\", \"Own-child\", 0, 0, 40, \"United-States\", 6.0, [9, [6], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 5.0, [15, [5], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [6, 9, 26, 37, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 249409.0, 9.0, 40.0]], [18.61748810213838, 1.382511897861618], [0.9308744051069191, 0.0691255948930809], 0.0, \"<=50K\"], [\"Private\", 124191, \"Masters\", 14, \"Married-civ-spouse\", \"Exec-managerial\", \"Husband\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 3.0, [16, [3], [1.0]], 0.0, [7, [0], [1.0]], 2.0, [15, [2], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 12, 25, 34, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 124191.0, 14.0, 40.0]], [9.204796606787331, 10.79520339321267], [0.46023983033936655, 0.5397601696606336], 1.0, \">50K\"], [\"Private\", 198282, \"Bachelors\", 13, \"Married-civ-spouse\", \"Exec-managerial\", \"Husband\", 15024, 0, 60, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 2.0, [15, [2], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 25, 34, 47, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 198282.0, 13.0, 15024.0, 60.0]], [5.460582974404719, 14.539417025595283], [0.273029148720236, 0.7269708512797641], 1.0, \">50K\"], [\"Self-emp-not-inc\", 149116, \"Masters\", 14, \"Never-married\", \"Prof-specialty\", \"Not-in-family\", 0, 0, 50, \"United-States\", 1.0, [9, [1], [1.0]], 3.0, [16, [3], [1.0]], 1.0, [7, [1], [1.0]], 0.0, [15, [0], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 12, 26, 32, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 149116.0, 14.0, 50.0]], [14.92553205230651, 5.074467947693491], [0.7462766026153255, 0.25372339738467453], 0.0, \"<=50K\"], [\"Private\", 188300, \"Some-college\", 10, \"Never-married\", \"Tech-support\", \"Own-child\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 11.0, [15, [11], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 26, 43, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 188300.0, 10.0, 40.0]], [18.432617713253986, 1.5673822867460143], [0.9216308856626994, 0.07836911433730072], 0.0, \"<=50K\"], [\"Private\", 103432, \"HS-grad\", 9, \"Never-married\", \"Craft-repair\", \"Not-in-family\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 1.0, [15, [1], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 33, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 103432.0, 9.0, 40.0]], [17.90947496926574, 2.0905250307342613], [0.8954737484632869, 0.10452625153671305], 0.0, \"<=50K\"], [\"Self-emp-inc\", 317660, \"HS-grad\", 9, \"Married-civ-spouse\", \"Craft-repair\", \"Husband\", 7688, 0, 40, \"United-States\", 5.0, [9, [5], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 1.0, [15, [1], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [5, 9, 25, 33, 47, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 317660.0, 9.0, 7688.0, 40.0]], [9.521777013358676, 10.478222986641327], [0.4760888506679337, 0.5239111493320663], 1.0, \">50K\"], [\"?\", 304873, \"10th\", 6, \"Never-married\", \"?\", \"Own-child\", 34095, 0, 32, \"United-States\", 3.0, [9, [3], [1.0]], 7.0, [16, [7], [1.0]], 1.0, [7, [1], [1.0]], 7.0, [15, [7], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [3, 16, 26, 39, 49, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 304873.0, 6.0, 34095.0, 32.0]], [14.070054175341141, 5.929945824658858], [0.7035027087670571, 0.2964972912329429], 0.0, \"<=50K\"], [\"Private\", 194901, \"11th\", 7, \"Never-married\", \"Handlers-cleaners\", \"Own-child\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 5.0, [16, [5], [1.0]], 1.0, [7, [1], [1.0]], 9.0, [15, [9], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 14, 26, 41, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 194901.0, 7.0, 40.0]], [18.841432491842458, 1.1585675081575428], [0.9420716245921229, 0.05792837540787714], 0.0, \"<=50K\"], [\"Local-gov\", 189265, \"HS-grad\", 9, \"Never-married\", \"Adm-clerical\", \"Not-in-family\", 0, 0, 40, \"United-States\", 2.0, [9, [2], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 3.0, [15, [3], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [2, 9, 26, 35, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 189265.0, 9.0, 40.0]], [17.979841837622782, 2.0201581623772236], [0.8989920918811388, 0.10100790811886114], 0.0, \"<=50K\"], [\"Private\", 124692, \"HS-grad\", 9, \"Married-civ-spouse\", \"Handlers-cleaners\", \"Husband\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 9.0, [15, [9], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 25, 41, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 124692.0, 9.0, 40.0]], [14.839048363482162, 5.160951636517839], [0.7419524181741081, 0.25804758182589194], 0.0, \"<=50K\"], [\"Private\", 432376, \"Bachelors\", 13, \"Never-married\", \"Sales\", \"Other-relative\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 1.0, [7, [1], [1.0]], 4.0, [15, [4], [1.0]], 5.0, [6, [5], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 26, 36, 52, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 432376.0, 13.0, 40.0]], [15.836681779108366, 4.163318220891633], [0.7918340889554183, 0.20816591104458165], 0.0, \"<=50K\"], [\"Private\", 65324, \"Prof-school\", 15, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 9.0, [16, [9], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 18, 25, 32, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 65324.0, 15.0, 40.0]], [7.754388151189234, 12.245611848810768], [0.3877194075594617, 0.6122805924405383], 1.0, \">50K\"], [\"Self-emp-not-inc\", 335605, \"HS-grad\", 9, \"Married-civ-spouse\", \"Other-service\", \"Husband\", 0, 1887, 50, \"Canada\", 1.0, [9, [1], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 5.0, [15, [5], [1.0]], 0.0, [6, [0], [1.0]], 5.0, [42, [5], [1.0]], [100, [1, 9, 25, 37, 47, 58, 95, 96, 98, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 335605.0, 9.0, 1887.0, 50.0]], [13.79685733085302, 6.203142669146979], [0.689842866542651, 0.31015713345734897], 0.0, \"<=50K\"], [\"Private\", 377869, \"Some-college\", 10, \"Married-civ-spouse\", \"Sales\", \"Wife\", 4064, 0, 25, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 4.0, [15, [4], [1.0]], 4.0, [6, [4], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 25, 36, 51, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 377869.0, 10.0, 4064.0, 25.0]], [9.95557550576848, 10.04442449423152], [0.49777877528842396, 0.502221224711576], 1.0, \">50K\"], [\"Private\", 102864, \"HS-grad\", 9, \"Never-married\", \"Machine-op-inspct\", \"Own-child\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 6.0, [15, [6], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 38, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 102864.0, 9.0, 40.0]], [18.448141733138897, 1.551858266861105], [0.9224070866569448, 0.07759291334305525], 0.0, \"<=50K\"], [\"Private\", 95647, \"9th\", 5, \"Married-civ-spouse\", \"Handlers-cleaners\", \"Husband\", 0, 0, 50, \"United-States\", 0.0, [9, [0], [1.0]], 10.0, [16, [10], [1.0]], 0.0, [7, [0], [1.0]], 9.0, [15, [9], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 19, 25, 41, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 95647.0, 5.0, 50.0]], [14.452677580064645, 5.547322419935356], [0.7226338790032323, 0.27736612099676783], 0.0, \"<=50K\"], [\"Self-emp-inc\", 303090, \"Some-college\", 10, \"Married-civ-spouse\", \"Sales\", \"Husband\", 0, 0, 50, \"United-States\", 5.0, [9, [5], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 4.0, [15, [4], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [5, 10, 25, 36, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 303090.0, 10.0, 50.0]], [14.126874122686369, 5.873125877313632], [0.7063437061343184, 0.2936562938656816], 0.0, \"<=50K\"], [\"Local-gov\", 197371, \"Assoc-voc\", 11, \"Married-civ-spouse\", \"Craft-repair\", \"Husband\", 0, 0, 40, \"United-States\", 2.0, [9, [2], [1.0]], 4.0, [16, [4], [1.0]], 0.0, [7, [0], [1.0]], 1.0, [15, [1], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [2, 13, 25, 33, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 197371.0, 11.0, 40.0]], [14.788410539811604, 5.2115894601883985], [0.73942052699058, 0.26057947300941986], 0.0, \"<=50K\"], [\"Private\", 247552, \"Some-college\", 10, \"Married-civ-spouse\", \"Sales\", \"Husband\", 0, 0, 56, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 4.0, [15, [4], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 25, 36, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 247552.0, 10.0, 56.0]], [14.408178708450626, 5.591821291549375], [0.7204089354225313, 0.27959106457746874], 0.0, \"<=50K\"], [\"Private\", 102632, \"HS-grad\", 9, \"Never-married\", \"Craft-repair\", \"Not-in-family\", 0, 0, 41, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 1.0, [15, [1], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 33, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 102632.0, 9.0, 41.0]], [17.90947496926574, 2.0905250307342613], [0.8954737484632869, 0.10452625153671305], 0.0, \"<=50K\"], [\"Private\", 199915, \"Some-college\", 10, \"Never-married\", \"Other-service\", \"Own-child\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 5.0, [15, [5], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 26, 37, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 199915.0, 10.0, 40.0]], [18.657878411906125, 1.342121588093871], [0.9328939205953064, 0.06710607940469357], 0.0, \"<=50K\"], [\"Private\", 118853, \"Bachelors\", 13, \"Married-civ-spouse\", \"Exec-managerial\", \"Husband\", 0, 0, 60, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 2.0, [15, [2], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 25, 34, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 118853.0, 13.0, 60.0]], [8.831470813128837, 11.168529186871165], [0.44157354065644183, 0.5584264593435583], 1.0, \">50K\"], [\"Private\", 77143, \"Bachelors\", 13, \"Never-married\", \"Exec-managerial\", \"Own-child\", 0, 0, 40, \"Germany\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 1.0, [7, [1], [1.0]], 2.0, [15, [2], [1.0]], 2.0, [6, [2], [1.0]], 4.0, [42, [4], [1.0]], [100, [0, 11, 26, 34, 49, 57, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 77143.0, 13.0, 40.0]], [16.56898886911731, 3.431011130882691], [0.8284494434558655, 0.17155055654413456], 0.0, \"<=50K\"], [\"State-gov\", 267989, \"Bachelors\", 13, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 0, 50, \"United-States\", 4.0, [9, [4], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [4, 11, 25, 32, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 267989.0, 13.0, 50.0]], [9.585526129116385, 10.414473870883617], [0.47927630645581926, 0.5207236935441808], 1.0, \">50K\"], [\"Private\", 301606, \"Some-college\", 10, \"Never-married\", \"Other-service\", \"Own-child\", 0, 0, 35, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 5.0, [15, [5], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 26, 37, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 301606.0, 10.0, 35.0]], [18.657878411906125, 1.342121588093871], [0.9328939205953064, 0.06710607940469357], 0.0, \"<=50K\"], [\"Private\", 287828, \"Bachelors\", 13, \"Married-civ-spouse\", \"Exec-managerial\", \"Wife\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 2.0, [15, [2], [1.0]], 4.0, [6, [4], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 25, 34, 51, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 287828.0, 13.0, 40.0]], [9.615648990810158, 10.384351009189844], [0.48078244954050786, 0.5192175504594922], 1.0, \">50K\"], [\"Private\", 111697, \"Some-college\", 10, \"Never-married\", \"Adm-clerical\", \"Own-child\", 0, 1719, 28, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 3.0, [15, [3], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 26, 35, 49, 53, 95, 96, 98, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 111697.0, 10.0, 1719.0, 28.0]], [18.524264359561958, 1.4757356404380455], [0.9262132179780977, 0.07378678202190227], 0.0, \"<=50K\"], [\"Private\", 114937, \"Assoc-acdm\", 12, \"Married-civ-spouse\", \"Adm-clerical\", \"Husband\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 6.0, [16, [6], [1.0]], 0.0, [7, [0], [1.0]], 3.0, [15, [3], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 15, 25, 35, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 114937.0, 12.0, 40.0]], [14.795518962068877, 5.204481037931126], [0.7397759481034437, 0.2602240518965563], 0.0, \"<=50K\"], [\"?\", 129305, \"HS-grad\", 9, \"Married-civ-spouse\", \"?\", \"Husband\", 0, 0, 40, \"United-States\", 3.0, [9, [3], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 7.0, [15, [7], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [3, 9, 25, 39, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 129305.0, 9.0, 40.0]], [15.014134738300372, 4.9858652616996295], [0.7507067369150187, 0.24929326308498148], 0.0, \"<=50K\"], [\"Private\", 365739, \"Some-college\", 10, \"Divorced\", \"Craft-repair\", \"Not-in-family\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 2.0, [7, [2], [1.0]], 1.0, [15, [1], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 27, 33, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 365739.0, 10.0, 40.0]], [17.849313602301354, 2.1506863976986486], [0.8924656801150676, 0.10753431988493241], 0.0, \"<=50K\"], [\"Private\", 69621, \"Assoc-acdm\", 12, \"Never-married\", \"Sales\", \"Not-in-family\", 0, 0, 60, \"United-States\", 0.0, [9, [0], [1.0]], 6.0, [16, [6], [1.0]], 1.0, [7, [1], [1.0]], 4.0, [15, [4], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 15, 26, 36, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 69621.0, 12.0, 60.0]], [17.24813712394819, 2.751862876051808], [0.8624068561974095, 0.1375931438025904], 0.0, \"<=50K\"], [\"Private\", 43323, \"HS-grad\", 9, \"Never-married\", \"Other-service\", \"Not-in-family\", 0, 1762, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 5.0, [15, [5], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 37, 48, 53, 95, 96, 98, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 43323.0, 9.0, 1762.0, 40.0]], [18.32570022614691, 1.6742997738530931], [0.9162850113073454, 0.08371498869265465], 0.0, \"<=50K\"], [\"Self-emp-not-inc\", 120985, \"HS-grad\", 9, \"Married-civ-spouse\", \"Craft-repair\", \"Husband\", 4386, 0, 35, \"United-States\", 1.0, [9, [1], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 1.0, [15, [1], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 9, 25, 33, 47, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 120985.0, 9.0, 4386.0, 35.0]], [9.803081599122933, 10.19691840087707], [0.49015407995614657, 0.5098459200438534], 1.0, \">50K\"], [\"Private\", 254202, \"Bachelors\", 13, \"Married-civ-spouse\", \"Sales\", \"Husband\", 0, 0, 50, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 4.0, [15, [4], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 25, 36, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 254202.0, 13.0, 50.0]], [9.827231581124645, 10.172768418875357], [0.49136157905623223, 0.5086384209437679], 1.0, \">50K\"], [\"Private\", 146195, \"Assoc-acdm\", 12, \"Divorced\", \"Tech-support\", \"Not-in-family\", 0, 0, 36, \"United-States\", 0.0, [9, [0], [1.0]], 6.0, [16, [6], [1.0]], 2.0, [7, [2], [1.0]], 11.0, [15, [11], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 15, 27, 43, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 146195.0, 12.0, 36.0]], [17.849313602301354, 2.1506863976986486], [0.8924656801150676, 0.10753431988493241], 0.0, \"<=50K\"], [\"Federal-gov\", 125933, \"Masters\", 14, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 0, 40, \"Iran\", 6.0, [9, [6], [1.0]], 3.0, [16, [3], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 22.0, [42, [22], [1.0]], [100, [6, 12, 25, 32, 47, 75, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 125933.0, 14.0, 40.0]], [8.559544113131464, 11.440455886868538], [0.4279772056565732, 0.5720227943434268], 1.0, \">50K\"], [\"Self-emp-not-inc\", 56920, \"HS-grad\", 9, \"Married-civ-spouse\", \"Craft-repair\", \"Husband\", 0, 0, 60, \"United-States\", 1.0, [9, [1], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 1.0, [15, [1], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 9, 25, 33, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 56920.0, 9.0, 60.0]], [14.47526405404523, 5.52473594595477], [0.7237632027022615, 0.27623679729773853], 0.0, \"<=50K\"], [\"Private\", 163127, \"Assoc-voc\", 11, \"Married-civ-spouse\", \"Adm-clerical\", \"Wife\", 0, 0, 35, \"United-States\", 0.0, [9, [0], [1.0]], 4.0, [16, [4], [1.0]], 0.0, [7, [0], [1.0]], 3.0, [15, [3], [1.0]], 4.0, [6, [4], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 13, 25, 35, 51, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 163127.0, 11.0, 35.0]], [14.268019803819838, 5.731980196180162], [0.7134009901909919, 0.2865990098090081], 0.0, \"<=50K\"], [\"Private\", 34310, \"Some-college\", 10, \"Never-married\", \"Sales\", \"Own-child\", 0, 0, 20, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 4.0, [15, [4], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 26, 36, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 34310.0, 10.0, 20.0]], [18.432617713253986, 1.5673822867460143], [0.9216308856626994, 0.07836911433730072], 0.0, \"<=50K\"], [\"Private\", 81973, \"Some-college\", 10, \"Married-civ-spouse\", \"Craft-repair\", \"Husband\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 1.0, [15, [1], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 25, 33, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 81973.0, 10.0, 40.0]], [14.681471734838041, 5.318528265161961], [0.7340735867419019, 0.265926413258098], 0.0, \"<=50K\"], [\"Self-emp-inc\", 66614, \"HS-grad\", 9, \"Married-civ-spouse\", \"Craft-repair\", \"Husband\", 0, 0, 40, \"United-States\", 5.0, [9, [5], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 1.0, [15, [1], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [5, 9, 25, 33, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 66614.0, 9.0, 40.0]], [14.489653075591255, 5.510346924408747], [0.7244826537795628, 0.27551734622043733], 0.0, \"<=50K\"], [\"Private\", 232782, \"Some-college\", 10, \"Never-married\", \"Sales\", \"Own-child\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 4.0, [15, [4], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 26, 36, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 232782.0, 10.0, 40.0]], [18.432617713253986, 1.5673822867460143], [0.9216308856626994, 0.07836911433730072], 0.0, \"<=50K\"], [\"Private\", 316868, \"Some-college\", 10, \"Never-married\", \"Other-service\", \"Own-child\", 0, 0, 30, \"Mexico\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 5.0, [15, [5], [1.0]], 2.0, [6, [2], [1.0]], 1.0, [42, [1], [1.0]], [100, [0, 10, 26, 37, 49, 54, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 316868.0, 10.0, 30.0]], [18.657878411906125, 1.342121588093871], [0.9328939205953064, 0.06710607940469357], 0.0, \"<=50K\"], [\"Private\", 196584, \"Assoc-voc\", 11, \"Never-married\", \"Prof-specialty\", \"Not-in-family\", 0, 1564, 40, \"United-States\", 0.0, [9, [0], [1.0]], 4.0, [16, [4], [1.0]], 1.0, [7, [1], [1.0]], 0.0, [15, [0], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 13, 26, 32, 48, 53, 95, 96, 98, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 196584.0, 11.0, 1564.0, 40.0]], [16.995787298944347, 3.0042127010556547], [0.8497893649472174, 0.15021063505278273], 0.0, \"<=50K\"], [\"Private\", 105376, \"Some-college\", 10, \"Never-married\", \"Tech-support\", \"Other-relative\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 11.0, [15, [11], [1.0]], 5.0, [6, [5], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 26, 43, 52, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 105376.0, 10.0, 40.0]], [17.622123869180047, 2.3778761308199523], [0.8811061934590023, 0.11889380654099761], 0.0, \"<=50K\"], [\"Private\", 185814, \"HS-grad\", 9, \"Never-married\", \"Transport-moving\", \"Unmarried\", 0, 0, 30, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 8.0, [15, [8], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 40, 50, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 185814.0, 9.0, 30.0]], [17.83501937594976, 2.1649806240502394], [0.8917509687974879, 0.10824903120251197], 0.0, \"<=50K\"], [\"Private\", 175374, \"Some-college\", 10, \"Married-civ-spouse\", \"Other-service\", \"Husband\", 0, 0, 24, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 5.0, [15, [5], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 25, 37, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 175374.0, 10.0, 24.0]], [15.783038004298891, 4.216961995701109], [0.7891519002149445, 0.21084809978505542], 0.0, \"<=50K\"], [\"Private\", 108293, \"HS-grad\", 9, \"Widowed\", \"Other-service\", \"Unmarried\", 0, 0, 24, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 4.0, [7, [4], [1.0]], 5.0, [15, [5], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 29, 37, 50, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 108293.0, 9.0, 24.0]], [17.91518952856918, 2.0848104714308224], [0.895759476428459, 0.10424052357154112], 0.0, \"<=50K\"], [\"Private\", 181232, \"11th\", 7, \"Married-civ-spouse\", \"Craft-repair\", \"Husband\", 0, 2179, 40, \"United-States\", 0.0, [9, [0], [1.0]], 5.0, [16, [5], [1.0]], 0.0, [7, [0], [1.0]], 1.0, [15, [1], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 14, 25, 33, 47, 53, 95, 96, 98, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 181232.0, 7.0, 2179.0, 40.0]], [12.476871354793365, 7.523128645206638], [0.6238435677396682, 0.37615643226033185], 0.0, \"<=50K\"], [\"?\", 174662, \"Some-college\", 10, \"Divorced\", \"?\", \"Not-in-family\", 0, 0, 40, \"United-States\", 3.0, [9, [3], [1.0]], 1.0, [16, [1], [1.0]], 2.0, [7, [2], [1.0]], 7.0, [15, [7], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [3, 10, 27, 39, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 174662.0, 10.0, 40.0]], [17.906929181443548, 2.093070818556455], [0.8953464590721772, 0.10465354092782274], 0.0, \"<=50K\"], [\"Local-gov\", 186009, \"Some-college\", 10, \"Divorced\", \"Adm-clerical\", \"Unmarried\", 0, 0, 38, \"Mexico\", 2.0, [9, [2], [1.0]], 1.0, [16, [1], [1.0]], 2.0, [7, [2], [1.0]], 3.0, [15, [3], [1.0]], 3.0, [6, [3], [1.0]], 1.0, [42, [1], [1.0]], [100, [2, 10, 27, 35, 50, 54, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 186009.0, 10.0, 38.0]], [17.850821432684334, 2.1491785673156665], [0.8925410716342167, 0.10745892836578333], 0.0, \"<=50K\"], [\"Private\", 198183, \"HS-grad\", 9, \"Never-married\", \"Adm-clerical\", \"Not-in-family\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 3.0, [15, [3], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 35, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 198183.0, 9.0, 40.0]], [18.058022210536322, 1.9419777894636812], [0.902901110526816, 0.09709888947318404], 0.0, \"<=50K\"], [\"Private\", 163003, \"Bachelors\", 13, \"Never-married\", \"Exec-managerial\", \"Other-relative\", 0, 0, 40, \"Philippines\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 1.0, [7, [1], [1.0]], 2.0, [15, [2], [1.0]], 5.0, [6, [5], [1.0]], 3.0, [42, [3], [1.0]], [100, [0, 11, 26, 34, 52, 56, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 163003.0, 13.0, 40.0]], [15.622567967401524, 4.377432032598474], [0.7811283983700762, 0.2188716016299237], 0.0, \"<=50K\"], [\"Private\", 296158, \"HS-grad\", 9, \"Never-married\", \"Craft-repair\", \"Own-child\", 0, 0, 35, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 1.0, [15, [1], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 33, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 296158.0, 9.0, 35.0]], [18.448141733138897, 1.551858266861105], [0.9224070866569448, 0.07759291334305525], 0.0, \"<=50K\"], [\"?\", 252903, \"HS-grad\", 9, \"Divorced\", \"?\", \"Not-in-family\", 0, 0, 45, \"United-States\", 3.0, [9, [3], [1.0]], 0.0, [16, [0], [1.0]], 2.0, [7, [2], [1.0]], 7.0, [15, [7], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [3, 9, 27, 39, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 252903.0, 9.0, 45.0]], [17.054730924316267, 2.945269075683734], [0.8527365462158134, 0.1472634537841867], 0.0, \"<=50K\"], [\"Private\", 187715, \"HS-grad\", 9, \"Married-civ-spouse\", \"Craft-repair\", \"Husband\", 0, 0, 46, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 1.0, [15, [1], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 25, 33, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 187715.0, 9.0, 46.0]], [14.49885588455786, 5.501144115442139], [0.724942794227893, 0.27505720577210696], 0.0, \"<=50K\"], [\"Private\", 214542, \"Bachelors\", 13, \"Never-married\", \"Handlers-cleaners\", \"Not-in-family\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 1.0, [7, [1], [1.0]], 9.0, [15, [9], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 26, 41, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 214542.0, 13.0, 40.0]], [16.982810677846746, 3.0171893221532553], [0.8491405338923373, 0.15085946610766277], 0.0, \"<=50K\"], [\"Self-emp-not-inc\", 494223, \"Some-college\", 10, \"Separated\", \"Sales\", \"Unmarried\", 0, 1816, 2, \"United-States\", 1.0, [9, [1], [1.0]], 1.0, [16, [1], [1.0]], 3.0, [7, [3], [1.0]], 4.0, [15, [4], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 10, 28, 36, 50, 53, 95, 96, 98, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 494223.0, 10.0, 1816.0, 2.0]], [16.39492106634103, 3.60507893365897], [0.8197460533170515, 0.1802539466829485], 0.0, \"<=50K\"], [\"Private\", 191535, \"HS-grad\", 9, \"Divorced\", \"Craft-repair\", \"Not-in-family\", 0, 0, 60, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 2.0, [7, [2], [1.0]], 1.0, [15, [1], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 27, 33, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 191535.0, 9.0, 60.0]], [17.119916008534368, 2.8800839914656318], [0.8559958004267184, 0.1440041995732816], 0.0, \"<=50K\"], [\"Private\", 228456, \"Bachelors\", 13, \"Separated\", \"Other-service\", \"Other-relative\", 0, 0, 50, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 3.0, [7, [3], [1.0]], 5.0, [15, [5], [1.0]], 5.0, [6, [5], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 28, 37, 52, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 228456.0, 13.0, 50.0]], [14.889298437283617, 5.110701562716381], [0.7444649218641809, 0.25553507813581905], 0.0, \"<=50K\"], [\"?\", 38317, \"1st-4th\", 2, \"Divorced\", \"?\", \"Not-in-family\", 0, 0, 20, \"United-States\", 3.0, [9, [3], [1.0]], 14.0, [16, [14], [1.0]], 2.0, [7, [2], [1.0]], 7.0, [15, [7], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [3, 23, 27, 39, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 38317.0, 2.0, 20.0]], [17.906929181443548, 2.093070818556455], [0.8953464590721772, 0.10465354092782274], 0.0, \"<=50K\"], [\"Private\", 252752, \"HS-grad\", 9, \"Never-married\", \"Other-service\", \"Unmarried\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 5.0, [15, [5], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 37, 50, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 252752.0, 9.0, 40.0]], [18.204579941273337, 1.7954200587266589], [0.910228997063667, 0.08977100293633296], 0.0, \"<=50K\"], [\"Self-emp-inc\", 78374, \"Masters\", 14, \"Divorced\", \"Exec-managerial\", \"Unmarried\", 0, 0, 40, \"United-States\", 5.0, [9, [5], [1.0]], 3.0, [16, [3], [1.0]], 2.0, [7, [2], [1.0]], 2.0, [15, [2], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [5, 12, 27, 34, 50, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 78374.0, 14.0, 40.0]], [14.22302349376034, 5.7769765062396585], [0.711151174688017, 0.2888488253119829], 0.0, \"<=50K\"], [\"Private\", 88419, \"HS-grad\", 9, \"Never-married\", \"Exec-managerial\", \"Not-in-family\", 0, 0, 40, \"England\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 2.0, [15, [2], [1.0]], 1.0, [6, [1], [1.0]], 11.0, [42, [11], [1.0]], [100, [0, 9, 26, 34, 48, 64, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 88419.0, 9.0, 40.0]], [17.76053722601131, 2.2394627739886928], [0.8880268613005654, 0.11197313869943464], 0.0, \"<=50K\"], [\"Self-emp-not-inc\", 201080, \"Masters\", 14, \"Married-civ-spouse\", \"Sales\", \"Husband\", 0, 0, 40, \"United-States\", 1.0, [9, [1], [1.0]], 3.0, [16, [3], [1.0]], 0.0, [7, [0], [1.0]], 4.0, [15, [4], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 12, 25, 36, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 201080.0, 14.0, 40.0]], [10.055992640178335, 9.944007359821667], [0.5027996320089168, 0.49720036799108336], 0.0, \"<=50K\"], [\"Private\", 207157, \"Some-college\", 10, \"Divorced\", \"Other-service\", \"Unmarried\", 0, 0, 40, \"Mexico\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 2.0, [7, [2], [1.0]], 5.0, [15, [5], [1.0]], 3.0, [6, [3], [1.0]], 1.0, [42, [1], [1.0]], [100, [0, 10, 27, 37, 50, 54, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 207157.0, 10.0, 40.0]], [18.032102213358755, 1.967897786641245], [0.9016051106679377, 0.09839488933206225], 0.0, \"<=50K\"], [\"Federal-gov\", 235485, \"Assoc-acdm\", 12, \"Never-married\", \"Exec-managerial\", \"Not-in-family\", 0, 0, 42, \"United-States\", 6.0, [9, [6], [1.0]], 6.0, [16, [6], [1.0]], 1.0, [7, [1], [1.0]], 2.0, [15, [2], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [6, 15, 26, 34, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 235485.0, 12.0, 42.0]], [17.318627299169997, 2.6813727008300035], [0.8659313649584999, 0.13406863504150018], 0.0, \"<=50K\"], [\"State-gov\", 102628, \"Masters\", 14, \"Widowed\", \"Protective-serv\", \"Unmarried\", 0, 0, 40, \"United-States\", 4.0, [9, [4], [1.0]], 3.0, [16, [3], [1.0]], 4.0, [7, [4], [1.0]], 12.0, [15, [12], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [4, 12, 29, 44, 50, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 102628.0, 14.0, 40.0]], [14.585941293481937, 5.414058706518062], [0.7292970646740968, 0.27070293532590306], 0.0, \"<=50K\"], [\"Private\", 25828, \"11th\", 7, \"Never-married\", \"Handlers-cleaners\", \"Own-child\", 0, 0, 16, \"United-States\", 0.0, [9, [0], [1.0]], 5.0, [16, [5], [1.0]], 1.0, [7, [1], [1.0]], 9.0, [15, [9], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 14, 26, 41, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 25828.0, 7.0, 16.0]], [18.841432491842458, 1.1585675081575428], [0.9420716245921229, 0.05792837540787714], 0.0, \"<=50K\"], [\"Local-gov\", 54826, \"Assoc-voc\", 11, \"Widowed\", \"Prof-specialty\", \"Not-in-family\", 0, 0, 20, \"United-States\", 2.0, [9, [2], [1.0]], 4.0, [16, [4], [1.0]], 4.0, [7, [4], [1.0]], 0.0, [15, [0], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [2, 13, 29, 32, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 54826.0, 11.0, 20.0]], [17.281606288601566, 2.718393711398434], [0.8640803144300783, 0.1359196855699217], 0.0, \"<=50K\"], [\"Private\", 124953, \"HS-grad\", 9, \"Never-married\", \"Other-service\", \"Not-in-family\", 0, 1980, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 5.0, [15, [5], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 37, 48, 53, 95, 96, 98, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 124953.0, 9.0, 1980.0, 40.0]], [17.708606944884338, 2.2913930551156634], [0.8854303472442169, 0.11456965275578317], 0.0, \"<=50K\"], [\"State-gov\", 175325, \"HS-grad\", 9, \"Married-civ-spouse\", \"Protective-serv\", \"Husband\", 0, 0, 40, \"United-States\", 4.0, [9, [4], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 12.0, [15, [12], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [4, 9, 25, 44, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 175325.0, 9.0, 40.0]], [14.770957661355512, 5.22904233864449], [0.7385478830677756, 0.2614521169322245], 0.0, \"<=50K\"], [\"Private\", 96062, \"Some-college\", 10, \"Married-civ-spouse\", \"Sales\", \"Husband\", 0, 1977, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 4.0, [15, [4], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 25, 36, 47, 53, 95, 96, 98, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 96062.0, 10.0, 1977.0, 40.0]], [12.31878679033016, 7.681213209669844], [0.6159393395165079, 0.38406066048349213], 0.0, \"<=50K\"], [\"Private\", 428030, \"Bachelors\", 13, \"Never-married\", \"Craft-repair\", \"Not-in-family\", 0, 0, 50, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 1.0, [7, [1], [1.0]], 1.0, [15, [1], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 26, 33, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 428030.0, 13.0, 50.0]], [16.039432690679643, 3.960567309320357], [0.8019716345339821, 0.19802836546601785], 0.0, \"<=50K\"], [\"State-gov\", 149624, \"Bachelors\", 13, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 0, 40, \"United-States\", 4.0, [9, [4], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [4, 11, 25, 32, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 149624.0, 13.0, 40.0]], [9.459762207157011, 10.54023779284299], [0.47298811035785054, 0.5270118896421495], 1.0, \">50K\"], [\"Private\", 253814, \"HS-grad\", 9, \"Married-spouse-absent\", \"Sales\", \"Unmarried\", 0, 0, 25, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 5.0, [7, [5], [1.0]], 4.0, [15, [4], [1.0]], 3.0, [6, [3], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 30, 36, 50, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 253814.0, 9.0, 25.0]], [17.545628963245598, 2.4543710367544027], [0.8772814481622799, 0.12271855183772014], 0.0, \"<=50K\"], [\"Private\", 312956, \"HS-grad\", 9, \"Never-married\", \"Craft-repair\", \"Own-child\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 1.0, [15, [1], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 33, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 312956.0, 9.0, 40.0]], [18.448141733138897, 1.551858266861105], [0.9224070866569448, 0.07759291334305525], 0.0, \"<=50K\"], [\"Private\", 483777, \"HS-grad\", 9, \"Never-married\", \"Handlers-cleaners\", \"Not-in-family\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 9.0, [15, [9], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 41, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 483777.0, 9.0, 40.0]], [17.969546059900928, 2.030453940099076], [0.8984773029950462, 0.10152269700495378], 0.0, \"<=50K\"], [\"Private\", 183930, \"HS-grad\", 9, \"Never-married\", \"Other-service\", \"Own-child\", 0, 0, 12, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 5.0, [15, [5], [1.0]], 2.0, [6, [2], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 26, 37, 49, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 183930.0, 9.0, 12.0]], [18.67340243179104, 1.3265975682089617], [0.933670121589552, 0.06632987841044809], 0.0, \"<=50K\"], [\"Private\", 37274, \"Bachelors\", 13, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 0, 65, \"United-States\", 0.0, [9, [0], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 11, 25, 32, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 37274.0, 13.0, 65.0]], [9.31460572528002, 10.685394274719982], [0.465730286264001, 0.5342697137359991], 1.0, \">50K\"], [\"Local-gov\", 181344, \"Some-college\", 10, \"Married-civ-spouse\", \"Exec-managerial\", \"Husband\", 0, 0, 38, \"United-States\", 2.0, [9, [2], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 2.0, [15, [2], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [2, 10, 25, 34, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 181344.0, 10.0, 38.0]], [13.336734175268472, 6.663265824731531], [0.6668367087634235, 0.3331632912365765], 0.0, \"<=50K\"], [\"Private\", 114580, \"Some-college\", 10, \"Divorced\", \"Adm-clerical\", \"Not-in-family\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 2.0, [7, [2], [1.0]], 3.0, [15, [3], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 27, 35, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 114580.0, 10.0, 40.0]], [17.997860843571935, 2.002139156428068], [0.8998930421785966, 0.10010695782140339], 0.0, \"<=50K\"], [\"Private\", 633742, \"Some-college\", 10, \"Never-married\", \"Craft-repair\", \"Not-in-family\", 0, 0, 45, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 1.0, [7, [1], [1.0]], 1.0, [15, [1], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 26, 33, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 633742.0, 10.0, 45.0]], [17.281933330922293, 2.7180666690777073], [0.8640966665461146, 0.13590333345388536], 0.0, \"<=50K\"], [\"Private\", 286370, \"7th-8th\", 4, \"Married-civ-spouse\", \"Machine-op-inspct\", \"Husband\", 0, 0, 40, \"Mexico\", 0.0, [9, [0], [1.0]], 8.0, [16, [8], [1.0]], 0.0, [7, [0], [1.0]], 6.0, [15, [6], [1.0]], 0.0, [6, [0], [1.0]], 1.0, [42, [1], [1.0]], [100, [0, 17, 25, 38, 47, 54, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 286370.0, 4.0, 40.0]], [15.216025698140516, 4.783974301859485], [0.7608012849070258, 0.23919871509297425], 0.0, \"<=50K\"], [\"Federal-gov\", 29054, \"Some-college\", 10, \"Married-civ-spouse\", \"Adm-clerical\", \"Husband\", 0, 0, 42, \"United-States\", 6.0, [9, [6], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 3.0, [15, [3], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [6, 10, 25, 35, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 29054.0, 10.0, 42.0]], [14.794833045203232, 5.205166954796772], [0.7397416522601614, 0.2602583477398385], 0.0, \"<=50K\"], [\"Private\", 304030, \"HS-grad\", 9, \"Married-civ-spouse\", \"Adm-clerical\", \"Husband\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 0.0, [16, [0], [1.0]], 0.0, [7, [0], [1.0]], 3.0, [15, [3], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 9, 25, 35, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 304030.0, 9.0, 40.0]], [14.886196138176112, 5.11380386182389], [0.7443098069088054, 0.25569019309119445], 0.0, \"<=50K\"], [\"Self-emp-not-inc\", 143129, \"Bachelors\", 13, \"Divorced\", \"Exec-managerial\", \"Not-in-family\", 0, 0, 40, \"United-States\", 1.0, [9, [1], [1.0]], 2.0, [16, [2], [1.0]], 2.0, [7, [2], [1.0]], 2.0, [15, [2], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 11, 27, 34, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 143129.0, 13.0, 40.0]], [15.575456979562608, 4.424543020437391], [0.7787728489781304, 0.22122715102186957], 0.0, \"<=50K\"], [\"?\", 135105, \"Bachelors\", 13, \"Divorced\", \"?\", \"Not-in-family\", 0, 0, 50, \"United-States\", 3.0, [9, [3], [1.0]], 2.0, [16, [2], [1.0]], 2.0, [7, [2], [1.0]], 7.0, [15, [7], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [3, 11, 27, 39, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 135105.0, 13.0, 50.0]], [14.748724891906743, 5.251275108093256], [0.7374362445953372, 0.2625637554046628], 0.0, \"<=50K\"], [\"Private\", 99928, \"Masters\", 14, \"Married-civ-spouse\", \"Prof-specialty\", \"Wife\", 0, 0, 50, \"United-States\", 0.0, [9, [0], [1.0]], 3.0, [16, [3], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 4.0, [6, [4], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 12, 25, 32, 51, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 99928.0, 14.0, 50.0]], [10.124924745447167, 9.875075254552833], [0.5062462372723584, 0.49375376272764165], 0.0, \"<=50K\"], [\"State-gov\", 109567, \"Doctorate\", 16, \"Married-civ-spouse\", \"Prof-specialty\", \"Husband\", 0, 0, 1, \"United-States\", 4.0, [9, [4], [1.0]], 12.0, [16, [12], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [4, 21, 25, 32, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 109567.0, 16.0, 1.0]], [8.63676240875909, 11.36323759124091], [0.4318381204379545, 0.5681618795620456], 1.0, \">50K\"], [\"Private\", 155222, \"Some-college\", 10, \"Divorced\", \"Machine-op-inspct\", \"Not-in-family\", 0, 0, 28, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 2.0, [7, [2], [1.0]], 6.0, [15, [6], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 27, 38, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 155222.0, 10.0, 28.0]], [17.849313602301354, 2.1506863976986486], [0.8924656801150676, 0.10753431988493241], 0.0, \"<=50K\"], [\"Private\", 159567, \"Some-college\", 10, \"Married-civ-spouse\", \"Machine-op-inspct\", \"Husband\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 6.0, [15, [6], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 25, 38, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 159567.0, 10.0, 40.0]], [14.880900639533383, 5.119099360466619], [0.7440450319766692, 0.25595496802333095], 0.0, \"<=50K\"], [\"Local-gov\", 523910, \"Bachelors\", 13, \"Married-civ-spouse\", \"Craft-repair\", \"Husband\", 0, 0, 40, \"United-States\", 2.0, [9, [2], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 1.0, [15, [1], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [2, 11, 25, 33, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 523910.0, 13.0, 40.0]], [10.54181322061976, 9.45818677938024], [0.527090661030988, 0.472909338969012], 0.0, \"<=50K\"], [\"Private\", 120939, \"Some-college\", 10, \"Married-civ-spouse\", \"Tech-support\", \"Husband\", 0, 0, 45, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 0.0, [7, [0], [1.0]], 11.0, [15, [11], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 25, 43, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 120939.0, 10.0, 45.0]], [14.408178708450626, 5.591821291549375], [0.7204089354225313, 0.27959106457746874], 0.0, \"<=50K\"], [\"Federal-gov\", 130760, \"Bachelors\", 13, \"Married-civ-spouse\", \"Tech-support\", \"Husband\", 0, 0, 24, \"United-States\", 6.0, [9, [6], [1.0]], 2.0, [16, [2], [1.0]], 0.0, [7, [0], [1.0]], 11.0, [15, [11], [1.0]], 0.0, [6, [0], [1.0]], 0.0, [42, [0], [1.0]], [100, [6, 11, 25, 43, 47, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 130760.0, 13.0, 24.0]], [10.521888094454154, 9.478111905545846], [0.5260944047227076, 0.4739055952772923], 0.0, \"<=50K\"], [\"Private\", 197387, \"5th-6th\", 3, \"Married-civ-spouse\", \"Transport-moving\", \"Other-relative\", 0, 0, 40, \"Mexico\", 0.0, [9, [0], [1.0]], 13.0, [16, [13], [1.0]], 0.0, [7, [0], [1.0]], 8.0, [15, [8], [1.0]], 5.0, [6, [5], [1.0]], 1.0, [42, [1], [1.0]], [100, [0, 22, 25, 40, 52, 54, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 197387.0, 3.0, 40.0]], [16.555915257769403, 3.444084742230595], [0.8277957628884703, 0.17220423711152977], 0.0, \"<=50K\"], [\"Private\", 99374, \"Some-college\", 10, \"Divorced\", \"Craft-repair\", \"Not-in-family\", 0, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 1.0, [16, [1], [1.0]], 2.0, [7, [2], [1.0]], 1.0, [15, [1], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 10, 27, 33, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 99374.0, 10.0, 40.0]], [17.849313602301354, 2.1506863976986486], [0.8924656801150676, 0.10753431988493241], 0.0, \"<=50K\"], [\"Federal-gov\", 56795, \"Masters\", 14, \"Never-married\", \"Exec-managerial\", \"Not-in-family\", 14084, 0, 55, \"United-States\", 6.0, [9, [6], [1.0]], 3.0, [16, [3], [1.0]], 1.0, [7, [1], [1.0]], 2.0, [15, [2], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [6, 12, 26, 34, 48, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 56795.0, 14.0, 14084.0, 55.0]], [8.991841877654018, 11.008158122345984], [0.4495920938827009, 0.5504079061172992], 1.0, \">50K\"], [\"Private\", 138992, \"Masters\", 14, \"Married-civ-spouse\", \"Prof-specialty\", \"Other-relative\", 7298, 0, 40, \"United-States\", 0.0, [9, [0], [1.0]], 3.0, [16, [3], [1.0]], 0.0, [7, [0], [1.0]], 0.0, [15, [0], [1.0]], 5.0, [6, [5], [1.0]], 0.0, [42, [0], [1.0]], [100, [0, 12, 25, 32, 52, 53, 95, 96, 97, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 138992.0, 14.0, 7298.0, 40.0]], [9.839133145436094, 10.160866854563906], [0.4919566572718047, 0.5080433427281953], 1.0, \">50K\"], [\"Self-emp-not-inc\", 32921, \"HS-grad\", 9, \"Never-married\", \"Sales\", \"Not-in-family\", 0, 0, 40, \"United-States\", 1.0, [9, [1], [1.0]], 0.0, [16, [0], [1.0]], 1.0, [7, [1], [1.0]], 4.0, [15, [4], [1.0]], 1.0, [6, [1], [1.0]], 0.0, [42, [0], [1.0]], [100, [1, 9, 26, 36, 48, 53, 95, 96, 99], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 32921.0, 9.0, 40.0]], [17.85232889811733, 2.147671101882669], [0.8926164449058666, 0.10738355509413344], 0.0, \"<=50K\"]]}]}\n"
     ]
    }
   ],
   "source": [
    "no_of_records_to_score = 200\n",
    "payload_scoring, scoring_response = sample_scoring(no_of_records_to_score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7166ec0e5378460782597ef4debbd0d6"
   },
   "source": [
    "### List the original model predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "id": "18debd5a405e42af9f72b28dd8af58ba"
   },
   "outputs": [],
   "source": [
    "# for i in range(no_of_records_to_score):\n",
    "#     print(scoring_response['predictions'][0]['values'][i][-1:][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4f60bc6f2f8c444988bc6e7b498a4331"
   },
   "source": [
    "## Get the token for calling OpenScale API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "id": "b6e98dc87c5a4f47817899ed8af6325d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6ImM1NG4yLWRFSU9NS0ZsNHUwZFZyaW5VcE1EazdRSFdPV2h2Y19oQnpleW8ifQ.eyJ1c2VybmFtZSI6ImNwYWRtaW4iLCJyb2xlIjoiQWRtaW4iLCJwZXJtaXNzaW9ucyI6WyJtYW5hZ2VfY2F0YWxvZyIsImFkZF9jYXRhbG9nX2Fzc2V0c190b19wcm9qZWN0cyIsImFkbWluaXN0cmF0b3IiLCJjYW5fcHJvdmlzaW9uIiwibW9uaXRvcl9wbGF0Zm9ybSIsImNvbmZpZ3VyZV9wbGF0Zm9ybSIsInZpZXdfcGxhdGZvcm1faGVhbHRoIiwiY29uZmlndXJlX2F1dGgiLCJtYW5hZ2VfdXNlcnMiLCJtYW5hZ2VfZ3JvdXBzIiwibWFuYWdlX3NlcnZpY2VfaW5zdGFuY2VzIiwibWFuYWdlX3ZhdWx0c19hbmRfc2VjcmV0cyIsInNoYXJlX3NlY3JldHMiLCJhZGRfdmF1bHRzIiwibWFuYWdlX2xvY2F0aW9ucyIsIm1hbmFnZV9kYXRhX3BsYW5lcyIsInZpZXdfbG9jYXRpb25zIiwidmlld19kYXRhX3BsYW5lcyIsImNyZWF0ZV9wcm9qZWN0IiwiY3JlYXRlX3NwYWNlIl0sImdyb3VwcyI6WzEwMDAwXSwic3ViIjoiY3BhZG1pbiIsImlzcyI6IktOT1hTU08iLCJhdWQiOiJEU1giLCJ1aWQiOiIxMDAwMzMxMDAxIiwiYXV0aGVudGljYXRvciI6ImV4dGVybmFsIiwiaWFtIjp7ImFjY2Vzc1Rva2VuIjoiZTFkMTNkMDVmMDIxMGZiZTJmNWM3NjE5ZDIwODQyYmZiOWNlYmQzNTdhZmFjYzUxZjBlMWYwNTdlNmUwYmFiNDA1Nzg5ZmUwMDY1NjYwODMzY2ZmM2ZlMTQ3MDI1OGRiYWZjMTg5ODViYjczY2RjZDQ0ZmJhN2I3NTQ2ZjIwYmE2NDc3ODFkNWMyOGUyNzE3NTY3NGM2YzVmZTcxM2E2ODgyZjVjODBmMGRlN2M2NjdhYWVlZTk5ZTgwYWY1ZjFlY2JiOWJhZjdkMmFhMDkyYzM1MWRhMTg5ZTdiNTAzYmM1Mjc5MTkxZGFjZDhhNDdhNDhlMmFmNjkxMWYxYTcwNDcwMTMwNmYzNjViMzcwNmUxZjEwOWQ5N2I3MzRmNGFjYjU1MjNiNDZhNzdjMTY3NGQzNTA4ZGRkMGY3ZGZhMjYxNmQ4N2IwZjRjNGY1MmU5YTY4ZThlMDZjYmU0MDFjNWE1MmM3YjU4M2NkNDA3N2VlZGZkMjhmY2ZmOTY5ZmVjZGNmMWU1ZTQ2MTUwZWNjMDE1MzMxMDA5ODM0ZjkwZWU5MTVmMmMwMmVhM2Y2ZDlmNDE5MDI4NzBkMTlhMzc1NWNjNTYxMTc1MmIxN2E3OWI0NjZkMjAzZTYyNTM4YzcyNmQ2N2NhMWQzMDJiYmQ0ZGYwNzAyNDM2Yjk0YjI3ZGZkY2I1OGJmNDI2ZTU0OTA3ZWNkOTA5ZTdlYWNmYTg4YTc4NjFjYzFjOWQzNThmY2Y1NDE0MmNiMjFkOTg4MmZmNDEzMmNmM2YwNjRlYmVlMjUwNGRhYmRhOGYzYWZmYWRkNzM3M2YxM2U2NjI2NDMxYTA3ZTExNDA2NzYwYTgzYzZjNGI2YzgyODdlNDViZmZjMWVjYjk1MGNhMTUyZjYwZGM1MzMyZjI5ZTUxZWE4YTg4NmYxYzlhYTMyNzhiZjQxZGNiM2UwYTJlY2VjM2M1OTNkMjA1NzAwOWI1YmYyZDJlMjBlOWM1ZTM0ZTI2ZjgyYjUxYmRlZGI2ZDViNjM1MjY0Yjk3MWQ3M2VhZjJlMzgwMTVhZDZlZWZhZDAzNDhiNjFmMjgyYjVjOGUzMjZjN2FlYzNhN2U4YTc1YmM4YjRjZDNmNjUyMTIxNTc3ODQxNzkxZjAyNWZjMWMxMzRlZjRiNzZjYjFlMTkzODFlYzU3N2FjMTFkZTBiMjk0N2U4MmRmZjMzNzUxZTU2NmY5YmUxOTRlZDc2YzgzMzllNThkMDE2Nzk5OWM0OWYxYzc4NDdiOTA4YzY2YTkxNTMwMDZhNmExNmNjNmEzNTE3ODEwZmI4ZjEwMjc1NWM5MjBkYzAyZjdlNGViODYwZDBjMGNhYWY4YmYwNTE5ZTY0OSJ9LCJkaXNwbGF5X25hbWUiOiJjcGFkbWluIiwiYXBpX3JlcXVlc3QiOmZhbHNlLCJpYXQiOjE3MjI5NTA4MzgsImV4cCI6MTcyMjk5NDAzOH0.PfSBCWZJvXeNU4ezAC7s9ZMV4AqWnOHINPPo_MXaFqB5WrW43fcGa7MRDPIet7ftldFA7OXEG1_zKbqp82t4UltmQZyBONdEtFImzmRMNmkEgj02tU5xRhmdDuOrIzHHYpdaQkYXben-ZAfxhB2JOqnJjhS_tWaVXHCpIbeENDkQ2TUXIFWNChV-eO3DapjtZWKlp2CBTG04WDoUQ7rKo8gc9MhY-jjVDBXhfe1eHXFfnA03aSpGJ82r0ISFITx8DkXz-Fg3cY7iTT-YSBpxSb6SVADmREUFqJKlo5t_a0gUnQA-EB3nIPlCGqGN6qlbq-CDHlNej8g5bR1FPdYhfw\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "import urllib3\n",
    "from http import HTTPStatus\n",
    "\n",
    "def get_iamtoken(url, username, password):\n",
    "    fqdn = urllib3.util.parse_url(url).netloc\n",
    "    domain = '.'.join(fqdn.split('.')[1:])\n",
    "    token_url = 'https://cp-console.{}/idprovider/v1/auth/identitytoken'.format(domain)\n",
    "    data = {\n",
    "        'grant_type': 'password',\n",
    "        'username': username,\n",
    "        'password': password,\n",
    "        'scope': 'openid'\n",
    "    }\n",
    "    return requests.post(token_url, data, verify=False)\n",
    "\n",
    "def get_accesstoken(url, username, iamtoken):\n",
    "    url = '{}/v1/preauth/validateAuth'.format(url)\n",
    "    headers = {\n",
    "        'Content-type': 'application/json',\n",
    "        'username': username,\n",
    "        'iam-token': iamtoken\n",
    "    }\n",
    "    return requests.get(url, headers=headers, verify=False)\n",
    "\n",
    "def generate_access_token():\n",
    "    url=WOS_CREDENTIALS['url']\n",
    "    username=WOS_CREDENTIALS['username']\n",
    "    password=WOS_CREDENTIALS['password'] \n",
    "    response = get_iamtoken(url,username,password)\n",
    "    #service is not available when iamintegration=false so fall back to old way of generating code\n",
    "    if response.status_code==HTTPStatus.SERVICE_UNAVAILABLE:\n",
    "        url = '{}/v1/preauth/validateAuth'.format(url)\n",
    "        headers = {'Content-type': 'application/json'}\n",
    "        data = {\n",
    "            'grant_type': 'password',\n",
    "            'username': username,\n",
    "            'password': password\n",
    "        }\n",
    "        return requests.get(url, headers=headers, auth= (username,password),verify=False).json()['accessToken']\n",
    "        \n",
    "    else:\n",
    "        return get_accesstoken(url,username, response.json()['access_token']).json()['accessToken']\n",
    "\n",
    "access_token = generate_access_token()\n",
    "print(access_token)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "id": "cfecded322744744881cc4e2e944212e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://cpd-cpd-instance.apps.wos415nfs2672.cp.fyre.ibm.com/openscale/00000000-0000-0000-0000-000000000000/v2/subscriptions/c274e76c-6bb9-43ce-ba64-127a3db95ede/predictions\n",
      "\n",
      ">>>>>>>>>>>>>>>\n",
      "\n",
      "{'fields': ['workclass', 'fnlwgt', 'education', 'education-num', 'Marital', 'occupation', 'relationship', 'capitalgain', 'loss', 'hoursper', 'citizen_status'], 'values': [['State-gov', 77516, 'Bachelors', 13, 'Never-married', 'Adm-clerical', 'Not-in-family', 2174, 0, 40, 'United-States'], ['Self-emp-not-inc', 83311, 'Bachelors', 13, 'Married-civ-spouse', 'Exec-managerial', 'Husband', 0, 0, 13, 'United-States'], ['Private', 215646, 'HS-grad', 9, 'Divorced', 'Handlers-cleaners', 'Not-in-family', 0, 0, 40, 'United-States'], ['Private', 234721, '11th', 7, 'Married-civ-spouse', 'Handlers-cleaners', 'Husband', 0, 0, 40, 'United-States'], ['Private', 338409, 'Bachelors', 13, 'Married-civ-spouse', 'Prof-specialty', 'Wife', 0, 0, 40, 'Cuba'], ['Private', 284582, 'Masters', 14, 'Married-civ-spouse', 'Exec-managerial', 'Wife', 0, 0, 40, 'United-States'], ['Private', 160187, '9th', 5, 'Married-spouse-absent', 'Other-service', 'Not-in-family', 0, 0, 16, 'Jamaica'], ['Self-emp-not-inc', 209642, 'HS-grad', 9, 'Married-civ-spouse', 'Exec-managerial', 'Husband', 0, 0, 45, 'United-States'], ['Private', 45781, 'Masters', 14, 'Never-married', 'Prof-specialty', 'Not-in-family', 14084, 0, 50, 'United-States'], ['Private', 159449, 'Bachelors', 13, 'Married-civ-spouse', 'Exec-managerial', 'Husband', 5178, 0, 40, 'United-States'], ['Private', 280464, 'Some-college', 10, 'Married-civ-spouse', 'Exec-managerial', 'Husband', 0, 0, 80, 'United-States'], ['State-gov', 141297, 'Bachelors', 13, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 0, 40, 'India'], ['Private', 122272, 'Bachelors', 13, 'Never-married', 'Adm-clerical', 'Own-child', 0, 0, 30, 'United-States'], ['Private', 205019, 'Assoc-acdm', 12, 'Never-married', 'Sales', 'Not-in-family', 0, 0, 50, 'United-States'], ['Private', 121772, 'Assoc-voc', 11, 'Married-civ-spouse', 'Craft-repair', 'Husband', 0, 0, 40, '?'], ['Private', 245487, '7th-8th', 4, 'Married-civ-spouse', 'Transport-moving', 'Husband', 0, 0, 45, 'Mexico'], ['Self-emp-not-inc', 176756, 'HS-grad', 9, 'Never-married', 'Farming-fishing', 'Own-child', 0, 0, 35, 'United-States'], ['Private', 186824, 'HS-grad', 9, 'Never-married', 'Machine-op-inspct', 'Unmarried', 0, 0, 40, 'United-States'], ['Private', 28887, '11th', 7, 'Married-civ-spouse', 'Sales', 'Husband', 0, 0, 50, 'United-States'], ['Self-emp-not-inc', 292175, 'Masters', 14, 'Divorced', 'Exec-managerial', 'Unmarried', 0, 0, 45, 'United-States'], ['Private', 193524, 'Doctorate', 16, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 0, 60, 'United-States'], ['Private', 302146, 'HS-grad', 9, 'Separated', 'Other-service', 'Unmarried', 0, 0, 20, 'United-States'], ['Federal-gov', 76845, '9th', 5, 'Married-civ-spouse', 'Farming-fishing', 'Husband', 0, 0, 40, 'United-States'], ['Private', 117037, '11th', 7, 'Married-civ-spouse', 'Transport-moving', 'Husband', 0, 2042, 40, 'United-States'], ['Private', 109015, 'HS-grad', 9, 'Divorced', 'Tech-support', 'Unmarried', 0, 0, 40, 'United-States'], ['Local-gov', 216851, 'Bachelors', 13, 'Married-civ-spouse', 'Tech-support', 'Husband', 0, 0, 40, 'United-States'], ['Private', 168294, 'HS-grad', 9, 'Never-married', 'Craft-repair', 'Own-child', 0, 0, 40, 'United-States'], ['?', 180211, 'Some-college', 10, 'Married-civ-spouse', '?', 'Husband', 0, 0, 60, 'South'], ['Private', 367260, 'HS-grad', 9, 'Divorced', 'Exec-managerial', 'Not-in-family', 0, 0, 80, 'United-States'], ['Private', 193366, 'HS-grad', 9, 'Married-civ-spouse', 'Craft-repair', 'Husband', 0, 0, 40, 'United-States'], ['Local-gov', 190709, 'Assoc-acdm', 12, 'Never-married', 'Protective-serv', 'Not-in-family', 0, 0, 52, 'United-States'], ['Private', 266015, 'Some-college', 10, 'Never-married', 'Sales', 'Own-child', 0, 0, 44, 'United-States'], ['Private', 386940, 'Bachelors', 13, 'Divorced', 'Exec-managerial', 'Own-child', 0, 1408, 40, 'United-States'], ['Federal-gov', 59951, 'Some-college', 10, 'Married-civ-spouse', 'Adm-clerical', 'Own-child', 0, 0, 40, 'United-States'], ['State-gov', 311512, 'Some-college', 10, 'Married-civ-spouse', 'Other-service', 'Husband', 0, 0, 15, 'United-States'], ['Private', 242406, '11th', 7, 'Never-married', 'Machine-op-inspct', 'Unmarried', 0, 0, 40, 'Puerto-Rico'], ['Private', 197200, 'Some-college', 10, 'Never-married', 'Machine-op-inspct', 'Own-child', 0, 0, 40, 'United-States'], ['Private', 544091, 'HS-grad', 9, 'Married-AF-spouse', 'Adm-clerical', 'Wife', 0, 0, 25, 'United-States'], ['Private', 84154, 'Some-college', 10, 'Married-civ-spouse', 'Sales', 'Husband', 0, 0, 38, '?'], ['Self-emp-not-inc', 265477, 'Assoc-acdm', 12, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 0, 40, 'United-States'], ['Private', 507875, '9th', 5, 'Married-civ-spouse', 'Machine-op-inspct', 'Husband', 0, 0, 43, 'United-States'], ['Self-emp-not-inc', 88506, 'Bachelors', 13, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 0, 40, 'United-States'], ['Private', 172987, 'Bachelors', 13, 'Married-civ-spouse', 'Tech-support', 'Husband', 0, 0, 50, 'United-States'], ['Private', 94638, 'HS-grad', 9, 'Separated', 'Adm-clerical', 'Unmarried', 0, 0, 40, 'United-States'], ['Private', 289980, 'HS-grad', 9, 'Never-married', 'Handlers-cleaners', 'Not-in-family', 0, 0, 35, 'United-States'], ['Federal-gov', 337895, 'Bachelors', 13, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 0, 40, 'United-States'], ['Private', 144361, 'HS-grad', 9, 'Married-civ-spouse', 'Machine-op-inspct', 'Husband', 0, 0, 38, 'United-States'], ['Private', 128354, 'Masters', 14, 'Divorced', 'Exec-managerial', 'Unmarried', 0, 0, 40, 'United-States'], ['State-gov', 101603, 'Assoc-voc', 11, 'Married-civ-spouse', 'Craft-repair', 'Husband', 0, 0, 40, 'United-States'], ['Private', 271466, 'Assoc-voc', 11, 'Never-married', 'Prof-specialty', 'Not-in-family', 0, 0, 43, 'United-States'], ['Private', 32275, 'Some-college', 10, 'Married-civ-spouse', 'Exec-managerial', 'Wife', 0, 0, 40, 'United-States'], ['Private', 226956, 'HS-grad', 9, 'Never-married', 'Other-service', 'Own-child', 0, 0, 30, '?'], ['Private', 51835, 'Prof-school', 15, 'Married-civ-spouse', 'Prof-specialty', 'Wife', 0, 1902, 60, 'Honduras'], ['Federal-gov', 251585, 'Bachelors', 13, 'Divorced', 'Exec-managerial', 'Not-in-family', 0, 0, 55, 'United-States'], ['Self-emp-inc', 109832, 'HS-grad', 9, 'Divorced', 'Exec-managerial', 'Not-in-family', 0, 0, 60, 'United-States'], ['Private', 237993, 'Some-college', 10, 'Married-civ-spouse', 'Tech-support', 'Husband', 0, 0, 40, 'United-States'], ['Private', 216666, '5th-6th', 3, 'Married-civ-spouse', 'Machine-op-inspct', 'Husband', 0, 0, 40, 'Mexico'], ['Private', 56352, 'Assoc-voc', 11, 'Married-civ-spouse', 'Other-service', 'Husband', 0, 0, 40, 'Puerto-Rico'], ['Private', 147372, 'HS-grad', 9, 'Married-civ-spouse', 'Adm-clerical', 'Husband', 0, 0, 48, 'United-States'], ['Private', 188146, 'HS-grad', 9, 'Married-civ-spouse', 'Machine-op-inspct', 'Husband', 5013, 0, 40, 'United-States'], ['Private', 59496, 'Bachelors', 13, 'Married-civ-spouse', 'Sales', 'Husband', 2407, 0, 40, 'United-States'], ['?', 293936, '7th-8th', 4, 'Married-spouse-absent', '?', 'Not-in-family', 0, 0, 40, '?'], ['Private', 149640, 'HS-grad', 9, 'Married-civ-spouse', 'Transport-moving', 'Husband', 0, 0, 40, 'United-States'], ['Private', 116632, 'Doctorate', 16, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 0, 45, 'United-States'], ['Private', 105598, 'Some-college', 10, 'Divorced', 'Tech-support', 'Not-in-family', 0, 0, 58, 'United-States'], ['Private', 155537, 'HS-grad', 9, 'Married-civ-spouse', 'Craft-repair', 'Husband', 0, 0, 40, 'United-States'], ['Private', 183175, 'Some-college', 10, 'Divorced', 'Adm-clerical', 'Not-in-family', 0, 0, 40, 'United-States'], ['Private', 169846, 'HS-grad', 9, 'Married-civ-spouse', 'Adm-clerical', 'Wife', 0, 0, 40, 'United-States'], ['Self-emp-inc', 191681, 'Some-college', 10, 'Married-civ-spouse', 'Exec-managerial', 'Husband', 0, 0, 50, 'United-States'], ['?', 200681, 'Some-college', 10, 'Never-married', '?', 'Own-child', 0, 0, 40, 'United-States'], ['Private', 101509, 'Some-college', 10, 'Never-married', 'Prof-specialty', 'Own-child', 0, 0, 32, 'United-States'], ['Private', 309974, 'Bachelors', 13, 'Separated', 'Sales', 'Own-child', 0, 0, 40, 'United-States'], ['Self-emp-not-inc', 162298, 'Bachelors', 13, 'Married-civ-spouse', 'Sales', 'Husband', 0, 0, 70, 'United-States'], ['Private', 211678, 'Some-college', 10, 'Never-married', 'Machine-op-inspct', 'Not-in-family', 0, 0, 40, 'United-States'], ['Private', 124744, 'Some-college', 10, 'Married-civ-spouse', 'Prof-specialty', 'Other-relative', 0, 0, 20, 'United-States'], ['Private', 213921, 'HS-grad', 9, 'Never-married', 'Other-service', 'Own-child', 0, 0, 40, 'Mexico'], ['Private', 32214, 'Assoc-acdm', 12, 'Married-civ-spouse', 'Adm-clerical', 'Husband', 0, 0, 40, 'United-States'], ['?', 212759, '10th', 6, 'Married-civ-spouse', '?', 'Husband', 0, 0, 2, 'United-States'], ['Private', 309634, '11th', 7, 'Never-married', 'Other-service', 'Own-child', 0, 0, 22, 'United-States'], ['Local-gov', 125927, '7th-8th', 4, 'Married-civ-spouse', 'Farming-fishing', 'Husband', 0, 0, 40, 'United-States'], ['Private', 446839, 'HS-grad', 9, 'Never-married', 'Sales', 'Not-in-family', 0, 0, 30, 'United-States'], ['Private', 276515, 'Bachelors', 13, 'Married-civ-spouse', 'Other-service', 'Husband', 0, 0, 40, 'Cuba'], ['Private', 51618, 'HS-grad', 9, 'Married-civ-spouse', 'Other-service', 'Wife', 0, 0, 40, 'United-States'], ['Private', 159937, 'HS-grad', 9, 'Married-civ-spouse', 'Sales', 'Husband', 0, 0, 48, 'United-States'], ['Private', 343591, 'HS-grad', 9, 'Divorced', 'Craft-repair', 'Not-in-family', 14344, 0, 40, 'United-States'], ['Private', 346253, 'HS-grad', 9, 'Divorced', 'Sales', 'Own-child', 0, 0, 35, 'United-States'], ['Local-gov', 268234, 'HS-grad', 9, 'Married-civ-spouse', 'Protective-serv', 'Husband', 0, 0, 40, 'United-States'], ['Private', 202051, 'Masters', 14, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 0, 50, 'United-States'], ['Private', 54334, '9th', 5, 'Never-married', 'Sales', 'Not-in-family', 0, 0, 40, 'United-States'], ['Federal-gov', 410867, 'Doctorate', 16, 'Never-married', 'Prof-specialty', 'Not-in-family', 0, 0, 50, 'United-States'], ['Private', 249977, 'Assoc-voc', 11, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 0, 40, 'United-States'], ['Private', 286730, 'Some-college', 10, 'Divorced', 'Craft-repair', 'Unmarried', 0, 0, 40, 'United-States'], ['Private', 212563, 'Some-college', 10, 'Divorced', 'Machine-op-inspct', 'Unmarried', 0, 0, 25, 'United-States'], ['Private', 117747, 'HS-grad', 9, 'Married-civ-spouse', 'Sales', 'Wife', 0, 1573, 35, '?'], ['Local-gov', 226296, 'Bachelors', 13, 'Married-civ-spouse', 'Protective-serv', 'Husband', 0, 0, 40, 'United-States'], ['Local-gov', 115585, 'Some-college', 10, 'Never-married', 'Handlers-cleaners', 'Not-in-family', 0, 0, 50, 'United-States'], ['Self-emp-not-inc', 191277, 'Doctorate', 16, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 1902, 60, 'United-States'], ['Private', 202683, 'Some-college', 10, 'Married-civ-spouse', 'Sales', 'Husband', 0, 0, 48, 'United-States'], ['Private', 171095, 'Assoc-acdm', 12, 'Divorced', 'Exec-managerial', 'Unmarried', 0, 0, 40, 'England'], ['Federal-gov', 249409, 'HS-grad', 9, 'Never-married', 'Other-service', 'Own-child', 0, 0, 40, 'United-States'], ['Private', 124191, 'Masters', 14, 'Married-civ-spouse', 'Exec-managerial', 'Husband', 0, 0, 40, 'United-States'], ['Private', 198282, 'Bachelors', 13, 'Married-civ-spouse', 'Exec-managerial', 'Husband', 15024, 0, 60, 'United-States'], ['Self-emp-not-inc', 149116, 'Masters', 14, 'Never-married', 'Prof-specialty', 'Not-in-family', 0, 0, 50, 'United-States'], ['Private', 188300, 'Some-college', 10, 'Never-married', 'Tech-support', 'Own-child', 0, 0, 40, 'United-States'], ['Private', 103432, 'HS-grad', 9, 'Never-married', 'Craft-repair', 'Not-in-family', 0, 0, 40, 'United-States'], ['Self-emp-inc', 317660, 'HS-grad', 9, 'Married-civ-spouse', 'Craft-repair', 'Husband', 7688, 0, 40, 'United-States'], ['?', 304873, '10th', 6, 'Never-married', '?', 'Own-child', 34095, 0, 32, 'United-States'], ['Private', 194901, '11th', 7, 'Never-married', 'Handlers-cleaners', 'Own-child', 0, 0, 40, 'United-States'], ['Local-gov', 189265, 'HS-grad', 9, 'Never-married', 'Adm-clerical', 'Not-in-family', 0, 0, 40, 'United-States'], ['Private', 124692, 'HS-grad', 9, 'Married-civ-spouse', 'Handlers-cleaners', 'Husband', 0, 0, 40, 'United-States'], ['Private', 432376, 'Bachelors', 13, 'Never-married', 'Sales', 'Other-relative', 0, 0, 40, 'United-States'], ['Private', 65324, 'Prof-school', 15, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 0, 40, 'United-States'], ['Self-emp-not-inc', 335605, 'HS-grad', 9, 'Married-civ-spouse', 'Other-service', 'Husband', 0, 1887, 50, 'Canada'], ['Private', 377869, 'Some-college', 10, 'Married-civ-spouse', 'Sales', 'Wife', 4064, 0, 25, 'United-States'], ['Private', 102864, 'HS-grad', 9, 'Never-married', 'Machine-op-inspct', 'Own-child', 0, 0, 40, 'United-States'], ['Private', 95647, '9th', 5, 'Married-civ-spouse', 'Handlers-cleaners', 'Husband', 0, 0, 50, 'United-States'], ['Self-emp-inc', 303090, 'Some-college', 10, 'Married-civ-spouse', 'Sales', 'Husband', 0, 0, 50, 'United-States'], ['Local-gov', 197371, 'Assoc-voc', 11, 'Married-civ-spouse', 'Craft-repair', 'Husband', 0, 0, 40, 'United-States'], ['Private', 247552, 'Some-college', 10, 'Married-civ-spouse', 'Sales', 'Husband', 0, 0, 56, 'United-States'], ['Private', 102632, 'HS-grad', 9, 'Never-married', 'Craft-repair', 'Not-in-family', 0, 0, 41, 'United-States'], ['Private', 199915, 'Some-college', 10, 'Never-married', 'Other-service', 'Own-child', 0, 0, 40, 'United-States'], ['Private', 118853, 'Bachelors', 13, 'Married-civ-spouse', 'Exec-managerial', 'Husband', 0, 0, 60, 'United-States'], ['Private', 77143, 'Bachelors', 13, 'Never-married', 'Exec-managerial', 'Own-child', 0, 0, 40, 'Germany'], ['State-gov', 267989, 'Bachelors', 13, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 0, 50, 'United-States'], ['Private', 301606, 'Some-college', 10, 'Never-married', 'Other-service', 'Own-child', 0, 0, 35, 'United-States'], ['Private', 287828, 'Bachelors', 13, 'Married-civ-spouse', 'Exec-managerial', 'Wife', 0, 0, 40, 'United-States'], ['Private', 111697, 'Some-college', 10, 'Never-married', 'Adm-clerical', 'Own-child', 0, 1719, 28, 'United-States'], ['Private', 114937, 'Assoc-acdm', 12, 'Married-civ-spouse', 'Adm-clerical', 'Husband', 0, 0, 40, 'United-States'], ['?', 129305, 'HS-grad', 9, 'Married-civ-spouse', '?', 'Husband', 0, 0, 40, 'United-States'], ['Private', 365739, 'Some-college', 10, 'Divorced', 'Craft-repair', 'Not-in-family', 0, 0, 40, 'United-States'], ['Private', 69621, 'Assoc-acdm', 12, 'Never-married', 'Sales', 'Not-in-family', 0, 0, 60, 'United-States'], ['Private', 43323, 'HS-grad', 9, 'Never-married', 'Other-service', 'Not-in-family', 0, 1762, 40, 'United-States'], ['Self-emp-not-inc', 120985, 'HS-grad', 9, 'Married-civ-spouse', 'Craft-repair', 'Husband', 4386, 0, 35, 'United-States'], ['Private', 254202, 'Bachelors', 13, 'Married-civ-spouse', 'Sales', 'Husband', 0, 0, 50, 'United-States'], ['Private', 146195, 'Assoc-acdm', 12, 'Divorced', 'Tech-support', 'Not-in-family', 0, 0, 36, 'United-States'], ['Federal-gov', 125933, 'Masters', 14, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 0, 40, 'Iran'], ['Self-emp-not-inc', 56920, 'HS-grad', 9, 'Married-civ-spouse', 'Craft-repair', 'Husband', 0, 0, 60, 'United-States'], ['Private', 163127, 'Assoc-voc', 11, 'Married-civ-spouse', 'Adm-clerical', 'Wife', 0, 0, 35, 'United-States'], ['Private', 34310, 'Some-college', 10, 'Never-married', 'Sales', 'Own-child', 0, 0, 20, 'United-States'], ['Private', 81973, 'Some-college', 10, 'Married-civ-spouse', 'Craft-repair', 'Husband', 0, 0, 40, 'United-States'], ['Self-emp-inc', 66614, 'HS-grad', 9, 'Married-civ-spouse', 'Craft-repair', 'Husband', 0, 0, 40, 'United-States'], ['Private', 232782, 'Some-college', 10, 'Never-married', 'Sales', 'Own-child', 0, 0, 40, 'United-States'], ['Private', 316868, 'Some-college', 10, 'Never-married', 'Other-service', 'Own-child', 0, 0, 30, 'Mexico'], ['Private', 196584, 'Assoc-voc', 11, 'Never-married', 'Prof-specialty', 'Not-in-family', 0, 1564, 40, 'United-States'], ['Private', 105376, 'Some-college', 10, 'Never-married', 'Tech-support', 'Other-relative', 0, 0, 40, 'United-States'], ['Private', 185814, 'HS-grad', 9, 'Never-married', 'Transport-moving', 'Unmarried', 0, 0, 30, 'United-States'], ['Private', 175374, 'Some-college', 10, 'Married-civ-spouse', 'Other-service', 'Husband', 0, 0, 24, 'United-States'], ['Private', 108293, 'HS-grad', 9, 'Widowed', 'Other-service', 'Unmarried', 0, 0, 24, 'United-States'], ['Private', 181232, '11th', 7, 'Married-civ-spouse', 'Craft-repair', 'Husband', 0, 2179, 40, 'United-States'], ['?', 174662, 'Some-college', 10, 'Divorced', '?', 'Not-in-family', 0, 0, 40, 'United-States'], ['Local-gov', 186009, 'Some-college', 10, 'Divorced', 'Adm-clerical', 'Unmarried', 0, 0, 38, 'Mexico'], ['Private', 198183, 'HS-grad', 9, 'Never-married', 'Adm-clerical', 'Not-in-family', 0, 0, 40, 'United-States'], ['Private', 163003, 'Bachelors', 13, 'Never-married', 'Exec-managerial', 'Other-relative', 0, 0, 40, 'Philippines'], ['Private', 296158, 'HS-grad', 9, 'Never-married', 'Craft-repair', 'Own-child', 0, 0, 35, 'United-States'], ['?', 252903, 'HS-grad', 9, 'Divorced', '?', 'Not-in-family', 0, 0, 45, 'United-States'], ['Private', 187715, 'HS-grad', 9, 'Married-civ-spouse', 'Craft-repair', 'Husband', 0, 0, 46, 'United-States'], ['Private', 214542, 'Bachelors', 13, 'Never-married', 'Handlers-cleaners', 'Not-in-family', 0, 0, 40, 'United-States'], ['Self-emp-not-inc', 494223, 'Some-college', 10, 'Separated', 'Sales', 'Unmarried', 0, 1816, 2, 'United-States'], ['Private', 191535, 'HS-grad', 9, 'Divorced', 'Craft-repair', 'Not-in-family', 0, 0, 60, 'United-States'], ['Private', 228456, 'Bachelors', 13, 'Separated', 'Other-service', 'Other-relative', 0, 0, 50, 'United-States'], ['?', 38317, '1st-4th', 2, 'Divorced', '?', 'Not-in-family', 0, 0, 20, 'United-States'], ['Private', 252752, 'HS-grad', 9, 'Never-married', 'Other-service', 'Unmarried', 0, 0, 40, 'United-States'], ['Self-emp-inc', 78374, 'Masters', 14, 'Divorced', 'Exec-managerial', 'Unmarried', 0, 0, 40, 'United-States'], ['Private', 88419, 'HS-grad', 9, 'Never-married', 'Exec-managerial', 'Not-in-family', 0, 0, 40, 'England'], ['Self-emp-not-inc', 201080, 'Masters', 14, 'Married-civ-spouse', 'Sales', 'Husband', 0, 0, 40, 'United-States'], ['Private', 207157, 'Some-college', 10, 'Divorced', 'Other-service', 'Unmarried', 0, 0, 40, 'Mexico'], ['Federal-gov', 235485, 'Assoc-acdm', 12, 'Never-married', 'Exec-managerial', 'Not-in-family', 0, 0, 42, 'United-States'], ['State-gov', 102628, 'Masters', 14, 'Widowed', 'Protective-serv', 'Unmarried', 0, 0, 40, 'United-States'], ['Private', 25828, '11th', 7, 'Never-married', 'Handlers-cleaners', 'Own-child', 0, 0, 16, 'United-States'], ['Local-gov', 54826, 'Assoc-voc', 11, 'Widowed', 'Prof-specialty', 'Not-in-family', 0, 0, 20, 'United-States'], ['Private', 124953, 'HS-grad', 9, 'Never-married', 'Other-service', 'Not-in-family', 0, 1980, 40, 'United-States'], ['State-gov', 175325, 'HS-grad', 9, 'Married-civ-spouse', 'Protective-serv', 'Husband', 0, 0, 40, 'United-States'], ['Private', 96062, 'Some-college', 10, 'Married-civ-spouse', 'Sales', 'Husband', 0, 1977, 40, 'United-States'], ['Private', 428030, 'Bachelors', 13, 'Never-married', 'Craft-repair', 'Not-in-family', 0, 0, 50, 'United-States'], ['State-gov', 149624, 'Bachelors', 13, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 0, 40, 'United-States'], ['Private', 253814, 'HS-grad', 9, 'Married-spouse-absent', 'Sales', 'Unmarried', 0, 0, 25, 'United-States'], ['Private', 312956, 'HS-grad', 9, 'Never-married', 'Craft-repair', 'Own-child', 0, 0, 40, 'United-States'], ['Private', 483777, 'HS-grad', 9, 'Never-married', 'Handlers-cleaners', 'Not-in-family', 0, 0, 40, 'United-States'], ['Private', 183930, 'HS-grad', 9, 'Never-married', 'Other-service', 'Own-child', 0, 0, 12, 'United-States'], ['Private', 37274, 'Bachelors', 13, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 0, 65, 'United-States'], ['Local-gov', 181344, 'Some-college', 10, 'Married-civ-spouse', 'Exec-managerial', 'Husband', 0, 0, 38, 'United-States'], ['Private', 114580, 'Some-college', 10, 'Divorced', 'Adm-clerical', 'Not-in-family', 0, 0, 40, 'United-States'], ['Private', 633742, 'Some-college', 10, 'Never-married', 'Craft-repair', 'Not-in-family', 0, 0, 45, 'United-States'], ['Private', 286370, '7th-8th', 4, 'Married-civ-spouse', 'Machine-op-inspct', 'Husband', 0, 0, 40, 'Mexico'], ['Federal-gov', 29054, 'Some-college', 10, 'Married-civ-spouse', 'Adm-clerical', 'Husband', 0, 0, 42, 'United-States'], ['Private', 304030, 'HS-grad', 9, 'Married-civ-spouse', 'Adm-clerical', 'Husband', 0, 0, 40, 'United-States'], ['Self-emp-not-inc', 143129, 'Bachelors', 13, 'Divorced', 'Exec-managerial', 'Not-in-family', 0, 0, 40, 'United-States'], ['?', 135105, 'Bachelors', 13, 'Divorced', '?', 'Not-in-family', 0, 0, 50, 'United-States'], ['Private', 99928, 'Masters', 14, 'Married-civ-spouse', 'Prof-specialty', 'Wife', 0, 0, 50, 'United-States'], ['State-gov', 109567, 'Doctorate', 16, 'Married-civ-spouse', 'Prof-specialty', 'Husband', 0, 0, 1, 'United-States'], ['Private', 155222, 'Some-college', 10, 'Divorced', 'Machine-op-inspct', 'Not-in-family', 0, 0, 28, 'United-States'], ['Private', 159567, 'Some-college', 10, 'Married-civ-spouse', 'Machine-op-inspct', 'Husband', 0, 0, 40, 'United-States'], ['Local-gov', 523910, 'Bachelors', 13, 'Married-civ-spouse', 'Craft-repair', 'Husband', 0, 0, 40, 'United-States'], ['Private', 120939, 'Some-college', 10, 'Married-civ-spouse', 'Tech-support', 'Husband', 0, 0, 45, 'United-States'], ['Federal-gov', 130760, 'Bachelors', 13, 'Married-civ-spouse', 'Tech-support', 'Husband', 0, 0, 24, 'United-States'], ['Private', 197387, '5th-6th', 3, 'Married-civ-spouse', 'Transport-moving', 'Other-relative', 0, 0, 40, 'Mexico'], ['Private', 99374, 'Some-college', 10, 'Divorced', 'Craft-repair', 'Not-in-family', 0, 0, 40, 'United-States'], ['Federal-gov', 56795, 'Masters', 14, 'Never-married', 'Exec-managerial', 'Not-in-family', 14084, 0, 55, 'United-States'], ['Private', 138992, 'Masters', 14, 'Married-civ-spouse', 'Prof-specialty', 'Other-relative', 7298, 0, 40, 'United-States'], ['Self-emp-not-inc', 32921, 'HS-grad', 9, 'Never-married', 'Sales', 'Not-in-family', 0, 0, 40, 'United-States']], 'meta': {'fields': ['race', 'age', 'sex'], 'values': [['White', 39, 'Male'], ['White', 50, 'Male'], ['White', 38, 'Male'], ['Black', 53, 'Male'], ['Black', 28, 'Female'], ['White', 37, 'Female'], ['Black', 49, 'Female'], ['White', 52, 'Male'], ['White', 31, 'Female'], ['White', 42, 'Male'], ['Black', 37, 'Male'], ['Asian-Pac-Islander', 30, 'Male'], ['White', 23, 'Female'], ['Black', 32, 'Male'], ['Asian-Pac-Islander', 40, 'Male'], ['Amer-Indian-Eskimo', 34, 'Male'], ['White', 25, 'Male'], ['White', 32, 'Male'], ['White', 38, 'Male'], ['White', 43, 'Female'], ['White', 40, 'Male'], ['Black', 54, 'Female'], ['Black', 35, 'Male'], ['White', 43, 'Male'], ['White', 59, 'Female'], ['White', 56, 'Male'], ['White', 19, 'Male'], ['Asian-Pac-Islander', 54, 'Male'], ['White', 39, 'Male'], ['White', 49, 'Male'], ['White', 23, 'Male'], ['Black', 20, 'Male'], ['White', 45, 'Male'], ['White', 30, 'Male'], ['Black', 22, 'Male'], ['White', 48, 'Male'], ['White', 21, 'Male'], ['White', 19, 'Female'], ['White', 31, 'Male'], ['White', 48, 'Male'], ['White', 31, 'Male'], ['White', 53, 'Male'], ['White', 24, 'Male'], ['White', 49, 'Female'], ['White', 25, 'Male'], ['Black', 57, 'Male'], ['White', 53, 'Male'], ['White', 44, 'Female'], ['White', 41, 'Male'], ['White', 29, 'Male'], ['Other', 25, 'Female'], ['White', 18, 'Female'], ['White', 47, 'Female'], ['White', 50, 'Male'], ['White', 47, 'Male'], ['White', 43, 'Male'], ['White', 46, 'Male'], ['White', 35, 'Male'], ['White', 41, 'Male'], ['White', 30, 'Male'], ['White', 30, 'Male'], ['White', 32, 'Male'], ['White', 48, 'Male'], ['White', 42, 'Male'], ['White', 29, 'Male'], ['White', 36, 'Male'], ['White', 28, 'Female'], ['White', 53, 'Female'], ['White', 49, 'Male'], ['White', 25, 'Male'], ['White', 19, 'Male'], ['Black', 31, 'Female'], ['White', 29, 'Male'], ['White', 23, 'Male'], ['White', 79, 'Male'], ['White', 27, 'Male'], ['White', 40, 'Male'], ['White', 67, 'Male'], ['White', 18, 'Female'], ['White', 31, 'Male'], ['White', 18, 'Male'], ['White', 52, 'Male'], ['White', 46, 'Female'], ['White', 59, 'Male'], ['White', 44, 'Female'], ['White', 53, 'Female'], ['White', 49, 'Male'], ['White', 33, 'Male'], ['White', 30, 'Male'], ['White', 43, 'Female'], ['White', 57, 'Male'], ['White', 37, 'Female'], ['Black', 28, 'Female'], ['Asian-Pac-Islander', 30, 'Female'], ['White', 34, 'Male'], ['White', 29, 'Male'], ['White', 48, 'Male'], ['White', 37, 'Male'], ['White', 48, 'Female'], ['Black', 32, 'Male'], ['White', 76, 'Male'], ['White', 44, 'Male'], ['White', 47, 'Female'], ['White', 20, 'Female'], ['White', 29, 'Male'], ['White', 32, 'Male'], ['White', 17, 'Female'], ['White', 30, 'Male'], ['White', 31, 'Female'], ['White', 42, 'Male'], ['White', 24, 'Male'], ['White', 38, 'Male'], ['White', 56, 'Male'], ['White', 28, 'Female'], ['White', 36, 'Female'], ['White', 53, 'Male'], ['White', 56, 'Male'], ['Black', 49, 'Male'], ['White', 55, 'Male'], ['White', 22, 'Male'], ['White', 21, 'Female'], ['White', 40, 'Male'], ['Black', 30, 'Male'], ['White', 29, 'Male'], ['Black', 19, 'Male'], ['White', 47, 'Female'], ['White', 20, 'Female'], ['White', 31, 'Male'], ['White', 35, 'Male'], ['White', 39, 'Male'], ['White', 28, 'Female'], ['White', 24, 'Female'], ['White', 38, 'Male'], ['White', 37, 'Male'], ['Black', 46, 'Female'], ['White', 38, 'Male'], ['White', 43, 'Male'], ['White', 27, 'Female'], ['White', 20, 'Male'], ['Asian-Pac-Islander', 49, 'Male'], ['White', 61, 'Male'], ['White', 27, 'Female'], ['White', 19, 'Male'], ['White', 45, 'Female'], ['White', 70, 'Male'], ['Black', 31, 'Female'], ['White', 22, 'Male'], ['White', 36, 'Female'], ['White', 64, 'Male'], ['White', 43, 'Female'], ['White', 47, 'Female'], ['White', 34, 'Female'], ['Asian-Pac-Islander', 33, 'Female'], ['White', 21, 'Male'], ['White', 52, 'Male'], ['White', 48, 'Male'], ['White', 23, 'Male'], ['Black', 71, 'Male'], ['White', 29, 'Male'], ['Black', 42, 'Male'], ['White', 68, 'Female'], ['White', 25, 'Female'], ['Asian-Pac-Islander', 44, 'Female'], ['Asian-Pac-Islander', 28, 'Female'], ['White', 45, 'Male'], ['White', 36, 'Female'], ['White', 39, 'Male'], ['White', 46, 'Male'], ['White', 18, 'Male'], ['White', 66, 'Female'], ['White', 27, 'Male'], ['White', 28, 'Male'], ['White', 51, 'Male'], ['White', 27, 'Male'], ['White', 28, 'Male'], ['White', 27, 'Female'], ['Black', 21, 'Male'], ['Black', 34, 'Male'], ['White', 18, 'Male'], ['White', 33, 'Male'], ['Black', 44, 'Male'], ['White', 43, 'Female'], ['Black', 30, 'Male'], ['White', 40, 'Male'], ['White', 37, 'Male'], ['Black', 34, 'Male'], ['White', 41, 'Female'], ['White', 53, 'Female'], ['White', 31, 'Female'], ['White', 58, 'Male'], ['Black', 38, 'Female'], ['White', 24, 'Male'], ['Black', 41, 'Male'], ['White', 47, 'Male'], ['White', 41, 'Male'], ['White', 23, 'Male'], ['White', 36, 'Male'], ['White', 40, 'Female'], ['White', 35, 'Male'], ['White', 24, 'Male']]}}\n",
      "\n",
      ">>>>>>>>>>>>>>>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "DEBIASING_PREDICTIONS_URL = WOS_CREDENTIALS['url'] + \"/openscale/{0}/v2/subscriptions/{1}/predictions\".format(data_mart_id,subscription_id)\n",
    "print(DEBIASING_PREDICTIONS_URL)\n",
    "\n",
    "headers = {}\n",
    "headers[\"Content-Type\"] = \"application/json\"\n",
    "headers[\"Accept\"] = \"application/json\"\n",
    "headers[\"Authorization\"] = \"Bearer {}\".format(access_token)\n",
    "\n",
    "debiased_scoring_payload = payload_scoring['input_data'][0]\n",
    "print('\\n>>>>>>>>>>>>>>>\\n')\n",
    "print(debiased_scoring_payload)\n",
    "print('\\n>>>>>>>>>>>>>>>\\n')\n",
    "\n",
    "response = requests.post(DEBIASING_PREDICTIONS_URL, data=json.dumps(debiased_scoring_payload), headers=headers, verify=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "a401c2184e6048db899ddcf9bfb450f1"
   },
   "source": [
    "## Listing those predictions whose original model prediction is different from the debiased prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "id": "e3f46a6761dc4f5a87d45ab10fd17dc9"
   },
   "outputs": [],
   "source": [
    "predictedLabel_index = response.json()['fields'].index('predictedLabel')\n",
    "debiased_prediction_index = response.json()['fields'].index('debiased_prediction')\n",
    "\n",
    "for j in range(no_of_records_to_score):\n",
    "    scored_record = response.json()['values'][j]\n",
    "    predictedLabel = scored_record[predictedLabel_index]\n",
    "    debiased_prediction = scored_record[debiased_prediction_index]\n",
    "    if predictedLabel != debiased_prediction:\n",
    "        print('==========')\n",
    "        print(scored_record)\n",
    "        print('predictedLabel:' + str(predictedLabel) + ', debiased_prediction:' + str(debiased_prediction))\n",
    "        print('==========')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c16ee2e26b694f2e965bd37f38c5377e"
   },
   "source": [
    "## Additional data to help debugging"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "id": "2fd4853741be46d6b1330fb259ed0fc6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model id: eed8b225-19e5-4460-9b6f-7271dc1e3ff2\n",
      "Deployment id: 038b8c68-6615-4c0d-8d46-15783d6ab9c8\n",
      "OpenScale Datamart id: 00000000-0000-0000-0000-000000000000\n",
      "OpenScale Subscription id: c274e76c-6bb9-43ce-ba64-127a3db95ede\n",
      "OpenScale Fairness Monitor Instance id: 841a81b1-b4cf-4e98-87de-1d374d43df63\n",
      "OpenScale Fairness Monitoring Run id: 6c53aa46-439a-4008-bbfc-165132dff3ff\n"
     ]
    }
   ],
   "source": [
    "print(\"Model id: {}\".format(model_uid))\n",
    "print(\"Deployment id: {}\".format(deployment_uid))\n",
    "print(\"OpenScale Datamart id: {}\".format(data_mart_id))\n",
    "print(\"OpenScale Subscription id: {}\".format(subscription_id))\n",
    "print(\"OpenScale Fairness Monitor Instance id: {}\".format(fairness_monitor_instance_id))\n",
    "print(\"OpenScale Fairness Monitoring Run id: {}\".format(fairness_monitoring_run_id))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "914d7fc6498141b9b1562bc1a43cf0ac"
   },
   "source": [
    "## Conclusion\n",
    "\n",
    "As part of this notebook we did the following tasks\n",
    "\n",
    "- Created and trained an Income classification model. We made sure to remove the sensitive attributes - age, sex and race while training the model.\n",
    "- Identified a Space to be associated with the model and its deployment.\n",
    "- Deployed the model to the space and scored it with additional meta fields.\n",
    "- Configured OpenScale and subscribed the deployment.\n",
    "- Configured fairness on the meta fields (sensitive attributes) age and sex.\n",
    "- Ran fairness monitor\n",
    "- Noticed that Indirect Bias exists against age attribute, as it can be visualised in the OpenScale dashboard.\n",
    "- Did an on-demand evaluation of the fairness monitor as well.\n",
    "- Call the active debias API, otherwise called as OpenScale predictions API, to notice that from the set of scored records indeed there exists some records for which debiased prediction is different from the original prediction.  \n",
    "- The above step proves that OpenScale is successfully able to debiased the model prediction even on the meta/sensitive attributes.\n",
    "\n",
    "That's all for now. Thank You!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}