tutorials/GENBAIT_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6406f396",
   "metadata": {},
   "source": [
    "\n",
    "# GENBAIT Tutorial\n",
    "\n",
    "This notebook provides a detailed walkthrough of the GENBAIT Python package for selecting optimal baits in BioID experiments using Genetic Algorithms (GA) and various clustering metrics. We will guide you through loading BioID data, running a genetic algorithm for bait selection, and evaluating the results using different NMF-derived and non NMF-derived metrics. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c799c897",
   "metadata": {},
   "source": [
    "## Importing required modules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "9315d848",
   "metadata": {},
   "outputs": [],
   "source": [
    "import genbait as gb\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2af1f3f",
   "metadata": {},
   "source": [
    "\n",
    "## Setting Input, Output Directories, and Parameters for NMF and GA\n",
    "\n",
    "In this section, we define the input and output directories for file handling, as well as key parameters for Non-negative Matrix Factorization (NMF) and the Genetic Algorithm (GA) used in feature (bait) selection.\n",
    "\n",
    "- **input_directory**: This variable defines the path where the input files, such as the SAINT file and other data, are stored.\n",
    "- **output_directory**: This variable specifies the location where the output files will be saved after processing, including the results of the genetic algorithm and NMF computations.\n",
    "\n",
    "- **primary_baits**: A list of primary baits that can be used to filter the dataset. If set to `None`, all baits from the dataset will be used. You can uncomment this line to provide specific bait proteins.\n",
    "\n",
    "- **n_components**: Specifies the number of NMF components (latent features). This parameter controls how many underlying factors will be extracted from the BioID dataset. Typically, these components represent key biological factors.\n",
    "\n",
    "- **n_baits_to_select**: Defines the number of baits (features) to be selected by the genetic algorithm. The goal of the GA is to find an optimal subset of baits that retain the biological significance of the dataset.\n",
    "\n",
    "- **subset_range**: This is the range of the number of baits that can be selected by the genetic algorithm. It is calculated based on the `n_baits_to_select` value to ensure that the number of selected baits remains within a valid range (neither too many nor too few).\n",
    "  - Function: `calculate_subset_range`\n",
    "  - Input: `n_baits_to_select` (the desired number of baits to select).\n",
    "  - Output: A tuple representing the valid range of baits that the GA can select.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "85eb1c06",
   "metadata": {},
   "outputs": [],
   "source": [
    "input_directory = 'data/input_files/'  \n",
    "output_directory = 'data/output_files/'  \n",
    "\n",
    "# primary_baits = ['Gene1', 'Gene2', 'Gene3'] \n",
    "\n",
    "n_components = 20  \n",
    "n_baits_to_select = 50  \n",
    "\n",
    "subset_range = gb.calculate_subset_range(n_baits_to_select)  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41f16c44",
   "metadata": {},
   "source": [
    "## Loading and Normalizing the BioID Data\n",
    "\n",
    "In this step, we load and preprocess the BioID data using the `load_bioid_data` function. This function reads the SAINT file and performs preprocessing steps such as calculating average control intensities, correcting for control values, and normalizing the data.\n",
    "\n",
    "- **Function**: `load_bioid_data`\n",
    "- **Parameters**:\n",
    "  - `input_file_path`: The path to the SAINT file (located in the `input_directory`).\n",
    "  - `output_file_directory`: The directory where the processed and normalized data will be saved. The normalized DataFrame will also be stored as a CSV file for further use.\n",
    "  \n",
    "- **Process**:\n",
    "  - **Step 1**: The SAINT file is read and baits (proteins) are processed.\n",
    "  - **Step 2**: Controls are averaged, and prey spectral counts are adjusted by subtracting these control averages.\n",
    "  - **Step 3**: The data is filtered by a False Discovery Rate (FDR) threshold, pivoted to a matrix format (baits as rows, preys as columns), and normalized using MinMax scaling.\n",
    "\n",
    "- **Output**: \n",
    "  - Returns the normalized DataFrame (`df_norm`), which can be used in subsequent analyses such as feature selection and clustering.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e47b4a39",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_norm = gb.load_bioid_data(\n",
    "    input_file_path=f'{input_directory}saint-latest.txt',\n",
    "    output_file_directory=output_directory\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "2242fa58",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>PreyGene</th>\n",
       "      <th>AAAS</th>\n",
       "      <th>AAK1</th>\n",
       "      <th>AAR2</th>\n",
       "      <th>AARS2</th>\n",
       "      <th>AASDH</th>\n",
       "      <th>AASS</th>\n",
       "      <th>AATF</th>\n",
       "      <th>ABCA3</th>\n",
       "      <th>ABCB1</th>\n",
       "      <th>ABCB10</th>\n",
       "      <th>...</th>\n",
       "      <th>ZNF830</th>\n",
       "      <th>ZNF850</th>\n",
       "      <th>ZNF91</th>\n",
       "      <th>ZRANB2</th>\n",
       "      <th>ZSCAN18</th>\n",
       "      <th>ZSCAN21</th>\n",
       "      <th>ZW10</th>\n",
       "      <th>ZWINT</th>\n",
       "      <th>ZYX</th>\n",
       "      <th>ZZZ3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bait</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AARS2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.347015</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.266667</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ACBD5</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ACTB</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.384215</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ACTR1A</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ACTR2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>VCL</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.278706</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>VIM</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.123211</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ZFPL1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ZNF330_target</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.381833</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.210237</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.579048</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ZYX</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>192 rows × 4424 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "PreyGene       AAAS      AAK1  AAR2  AARS2  AASDH      AASS      AATF  ABCA3  \\\n",
       "Bait                                                                           \n",
       "AARS2           0.0  0.000000   0.0    0.0    0.0  0.347015  0.000000    0.0   \n",
       "ACBD5           0.0  0.000000   0.0    0.0    0.0  0.000000  0.000000    0.0   \n",
       "ACTB            0.0  0.000000   0.0    0.0    0.0  0.000000  0.000000    0.0   \n",
       "ACTR1A          0.0  0.000000   0.0    0.0    0.0  0.000000  0.000000    0.0   \n",
       "ACTR2           0.0  0.000000   0.0    0.0    0.0  0.000000  0.000000    0.0   \n",
       "...             ...       ...   ...    ...    ...       ...       ...    ...   \n",
       "VCL             0.0  0.000000   0.0    0.0    0.0  0.000000  0.000000    0.0   \n",
       "VIM             0.0  0.123211   0.0    0.0    0.0  0.000000  0.000000    0.0   \n",
       "ZFPL1           0.0  0.000000   0.0    0.0    0.0  0.000000  0.000000    0.0   \n",
       "ZNF330_target   0.0  0.000000   0.0    0.0    0.0  0.000000  0.381833    0.0   \n",
       "ZYX             0.0  1.000000   0.0    0.0    0.0  0.000000  0.000000    0.0   \n",
       "\n",
       "PreyGene       ABCB1    ABCB10  ...  ZNF830  ZNF850  ZNF91    ZRANB2  ZSCAN18  \\\n",
       "Bait                            ...                                             \n",
       "AARS2            0.0  0.266667  ...     0.0     0.0    0.0  0.000000      0.0   \n",
       "ACBD5            0.0  0.000000  ...     0.0     0.0    0.0  0.000000      0.0   \n",
       "ACTB             0.0  0.000000  ...     0.0     0.0    0.0  0.000000      0.0   \n",
       "ACTR1A           0.0  0.000000  ...     0.0     0.0    0.0  0.000000      0.0   \n",
       "ACTR2            0.0  0.000000  ...     0.0     0.0    0.0  0.000000      0.0   \n",
       "...              ...       ...  ...     ...     ...    ...       ...      ...   \n",
       "VCL              0.0  0.000000  ...     0.0     0.0    0.0  0.000000      0.0   \n",
       "VIM              0.0  0.000000  ...     0.0     0.0    0.0  0.000000      0.0   \n",
       "ZFPL1            0.0  0.000000  ...     0.0     0.0    0.0  0.000000      0.0   \n",
       "ZNF330_target    0.0  0.000000  ...     0.0     0.0    0.0  0.210237      0.0   \n",
       "ZYX              0.0  0.000000  ...     0.0     0.0    0.0  0.000000      0.0   \n",
       "\n",
       "PreyGene        ZSCAN21  ZW10  ZWINT       ZYX  ZZZ3  \n",
       "Bait                                                  \n",
       "AARS2          0.000000   0.0    0.0  0.000000   0.0  \n",
       "ACBD5          0.000000   0.0    0.0  0.000000   0.0  \n",
       "ACTB           0.000000   0.0    0.0  0.384215   0.0  \n",
       "ACTR1A         0.000000   0.0    0.0  0.000000   0.0  \n",
       "ACTR2          0.000000   0.0    0.0  0.000000   0.0  \n",
       "...                 ...   ...    ...       ...   ...  \n",
       "VCL            0.000000   0.0    0.0  0.278706   0.0  \n",
       "VIM            0.000000   0.0    0.0  0.000000   0.0  \n",
       "ZFPL1          0.000000   0.0    0.0  0.000000   0.0  \n",
       "ZNF330_target  0.579048   0.0    0.0  0.000000   0.0  \n",
       "ZYX            0.000000   0.0    0.0  0.000000   0.0  \n",
       "\n",
       "[192 rows x 4424 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_norm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69cd74f5",
   "metadata": {},
   "source": [
    "## Running the Genetic Algorithm\n",
    "\n",
    "In this step, we use the `run_ga` function to run a genetic algorithm (GA) for selecting an optimal subset of features (baits) from the BioID dataset. The GA evolves a population of possible solutions over multiple generations to find the best subset of baits.\n",
    "\n",
    "- **Function**: `run_ga`\n",
    "- **Parameters**:\n",
    "  - `df_norm`: The normalized BioID data (DataFrame).\n",
    "  - `n_components`: The number of NMF components used in the analysis.\n",
    "  - `subset_range`: The range of the number of selected baits (features).\n",
    "  - `population_size`: The number of individuals in the GA population. In this case, it's set to 50.\n",
    "  - `n_generations`: The number of generations for which the GA will run. In this case, it's set to 3.\n",
    "\n",
    "- **Returns**:\n",
    "  - `pop`: The final population of individuals after the GA has evolved over the specified generations.\n",
    "  - `logbook`: A logbook that contains statistical information about the evolution process, including metrics like the average fitness, minimum fitness, and maximum fitness for each generation.\n",
    "  - `hof`: The Hall of Fame, which contains the best individuals (i.e., subsets of baits) discovered during the GA run.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3b5ac92c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gen\tnevals\tavg      \tstd     \tmin    \tmax     \n",
      "0  \t50    \t-0.496768\t0.460882\t-1.5312\t0.646729\n",
      "1  \t33    \t-0.156691\t0.408448\t-1.32808\t0.668128\n",
      "2  \t33    \t0.0120869\t0.35166 \t-1.01133\t0.662135\n",
      "3  \t28    \t0.199545 \t0.326013\t-0.481102\t0.709199\n"
     ]
    }
   ],
   "source": [
    "pop, logbook, hof = gb.run_ga(\n",
    "    df_norm=df_norm,\n",
    "    n_components=n_components,\n",
    "    subset_range=subset_range,\n",
    "    population_size=50,\n",
    "    n_generations=3\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "434049bd",
   "metadata": {},
   "source": [
    "## Saving GA results\n",
    "Here, we save the results of the GA using the `save_ga_results` function, which saves the population, logbook, and Hall of Fame to specified file paths.\n",
    "\n",
    "- **Parameters**:\n",
    "  - `population`: The final population after the GA run.\n",
    "  - `logbook`: The logbook tracking the evolution of the population.\n",
    "  - `hof`: Hall of Fame containing the best individuals found by the GA.\n",
    "  - `pop_file_path`, `logbook_file_path`, `hof_file_path`: Paths to save the files.\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "685a97c3",
   "metadata": {},
   "outputs": [],
   "source": [
    "gb.save_ga_results(\n",
    "    population=pop,\n",
    "    logbook=logbook,\n",
    "    hof=hof,\n",
    "    pop_file_path=f\"{output_directory}/popfile.pkl\",\n",
    "    logbook_file_path=f\"{output_directory}/logbookfile.pkl\",\n",
    "    hof_file_path=f\"{output_directory}/hoffile.pkl\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "feff2e44",
   "metadata": {},
   "source": [
    "## Loading GA results\n",
    "Here we can load previously saved population and logbook from their respective file paths"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "f8f79ab6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# population, logbook, hof = gb.load_ga_results(\n",
    "#     pop_file_path=f\"{output_directory}/popfile.pkl\",\n",
    "#     logbook_file_path=f\"{output_directory}/logbookfile.pkl\",\n",
    "#     hof_file_path=f\"{output_directory}/hoffile.pkl\"\n",
    "# )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "647aea0a",
   "metadata": {},
   "source": [
    "## Calculating GA result\n",
    "\n",
    "In this step, we use the `calculate_ga_results` function to retrieve the best subset of baits selected by the genetic algorithm and calculate the NMF correlation between the original and selected subset. This function returns the selected baits and their corresponding NMF correlation values.\n",
    "\n",
    "- **Function**: `calculate_ga_results`\n",
    "- **Parameters**:\n",
    "  - `hof`: The Hall of Fame from the genetic algorithm, which contains the best individuals (subsets of baits).\n",
    "  - `df_norm`: The normalized BioID data (DataFrame).\n",
    "  - `n_components`: The number of NMF components used in the analysis.\n",
    "  \n",
    "- **Returns**:\n",
    "  - `selected_baits`: The list of baits selected by the genetic algorithm as the optimal subset.\n",
    "  - `mean_nmf_correlation`: The mean correlation between the NMF basis matrix of the original dataset and the subset dataset.\n",
    "  - `all_nmf_correaltions`: A list of correlation values for each NMF component.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "d15e354c",
   "metadata": {},
   "outputs": [],
   "source": [
    "selected_baits, mean_nmf_correlation, all_nmf_correaltions = gb.calculate_ga_results(hof, df_norm, n_components)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "cea6c9a1-cfb2-4c9f-981b-ae806e841d38",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['AKAP1',\n",
       " 'ANK3',\n",
       " 'ASF1A',\n",
       " 'ATP2A1',\n",
       " 'CALR3',\n",
       " 'CBX3',\n",
       " 'CLTA',\n",
       " 'COX8A_target',\n",
       " 'CS',\n",
       " 'CTNNA1',\n",
       " 'CYP2C1_sigseq',\n",
       " 'DHX40',\n",
       " 'EBAG9',\n",
       " 'EPB41L4A_Cterm',\n",
       " 'FLOT1',\n",
       " 'GFAP',\n",
       " 'GORASP1',\n",
       " 'HNRNPA1',\n",
       " 'LNPK_Nterm',\n",
       " 'LRRC59_Nterm',\n",
       " 'MAPRE1',\n",
       " 'MARCKS_target',\n",
       " 'MLLT4',\n",
       " 'NDC80',\n",
       " 'NOP56',\n",
       " 'NPM1',\n",
       " 'NUP155_Nterm',\n",
       " 'PARP1',\n",
       " 'PRPF8',\n",
       " 'RAB5A',\n",
       " 'RAB5C',\n",
       " 'RAB9A',\n",
       " 'RHOB',\n",
       " 'RPL31',\n",
       " 'RPS24',\n",
       " 'SEC61B_Cterm',\n",
       " 'SFXN1',\n",
       " 'STX7',\n",
       " 'SYNE3_Cterm',\n",
       " 'SYNE3_Nterm',\n",
       " 'TERF2IP',\n",
       " 'TRIM36_Nterm',\n",
       " 'TUBA1A',\n",
       " 'TUBA4A',\n",
       " 'TUBB']"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "selected_baits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "67417d74",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7091985502317943"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_nmf_correlation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "b3ae8f28",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.79226554, 0.85351557, 0.15611431, 0.93416164, 0.96406943,\n",
       "       0.64759528, 0.96251264, 0.95233955, 0.77993336, 0.93445555,\n",
       "       0.51533336, 0.03571605, 0.99051509, 0.48101593, 0.40425275,\n",
       "       0.86361245, 0.93234166, 0.81742301, 0.17157149, 0.99522634])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_nmf_correaltions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c652def",
   "metadata": {},
   "source": [
    "## Calculating NMF Cosine similarity\n",
    "\n",
    "In this step, we use the `calculate_nmf_cosine_similarity` function to compute the cosine similarity between the original and selected subset baits. This similarity gives insight into how well the selected baits represent the original data.\n",
    "\n",
    "- **Function**: `calculate_nmf_cosine_similarity`\n",
    "- **Parameters**:\n",
    "  - `df_norm`: The normalized BioID data (DataFrame).\n",
    "  - `selected_baits`: The subset of baits selected by the genetic algorithm.\n",
    "  - `n_components`: The number of NMF components used in the analysis.\n",
    "  \n",
    "- **Returns**:\n",
    "  - `mean_nmf_cos_similarity_score`: The mean cosine similarity score between the NMF components of the original dataset and the selected subset.\n",
    "  - `all_nmf_cos_similarity_scores`: A list of cosine similarity scores for each NMF component, showing the similarity for individual components.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "1f72eca0",
   "metadata": {},
   "outputs": [],
   "source": [
    "mean_nmf_cos_similarity_score, all_nmf_cos_similarity_scores = gb.calculate_nmf_cosine_similarity(df_norm, selected_baits, n_components)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "6eba10b1-9b69-443c-9c04-4950d82879c6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7245421537225744"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_nmf_cos_similarity_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "4b6ebe28-5aaf-4a78-8995-6587f1f226b1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.8125022 , 0.86230352, 0.19511033, 0.93815166, 0.96538689,\n",
       "       0.67531421, 0.96437863, 0.95253445, 0.78516601, 0.93861229,\n",
       "       0.54977818, 0.08476043, 0.99135444, 0.52194027, 0.42729313,\n",
       "       0.86739639, 0.93356794, 0.82899058, 0.20088615, 0.99541539])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_nmf_cos_similarity_scores"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe9b3f3c",
   "metadata": {},
   "source": [
    "## Calculating NMF KL divergence\n",
    "\n",
    "\n",
    "In this step, we use the `calculate_nmf_kl_divergence` function to compute the Kullback-Leibler (KL) divergence between the NMF components of the original data and the selected baits. KL divergence measures how one probability distribution diverges from a second, expected probability distribution. This is useful for understanding the differences between the original data and the subset selected by the genetic algorithm.\n",
    "\n",
    "- **Function**: `calculate_nmf_kl_divergence`\n",
    "- **Parameters**:\n",
    "  - `df_norm`: The normalized BioID data (DataFrame).\n",
    "  - `selected_baits`: The list of baits (features) selected by the GA.\n",
    "  - `n_components`: The number of NMF components used in the analysis.\n",
    "\n",
    "- **Returns**:\n",
    "  - `mean_nmf_kl_divergence_score`: The average KL divergence score across all NMF components.\n",
    "  - `all_nmf_kl_divergence_scores`: The individual KL divergence scores for each component.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "6eaeaf5f",
   "metadata": {},
   "outputs": [],
   "source": [
    "mean_nmf_kl_divergence_score, all_nmf_kl_divergence_scores = gb.calculate_nmf_kl_divergence(df_norm, selected_baits, n_components)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "0264a34e-1937-471f-95f0-d6fe36181948",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.137286525421461"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_nmf_kl_divergence_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "8649aa3e-7fba-496b-b373-9cea7200d1ab",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 3.44579802,  3.98665558, 16.24365059,  1.65621653,  1.39011435,\n",
       "        6.24679521,  2.3820665 ,  3.7649966 ,  7.06126624,  2.62323316,\n",
       "        8.99116679, 18.10645507,  0.36588091,  5.95561483,  9.76928944,\n",
       "        6.23863719,  3.1108764 ,  3.90540797, 16.65477775,  0.84683137])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_nmf_kl_divergence_scores"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61ac6e11",
   "metadata": {},
   "source": [
    "## Calculating NMF GO Jaccard Index\n",
    "\n",
    "In this step, we use the `calculate_nmf_go_jaccard` function to compute the Jaccard Index between the Gene Ontology (GO) Cellular Component (CC) terms associated with the original data and the selected baits. The Jaccard Index is used to measure the similarity between two sets, specifically the overlap between GO terms of the original and selected baits.\n",
    "\n",
    "- **Function**: `calculate_nmf_go_jaccard`\n",
    "- **Parameters**:\n",
    "  - `df_norm`: The normalized BioID data (DataFrame).\n",
    "  - `selected_baits`: The list of baits (features) selected by the genetic algorithm.\n",
    "  - `n_components`: The number of NMF components used in the analysis.\n",
    "\n",
    "- **Returns**:\n",
    "  - `mean_nmf_go_jaccard_index`: The average Jaccard index score across all NMF components.\n",
    "  - `all_nmf_go_jaccard_indices`: The individual Jaccard index scores for each component.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "15343055",
   "metadata": {},
   "outputs": [],
   "source": [
    "mean_nmf_go_jaccard_index, all_nmf_go_jaccard_indices = gb.calculate_nmf_go_jaccard(df_norm, selected_baits, n_components=n_components)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "58bdc360-e623-4628-b572-066ee10539cf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5429466839122716"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_nmf_go_jaccard_index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "9fd9f1bd-8708-4cbf-88dc-9ca79143ca6d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{2: 0.31868131868131866,\n",
       " 6: 0.6785714285714286,\n",
       " 4: 0.9393939393939394,\n",
       " 5: 0.5151515151515151,\n",
       " 9: 0.6947368421052632,\n",
       " 0: 0.43005181347150256,\n",
       " 1: 0.5423728813559322,\n",
       " 7: 0.3942307692307692,\n",
       " 3: 0.8783783783783784,\n",
       " 8: 0.6461538461538462,\n",
       " 11: 0.21495327102803738,\n",
       " 13: 0.3783783783783784,\n",
       " 16: 0.9361702127659575,\n",
       " 12: 0.6363636363636364,\n",
       " 18: 0.1794871794871795,\n",
       " 15: 0.5393258426966292,\n",
       " 17: 0.5080645161290323,\n",
       " 10: 0.6190476190476191,\n",
       " 14: 0.48333333333333334,\n",
       " 19: 0.32608695652173914}"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_nmf_go_jaccard_indices"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b92304b",
   "metadata": {},
   "source": [
    "## Calculating NMF ARI (Adjusted Rand Index)\n",
    "\n",
    "In this step, we use the `calculate_nmf_ari` function to compute the Adjusted Rand Index (ARI) between the clustering of the original data and the selected baits. ARI measures the similarity of the clustering assignments between two different sets, where a higher ARI indicates better alignment between the clusters.\n",
    "\n",
    "- **Function**: `calculate_nmf_ari`\n",
    "- **Parameters**:\n",
    "  - `df_norm`: The normalized BioID data (DataFrame).\n",
    "  - `selected_baits`: The list of baits (features) selected by the genetic algorithm.\n",
    "  - `n_components`: The number of NMF components used in the analysis.\n",
    "\n",
    "- **Returns**:\n",
    "  - `nmf_ari_score`: The ARI score indicating how well the selected baits' clustering matches the clustering of the original data.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "41b5c40d",
   "metadata": {},
   "outputs": [],
   "source": [
    "nmf_ari_score = gb.calculate_nmf_ari(df_norm, selected_baits, n_components)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "db751505-34a9-4aea-9437-ce1d10e65903",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.5116111477977049"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nmf_ari_score"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d804709",
   "metadata": {},
   "source": [
    "## Calculating Remaining Preys Percentage and Count\n",
    "\n",
    "In this step, we use the `calculate_remaining_preys_percentage` function to compute the percentage and count of preys (proteins) that remain in the selected subset of data after applying the genetic algorithm. This helps assess the coverage of preys in the selected baits.\n",
    "\n",
    "- **Function**: `calculate_remaining_preys_percentage`\n",
    "- **Parameters**:\n",
    "  - `df_norm`: The normalized BioID data (DataFrame).\n",
    "  - `selected_baits`: The list of baits (features) selected by the genetic algorithm.\n",
    "  - `n_components`: The number of NMF components used in the analysis.\n",
    "\n",
    "- **Returns**:\n",
    "  - `remaining_preys_percentage`: The percentage of preys that remain in the selected baits.\n",
    "  - `remaining_preys_count`: The total number of preys remaining in the selected subset.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "e815593a",
   "metadata": {},
   "outputs": [],
   "source": [
    "remaining_preys_percentage, remaining_preys_count = gb.calculate_remaining_preys_percentage(df_norm, selected_baits, n_components)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "32ced1d4-cc49-4596-a6ef-ad985fe998c0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7174502712477396"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "remaining_preys_percentage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "8ca06e86-3410-4c7b-abb8-8c3652f5c7a7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3174"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "remaining_preys_count"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "751a39d2",
   "metadata": {},
   "source": [
    "## Calculating GO Retrieval Percentage\n",
    "\n",
    "In this step, we use the `calculate_go_retrieval_percentage` function to calculate the Gene Ontology (GO) retrieval percentage for the selected baits. GO retrieval assesses the functional enrichment of the selected baits by evaluating how many of the preys are associated with known GO terms (specifically Cellular Component, or GO:CC) based on the provided GAF (Gene Annotation Format) file.\n",
    "\n",
    "- **Function**: `calculate_go_retrieval_percentage`\n",
    "- **Parameters**:\n",
    "  - `df_norm`: The normalized BioID data (DataFrame).\n",
    "  - `selected_baits`: The list of baits (features) selected by the genetic algorithm.\n",
    "  - `gaf_path`: The file path to the GAF file, which contains GO annotations for various genes/proteins.\n",
    "\n",
    "- **Returns**:\n",
    "  - `go_retrieval_percentage`: The percentage of preys in the selected baits that are associated with known GO:CC terms.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "7bb0e4f7",
   "metadata": {},
   "outputs": [],
   "source": [
    "go_retrieval_percentage = gb.calculate_go_retrieval_percentage(df_norm, selected_baits, gaf_path=f'{input_directory}goa_human.gaf')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "8453bb69-a227-4539-9b42-1464ad1f8985",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "90.70660522273425"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "go_retrieval_percentage"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fa9524a",
   "metadata": {},
   "source": [
    "## Calculating Leiden ARI (Adjusted Rand Index)\n",
    "\n",
    "In this step, we use the `calculate_leiden_ari` function to calculate the Adjusted Rand Index (ARI) for clustering the selected baits using the Leiden algorithm. The Leiden algorithm is a community detection algorithm commonly used for graph-based clustering. By calculating the ARI, we can compare how well the clustering from the selected baits corresponds to the clustering of the entire dataset across different resolutions. ARI measures the similarity between two clusterings, comparing the clustering of the full dataset with the clustering of the selected subset of baits.\n",
    "\n",
    "- **Function**: `calculate_leiden_ari`\n",
    "- **Parameters**:\n",
    "  - `df_norm`: The normalized BioID data (DataFrame).\n",
    "  - `selected_baits`: The list of baits (features) selected by the genetic algorithm.\n",
    "  - `resolutions`: List of resolutions for the Leiden clustering algorithm. Higher resolutions lead to more clusters.\n",
    "  - `seed`: (Optional) Random seed for reproducibility (default is 4).\n",
    "\n",
    "- **Returns**:\n",
    "  - `leiden_results`: A dictionary where the keys are resolution values, and the values are the ARI scores between the clustering of the original data and the selected baits.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "eb7bf4bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "leiden_results = gb.calculate_leiden_ari(df_norm, selected_baits, resolutions=[0.5, 1.0, 1.5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "59b2813d-4d3d-4696-9a75-fa1aba4a461b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Resolution: 0.5, ARI: 0.5218981053123726\n",
      "Resolution: 1.0, ARI: 0.5272409033404254\n",
      "Resolution: 1.5, ARI: 0.4647442350933481\n"
     ]
    }
   ],
   "source": [
    "for resolution, ari in leiden_results.items():\n",
    "    print(f\"Resolution: {resolution}, ARI: {ari}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9177bf8e",
   "metadata": {},
   "source": [
    "## Calculating GMM ARI (Adjusted Rand Index)\n",
    "\n",
    "In this step, we use the `calculate_gmm_ari` function to calculate the Adjusted Rand Index (ARI) for Gaussian Mixture Model (GMM) clustering of the selected baits. The Gaussian Mixture Model (GMM) is used to cluster the data into a specified number of clusters. The ARI score measures the similarity between the clustering of the full dataset and the selected baits subset.\n",
    "\n",
    "- **Function**: `calculate_gmm_ari`\n",
    "- **Parameters**:\n",
    "  - `df_norm`: The normalized BioID data (DataFrame).\n",
    "  - `selected_baits`: The list of baits (features) selected by the genetic algorithm.\n",
    "  - `cluster_numbers`: List of cluster numbers to fit the Gaussian Mixture Model (GMM). Each number represents how many clusters are formed.\n",
    "  - `seed`: (Optional) Random seed for reproducibility (default is 4).\n",
    "\n",
    "- **Returns**:\n",
    "  - `gmm_results`: A dictionary where the keys are cluster numbers, and the values are the ARI scores comparing the clustering of the full dataset and the selected baits subset for each cluster number.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "be504268",
   "metadata": {},
   "outputs": [],
   "source": [
    "gmm_results = gb.calculate_gmm_ari(df_norm, selected_baits, cluster_numbers=[15, 20, 25, 30])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "0ccb2caa-8b03-48ab-8eae-0438e3ea612b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cluster Number: 15, ARI: 0.36254999357590995\n",
      "Cluster Number: 20, ARI: 0.3834121704190322\n",
      "Cluster Number: 25, ARI: 0.473268923990476\n",
      "Cluster Number: 30, ARI: 0.3604236140421185\n"
     ]
    }
   ],
   "source": [
    "for cluster_number, ari in gmm_results.items():\n",
    "    print(f\"Cluster Number: {cluster_number}, ARI: {ari}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d30fadcd",
   "metadata": {},
   "source": [
    "## Calculating GMM Mean Correlation\n",
    "\n",
    "In this step, we use the `calculate_gmm_mean_correlation` function to compute the mean correlation values between the Gaussian Mixture Model (GMM) clustering of the full dataset and the selected baits subset. Gaussian Mixture Model (GMM) soft clustering is performed on both the full dataset and the selected baits subset for each specified cluster number. The function computes the mean correlation between the probability distributions of the original and subset clusters, indicating how well the selected baits maintain the clustering structure of the full dataset.\n",
    "\n",
    "- **Function**: `calculate_gmm_mean_correlation`\n",
    "- **Parameters**:\n",
    "  - `df_norm`: The normalized BioID data (DataFrame).\n",
    "  - `selected_baits`: The list of baits (features) selected by the genetic algorithm.\n",
    "  - `cluster_numbers`: List of cluster numbers to fit the Gaussian Mixture Model (GMM). Each number represents how many clusters are formed.\n",
    "  - `seed`: (Optional) Random seed for reproducibility (default is 4).\n",
    "\n",
    "- **Returns**:\n",
    "  - `gmm_corr_results`: A dictionary where the keys are cluster numbers and the values are the mean correlation values between the soft clustering of the full dataset and the selected baits subset for each cluster number.\n",
    "\n",
    "\n",
    "  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "d52e86d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "gmm_corr_results = gb.calculate_gmm_mean_correlation(df_norm, selected_baits, cluster_numbers=[15, 20, 25, 30])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "40ed3967-f726-4a08-a31e-7cac61385b16",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cluster Number: 15, Mean Correlation: 0.2727451914291265\n",
      "Cluster Number: 20, Mean Correlation: 0.3158209295933284\n",
      "Cluster Number: 25, Mean Correlation: 0.31291794460313144\n",
      "Cluster Number: 30, Mean Correlation: 0.27618404245265055\n"
     ]
    }
   ],
   "source": [
    "for cluster_number, mean_corr in gmm_corr_results.items():\n",
    "    print(f\"Cluster Number: {cluster_number}, Mean Correlation: {mean_corr}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa31bf23-c9f2-4cf7-b0b9-855d80b7e3be",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}