{ "cells": [ { "cell_type": "markdown", "id": "4fb0d430", "metadata": {}, "source": [ "# Logistic Regression" ] }, { "cell_type": "markdown", "id": "7e251389", "metadata": {}, "source": [ "In this chapter the logistic regression algorithm is applied to the macro- and microstrucutral data. The procedure is analogous to both kind of data. First, the data is prepared in terms of making it suitable to use it in the code efficiently. Second, the logistic regression model is built, defining the input and output variables, scaling the relevant variables and splitting the data set into training and testing samples and subsequently training the model. Followed by evulating the model, we will refer to different metrics that provide information on how the model performs. \n", "\n", "We first start with **cortical thickness (CT)**." ] }, { "cell_type": "markdown", "id": "623b534c", "metadata": {}, "source": [ "## 1. Macro-structural data: Cortical Thickness (CT)" ] }, { "cell_type": "markdown", "id": "150941c9", "metadata": {}, "source": [ "### 1.1 Data preperation" ] }, { "cell_type": "markdown", "id": "ed303c0c", "metadata": {}, "source": [ "In the beginning, all relevant modules needed for the analysis are imported. " ] }, { "cell_type": "code", "execution_count": 1, "id": "4c372d6b", "metadata": {}, "outputs": [], "source": [ "#import relevant modules\n", "\n", "import os\n", "import pandas as pd\n", "import numpy as np\n", "import seaborn as sns\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline\n", "\n", "\n", "from sklearn.model_selection import train_test_split\n", "from sklearn import metrics \n", "from sklearn.metrics import confusion_matrix\n", "from sklearn.linear_model import LogisticRegression\n", "from sklearn.preprocessing import StandardScaler" ] }, { "cell_type": "markdown", "id": "dc556abe", "metadata": {}, "source": [ "Again, to read the data, the os.pardir() function is used to make the code reproducible independent of different operating systems. " ] }, { "cell_type": "code", "execution_count": 2, "id": "c724d3cf", "metadata": {}, "outputs": [], "source": [ "#read data\n", "\n", "CT_Dublin_path = os.path.join(os.pardir, 'data', 'PARC_500.aparc_thickness_Dublin.csv')\n", "CT_Dublin = pd.read_csv(CT_Dublin_path)" ] }, { "cell_type": "code", "execution_count": 3, "id": "dab3630c", "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Subject IDAgeSexGrouplh_bankssts_part1_thicknesslh_bankssts_part2_thicknesslh_caudalanteriorcingulate_part1_thicknesslh_caudalmiddlefrontal_part1_thicknesslh_caudalmiddlefrontal_part2_thicknesslh_caudalmiddlefrontal_part3_thickness...rh_supramarginal_part5_thicknessrh_supramarginal_part6_thicknessrh_supramarginal_part7_thicknessrh_frontalpole_part1_thicknessrh_temporalpole_part1_thicknessrh_transversetemporal_part1_thicknessrh_insula_part1_thicknessrh_insula_part2_thicknessrh_insula_part3_thicknessrh_insula_part4_thickness
0CON922521212.1802.3822.3462.5262.7472.544...2.8172.3252.4303.0043.9792.3293.6202.7763.2823.347
1CON922928212.3941.9732.5342.4392.4852.435...2.6112.4182.3172.7943.8512.0343.5882.6543.1243.214
2CON923129212.5512.5671.9542.4392.4282.190...2.7772.3092.3902.3654.0392.3373.6572.4952.6692.886
3GASP303761122.1871.9232.1602.4102.3812.277...2.2652.3062.1292.2813.5052.2753.1212.3332.6042.731
4GASP304047121.8621.7502.1292.5162.2442.169...2.5822.3142.0472.3893.2722.4453.1712.2162.6592.657
..................................................................
103RPG901931122.2402.1501.9952.2542.1642.008...2.2732.2882.3952.1053.2672.2573.2312.5742.9202.899
104RPG910242222.2692.1242.5312.5022.2502.183...2.3022.1822.1822.3272.8812.1243.1592.4502.7532.791
105RPG911941122.2732.5592.5782.4632.4632.053...2.5342.6042.4492.3703.1112.1903.4802.2942.5712.875
106RPG912151121.9402.4382.2722.2722.6102.099...2.6382.2252.0132.1153.8532.2313.1872.5102.7592.838
107RPG912656122.1082.2692.1452.1922.4431.977...2.0132.2512.0212.4193.6791.9703.1922.5512.8552.985
\n", "

108 rows × 312 columns

\n", "
" ], "text/plain": [ " Subject ID Age Sex Group lh_bankssts_part1_thickness \\\n", "0 CON9225 21 2 1 2.180 \n", "1 CON9229 28 2 1 2.394 \n", "2 CON9231 29 2 1 2.551 \n", "3 GASP3037 61 1 2 2.187 \n", "4 GASP3040 47 1 2 1.862 \n", ".. ... ... ... ... ... \n", "103 RPG9019 31 1 2 2.240 \n", "104 RPG9102 42 2 2 2.269 \n", "105 RPG9119 41 1 2 2.273 \n", "106 RPG9121 51 1 2 1.940 \n", "107 RPG9126 56 1 2 2.108 \n", "\n", " lh_bankssts_part2_thickness lh_caudalanteriorcingulate_part1_thickness \\\n", "0 2.382 2.346 \n", "1 1.973 2.534 \n", "2 2.567 1.954 \n", "3 1.923 2.160 \n", "4 1.750 2.129 \n", ".. ... ... \n", "103 2.150 1.995 \n", "104 2.124 2.531 \n", "105 2.559 2.578 \n", "106 2.438 2.272 \n", "107 2.269 2.145 \n", "\n", " lh_caudalmiddlefrontal_part1_thickness \\\n", "0 2.526 \n", "1 2.439 \n", "2 2.439 \n", "3 2.410 \n", "4 2.516 \n", ".. ... \n", "103 2.254 \n", "104 2.502 \n", "105 2.463 \n", "106 2.272 \n", "107 2.192 \n", "\n", " lh_caudalmiddlefrontal_part2_thickness \\\n", "0 2.747 \n", "1 2.485 \n", "2 2.428 \n", "3 2.381 \n", "4 2.244 \n", ".. ... \n", "103 2.164 \n", "104 2.250 \n", "105 2.463 \n", "106 2.610 \n", "107 2.443 \n", "\n", " lh_caudalmiddlefrontal_part3_thickness ... \\\n", "0 2.544 ... \n", "1 2.435 ... \n", "2 2.190 ... \n", "3 2.277 ... \n", "4 2.169 ... \n", ".. ... ... \n", "103 2.008 ... \n", "104 2.183 ... \n", "105 2.053 ... \n", "106 2.099 ... \n", "107 1.977 ... \n", "\n", " rh_supramarginal_part5_thickness rh_supramarginal_part6_thickness \\\n", "0 2.817 2.325 \n", "1 2.611 2.418 \n", "2 2.777 2.309 \n", "3 2.265 2.306 \n", "4 2.582 2.314 \n", ".. ... ... \n", "103 2.273 2.288 \n", "104 2.302 2.182 \n", "105 2.534 2.604 \n", "106 2.638 2.225 \n", "107 2.013 2.251 \n", "\n", " rh_supramarginal_part7_thickness rh_frontalpole_part1_thickness \\\n", "0 2.430 3.004 \n", "1 2.317 2.794 \n", "2 2.390 2.365 \n", "3 2.129 2.281 \n", "4 2.047 2.389 \n", ".. ... ... \n", "103 2.395 2.105 \n", "104 2.182 2.327 \n", "105 2.449 2.370 \n", "106 2.013 2.115 \n", "107 2.021 2.419 \n", "\n", " rh_temporalpole_part1_thickness rh_transversetemporal_part1_thickness \\\n", "0 3.979 2.329 \n", "1 3.851 2.034 \n", "2 4.039 2.337 \n", "3 3.505 2.275 \n", "4 3.272 2.445 \n", ".. ... ... \n", "103 3.267 2.257 \n", "104 2.881 2.124 \n", "105 3.111 2.190 \n", "106 3.853 2.231 \n", "107 3.679 1.970 \n", "\n", " rh_insula_part1_thickness rh_insula_part2_thickness \\\n", "0 3.620 2.776 \n", "1 3.588 2.654 \n", "2 3.657 2.495 \n", "3 3.121 2.333 \n", "4 3.171 2.216 \n", ".. ... ... \n", "103 3.231 2.574 \n", "104 3.159 2.450 \n", "105 3.480 2.294 \n", "106 3.187 2.510 \n", "107 3.192 2.551 \n", "\n", " rh_insula_part3_thickness rh_insula_part4_thickness \n", "0 3.282 3.347 \n", "1 3.124 3.214 \n", "2 2.669 2.886 \n", "3 2.604 2.731 \n", "4 2.659 2.657 \n", ".. ... ... \n", "103 2.920 2.899 \n", "104 2.753 2.791 \n", "105 2.571 2.875 \n", "106 2.759 2.838 \n", "107 2.855 2.985 \n", "\n", "[108 rows x 312 columns]" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "CT_Dublin" ] }, { "cell_type": "markdown", "id": "fc679ec0", "metadata": {}, "source": [ "The data contains variables such as SubjectID, Age and Sex which are not relevant for the classification. Hence, we adjust the dataframe accordingly." ] }, { "cell_type": "code", "execution_count": 4, "id": "e7eaad40", "metadata": {}, "outputs": [], "source": [ "#adjust dataframe\n", "\n", "CT_Dublin_adj = CT_Dublin.drop(['Subject ID','Age', 'Sex'], axis=1)" ] }, { "cell_type": "code", "execution_count": 5, "id": "1277a861", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Grouplh_bankssts_part1_thicknesslh_bankssts_part2_thicknesslh_caudalanteriorcingulate_part1_thicknesslh_caudalmiddlefrontal_part1_thicknesslh_caudalmiddlefrontal_part2_thicknesslh_caudalmiddlefrontal_part3_thicknesslh_caudalmiddlefrontal_part4_thicknesslh_cuneus_part1_thicknesslh_cuneus_part2_thickness...rh_supramarginal_part5_thicknessrh_supramarginal_part6_thicknessrh_supramarginal_part7_thicknessrh_frontalpole_part1_thicknessrh_temporalpole_part1_thicknessrh_transversetemporal_part1_thicknessrh_insula_part1_thicknessrh_insula_part2_thicknessrh_insula_part3_thicknessrh_insula_part4_thickness
012.1802.3822.3462.5262.7472.5442.5821.8162.228...2.8172.3252.4303.0043.9792.3293.6202.7763.2823.347
112.3941.9732.5342.4392.4852.4352.4581.7231.821...2.6112.4182.3172.7943.8512.0343.5882.6543.1243.214
212.5512.5671.9542.4392.4282.1902.3772.0261.800...2.7772.3092.3902.3654.0392.3373.6572.4952.6692.886
322.1871.9232.1602.4102.3812.2772.3611.5851.750...2.2652.3062.1292.2813.5052.2753.1212.3332.6042.731
421.8621.7502.1292.5162.2442.1692.2201.6461.717...2.5822.3142.0472.3893.2722.4453.1712.2162.6592.657
..................................................................
10322.2402.1501.9952.2542.1642.0082.2981.9181.717...2.2732.2882.3952.1053.2672.2573.2312.5742.9202.899
10422.2692.1242.5312.5022.2502.1832.4081.5391.611...2.3022.1822.1822.3272.8812.1243.1592.4502.7532.791
10522.2732.5592.5782.4632.4632.0532.5261.7331.859...2.5342.6042.4492.3703.1112.1903.4802.2942.5712.875
10621.9402.4382.2722.2722.6102.0992.5381.9311.792...2.6382.2252.0132.1153.8532.2313.1872.5102.7592.838
10722.1082.2692.1452.1922.4431.9772.4531.5901.715...2.0132.2512.0212.4193.6791.9703.1922.5512.8552.985
\n", "

108 rows × 309 columns

\n", "
" ], "text/plain": [ " Group lh_bankssts_part1_thickness lh_bankssts_part2_thickness \\\n", "0 1 2.180 2.382 \n", "1 1 2.394 1.973 \n", "2 1 2.551 2.567 \n", "3 2 2.187 1.923 \n", "4 2 1.862 1.750 \n", ".. ... ... ... \n", "103 2 2.240 2.150 \n", "104 2 2.269 2.124 \n", "105 2 2.273 2.559 \n", "106 2 1.940 2.438 \n", "107 2 2.108 2.269 \n", "\n", " lh_caudalanteriorcingulate_part1_thickness \\\n", "0 2.346 \n", "1 2.534 \n", "2 1.954 \n", "3 2.160 \n", "4 2.129 \n", ".. ... \n", "103 1.995 \n", "104 2.531 \n", "105 2.578 \n", "106 2.272 \n", "107 2.145 \n", "\n", " lh_caudalmiddlefrontal_part1_thickness \\\n", "0 2.526 \n", "1 2.439 \n", "2 2.439 \n", "3 2.410 \n", "4 2.516 \n", ".. ... \n", "103 2.254 \n", "104 2.502 \n", "105 2.463 \n", "106 2.272 \n", "107 2.192 \n", "\n", " lh_caudalmiddlefrontal_part2_thickness \\\n", "0 2.747 \n", "1 2.485 \n", "2 2.428 \n", "3 2.381 \n", "4 2.244 \n", ".. ... \n", "103 2.164 \n", "104 2.250 \n", "105 2.463 \n", "106 2.610 \n", "107 2.443 \n", "\n", " lh_caudalmiddlefrontal_part3_thickness \\\n", "0 2.544 \n", "1 2.435 \n", "2 2.190 \n", "3 2.277 \n", "4 2.169 \n", ".. ... \n", "103 2.008 \n", "104 2.183 \n", "105 2.053 \n", "106 2.099 \n", "107 1.977 \n", "\n", " lh_caudalmiddlefrontal_part4_thickness lh_cuneus_part1_thickness \\\n", "0 2.582 1.816 \n", "1 2.458 1.723 \n", "2 2.377 2.026 \n", "3 2.361 1.585 \n", "4 2.220 1.646 \n", ".. ... ... \n", "103 2.298 1.918 \n", "104 2.408 1.539 \n", "105 2.526 1.733 \n", "106 2.538 1.931 \n", "107 2.453 1.590 \n", "\n", " lh_cuneus_part2_thickness ... rh_supramarginal_part5_thickness \\\n", "0 2.228 ... 2.817 \n", "1 1.821 ... 2.611 \n", "2 1.800 ... 2.777 \n", "3 1.750 ... 2.265 \n", "4 1.717 ... 2.582 \n", ".. ... ... ... \n", "103 1.717 ... 2.273 \n", "104 1.611 ... 2.302 \n", "105 1.859 ... 2.534 \n", "106 1.792 ... 2.638 \n", "107 1.715 ... 2.013 \n", "\n", " rh_supramarginal_part6_thickness rh_supramarginal_part7_thickness \\\n", "0 2.325 2.430 \n", "1 2.418 2.317 \n", "2 2.309 2.390 \n", "3 2.306 2.129 \n", "4 2.314 2.047 \n", ".. ... ... \n", "103 2.288 2.395 \n", "104 2.182 2.182 \n", "105 2.604 2.449 \n", "106 2.225 2.013 \n", "107 2.251 2.021 \n", "\n", " rh_frontalpole_part1_thickness rh_temporalpole_part1_thickness \\\n", "0 3.004 3.979 \n", "1 2.794 3.851 \n", "2 2.365 4.039 \n", "3 2.281 3.505 \n", "4 2.389 3.272 \n", ".. ... ... \n", "103 2.105 3.267 \n", "104 2.327 2.881 \n", "105 2.370 3.111 \n", "106 2.115 3.853 \n", "107 2.419 3.679 \n", "\n", " rh_transversetemporal_part1_thickness rh_insula_part1_thickness \\\n", "0 2.329 3.620 \n", "1 2.034 3.588 \n", "2 2.337 3.657 \n", "3 2.275 3.121 \n", "4 2.445 3.171 \n", ".. ... ... \n", "103 2.257 3.231 \n", "104 2.124 3.159 \n", "105 2.190 3.480 \n", "106 2.231 3.187 \n", "107 1.970 3.192 \n", "\n", " rh_insula_part2_thickness rh_insula_part3_thickness \\\n", "0 2.776 3.282 \n", "1 2.654 3.124 \n", "2 2.495 2.669 \n", "3 2.333 2.604 \n", "4 2.216 2.659 \n", ".. ... ... \n", "103 2.574 2.920 \n", "104 2.450 2.753 \n", "105 2.294 2.571 \n", "106 2.510 2.759 \n", "107 2.551 2.855 \n", "\n", " rh_insula_part4_thickness \n", "0 3.347 \n", "1 3.214 \n", "2 2.886 \n", "3 2.731 \n", "4 2.657 \n", ".. ... \n", "103 2.899 \n", "104 2.791 \n", "105 2.875 \n", "106 2.838 \n", "107 2.985 \n", "\n", "[108 rows x 309 columns]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "CT_Dublin_adj" ] }, { "cell_type": "markdown", "id": "074688be", "metadata": {}, "source": [ "As the dataframe shows, the Group variable contains information of whether the samples belong to control or patient. In this case, 1 indicates control and 2 patient. In order to perform a **Logistic Regression**, the labels of the outputs require to be 0 and 1 since the probability of an instance belonging to a default class is computed." ] }, { "cell_type": "code", "execution_count": 6, "id": "aca4d140", "metadata": {}, "outputs": [], "source": [ "#label group 1 as 0 and 2 as 1\n", "\n", "CT_Dublin_adj['Group'] = CT_Dublin_adj['Group'].replace([1,2],[0, 1])" ] }, { "cell_type": "code", "execution_count": 7, "id": "408610ce", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Grouplh_bankssts_part1_thicknesslh_bankssts_part2_thicknesslh_caudalanteriorcingulate_part1_thicknesslh_caudalmiddlefrontal_part1_thicknesslh_caudalmiddlefrontal_part2_thicknesslh_caudalmiddlefrontal_part3_thicknesslh_caudalmiddlefrontal_part4_thicknesslh_cuneus_part1_thicknesslh_cuneus_part2_thickness...rh_supramarginal_part5_thicknessrh_supramarginal_part6_thicknessrh_supramarginal_part7_thicknessrh_frontalpole_part1_thicknessrh_temporalpole_part1_thicknessrh_transversetemporal_part1_thicknessrh_insula_part1_thicknessrh_insula_part2_thicknessrh_insula_part3_thicknessrh_insula_part4_thickness
002.1802.3822.3462.5262.7472.5442.5821.8162.228...2.8172.3252.4303.0043.9792.3293.6202.7763.2823.347
102.3941.9732.5342.4392.4852.4352.4581.7231.821...2.6112.4182.3172.7943.8512.0343.5882.6543.1243.214
202.5512.5671.9542.4392.4282.1902.3772.0261.800...2.7772.3092.3902.3654.0392.3373.6572.4952.6692.886
312.1871.9232.1602.4102.3812.2772.3611.5851.750...2.2652.3062.1292.2813.5052.2753.1212.3332.6042.731
411.8621.7502.1292.5162.2442.1692.2201.6461.717...2.5822.3142.0472.3893.2722.4453.1712.2162.6592.657
..................................................................
10312.2402.1501.9952.2542.1642.0082.2981.9181.717...2.2732.2882.3952.1053.2672.2573.2312.5742.9202.899
10412.2692.1242.5312.5022.2502.1832.4081.5391.611...2.3022.1822.1822.3272.8812.1243.1592.4502.7532.791
10512.2732.5592.5782.4632.4632.0532.5261.7331.859...2.5342.6042.4492.3703.1112.1903.4802.2942.5712.875
10611.9402.4382.2722.2722.6102.0992.5381.9311.792...2.6382.2252.0132.1153.8532.2313.1872.5102.7592.838
10712.1082.2692.1452.1922.4431.9772.4531.5901.715...2.0132.2512.0212.4193.6791.9703.1922.5512.8552.985
\n", "

108 rows × 309 columns

\n", "
" ], "text/plain": [ " Group lh_bankssts_part1_thickness lh_bankssts_part2_thickness \\\n", "0 0 2.180 2.382 \n", "1 0 2.394 1.973 \n", "2 0 2.551 2.567 \n", "3 1 2.187 1.923 \n", "4 1 1.862 1.750 \n", ".. ... ... ... \n", "103 1 2.240 2.150 \n", "104 1 2.269 2.124 \n", "105 1 2.273 2.559 \n", "106 1 1.940 2.438 \n", "107 1 2.108 2.269 \n", "\n", " lh_caudalanteriorcingulate_part1_thickness \\\n", "0 2.346 \n", "1 2.534 \n", "2 1.954 \n", "3 2.160 \n", "4 2.129 \n", ".. ... \n", "103 1.995 \n", "104 2.531 \n", "105 2.578 \n", "106 2.272 \n", "107 2.145 \n", "\n", " lh_caudalmiddlefrontal_part1_thickness \\\n", "0 2.526 \n", "1 2.439 \n", "2 2.439 \n", "3 2.410 \n", "4 2.516 \n", ".. ... \n", "103 2.254 \n", "104 2.502 \n", "105 2.463 \n", "106 2.272 \n", "107 2.192 \n", "\n", " lh_caudalmiddlefrontal_part2_thickness \\\n", "0 2.747 \n", "1 2.485 \n", "2 2.428 \n", "3 2.381 \n", "4 2.244 \n", ".. ... \n", "103 2.164 \n", "104 2.250 \n", "105 2.463 \n", "106 2.610 \n", "107 2.443 \n", "\n", " lh_caudalmiddlefrontal_part3_thickness \\\n", "0 2.544 \n", "1 2.435 \n", "2 2.190 \n", "3 2.277 \n", "4 2.169 \n", ".. ... \n", "103 2.008 \n", "104 2.183 \n", "105 2.053 \n", "106 2.099 \n", "107 1.977 \n", "\n", " lh_caudalmiddlefrontal_part4_thickness lh_cuneus_part1_thickness \\\n", "0 2.582 1.816 \n", "1 2.458 1.723 \n", "2 2.377 2.026 \n", "3 2.361 1.585 \n", "4 2.220 1.646 \n", ".. ... ... \n", "103 2.298 1.918 \n", "104 2.408 1.539 \n", "105 2.526 1.733 \n", "106 2.538 1.931 \n", "107 2.453 1.590 \n", "\n", " lh_cuneus_part2_thickness ... rh_supramarginal_part5_thickness \\\n", "0 2.228 ... 2.817 \n", "1 1.821 ... 2.611 \n", "2 1.800 ... 2.777 \n", "3 1.750 ... 2.265 \n", "4 1.717 ... 2.582 \n", ".. ... ... ... \n", "103 1.717 ... 2.273 \n", "104 1.611 ... 2.302 \n", "105 1.859 ... 2.534 \n", "106 1.792 ... 2.638 \n", "107 1.715 ... 2.013 \n", "\n", " rh_supramarginal_part6_thickness rh_supramarginal_part7_thickness \\\n", "0 2.325 2.430 \n", "1 2.418 2.317 \n", "2 2.309 2.390 \n", "3 2.306 2.129 \n", "4 2.314 2.047 \n", ".. ... ... \n", "103 2.288 2.395 \n", "104 2.182 2.182 \n", "105 2.604 2.449 \n", "106 2.225 2.013 \n", "107 2.251 2.021 \n", "\n", " rh_frontalpole_part1_thickness rh_temporalpole_part1_thickness \\\n", "0 3.004 3.979 \n", "1 2.794 3.851 \n", "2 2.365 4.039 \n", "3 2.281 3.505 \n", "4 2.389 3.272 \n", ".. ... ... \n", "103 2.105 3.267 \n", "104 2.327 2.881 \n", "105 2.370 3.111 \n", "106 2.115 3.853 \n", "107 2.419 3.679 \n", "\n", " rh_transversetemporal_part1_thickness rh_insula_part1_thickness \\\n", "0 2.329 3.620 \n", "1 2.034 3.588 \n", "2 2.337 3.657 \n", "3 2.275 3.121 \n", "4 2.445 3.171 \n", ".. ... ... \n", "103 2.257 3.231 \n", "104 2.124 3.159 \n", "105 2.190 3.480 \n", "106 2.231 3.187 \n", "107 1.970 3.192 \n", "\n", " rh_insula_part2_thickness rh_insula_part3_thickness \\\n", "0 2.776 3.282 \n", "1 2.654 3.124 \n", "2 2.495 2.669 \n", "3 2.333 2.604 \n", "4 2.216 2.659 \n", ".. ... ... \n", "103 2.574 2.920 \n", "104 2.450 2.753 \n", "105 2.294 2.571 \n", "106 2.510 2.759 \n", "107 2.551 2.855 \n", "\n", " rh_insula_part4_thickness \n", "0 3.347 \n", "1 3.214 \n", "2 2.886 \n", "3 2.731 \n", "4 2.657 \n", ".. ... \n", "103 2.899 \n", "104 2.791 \n", "105 2.875 \n", "106 2.838 \n", "107 2.985 \n", "\n", "[108 rows x 309 columns]" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "CT_Dublin_adj" ] }, { "cell_type": "code", "execution_count": 8, "id": "402f2a08", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(108, 309)" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#get shape of df_adj\n", "\n", "CT_Dublin_adj.shape" ] }, { "cell_type": "markdown", "id": "0c3996d6", "metadata": {}, "source": [ "Because the LogisticRegression function from sklearn requires the inputs to be numpy arrays, in the following step the dataframe is converted to a numpy array." ] }, { "cell_type": "code", "execution_count": 9, "id": "8c20c482", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0. , 2.18 , 2.382, ..., 2.776, 3.282, 3.347],\n", " [0. , 2.394, 1.973, ..., 2.654, 3.124, 3.214],\n", " [0. , 2.551, 2.567, ..., 2.495, 2.669, 2.886],\n", " ...,\n", " [1. , 2.273, 2.559, ..., 2.294, 2.571, 2.875],\n", " [1. , 1.94 , 2.438, ..., 2.51 , 2.759, 2.838],\n", " [1. , 2.108, 2.269, ..., 2.551, 2.855, 2.985]])" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#dataframe as numpy array \n", "\n", "CT_Dublin_adj.to_numpy()" ] }, { "cell_type": "markdown", "id": "5998c14a", "metadata": {}, "source": [ "### 1.2 Building the model" ] }, { "cell_type": "markdown", "id": "a3e7df3f", "metadata": {}, "source": [ "In the next steps, the **logistic regression** model is built. Firstly, the input and output should be defined. Our input contains the **CT** for all of the 308 brain regions, meaning that there are n=308 features in total. The output is within the Group variable containing label information." ] }, { "cell_type": "code", "execution_count": 10, "id": "2cd70850", "metadata": {}, "outputs": [], "source": [ "#define input\n", "\n", "X = CT_Dublin_adj.iloc[:,1:309].values" ] }, { "cell_type": "code", "execution_count": 11, "id": "4a858fdd", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[2.18 , 2.382, 2.346, ..., 2.776, 3.282, 3.347],\n", " [2.394, 1.973, 2.534, ..., 2.654, 3.124, 3.214],\n", " [2.551, 2.567, 1.954, ..., 2.495, 2.669, 2.886],\n", " ...,\n", " [2.273, 2.559, 2.578, ..., 2.294, 2.571, 2.875],\n", " [1.94 , 2.438, 2.272, ..., 2.51 , 2.759, 2.838],\n", " [2.108, 2.269, 2.145, ..., 2.551, 2.855, 2.985]])" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X" ] }, { "cell_type": "code", "execution_count": 12, "id": "d44d0e78", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(108, 308)" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X.shape" ] }, { "cell_type": "code", "execution_count": 13, "id": "aaa56b67", "metadata": {}, "outputs": [], "source": [ "#output\n", "\n", "y = CT_Dublin_adj.iloc[:,[0]].values" ] }, { "cell_type": "code", "execution_count": 14, "id": "73f341d3", "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "array([[0],\n", " [0],\n", " [0],\n", " [1],\n", " [1],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [0],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1],\n", " [1]])" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y" ] }, { "cell_type": "code", "execution_count": 15, "id": "c0d23e00", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(108, 1)" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y.shape" ] }, { "cell_type": "markdown", "id": "4728103a", "metadata": {}, "source": [ "The numpy.ravel() functions returns contiguous flattened array (1D array with all the input-array elements and with the same type as it). This step is required for the upcoming analyses. " ] }, { "cell_type": "code", "execution_count": 16, "id": "5377250a", "metadata": {}, "outputs": [], "source": [ "y = y.ravel()" ] }, { "cell_type": "code", "execution_count": 17, "id": "6401e42c", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(108,)" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y.shape" ] }, { "cell_type": "markdown", "id": "2cce2797", "metadata": {}, "source": [ "Now having defined our input and ouput data, to build the logistic regression model we need to split our data into train and test sets. For this, we use the train_test_split splitter function from Sklearn. The training set is the dataset on which the model is trained. This data is seen and learned by the model. The test set is a a subset of the training set and utilized for an accurate evaluation of a final model fit.\n", "With that function, the data gets divided into X_train, X_test, y_train and y_test. X_train and y_train are used for training and fitting the model. The X_test and y_test sets, however, are used for training the model if the correct labels were predicted. " ] }, { "cell_type": "markdown", "id": "2e75fed4", "metadata": {}, "source": [ "But before splitting the data into training and testing set, we use the StandardScaler() function to standardize our data. The function standardizes every feature (each column) indivudally by substracting the mean and then scaling to unit variance (dividing all the values by the standard deviation). As a result, we get a distribution with a mean equal to 0 and with a standard deviation equal to 1. " ] }, { "cell_type": "markdown", "id": "9c7abf57", "metadata": {}, "source": [ "Also, with such a small sample, the N=27 participants (108 * 25%) in the testing sample could differ considerably from the training sample by chance. To tackle this problem, we can run the cross validation for 5000 iterations." ] }, { "cell_type": "code", "execution_count": 29, "id": "090318e9", "metadata": {}, "outputs": [], "source": [ "n_iter = 5000\n", "y_preds = []\n", "y_tests = []\n", "\n", "\n", "# scale before splitting into test and train samples\n", "X_sc = StandardScaler().fit_transform(X)\n", "\n", "for i in range(n_iter):\n", " \n", " # take a new testing and training sample\n", " X_train, X_test, y_train, y_test = train_test_split(X_sc, y, test_size = 0.25, random_state = i)\n", " y_tests.append(y_test) # store the y_test sample\n", " \n", " # fit the logistic regression\n", " classifier = LogisticRegression(random_state = i, solver ='liblinear')\n", " classifier.fit(X_train, y_train)\n", " \n", " # get the y predictions and store\n", " y_pred = classifier.predict(X_test)\n", " y_preds.append(y_pred)" ] }, { "cell_type": "markdown", "id": "8f821651", "metadata": {}, "source": [ "The test size indicates the size of the test subset, a random sampling without replacement about 75% of the rows , the remaining 25% is put into the test set. The random_state parameter allows you to reproduce the same train test split each time when the code is run. With a different value for random_state, different information would flow into the train and test sets. " ] }, { "cell_type": "markdown", "id": "c555676e", "metadata": {}, "source": [ "The following outputs show the first five lines of the iterations for our predicted y values and y testing values." ] }, { "cell_type": "code", "execution_count": 19, "id": "1ce39e57", "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "[array([1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1,\n", " 1, 1, 0, 1, 0]),\n", " array([1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 0, 1,\n", " 0, 1, 0, 1, 0]),\n", " array([1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1,\n", " 1, 1, 0, 1, 1]),\n", " array([1, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1,\n", " 1, 1, 0, 0, 1]),\n", " array([1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0,\n", " 1, 0, 0, 0, 1])]" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_preds[:5]" ] }, { "cell_type": "code", "execution_count": 20, "id": "c901050c", "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "[array([1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0,\n", " 0, 1, 0, 1, 0]),\n", " array([1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 1,\n", " 0, 1, 0, 0, 0]),\n", " array([0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1,\n", " 0, 1, 0, 1, 0]),\n", " array([1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0,\n", " 1, 0, 0, 0, 0]),\n", " array([0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0,\n", " 1, 0, 0, 0, 1])]" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_tests[:5]" ] }, { "cell_type": "markdown", "id": "390aa641", "metadata": {}, "source": [ "In the following, we can concatenate the the y_pred and y_test results from each iteration and use this to compute the confusion matrix. " ] }, { "cell_type": "code", "execution_count": 40, "id": "1481ef01", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1, 0, 1, ..., 1, 1, 1])" ] }, "execution_count": 40, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_preds = np.concatenate([y_preds])\n", "y_preds" ] }, { "cell_type": "code", "execution_count": 41, "id": "eb27b859", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1, 0, 0, ..., 1, 0, 1])" ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_tests = np.concatenate([y_tests])\n", "y_tests" ] }, { "cell_type": "markdown", "id": "d487eed3", "metadata": {}, "source": [ "### 1.3 Model evaluation" ] }, { "cell_type": "markdown", "id": "610c1784", "metadata": {}, "source": [ "In the next section, we will have a look at how the logistic regression model performs and evaluate it. To evaluate the model, a look at different measurements such as the **confusion matrix**, **accuracy, precision and recall** is helpful." ] }, { "cell_type": "markdown", "id": "3534e34a", "metadata": {}, "source": [ "#### 1.3.1 Confusion Matrix" ] }, { "cell_type": "markdown", "id": "9f048bc1", "metadata": {}, "source": [ "The **confusion matrix** provides information on the quality of the logistic regression model since it shows the predicted values from the model compared to the actual values from the test dataset. We scale this by the sum of the array to get probabilities for **hits, misses, false positives and true negatives**." ] }, { "cell_type": "code", "execution_count": 45, "id": "58f882a8", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Confusion Matrix : \n", " [[0.43165926 0.31014074]\n", " [0.00807407 0.25012593]]\n" ] } ], "source": [ "cm_CT = confusion_matrix(y_tests, y_preds)\n", "\n", "cm_CT_f = cm_CT/np.sum(cm_CT)\n", "\n", "print(\"Confusion Matrix : \\n\", cm_CT_f)" ] }, { "cell_type": "markdown", "id": "bc49f766", "metadata": {}, "source": [ "To make the confusion matrix visually more appealing and more informative, we run the following code." ] }, { "cell_type": "code", "execution_count": 46, "id": "1eeac9d6", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Text(0.5, 257.44, 'Predicted label')" ] }, "execution_count": 46, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaUAAAFBCAYAAADXB7A6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAyMElEQVR4nO3dd5wV1fnH8c+zu/QiVUC6UgQsqIhGFBUBQQxogpGYYg2xmxhrjKgYE1ssyY9EUVGjUWzRoKCIKGJDKSIIiixFioCy9M7uPr8/Zna9u2y5d9nLzr1+377m5Z0zZ845c7l7n3vOnJkxd0dERCQKMqq6ASIiIgUUlEREJDIUlEREJDIUlEREJDIUlEREJDIUlEREJDIUlCLKzGqZ2atmttHMXtiLcn5hZm9WZtuqgpm9bmbnVnDfP5vZWjNbXdntShYze8jMbt7LMk4ysxUV3PcJM/tzGdu3mNmB5ZTRzszczLIq0gb5YVJQ2ktmdo6ZzQj/SFeFX57HV0LRQ4FmQGN3P6uihbj7f9y9fyW0p4jwC8/N7OVi6YeH6VPiLOdWM3u6vHzuPtDdn6xAO9sAfwC6unvzRPcvpUwzsyvN7HMz22pmK8zsBTM7tILlnWdm78emufvF7n57ZbS3hPrahJ/XgsXD4yhYP6G8Mty9rrsvTkb75IdNQWkvmNnVwAPAXwgCSBvgn8CQSii+LfCVu+dWQlnJ8h3wIzNrHJN2LvBVZVUQBoC9+Zy2AXLc/dsK1F3aL/wHgauAK4FGQCfgFWBQJdaRNO6+LAwqdd29bph8eEzae/u6TSKF3F1LBRZgP2ALcFYZeWoQBK1vwuUBoEa47SRgBcGv+G+BVcD54bbbgF3A7rCOC4Fbgadjym4HOJAVrp8HLAY2A0uAX8Skvx+z33HAdGBj+P/jYrZNAW4HPgjLeRNoUsqxFbT/IeCyMC0TWAmMAKbE5H0QWA5sAmYCJ4TpA4od52cx7bgjbMd2oEOYdlG4/V/ASzHl3wVMBqxYG/uG++eH5T8Rpg8G5gEbwnK7xOyzFLgemAPsLHh/Y7Z3BPKAnuV8Nv5NELS/Bv4EZMT8e3wA3A/kAC8BO8IytwAbwnxPAH+OKXMIMDt8DxcBA8L084Evwn+vxcBvi/8bxfFZdqBDsbQngFHA+LDsj4GDStoHqAX8LTzWjcD7YVo7in5Gfxq+v4fEbDsXWAasBW6KKT8DuCE81hzgeaBRuK0m8HSYvoHgc9ysrL8DLamzVHkDUnUh+ELNLf6lVSzPSGAasD/QFPgQuD3cdlK4/0igGnAasA1oGG6/laJBqPh64R88UCf8suocbmsBdAtfn0cYlAh+1a8HfhXu9/NwvXG4fUr4JdAp/FKZAtxZyrGdRBCUjgM+DtNOAyYCF1E0KP0SaBzW+QdgNVCzpOOKaccyoFu4TzWKBqXaBL2x84ATwi+0VmW1M2a9E7AV6BeWex2QDVQPty8l+PJvDdQqobyLga/L+Wz8G/gfUC/8d/oKuDDm3yMXuCI8tloU++EQ5nuCMCgBPQm+7PsRfFm3BA4Otw0CDgIMOJHgM3RkScdeRntLC0o5Yd1ZwH+AsSXtQxC8poTtygw/EzUo+hk9P3yfC/Yp2PZI+B4cTvAjoEu4/SqCv51WYVkPA8+G234LvBp+DjKBo4D6lPF3oCV1Fg3fVVxjYK2XPbz2C2Cku3/r7t8R9IB+FbN9d7h9t7tPIPil3LmC7ckHDjGzWu6+yt3nlZBnELDQ3Z9y91x3fxb4EvhxTJ7H3f0rd99O8Ou0e1mVuvuHQCMz6wz8muALuXiep909J6zzbwRfMuUd5xPuPi/cZ3ex8rYRvI/3EfxivsLd4z2hfzYw3t0nheXeS/CleFxMnr+7+/LwPSiuMUGvtkRmlgkMA250983uvpSgFxH77/6Nu/8jPLaS6ijuQmBM2OZ8d1/p7l8CuPt4d1/kgXcJerflnhOK08vu/kn4Gf8PJXwWwqHVC4CrwnblufuH7r4zJtvvgGuBk9w9u1gRt7n7dnf/DPiMIDhBEPxvcvcVYVm3AkPD4c7dBP8OHcL6Zrr7pnC/eP4OJMIUlCouB2hSzjmBAwiGNAp8HaYVllEsqG0D6pIgd99K8GV7MbDKzMab2cFxtKegTS1j1mNnqMXbnqeAy4GTgZeLbzSza8zsi3Am4QaC4a0m5ZS5vKyN7v4xwTCNEQTPeBV5D9w9P6wr9j0oq+4cgl/gpWlC0AMr/u8eb/klaU3Qg92DmQ00s2lmti58b0+j/Pc2XvF8FpoQDKeV2L7QtcCoUn44lFZHW+BlM9sQHtcXBEOczQg+bxOBsWb2jZndbWbVEvg7kAhTUKq4jwiGG84oI883BH9cBdqEaRWxlWC4okCRmWTuPtHd+xF8YX5JMCxSXnsK2rSygm0q8BRwKTAh7MUUCmdyXQf8jGBosgHBUJQVNL2UMsu8fb2ZXUbQ4/omLD9eRd4DMzOCL/3Y96CsuicDrcysRynb1xL8ki/+715W+eXdqn85wRBdEWZWg+Cc1L0E51QaABP4/r3dF9YSnBPbo30x+gN/MrOfJlDucmCguzeIWWqGvbHd7n6bu3cl6OGeTtBLj/fvQCJMQamC3H0jwQn9UWZ2hpnVNrNq4S/Xu8NszxL8MTY1syZh/nKnP5diNtA7nM67H3BjwQYza2ZmQ8ysDkGg3EIwjFHcBKBTOI09y8zOBroCr1WwTQC4+xKC8xk3lbC5HsE5lO+ALDMbQTD+X2AN0C6RGXZm1gn4M8G5ql8B15lZ9zh3fx4YZGanmFk1gnNcOwnO95XL3RcSzLB8NpwWX93MaprZMDO7wd3zwjruMLN6ZtYWuJqy/93XEAS66qVsfww4P2xzhpm1DHsA1QkC83dArpkNJAgA+0zY0xwD3GdmB5hZppn9KAyYBeYRnIMdZWaD4yz6IYL3sC1A+Dc0JHx9spkdGg6VbiL4EZCfwN+BRJiC0l4Iz49cTTC76juCX3eXE0wPhuCLcwbBTK65wKwwrSJ1TQKeC8uaSdFAkhG24xtgHUGAuKSEMnIIflX+gWAY6jrgdHdfW5E2FSv7fXcvqRc4EXiD4GT/1wS/qmOHrwouDM4xs1nl1RMOlz4N3OXun4VB4o/AU8W+CEtr5wKCYPYPgl/5PwZ+7O67yts3xpXA/xGc4N9AMHR1JsHJdwgmMWwlGF58H3iG4Iu7NG8TfHGvNrM9/i3c/ROCiQL3E/Qy3wXauvvmsC3PE0xYOQcYl8BxVJZrCD7f0wk+f3dR7LslPGd0OvBIGDzL8yDBsbxpZpsJJj0cE25rDrxIEJC+IHg/niLOvwOJNnPXQ/5ERCQa1FMSEZHIUFASEZHIUFASEZHIUFASEZHIUFASEZHIUFASEZHIUFASEZHIUFASEZHIUFASEZHIUFASEZHIUFASEZHIUFASEZHIUFASEZHIUFASEZHIUFASEZHIUFASEZHIUFASEZHIUFASEZHIUFCSKmNmeWY228w+N7MXzKz2XpT1hJkNDV8/amZdy8h7kpkdV4E6lppZk3jTi+XZkmBdt5rZNYm2USTVKShJVdru7t3d/RBgF3Bx7EYzy6pIoe5+kbvPLyPLSUDCQUlEkk9BSaLiPaBD2It5z8zGAfPNLNPM7jGz6WY2x8x+C2CB/zOzBWb2FrB/QUFmNsXMeoSvB5jZLDP7zMwmm1k7guD3+7CXdoKZNTWzl8I6pptZr3Dfxmb2ppnNM7NHASvvIMzsFTObGe4zvNi2+8P0yWbWNEw7yMzeCPd5z8wOrpR3UyRFVeiXqEhlCntEA4E3wqQjgUPcfUn4xb7R3Y82sxrAB2b2JnAE0BnoCjQD5gNjipXbFHgE6B2W1cjd15nZQ8AWd783zPcMcL+7v29mbYCJQBfgFuB9dx9pZoOAC+M4nAvCOmoB083sJXfPAeoAM9z992Y2Iiz7cmA0cLG7LzSzY4B/An0q8DaKpAUFJalKtcxsdvj6PeAxgmG1T9x9SZjeHzis4HwRsB/QEegNPOvuecA3ZvZ2CeUfC0wtKMvd15XSjr5AV7PCjlB9M6sb1vGTcN/xZrY+jmO60szODF+3DtuaA+QDz4XpTwP/Des4Dnghpu4acdQhkrYUlKQqbXf37rEJ4Zfz1tgk4Ap3n1gs32mV2I4M4Fh331FCW+JmZicRBLgfufs2M5sC1Cwlu4f1bij+Hoj8kOmckkTdROASM6sGYGadzKwOMBU4Ozzn1AI4uYR9pwG9zax9uG+jMH0zUC8m35vAFQUrZtY9fDkVOCdMGwg0LKet+wHrw4B0MEFPrUAGUNDbO4dgWHATsMTMzgrrMDM7vJw6RNKagpJE3aME54tmmdnnwMMEPfyXgYXhtn8DHxXf0d2/A4YTDJV9xvfDZ68CZxZMdACuBHqEEynm8/0swNsIgto8gmG8ZeW09Q0gy8y+AO4kCIoFtgI9w2PoA4wM038BXBi2bx4wJI73RCRtmbtXdRtEREQA9ZRERCRCFJRERCQyIjv7rlabn2tcUfa5FmecXdVNkB+YxX8/I7FpnuVI9Ltz+7JnK7X+vRXZoCQiIokzS+0BMAUlEZE0Yil+VkZBSUQkjainJCIikaGgJCIikZHo7bGiRkFJRCStqKckIiIRoeE7ERGJDAUlERGJDE0JFxGRyFBPSUREIkNBSUREIkNBSUREIsPQdUoiIhIR6imJiEhkpHpQSu3Wi4hIEWYZCS3xlWkDzGyBmWWb2Q1l5PupmbmZ9YhJuzHcb4GZnVpeXeopiYiklcrta5hZJjAK6AesAKab2Th3n18sXz3gKuDjmLSuwDCgG3AA8JaZdXL3vH3TehERqVJJ6Cn1BLLdfbG77wLGAkNKyHc7cBewIyZtCDDW3Xe6+xIgOyyvVApKIiJpJNGgZGbDzWxGzDK8WJEtgeUx6yvCtJg67UigtbuPT3Tf4jR8JyKSRhK9zZC7jwZGV7i+oLt1H3BeRcuIpaAkIpJGkjD7biXQOma9VZhWoB5wCDAlfJZTc2CcmQ2OY989aPhORCSNmFlCSxymAx3NrL2ZVSeYuDCuYKO7b3T3Ju7ezt3bAdOAwe4+I8w3zMxqmFl7oCPwSVmVqackIpJGKrun5O65ZnY5MBHIBMa4+zwzGwnMcPdxZew7z8yeB+YDucBlZc28AwUlEZG0koxHV7j7BGBCsbQRpeQ9qdj6HcAd8daloCQikkZS/Y4OCkoiImlEQUlERCJDT54VEZHoUE9JRESiQsN3IiISGXFeexRZCkoiImlE55RERCQyNHwnIiLRoeE7ERGJjEwFJRERiQr1lEREJDJS+5SSgpKISDpx9ZRERCQyUjsmKSiJiKSVjNSOSgpKIiLpRMN3IiISGakdkxSURETSiobvREQkMjR8JyIikZHaMSnVL7MSEZEiMiyxJQ5mNsDMFphZtpndUML2i81srpnNNrP3zaxrmN7OzLaH6bPN7KHy6lJPSUQknVRyT8nMMoFRQD9gBTDdzMa5+/yYbM+4+0Nh/sHAfcCAcNsid+8eb33qKYmIpBE3S2iJQ08g290Xu/suYCwwpEid7ptiVusAXtH2KyiJiKSTBIfvzGy4mc2IWYYXK7ElsDxmfUWYVoSZXWZmi4C7gStjNrU3s0/N7F0zO6G85mv4TkQknSQ4fOfuo4HRe1utu48CRpnZOcCfgHOBVUAbd88xs6OAV8ysW7GeVRHqKYmIpBOzxJbyrQRax6y3CtNKMxY4A8Ddd7p7Tvh6JrAI6FRWZQpKIiLppPJn300HOppZezOrDgwDxsVmMLOOMauDgIVhetNwogRmdiDQEVhcVmUavhMRSSeVPPvO3XPN7HJgIpAJjHH3eWY2Epjh7uOAy82sL7AbWE8wdAfQGxhpZruBfOBid19XVn0KSiIi6SQJd3Rw9wnAhGJpI2JeX1XKfi8BLyVSl4KSiEg60W2GREQkMlJ8poCCkohIOlFPSUREIiO1Y1Kqd/RSW78TD+ezd/7G51Pv55pLB5ea74yBPdm+7FmOPOxAAHocfhDTXv8r017/Kx+/cSeDT+1RmPehe37L17MeYsaku/co55LzTmX22/cy8617uOOP5wDQplUT1n31ZGF5f//LhYX5h/74WD6ZeBcz37qHP9/488o6bKlCvbvsz1s3ncLbN/fl4r4d99h+Tq92vH7Dybx23ck8f9UJdGheD4AGtavxnyt6Mfee07l16GFF9jmk9X68fsPJvH1zX0b89NDC9IHdD+CNG/uQ/cAQDm3dYI+6DmhYi7n3nM5FfToUpt11zhF8csdAXr+hTyUd8Q+PZ1hCS9Sop1RFMjKMB/58PoN+8RdWrsrh/Vfv4LVJM/lyYdFr0urWqcllFwzgk1kLC9PmLVhOr9NvIi8vn+b7N+DjN+5k/FuzyMvL56kX3uWhJyfy6P2XFimn94+6cnr/o+g54AZ27cqlaeP6hdsWf72GYwfeWCR/owZ1+csff8Fxg/7I2nWbeeS+SzipVzemfDAvCe+G7AsZBreddTi/HvUBqzds55VrTuKtz1eTvXpzYZ5xM1fwzAdLATjlkObcdOYhnP+vj9iZm8/947+gU4v6dGpRv0i5t/+sOzeOnc3spesZc/GPOLHL/rz7xbd8tWoTlzz2CXec3b3E9tx05iG8O39NkbQXP17Gv6cu5t5fHlWpx/6DkuLDd+opVZGju3dg0dLVLF32Lbt35/HCqx9xev8ee+S75Zqf8bd/vcqOnbsL07bv2EVeXj4ANWpUw2NuffjBJ1+ybsOWPcoZ/qt+3PvPcezalQvAdzml3uUDgPZt9id76WrWrgu+sN5+fy5nDDwm4eOU6Di8bUO+/m4Ly3O2sTvPeW3WCvod2rxIni07cgtf166eWfjZ2r4rjxmL17Fzd36R/E3r16BuzSxmL10PwMufLKPfYS0AWLRmC0u+3fOzCNDv0BYsz9nGwpiACDB9UQ4btu0ucR+JkyW4REzSgpKZHWxm15vZ38PlejPrkqz6Us0BzRuy4pucwvWVq3Jo2axhkTzdD2lHqxaNeOPtT/fY/+juBzHzrXuY8ebdXPnHRwuDVGk6tG9Or54HM/V/t/Pm8yM4KhwKBGjXuikfTfgrbz4/gl49OwOw6Os1dDqwBW1aNSEzM4PB/XvQ6oBGe3PIUsWaN6jFqg3bC9dXbdhBs/1q7ZHvVye0550R/bh+SDdGvjSn7DL3q8XqmDJXb9hB8xLKjFW7eia/7duRv7/+ZYJHIHFJwvOU9qWkDN+Z2fXAzwnugfRJmNwKeNbMxrr7ncmoN52YGXfd/Ct+84d/lbh9+uxFHNX3Wjp3OIBH77uEiVM+Y+fO0n9hZmVl0mi/uvQecjM9Dj+Ip/95FV2Ov4rV326g07FXsG7DFo44tD3PP/IHjux7LRs2buXKm8bw9KiryM/PZ9rMhRzYdv9kHa5EyFPvLeGp95Yw+KhWXNa/M9f+Z1alln/VwIMZMyWbbbvyKrVcCaX48F2yzildCHRz9yLfkmZ2HzAPKDEohbdMHw6Q1bAHWXU7lJQtLXyzej2tDmhcuN6yRWNWrllfuF6vbk26dm7Nm88FF003a7ofLz52DUMvvJdZc76/ddSC7G/YsnUn3Tq3LpJe3MpV63jljeD3wYzPFpHvTpNG9Vi7bjPrdgVDLJ/OXcLir9fQ8cAWzJqzmAlvzWLCW8EX0gXn9CEvv+zemETb6g3badHg+15MiwY1WbNxe6n5X521gtt/djjX/qeMMjdup3lMmc0b1GR1GWUCdG/XkIHdW3LD4EOoX6sa+e7s3J3HU+8tif9gpHSpHZOSNnyXDxxQQnqLcFuJ3H20u/dw9x7pHJAgCAwd2jenbeumVKuWyVk//hHjJ80s3L5p83Zadx/Owb2u5OBeV/LJp9mFAalt66ZkZgb/dG1aNqFzhwP4evl3Zdb36pszOPFHXYFgKK96tSzWrttMk0b1yAi78O3a7E+H9s1Z8nVw8rlgMkSD/eow/Ff9ePzZtyv9fZB9Z86yDbRrWpdWjWpTLdM4/chWvDV3dZE87ZrWKXx9crfmLP2u5HNCBb7btJMtO3Lp3i4Yej6zZ5s9yizu7Affp/dtb9L7tjd5/N1F/HPSVwpIlUnDdyX6HTDZzBby/cOh2gAdgMuTVGdKycvL5/c3P8GrT91IZmYGTz43hS++WsHNVw9l1twlRQJUcccd3ZlrLh3C7t255Oc7V900hpz1wQnjJ/9xBSf8qAtNGtYj++P/4/b7XuTJ56bw5HPv8PA9FzNj0t3s2pXLRVcHw4LHH9OFm/9wVmFZV/zxMdZv3ArAvbeey6Fd2wDw1wf+S/aSsr9sJNry8p1bX5zDk5ceR0aG8cK0r1m4ejO/O+1g5i7bwOTPV/OrEw6kV+em5OY5G7fv4pqnvx+6m3pLf+rWzKJaVgb9DmvBuf/8kOzVmxnx/Gfc/YsjqVk9k3fnr2FKOKOu/2EtuGXoYTSqW53Hfnss81du5Lx/fVRmGx88twfHdGhCw7rV+WDkqTw44Uuen/Z1Ut+XtBPBQJMIc6/wU2vLLtgsg+AxugVPKFwJTHf3uAaSa7X5eXIaJlKGFmecXdVNkB+YxX8/o1KjyIEXvZDQd+fiR8+KVBRL2nVK7p4PTEtW+SIiUoIU7ynp4lkRkXSi2XciIhIZ6imJiEhkpPh9ehSURETSiYbvREQkKjwztbtKCkoiIukktWOSgpKISFpJ8YkOKR5TRUSkCLPElriKtAFmtsDMss3shhK2X2xmc81stpm9b2ZdY7bdGO63wMxOLa8u9ZRERNJJJfeUzCwTGAX0A1YA081snLvPj8n2jLs/FOYfDNwHDAiD0zCgG8H9UN8ys05l3dlHPSURkXRS+Q/56wlku/tid99F8EiiIbEZ3D32qaF1gIJbHQ0Bxrr7TndfAmSH5ZVKPSURkTTiCfaUYh8ZFBrt7qNj1lvy/Y21Iegt7fEYajO7DLgaqA70idk39nZzK/j+fqglUlASEUknCQalMACNLjdj+eWMAkaZ2TnAn4BzK1KOhu9ERNJJ5U90WAm0jllvFaaVZixwRgX3VVASEUkrGQku5ZsOdDSz9mZWnWDiwrjYDGbWMWZ1ELAwfD0OGGZmNcysPdAR+KSsyjR8JyKSTir5NkPunmtmlwMTgUxgjLvPM7ORwAx3HwdcbmZ9gd3AesKhuzDf88B8IBe4rLxn6ikoiYikkyRcPOvuE4AJxdJGxLy+qox97wDuiLcuBSURkXSS4nd0UFASEUkjrruEi4hIZKT49DUFJRGRdKKekoiIRIbOKYmISGQoKImISGSkdkxSUBIRSSeJ3pA1ahSURETSiSY6iIhIZKinJCIikZHaMUlBSUQknWTo4lkREYmKFD+lpKAkIpJO0jYomdlmwAtWw/97+NrdvX6S2yYiIgmyFI9KpQYld6+3LxsiIiJ7L8VjUnz3kzWz483s/PB1k/CxtiIiEjFmiS1RU+45JTO7BegBdAYeB6oDTwO9kts0ERFJlP0AZt+dCRwBzAJw92/MTEN7IiIRFMXeTyLiCUq73N3NzAHMrE6S2yQiIhWU4jd0iOuc0vNm9jDQwMx+A7wFPJLcZomISEUk45ySmQ0wswVmlm1mN5Sw/Wozm29mc8xsspm1jdmWZ2azw2VceXWV21Ny93vNrB+wCegEjHD3SfEdioiI7EuVPXxnZpnAKKAfsAKYbmbj3H1+TLZPgR7uvs3MLgHuBs4Ot2139+7x1hfvxbNzgVoE1ynNjbdwERHZt5JwnVJPINvdF4fljwWGAIVByd3fick/DfhlRSsrd/jOzC4CPgF+AgwFppnZBRWtUEREkscyElzMhpvZjJhleLEiWwLLY9ZXhGmluRB4PWa9ZljuNDM7o7z2x9NTuhY4wt1zAMysMfAhMCaOfUVEZB9KtKPk7qOB0ZVTt/2S4BKiE2OS27r7SjM7EHjbzOa6+6LSyohnokMOsDlmfXOYJiIiEZOEiQ4rgdYx663CtGL1Wl/gJmCwu+8sSHf3leH/FwNTCC4xKlVZ9767OnyZDXxsZv8jOKc0BJgTx4GIiMg+lln5F89OBzqGd/JZCQwDzonNYGZHAA8DA9z925j0hsA2d99pZk0Ibrpwd1mVlTV8V3CB7KJwKfC/OA9ERET2scqe5+DuuWZ2OTARyATGuPs8MxsJzHD3ccA9QF3ghXCixTJ3Hwx0AR42s3yCkbk7i83a20NZN2S9rVKOSERE9plk3NHB3ScAE4qljYh53beU/T4EDk2krnjufdcUuA7oBtSMqaxPIhWJiEjyWYrf0iGe0cf/AF8C7YHbgKUEY4wiIhIxqX6X8HiCUmN3fwzY7e7vuvsFgHpJIiIRlOpBKZ7rlHaH/19lZoOAb4BGyWuSiIhUVBQDTSLiCUp/NrP9gD8A/wDqA79PaqtERKRCUvyUUlw3ZH0tfLkRODm5zRERkb2Rtj0lM/sHwcWyJXL3K5PSIhERqbB0fvLsjH3WChERqRRp21Ny9yf3ZUNERGTvJeHRFftUvM9TEhGRFJDiMUlBSUQknSgoiYhIZKRtUKrq2Xfbl+l+sLLvHfrvNVXdBJG9ks7XKWn2nYhIiknboKTZdyIiqSfDSh3gSgnxPrrieqArenSFiEikpXpPKd5HV3yBHl0hIhJ5GQkuUaNHV4iIpJEM84SWqNGjK0RE0kiqD9/p0RUiImkkikNyidCjK0RE0kja95TM7HFKuIg2PLckIiIRYkk4T2RmA4AHgUzgUXe/s9j2q4GLgFzgO+ACd/863HYu8Kcw65/Lu9wonuG712Je1wTOJDivJCIiEVPZPSUzywRGAf2AFcB0Mxvn7vNjsn0K9HD3bWZ2CXA3cLaZNQJuAXoQdG5mhvuuL62+eIbvXirWwGeB9xM8LhER2QeScE6pJ5Dt7osBzGwsMAQoDEru/k5M/mnAL8PXpwKT3H1duO8kYADwbGmVVaT9HYH9K7CfiIgkWaJTws1suJnNiFmGFyuyJbA8Zn1FmFaaC4HXK7hvXOeUNlP0nNJqgjs8iIhIxCQ6fOfuo4HRlVG3mf2SYKjuxIqWEc/wXb2KFi4iIvtWEobvVgKtY9ZbhWlFmFlf4CbgRHffGbPvScX2nVJWZeW238wmx5MmIiJVL8MSW+IwHehoZu3NrDowDBgXm8HMjgAeBga7+7cxmyYC/c2soZk1BPqHaaUq63lKNYHaQJOwsILm16ecMUEREakalX3rIHfPNbPLCYJJJjDG3eeZ2UhghruPA+4B6gIvWPCUwWXuPtjd15nZ7Xx/v9SRBZMeSlPW8N1vgd8BBwAz+T4obQL+r0JHJyIiSZWMi2fdfQIwoVjaiJjXfcvYdwwwJt66ynqe0oPAg2Z2hbv/I94CRUSk6qT6bYbiaX++mTUoWAnHBi9NXpNERKSiUv0u4fEEpd+4+4aClfBK3N8krUUiIlJhSZjosE/Fc5uhTDMzd3covOVE9eQ2S0REKiKKgSYR8QSlN4DnzOzhcP23YZqIiERMqp9TiicoXQ8MBy4J1ycBjyStRSIiUmFZGdE7T5SIcoOqu+e7+0PuPtTdhxLchE+z8UREIigjwSVq4ukpFVyt+3PgZ8AS4L/JbJSIiFRM2p5TMrNOBIHo58Ba4DnA3F1PnxURiahkPORvXyqrp/Ql8B5wurtnA5jZ7/dJq0REpEJSvadU1pDiT4BVwDtm9oiZncL3txoSEZEISvVzSqW2yd1fcfdhwMHAOwT3wdvfzP5lZv33UftERCQBaX9HB3ff6u7PuPuPCZ6F8Sl6yJ+ISCT9EO7oUCi8xVClPaVQREQqVxQDTSISCkoiIhJtmVXdgL2koCQikkaieJ4oEQpKIiJpRMN3IiISGQpKIiISGZkKSiIiEhWp3lOK4gW9IiJSQcm4eNbMBpjZAjPLNrMbStje28xmmVmumQ0tti3PzGaHy7jy6lJPSUQkjVR2Tyl82vgooB+wAphuZuPcfX5MtmXAecA1JRSx3d27x1ufgpKISBpJwnVKPYFsd18MYGZjgSEEz9YDwN2Xhtvy97YyDd+JiKSRRG8zZGbDzWxGzDK8WJEtgeUx6yvCtHjVDMudZmZnlJdZPSURkTSS6MWz7p7sW8e1dfeVZnYg8LaZzXX3RaVlVlASEUkjSZgSvhJoHbPeKkyLi7uvDP+/2MymAEcApQYlDd+JiKSRJNwlfDrQ0czam1l1YBhQ7iw6ADNraGY1wtdNgF7EnIsqsf1xNUlERFJCZQcld88FLgcmAl8Az7v7PDMbaWaDAczsaDNbAZwFPGxm88LduwAzzOwzgufy3Vls1t4eNHwnIpJGknHxrLtPACYUSxsR83o6wbBe8f0+BA5NpC4FJRGRNJKpu4SLiEhUpPo5GQUlEZE0kur3vlNQEhFJIwpKIiISGTqnJCIikaGekoiIRIaCkoiIRIaCkoiIRIYehy4iIpGR6F3Co0ZBSUQkjejiWSnX1KkzueOOR8jPz+ess/oxfPhZRbbv2rWb6667j3nzFtGgQT3uv/86WrVqBsDDD7/Aiy9OIiMjgz/9aTgnnHAkAE888QovvPAmZkanTu3461+vokaN6ixfvpqrr76HDRs2063bQdx999VUr16Nv/zlET7+eC4AO3bsJCdnIzNmjC1sw5Yt2zjttEvp2/dYRoy4GIDPP8/mxhsfYMeOXZx44lHcdNNwzIzf/e4uliwJ7ly/efNW6tWrw//+9/ekv4+y93od0JDrjz6QTDP+m72axz5fUWT7r7u05Ccdm5Pnzroduxnx4Ves2roTgNm/PJ6FG7YCsGrrTq58J7ivZsu6Nbj7hINpUKMa89dt4cb3F5Cb7xy1f32uO/ogOjWsw3VTv2TSsrUAdG5Yh5uP7UCdapnkO4yeu4yJS9cWtuGK7m3p364J+fnw3FereObLb/bFW5M2dE5JypSXl8fIkQ/x+OO306xZY4YOvZo+fY6hQ4c2hXleeOFN6tevy6RJoxk/fir33vsEDzxwPdnZyxg/firjx49izZoczj//ZiZOfIi1azfw73+/yoQJ/6RmzRpcddWdjB8/lZ/8pC/33vsE5503hEGDejNixChefHES55xzGn/8428K63vqqVeZP39xkXY+8MDTHH10tyJpt976T26//XIOP7wzv/nNrUydOpMTT+zBAw9cX5jnzjsfo27d2kl696QyZRjcdMxBDJ/0Oau37WTsad15Z/k6Fm/cVpjni3VbGDb+U3bk5fOzTi24+qj2XDv1SwB25uVz1muf7lHu749sz1NffMMbS7/j5mM68JMOzXn+q1Ws2rqTmz9YwLndit6nc0duPn98fwHLNu+gaa3qPDfoCD5cuZ7Nu/M446BmNK9Tg8GvzMSBRjWrJfU9SUepfk4p1Xt6kTdnzkLatm1B69bNqV69GoMG9Wby5I+L5Hn77Y8588xTADj11F589NFnuDuTJ3/MoEG9qV69Gq1bN6dt2xbMmbMQgLy8fHbs2EVubh47duxk//0b4e5MmzaHU0/tBcCZZ57C5MnT9mjT+PFTOf303oXrn3+eTU7OBnr1OqIw7dtv17Flyza6dz8YM+OMM/rsUZa78/rr73P66SdWzpslSXVo43os27yDFVt2kJvvvL70O05u3ahInulrNrIjLx+AOWs30ax29XLL7dm8AZO+/g6AcYvW0KdNYwC+2bqTrzZsw4ud4vh683aWbd4BwHfbd7Fuxy4ahsHnZ51b8NCcZRTssm7H7ooe7g9WhnlCS9QoKCXZmjU5NG/epHC9WbPGrFmTs0eeFi2CPFlZmdSrV4f16zeVsG8T1qzJoVmzxlxwwZmcfPIFHH/8r6lbtw7HH38k69dvon79umRlZQLQvPmeda1c+S0rVqzh2GMPAyA/P5+77nqM66+/oMx2N2/eZI+yZsyYR+PGDWjX7oCKvj2yD+1fuwarw6E4gDXbdtGsdo1S8/+kQ3PeX7m+cL16ZgZjT+vO0wMPp0/rIPA0qJHF5l255IXfbau37WT/WuUHsgKHNK5LtYwMlodBqnW9mgxo15Sxp3XnX6d0o029mokcogBZGYktUbPPm2Rm55exbbiZzTCzGaNHP7cvm5VSNm7cwuTJHzN58qO8996TbN++g//975249h0/fiqnntqLzMwgcD3zzAR69+5RJADF67XXiva4JH2c3r4pXRvX5fF5359zOvWlTxg2YTY3vLeA644+kFZ19y5gNKlVjb8c35mbP/yqsGdUPSODnXn5DJswmxcXrmbkcZ32qo4foowEl6ipinNKtwGPl7TB3UcDo4O1r6LXr6yAZs0as3r19ydxC3o6xfOsWrWW5s2bkJubx+bNW2nYsH4J+66lWbPGfPjhbFq1akajRvsB0L//cXz66RcMHnwSmzZtITc3j6ysTFav3rOuCRPeK5zIAPDpp18yc+Y8nn12Alu3bmf37lxq167Jr389uEjdq1evLVJWbm4ekyZ9xH//e3/lvFGSdN9u20nzOt/3jJrVrs6abTv3yHdsiwb85tA2nP/mHHbnf/9n+O32XQCs2LKDGas30qVRXSYtW0u96llkGuQ5NK9dozBfWepUy2RUn0P4x6dfM2ft5sL0Ndt2MjmcEDF5WQ63KyglzHROaU9mNqeUZS7QLBl1RtWhh3Zk6dJvWL58Nbt27Wb8+Kn06dOzSJ4+fY7h5ZcnAzBx4gcce+xhmBl9+vRk/Pip7Nq1m+XLV7N06TccdlhHDjigKZ999iXbt+/A3fnoo8846KDWmBnHHHMYEyd+AMDLL0+mT59jCutZtGg5mzZt4YgjDi5M+9vfrmHKlMd5++1gCO+MM/pwzTXnsf/+jahbtzazZ3+Ju/PKK29zyinHFu734YezOfDAlhXqYUnV+DxnM23r1aRl3RpkZRgD2zVlyvJ1RfIc3KgOI47twBXvzCtyPqd+9SyqhdO6GtTIovv+9VkUTpCYvnoD/do2BWDwQc14Z3nRYd7isjKMB07qyquL1xTOyCvw9vIcjm7eAIAezfbj603b9+qYf4gswSVqktVTagacCqwvlm7Ah0mqM5KysjIZMeJiLrroFvLy8vnpT/vSsWNbHnzwaQ45pCOnnHIMQ4f249pr76Nfv+Hst19d7r//OgA6dmzLwIHHc9ppl5KZGZSTmZnJ4Yd35tRTe3Hmmb8jKyuTLl0O5OyzBwBw7bXn8fvf380DDzxNly4HctZZ/QvbMmHCe5x22glYnD+lbrnlksIp4b17H0Xv3kfFlDWVQYM0wSGV5Dn85ZNFPNT3EDLNeDl7DYs2buOyw9syL2czU1as4w9Htad2ViZ/O7EL8P3U7/b71eKWYzuS706GGY99vrxw1t79s5Zyd++DuaJ7W75ct4X/LlwNQLfGdXnwpK7Uq57Fia0bcWn3Npw5bhYD2jbhqGb1aVAjiyEHBb9R//TBVyxYv5XH5i7nzhMO5tddWrItN49bPlpYNW9WCkv1npJ58akxlVGo2WPA4+7+fgnbnnH3c8ovJT2G7yS1HPrvNVXdBPmBmfvrEyo1jMxaOz6h784jmwyKVBhLyvCdu19YUkAKt8URkEREpCLMPKElvjJtgJktMLNsM7uhhO29zWyWmeWa2dBi2841s4Xhcm55deniWRGRNFLZ3R4zywRGAf2AFcB0Mxvn7vNjsi0DzgOuKbZvI+AWoAfgwMxw3+KndgpFcUagiIhUkFliSxx6AtnuvtjddwFjgSGxGdx9qbvPAfKL7XsqMMnd14WBaBIwoKzKFJRERNJIEmbftQSWx6yvCNOSsq+CkohIGsmwxJbYmxaEy/CqbL/OKYmIpJFEzykVvWlBiVYCrWPWW4Vp8VgJnFRs3yll7aCekohIGknCOaXpQEcza29m1YFhwLg4mzMR6G9mDc2sIdA/TCuVgpKISBqp7HNK7p4LXE4QTL4Annf3eWY20swGA5jZ0Wa2AjgLeNjM5oX7rgNuJwhs04GRYVqpNHwnIpJGknElrLtPACYUSxsR83o6wdBcSfuOAcbEW5eCkohIGtGTZ0VEJDJSPCYpKImIpJN4bx0UVQpKIiJpRD0lERGJjFR/dIWCkohIGkn163wUlERE0oh6SiIiEhkpHpMUlERE0ol6SiIiEhkpHpMUlERE0onu6CAiIpGR4jFJQUlEJJ3ojg4iIhIZ6imJiEhkaPadiIhERorHJAUlEZF0otsMiYhIZGj4TkREIiS1o5KCkohIGjEFJRERiQqz1D6rlNqtFxGRYizBJY4SzQaY2QIzyzazG0rYXsPMngu3f2xm7cL0dma23cxmh8tD5dWlnpKISBqxSu5rmFkmMAroB6wAppvZOHefH5PtQmC9u3cws2HAXcDZ4bZF7t493vrUUxIRSSNmGQktcegJZLv7YnffBYwFhhTLMwR4Mnz9InCKWcXmASooiYiklcSG78xsuJnNiFmGFyuwJbA8Zn1FmFZiHnfPBTYCjcNt7c3sUzN718xOKK/1Gr4TEUkjic6+c/fRwOjktIZVQBt3zzGzo4BXzKybu28qbQf1lERE0ogl+F8cVgKtY9ZbhWkl5jGzLGA/IMfdd7p7DoC7zwQWAZ3KqkxBSUQkrWQkuJRrOtDRzNqbWXVgGDCuWJ5xwLnh66HA2+7uZtY0nCiBmR0IdAQWl1WZhu9ERNJIBecXlMrdc83scmAikAmMcfd5ZjYSmOHu44DHgKfMLBtYRxC4AHoDI81sN5APXOzu68qqT0FJRCStVP4dHdx9AjChWNqImNc7gLNK2O8l4KVE6lJQEhFJI7rNkIiIREhqTxVQUBIRSSPqKYmISGRU9kSHfU1BSUQkrSgoiYhIRFT2DVn3NQUlEZG0op6SiIhEhM4piYhIhCgoiYhIROickoiIRIh6SiIiEhG6eFZERCJDEx1ERCRCdE5JREQiQsN3IiISIQpKIiISETqnJCIiEaJzSiIiEhGpfk7J3L2q2yCVzMyGu/voqm6H/HDoMyeVJbX7eVKa4VXdAPnB0WdOKoWCkoiIRIaCkoiIRIaCUnrS2L7sa/rMSaXQRAcREYkM9ZRERCQyFJRERCQyFJTSiJkNMLMFZpZtZjdUdXsk/ZnZGDP71sw+r+q2SHpQUEoTZpYJjAIGAl2Bn5tZ16ptlfwAPAEMqOpGSPpQUEofPYFsd1/s7ruAscCQKm6TpDl3nwqsq+p2SPpQUEofLYHlMesrwjQRkZShoCQiIpGhoJQ+VgKtY9ZbhWkiIilDQSl9TAc6mll7M6sODAPGVXGbREQSoqCUJtw9F7gcmAh8ATzv7vOqtlWS7szsWeAjoLOZrTCzC6u6TZLadJshERGJDPWUREQkMhSUREQkMhSUREQkMhSUREQkMhSUREQkMhSUREQkMhSUREQkMv4fq5ceWgAaA5kAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "class_names=[0,1]\n", "\n", "fig, ax = plt.subplots() \n", "tick_marks = np.arange(len(class_names)) \n", "plt.xticks(tick_marks, class_names) \n", "plt.yticks(tick_marks, class_names) \n", "sns.heatmap(pd.DataFrame(cm_CT_f), annot=True, cmap=\"YlGnBu\" ,fmt='g') \n", "ax.xaxis.set_label_position(\"top\") \n", "plt.tight_layout() \n", "plt.title('Confusion Matrix for Cortical Thickness', y=1.1) \n", "plt.ylabel('Actual label') \n", "plt.xlabel('Predicted label')" ] }, { "cell_type": "markdown", "id": "02311028", "metadata": {}, "source": [ "The confusion matrix can be interpreted as follows:\n", "\n", "- Bottom righ square: **True positives** or **hits** -> predicted a subject to be patient and he is\n", "- Upper left square: **True negatives** -> predicted a subject to be control and he is\n", "- Upper right square: **False positive (Type 1 Error)** -> predicted a subject as patient but he is control\n", "- Bottom left square: **False negative (Type 2 Error)** or **misses** -> predicted a subject as control but he is patient" ] }, { "cell_type": "markdown", "id": "d5d2c2dc", "metadata": {}, "source": [ "The results show that the probability for **hits** is around 25%, for **true negatives** around 43%. The probability for **misses** is around 0.8%. The probability for **false positive** cases is around 31%.\n", "As we can see, the model is better at classifying controls than patients. \n", "However, based on these values we can compute other measures such as **accuracy, precision, recall and F1-Score** indicating the qualtiy of the model." ] }, { "cell_type": "markdown", "id": "9103b861", "metadata": {}, "source": [ "#### 1.3.2 Model accuracy, precision, recall F1-Score" ] }, { "cell_type": "markdown", "id": "c72e03e4", "metadata": {}, "source": [ "The measures indicate the following: \n", "\n", "- **Accuracy**: percentage of correct predictions \n", "- **Precision**: correct positive predictions relative to total positive predictions. In other words: From all the cases predicted as positive, how many are actually positive. \n", "- **Recall**: the correct positive predictions relative to total actual positives. In other words: From all the positive cases, how many were predicted correctly. \n", "- **F1-Score**: combines the precision and recall of a classifier into a single metric by taking their harmonic mean. It is primarily used to compare the performance of two classifiers." ] }, { "cell_type": "markdown", "id": "ef8b53e9", "metadata": {}, "source": [ "To compute these measures, we can simply use ```sklearn.metrics module```." ] }, { "cell_type": "code", "execution_count": 48, "id": "2eeec99b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Accuracy: 0.6817851851851852\n", "Precision: 0.446440848273309\n", "Recall: 0.9687293800384428\n", "F1-Score: 0.6112061397554596\n" ] } ], "source": [ "print(\"Accuracy:\",metrics.accuracy_score(y_tests, y_preds)) \n", "\n", "print(\"Precision:\",metrics.precision_score(y_tests, y_preds)) \n", "\n", "print(\"Recall:\",metrics.recall_score(y_tests, y_preds)) \n", "\n", "print(\"F1-Score:\", metrics.f1_score(y_tests, y_preds))" ] }, { "cell_type": "markdown", "id": "ec04efc1", "metadata": {}, "source": [ "So as the values show, our logistic regression model for **cortical thickness** makes 68.18% of the time correct predictions. 44.64% represent the proportion of the model's prediction of psychosis where psychosis is actually present and 96.87% relate to the proportion of all cases of psychosis that the model accurately predicted. " ] }, { "cell_type": "markdown", "id": "4c3ab202", "metadata": {}, "source": [ "## 2. Micro-structural data: mean diffusivity (MD) and fractional anisotropy (FA)" ] }, { "cell_type": "markdown", "id": "d4cc3ede", "metadata": {}, "source": [ "In the following section, the logistic regression model is computed for **micro-structural** data in an analogous way as for the macro-structural data. We first start with **MD**." ] }, { "cell_type": "markdown", "id": "a2ba209f", "metadata": {}, "source": [ "### 2.1 Mean diffusivity" ] }, { "cell_type": "markdown", "id": "c39656f1", "metadata": {}, "source": [ "First, we prepare our data to use it in the code efficiently and adjust it for the logistic regression accordingly." ] }, { "cell_type": "markdown", "id": "56a83991", "metadata": {}, "source": [ "#### 2.1.1 Data preparation" ] }, { "cell_type": "code", "execution_count": 50, "id": "d7217d4a", "metadata": {}, "outputs": [], "source": [ "#read data\n", "\n", "MD_Dublin_path = os.path.join(os.pardir, 'data', 'PARC_500.aparc_MD_cortexAv_mean_Dublin.csv')\n", "MD_Dublin = pd.read_csv(MD_Dublin_path)" ] }, { "cell_type": "code", "execution_count": 51, "id": "d8f1126d", "metadata": {}, "outputs": [], "source": [ "#adjust dataframe\n", "\n", "MD_Dublin_adj = MD_Dublin.drop(['Subject ID','Age', 'Sex'], axis=1)" ] }, { "cell_type": "code", "execution_count": 52, "id": "6a0c154b", "metadata": {}, "outputs": [], "source": [ "#label group 1 as 0 and 2 as 1\n", "\n", "MD_Dublin_adj['Group'] = MD_Dublin_adj['Group'].replace([1,2],[0, 1])" ] }, { "cell_type": "code", "execution_count": 53, "id": "e12a8836", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Grouplh_bankssts_part1_thicknesslh_bankssts_part2_thicknesslh_caudalanteriorcingulate_part1_thicknesslh_caudalmiddlefrontal_part1_thicknesslh_caudalmiddlefrontal_part2_thicknesslh_caudalmiddlefrontal_part3_thicknesslh_caudalmiddlefrontal_part4_thicknesslh_cuneus_part1_thicknesslh_cuneus_part2_thickness...rh_supramarginal_part5_thicknessrh_supramarginal_part6_thicknessrh_supramarginal_part7_thicknessrh_frontalpole_part1_thicknessrh_temporalpole_part1_thicknessrh_transversetemporal_part1_thicknessrh_insula_part1_thicknessrh_insula_part2_thicknessrh_insula_part3_thicknessrh_insula_part4_thickness
000.9110.9310.8911.0480.8810.9391.1240.9861.045...0.9281.0671.0960.8921.2381.0211.1660.9000.9070.937
100.8610.9130.8460.9270.8880.8940.9241.0401.093...0.8780.9851.0451.0011.1961.0831.1430.9170.9230.960
200.8170.8270.8280.8280.7800.8430.8250.8480.838...0.8470.8490.8190.9520.9330.9421.0590.7940.8340.860
300.8870.9050.8780.9320.8200.8880.9700.9180.900...0.9570.9850.9891.0751.1501.0170.9860.8880.9160.928
400.8870.8540.9051.0110.9460.9221.0341.1261.114...0.8710.9520.9871.3250.9961.0941.0640.9660.9890.977
..................................................................
11010.8430.8550.9401.0170.9540.8401.1281.0120.997...0.9381.0621.1430.9031.3641.2841.2181.0170.9721.028
11110.9110.9140.9261.0010.9181.1151.0361.0261.001...0.9571.0851.0981.0591.2681.0891.1730.9901.0651.021
11210.8900.8990.8860.9300.8830.8820.8831.1901.101...0.9161.0100.9740.9681.3051.1681.2650.9810.9750.972
11310.9200.9860.8830.8790.7940.9831.0291.0761.053...0.9420.9850.9901.1991.3531.1871.4440.9471.0471.085
11410.9700.8680.9400.9670.9301.0220.9571.1121.004...1.0741.1221.2150.8911.3331.2951.2841.1251.1101.139
\n", "

115 rows × 309 columns

\n", "
" ], "text/plain": [ " Group lh_bankssts_part1_thickness lh_bankssts_part2_thickness \\\n", "0 0 0.911 0.931 \n", "1 0 0.861 0.913 \n", "2 0 0.817 0.827 \n", "3 0 0.887 0.905 \n", "4 0 0.887 0.854 \n", ".. ... ... ... \n", "110 1 0.843 0.855 \n", "111 1 0.911 0.914 \n", "112 1 0.890 0.899 \n", "113 1 0.920 0.986 \n", "114 1 0.970 0.868 \n", "\n", " lh_caudalanteriorcingulate_part1_thickness \\\n", "0 0.891 \n", "1 0.846 \n", "2 0.828 \n", "3 0.878 \n", "4 0.905 \n", ".. ... \n", "110 0.940 \n", "111 0.926 \n", "112 0.886 \n", "113 0.883 \n", "114 0.940 \n", "\n", " lh_caudalmiddlefrontal_part1_thickness \\\n", "0 1.048 \n", "1 0.927 \n", "2 0.828 \n", "3 0.932 \n", "4 1.011 \n", ".. ... \n", "110 1.017 \n", "111 1.001 \n", "112 0.930 \n", "113 0.879 \n", "114 0.967 \n", "\n", " lh_caudalmiddlefrontal_part2_thickness \\\n", "0 0.881 \n", "1 0.888 \n", "2 0.780 \n", "3 0.820 \n", "4 0.946 \n", ".. ... \n", "110 0.954 \n", "111 0.918 \n", "112 0.883 \n", "113 0.794 \n", "114 0.930 \n", "\n", " lh_caudalmiddlefrontal_part3_thickness \\\n", "0 0.939 \n", "1 0.894 \n", "2 0.843 \n", "3 0.888 \n", "4 0.922 \n", ".. ... \n", "110 0.840 \n", "111 1.115 \n", "112 0.882 \n", "113 0.983 \n", "114 1.022 \n", "\n", " lh_caudalmiddlefrontal_part4_thickness lh_cuneus_part1_thickness \\\n", "0 1.124 0.986 \n", "1 0.924 1.040 \n", "2 0.825 0.848 \n", "3 0.970 0.918 \n", "4 1.034 1.126 \n", ".. ... ... \n", "110 1.128 1.012 \n", "111 1.036 1.026 \n", "112 0.883 1.190 \n", "113 1.029 1.076 \n", "114 0.957 1.112 \n", "\n", " lh_cuneus_part2_thickness ... rh_supramarginal_part5_thickness \\\n", "0 1.045 ... 0.928 \n", "1 1.093 ... 0.878 \n", "2 0.838 ... 0.847 \n", "3 0.900 ... 0.957 \n", "4 1.114 ... 0.871 \n", ".. ... ... ... \n", "110 0.997 ... 0.938 \n", "111 1.001 ... 0.957 \n", "112 1.101 ... 0.916 \n", "113 1.053 ... 0.942 \n", "114 1.004 ... 1.074 \n", "\n", " rh_supramarginal_part6_thickness rh_supramarginal_part7_thickness \\\n", "0 1.067 1.096 \n", "1 0.985 1.045 \n", "2 0.849 0.819 \n", "3 0.985 0.989 \n", "4 0.952 0.987 \n", ".. ... ... \n", "110 1.062 1.143 \n", "111 1.085 1.098 \n", "112 1.010 0.974 \n", "113 0.985 0.990 \n", "114 1.122 1.215 \n", "\n", " rh_frontalpole_part1_thickness rh_temporalpole_part1_thickness \\\n", "0 0.892 1.238 \n", "1 1.001 1.196 \n", "2 0.952 0.933 \n", "3 1.075 1.150 \n", "4 1.325 0.996 \n", ".. ... ... \n", "110 0.903 1.364 \n", "111 1.059 1.268 \n", "112 0.968 1.305 \n", "113 1.199 1.353 \n", "114 0.891 1.333 \n", "\n", " rh_transversetemporal_part1_thickness rh_insula_part1_thickness \\\n", "0 1.021 1.166 \n", "1 1.083 1.143 \n", "2 0.942 1.059 \n", "3 1.017 0.986 \n", "4 1.094 1.064 \n", ".. ... ... \n", "110 1.284 1.218 \n", "111 1.089 1.173 \n", "112 1.168 1.265 \n", "113 1.187 1.444 \n", "114 1.295 1.284 \n", "\n", " rh_insula_part2_thickness rh_insula_part3_thickness \\\n", "0 0.900 0.907 \n", "1 0.917 0.923 \n", "2 0.794 0.834 \n", "3 0.888 0.916 \n", "4 0.966 0.989 \n", ".. ... ... \n", "110 1.017 0.972 \n", "111 0.990 1.065 \n", "112 0.981 0.975 \n", "113 0.947 1.047 \n", "114 1.125 1.110 \n", "\n", " rh_insula_part4_thickness \n", "0 0.937 \n", "1 0.960 \n", "2 0.860 \n", "3 0.928 \n", "4 0.977 \n", ".. ... \n", "110 1.028 \n", "111 1.021 \n", "112 0.972 \n", "113 1.085 \n", "114 1.139 \n", "\n", "[115 rows x 309 columns]" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" } ], "source": [ "MD_Dublin_adj" ] }, { "cell_type": "markdown", "id": "a2dba246", "metadata": {}, "source": [ "Very important is, not to forget to convert the dataframe into an numpy array!" ] }, { "cell_type": "code", "execution_count": 54, "id": "aa974bc3", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0. , 0.911, 0.931, ..., 0.9 , 0.907, 0.937],\n", " [0. , 0.861, 0.913, ..., 0.917, 0.923, 0.96 ],\n", " [0. , 0.817, 0.827, ..., 0.794, 0.834, 0.86 ],\n", " ...,\n", " [1. , 0.89 , 0.899, ..., 0.981, 0.975, 0.972],\n", " [1. , 0.92 , 0.986, ..., 0.947, 1.047, 1.085],\n", " [1. , 0.97 , 0.868, ..., 1.125, 1.11 , 1.139]])" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#dataframe as numpy array \n", "\n", "MD_Dublin_adj.to_numpy()" ] }, { "cell_type": "markdown", "id": "9317379a", "metadata": {}, "source": [ "#### 2.1.2 Build the model" ] }, { "cell_type": "markdown", "id": "e57ed474", "metadata": {}, "source": [ "In the next step, the input and output for our model is defined. For that, we use the **MD** for each 308 cortical region as input and the label whether a participant belongs to the control or patient group as output." ] }, { "cell_type": "code", "execution_count": 55, "id": "0decd525", "metadata": {}, "outputs": [], "source": [ "#define input\n", "\n", "X_MD = MD_Dublin_adj.iloc[:,1:309].values" ] }, { "cell_type": "code", "execution_count": 56, "id": "e4fe91ae", "metadata": {}, "outputs": [], "source": [ "#output\n", "\n", "y_MD = MD_Dublin_adj.iloc[:,[0]].values" ] }, { "cell_type": "code", "execution_count": 57, "id": "e7089e2f", "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "(115, 1)" ] }, "execution_count": 57, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_MD.shape" ] }, { "cell_type": "markdown", "id": "079848d7", "metadata": {}, "source": [ "To return a 1D flattened array since its required for further analyses, the .ravel() function is used!" ] }, { "cell_type": "code", "execution_count": 58, "id": "d8c37219", "metadata": {}, "outputs": [], "source": [ "y_MD = y_MD.ravel()" ] }, { "cell_type": "code", "execution_count": 59, "id": "0ca72716", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(115,)" ] }, "execution_count": 59, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_MD.shape" ] }, { "cell_type": "code", "execution_count": 60, "id": "3a491660", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0.911, 0.931, 0.891, ..., 0.9 , 0.907, 0.937],\n", " [0.861, 0.913, 0.846, ..., 0.917, 0.923, 0.96 ],\n", " [0.817, 0.827, 0.828, ..., 0.794, 0.834, 0.86 ],\n", " ...,\n", " [0.89 , 0.899, 0.886, ..., 0.981, 0.975, 0.972],\n", " [0.92 , 0.986, 0.883, ..., 0.947, 1.047, 1.085],\n", " [0.97 , 0.868, 0.94 , ..., 1.125, 1.11 , 1.139]])" ] }, "execution_count": 60, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_MD" ] }, { "cell_type": "markdown", "id": "741bf0a4", "metadata": {}, "source": [ "Again, we build our model with 5000 iterations." ] }, { "cell_type": "code", "execution_count": 61, "id": "f2c9353a", "metadata": {}, "outputs": [], "source": [ "n_iter_MD = 5000\n", "y_preds_MD = []\n", "y_tests_MD = []\n", "\n", "# scale before splitting into test and train samples\n", "X_sc_MD = StandardScaler().fit_transform(X_MD)\n", "\n", "for i in range(n_iter):\n", " # take a new testing and training sample\n", " X_train_MD, X_test_MD, y_train_MD, y_test_MD = train_test_split(X_sc_MD, y_MD, test_size = 0.25, random_state = i)\n", " y_tests_MD.append(y_test_MD) # store the y_test sample\n", " \n", " # fit the logistic regression\n", " classifier_MD = LogisticRegression(random_state = i, solver ='liblinear')\n", " classifier_MD.fit(X_train_MD, y_train_MD)\n", " \n", " # get the y predictions and store\n", " y_pred_MD = classifier_MD.predict(X_test_MD)\n", " y_preds_MD.append(y_pred_MD)" ] }, { "cell_type": "markdown", "id": "3272350b", "metadata": {}, "source": [ "In the following steps, we again concatenate the values to compute the confusion matrix." ] }, { "cell_type": "code", "execution_count": 72, "id": "1d9afdd9", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1, 0, 0, ..., 1, 0, 1])" ] }, "execution_count": 72, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_preds_MD = np.concatenate([y_preds_MD])\n", "y_preds_MD" ] }, { "cell_type": "code", "execution_count": 93, "id": "71323df1", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0, 0, 0, ..., 0, 0, 1])" ] }, "execution_count": 93, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_tests_MD = np.concatenate([y_tests_MD])\n", "y_tests_MD" ] }, { "cell_type": "markdown", "id": "6ad84688", "metadata": {}, "source": [ "#### 2.1.3 Model evaluation" ] }, { "cell_type": "markdown", "id": "317b6dff", "metadata": {}, "source": [ "Next, we will again have a look at the confusion matrix and then compute the evaluation measures as for the **CT**. To get probabilities in the confusion matrix, we again scale the confusion matrix by the sum of the array." ] }, { "cell_type": "markdown", "id": "6f022c85", "metadata": {}, "source": [ "##### 2.1.3.1 Confusion matrix" ] }, { "cell_type": "code", "execution_count": 94, "id": "b92c0047", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Confusion Matrix : \n", " [[0.53733103 0.17482759]\n", " [0.07247586 0.21536552]]\n" ] } ], "source": [ "#confusion matrix\n", "\n", "cm_MD = confusion_matrix(y_tests_MD, y_preds_MD)\n", "\n", "cm_MD_f = cm_MD/np.sum(cm_MD)\n", " \n", "print (\"Confusion Matrix : \\n\", cm_MD_f)" ] }, { "cell_type": "markdown", "id": "1ba0fc55", "metadata": {}, "source": [ "To plot the confusion matrix visually more appealing again, the following code can be run." ] }, { "cell_type": "code", "execution_count": 95, "id": "6772de47", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Text(0.5, 257.44, 'Predicted label')" ] }, "execution_count": 95, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZ8AAAFBCAYAAACo1qLLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAsWklEQVR4nO3dd5gUVdbH8e+ZGUAkJwEJiogoghgQMbsoiguKruAiZkXMOSvG1dVdI68ZI2IAdUVBWBFRV1EUUFEERUGRDJKDCBPO+0fVDD3DMNM9TNdMt7+PTz123bp161ZNU6fuvdVV5u6IiIhEKaOiKyAiIn8+Cj4iIhI5BR8REYmcgo+IiEROwUdERCKn4CMiIpFT8EkhZlbdzEaZ2Woze30byjnVzN4rz7pVFDM71MxmlnHdtmY21czWmtll5V23VGFmT5rZLTHzF5rZEjNbZ2YNzOxgM/spnD9hG7bzXzM7M458Zf6bSgpxd03lPAH9gCnAOmAR8F/gkHIo93RgEpBV0fsY0XF0YNcklv8s8FA5lnd7WOfLi6RfHqbfXgHHcA6wAVgLrAI+Ay4AMraSv0qYv2NM2vii+1QB+3BURW1fU3ImtXzKmZldBTwM/BNoDLQEHgd6lUPxOwE/untOOZSV8swsaxuL2AmYXs7b/hE4o0jamWF6RTnO3WsR7O+9wPUEgbc4jYHtKHxcynycRLaqoqNfOk1AHYLWTp8S8lQjCE4Lw+lhoFq47AhgPnA1sJSg1XR2uOwOYBOQHW7jXIIr7Zdiyt6Z4Ao7K5w/C/iZ4Kr3F+DUmPQJMesdBEwGVof/Pyhm2UfAP4BPw3LeAxpuZd/y639dTP1PAP5KcPJdAdwUk78zMJHginwR8ChQNVz2cbgv68P9/XtM+dcDi4Gh+WnhOq3Dbewbzu8I/AYcUUxdPwBygT/C8ncL/34vhuv8CgwkbCGEx+xT4CFgOXBXMWXeDrwEfA/sGabtCcwI02+PydsTmMrm1sheMctuAGaHx3sGcGLMsrOACcD9wMrw73psCd+3ORRpNYTHPQ9oH86/ANwVHoP14XFfFx6j2WHeDWFataJlEvM9JAhcL4XHaBXB96lxzHepf1jGqvzth8sahdvYocjfdGiR7V8HjAYuLbJP38YeJ02Vf1LLp3wdSPCPb0QJeW4GugB7Ax0JTgQDY5Y3ITgJNiMIMI+ZWT13v42gNTXc3Wu6+9auXAEwsxrA/xGcmGoRBJipxeSrT/CP+f+ABsCDwGgzaxCTrR9wNsGJoSpwTQmbbkJwDJoBtwJPA6cB+wGHAreYWaswby5wJdCQ4NgdCVwE4O6HhXk6hvs7PKb8+gRX4wNiN+zuswkC00tmtj3wPDDE3T8qWkl37wp8AlwSlv8j8AjBsd8FOJygBXN2zGoHEATzxsDdJRyDoWxu/ZwZzhcws32A54DzCY75U8BIM6sWZpkdHqs6BBcdL5lZ0yL1mElw3P4NPGtmVkJ9iu77JIIgfmiR9B8JgiVAXXfv6u6tgbkEraea7r6xlOLPDOvdIty3CwgCR+x2NgJvAqfEJJ8M/M/dlxbJe3qR7f8bGELwnQLAzDoSfN9Gl7bvUnko+JSvBsAyL7lb7FTgTndf6u6/EZxcTo9Znh0uz3b3MQRXe23LWJ88oL2ZVXf3Re5eXNdJD+Andx/q7jnu/irwA3BcTJ7n3f1Hd98AvEYQOLcmG7jb3bOBYQQnyEHuvjbc/gyCoIu7f+nun4fbnUNwEj48jn26zd03hvUpxN2fBmYBXwBNCYJ9qcwsE+gL3BjWdQ7wAIX/Ngvd/ZGwvltsO8ZLwClmViUs86UiywcAT7n7F+6e6+5DgI0EFyW4++vuvtDd88Kg+xPBRUq+X939aXfPJTgRNyUIiIlYSBDEy1s2wb+DXcN9+9Ld1xST7xWCY5OvX5gWj5HAbmbWJpw/neCibFNZKy3RU/ApX8uBhqWMRexI0KWT79cwraCMIsHrd6BmohVx9/UEXVUXAIvMbLSZ7R5HffLr1CxmfnEC9VkenhRh8xXvkpjlG/LXN7PdzOwdM1tsZmsIWnYNSygb4Dd3/6OUPE8D7YFH4rhSz9eQYLC96N8m9jjMi6cgd59LEAD/SRDYi663E3C1ma3KnwhaCjsCmNkZ4V14+cvaU/i4FPw93P338GOi35FmBF2U5W0oMBYYZmYLzezfYRAu6kNgezM7wMx2JrigKanHoED49x8OnGZmGQQtqKElryWVjYJP+ZpIcAV7Qgl5FhKcfPK1DNPKYj2wfcx8k9iF7j7W3bsRXBn/QHBSLq0++XVaUMY6JeIJgnq1cffawE1Aad1HJT6G3cxqEoyjPQvcHnYrxmMZwVV70b9N7HFI5BHwLxKM3b1YzLJ5BK3DujHT9u7+qpntRPB3ugRo4O51ge8o/bjEzcz2Jwg+E8pYxFa/d2GL/Q53b0fQ1duTLW/AILxAeY0gcJwCvOPua7eyveKO+xCCXoQjgd/dfWJZdkQqjoJPOXL31QTjHI+Z2Qlmtr2ZVTGzY83s32G2V4GBZtbIzBqG+Yt2y8RrKnCYmbU0szrAjfkLzKyxmfUKx342EnTf5RVTxhiCLox+ZpZlZn8H2gHvlLFOiagFrAHWha2yC4ssX0Iw/pKIQcAUd+9PMAbwZDwrxZwM7zazWmEQuIqy/22GA0eHZRb1NHBBeNVvZlbDzHqYWS2gBsHJ9jcAMzuboOWzzcystpn1JOgOfcndp5WxqKlA3/C73QnoHbONv5hZh7Abcw1BQC/uewdBN9vfCYJISV1uW3wPwmCTR9A1qlZPClLwKWfu/gDBSWsgwQlkHsFV7FthlrsIfgP0LTAN+CpMK8u2xhGc5L4FvqRwwMgI67GQoHvlcLY8uePuywmuTq8m6Da8Dujp7svKUqcEXUPQ17+W4IQ8vMjy24EhYffTyaUVZma9gO5s3s+rgH3N7NQ463MpwVX9zwStglcIbgxImLtvcPf3tzIuNQU4j+DuvpUEXXRnhctmEJxQJxKcdDsQ3GW3LUaZ2VqC7+LNBDeVnF3yKiW6heDOwpUEY5axgaMJ8AZB4Pke+B9bCQ7u/gXB8d6R4LdwW3MPwQXbKjOLvdnlRYLjU9YLBKlA5q6XyYlI6jGzM4AB7n5IRddFEqeWj4iknPBW+ouAwRVdFykbBR8RSSlmdgxBl/YS4r89WyoZdbuJiEjk1PIREZHIKfiIiEjkFHxERCRyCj4iIhI5BR8REYmcgo+IiEROwUdERCKn4CMiIpFT8BERkcgp+IiISOQUfEREJHIKPiIiEjkFHxERiZyCj4iIRE7BR0REIqfgIxXGzHLNbKqZfWdmr4dvpyxrWS+YWe/w8zNm1q6EvEeY2UFl2MYcM2sYb3qRPOsS3NbtZnZNonUUSRUKPlKRNrj73u7eHtgEXBC70MyyylKou/d39xklZDkCSDj4iEj5UfCRyuITYNewVfKJmY0EZphZppndZ2aTzexbMzsfwAKPmtlMM3sf2CG/IDP7yMw6hZ+7m9lXZvaNmY03s50JgtyVYavrUDNrZGb/Cbcx2cwODtdtYGbvmdl0M3sGsNJ2wszeMrMvw3UGFFn2UJg+3swahWmtzezdcJ1PzGz3cjmaIpVcma4sRcpT2MI5Fng3TNoXaO/uv4Qn8NXuvr+ZVQM+NbP3gH2AtkA7oDEwA3iuSLmNgKeBw8Ky6rv7CjN7Eljn7veH+V4BHnL3CWbWEhgL7AHcBkxw9zvNrAdwbhy7c064jerAZDP7j7svB2oAU9z9SjO7NSz7EmAwcIG7/2RmBwCPA13LcBhFUoqCj1Sk6mY2Nfz8CfAsQXfYJHf/JUw/GtgrfzwHqAO0AQ4DXnX3XGChmX1QTPldgI/zy3L3FVupx1FAO7OChk1tM6sZbuNv4bqjzWxlHPt0mZmdGH5uEdZ1OZAHDA/TXwLeDLdxEPB6zLarxbENkZSn4CMVaYO77x2bEJ6E18cmAZe6+9gi+f5ajvXIALq4+x/F1CVuZnYEQSA70N1/N7OPgO22kt3D7a4qegxE/gw05iOV3VjgQjOrAmBmu5lZDeBj4O/hmFBT4C/FrPs5cJiZtQrXrR+mrwVqxeR7D7g0f8bM9g4/fgz0C9OOBeqVUtc6wMow8OxO0PLKlwHkt976EXTnrQF+MbM+4TbMzDqWsg2RtKDgI5XdMwTjOV+Z2XfAUwQt9hHAT+GyF4GJRVd099+AAQRdXN+wudtrFHBi/g0HwGVAp/CGhhlsvuvuDoLgNZ2g+21uKXV9F8gys++BewmCX771QOdwH7oCd4bppwLnhvWbDvSK45iIpDxz94qug4iI/Mmo5SMiIpFT8BERkchV2rvdqrc8Rf2BErnPp55a0VWQP5mO9XsmdltlKRI9d26Y+2q5bj9elTb4iIhI4sxSo0NLwUdEJI1YioymKPiIiKQRtXxERCRyCj4iIhK5RB8LVVEUfERE0opaPiIiEjF1u4mISOQUfEREJHK61VpERCKnlo+IiEROwUdERCKn4CMiIpEz9DsfERGJmFo+IiISOQUfERGJXKoEn9SopYiIxCkjwal0ZtbdzGaa2Swzu6GY5WeZ2W9mNjWc+pdWplo+IiJppLxbPmaWCTwGdAPmA5PNbKS7zyiSdbi7XxJvuQo+IiJpJAndbp2BWe7+c1C+DQN6AUWDT0LU7SYikkaMjMQmswFmNiVmGlCkyGbAvJj5+WFaUSeZ2bdm9oaZtSitnmr5iIikkURbPu4+GBi8jZsdBbzq7hvN7HxgCNC1pBXU8hERSSNmltAUhwVAbEumeZhWwN2Xu/vGcPYZYL/SClXwERFJI2YZCU1xmAy0MbNWZlYV6AuMLLxNaxozezzwfWmFqttNRCSNlPcrFdw9x8wuAcYCmcBz7j7dzO4Eprj7SOAyMzseyAFWAGeVVq6Cj4hIGknGj0zdfQwwpkjarTGfbwRuTKRMBR8RkTSSKk84UPAREUkjepOpiIhETy0fERGJmrrdREQkcnH+dqfCKfiIiKQRjfmIiEjk1O0mIiLRU7ebiIhELlPBR0REoqaWj4iIRC41hnwUfERE0omr5SMiIpFLjdij4CMiklYyUiP6KPiIiKQTdbuJiEjkUiP2KPiIiKQVdbuJiEjk1O0mIiKRS43Yo+AjIpJW1O0mIiKRS43Yo+AjIpJO9IQDERGJnrrdREQkcqkRexR8RETSirrdREQkcup2ExGRyKVG7FHwERFJK+p2ExGRyCn4iIhI5PQabRERiZxaPiIiErnUiD0KPhWp2+Eduf/2M8jMzOCFYR9y/+MjCy0/rfdh/PPmU1m4eAUATw55jxeGfUjLZg0ZNvgqMjKMKlWyeOKFsTzz0vvUrLEd779xW8H6zZo2YNiICVx7x4v0P+0ozj+jG7m5eaz//Q8uvuEZfvhpAZ06tubRe/sDYGbc/dAbjBw7Jdjefedz7JH78NvyNXTqdl1ER0WSaerEH3j+4bfIy83jyOMP4IQzjiy0fMbXsxny8Nv8OnsRV9x5Gl26dgTguy9nMWTQ2wX5Fv66lMvvPI3Oh3coSHvuwRF8+M4khn5wDwDLFq/ksX+8yvq1G8jLc/pd1IN9D9qDbyfN5OXHx5CTnUNWlSxOv6Qn7Tu1AWDCe18xYsh4zIx6DWtz6e39qF23ZrIPS1px3WotJcnIMB6+62x6nPpPFixazoRRd/POuC/54acFhfL9Z9RErrz1hUJpi5au5IgTb2XTphxqbF+NL8fdx+hxX7JoyUq6HHtjQb5PR9/NW/+dBMDwtz7lmZfeB6BHt/341y2n0+uMe5k+cx4H97yZ3Nw8muxQly/evZfR739Fbm4eQ1//H08OGcszD12U3IMhkcjLzePZB95k4KDzabBDHW4852E6HbonzVs1KcjTsEk9LrqlL6Ne/qjQuu3325X7XrwagHWrf+fSPv+k4wFtC5bP/n4e69duKLTOf154nwOP3Juj/3YQ839ZzD1XPcO+IwZSq04Nrr/vHOo3qsPc2Yu4+4rBPDXqNnJzcnnh4bd58JVrqV23Ji89Oop33/iUk/sfk7yDko5SpNstRYam0s/+e+/K7DmLmTN3KdnZubw+aiI9j+4U17rZ2bls2pQDQLWqVcgo5kpn11ZN2KFBHT6d9AMAa9dtPjHUqF4Ndwdgwx+byM3NC8qqVoUwGYBPJ/3AilXryrR/UvnMmjGXJs0b0LhZA7KqZHHQUfsw+ePphfLs0LQ+O+26I1bC1fPnH37DPgfuTrXtqgJBUHvp0VGcdnHPQvkM+H39HwD8vu4P6jWsDUCrts2p36gOAC12acKmjdlkb8rBAXdn44ZNuDu/r99I/XAdSYAlOFWQpLV8zGx3oBfQLExaAIx09++Ttc1UsmOTesxfuLxgfsGi5XTee9ct8vX6a2cOPmAPZv2yiOvueJH5i4IuuOZN6/PmC9fTeufG3HT3yyxasrLQen2OP4g3Rk0slHb+Gd247LweVK2SRfe+dxWk7793a568/wJaNmvIuVc8VhCMJL2s+G01DXaoWzDfYIc6/DR9bsLlfPr+VHr2Pbxg/t03JrDfIXsWBJd8ffofw12XP8W7r09g4x+buOX/zt+irC8+/JZd2janStXgVHTetSdxzWn3U616VZq2aET/a/6WcP3+9FKk2y0pLR8zux4YRhBXJ4WTAa+a2Q3J2GY6GvP+V+x+0GV0PuZ6xn8yjacf3Nz9NX/RCjofcz3tD7uS03ofxg4N6xRat8/xB/LayM8KpT314jj2PPQKBt7zCjdcdmJB+uSps9nvqGs55LibufbiXlSrViW5OyYpa+WyNcydvYiOXYIutxW/rWbiB99wbJ9Dtsj76bivOaLH/jw58lZufKA/j9zxKnl5my9s5v28mJcfH8151/cGICcnl/fe/Ix/DbmKp0bdRsvWTRnx4vhodiydmCU2VZBkdbudC+zv7ve6+0vhdC/QOVxWLDMbYGZTzGxKzrpZSapa5bBw8Uqa79igYL5Z0wYsKNJ6WbFqXUH32vOvfsA+HVptUc6iJSuZPnM+B3fe3P/eYY+WZGVm8vW0X4rd9msjJ3JcMV18M2ctZN36jezZtkWZ9kkqt/qN6rB86aqC+eVLVxd0f8Vr4vipdD68A1lZmQDM+XEBi+cv57I+93DxiXex6Y9sLu39TwA+GPUFBx4Z3LCwW4edyd6UzdpV68Ntr+L+G57n4ltOoUnzhgVlATRp3hAz48AjO/LjtDnbsst/TinS7Zas4JMH7FhMetNwWbHcfbC7d3L3Tlk1t+yCSidTvpnNrq2asFOLRlSpkkmf4w5k9LgvC+VpEtNF0rPbfsycFfzjbNakPtuFrZO6dWpw0P5t+XH2ooK8J/c6aItWT+udNw8qH3vkPsyasxiAnVo0IjMz+Bq0bNaQtrvuyK/zfiu/HZVKo/UeLVg0bxlLFy4nJzuHz97/mk6H7plQGZ+O+5qDu+1TML/vwe14evTtPDZiII+NGEjV7arwyBs3AdCwcT2+m/ITAPPnLCF7Uw6169Vk/doN3Hv1M/S7qAe7d9x8QVW/UR3mz1nCmpXBOOO3k36k2c6Nt3W3/3wyLLGpgiRrzOcKYLyZ/QTMC9NaArsClyRpmyklNzePK295gVFDbyQzM4Mhwz/i+x/nc8tVvflq2i+MHvclF53dnR7d9iMnJ5eVq9Zx3tVPAtC2TTPuHXga7o6Z8fDgd5g+c15B2Sf17MIJZ/670PYuPOto/nJIB7Kzc1i1ej3nXfUEAAft35ZrLupFdnYOeXnO5Tc/x/KVawEY8silHHrgHjSsV4tZXzzKPx58gyHDP4rmAEm5y8zK5Jyr/8bdVwwmL8/5S8/OtNilCcMHv0vrPZrT6dD2zJoxl/tveIH1azfw5YQZvPbMWB58JbjNfumiFSxbsop2++wS1/bOuOw4nrrndUYP+xjMuGhgX8yMd9+YwOL5y3njuXG88dw4AAY+PID6jerQ+5yjue3Cx8jMyqRhk3pcfEvfpB2PtJUiYz7msbc3lWfBZhkE3WyxNxxMdvfceNav3vKU5FRMpASfTz21oqsgfzId6/cs12ixS//XEzp3/vxMnwqJVkm7283d84DPk1W+iIgUI0VaPvqRqYhIOkmRH5kq+IiIpBO1fEREJHIp8tyaFKmmiIjEJQk/MjWz7mY208xmlfSgADM7yczczEp9VphaPiIiacQzy7dNYWaZwGNAN2A+MNnMRrr7jCL5agGXA1/EU65aPiIi6SQjwal0nYFZ7v6zu28ieHRar2Ly/QP4F/BHvNUUEZF0keATDmIfaxZOA4qU2IzNDwuAoPXTLDaDme0LtHD30fFWU91uIiLpJMFbrd19MDC47JuzDOBB4KxE1lPwERFJJ+V/q/UCIPZpw83DtHy1gPbARxYEvibASDM73t2nbK1QBR8RkXRS/j/zmQy0MbNWBEGnL9Avf6G7rwYaFmze7CPgmpICDyj4iIikFS/nlo+755jZJcBYIBN4zt2nm9mdwBR3H1mWchV8RETSSRKecODuY4AxRdJu3UreI+IpU8FHRCSd6NluIiISuRT5AY2Cj4hIOlHLR0REIqenWouISOQUfEREJGqubjcREYmcbjgQEZHIqeUjIiKR05iPiIhETsFHREQilxqxR8FHRCSdlPeDRZNFwUdEJJ3ohgMREYmcWj4iIhK51Ig9Cj4iIukkQz8yFRGRqKXIkI+Cj4hIOkn54GNmawHPnw3/7+Fnd/faSa6biIgkyFIk+mw1+Lh7rSgrIiIi2y5FYk98zz81s0PM7Ozwc0Mza5XcaomISFmYJTZVlFLHfMzsNqAT0BZ4HqgKvAQcnNyqiYhIoiyN7nY7EdgH+ArA3ReambrkREQqoVTpdosn+GxydzczBzCzGkmuk4iIlFGKPOAgrjGf18zsKaCumZ0HvA88ndxqiYhIWaTNmI+7329m3YA1wG7Are4+Luk1ExGRhKVTtxvANKA6we98piWvOiIisi1S5Xc+pXa7mVl/YBLwN6A38LmZnZPsiomISOIsI7GposTT8rkW2MfdlwOYWQPgM+C5ZFZMREQSlyINn7iCz3Jgbcz82jBNREQqmZQPPmZ2VfhxFvCFmb1NMObTC/g2grqJiEiCMtPgR6b5PySdHU753k5edUREZFukfMvH3e+IsiIiIrLtUj745DOzRsB1wJ7Advnp7t41ifUSEZEysBR5xEE8vYMvAz8ArYA7gDnA5CTWSUREyihVnnAQT/Bp4O7PAtnu/j93PwdQq0dEpBJKleATz63W2eH/F5lZD2AhUD95VRIRkbJKmzEf4C4zqwNcDTwC1AauTGqtRESkTFJkyCeuB4u+E35cDfwludUREZFtkfItHzN7hOBHpcVy98uSUiMRESmzdHiT6ZTIaiEiIuUi5Vs+7j4kyoqIiMi2S5VXKsT7Ph8REUkBKRJ7FHxERNJJqgSfFBmaEhGReCTjR6Zm1t3MZprZLDO7oZjlF5jZNDObamYTzKxdaWVW2rvdNszVc00lesNm/1zRVZA/mY7l/JP98v6dj5llAo8B3YD5wGQzG+nuM2KyveLuT4b5jwceBLqXVK7udhMRSSNJ+JFpZ2CWu/8MYGbDCN7rVhB83H1NTP4alNBwyae73URE0kiGlXreT1QzYF7M/HzggKKZzOxi4CqgKnE8/zPeVypcD7RDr1QQEanUEm35mNkAYEBM0mB3H5zodt39MeAxM+sHDATOLCl/PHe7vQwMB3oAF4QF/pZoxUREJPkSvYssDDQlBZsFQIuY+eZh2tYMA54obbt6pYKISBrJME9oisNkoI2ZtTKzqkBfYGRsBjNrEzPbA/iptEL1SgURkTRS3jccuHuOmV0CjAUygefcfbqZ3QlMcfeRwCVmdhRBvFhJKV1uoFcqiIiklWT8eNPdxwBjiqTdGvP58kTL1CsVRETSSNq8z8fMnqeYe7bDsR8REalErPxvtU6KeLrd3on5vB1wIsG4j4iIVDJp0/Jx9//EzpvZq8CEpNVIRETKLFUe2FmWp1q3AXYo74qIiMi2S8ITDpIinjGftRQe81lM8MQDERGpZNKp261WFBUREZFtlyrdbqXW08zGx5MmIiIVL8MSmypKSe/z2Q7YHmhoZvWA/GrWJnjKqYiIVDLpMOZzPnAFsCPwJZuDzxrg0eRWS0REyiLlx3zcfRAwyMwudfdHIqyTiIiUUdqM+QB5ZlY3f8bM6pnZRcmrkoiIlFUSnmqdnHrGkec8d1+VP+PuK4HzklYjEREps5S/4SBGppmZuzuAmWUSvCZVREQqmZQf84nxLjDczJ4K588P00REpJJJlTGfeILP9QTv974wnB8HPJ20GomISJllZaTGrdalBkl3z3P3J929t7v3BmYQvFROREQqmYwEp4oS14NFzWwf4BTgZOAX4M1kVkpERMom5cd8zGw3goBzCrAMGA6Yu+ttpiIilVQ6vEzuB+AToKe7zwIwsysjqZWIiJRJqrR8Sury+xuwCPjQzJ42syPZ/IgdERGphFJlzGer23b3t9y9L7A78CHBc952MLMnzOzoiOonIiIJSJsnHLj7end/xd2PA5oDX6OXyYmIVErp9ISDAuGjdQaHk4iIVDKpMuaTUPAREZHKLbOiKxAnBR8RkTSSDi+TExGRFKNuNxERiZyCj4iIRC5TwUdERKKmlo+IiERONxyIiEjk1PIREZHI6Xc+IiISObV8REQkchrzERGRyOlWaxERiZy63UREJHIKPiIiEjkFHxERiVymbjgQEZGolfp66kpCwUdEJI2o201ERCKn4CMiIpHTmI+IiEQuVVo+qTI2JSIicciwxKZ4mFl3M5tpZrPM7IZill9lZjPM7FszG29mO5Vaz8R3TUREKqvyDj5mlgk8BhwLtANOMbN2RbJ9DXRy972AN4B/l1rPRHdMREQqr0xLbIpDZ2CWu//s7puAYUCv2Azu/qG7/x7Ofg40L61QBR8RkTSSYZ7QZGYDzGxKzDSgSJHNgHkx8/PDtK05F/hvafXUDQciImkk0RaFuw8GBpfHts3sNKATcHhpeRV8Ivbxx19y991Pk5eXR58+3RgwoE+h5Zs2ZXPddQ8yffps6tatxUMPXUfz5o0ZOfIjnn32zYJ8M2fOYcSIh9l55x25/PJ/MXfuIjIzM/jLXzpzzTVnFSpz7NhPueyye3njjQfp0KHNVsvaY49dOP30G1m6dCXbbVcVgOeeu5MGDeqyYMFSbrppECtWrKFu3Zrcd9/VNGnSMHkHSpLipynf89+n3sTz8tj3mC4cenK3Qss/e/NDvho7kYzMDLavU5MTruhH3cb1ARh6yxPM/+FXWrZrxal3nF+wzogHX2bOtFlsV6M6ACdc2Y+mrZvzw8RpfDB0NJaRQUZGBt3PP5Gd9mwNwKqlKxg5aBirl63CgFPvPJ96jRvg7ox/cTQzPpmKZWaw/18PpkuvUs9jEiMJd7stAFrEzDcP0woxs6OAm4HD3X1jaYUq+EQoNzeXO+98kuef/weNGzegd++r6Nr1AHbdtWVBntdff4/atWsybtxgRo/+mPvvf4GHH76e448/guOPPwIIgsXFF9/NHnvswoYNf3DOOSfSpctebNqUzVlnDeR//5vC4Yd3AmDdut958cVRdOzYtmAbWysr3/33X02HDm0K1f1f/3qOE07oyoknHsnEid/wwANDuO++q5N0pCQZ8nLzGP3465xx90XUbliXwVc8QNsuHdihZZOCPE1bN2fAoGuoul1VJo2ewHvPjeTkG88C4OCTupK9MZspYz7douyjz+3FnofsXSit1d67cWGX9pgZi39ZwOv3vMClg28GYMQDL3PY37vRet/d2bhhI2bBGXPquC9Y89sqLhl8ExkZGaxbtTY5ByONJeF9PpOBNmbWiiDo9AX6xWYws32Ap4Du7r40nkI15hOhb7/9iZ12akqLFk2oWrUKPXocxvjxXxTK88EHX3DiiUcCcMwxBzNx4je4F/7R2OjRH9Ojx6EAVK++HV267AVA1apVaNeuNUuWLC/IO2jQy5x33klUq1al2DrFllWS2bPnFmynS5e9tqi3VH4LfvyV+js2on7ThmRVyaL9Yfvyw8RphfK06tiGqmGrt8XuO7Nm2aqCZbvs3Zaq1avFvb1q1asVBJXsPzZB+Hnp3MXk5ebSet/dC/Llb3PymE85vN8xZGQEp6aadWuVbWf/xBId8ymNu+cAlwBjge+B19x9upndaWbHh9nuA2oCr5vZVDMbWVq5avlEaMmS5YW6qho3bsC33/64RZ6mTYM8WVmZ1KpVg5Ur11C/fp2CPGPGfMLjjw/covw1a9bx4YeTOPPM4PswffosFi/+jSOO2L9QN1us4sq66aZBZGRkcPTRB3HRRX/HzNh991a8995EzjzzeMaNm8j69RtYuXIN9erVLtvBkMitWb6aOg3rFszXaViX+TN/3Wr+r8Z+TptOe8RV9vgho/nfK+/Sau/d6Hb28WRVCU4t33/2De+/8A7rV63j1DuCcezl85eyXY3qDLvrWVYuXs4u+7Sl21nHkZGZwYpFy/ju46/54bNv2b5OTf56wd9o0GyHsu/0n1BWEpoU7j4GGFMk7daYz0clWmbkLR8zO7uEZQV3XQwePDzKaqWMb76ZSfXq1dhtt8K/4crJyeWqq+7j9NOPo0WLJuTl5XHvvc9y/fXnJlTW/fdfw6hRj/Lyy/fy5ZfTefvtDwG47rpzmDz5O0444XImTfqOxo0bkJmphnO6+uaDySz8aS4H9z6y1LxHndWTSwffxIBB17Bh7e9MeP39gmV7HNSRSwffTN9bzuWDocG5Ky8vj1+n/8zR5/ZiwKCrWbloGV+/H7Skc7NzyKqaxfn/dw37dT+Qtx5+NTk7mMYyEpwqSkVs+46tLXD3we7eyd07DRjw9yjrFInGjRuwePGygvklS5bTuHGDLfIsWhTkycnJZe3a9YVaF0E32WFblH3LLY+y8847ctZZwe3369dv4Mcff+WMM26ia9dzmTp1JhdeeBfTpv1UYln59alZc3t69jy8oGXWuHEDHn30Jt56axBXXnk6ALVr1yzzsZDo1W5Qh9Ux3Wirl62iVoM6W+Sb/fVMPh4+jlNuO6+gBVOSWvXrYGZkVclin24HsKCY1tTOHXZl5eLlrF+9jtoN69Jkl2bUb9qQzMxM9jhwLxbNmh/UsWFd2h3UEYA9DtqLJb8sLOPe/nmZJTZVlKQEn/ARC8VN04DGydhmKujQoQ1z5ixk3rzFbNqUzejRH9O1a+dCebp2PYARI8YDwV1qXbrsVdBvnpeXx3//O2GLgPHQQ0NZt249N910XkFarVo1+OKLV/jgg2f54INn2XvvtjzxxMCCGwmKKysnJ5cVK1YDkJ2dw0cfTaZNm6BVtGLFavLy8gAYPPh1Tjop4Va2VLAdd2vJioW/sXLxcnKyc/ju46/YvUv7QnkWzZ7PqEeG0+/W/nGPt6wNvzPuzg8Tp7HDzk0BWL7wt4LxyoWz5pGTncP2tWvQrE1L/li/gfWr1wHw8zc/0ii86WH3Azvwy7fBBdKcabNo0KzRtu/4n4wlOFWUZI35NAaOAVYWSTfgsyRts9LLysrk1lsvoH//28jNzeOkk46iTZudGDToJdq3b8ORRx5A797duPbaB+nWbQB16tTkoYeuK1h/8uTpNG3aiBYtNt+dtHjxMp588jV22aU5J554BQCnndaDPn2OKbEuxZW1aVM2/fvfRnZ2Lnl5uRx44N6cfPLRAEya9B0PPjgEM6NTpz257bYLy/HISBQyMzP564UnMXTgE+Tl5bHP0V3YYaemfDB0DDu2acHuXTrw3rNvs+mPjbx2zwsA1GlUj363BRc1z147iGXzlrDpj008cPqt9LriFHbdbw/+8++hYSBxmuzSjJ6XBL0WMz79hm/GTyYzK5OsqlXoc8OZmBmWaRxzbi+G3Pgo7rBjmxbs1/1AAA7pcxT/uW8oE0d8RNXq1eh1+SkVcahSWkW2ZhJhRe+kKpdCzZ4Fnnf3CcUse8Xd+xWzWhE/psZzwSWtDJv9c0VXQf5k+rbuXq7h4qtloxM6d+7bsEeFhKuktHzcfauj3PEFHhERKQvT+3xERCRqKdLrpuAjIpJOUmXMR8FHRCSNpEjsUfAREUknqfIabQUfEZE0kiKxR8FHRCSdaMxHREQilyKxR8FHRCSdKPiIiEjkdMOBiIhELkVij4KPiEg60eN1REQkcmr5iIhI5HSrtYiIRC5VXm6v4CMikkbU8hERkcilSOxR8BERSSdq+YiISORSJPYo+IiIpBM94UBERCKXIrFHwUdEJJ3oCQciIhI5tXxERCRyuttNREQilyKxR8FHRCSd6PE6IiISOXW7iYhIBUiN6KPgIyKSRkzBR0REomaWGqM+Cj4iImlFLR8REYmYpcj9bgo+IiJpRN1uIiJSAdTtJiIiEdPdbiIiEjkFHxERqQAa8xERkYhZijxfR8FHRCStKPiIiEjEUmXMJzU6B0VEJE4ZCU6lM7PuZjbTzGaZ2Q3FLD/MzL4ysxwz6x1vLUVEJE1Ygv+VWp5ZJvAYcCzQDjjFzNoVyTYXOAt4Jd56qttNRCSNJOGGg87ALHf/OSx/GNALmJGfwd3nhMvy4i1ULR8RkbRiCU1mNsDMpsRMA4oU2AyYFzM/P0zbJmr5iIikkUQfLOrug4HByanN1in4iIiklXLvdlsAtIiZbx6mbRMFHxGRNJKEMZ/JQBsza0UQdPoC/ba1UI35iIiklcTGfErj7jnAJcBY4HvgNXefbmZ3mtnxAGa2v5nNB/oAT5nZ9NLKVctHRCSNJONlcu4+BhhTJO3WmM+TCbrj4qbgIyKSVlLjCQcKPiIiaSRVHq+j4CMikkb0VGsREakAqXEfmYKPiEgaUbebiIhUAAUfERGJmMZ8RESkAmjMR0REIpYqYz7m7hVdBylnZjYgfFKtSCT0nZNEpUb7TBJV9H0cIsmm75wkRMFHREQip+AjIiKRU/BJT+p7l6jpOycJ0Q0HIiISObV8REQkcgo+IiISOQWfNGJm3c1sppnNMrMbKro+kv7M7DkzW2pm31V0XSS1KPikCTPLBB4DjgXaAaeYWbuKrZX8CbwAdK/oSkjqUfBJH52BWe7+s7tvAoYBvSq4TpLm3P1jYEVF10NSj4JP+mgGzIuZnx+miYhUOgo+IiISOQWf9LEAaBEz3zxMExGpdBR80sdkoI2ZtTKzqkBfYGQF10lEpFgKPmnC3XOAS4CxwPfAa+4+vWJrJenOzF4FJgJtzWy+mZ1b0XWS1KDH64iISOTU8hERkcgp+IiISOQUfEREJHIKPiIiEjkFHxERiZyCj4iIRE7BR0REIvf/lROYtP5DIX4AAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "class_names=[0,1]\n", "\n", "fig, ax = plt.subplots() \n", "tick_marks = np.arange(len(class_names)) \n", "plt.xticks(tick_marks, class_names) \n", "plt.yticks(tick_marks, class_names) \n", "sns.heatmap(pd.DataFrame(cm_MD_f), annot=True, cmap=\"YlGnBu\" ,fmt='g') \n", "ax.xaxis.set_label_position(\"top\") \n", "plt.tight_layout() \n", "plt.title('Confusion matrix for Mean Diffusivity', y=1.1) \n", "plt.ylabel('Actual label') \n", "plt.xlabel('Predicted label')" ] }, { "cell_type": "markdown", "id": "73f431f5", "metadata": {}, "source": [ "The confusion matrix reveals that the probability for **hits** is around 22%, for **true negatives** around 54%. The probability for **misses** is around 7% and for **false positive** cases around 17%. " ] }, { "cell_type": "markdown", "id": "147a9322", "metadata": {}, "source": [ "##### 2.1.3.2 Model accuracy, precision, recall and F1-Score" ] }, { "cell_type": "code", "execution_count": 96, "id": "dc865b19", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Accuracy: 0.752696551724138\n", "Precision: 0.5519459860723249\n", "Recall: 0.7482090231688909\n", "F1-Score: 0.6352642018003356\n" ] } ], "source": [ "#compute accuracy, precision, recall\n", "\n", "print(\"Accuracy:\",metrics.accuracy_score(y_tests_MD, y_preds_MD)) \n", "\n", "print(\"Precision:\",metrics.precision_score(y_tests_MD, y_preds_MD)) \n", "\n", "print(\"Recall:\",metrics.recall_score(y_tests_MD, y_preds_MD)) \n", "\n", "print(\"F1-Score:\", metrics.f1_score(y_tests_MD, y_preds_MD))" ] }, { "cell_type": "markdown", "id": "ea7e839a", "metadata": {}, "source": [ "As the values reveal, our logistic regression model for **mean diffusivity** makes 75.26% of the time correct predictions. 55.20% represent the proportion of the model's prediction of psychosis where psychosis is actually present and 74.82% relate to the proportion of all cases of psychosis that the model accurately predicted. " ] }, { "cell_type": "markdown", "id": "76d6bc86", "metadata": {}, "source": [ "### 2.2 Fractional anisotropy" ] }, { "cell_type": "markdown", "id": "a84e891f", "metadata": {}, "source": [ "Now, the same procedure is applied for **FA**. " ] }, { "cell_type": "markdown", "id": "cd038dc5", "metadata": {}, "source": [ "#### 2.2.1 Data preparation" ] }, { "cell_type": "markdown", "id": "8d9b53c7", "metadata": {}, "source": [ "First, the data is adjusted accordingly. " ] }, { "cell_type": "code", "execution_count": 97, "id": "4ed0d86d", "metadata": {}, "outputs": [], "source": [ "#read data\n", "\n", "FA_Dublin_path = os.path.join(os.pardir, 'data', 'PARC_500.aparc_FA_cortexAv_mean_Dublin.csv')\n", "FA_Dublin = pd.read_csv(FA_Dublin_path)" ] }, { "cell_type": "code", "execution_count": 98, "id": "2245eded", "metadata": {}, "outputs": [], "source": [ "#adjust dataframe\n", "\n", "FA_Dublin_adj = FA_Dublin.drop(['Subject ID','Age', 'Sex'], axis=1)" ] }, { "cell_type": "code", "execution_count": 99, "id": "8e2354f2", "metadata": {}, "outputs": [], "source": [ "#label group 1 as 0 and 2 as 1\n", "\n", "FA_Dublin_adj['Group'] = FA_Dublin_adj['Group'].replace([1,2],[0, 1])" ] }, { "cell_type": "code", "execution_count": 100, "id": "206f4afb", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0. , 0.322, 0.147, ..., 0.157, 0.147, 0.137],\n", " [0. , 0.302, 0.155, ..., 0.152, 0.148, 0.152],\n", " [0. , 0.324, 0.18 , ..., 0.171, 0.174, 0.143],\n", " ...,\n", " [1. , 0.323, 0.173, ..., 0.181, 0.143, 0.151],\n", " [1. , 0.311, 0.174, ..., 0.162, 0.145, 0.123],\n", " [1. , 0.294, 0.164, ..., 0.145, 0.147, 0.127]])" ] }, "execution_count": 100, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#dataframe as numpy array \n", "\n", "FA_Dublin_adj.to_numpy()" ] }, { "cell_type": "markdown", "id": "714a804d", "metadata": {}, "source": [ "#### 2.2.2 Build the model" ] }, { "cell_type": "markdown", "id": "1bc4e3c0", "metadata": {}, "source": [ "Again, the input and output variables are defined. Here, we take the **FA** values for every brain region as input." ] }, { "cell_type": "code", "execution_count": 101, "id": "ffa88998", "metadata": {}, "outputs": [], "source": [ "#define input\n", "\n", "X_FA = FA_Dublin_adj.iloc[:,1:309].values" ] }, { "cell_type": "code", "execution_count": 102, "id": "4a6fdc05", "metadata": {}, "outputs": [], "source": [ "#output\n", "\n", "y_FA = FA_Dublin_adj.iloc[:,[0]].values" ] }, { "cell_type": "code", "execution_count": 103, "id": "f7a678ed", "metadata": {}, "outputs": [], "source": [ "y_FA = y_FA.ravel()" ] }, { "cell_type": "markdown", "id": "09e07afb", "metadata": {}, "source": [ "And finally run the model." ] }, { "cell_type": "code", "execution_count": 104, "id": "8417be69", "metadata": {}, "outputs": [], "source": [ "n_iter_FA = 5000\n", "y_preds_FA = []\n", "y_tests_FA = []\n", "\n", "# scale before splitting into test and train samples\n", "X_sc_FA = StandardScaler().fit_transform(X_FA)\n", "\n", "for i in range(n_iter):\n", " # take a new testing and training sample\n", " X_train_FA, X_test_FA, y_train_FA, y_test_FA = train_test_split(X_sc_FA, y_FA, test_size = 0.25, random_state = i)\n", " y_tests_FA.append(y_test_FA) # store the y_test sample\n", " \n", " # fit the logistic regression\n", " classifier_FA = LogisticRegression(random_state = i, solver ='liblinear')\n", " classifier_FA.fit(X_train_FA, y_train_FA)\n", " \n", " # get the y predictions and store\n", " y_pred_FA = classifier_FA.predict(X_test_FA)\n", " y_preds_FA.append(y_pred_FA)" ] }, { "cell_type": "markdown", "id": "7f35473b", "metadata": {}, "source": [ "Concatenate the values from each iteration to compute our confusion matrix!" ] }, { "cell_type": "code", "execution_count": 112, "id": "4db6e2b9", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1, 0, 0, ..., 1, 0, 1])" ] }, "execution_count": 112, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_preds_FA = np.concatenate([y_preds_FA])\n", "y_preds_FA" ] }, { "cell_type": "code", "execution_count": 114, "id": "04ffbc41", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0, 0, 0, ..., 0, 0, 1])" ] }, "execution_count": 114, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_tests_FA = np.concatenate([y_tests_FA])\n", "y_tests_FA" ] }, { "cell_type": "markdown", "id": "341d0dfd", "metadata": {}, "source": [ "#### 2.2.3 Model evaluation" ] }, { "cell_type": "markdown", "id": "5afe1404", "metadata": {}, "source": [ "Again, to get the probabilites we scale the confusion matrix with sum of its array. " ] }, { "cell_type": "markdown", "id": "dced73e0", "metadata": {}, "source": [ "##### 2.2.3.1 Confusion matrix" ] }, { "cell_type": "code", "execution_count": 117, "id": "9fc70ea9", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Confusion Matrix : \n", " [[0.44015862 0.272 ]\n", " [0.05826897 0.22957241]]\n" ] } ], "source": [ "#confusion matrix\n", "\n", "cm_FA = confusion_matrix(y_tests_FA, y_preds_FA)\n", "\n", "cm_FA_f = cm_FA / np.sum(cm_FA)\n", " \n", " \n", "print (\"Confusion Matrix : \\n\", cm_FA_f)" ] }, { "cell_type": "markdown", "id": "57452435", "metadata": {}, "source": [ "To have a nicer plot, again we run the following code!" ] }, { "cell_type": "code", "execution_count": 118, "id": "6a037656", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Text(0.5, 257.44, 'Predicted label')" ] }, "execution_count": 118, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaUAAAFBCAYAAADXB7A6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAup0lEQVR4nO3dd5wU9f3H8dfn7ui9Hr0oCCJGNAgxNkIsqFE0YmKNGpUYxUSw56eo2DWxxBAFFTVRFFsUFUUsWFEPFRQQlCYdpPdyd5/fHzN37B1Xdo9bbnZ9P3nMg53vzHy/353dm89+y86auyMiIhIFGVVdARERkQIKSiIiEhkKSiIiEhkKSiIiEhkKSiIiEhkKSiIiEhkKSnuYmdUys1fNbJ2ZPb8b+ZxlZm9VZt2qipkdbmazKnhsFzObYmYbzOwvlV23PcXMpptZnySX0cHM3MyykpT/RjPbKxl5y0+HglIpzOxMM5sc/qEtNbM3zOywSsh6AJANNHH30yqaibs/7e7HVEJ9kiq8CHYqax93/9Ddu1SwiKuB99y9nrv/s4J5FDKzm8xsR/i6FyxX726+xcp4wsxujU1z9/3cfWJlllNRZjbRzNaYWY1EjnP3uu4+dzfLLvf9IulNQakEZjYEuB+4nSCAtAP+DfSvhOzbA9+5e24l5JXyKuFTe3tgeiWXPSa8wBYsd5dwbGZFyow6M+sAHA44cFLV1mZXyWrlSYS4u5aYBWgAbAROK2OfGgRBa0m43A/UCLf1ARYBVwArgKXA+eG2m4HtwI6wjAuAm4CnYvLuQHBByArXzwPmAhuAecBZMekfxRz3SyAHWBf+/8uYbROBW4CPw3zeApqW8twK6n91TP1PBo4HvgNWA3+L2b8XMAlYG+77L6B6uO2D8LlsCp/v72PyvwZYBvy3IC08Zu+wjIPC9VbAj0CfEur6LpAHbA3z3yd8/f4THvMDcD2QEXPOPgbuA1YBt5aQZ5HXIyb9CeAhYFz4fI4CTgC+AtYDC4Gbih1zGPBJeG4WhuUPDF//7WGdXw33nQ8ctTvvr3B7qXWi2HurlNd/aHiO7gVeK+EcDAdeJ3gffQbsHbPdgU7h4+OBGeF+i4ErY/a7CJgdvs5jgVYJvl/iOT9/A1aG57Xgb+ZgYDmQGVOX3wJTq/q6oyXmfVbVFYjaAvQDcsv5wx0GfAo0B5qFF55bwm19wuOHAdXCP87NQKNw+00UDULF1wsvHECd8OLSJdzWEtgvfHweYVACGgNrgHPC484I15uE2ycCcwgu2rXC9TtLeW4F9R8a1v8iggv8aKAesB+wBegY7v9z4BdhuR2Ab4HLY/IrvFAVy/+u8OJSi5igFO5zEcEFrTYwHvh7Ga/FRODCmPX/AK+Ede1AEEgviDlnucBlYX1rlZBfkdcjJv0JgoB/KEEPQ82w3vuH6z8juOCdHO7fnuCCfEZ4HpsAPWLyurVY/vPZGZR25/1VVp06UH5Qmg1cEr6uO4DsYudgFcEHkSzgaeDZkl5rgmB5ePi4ETs/ZPQlCBYHha//g8AHCb5f4jk/94b7H0kQ5Ar+hmYAx8Xk/z/giqq+7miJeQ9WdQWitgBnAcvK2WcOcHzM+rHA/PBxH4KLdlbM9hXAL8LHN5FYUFoLnEqxCyhFg9I5wOfFtk8CzgsfTwSuj9l2CfBmKc+toP6Z4Xq9sD69Y/b5gvBCV8LxlwP/i1kv6SKzHahZLG1RsXzGAt8AXxN+Ci6lvImEQQnIDPPuFrP9T8DEmHO2oJzX9qYwj7UxSyuCC/J/yjn2fuC+8PF1seeh2H5PUHZQqvD7q5w6Fb63Stn3MIJA1DRcnwkMLlbvR2PWjwdmlvRaAwvCc1+/WBmPAXfHrNcNy+yQwPulvPOTC9SJ2f4ccEP4+Brg6fBxY4KA3rKs11XLnl00prSrVUDTcvquWxF0DRX4IUwrzMOLjhltJvjjS4i7byLowrgYWGpmr5tZ1zjqU1Cn1jHryxKozyp3zwsfbwn/Xx6zfUvB8Wa2j5m9ZmbLzGw9wThc0zLyBvjR3beWs88jQHfgQXffVs6+BZoStB6Kvzax52FhHPk85+4NY5YlJR1rZr3N7D0z+9HM1hG8TgXPvS3BxbMiKvz+KqdO5TkXeMvdV4bro8O0WPG+j04lCFo/mNn7ZnZImF7kubn7RoK/uda7ZlGo+PulvPOzJvzbKWn7U8CJZlYH+B3wobsvLaNs2cMUlHY1CdhGMI5SmiUE3TMF2oVpFbGJoJuqQIvYje4+3t2PJui6m0lwsS6vPgV1WlzBOiXiIYJ6dXb3+gR9+VbOMV7WRjOrS/AJ/zHgJjNrHGddVhJ86i7+2sSehzLLLkfxY0cTtOjaunsD4GF2PveFBONj8eRT3O68v8qqU6nMrBbBRfrI8APGMmAwcICZHRBn2YXcPcfd+xN0sb1M0FqBYs8tDA5NKPu9Wvx8lXd+GoX57rLd3RcT/I3/lqCH4b/xPSPZUxSUinH3dQTjKcPN7GQzq21m1czsODMrmIX1DHC9mTUzs6bh/k9VsMgpwBFm1s7MGhB0+wBgZtlm1j/8A9tGMPibX0Ie44B9wmnsWWb2e6Ab8FoF65SIegTjXhvDVtyfi21fDiT63ZUHgMnufiHBoPrD8RwUtu6eA24zs3pm1h4YQsVfm/LUA1a7+1Yz6wWcGbPtaeAoM/td+Jo0MbMe4bbyzsnuvL/KqlNZTiaYNNIN6BEu+wIfAn+IMw8AzKx6+D26Bu6+g+D9UfC+fQY438x6hFPObwc+c/f54fZ43i/xnJ+bw3ocDvwGiP1O4H8IJvLsD7yUyHOT5FNQKoG7/4PgYnY9wSD/QmAQwSc+gFuByQTjHd8AX4ZpFSlrAjAmzOsLigaSjLAeSwhmKh3Jrhd93H0VwR/eFQRdIVcDv4nphkmmKwkufBsIWnFjim2/CXjSzNaa2e/Ky8zM+hNMNil4nkOAg8zsrDjrcxlB63Mu8BFBy2FUnMcm6hJgmJltILgwFrQGcPcFBN1XVxC8dlOAghbHY0C38Jy8XEK+u/P+KrVO5TgXeNzdF7j7soKFYDblWRWYin0OMD/s0r2YYKwWd38buAF4kWAyxN7A6THH3UT575fyzs8ygok+Swg+HFzs7jNjtv+PoKX1P3ffnODzkiQz993pzRARiY7wrhhPuXubcvabA/wpDJISIWopichPipmdSjBO9W5V10V2pW9Hi8hPhplNJBg3O8fdSxqflSqm7jsREYkMdd+JiEhkKCiJiEhkKCiJiEhkKCiJiEhkKCiJiEhkKCiJiEhkKCiJiEhkKCiJiEhkKCiJiEhkKCiJiEhkKCiJiEhkKCiJiEhkKCiJiEhkKCiJiEhkKCiJiEhkKChJlTGzPDObYmbTzOx5M6u9G3k9YWYDwsePmlm3MvbtY2a/rEAZ882sabzpxfbZmGBZN5nZlYnWUSTVKShJVdri7j3cvTuwHbg4dqOZVeiXkd39QnefUcYufYCEg5KIJJ+CkkTFh0CnsBXzoZmNBWaYWaaZ3WNmOWb2tZn9CcAC/zKzWWb2NtC8ICMzm2hmPcPH/czsSzObambvmFkHguA3OGylHW5mzczsxbCMHDM7NDy2iZm9ZWbTzexRwMp7Emb2spl9ER4zsNi2+8L0d8ysWZi2t5m9GR7zoZl1rZSzKZKiKvRJVKQyhS2i44A3w6SDgO7uPi+8sK9z94PNrAbwsZm9BRwIdAG6AdnADGBUsXybAY8AR4R5NXb31Wb2MLDR3f8e7jcauM/dPzKzdsB4YF/gRuAjdx9mZicAF8TxdP4YllELyDGzF919FVAHmOzug81saJj3IGAkcLG7f29mvYF/A30rcBpF0oKCklSlWmY2JXz8IfAYQbfa5+4+L0w/BvhZwXgR0ADoDBwBPOPuecASM3u3hPx/AXxQkJe7ry6lHkcB3cwKG0L1zaxuWMZvw2NfN7M1cTynv5jZKeHjtmFdVwH5wJgw/SngpbCMXwLPx5RdI44yRNKWgpJUpS3u3iM2Ibw4b4pNAi5z9/HF9ju+EuuRAfzC3beWUJe4mVkfggB3iLtvNrOJQM1Sdvew3LXFz4HIT5nGlCTqxgN/NrNqAGa2j5nVAT4Afh+OObUEflXCsZ8CR5hZx/DYxmH6BqBezH5vAZcVrJhZj/DhB8CZYdpxQKNy6toAWBMGpK4ELbUCGUBBa+9Mgm7B9cA8MzstLMPM7IByyhBJawpKEnWPEowXfWlm04ARBC38/wHfh9v+A0wqfqC7/wgMJOgqm8rO7rNXgVMKJjoAfwF6hhMpZrBzFuDNBEFtOkE33oJy6vomkGVm3wJ3EgTFApuAXuFz6AsMC9PPAi4I6zcd6B/HORFJW+buVV0HERERQC0lERGJEAUlERGJjMjOvqvV7gz1K8oe1+WOS6q6CvITM+WswxOb5lmORK+dWxY8U6nl767IBiUREUmcWWp3gCkoiYikEUvxURkFJRGRNKKWkoiIRIaCkoiIREait8eKGgUlEZG0opaSiIhEhLrvREQkMhSUREQkMjQlXEREIkMtJRERiQwFJRERiQwFJRERiQxD31MSEZGIUEtJREQiQ0FJREQiQ0FJREQiREFJREQiQi0lERGJDAUlERGJDN1mSEREIkMtJRERiYxU/5G/1A6pIiJShFlGQkt8eVo/M5tlZrPN7Noy9jvVzNzMesakXRceN8vMji2vLLWURETSSGWPKZlZJjAcOBpYBOSY2Vh3n1Fsv3rAX4HPYtK6AacD+wGtgLfNbB93zyutPLWURETSSBJaSr2A2e4+1923A88C/UvY7xbgLmBrTFp/4Fl33+bu84DZYX6lUlASEUkjSQhKrYGFMeuLwrSYMu0goK27v57oscWp+05EJI0k2n1nZgOBgTFJI919ZALHZwD3AuclVHApFJRERNJJglPCwwBUVhBaDLSNWW8TphWoB3QHJoYz/1oAY83spDiO3YWCkohIGknC95RygM5m1pEgoJwOnFmw0d3XAU13lm8TgSvdfbKZbQFGm9m9BBMdOgOfl1WYgpKISBqp7O8puXuumQ0CxgOZwCh3n25mw4DJ7j62jGOnm9lzwAwgF7i0rJl3oKAkIpJWknGbIXcfB4wrlja0lH37FFu/Dbgt3rIUlERE0ohuMyQiItGR4rcZUlASEUknmQpKIiISFWopiYhIZKT2kJKCkohIOnG1lEREJDJSOyYpKImIpJWM1I5KCkoiIulE3XciIhIZqR2TFJRERNKKuu9ERCQy1H0nIiKRkdoxSUFJRCStqPtOREQiI7VjkoKSiEg60R0dREQkOtR9JyIikZHaMUlBSUQkraj7TkREIkPddyIiEhmpHZMUlERE0oq670REJDIUlEREJDJS/OfQU7z6IiJShFliS1xZWj8zm2Vms83s2hK2X2xm35jZFDP7yMy6hekdzGxLmD7FzB4uryy1lERE0kkl996ZWSYwHDgaWATkmNlYd58Rs9tod3843P8k4F6gX7htjrv3iLc8tZSq0NFHHsDU9/7BtA/u48pLTip1v5OP68WWBc9w0M/2KpLetlUTfvz2cS4feEK5eV587jFM++A+tix4hiaN6hWmH/6LfVk27TE+feMOPn3jDq77628Lt136x35MnnA3X7x9D4MuOK4ynrKkiF+2bMTLJ/6csSf15PxubXbZfnbX1rz4m5/z3PEHMeLX+9OyTg0AemY3YMxxBxYun51+KL9q0wSA23/ZhZdP/DkvnHAQN/2iM1kpPvYRVZ5hCS1x6AXMdve57r4deBboX6RM9/Uxq3UAr2j91VKqIhkZxv23ns8JZ93O4qWr+OjV23htwhfM/H5xkf3q1qnJpX/sx+dffr9LHncNPYe3Jk6JK89Jk79j3Dtf8taYobvk83HOTE49/54iad32acP5Z/Tl8BOvZ/uOXMb+91rGvf0lc39YXjknQCIrw+C6g/fm4nensXzzNp7u14P3F61m7vrNhfvMXLORs974iq15+ZzWuSWXH9iRaz6ayeTl6/j9G18BUL96Fq+e1JNJS9cAMG7+Cv72ySwA7ji0C6d0asHz3y/d808w3SUY7M1sIDAwJmmku4+MWW8NLIxZXwT0LiGfS4EhQHWgb8ymjmb2FbAeuN7dPyyrPmopVZGDe3RizvxlzF+wgh078nj+1Un85pieu+x345W/4x8PvcrWbTuKpJ94TE/mL1jBjO8WxZXn1OnzWbBoZdz169q5NTlfzWbL1u3k5eXz4affcvJxvSr4bCWVdG9Sj4UbtrJ441Zy853xP/xIn7aNi+wzefk6tublA/D1yvVk166+Sz5Ht2vKx0vWFO730ZI1hdumr9pQ4jFSCSyxxd1HunvPmGVkKTmXyd2Hu/vewDXA9WHyUqCdux9IELBGm1n9svJJWlAys65mdo2Z/TNcrjGzfZNVXqpp1aIRi5asKlxfvHQVrbMbFdmnR/cOtGnZmDff/apIep3aNbjizydy2/0vJpxnSXof1JnP3ryTl5+8hn33Cbpqps9ayKG9utK4YV1q1axOv1/1oE3LJgk/T0k9zWvVYNnmbYXryzdvp3mtGqXuf8reLYoEnALHtm/GGz/8uEt6lhkndMzm4xKOkUqQYYkt5VsMtI1ZbxOmleZZ4GQAd9/m7qvCx18Ac4B9yiosKd13ZnYNcEZYuc/D5DbAM2b2rLvfmYxy04mZcdcN53DRFQ/tsu36wQN48LE32BRz4aioKdPm0+WQy9i0eRvH/qoHzz0yhP2PHMKs2Uv4x0NjefXp69i8eRtTZ/xAXn7+bpcn6eX4Ds3o1qQuF0z4ukh605rV6NSwDpNKCDx/67U3X65Yx1c/rt9lm1SCyh+rywE6m1lHgmB0OnBm0SKts7sXjDGcAHwfpjcDVrt7npntBXQG5pZVWLLGlC4A9nP3In1OZnYvMB0oMSjF9m1mNepJVt1OSape1VuybA1tWu1sebRu2YTFy3f+AderW5NuXdoWjgFlN2vAC49dyYAL/s7BB3bilON7c9t1Z9Kgfm3y3dm6bQdffTOvzDxLsmHjlsLH49+bwgO3/pEmjeqxas0GnhwzkSfHTATg5qt/z+KlqyvjqUvErdiyjRa1d7aMsmtXZ8WWXT8A9W7RkAu7t+OCCV+zI7/ouPYx7Zvx3sKV5HrR9D/t345GNapxy2ffJqfyUumz79w918wGAeOBTGCUu083s2HAZHcfCwwys6OAHcAa4Nzw8COAYWa2A8gHLnb3Mi8kyQpK+UAr4Idi6S3DbSUK+zJHAtRqd0aFZ2+kgslT59CpYwvat23GkmWrOe3EQzjvL/8q3L5+wxba9tg59jh+zA1cd9vTfPn1XI4acHNh+v8NPpVNm7by8JNvkZmZUWaeJclu1oDlP64DoOcBe5ORYaxaswGAZk3q8+Oq9bRt1YT+/Q7myJN3nSQh6Wf6qg20q1eTVnVqsGLLdo5t34y/fTyryD5dGtXh+l6duPS9aawpNt4J0K9DM/45ZX6RtFP2zuaXLRsx8J1vKj41S8qXhBuyuvs4YFyxtKExj/9aynEvAi+WtK00yQpKlwPvmNn37Jy10Q7oBAxKUpkpJS8vn8E3PMGr/72OzMwMnhwzkW+/W8QNQwbw5TfzeH3CF5WWJ8Al5x/LkItPJLtZQ3Leuos33/2KS655hFOO781F5xxNbm4eW7du5w+D/lmY3zMjBtO4UV127Mjj8hseZ13M7CtJX3kOd06ew0N9u5NhxitzljNn3Wb+/LP2zFi1gfcXr2bwgR2pnZXJPYcFw8RLN2/j8veDr620qlODFrVr8MXydUXy/b9enVm6aSv/OeYAAN5ZuIqR0xbs2Sf3U5Didwk39+R8ZjGzDIL57a3DpMVAjrvnxXN8ureUJJq63HFJVVdBfmKmnHV4pUaRvS58PqFr59xHT4tUFEva95TcPR/4NFn5i4hICVK8paQvz4qIpJMUv1OGgpKISDpRS0lERCIjxe/To6AkIpJO1H0nIiJR4Zmp3VRSUBIRSSepHZMUlERE0oomOoiISGRoTElERCJDLSUREYmM1I5JCkoiIunE1VISEZHIUFASEZHI0EQHERGJDH1PSUREIkMtJRERiQyNKYmISGQoKImISFS4uu9ERCQyNNFBREQiQy0lERGJjBQfU0rxhp6IiBSRYYktcTCzfmY2y8xmm9m1JWy/2My+MbMpZvaRmXWL2XZdeNwsMzu23Oon9GRFRCTaLMGlvOzMMoHhwHFAN+CM2KATGu3u+7t7D+Bu4N7w2G7A6cB+QD/g32F+pVJQEhFJI55hCS1x6AXMdve57r4deBboX6RM9/Uxq3UADx/3B551923uPg+YHeZXKo0piYikkwQnOpjZQGBgTNJIdx8Zs94aWBizvgjoXUI+lwJDgOpA35hjPy12bOuy6qOgJCKSThKc6BAGoJHl7lh+PsOB4WZ2JnA9cG5F8lH3nYhIOqnkMSVgMdA2Zr1NmFaaZ4GTK3isgpKISDrJyEhsiUMO0NnMOppZdYKJC2NjdzCzzjGrJwDfh4/HAqebWQ0z6wh0Bj4vqzB134mIpJHK/u6su+ea2SBgPJAJjHL36WY2DJjs7mOBQWZ2FLADWEPYdRfu9xwwA8gFLnX3vLLKU1ASEUkjybihg7uPA8YVSxsa8/ivZRx7G3BbvGWVGpTMbAM7p/UVPE0PH7u714+3EBER2TMsXW8z5O719mRFRERk96V4TIpvooOZHWZm54ePm4YDViIiEjFmiS1RU+6YkpndCPQEugCPE3wx6ing0ORWTUREEmUpPqc6nokOpwAHAl8CuPsSM1PXnohIBEWx9ZOIeILSdnd3M3MAM6uT5DqJiEgFpfgvV8Q1pvScmY0AGprZRcDbwCPJrZaIiFRE2o8pufvfzexoYD2wDzDU3SckvWYiIpKwKAaaRMT75dlvgFoE31P6JnnVERGR3ZHq31Mqt/vOzC4kuFfRb4EBwKdm9sdkV0xERBJnGYktURNPS+kq4EB3XwVgZk2AT4BRyayYiIgkLsUbSnEFpVXAhpj1DWGaiIhETNoGJTMbEj6cDXxmZq8QjCn1B77eA3UTEZEEZUawSy4RZbWUCr4gOydcCrySvOqIiMjuSNuWkrvfvCcrIiIiuy9tg1IBM2sGXA3sB9QsSHf3vkmsl4iIVICl+C0d4ul9fBqYCXQEbgbmE/w8roiIREyq39EhnqDUxN0fA3a4+/vu/kdArSQRkQhK9aAUz5TwHeH/S83sBGAJ0Dh5VRIRkYqKYqBJRDxB6VYzawBcATwI1AcGJ7VWIiJSISk+pBTXDVlfCx+uA36V3OqIiMjuSNuWkpk9SPBl2RK5+1+SUiMREamwKN7PLhFltZQm77FaiIhIpUjblpK7P7knKyIiIrsvGT9dYWb9gAeATOBRd7+z2PYhwIVALvAj8Ed3/yHclsfOnzxa4O4nlVVWvL+nJCIiKaCyY5KZZQLDgaOBRUCOmY119xkxu30F9HT3zWb2Z+Bu4Pfhti3u3iPe8lK891FERGIl4XtKvYDZ7j7X3bcDzxLcmLuQu7/n7pvD1U+BNhWtv4KSiEgaSUJQag0sjFlfFKaV5gLgjZj1mmY22cw+NbOTyysssrPvtizQ/WBlzzv7/aVVXQWR3ZLo95TMbCAwMCZppLuPrEjZZnY20BM4Mia5vbsvNrO9gHfN7Bt3n1NyDpp9JyKSVhINSmEAKisILQbaxqy3CdOKMLOjgP8DjnT3bTH5Lw7/n2tmE4EDKfpzSEVo9p2ISBrJsFI7uCoqB+hsZh0JgtHpwJmxO5jZgcAIoJ+7r4hJbwRsdvdtZtYUOJRgEkSp4v3pimuAbuinK0REIq2ybzPk7rlmNggYTzAlfJS7TzezYcBkdx8L3APUBZ4Pp6QXTP3eFxhhZvkEcxjuLDZrbxfxTAl/GhgDnABcDJxLMA9dREQiJhmz19x9HDCuWNrQmMdHlXLcJ8D+iZSln64QEUkjGeYJLVGjn64QEUkjaX+XcPTTFSIiKSPVv3yqn64QEUkjad9SMrPHKeFLtOHYkoiIRIhFcJwoEfF0370W87gmcArBuJKIiERM2reU3P3F2HUzewb4KGk1EhGRCkv7MaUSdAaaV3ZFRERk90Vxmnci4hlT2kDRMaVlBHd4EBGRiPkpdN/V2xMVERGR3Zfq3Xfl1t/M3oknTUREql6GJbZETVm/p1QTqA00De/0WlD9+pT9A08iIlJF0nlM6U/A5UAr4At2BqX1wL+SWy0REamIKLZ+ElHW7yk9ADxgZpe5+4N7sE4iIlJBaT+mBOSbWcOCFTNrZGaXJK9KIiJSUal+l/B4gtJF7r62YMXd1wAXJa1GIiJSYWk70SFGppmZuzuAmWUC1ZNbLRERqYgoBppExBOU3gTGmNmIcP1PYZqIiERMqo8pxROUrgEGAn8O1ycAjyStRiIiUmFZGdEbJ0pEuUHV3fPd/WF3H+DuA4AZBD/2JyIiEZOR4BI1cd2Q1cwOBM4AfgfMA15KZqVERKRi0nZMycz2IQhEZwArgTGAubt+fVZEJKLS+Uf+ZgIfAr9x99kAZjZ4j9RKREQqJNVbSmV1Kf4WWAq8Z2aPmNmv2XmrIRERiaBUH1MqtU7u/rK7nw50Bd4juA9eczN7yMyO2UP1ExGRBCTjjg5m1s/MZpnZbDO7toTtQ8xshpl9bWbvmFn7mG3nmtn34XJuufUvbwd33+Tuo939RKAN8BX6kT8RkUiq7Ds6hDdMGA4cB3QDzjCzbsV2+wro6e4/A14A7g6PbQzcCPQGegE3hr86UXr9E3my7r7G3Ue6+68TOU5ERPaMJNxmqBcw293nuvt24Fmgf+wO7v6eu28OVz8laMAAHAtMcPfV4S3qJgD9yqx//E9VRESiLjPBxcwGmtnkmGVgsSxbAwtj1hdR9m/qXQC8UcFj4/uekoiIpIZE7/zt7iOBkZVRtpmdDfQEjqxoHmopiYikkSR03y0G2sastwnTijCzo4D/A05y922JHFuk/nFVSUREUkISglIO0NnMOppZdeB0YGzsDuFdf0YQBKQVMZvGA8eEv8PXCDgmTCuVuu9ERNJIZiV/m9Tdc81sEEEwyQRGuft0MxsGTHb3scA9QF3geTMDWODuJ7n7ajO7hSCwAQxz99VllaegJCKSRpJxRwd3HweMK5Y2NObxUWUcOwoYFW9ZCkoiImkkij9xnggFJRGRNJLq975TUBIRSSOZVV2B3aSgJCKSRtRSEhGRyNCYkoiIREZlTwnf0xSURETSiLrvREQkMhSUREQkMhSUREQkMjI10UFERKIi1e+yraAkIpJG1H0nIiKRoaAkIiKRoTElERGJDLWUREQkMhSUREQkMhSUREQkMnTvOxERiQzdJVxERCJDX56VCvvggy+47bZHyM/P57TTjmbgwNOKbN++fQdXX30v06fPoWHDetx339W0aZPNokXLOf74S+jYsTUABxzQhWHDLgXgtdfeZ8SI5wGjefPG3HPPEBo3bsBdd43ivfc+p1q1arRr14I77vgr9evXBWDmzHnceONwNm7cTEZGBi+8cC81alRn3LgPeeih58jPz6NPn15cddV5e/L0SBKsmzaNBWPG4Pn5NDvsMFoed1yR7csmTODHjz7CMjLIqlePjueeS40mTdi8cCHzn36avC1bsIwMWh5/PE0OPhiA9TNnsvD55/G8PGq3b0/HP/wBy8xk/axZzB4+nOpNmwLQ6KCDaP2b37Bl2TLmjBxZWOa2lStpfdJJtDjqKBa+8AJrp07FsrKo0awZHc87j6zatffcCUoDGlOSCsnLy2PYsId5/PFbyM5uwoABQ+jbtzedOrUr3Of559+ifv26TJgwktdf/4C///0J7r//GgDatWvBK6/8s0ieubl53HbbI7z++nAaN27A3Xc/ztNPv85ll53JoYf24IorziUrK5N77nmCESNe4KqrziM3N4+rrrqXe+4ZQteuHVmzZj1ZWZmsWbOeu+8exUsv3U/jxg245pr7mDRpKocccsAePU9SeTw/nx9Gj2afwYOp3qgRM26/nYYHHECtVq0K96ndti3d/vY3MmvUYMXEiSx88UU6DRxIRvXq7HX++dTMzmb72rXMuPVWGuy3H5k1azL38cfpOmQINbOzWfzKK6ycNIlmhx0GQN3OndnnssuK1KNWixZ0Hzq0sE5Trr6aRgceCED9ffelzSmnYJmZLHzxRZa+8QZtTz11D52h9JDqY0qp3tJLWV9//T3t27ekbdsWVK9ejRNOOIJ33vmsyD7vvvsZp5zyawCOPfZQJk2ainvp/cXujruzZcs23J2NGzfTvHljAA477CCysjIB6NGjC8uWrQTg44+/okuXDnTt2hGARo3qk5mZycKFy2jfvhWNGzcA4JBDDmD8+I8r9yTIHrVp3jxqNG9OzWbNyMjKovHBB7Nm6tQi+9Tv2pXMGjUAqLPXXuxYswaAmtnZ1MzOBqB6w4Zk1a9P7oYN5G7aREZmZuG2+t26sebLL+Ou0/pvv6Vms2bUaNIEgAb77YdlBu/TunvtxfawfIlfhnlCS9QoKFWR5ctX0aJF08L17OwmLF++apd9WrYM9snKyqRevTqsWbMegEWLlnPyyX/l7LOvZfLk6QBUq5bFTTddwoknDuLww89lzpyFDBhw9C5lv/jiBI444ucAzJu3GDO44IKhnHLKX3nkkRcBaN++FfPmLWbRouXk5ubxzjufFgYySU3b166leuPGhevVGzYsDDolWfnRRzTo3n2X9I3z5uG5udRo1oysunXx/Hw2zZ8PwOovvmD76tU79507l2nDhvHdAw+wZcmSXfJanZND47AbsLgfP/64xPKlbFkZiS1Rs8erZGbnl7FtoJlNNrPJI0eO2ZPVSinNmzfmvfdG8fLLD3DttRdyxRV/Z+PGzezYkcszz4zj5Zcf4MMPn6RLlw6MGPFCkWMfemgMmZmZnHRSHyDoRvziixncc88VjB59F2+/PYlJk6bSoEFdbrrpEgYPvpuzzrqG1q2zyciI4DtYkmLlp5+y6YcfaHHMMUXSt69dy7xRo+h43nlYRgZmxt4XXcSC555jxu23k1mzJoTvkzrt2nHAHXfQfehQmvfty/f//neRvPJzc1k7dSqNe/bcpfwlr7+OZWTQpHfv5D3JNJWR4BIPM+tnZrPMbLaZXVvC9iPM7EszyzWzAcW25ZnZlHAZW15ZVTGmdDPweEkb3H0kEI6Afhe9dmUlys5uUqTlsXz5KrKzm+yyz9KlK2nRoim5uXls2LCJRo3qY2ZUr14NgO7dO9GuXQvmzVtc2LXXrl1LAI477jBGjtwZlF566W0mTszhiSduxSzoeG7RoikHH9y9sJvuiCN6Mn36HA455AD69u1F3769ABgz5k0FpRRXvWHDIq2Y7WvXUq1Ro132WzdjBkvHjaPrlVeSUa1aYXreli18/+CDtD75ZOrutVdhet2992bfq68Ojp0+na3LlwOQWatW4T4N99+fH0aPZseGDVSrVy/Yd9o0ardrR7X69YuUv/KTT1j7zTd0GTy48H0q8avsU2ZmmcBw4GhgEZBjZmPdfUbMbguA84ArS8hii7v3iLe8pFxlzOzrUpZvgOxklJlq9t+/M/PnL2HhwmVs376D11//oDAAFOjbtzf/+987AIwf/zG/+MXPMDNWr15HXl4eAAsXLmP+/CW0bduC7OwmzJmzkNWr1wHw8cdT2HvvtkAw0+/RR1/ioYduoFatmoVlHHbYQXz33Xy2bNlKbm4eOTnT6NQpOGbVqrUArFu3kdGjx3HaaUU/NUtqqdOhA9tWrGDbypXk5+ayOieHRgcUnbiyacECfnjqKTpfemmRYJGfm8v3Dz1Ek0MOofHPf17kmB3rgy7l/B07WDp+PM2PPDJIX7eu8IPSxnnzID+frLp1C49b/fnnNO5V9D2/bto0lo4fT+dLLy0c25LEWIJLHHoBs919rrtvB54F+sfu4O7z3f1rIH9365+sllI2cCxQvMPagE+SVGZKycrKZOjQi7nwwhvJy8vn1FOPonPn9jzwwFN0796ZX/+6NwMGHM1VV93L0UcPpEGDutx3X/BpNCdnGv/859NkZWWRkWHcfPOlNGwYfPq89NIzOOusa8nKyqJ162bcccflANxyywi2b9/B+effAOycRt6gQV3OO+9kBgwYgplxxBE96dMn6OO/7bZHmDlzXpjv6YVT0CU1WWYm7c44g1n33w/5+TQ99FBqtWrF4ldeoXb79jTq0YNFL7xA3rZtzB4xAoAajRvTedAgVk+ezMbvviN340ZWfhL8Ce91/vnUbtuWZW+9xdqvvwZ3mh15JPW7dgWC8aUV77+PZWaSUa0aew0cWNjyydu2jXXffkv7s88uUscfnnmG/NxcZt13HxBMduhQbB8pW6ItJTMbCAyMSRoZ9loVaA0sjFlfBCTSr1rTzCYDucCd7v5ymfUpazZXRZnZY8Dj7v5RCdtGu/uZ5eeS3t13Ek1nv7+0qqsgPzFPHXlkpXa4fbny9YSunQc1PaHM8sMxon7ufmG4fg7Q290HlbDvE8Br7v5CTFprd19sZnsB7wK/dvc5pZWXlJaSu19QxrY4ApKIiFSEVf4078VA25j1NmFaXNx9cfj/XDObCBwIlBqUNHItIpJGkjCmlAN0NrOOZlYdOB0odxYdgJk1MrMa4eOmwKHAjLKOUVASEUkjZokt5XH3XGAQMB74FnjO3aeb2TAzOyko0w42s0XAacAIM5seHr4vMNnMpgLvEYwplRmUdJshEZE0koxJ9O4+DhhXLG1ozOMcgm694sd9AuyfSFkKSiIiaUQ3ZBURkchI8ZikoCQikk5S/SYYCkoiImkkxWOSgpKISDpRUBIRkcjQRAcREYmMFI9JCkoiIukkCbcZ2qMUlERE0ohaSiIiEhmaEi4iIpGR6jc0VVASEUkjaimJiEhkpHhMUlASEUknaimJiEhkpHhMUlASEUknuqODiIhERorHJAUlEZF0ojs6iIhIZKilJCIikaHZdyIiEhkpHpMUlERE0oluMyQiIpGR6t13qR5URUSkCEtwiSNHs35mNsvMZpvZtSVsP8LMvjSzXDMbUGzbuWb2fbicW15ZaimJiKQRq+RRJTPLBIYDRwOLgBwzG+vuM2J2WwCcB1xZ7NjGwI1AT8CBL8Jj15RWnlpKIiJpxCwjoSUOvYDZ7j7X3bcDzwL9Y3dw9/nu/jWQX+zYY4EJ7r46DEQTgH5lFaagJCKSViq9+641sDBmfVGYlpRjFZRERNKIkZHYYjbQzCbHLAOrsv4aUxIRSSNxdskVcveRwMgydlkMtI1ZbxOmxWMx0KfYsRPLOkAtJRGRtFLp3Xc5QGcz62hm1YHTgbFxVmY8cIyZNTKzRsAxYVqpFJRERNKIJfivPO6eCwwiCCbfAs+5+3QzG2ZmJwGY2cFmtgg4DRhhZtPDY1cDtxAEthxgWJhWKnXfiYikkcqeEg7g7uOAccXShsY8ziHomivp2FHAqHjLUlASEUkrqd0BpqAkIpJGLMXvM6SgJCKSVhSUREQkIpIxprQnKSiJiKQVjSmJiEhEqKUkIiKRoYkOIiISIQpKIiISEaYxJRERiQ61lEREJCI0piQiIhGioCQiIhGhMSUREYkQtZRERCQi9OVZERGJDE10EBGRCNGYkoiIRIS670REJEIUlEREJCI0piQiIhGiMSUREYmIVB9TMnev6jpIJTOzge4+sqrrIT8des9JZUntdp6UZmBVV0B+cvSek0qhoCQiIpGhoCQiIpGhoJSe1Lcve5rec1IpNNFBREQiQy0lERGJDAUlERGJDAWlNGJm/cxslpnNNrNrq7o+kv7MbJSZrTCzaVVdF0kPCkppwswygeHAcUA34Awz61a1tZKfgCeAflVdCUkfCkrpoxcw293nuvt24FmgfxXXSdKcu38ArK7qekj6UFBKH62BhTHri8I0EZGUoaAkIiKRoaCUPhYDbWPW24RpIiIpQ0EpfeQAnc2so5lVB04HxlZxnUREEqKglCbcPRcYBIwHvgWec/fpVVsrSXdm9gwwCehiZovM7IKqrpOkNt1mSEREIkMtJRERiQwFJRERiQwFJRERiQwFJRERiQwFJRERiQwFJRERiQwFJRERiYz/B2sjUPTb0KxUAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "class_names=[0,1]\n", "\n", "fig, ax = plt.subplots() \n", "tick_marks = np.arange(len(class_names)) \n", "plt.xticks(tick_marks, class_names) \n", "plt.yticks(tick_marks, class_names) \n", "sns.heatmap(pd.DataFrame(cm_FA_f), annot=True, cmap=\"YlGnBu\" ,fmt='g') \n", "ax.xaxis.set_label_position(\"top\") \n", "plt.tight_layout() \n", "plt.title('Confusion matrix for Fractional Anisotropy', y=1.1) \n", "plt.ylabel('Actual label') \n", "plt.xlabel('Predicted label')" ] }, { "cell_type": "markdown", "id": "d3b1ac17", "metadata": {}, "source": [ "The confusion matrix shows that the probability for **hits** is around 23%, for **true negatives** around 44%. The probability for **misses** is around 6%. The probability for **false positive** cases is around 27%. " ] }, { "cell_type": "markdown", "id": "7a5f68b1", "metadata": {}, "source": [ "##### 2.2.3.2 Model accuracy, precision, recall and F1-Score" ] }, { "cell_type": "code", "execution_count": 120, "id": "b400b986", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Accuracy: 0.6697310344827586\n", "Precision: 0.45770542294577055\n", "Recall: 0.7975657090830678\n", "F1-Score: 0.5816275717468222\n" ] } ], "source": [ "#compute accuracy, precision, recall\n", "\n", "print(\"Accuracy:\",metrics.accuracy_score(y_tests_FA, y_preds_FA)) \n", "\n", "print(\"Precision:\",metrics.precision_score(y_tests_FA, y_preds_FA)) \n", "\n", "print(\"Recall:\",metrics.recall_score(y_tests_FA, y_preds_FA)) \n", "\n", "print(\"F1-Score:\", metrics.f1_score(y_tests_FA, y_preds_FA))" ] }, { "cell_type": "markdown", "id": "113adf3d", "metadata": {}, "source": [ "As the values reveal, our logistic regression model for **fractional anisotropy** makes 66.97% of the time correct predictions. 45.77% represent the proportion of the model's prediction of psychosis where psychosis is actually present and 79.75% relate to the proportion of all cases of psychosis that the model accurately predicted. " ] }, { "cell_type": "markdown", "id": "5914663f", "metadata": {}, "source": [ "## 3. Summary" ] }, { "cell_type": "markdown", "id": "542475a8", "metadata": {}, "source": [ "To summarize the performance measures of our logistic regression models for **CT** and **FA and MD**, we can simply create a table with the relevant data." ] }, { "cell_type": "markdown", "id": "4edf4326", "metadata": {}, "source": [ "For that, we need to import the tabulate function from the tabulate module. However, before this u have to run the ```pip install tabulate```command. Remove the hashtag in the next line, to install it." ] }, { "cell_type": "code", "execution_count": 165, "id": "e03942ec", "metadata": {}, "outputs": [], "source": [ "#!pip install tabulate\n", "\n", "from tabulate import tabulate" ] }, { "cell_type": "markdown", "id": "4b63d003", "metadata": {}, "source": [ "Next, we define our data and the column names and print the table." ] }, { "cell_type": "code", "execution_count": 184, "id": "a8014e5b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Measure Cortical Thickness Mean Diffusivity Fractional Anisotropy\n", "--------- -------------------- ------------------ -----------------------\n", "Accuracy 0.681785 0.752697 0.669731\n", "Precision 0.446441 0.551946 0.457705\n", "Recall 0.968729 0.748209 0.797566\n", "F1-Score 0.611206 0.635264 0.581628\n" ] } ], "source": [ "#create data\n", "data_LR = [[\"Accuracy\", metrics.accuracy_score(y_tests, y_preds), metrics.accuracy_score(y_tests_MD, y_preds_MD), metrics.accuracy_score(y_tests_FA, y_preds_FA)], \n", " [\"Precision\", metrics.precision_score(y_tests, y_preds), metrics.precision_score(y_tests_MD, y_preds_MD),metrics.precision_score(y_tests_FA, y_preds_FA)], \n", " [\"Recall\", metrics.recall_score(y_tests, y_preds), metrics.recall_score(y_tests_MD, y_preds_MD),metrics.recall_score(y_tests_FA, y_preds_FA)], \n", " [\"F1-Score\", metrics.f1_score(y_tests, y_preds), metrics.f1_score(y_tests_MD, y_preds_MD), metrics.f1_score(y_tests_FA, y_preds_FA)]]\n", " \n", "#define header names\n", "col_names = [\"Measure\", \"Cortical Thickness\", \"Mean Diffusivity\", \"Fractional Anisotropy\"]\n", " \n", "#display table\n", "print(tabulate(data_LR, headers=col_names))" ] }, { "attachments": {}, "cell_type": "markdown", "id": "7e26df31", "metadata": {}, "source": [ "What can be seen is, that **MD** has the highest **accuracy**, the highest **precision** and the highest **F1-Score**. **CT** the highest **recall** score. **Precision and Recall** values are inversely related. As one increases, the other decreases. So when we relate to the **F1-Score**, **MD** performs better.\n" ] }, { "cell_type": "markdown", "id": "f6fa1c9c", "metadata": {}, "source": [ "In the next pages, a different algorithm is applied to the same data." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.0" }, "vscode": { "interpreter": { "hash": "578504114d49301275c44c87035f08411733f9928d9347745d7de100c09f7611" } } }, "nbformat": 4, "nbformat_minor": 5 }