{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Άσκηση 04 Βελτιστοποίηση kNN και SVC στο Iris με την Optuna - solution.ipynb","provenance":[],"authorship_tag":"ABX9TyM4B3RufUluFqLPlmKt52If"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"markdown","metadata":{"id":"v6cFZvQRGgrm"},"source":["# Άσκηση 04 Βελτιστοποίηση kNN και SVC στο Iris με την Optuna\n","\n","![](https://res.cloudinary.com/practicaldev/image/fetch/s--XpO3kV9Z--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://dev-to-uploads.s3.amazonaws.com/i/r8wwzb70006nq7hy24ct.jpeg)\n","\n","\n","Χρησιμοποιώντας την βιβλιοθήκη Optuna βρείτε τις βέλτιστες υπερπαραμέτρους για τον kNN και τα SVM στο Iris dataset.\n"]},{"cell_type":"markdown","metadata":{"id":"o5e0O72xM3k9"},"source":["## Optuna tutorial: *Optimize Your Optimization*\n","\n","![](https://optuna.org/assets/img/optuna-logo.png) \n","\n","\n","Μελετήστε πρώτα τα τέσσερα παραδείγματα κάτω από το [\"Key Features\"](https://optuna.readthedocs.io/en/stable/tutorial/index.html#key-features) της τεκμηρίωσης της Optuna."]},{"cell_type":"markdown","metadata":{"id":"3gulCuSaMeQl"},"source":["\n","## Βήματα\n","\n","1. Ορίστε μια συνάρτηση - στόχο \"objective\" η οποία θα θα εκπαιδεύει τους ταξινομητές και θα επιστρέφει το μέσο τετραγωνικό σφάλμα στο test set.\n","2. Εντός της \"objective\" χρησιμοποιήστε τις μεθόδους trial.suggest_categorical, trial.suggest_int, και trial.suggest_loguniform γιά όλες τις επιλογές που πρέπει να γίνουν. Hint: ξεκινήστε με trial_suggest_categorical για το είδος του ταξινομητή, ώστε να \"χωρέσουν\" και οι δύο σε ένα function.\n","3. Βελτιστοποιήστε κάνοντας 100 δοκιμές ως εξής: \n","```python\n","import optuna\n","# depending on the definition of objective\n","# we can create study object with either minimize or maximize\n","study = optuna.create_study(direction='minimize')\n","# start tuning for the hyper-parameters\n","study.optimize(objective, n_trials=100)\n","```\n"]},{"cell_type":"markdown","metadata":{"id":"b8KaATEKMZ4r"},"source":["\n","## Υπερπαράμετροι, τιμές και εύρη τιμών προς εξερεύνηση\n","\n","[kNN](https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html)\n","\n","* n_neighbors: 3 με 11\n","* algorithm: \"ball_tree\", και \"kd_tree\"\n","* leaf_size: 1 με 50\n","* metric: \"euclidean\", \"manhattan\", \"chebyshev\", και \"minkowski\"\n","\n","\n","[SVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html)\n","\n","* C: 1e-10 με 1\n","* kernel: 'poly', 'rbf', 'sigmoid'\n","* degree: 1 με 50\n","* gamma: 0.001 με 10000\n","\n"]},{"cell_type":"markdown","metadata":{"id":"eJUl50T3hFIh"},"source":["## Επιλογή μοντέλου\n","\n","* Ποιο είναι το καλύτερο μοντέλαο kNN;\n","* Ποιο είναι το καλύτερο μοντέλο SVM;\n","* Ποιο έίναι το καλύτερο μοντέλο συνολικά;"]},{"cell_type":"code","metadata":{"id":"zROY29F1J4sj"},"source":["from sklearn.svm import SVC\n","from sklearn import datasets\n","from sklearn.metrics import mean_squared_error\n","from sklearn.neighbors import KNeighborsClassifier\n","from sklearn.model_selection import train_test_split\n","\n","\n","def objective(trial):\n","\n","    # Load data\n","    iris = datasets.load_iris()\n","    x = iris.data\n","    y = iris.target\n","    x_train,x_test,y_train,y_test = train_test_split(x,y,test_size=0.2)\n","    \n","    # Sample hyper parameters\n","    classifier_name = trial.suggest_categorical(\"classifier\", [\"KNeighborsClassifier\",\n","                                                               \"SVC\"])\n","    if classifier_name==\"KNeighborsClassifier\":\n","\n","        # Sample hyper parameters\n","        n_neighbors = trial.suggest_int('n_neighbors', 3, 11)\n","        algorithm = trial.suggest_categorical(\"algorithm\", \n","                                              [\"ball_tree\",\n","                                                \"kd_tree\"])\n","        leaf_size = trial.suggest_int('leaf_size', 1, 50)\n","        metric = trial.suggest_categorical('metric', \n","                                           [\"euclidean\",\"manhattan\", \n","                                            \"chebyshev\",\"minkowski\"])\n","        # Construct the model\n","        clf = KNeighborsClassifier(n_neighbors=n_neighbors,\n","                               algorithm=algorithm,\n","                               leaf_size=leaf_size,\n","                               metric=metric,\n","                               )\n","    elif classifier_name==\"SVC\":\n","\n","        # Sample hyper parameters\n","        C = trial.suggest_loguniform('C', 1e-10, 1)\n","        kernel = trial.suggest_categorical('kernel',['poly','rbf','sigmoid'])\n","        degree = trial.suggest_int('degree',1, 50)\n","        gamma = trial.suggest_loguniform('gamma',0.001,10000)\n","\n","        # Construct the model\n","        clf = SVC(C=C, kernel=kernel, degree=degree,gamma=gamma)\n","    \n","    # Train the model\n","    clf.fit(x_train,y_train)\n","\n","    # Evaluate the model\n","    y_pred_test = clf.predict(x_test)\n","    loss = mean_squared_error(y_test,y_pred_test)\n","    print(\"Test Score:\",clf.score(x_test,y_test))\n","    print(\"Train Score:\",clf.score(x_train,y_train))\n","    print(\"\\n=================\")\n","    return loss"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"nWizkqbEKTdN","executionInfo":{"status":"ok","timestamp":1635747722515,"user_tz":-120,"elapsed":8296,"user":{"displayName":"Giorgos Siolas","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GjxnZOAObbc3X0z9X2rs1N_1geznqhrotkq3KF-p_M=s64","userId":"10127542075805046236"}},"outputId":"873d99c1-2cdf-4570-cf65-58201e768ecc"},"source":["!pip install optuna"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Collecting optuna\n","  Downloading optuna-2.10.0-py3-none-any.whl (308 kB)\n","\u001b[K     |████████████████████████████████| 308 kB 7.4 MB/s \n","\u001b[?25hCollecting cliff\n","  Downloading cliff-3.9.0-py3-none-any.whl (80 kB)\n","\u001b[K     |████████████████████████████████| 80 kB 8.0 MB/s \n","\u001b[?25hRequirement already satisfied: scipy!=1.4.0 in /usr/local/lib/python3.7/dist-packages (from optuna) (1.4.1)\n","Collecting alembic\n","  Downloading alembic-1.7.4-py3-none-any.whl (209 kB)\n","\u001b[K     |████████████████████████████████| 209 kB 44.6 MB/s \n","\u001b[?25hCollecting colorlog\n","  Downloading colorlog-6.5.0-py2.py3-none-any.whl (11 kB)\n","Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from optuna) (21.0)\n","Collecting cmaes>=0.8.2\n","  Downloading cmaes-0.8.2-py3-none-any.whl (15 kB)\n","Requirement already satisfied: sqlalchemy>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from optuna) (1.4.25)\n","Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from optuna) (4.62.3)\n","Requirement already satisfied: PyYAML in /usr/local/lib/python3.7/dist-packages (from optuna) (3.13)\n","Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from optuna) (1.19.5)\n","Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->optuna) (2.4.7)\n","Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from sqlalchemy>=1.1.0->optuna) (4.8.1)\n","Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.7/dist-packages (from sqlalchemy>=1.1.0->optuna) (1.1.2)\n","Collecting Mako\n","  Downloading Mako-1.1.5-py2.py3-none-any.whl (75 kB)\n","\u001b[K     |████████████████████████████████| 75 kB 4.1 MB/s \n","\u001b[?25hRequirement already satisfied: importlib-resources in /usr/local/lib/python3.7/dist-packages (from alembic->optuna) (5.2.2)\n","Collecting autopage>=0.4.0\n","  Downloading autopage-0.4.0-py3-none-any.whl (20 kB)\n","Collecting stevedore>=2.0.1\n","  Downloading stevedore-3.5.0-py3-none-any.whl (49 kB)\n","\u001b[K     |████████████████████████████████| 49 kB 5.2 MB/s \n","\u001b[?25hRequirement already satisfied: PrettyTable>=0.7.2 in /usr/local/lib/python3.7/dist-packages (from cliff->optuna) (2.2.1)\n","Collecting pbr!=2.1.0,>=2.0.0\n","  Downloading pbr-5.6.0-py2.py3-none-any.whl (111 kB)\n","\u001b[K     |████████████████████████████████| 111 kB 48.8 MB/s \n","\u001b[?25hCollecting cmd2>=1.0.0\n","  Downloading cmd2-2.2.0-py3-none-any.whl (144 kB)\n","\u001b[K     |████████████████████████████████| 144 kB 57.6 MB/s \n","\u001b[?25hCollecting pyperclip>=1.6\n","  Downloading pyperclip-1.8.2.tar.gz (20 kB)\n","Requirement already satisfied: attrs>=16.3.0 in /usr/local/lib/python3.7/dist-packages (from cmd2>=1.0.0->cliff->optuna) (21.2.0)\n","Collecting colorama>=0.3.7\n","  Downloading colorama-0.4.4-py2.py3-none-any.whl (16 kB)\n","Requirement already satisfied: wcwidth>=0.1.7 in /usr/local/lib/python3.7/dist-packages (from cmd2>=1.0.0->cliff->optuna) (0.2.5)\n","Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from cmd2>=1.0.0->cliff->optuna) (3.7.4.3)\n","Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->sqlalchemy>=1.1.0->optuna) (3.6.0)\n","Requirement already satisfied: MarkupSafe>=0.9.2 in /usr/local/lib/python3.7/dist-packages (from Mako->alembic->optuna) (2.0.1)\n","Building wheels for collected packages: pyperclip\n","  Building wheel for pyperclip (setup.py) ... \u001b[?25l\u001b[?25hdone\n","  Created wheel for pyperclip: filename=pyperclip-1.8.2-py3-none-any.whl size=11136 sha256=b42e7674a54ce1ad5659e0db0acbaae656e6427a9b28c19209adbe397b7e4650\n","  Stored in directory: /root/.cache/pip/wheels/9f/18/84/8f69f8b08169c7bae2dde6bd7daf0c19fca8c8e500ee620a28\n","Successfully built pyperclip\n","Installing collected packages: pyperclip, pbr, colorama, stevedore, Mako, cmd2, autopage, colorlog, cmaes, cliff, alembic, optuna\n","Successfully installed Mako-1.1.5 alembic-1.7.4 autopage-0.4.0 cliff-3.9.0 cmaes-0.8.2 cmd2-2.2.0 colorama-0.4.4 colorlog-6.5.0 optuna-2.10.0 pbr-5.6.0 pyperclip-1.8.2 stevedore-3.5.0\n"]}]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"INFspD3VKHW-","executionInfo":{"status":"ok","timestamp":1635747733239,"user_tz":-120,"elapsed":7508,"user":{"displayName":"Giorgos Siolas","photoUrl":"https://lh3.googleusercontent.com/a-/AOh14GjxnZOAObbc3X0z9X2rs1N_1geznqhrotkq3KF-p_M=s64","userId":"10127542075805046236"}},"outputId":"d8cb422d-3316-4438-f083-6b9a3cd5f9ee"},"source":["import optuna\n","\n","# depending on the definition of objective\n","# we can create study object with either minimize or maximize\n","study = optuna.create_study(direction='minimize')\n","\n","# start tuning for the hyper-parameters\n","study.optimize(objective, n_trials=100)"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stderr","text":["\u001b[32m[I 2021-11-01 06:22:07,359]\u001b[0m A new study created in memory with name: no-name-5b2ae69b-af62-4aaf-a433-126b705769d4\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,381]\u001b[0m Trial 0 finished with value: 0.8 and parameters: {'classifier': 'SVC', 'C': 7.209139830340565e-09, 'kernel': 'sigmoid', 'degree': 44, 'gamma': 8625.006179740649}. Best is trial 0 with value: 0.8.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,406]\u001b[0m Trial 1 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 3, 'algorithm': 'ball_tree', 'leaf_size': 28, 'metric': 'manhattan'}. Best is trial 1 with value: 0.06666666666666667.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,419]\u001b[0m Trial 2 finished with value: 2.8666666666666667 and parameters: {'classifier': 'SVC', 'C': 3.1350657380153916e-05, 'kernel': 'sigmoid', 'degree': 38, 'gamma': 0.12048471595657338}. Best is trial 1 with value: 0.06666666666666667.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,436]\u001b[0m Trial 3 finished with value: 0.03333333333333333 and parameters: {'classifier': 'SVC', 'C': 2.662898901538909e-07, 'kernel': 'poly', 'degree': 8, 'gamma': 3.1874144291005178}. Best is trial 3 with value: 0.03333333333333333.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,451]\u001b[0m Trial 4 finished with value: 1.8333333333333333 and parameters: {'classifier': 'SVC', 'C': 1.8639253890241803e-07, 'kernel': 'sigmoid', 'degree': 1, 'gamma': 0.001612656896330983}. Best is trial 3 with value: 0.03333333333333333.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,475]\u001b[0m Trial 5 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'ball_tree', 'leaf_size': 38, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,491]\u001b[0m Trial 6 finished with value: 0.7333333333333333 and parameters: {'classifier': 'SVC', 'C': 0.0017096028419162418, 'kernel': 'rbf', 'degree': 44, 'gamma': 425.36331611390045}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,515]\u001b[0m Trial 7 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 4, 'algorithm': 'kd_tree', 'leaf_size': 4, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,526]\u001b[0m Trial 8 finished with value: 2.066666666666667 and parameters: {'classifier': 'SVC', 'C': 3.4123227360246604e-09, 'kernel': 'sigmoid', 'degree': 30, 'gamma': 1491.5062681969966}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,540]\u001b[0m Trial 9 finished with value: 1.9333333333333333 and parameters: {'classifier': 'SVC', 'C': 1.9525132783916855e-09, 'kernel': 'rbf', 'degree': 41, 'gamma': 1.6512861773315455}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,566]\u001b[0m Trial 10 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 11, 'algorithm': 'ball_tree', 'leaf_size': 46, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n"]},{"output_type":"stream","name":"stdout","text":["Test Score: 0.2\n","Train Score: 0.36666666666666664\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.03333333333333333\n","Train Score: 0.06666666666666667\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 1.0\n","\n","=================\n","Test Score: 0.26666666666666666\n","Train Score: 0.35\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.26666666666666666\n","Train Score: 0.35\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.23333333333333334\n","Train Score: 0.35833333333333334\n","\n","=================\n","Test Score: 0.26666666666666666\n","Train Score: 0.35\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.975\n","\n","=================\n"]},{"output_type":"stream","name":"stderr","text":["\u001b[32m[I 2021-11-01 06:22:07,599]\u001b[0m Trial 11 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'ball_tree', 'leaf_size': 44, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,647]\u001b[0m Trial 12 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 7, 'algorithm': 'kd_tree', 'leaf_size': 26, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,672]\u001b[0m Trial 13 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 7, 'algorithm': 'kd_tree', 'leaf_size': 27, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,695]\u001b[0m Trial 14 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 7, 'algorithm': 'kd_tree', 'leaf_size': 35, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,722]\u001b[0m Trial 15 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'ball_tree', 'leaf_size': 14, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,758]\u001b[0m Trial 16 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'kd_tree', 'leaf_size': 37, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,782]\u001b[0m Trial 17 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 5, 'algorithm': 'kd_tree', 'leaf_size': 16, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n"]},{"output_type":"stream","name":"stdout","text":["Test Score: 0.9333333333333333\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9666666666666667\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.9916666666666667\n","\n","=================\n"]},{"output_type":"stream","name":"stderr","text":["\u001b[32m[I 2021-11-01 06:22:07,822]\u001b[0m Trial 18 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 10, 'algorithm': 'ball_tree', 'leaf_size': 38, 'metric': 'manhattan'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,842]\u001b[0m Trial 19 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'ball_tree', 'leaf_size': 50, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,871]\u001b[0m Trial 20 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 11, 'algorithm': 'kd_tree', 'leaf_size': 47, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,894]\u001b[0m Trial 21 finished with value: 0.1 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 8, 'algorithm': 'kd_tree', 'leaf_size': 33, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,916]\u001b[0m Trial 22 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'ball_tree', 'leaf_size': 41, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,938]\u001b[0m Trial 23 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 8, 'algorithm': 'ball_tree', 'leaf_size': 50, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,959]\u001b[0m Trial 24 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 10, 'algorithm': 'ball_tree', 'leaf_size': 40, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:07,982]\u001b[0m Trial 25 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 6, 'algorithm': 'ball_tree', 'leaf_size': 44, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,009]\u001b[0m Trial 26 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 6, 'algorithm': 'ball_tree', 'leaf_size': 21, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n"]},{"output_type":"stream","name":"stdout","text":["Test Score: 0.9333333333333333\n","Train Score: 0.95\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9666666666666667\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.9\n","Train Score: 0.9666666666666667\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9666666666666667\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9666666666666667\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.9666666666666667\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 1.0\n","\n","=================\n"]},{"output_type":"stream","name":"stderr","text":["\u001b[32m[I 2021-11-01 06:22:08,038]\u001b[0m Trial 27 finished with value: 0.1 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 10, 'algorithm': 'ball_tree', 'leaf_size': 29, 'metric': 'manhattan'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,068]\u001b[0m Trial 28 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 7, 'algorithm': 'kd_tree', 'leaf_size': 21, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,095]\u001b[0m Trial 29 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 10, 'algorithm': 'ball_tree', 'leaf_size': 20, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,125]\u001b[0m Trial 30 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 8, 'algorithm': 'ball_tree', 'leaf_size': 41, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,154]\u001b[0m Trial 31 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 7, 'algorithm': 'kd_tree', 'leaf_size': 31, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,179]\u001b[0m Trial 32 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 6, 'algorithm': 'kd_tree', 'leaf_size': 32, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,208]\u001b[0m Trial 33 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 10, 'algorithm': 'kd_tree', 'leaf_size': 39, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n"]},{"output_type":"stream","name":"stdout","text":["Test Score: 0.9\n","Train Score: 0.975\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9583333333333334\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9666666666666667\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.9916666666666667\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.975\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.9666666666666667\n"]},{"output_type":"stream","name":"stderr","text":["\u001b[32m[I 2021-11-01 06:22:08,233]\u001b[0m Trial 34 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 7, 'algorithm': 'kd_tree', 'leaf_size': 22, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,253]\u001b[0m Trial 35 finished with value: 2.033333333333333 and parameters: {'classifier': 'SVC', 'C': 0.16679663721689622, 'kernel': 'poly', 'degree': 21, 'gamma': 0.0011157416580923386}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,283]\u001b[0m Trial 36 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 11, 'algorithm': 'kd_tree', 'leaf_size': 41, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,302]\u001b[0m Trial 37 finished with value: 0.6666666666666666 and parameters: {'classifier': 'SVC', 'C': 0.33386318725275244, 'kernel': 'rbf', 'degree': 19, 'gamma': 51.00335925976303}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,327]\u001b[0m Trial 38 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 10, 'algorithm': 'ball_tree', 'leaf_size': 40, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,346]\u001b[0m Trial 39 finished with value: 0.03333333333333333 and parameters: {'classifier': 'SVC', 'C': 0.00047916035951341016, 'kernel': 'poly', 'degree': 12, 'gamma': 0.04384490422139567}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,377]\u001b[0m Trial 40 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 5, 'algorithm': 'kd_tree', 'leaf_size': 34, 'metric': 'manhattan'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,419]\u001b[0m Trial 41 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 7, 'algorithm': 'kd_tree', 'leaf_size': 26, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n"]},{"output_type":"stream","name":"stdout","text":["Train Score: 0.9916666666666667\n","\n","=================\n","Test Score: 0.26666666666666666\n","Train Score: 0.35\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.3333333333333333\n","Train Score: 0.5\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9916666666666667\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9916666666666667\n","\n","=================\n"]},{"output_type":"stream","name":"stderr","text":["\u001b[32m[I 2021-11-01 06:22:08,450]\u001b[0m Trial 42 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 8, 'algorithm': 'kd_tree', 'leaf_size': 30, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,476]\u001b[0m Trial 43 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'kd_tree', 'leaf_size': 37, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,502]\u001b[0m Trial 44 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 8, 'algorithm': 'kd_tree', 'leaf_size': 12, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,534]\u001b[0m Trial 45 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 6, 'algorithm': 'kd_tree', 'leaf_size': 32, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,553]\u001b[0m Trial 46 finished with value: 1.7 and parameters: {'classifier': 'SVC', 'C': 1.585575118758267e-10, 'kernel': 'rbf', 'degree': 50, 'gamma': 53.836557757220156}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,581]\u001b[0m Trial 47 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'ball_tree', 'leaf_size': 5, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,608]\u001b[0m Trial 48 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 5, 'algorithm': 'kd_tree', 'leaf_size': 27, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,634]\u001b[0m Trial 49 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 3, 'algorithm': 'ball_tree', 'leaf_size': 49, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n"]},{"output_type":"stream","name":"stdout","text":["Test Score: 0.9333333333333333\n","Train Score: 0.9916666666666667\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.3\n","Train Score: 0.3416666666666667\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.95\n","\n","=================\n"]},{"output_type":"stream","name":"stderr","text":["\u001b[32m[I 2021-11-01 06:22:08,662]\u001b[0m Trial 50 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 10, 'algorithm': 'kd_tree', 'leaf_size': 8, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,685]\u001b[0m Trial 51 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 7, 'algorithm': 'kd_tree', 'leaf_size': 2, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,707]\u001b[0m Trial 52 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 7, 'algorithm': 'kd_tree', 'leaf_size': 24, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,732]\u001b[0m Trial 53 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 6, 'algorithm': 'kd_tree', 'leaf_size': 19, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,760]\u001b[0m Trial 54 finished with value: 0.1 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 11, 'algorithm': 'kd_tree', 'leaf_size': 43, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:08,785]\u001b[0m Trial 55 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'kd_tree', 'leaf_size': 35, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n"]},{"output_type":"stream","name":"stdout","text":["Test Score: 1.0\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9666666666666667\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.9\n","Train Score: 0.9916666666666667\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.975\n","\n","=================\n"]},{"output_type":"stream","name":"stderr","text":["\u001b[32m[I 2021-11-01 06:22:11,808]\u001b[0m Trial 56 finished with value: 0.0 and parameters: {'classifier': 'SVC', 'C': 0.01087790207756262, 'kernel': 'poly', 'degree': 30, 'gamma': 0.04806302627584933}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:11,835]\u001b[0m Trial 57 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 4, 'algorithm': 'kd_tree', 'leaf_size': 9, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:11,864]\u001b[0m Trial 58 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 6, 'algorithm': 'ball_tree', 'leaf_size': 31, 'metric': 'manhattan'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:11,899]\u001b[0m Trial 59 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 10, 'algorithm': 'ball_tree', 'leaf_size': 38, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:11,923]\u001b[0m Trial 60 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 11, 'algorithm': 'ball_tree', 'leaf_size': 39, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:11,946]\u001b[0m Trial 61 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 11, 'algorithm': 'kd_tree', 'leaf_size': 37, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:11,974]\u001b[0m Trial 62 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 8, 'algorithm': 'ball_tree', 'leaf_size': 50, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,003]\u001b[0m Trial 63 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'kd_tree', 'leaf_size': 36, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n"]},{"output_type":"stream","name":"stdout","text":["Test Score: 1.0\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.975\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.975\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9666666666666667\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9833333333333333\n","\n","=================\n"]},{"output_type":"stream","name":"stderr","text":["\u001b[32m[I 2021-11-01 06:22:12,031]\u001b[0m Trial 64 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 8, 'algorithm': 'ball_tree', 'leaf_size': 47, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,056]\u001b[0m Trial 65 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'ball_tree', 'leaf_size': 14, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,091]\u001b[0m Trial 66 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 8, 'algorithm': 'ball_tree', 'leaf_size': 44, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,115]\u001b[0m Trial 67 finished with value: 0.06666666666666667 and parameters: {'classifier': 'SVC', 'C': 0.007366144423491904, 'kernel': 'poly', 'degree': 31, 'gamma': 0.03473204041690176}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,141]\u001b[0m Trial 68 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 4, 'algorithm': 'kd_tree', 'leaf_size': 47, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,168]\u001b[0m Trial 69 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 3, 'algorithm': 'ball_tree', 'leaf_size': 9, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,194]\u001b[0m Trial 70 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 3, 'algorithm': 'kd_tree', 'leaf_size': 47, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,214]\u001b[0m Trial 71 finished with value: 1.7333333333333334 and parameters: {'classifier': 'SVC', 'C': 1.7847545766073692e-05, 'kernel': 'poly', 'degree': 33, 'gamma': 0.6388150203893904}. Best is trial 5 with value: 0.0.\u001b[0m\n"]},{"output_type":"stream","name":"stdout","text":["Test Score: 0.9666666666666667\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9666666666666667\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 1.0\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9666666666666667\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.975\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9583333333333334\n","\n","=================\n","Test Score: 0.26666666666666666\n","Train Score: 0.35\n","\n","=================\n"]},{"output_type":"stream","name":"stderr","text":["\u001b[32m[I 2021-11-01 06:22:12,251]\u001b[0m Trial 72 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 8, 'algorithm': 'kd_tree', 'leaf_size': 29, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,268]\u001b[0m Trial 73 finished with value: 1.9333333333333333 and parameters: {'classifier': 'SVC', 'C': 0.012282249721113796, 'kernel': 'poly', 'degree': 22, 'gamma': 0.006341195174808289}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,294]\u001b[0m Trial 74 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'kd_tree', 'leaf_size': 33, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,319]\u001b[0m Trial 75 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 7, 'algorithm': 'kd_tree', 'leaf_size': 17, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,348]\u001b[0m Trial 76 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 10, 'algorithm': 'kd_tree', 'leaf_size': 41, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,376]\u001b[0m Trial 77 finished with value: 0.13333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 10, 'algorithm': 'kd_tree', 'leaf_size': 9, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,398]\u001b[0m Trial 78 finished with value: 0.1 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 8, 'algorithm': 'ball_tree', 'leaf_size': 44, 'metric': 'manhattan'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,427]\u001b[0m Trial 79 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 4, 'algorithm': 'ball_tree', 'leaf_size': 43, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n"]},{"output_type":"stream","name":"stdout","text":["Test Score: 0.9666666666666667\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.26666666666666666\n","Train Score: 0.35\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.8666666666666667\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.9\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9583333333333334\n","\n","=================\n"]},{"output_type":"stream","name":"stderr","text":["\u001b[32m[I 2021-11-01 06:22:12,444]\u001b[0m Trial 80 finished with value: 1.8666666666666667 and parameters: {'classifier': 'SVC', 'C': 5.974089116035382e-07, 'kernel': 'poly', 'degree': 14, 'gamma': 16.252603412217834}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,473]\u001b[0m Trial 81 finished with value: 0.1 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 3, 'algorithm': 'kd_tree', 'leaf_size': 50, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,497]\u001b[0m Trial 82 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 4, 'algorithm': 'kd_tree', 'leaf_size': 46, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,512]\u001b[0m Trial 83 finished with value: 0.26666666666666666 and parameters: {'classifier': 'SVC', 'C': 0.00017490222165557058, 'kernel': 'poly', 'degree': 27, 'gamma': 0.29589237198029633}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,530]\u001b[0m Trial 84 finished with value: 1.6666666666666667 and parameters: {'classifier': 'SVC', 'C': 0.061407362324338396, 'kernel': 'poly', 'degree': 35, 'gamma': 0.010642490670927325}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,546]\u001b[0m Trial 85 finished with value: 2.2333333333333334 and parameters: {'classifier': 'SVC', 'C': 2.3520228803711233e-06, 'kernel': 'rbf', 'degree': 2, 'gamma': 5.540826288888942}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,571]\u001b[0m Trial 86 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'ball_tree', 'leaf_size': 24, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,607]\u001b[0m Trial 87 finished with value: 0.1 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 7, 'algorithm': 'kd_tree', 'leaf_size': 36, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,632]\u001b[0m Trial 88 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 3, 'algorithm': 'kd_tree', 'leaf_size': 45, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n"]},{"output_type":"stream","name":"stdout","text":["Test Score: 0.3333333333333333\n","Train Score: 0.3333333333333333\n","\n","=================\n","Test Score: 0.9\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.7333333333333333\n","Train Score: 0.65\n","\n","=================\n","Test Score: 0.3333333333333333\n","Train Score: 0.425\n","\n","=================\n","Test Score: 0.16666666666666666\n","Train Score: 0.375\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.9\n","Train Score: 0.9916666666666667\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.975\n","\n","=================\n"]},{"output_type":"stream","name":"stderr","text":["\u001b[32m[I 2021-11-01 06:22:12,660]\u001b[0m Trial 89 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 8, 'algorithm': 'ball_tree', 'leaf_size': 11, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,684]\u001b[0m Trial 90 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 7, 'algorithm': 'kd_tree', 'leaf_size': 23, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,709]\u001b[0m Trial 91 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 7, 'algorithm': 'kd_tree', 'leaf_size': 22, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,737]\u001b[0m Trial 92 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'ball_tree', 'leaf_size': 47, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,763]\u001b[0m Trial 93 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 10, 'algorithm': 'ball_tree', 'leaf_size': 39, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,792]\u001b[0m Trial 94 finished with value: 0.0 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 11, 'algorithm': 'ball_tree', 'leaf_size': 41, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,818]\u001b[0m Trial 95 finished with value: 0.06666666666666667 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 11, 'algorithm': 'ball_tree', 'leaf_size': 38, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,843]\u001b[0m Trial 96 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 9, 'algorithm': 'ball_tree', 'leaf_size': 43, 'metric': 'euclidean'}. Best is trial 5 with value: 0.0.\u001b[0m\n"]},{"output_type":"stream","name":"stdout","text":["Test Score: 0.9666666666666667\n","Train Score: 0.975\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.9666666666666667\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.975\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.9583333333333334\n","\n","=================\n","Test Score: 1.0\n","Train Score: 0.9833333333333333\n","\n","=================\n","Test Score: 0.9333333333333333\n","Train Score: 0.9666666666666667\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.975\n","\n","=================\n"]},{"output_type":"stream","name":"stderr","text":["\u001b[32m[I 2021-11-01 06:22:12,875]\u001b[0m Trial 97 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 10, 'algorithm': 'kd_tree', 'leaf_size': 34, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,920]\u001b[0m Trial 98 finished with value: 0.03333333333333333 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 8, 'algorithm': 'ball_tree', 'leaf_size': 40, 'metric': 'minkowski'}. Best is trial 5 with value: 0.0.\u001b[0m\n","\u001b[32m[I 2021-11-01 06:22:12,947]\u001b[0m Trial 99 finished with value: 0.1 and parameters: {'classifier': 'KNeighborsClassifier', 'n_neighbors': 6, 'algorithm': 'kd_tree', 'leaf_size': 25, 'metric': 'chebyshev'}. Best is trial 5 with value: 0.0.\u001b[0m\n"]},{"output_type":"stream","name":"stdout","text":["Test Score: 0.9666666666666667\n","Train Score: 0.9916666666666667\n","\n","=================\n","Test Score: 0.9666666666666667\n","Train Score: 0.9916666666666667\n","\n","=================\n","Test Score: 0.9\n","Train Score: 0.9833333333333333\n","\n","=================\n"]}]}]}