739 lines
22 KiB
Plaintext
739 lines
22 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7d017333",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Final Assessment Scratch Pad"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d3d00386",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Instructions"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ea516aa7",
|
|
"metadata": {},
|
|
"source": [
|
|
"1. Please use only this Jupyter notebook to work on your model, and **do not use any extra files**. If you need to define helper classes or functions, feel free to do so in this notebook.\n",
|
|
"2. This template is intended to be general, but it may not cover every use case. The sections are given so that it will be easier for us to grade your submission. If your specific use case isn't addressed, **you may add new Markdown or code blocks to this notebook**. However, please **don't delete any existing blocks**.\n",
|
|
"3. If you don't think a particular section of this template is necessary for your work, **you may skip it**. Be sure to explain clearly why you decided to do so."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "022cb4cd",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Report"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9c14a2d8",
|
|
"metadata": {},
|
|
"source": [
|
|
"**[TODO]**\n",
|
|
"\n",
|
|
"Please provide a summary of the ideas and steps that led you to your final model. Someone reading this summary should understand why you chose to approach the problem in a particular way and able to replicate your final model at a high level. Please ensure that your summary is detailed enough to provide an overview of your thought process and approach but also concise enough to be easily understandable. Also, please follow the guidelines given in the `main.ipynb`.\n",
|
|
"\n",
|
|
"This report should not be longer than **1-2 pages of A4 paper (up to around 1,000 words)**. Marks will be deducted if you do not follow instructions and you include too many words here. \n",
|
|
"\n",
|
|
"**[DELETE EVERYTHING FROM THE PREVIOUS TODO TO HERE BEFORE SUBMISSION]**\n",
|
|
"\n",
|
|
"##### Overview\n",
|
|
"**[TODO]**\n",
|
|
"\n",
|
|
"##### 1. Descriptive Analysis\n",
|
|
"**[TODO]**\n",
|
|
"\n",
|
|
"##### 2. Detection and Handling of Missing Values\n",
|
|
"**[TODO]**\n",
|
|
"\n",
|
|
"##### 3. Detection and Handling of Outliers\n",
|
|
"**[TODO]**\n",
|
|
"\n",
|
|
"##### 4. Detection and Handling of Class Imbalance \n",
|
|
"**[TODO]**\n",
|
|
"\n",
|
|
"##### 5. Understanding Relationship Between Variables\n",
|
|
"**[TODO]**\n",
|
|
"\n",
|
|
"##### 6. Data Visualization\n",
|
|
"**[TODO]** \n",
|
|
"##### 7. General Preprocessing\n",
|
|
"**[TODO]**\n",
|
|
" \n",
|
|
"##### 8. Feature Selection \n",
|
|
"**[TODO]**\n",
|
|
"\n",
|
|
"##### 9. Feature Engineering\n",
|
|
"**[TODO]**\n",
|
|
"\n",
|
|
"##### 10. Creating Models\n",
|
|
"**[TODO]**\n",
|
|
"\n",
|
|
"##### 11. Model Evaluation\n",
|
|
"**[TODO]**\n",
|
|
"\n",
|
|
"##### 12. Hyperparameters Search\n",
|
|
"**[TODO]**\n",
|
|
"\n",
|
|
"##### Conclusion\n",
|
|
"**[TODO]**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "49dcaf29",
|
|
"metadata": {},
|
|
"source": [
|
|
"---"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "27103374",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Workings (Not Graded)\n",
|
|
"\n",
|
|
"You will do your working below. Note that anything below this section will not be graded, but we might counter-check what you wrote in the report above with your workings to make sure that you actually did what you claimed to have done. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "0f4c6cd4",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Import Packages\n",
|
|
"\n",
|
|
"Here, we import some packages necessary to run this notebook. In addition, you may import other packages as well. Do note that when submitting your model, you may only use packages that are available in Coursemology (see `main.ipynb`)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 42,
|
|
"id": "cded1ed6",
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-28T06:46:52.407375Z",
|
|
"start_time": "2024-04-28T06:46:52.405317Z"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import pandas as pd\n",
|
|
"import os\n",
|
|
"import numpy as np"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "748c35d7",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Load Dataset\n",
|
|
"\n",
|
|
"The dataset `data.npy` consists of $N$ grayscale videos and their corresponding labels. Each video has a shape of (L, H, W). L represents the length of the video, which may vary between videos. H and W represent the height and width, which are consistent across all videos. \n",
|
|
"\n",
|
|
"A code snippet that loads the data is provided below."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c09da291",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Load Data"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 43,
|
|
"id": "6297e25a",
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-28T06:46:52.453152Z",
|
|
"start_time": "2024-04-28T06:46:52.428539Z"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Number of data sample: 2500\n",
|
|
"Shape of the first data sample: (10, 16, 16)\n",
|
|
"Shape of the third data sample: (8, 16, 16)\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"with open('data.npy', 'rb') as f:\n",
|
|
" data = np.load(f, allow_pickle=True).item()\n",
|
|
" X = data['data']\n",
|
|
" y = data['label']\n",
|
|
"\n",
|
|
"\n",
|
|
"print('Number of data sample:', len(X))\n",
|
|
"print('Shape of the first data sample:', X[0].shape)\n",
|
|
"print('Shape of the third data sample:', X[2].shape)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cbe832b6",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Data Exploration & Preparation"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2f6a464c",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 1. Descriptive Analysis"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 44,
|
|
"id": "3b1f62dd",
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-28T06:46:52.472306Z",
|
|
"start_time": "2024-04-28T06:46:52.454360Z"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"ename": "ValueError",
|
|
"evalue": "setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2250,) + inhomogeneous part.",
|
|
"output_type": "error",
|
|
"traceback": [
|
|
"\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
|
|
"\u001B[0;31mValueError\u001B[0m Traceback (most recent call last)",
|
|
"Cell \u001B[0;32mIn[44], line 10\u001B[0m\n\u001B[1;32m 8\u001B[0m X6 \u001B[38;5;241m=\u001B[39m np\u001B[38;5;241m.\u001B[39marray([video[:\u001B[38;5;241m6\u001B[39m] \u001B[38;5;28;01mfor\u001B[39;00m video \u001B[38;5;129;01min\u001B[39;00m X])\n\u001B[1;32m 9\u001B[0m \u001B[38;5;66;03m# Now that they are consistent, we can convert them to a numpy array\u001B[39;00m\n\u001B[0;32m---> 10\u001B[0m X6 \u001B[38;5;241m=\u001B[39m \u001B[43mnp\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43marray\u001B[49m\u001B[43m(\u001B[49m\u001B[43mX\u001B[49m\u001B[43m)\u001B[49m\n",
|
|
"\u001B[0;31mValueError\u001B[0m: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2250,) + inhomogeneous part."
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Remove nans from the input. This needs to be done in the model for training data as well\n",
|
|
"not_nan_indices = np.argwhere(~np.isnan(np.array(y))).squeeze()\n",
|
|
"y = [y[i] for i in not_nan_indices]\n",
|
|
"X = [X[i] for i in not_nan_indices]\n",
|
|
"y = np.array(y).astype(int)\n",
|
|
"\n",
|
|
"# Since each video varies in length, we will take the min length, 6, for each video\n",
|
|
"X6 = np.array([video[:6] for video in X])\n",
|
|
"# Now that they are consistent, we can convert them to a numpy array\n",
|
|
"X6 = np.array(X)\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"outputs": [],
|
|
"source": [
|
|
"pd.DataFrame(y).value_counts()\n",
|
|
"# From this, we know that we need to undersample or upsample the data. We will pick understampling as the data is quite large, and understampling will reduce the training time."
|
|
],
|
|
"metadata": {
|
|
"collapsed": false
|
|
},
|
|
"id": "dd66bb1efa4e602c",
|
|
"execution_count": null
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "adb61967",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 2. Detection and Handling of Missing Values"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "4bb9cdfb",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"np.isnan(X6).sum() # We know that there is quite a few NaNs in the data. However, I will not be figuring out which column / nan has this value. Instead we can just take the average of each image, adn use that as the input to the nan"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8adcb9cd",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 3. Detection and Handling of Outliers"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "ed1c17a1",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Check if there are outliers\n",
|
|
"# We can check if there are outliers by checking the max and min values of each video\n",
|
|
"np.max(X6, axis=3)\n",
|
|
"# From this we can see that there are values whic exceed 255, and thus, we can clip that."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d4916043",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 4. Detection and Handling of Class Imbalance"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 83,
|
|
"id": "ad3ab20e",
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-28T06:59:16.949196Z",
|
|
"start_time": "2024-04-28T06:59:16.943398Z"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": "0\n0 300\n1 300\n2 300\n3 300\n4 300\n5 300\nName: count, dtype: int64"
|
|
},
|
|
"execution_count": 83,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Handling Undersampling\n",
|
|
"pd.DataFrame(y).value_counts()\n",
|
|
"# There is a class imbalance, and we will need to undersample the data"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2552a795",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 5. Understanding Relationship Between Variables"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 44,
|
|
"id": "29ddbbcf",
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-28T06:46:52.478781Z",
|
|
"start_time": "2024-04-28T06:46:52.477156Z"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "757fb315",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 6. Data Visualization"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 44,
|
|
"id": "93f82e42",
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-28T06:46:52.483551Z",
|
|
"start_time": "2024-04-28T06:46:52.482068Z"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2a7eebcf",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Data Preprocessing"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ae3e3383",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 7. General Preprocessing"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"torch.Size([1800, 6, 16, 16])\n",
|
|
"(1800,)\n",
|
|
"<class 'numpy.ndarray'>\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"import torch\n",
|
|
"\n",
|
|
"# Reduce the data to 6 frames\n",
|
|
"X = np.array([video[:6] for video in X])\n",
|
|
"tensor_videos = torch.tensor(X, dtype=torch.float32)\n",
|
|
"# Clip values to 0 and 255\n",
|
|
"tensor_videos = np.clip(tensor_videos, 0, 255)\n",
|
|
"# Replace NaNs in each frame, with the average of the frame. This was generated with GPT\n",
|
|
"for i in range(tensor_videos.shape[0]):\n",
|
|
" for j in range(tensor_videos.shape[1]):\n",
|
|
" tensor_videos[i][j][torch.isnan(tensor_videos[i][j])] = torch.mean(tensor_videos[i][j][~torch.isnan(tensor_videos[i][j])])\n",
|
|
" \n",
|
|
"# Undersample the data for each of the 6 classes. Select max of 300 samples for each class\n",
|
|
"# Very much generated with the assitance of chatGPT with some modifications\n",
|
|
"# Get the indices of each class\n",
|
|
"indices = [np.argwhere(y == i).squeeze(1) for i in range(6)]\n",
|
|
"# Get the number of samples to take for each class\n",
|
|
"num_samples_to_take = 300\n",
|
|
"# Get the indices of the samples to take\n",
|
|
"indices_to_take = [np.random.choice(indices[i], num_samples_to_take, replace=True) for i in range(6)]\n",
|
|
"# Concatenate the indices\n",
|
|
"indices_to_take = np.concatenate(indices_to_take)\n",
|
|
"# Select the samples\n",
|
|
"tensor_videos = tensor_videos[indices_to_take]\n",
|
|
"y = y[indices_to_take]\n"
|
|
],
|
|
"metadata": {
|
|
"collapsed": false
|
|
},
|
|
"id": "19174365",
|
|
"execution_count": 82
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": "torch.Size([1800, 1, 6, 16, 16])"
|
|
},
|
|
"execution_count": 85,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# This is the extra channel dimention to work with the conv3d\n",
|
|
"tensor_videos = tensor_videos.unsqueeze(1)\n",
|
|
"tensor_videos.shape"
|
|
],
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-28T07:01:44.496557Z",
|
|
"start_time": "2024-04-28T07:01:44.492973Z"
|
|
}
|
|
},
|
|
"id": "8b6bcf332c355e9d",
|
|
"execution_count": 85
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fb3aa527",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 8. Feature Selection"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "a85808bf",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4921e8ca",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 9. Feature Engineering"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "dbcde626",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fa676c3f",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Modeling & Evaluation"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "589b37e4",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 10. Creating models"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 230,
|
|
"id": "d8dffd7d",
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-28T07:57:09.790124Z",
|
|
"start_time": "2024-04-28T07:57:09.780591Z"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from torch import nn\n",
|
|
"class CNN3D(nn.Module):\n",
|
|
" def __init__(self):\n",
|
|
" super(CNN3D, self).__init__()\n",
|
|
" self.conv1 = nn.Conv3d(1, 12, 2, 1,2)\n",
|
|
" self.mp = nn.AvgPool3d(2)\n",
|
|
" self.relu = nn.LeakyReLU()\n",
|
|
" self.fc1 = nn.Linear(3888, 6)\n",
|
|
" self.fc2 = nn.Linear(128, 6)\n",
|
|
" self.flatten = nn.Flatten()\n",
|
|
" def forward(self, x):\n",
|
|
" x = self.conv1(x)\n",
|
|
" x = self.mp(x)\n",
|
|
" x = self.relu(x)\n",
|
|
" \n",
|
|
" # print(x.shape)\n",
|
|
" \n",
|
|
" x = x.view(-1, 3888)\n",
|
|
" x = self.fc1(x)\n",
|
|
" # x = self.fc2(x)\n",
|
|
" return x\n",
|
|
" \n",
|
|
"def train(model, criterion, optimizer, loader, epochs = 10):\n",
|
|
" for epoch in range(epochs):\n",
|
|
" for idx, (inputs, labels) in enumerate(loader):\n",
|
|
" optimizer.zero_grad()\n",
|
|
" outputs = model(inputs)\n",
|
|
" loss = criterion(outputs, labels)\n",
|
|
" loss.backward()\n",
|
|
" optimizer.step()\n",
|
|
" print(f'Epoch {epoch}, Loss: {loss.item()}')\n",
|
|
" return model\n",
|
|
"def process_data(X, y):\n",
|
|
" y = np.array(y)\n",
|
|
" X = np.array([video[:6] for video in X])\n",
|
|
" tensor_videos = torch.tensor(X, dtype=torch.float32)\n",
|
|
" # Clip values to 0 and 255\n",
|
|
" tensor_videos = np.clip(tensor_videos, 0, 255)\n",
|
|
" # Replace NaNs in each frame, with the average of the frame. This was generated with GPT\n",
|
|
" for i in range(tensor_videos.shape[0]):\n",
|
|
" for j in range(tensor_videos.shape[1]):\n",
|
|
" tensor_videos[i][j][torch.isnan(tensor_videos[i][j])] = torch.mean(tensor_videos[i][j][~torch.isnan(tensor_videos[i][j])])\n",
|
|
" # Undersample the data for each of the 6 classes. Select max of 300 samples for each class\n",
|
|
" # Very much generated with the assitance of chatGPT with some modifications\n",
|
|
" # Get the indices of each class\n",
|
|
" indices = [np.argwhere(y == i).squeeze(1) for i in range(6)]\n",
|
|
" # Get the number of samples to take for each class\n",
|
|
" num_samples_to_take = 300\n",
|
|
" # Get the indices of the samples to take\n",
|
|
" indices_to_take = [np.random.choice(indices[i], num_samples_to_take, replace=True) for i in range(6)]\n",
|
|
" # Concatenate the indices\n",
|
|
" indices_to_take = np.concatenate(indices_to_take)\n",
|
|
" # Select the samples\n",
|
|
" tensor_videos = tensor_videos[indices_to_take].unsqueeze(1)\n",
|
|
" y = y[indices_to_take]\n",
|
|
" return torch.Tensor(tensor_videos), torch.Tensor(y).long()\n",
|
|
"class Model():\n",
|
|
" def __init__(self):\n",
|
|
" self.model = CNN3D()\n",
|
|
" self.criterion = nn.CrossEntropyLoss()\n",
|
|
" self.optimizer = torch.optim.Adam(self.model.parameters(), lr=0.001)\n",
|
|
" def fit(self, X, y):\n",
|
|
" X, y = process_data(X, y)\n",
|
|
" train_dataset = torch.utils.data.TensorDataset(X, y)\n",
|
|
" train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=32, shuffle=True)\n",
|
|
" train(self.model, self.criterion, self.optimizer, train_loader)\n",
|
|
" def predict(self, X):\n",
|
|
" self.model.eval()\n",
|
|
"\n",
|
|
" X = np.array([video[:6] for video in X])\n",
|
|
" tensor_videos = torch.tensor(X, dtype=torch.float32)\n",
|
|
" # Clip values to 0 and 255\n",
|
|
" tensor_videos = np.clip(tensor_videos, 0, 255)\n",
|
|
" # Replace NaNs in each frame, with the average of the frame. This was generated with GPT\n",
|
|
" for i in range(tensor_videos.shape[0]):\n",
|
|
" for j in range(tensor_videos.shape[1]):\n",
|
|
" tensor_videos[i][j][torch.isnan(tensor_videos[i][j])] = torch.mean(tensor_videos[i][j][~torch.isnan(tensor_videos[i][j])])\n",
|
|
" X = torch.Tensor(tensor_videos.unsqueeze(1))\n",
|
|
" return np.argmax(self.model(X).detach().numpy(), axis=1)\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "495bf3c0",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 11. Model Evaluation"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 217,
|
|
"id": "9245ab47",
|
|
"metadata": {
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-28T07:55:53.563103Z",
|
|
"start_time": "2024-04-28T07:55:53.544134Z"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from sklearn.model_selection import train_test_split\n",
|
|
"\n",
|
|
"with open('data.npy', 'rb') as f:\n",
|
|
" data = np.load(f, allow_pickle=True).item()\n",
|
|
" X = data['data']\n",
|
|
" y = data['label']\n",
|
|
"\n",
|
|
"\n",
|
|
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1)\n",
|
|
"\n",
|
|
"not_nan_indices = np.argwhere(~np.isnan(np.array(y_test))).squeeze()\n",
|
|
"y_test = [y_test[i] for i in not_nan_indices]\n",
|
|
"X_test = [X_test[i] for i in not_nan_indices]\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Epoch 0, Loss: 4.225716590881348\n",
|
|
"Epoch 1, Loss: 0.9198675155639648\n",
|
|
"Epoch 2, Loss: 1.7365752458572388\n",
|
|
"Epoch 3, Loss: 0.4570190906524658\n",
|
|
"Epoch 4, Loss: 0.11014104634523392\n",
|
|
"Epoch 5, Loss: 0.24420055747032166\n",
|
|
"Epoch 6, Loss: 0.03079795092344284\n",
|
|
"Epoch 7, Loss: 0.07790327817201614\n",
|
|
"Epoch 8, Loss: 0.07603466510772705\n",
|
|
"Epoch 9, Loss: 0.04154537618160248\n",
|
|
"F1 Score (macro): 0.51\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"model = Model()\n",
|
|
"model.fit(X_train, y_train)\n",
|
|
"\n",
|
|
"from sklearn.metrics import f1_score\n",
|
|
"\n",
|
|
"y_pred = model.predict(X_test)\n",
|
|
"print(\"F1 Score (macro): {0:.2f}\".format(f1_score(y_test, y_pred, average='macro'))) # You may encounter errors, you are expected to figure out what's the issue.\n"
|
|
],
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-28T07:57:38.644155Z",
|
|
"start_time": "2024-04-28T07:57:35.958882Z"
|
|
}
|
|
},
|
|
"id": "abb2d957f4a15bd2",
|
|
"execution_count": 235
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"F1 Score (macro): 0.60\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"\n"
|
|
],
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"ExecuteTime": {
|
|
"end_time": "2024-04-28T07:57:16.355215Z",
|
|
"start_time": "2024-04-28T07:57:16.281540Z"
|
|
}
|
|
},
|
|
"id": "37ff28a8da9dba6c",
|
|
"execution_count": 232
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8aa31404",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 12. Hyperparameters Search"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "81addd51",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.9.18"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|