yadunut/nus
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

maybe good?

Browse Source
This commit is contained in:
Yadunand Prem 2024-04-28 15:58:30 +08:00
parent 1312e694c3
commit d2e87aec97
No known key found for this signature in database
21 changed files with 3097 additions and 700 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
cs2109s/labs/environment.yml
View File

@ -11,5 +11,5 @@ dependencies:
- scikit-learn=1.0.2 - scikit-learn=1.0.2
- seaborn=0.12.0 - seaborn=0.12.0
- timeout-decorator=0.5.0 - timeout-decorator=0.5.0
- pytorch=1.11.0 - pytorch=1.12.1
- torchvision=0.12.0 - torchvision=0.12.0

605
cs2109s/labs/final 2/main.ipynb Normal file
View File

@ -0,0 +1,605 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "7d017333",
"metadata": {},
"source": [
"# Final Assessment: Making Prediction on a Dataset without Domain Knowledge"
]
},
{
"cell_type": "markdown",
"id": "09648cba",
"metadata": {},
"source": [
"**Release Date:** Saturday, 27 April 2024, 20:00\n",
"\n",
"**Due Date:** Sunday, 28 April 2024, 23:59"
]
},
{
"cell_type": "markdown",
"id": "022cb4cd",
"metadata": {},
"source": [
"## Overview\n",
"\n",
"In this assessment, your goal is to put into practice what you have learnt over the semester by building a model that can accurately predict the outcomes for new data samples from the same distribution from a given data set. Because the dataset's origin and domain are unknown, you cannot rely on any prior domain knowledge. Instead, you will need to analyze and comprehend the data based solely on its inherent characteristics and the given training samples. You may employ any techniques to uncover the data's nature, such as visualization, analysis, or experimentation.\n",
"\n",
"In general, the techniques learnt in class should be sufficient, but you may choose to explore new ones. For fairness and for practical reasons, your solution will be implemented with the libraries that we have used in CS2109S. You may find \"[ML for People in Hurry](https://colab.research.google.com/drive/1yHecT3sXevjdko9KmRSIOOJxXdf43xd6?usp=sharing)\" shared in Lecture 12 useful.\n",
"\n",
"Once you have some understanding of the data, you can create a model (or a set of models) that takes into account the data's characteristics. Your model can perform a series of preprocessing steps before training or making predictions. You will need to contemplate how you will train your model, what objective you will use, and how to evaluate its performance properly. Additionally, you may want to conduct a hyperparameter search to enhance your model's performance.\n",
"\n",
"There is no model answer to the problem. There are many possible approaches and many of them might perform equally well. \n",
"\n",
"### Constraints:\n",
"\n",
"* You are only allowed to use <span style=\"color:red\">**neural networks**</span> as the hypothesis class of your supervised learning models.\n",
"* Your model is allowed to take <span style=\"color:red\">**no more than 3 minutes**</span> on the Coursemology server and use <span style=\"color:red\">**at most 1 GB of memory**</span>.\n",
"* <span style=\"color:red\">**WARNING:</span> The use of any other hypothesis class is <span style=\"color:red\">strictly prohibited</span> and the failure to comply with this requirement <span style=\"color:red\">will result in zero marks for this assessment**</span>.\n",
"* You are allowed and you might find it useful to use unsupervised learning methods such as PCA and K-means clustering to preprocess your data, but preprocessing is optional.\n",
"\n",
"### Required Files:\n",
"\n",
"* `main.ipynb` (this Jupyter notebook)\n",
"* `scratchpad.ipynb` (Jupyter notebook for your \"scratch paper\") \n",
"* `environment.yml` (environment file)\n",
"* `data.npy` (dataset)\n",
"\n",
"\n",
"\n",
"### Policy on Citing AI:\n",
"\n",
"As explained in lecture, we have little choice but to allow the use of AI. Here are some guidelines for citing AI-generated content:\n",
"\n",
"**ChatGPT**: When referencing content generated by ChatGPT, Bing, or something similar, please include the link(s) to the conversation(s) in `scratchpad.ipynb` as part of the supporting evidence for your work.\n",
"\n",
"**GitHub Copilot**: For GitHub Copilot or something similar, please include one of the following in `scratchpad.ipynb`: (i) If your code was generated based on a prompt, include the prompt used along with a link to the generated code (if possible). (ii) If the code was produced through autocomplete without a specific prompt, kindly provide a screenshot of the autocomplete suggestion.\n",
"\n",
"We have caught students in the past attempting to explain away instances of plagiarism by falsely claiming that they received input from ChatGPT. It is important to note that these attempts were identified and flagged by our plagiarism checker. **Disciplinary action was taken against these students** for their actions.\n",
"\n",
"We would like to remind all students that academic integrity is of utmost importance, especially during final assessment. Any attempt to plagiarize will result in **severe disciplinary action**. We urge you to complete this final assessment with honesty and integrity, and refrain from resorting to any unethical practices.\n",
"\n",
"Please be aware that **our plagiarism detection tools are highly effective** and any attempt to cheat or plagiarize will likely be detected. It is in your best interest to demonstrate your own knowledge and skills in completing the final assessment.\n",
"\n",
"### Honour Code: \n",
"\n",
"* Note that plagiarism will not be condoned! \n",
"* Also, because this is an individual assessment, you **MUST NOT** discuss your approach or solution with your classmates. If you are caught doing so, you will be subject to disciplinary action because it will considered an act of academic dishonestly. \n",
"* You may check the internet for references, but if you submit code that is copied directly and subsequently modified from an online source, you need to provide us with the reference and URL! For good measure, please cite all references in your final solution write-up and include all your working in `scratchpad.ipynb`.\n",
"\n",
"### Latest Updates, Announcements, and Clarifications:\n",
"\n",
"We recommend regularly checking the [official forum thread for the final exam](https://coursemology.org/courses/2714/forums/homework/topics/official-final-assessment) to stay up-to-date with the latest updates and announcements. Additionally, you can use this thread to request clarifications regarding any aspect of the exam."
]
},
{
"cell_type": "markdown",
"id": "ef4d73fd",
"metadata": {},
"source": [
"## Jupyter Notebooks\n",
"\n",
"In this final assessment, you will be working with two Jupyter notebooks: `main.ipynb` and `scratchpad.ipynb`.\n",
"\n",
"### `main.ipynb`\n",
"\n",
"This notebook serves as the primary guide to completing the final exam. It includes instructions, guidelines, and a final model code template that you must complete and copy-paste into Coursemology for submission. You can think of this as the question paper. \n",
"\n",
"It is important to note that `main.ipynb` **should not be uploaded**.\n",
"\n",
"### `scratchpad.ipynb`\n",
"\n",
"`scratchpad.ipynb` is your \"scratch paper,\" where you will perform various analyses and transformations on the data we provided you with and experiment with different techniques. You can think of this as your answer booklet. \n",
"\n",
"It is important to document your thoughts in this notebook so that we can understand how you arrived at your final model. Note that your working will not be graded directly. Instead, you will have to summarize your work in a report of not more than 1,000 words. You should follow the format/sections given to help make the grading easier for the profs. Some sections are optional. If they are not applicable, just indicate \"Nil.\"\n",
"\n",
"Mostly, the working submitted will be used as a sanity check that you actually did the work you claim to have done, instead of copying it from some online source, or worse, copied from another student. If we find 2 submissions are suspiciously similar, we will check the working for the 2 students. It is highly implausible that 2 students will come up with the same working if they did not discuss their approach/answers. \n",
"\n",
"Once you have found your best model, you should copy and paste the model and its necessary components from `scratchpad.ipynb` to Coursemology for submission. It is important to **ensure that your model adheres to the model code template provided in `main.ipynb`**. You need to make sure that the code you submit in Coursemology is self-contained. We strongly encourage you to test your model locally using `main.ipynb` by copying the necessary parts from your scratchpad to `main.ipynb` and check that your model runs correctly, to avoid wasting your attempt on Coursemology. You will be giving only a limited number of attempts on Coursemology because it is extreme memory-intensive (aka expensive) to run these models and we don't want students to be spamming our servers. \n",
"\n",
"**Remember to upload `scratchpad.ipynb` when submitting your final exam.** "
]
},
{
"cell_type": "markdown",
"id": "c51cc898",
"metadata": {},
"source": [
"## Compute Resources\n",
"\n",
"### IMPORTANT: Limitations to Compute Resources\n",
"\n",
"At some level, machine learning can generally do better if we throw more resources into the problem. However for reasons of fairness and practicality, we need to limit the compute resources that you can use for this problem. The way that we decide to quantify this is to limit your model to take <span style=\"color:red\">**no more than 3 minutes**</span> on the Coursemology server and uses <span style=\"color:red\">**at most 1 GB of memory**</span>. The Coursemology server also **does not have a GPU,** so you should not use GPU in your model. You should also assume that your model will have access to only 1 CPU and not include multi-processor code in your submission. This means that you should start off with a simple model and then gradually increase the complexity of your model to make sure that you stay within the allocated compute resources.\n",
"\n",
"These limitations exist for 2 reasons: (i) first, we need to contain the costs of running this Final Assessment; and (ii) we need to ensure a fair playing field for all students. We need to make sure no student performs better than other students simply because of access to faster machines or more resources. \n",
"\n",
"### Available Compute Resources\n",
"\n",
"For this assessment, you will need access to some computing resources to run your code and train your machine learning models. Two popular options are Google Colaboratory and the School of Computing Compute Cluster.\n",
"\n",
"* **[Google Collaboratory](https://colab.research.google.com/)**, or \"Colab\" for short, is a free cloud-based platform provided by Google that allows you to write and run Python code using a Jupyter notebook interface. Colab provides access to a virtual machine with a GPU and sometimes even a TPU, which can speed up computation for tasks like training machine learning models. You can use Colab on your own computer without installing any software, and it provides access to a number of libraries and datasets. However, there may be limits on how much time, memory, and storage space you can use, and you may need to reauthorize your session frequently.\n",
"\n",
"* **[The School of Computing Compute Cluster](https://dochub.comp.nus.edu.sg/cf/guides/compute-cluster/start)** is a set of high-performance computing resources that are available to students, faculty, and researchers affiliated with the National University of Singapore's School of Computing. The cluster consists of multiple nodes, each with its own set of CPUs, memory, and storage. You can submit jobs to the cluster using the [Slurm workload manager](https://slurm.schedmd.com/documentation.html), which allocates resources to jobs based on availability and user-specified requirements. The Compute Cluster provides significantly more computing power than Colab, with the ability to scale up to hundreds or even thousands of cores. However, you need to apply for access to the cluster, and there may be limits on the amount of resources that can be used at any given time. Additionally, using the cluster requires some technical expertise and familiarity with the Linux command line interface.\n",
"\n",
"If you prefer not to use Google Colaboratory or the School of Computing Compute Cluster, you can also run your code on your own computer. However, keep in mind that your computer may not have as much processing power or memory as the other options, so your code may run more slowly and you will take more time to complete certain tasks."
]
},
{
"cell_type": "markdown",
"id": "7e323c3c",
"metadata": {},
"source": [
"## Scratch Pad"
]
},
{
"cell_type": "markdown",
"id": "bd2d0eaf",
"metadata": {},
"source": [
"The following are brief descriptions of each step in `scratchpad.ipynb`. Please note that the scratch pad only contains generic steps, and depending on the specific data you are working with, not all steps may be necessary. As a result, some sections of the notebook may potentially be left empty/blank (you can indicate \"Nil\").\n",
"\n",
"You should probably also limit the amount of time that you invest into each of the steps to avoid rushing at the end. We expect students to take abour 3-4 hours to complete this assessment and the suggested amount of times for each of the steps is given as a guide. You do not need to adhere strictly to our suggestions. \n",
"\n",
"You should include all your \"workings\" in scratchpad. Although you will only be graded on your 1,000-word executive summary, we might refer to your workings if there are concerns about plagiarism.\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "cbe832b6",
"metadata": {},
"source": [
"### Data Exploration & Preparation (60 mins)\n",
"\n",
"Before starting to create a model, it is important to explore and analyze the characteristics of the data. This can help us make informed decisions regarding the choice of approach, techniques, and models that we will use.\n",
"\n",
"When dealing with data, it is essential to understand the format in which it is presented. In machine learning, tabular data is typically provided in the form of a Pandas DataFrame, while tensor data, such as images, is given in the form of a Numpy ndarray.\n",
"\n",
"To help you get started with these data formats, the following guides can be useful:\n",
"\n",
"* [10 Minutes to pandas](https://pandas.pydata.org/docs/user_guide/10min.html): This guide provides a quick introduction to the Pandas library, including its key features and how to work with DataFrames.\n",
"* [Numpy Quickstart](https://numpy.org/doc/stable/user/quickstart.html): This guide covers the basics of the Numpy library and how to work with ndarrays, including creating arrays, indexing and slicing, and mathematical operations.\n",
"\n",
"**It is important to note that you have the option to skip most of these data exploration and preparation steps and mostly use the dataset as is (with a rudimentary preprocessing)**. Given that machine learning is machine learning, such a naive approach work (i.e. give you some answer). However, such an approach is unlikely to yield the best outcomes.\n",
"\n",
"\n",
"#### 1. Descriptive Analysis (5 mins)\n",
"\n",
"Descriptive analysis is used to understand the basic characteristics of the data. This includes analyzing the distribution of the data, measuring its central tendency (i.e., mean, median, mode), and checking the variability of the data (i.e., range, standard deviation, variance, interquartile range). This analysis can give us an overview of the data and help us to identify any potential issues or challenges that may need to be addressed.\n",
"\n",
"You may find the following resources helpful:\n",
"* [Pandas: how to calculate statistics?](https://pandas.pydata.org/docs/getting_started/intro_tutorials/06_calculate_statistics.html)\n",
"* [Numpy statistics](https://numpy.org/doc/stable/reference/routines.statistics.html)\n",
"\n",
"#### 2. Detection and Handling of Missing Values (10 mins)\n",
"\n",
"Missing values in the data can cause problems with our machine learning algorithms and may need to be handled. Detecting and handling missing values involves checking if there are any missing values in the data and figuring out the best way to handle them if necessary. This may involve imputing missing values with a certain value or method, or removing the rows or columns that contain missing values. You can follow the \"[10 Minutes to pandas](https://pandas.pydata.org/docs/user_guide/10min.html)\" guide on how to do this.\n",
"\n",
"#### 3. Detection and Handling of Outliers (10 mins)\n",
"\n",
"Outliers are data points that are significantly different from the majority of the data. They can have a significant impact on the performance of your model, and it is important to detect and handle them appropriately. For example, you may use statistical methods such as the interquartile range (IQR) and z-score to detect outliers. Once you have detected outliers, you need to decide how to handle them. After you found them, you can choose to remove them from the dataset, replace them with a more appropriate value (e.g., the mean or median), or leave them in the dataset and use a model that is robust to outliers. You may find [this guide](https://www.geeksforgeeks.org/detect-and-remove-the-outliers-using-python/) useful.\n",
"\n",
"#### 4. Detection and Handling of Class Imbalance (5 mins)\n",
"\n",
"Class imbalance is a problem that occurs when one class has significantly more instances than another. This can make it difficult to build a model that accurately predicts the minority class. Investigating the possibility of class imbalance and figuring out the best way to handle it if necessary is important. This may involve techniques such as oversampling or undersampling the minority class (as seen in PS5), using cost-sensitive learning (as seen in Tutorial 8), or other methods which you can explore yourself.\n",
"\n",
"#### 5. Understanding Relationship Between Variables (20 mins)\n",
"\n",
"Analyzing the relationship between variables in a dataset can reveal potential dependencies and offer insights for building accurate models. There are different ways to explore these dependencies:\n",
"\n",
"* **Linear dependencies:** It's possible to identify linear dependencies by verifying if certain attributes are multiples of other attributes by a constant factor.\n",
"\n",
"* **Correlations:** Another approach is to measure the correlations between variables, which indicates whether certain variables influence others. For instance, high correlation between the target variable and attributes A, B, and C suggests that A, B, and C are important factors for the target variable. Conversely, low correlation implies that they are not critical. [You can measure correlation using pandas](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.corr.html).\n",
"\n",
"By performing this analysis, we can determine which variables are most relevant to the problem and should be considered in our model. This can help us build a more effective and accurate machine learning model.\n",
"\n",
"#### 6. Data Visualization (10 mins)\n",
"\n",
"Visualizing the data can help us to see patterns that are visible to human eyes. Data visualization techniques can include scatter plots, histograms, heat maps, and other graphical methods. Visualization can be particularly useful when trying to identify relationships between variables or patterns in the data.\n",
"\n",
"You may use libraries such as [Matplotlib](https://matplotlib.org/stable/tutorials/introductory/pyplot.html) or [Seaborn](https://seaborn.pydata.org/tutorial/introduction.html) to do this.\n",
"\n",
"### Data Preprocessing (70 mins)\n",
"\n",
"Data preprocessing involves cleaning and transforming the data to maximize learning and prediction performance. This can include removing irrelevant variables, normalizing the data, scaling the data, or transforming the data using mathematical techniques.\n",
"\n",
"Scikit-learn website provides a [short guide](https://scikit-learn.org/stable/modules/preprocessing.html) on how to do data preprocessing using their library.\n",
"\n",
"**It is important to note that you can choose to skip more of the steps in feature selection and feature engineering and use all the features in the data set as is**. However, doing so may not yield the best outcomes. \n",
"\n",
"#### 7. General Preprocessing (10 mins)\n",
"\n",
"General preprocessing involves any other preprocessing that is necessary for the data, such as converting the data type of certain attributes or removing duplicates. think of this as implementing what needs to be done based on what you learnt in Step 2 above. \n",
"\n",
"#### 8. Feature Selection (30 mins)\n",
"\n",
"Feature selection is an important step in machine learning that involves identifying a subset of features that are most relevant to the problem. This helps to reduce the dimensionality of the data and improve the accuracy of the models. There are different techniques for feature selection, including:\n",
"\n",
"* **Removing uninformative features:** This involves removing features that are not useful for the task at hand. Two common methods to identify uninformative features are:\n",
" * Linearly dependent: features that are linear combinations of other features can be removed since they add no new information. A linear dependence test can be applied to identify linearly dependent features.\n",
" * Low or no correlation: features that have low or no correlation with the target variable can also be removed as they do not provide valuable information. A correlation analysis can be performed to identify such features \n",
" * See Understanding Relationship Between Variables on the explanation regarding the analysis.\n",
"\n",
"* **Sequentially removing features:** This involves iteratively removing the least significant feature until a desired number of features is reached. The idea is to remove features that have the least impact on the performance of the model. [Learn how to do this using Scikit-learn](https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SequentialFeatureSelector.html).\n",
"\n",
"* **Principal component analysis:** This is a dimensionality reduction technique that involves transforming the data into a new set of orthogonal variables, called principal components, that capture the most important information in the original data. By selecting a subset of these components, we can reduce the dimensionality of the data while retaining most of the information. [Learn how to do this using Scikit-learn](https://scikit-learn.org/stable/auto_examples/decomposition/plot_pca_iris.html).\n",
"\n",
"By applying feature selection techniques, we can identify the most relevant features for the problem and improve the accuracy and efficiency of our machine learning models.\n",
"\n",
"\n",
"#### 9. Feature Engineering (30 mins)\n",
"\n",
"Feature engineering is the process of creating new features from existing ones to improve the performance of machine learning models. It involves identifying and extracting useful information from the data by applying various techniques such as:\n",
"\n",
"* **Combining features:** This involves combining two or more existing features to create a new feature that captures more information than either of the original features alone. For example, if we have two features representing length and width, we can create a new feature that represents area by multiplying the two. \n",
"\n",
"* **Creating new features:** This involves creating new features from the existing ones using domain knowledge or other insights gained from the data analysis. For example, if we have a dataset of customer transactions, we can create new features such as total spending per customer or the number of items purchased in a single transaction. \n",
"\n",
"* **Mapping functions to features:** This involves applying mathematical functions to the features to create new features with higher-order terms or interactions between features. For example, we can create polynomial features by mapping a feature x to x^2 or x^3.\n",
"\n",
"By applying feature engineering techniques, we can create more informative features that capture the underlying patterns and relationships in the data, leading to better performance in the machine learning models, since compute resources are limited. \n",
"\n",
"\n",
"### Modeling & Evaluation (110 mins)\n",
"\n",
"After completing the above steps, it is time to build and evaluate models. This can involve creating a set of models that best fit the nature of the data, performing model evaluation, and doing hyperparameters search.\n",
"\n",
"The models should be evaluated thoroughly, and the best one should be chosen for submission.\n",
"\n",
"#### 10. Creating Models (30 mins)\n",
"\n",
"In this stage, we create models that are appropriate for the data. Since you are only allowed to use neural networks as the hypothesis class, then the choice of models boils down to the choice of the architectures of the neural networks. For example, you can use feedforward neural networks, convolutional neural networks, recurrent neural networks, and so on. You can even combine multiple architecture together to form a new architecture. Depending on the nature of the data, we can choose one or more of these models to build. We should be careful in selecting the models to ensure that they are suitable for the task we want to accomplish.\n",
"\n",
"Utilizing pre-built models from [PyTorch](https://pytorch.org/docs/stable/nn.html) can be beneficial. This library offers an extensive range of models that can be easily implemented and integrated. \n",
"\n",
"However, if needed, you can also create your own models and algorithms from scratch.\n",
"\n",
"#### 11. Model Evaluation (30 mins)\n",
"\n",
"Once we have created our models, we need to evaluate them to determine their performance. We should use a variety of metrics to evaluate the performance of each model, such as accuracy, precision, recall, F1 score, ROC curve, AUC, and so on. We should also use appropriate techniques to validate the models, such as cross-validation, train-test split, or hold-out validation. By doing this, we can determine which model is the best fit for our data. \n",
"\n",
"It's important to consider multiple models in the evaluation process to ensure that we are choosing the best one for our data. We may create and evaluate several models before selecting the best one. We should also consider the trade-offs between model complexity and accuracy to make sure that it can run in Coursemology.\n",
"\n",
"#### 12. Hyperparameters Search (50 mins)\n",
"\n",
"After choosing a model, we should optimize its hyperparameters to achieve the best performance. Hyperparameters are parameters that are not learned during training, such as the learning rate, the number of hidden layers (if applicable), or the regularization coefficient (if applicable). We can use various methods to search for the optimal hyperparameters, such as grid search, random search, or Bayesian optimization. The choice of the method depends on the complexity of the model and the size of the dataset. By tuning the hyperparameters, we can improve the performance of the model and make it more robust.\n",
"\n",
"There are many libraries to do hyperparameter search. You browse for them on [GitHub](https://github.com/topics/hyperparameter-optimization).\n",
"\n",
"In addition to using optimization libraries and functions, you can also manually perform simple hyperparameter tuning. This involves adjusting the hyperparameters of your model and evaluating its performance repeatedly until the best combination is achieved. However, keep in mind that manual tuning can be time-consuming and may not be as effective as more advanced techniques."
]
},
{
"cell_type": "markdown",
"id": "9045d90d",
"metadata": {},
"source": [
"## Tasks & Submission"
]
},
{
"cell_type": "markdown",
"id": "c950686e",
"metadata": {},
"source": [
"### Task 1: Model Implementation (80% Marks)\n",
"\n",
"Implement your model that you want to submit by completing the following functions:\n",
"* `__init__`: The constructor for Model class.\n",
"* `fit`: Fit/train the model using the input data. You may perform data handling and preprocessing here before training your model.\n",
"* `predict`: Predict using the model. If you perform data handling and preprocessing in the `fit` function, then you may want to do the same here.\n",
"\n",
"#### Dependencies\n",
"\n",
"It is crucial to note that your model may rely on specific versions of Python packages, including:\n",
"\n",
"* Python 3.10\n",
"* Numpy version 1.23\n",
"* Pandas version 1.4\n",
"* Scikit-Learn version 1.1\n",
"* PyTorch version 1.12\n",
"* Torchvision version 0.13\n",
"\n",
"To prevent any compatibility issues or unexpected errors during the execution of your code, ensure that you are using the correct versions of these packages. You can refer to `environment.yml` for a comprehensive list of packages that are pre-installed in Coursemology and can be used by your model. Note that if you do end up using libraries that are not installed on Coursemology, you might see an error like:\n",
"\n",
"\"Your code failed to evaluate correctly. There might be a syntax error, or perhaps execution failed to complete within the allocated time and memory limits.\"\n",
"\n",
"#### Model Template\n",
"\n",
"Note that you should copy and paste the code below *directly* into Coursemology for submission. You should probably test the code in this notebook on your local machine before uploading to Coursemology and using up an attempt.\n",
" "
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "a44b7aa4",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-28T04:16:20.742674Z",
"start_time": "2024-04-28T04:16:20.719852Z"
}
},
"outputs": [],
"source": [
"import torch\n",
"from torch import nn\n",
"class CNN(nn.Module):\n",
" def __init__(self, num_classes):\n",
" super(CNN, self).__init__()\n",
"\n",
" self.conv1 = nn.Conv2d(1,32,3,stride=1,padding=0)\n",
" self.conv2 = nn.Conv2d(32,64,3,stride=1,padding=0)\n",
" self.relu = nn.ReLU()\n",
" self.maxpool = nn.MaxPool2d(2)\n",
" self.fc1 = nn.Linear(256, 128) # Calculate input size based on output from conv2 and pooling\n",
" self.fc2 = nn.Linear(128, num_classes)\n",
" self.flatten = nn.Flatten()\n",
"\n",
" def forward(self, x):\n",
" x = self.conv1(x)\n",
" x = self.relu(x)\n",
" x = self.maxpool(x)\n",
" x = self.conv2(x)\n",
" x = self.relu(x)\n",
" x = self.maxpool(x)\n",
" x = self.flatten(x)\n",
" x = self.fc1(x)\n",
" x = self.relu(x)\n",
" x = self.fc2(x)\n",
" return x\n",
"\n",
"# video is a numpy array of shape (L, H, W)\n",
"def clean_batch(batch):\n",
" batch = np.array(batch)\n",
" print(batch.shape)\n",
" temp_x = batch.reshape(-1, 256)\n",
" np.nan_to_num(temp_x, copy=False)\n",
" col_mean = np.nanmean(temp_x, axis=0)\n",
" inds = np.where(np.isnan(temp_x))\n",
" temp_x[inds] = np.take(col_mean, inds[1])\n",
" temp_x = np.clip(temp_x, 1, 255)\n",
" batch = temp_x.reshape(-1, 1, 16,16)\n",
" return torch.tensor(batch, dtype=torch.float32)\n",
"def flatten_data(X, y):\n",
" not_nan_indices = np.argwhere(~np.isnan(np.array(y))).squeeze()\n",
" y = [y[i] for i in not_nan_indices]\n",
" X = [X[i] for i in not_nan_indices]\n",
" flattened_x = []\n",
" flattened_y = []\n",
" for idx, video in enumerate(X):\n",
" for frame in video:\n",
" flattened_x.append(frame)\n",
" flattened_y.append(y[idx])\n",
" flattened_x = clean_batch(flattened_x)\n",
" return flattened_x, torch.Tensor(np.array(flattened_y, dtype=np.int64)).long()\n",
"\n",
"class Model():\n",
" def __init__(self):\n",
" self.cnn = CNN(6)\n",
" def fit(self, X, y):\n",
" self.cnn.train()\n",
" X, y = flatten_data(X, y)\n",
" train_dataset = torch.utils.data.TensorDataset(X, y)\n",
" train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=320, shuffle=True)\n",
" criterion = nn.CrossEntropyLoss()\n",
" optimizer = torch.optim.Adam(self.cnn.parameters(), lr=0.001)\n",
" for epoch in range(70):\n",
" for idx, (inputs, labels) in enumerate(train_loader):\n",
" optimizer.zero_grad()\n",
" outputs = self.cnn(inputs)\n",
" loss = criterion(outputs, labels)\n",
" loss.backward()\n",
" optimizer.step()\n",
" print(f'Epoch {epoch}, Loss: {loss.item()}')\n",
" return self\n",
" def predict(self, X):\n",
" self.cnn.eval()\n",
" results = []\n",
" for idx, batch in enumerate(X):\n",
" batch = clean_batch(batch)\n",
" pred = self.cnn(batch)\n",
" result = torch.argmax(pred, axis=1)\n",
" results.append(torch.max(result))\n",
" return results\n"
]
},
{
"cell_type": "markdown",
"id": "e02178d7",
"metadata": {},
"source": [
"#### Local Evaluation\n",
"\n",
"You may test your solution locally by running the following code. Do note that the results may not reflect your performance in Coursemology. You should not be submitting the code below in Coursemology. The code here is meant only for you to do local testing."
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "4f4dd489",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-28T04:09:59.741093Z",
"start_time": "2024-04-28T04:09:59.732247Z"
}
},
"outputs": [],
"source": [
"# Import packages\n",
"import pandas as pd\n",
"import numpy as np\n",
"import os\n",
"from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, mean_squared_error, mean_absolute_error, r2_score\n",
"from sklearn.model_selection import train_test_split"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "3064e0ff",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-28T04:10:00.718747Z",
"start_time": "2024-04-28T04:10:00.689200Z"
}
},
"outputs": [],
"source": [
"# Load data\n",
"with open('data.npy', 'rb') as f:\n",
" data = np.load(f, allow_pickle=True).item()\n",
" X = data['data']\n",
" y = data['label']"
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "27c9fd10",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-28T04:18:50.184449Z",
"start_time": "2024-04-28T04:18:43.527661Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(16189, 16, 16)\n",
"Epoch 0, Loss: 1.0775409936904907\n",
"Epoch 1, Loss: 1.279036283493042\n",
"Epoch 2, Loss: 1.0776251554489136\n"
]
},
{
"ename": "KeyboardInterrupt",
"evalue": "",
"output_type": "error",
"traceback": [
"\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
"\u001B[0;31mKeyboardInterrupt\u001B[0m Traceback (most recent call last)",
"Cell \u001B[0;32mIn[21], line 15\u001B[0m\n\u001B[1;32m 13\u001B[0m \u001B[38;5;66;03m# Train and predict\u001B[39;00m\n\u001B[1;32m 14\u001B[0m model \u001B[38;5;241m=\u001B[39m Model()\n\u001B[0;32m---> 15\u001B[0m \u001B[43mmodel\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mfit\u001B[49m\u001B[43m(\u001B[49m\u001B[43mX_train\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43my_train\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 16\u001B[0m y_pred \u001B[38;5;241m=\u001B[39m model\u001B[38;5;241m.\u001B[39mpredict(X_test)\n\u001B[1;32m 18\u001B[0m \u001B[38;5;66;03m# Evaluate model predition\u001B[39;00m\n\u001B[1;32m 19\u001B[0m \u001B[38;5;66;03m# Learn more: https://scikit-learn.org/stable/modules/classes.html#module-sklearn.metrics\u001B[39;00m\n",
"Cell \u001B[0;32mIn[18], line 68\u001B[0m, in \u001B[0;36mModel.fit\u001B[0;34m(self, X, y)\u001B[0m\n\u001B[1;32m 66\u001B[0m outputs \u001B[38;5;241m=\u001B[39m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mcnn(inputs)\n\u001B[1;32m 67\u001B[0m loss \u001B[38;5;241m=\u001B[39m criterion(outputs, labels)\n\u001B[0;32m---> 68\u001B[0m \u001B[43mloss\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mbackward\u001B[49m\u001B[43m(\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 69\u001B[0m optimizer\u001B[38;5;241m.\u001B[39mstep()\n\u001B[1;32m 70\u001B[0m \u001B[38;5;28mprint\u001B[39m(\u001B[38;5;124mf\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mEpoch \u001B[39m\u001B[38;5;132;01m{\u001B[39;00mepoch\u001B[38;5;132;01m}\u001B[39;00m\u001B[38;5;124m, Loss: \u001B[39m\u001B[38;5;132;01m{\u001B[39;00mloss\u001B[38;5;241m.\u001B[39mitem()\u001B[38;5;132;01m}\u001B[39;00m\u001B[38;5;124m'\u001B[39m)\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/_tensor.py:522\u001B[0m, in \u001B[0;36mTensor.backward\u001B[0;34m(self, gradient, retain_graph, create_graph, inputs)\u001B[0m\n\u001B[1;32m 512\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m has_torch_function_unary(\u001B[38;5;28mself\u001B[39m):\n\u001B[1;32m 513\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m handle_torch_function(\n\u001B[1;32m 514\u001B[0m Tensor\u001B[38;5;241m.\u001B[39mbackward,\n\u001B[1;32m 515\u001B[0m (\u001B[38;5;28mself\u001B[39m,),\n\u001B[0;32m (...)\u001B[0m\n\u001B[1;32m 520\u001B[0m inputs\u001B[38;5;241m=\u001B[39minputs,\n\u001B[1;32m 521\u001B[0m )\n\u001B[0;32m--> 522\u001B[0m \u001B[43mtorch\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mautograd\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mbackward\u001B[49m\u001B[43m(\u001B[49m\n\u001B[1;32m 523\u001B[0m \u001B[43m \u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mgradient\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mretain_graph\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mcreate_graph\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43minputs\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[43minputs\u001B[49m\n\u001B[1;32m 524\u001B[0m \u001B[43m\u001B[49m\u001B[43m)\u001B[49m\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/autograd/__init__.py:266\u001B[0m, in \u001B[0;36mbackward\u001B[0;34m(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)\u001B[0m\n\u001B[1;32m 261\u001B[0m retain_graph \u001B[38;5;241m=\u001B[39m create_graph\n\u001B[1;32m 263\u001B[0m \u001B[38;5;66;03m# The reason we repeat the same comment below is that\u001B[39;00m\n\u001B[1;32m 264\u001B[0m \u001B[38;5;66;03m# some Python versions print out the first line of a multi-line function\u001B[39;00m\n\u001B[1;32m 265\u001B[0m \u001B[38;5;66;03m# calls in the traceback and some print out the last line\u001B[39;00m\n\u001B[0;32m--> 266\u001B[0m \u001B[43mVariable\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43m_execution_engine\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mrun_backward\u001B[49m\u001B[43m(\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;66;43;03m# Calls into the C++ engine to run the backward pass\u001B[39;49;00m\n\u001B[1;32m 267\u001B[0m \u001B[43m \u001B[49m\u001B[43mtensors\u001B[49m\u001B[43m,\u001B[49m\n\u001B[1;32m 268\u001B[0m \u001B[43m \u001B[49m\u001B[43mgrad_tensors_\u001B[49m\u001B[43m,\u001B[49m\n\u001B[1;32m 269\u001B[0m \u001B[43m \u001B[49m\u001B[43mretain_graph\u001B[49m\u001B[43m,\u001B[49m\n\u001B[1;32m 270\u001B[0m \u001B[43m \u001B[49m\u001B[43mcreate_graph\u001B[49m\u001B[43m,\u001B[49m\n\u001B[1;32m 271\u001B[0m \u001B[43m \u001B[49m\u001B[43minputs\u001B[49m\u001B[43m,\u001B[49m\n\u001B[1;32m 272\u001B[0m \u001B[43m \u001B[49m\u001B[43mallow_unreachable\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[38;5;28;43;01mTrue\u001B[39;49;00m\u001B[43m,\u001B[49m\n\u001B[1;32m 273\u001B[0m \u001B[43m \u001B[49m\u001B[43maccumulate_grad\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[38;5;28;43;01mTrue\u001B[39;49;00m\u001B[43m,\u001B[49m\n\u001B[1;32m 274\u001B[0m \u001B[43m\u001B[49m\u001B[43m)\u001B[49m\n",
"\u001B[0;31mKeyboardInterrupt\u001B[0m: "
]
}
],
"source": [
"# Split train and test\n",
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1)\n",
"\n",
"# Filter test data that contains no labels\n",
"# In Coursemology, the test data is guaranteed to have labels\n",
"not_nan_indices = np.argwhere(~np.isnan(np.array(y_test))).squeeze()\n",
"y_test = [y_test[i] for i in not_nan_indices]\n",
"X_test = [X_test[i] for i in not_nan_indices]\n",
"not_nan_indices = np.argwhere(~np.isnan(np.array(y_train))).squeeze()\n",
"y_train = [y_train[i] for i in not_nan_indices]\n",
"X_train = [X_train[i] for i in not_nan_indices]\n",
"\n",
"# Train and predict\n",
"model = Model()\n",
"model.fit(X_train, y_train)\n",
"y_pred = model.predict(X_test)\n",
"\n",
"# Evaluate model predition\n",
"# Learn more: https://scikit-learn.org/stable/modules/classes.html#module-sklearn.metrics\n",
"print(\"F1 Score (macro): {0:.2f}\".format(f1_score(y_test, y_pred, average='macro'))) # You may encounter errors, you are expected to figure out what's the issue."
]
},
{
"cell_type": "markdown",
"id": "16861aef",
"metadata": {},
"source": [
"#### Grading Scheme\n",
"\n",
"Your code implementation will be graded based on its performance ([**Macro F1 Score**](http://iamirmasoud.com/2022/06/19/understanding-micro-macro-and-weighted-averages-for-scikit-learn-metrics-in-multi-class-classification-with-example/)*) in the contest. Your model will be trained with the data that we provided you with this assesment. We will use score cutoffs that we will decide after the contest to determine your marks.\n",
"\n",
"The performance of your model will be determined by a separate test data set, drawn from the same population as the training set, but not provided to you earlier. The marks you will receive will depend on the **Macro F1 Score** of the predictions:\n",
"\n",
"* If your score is above the mean or median, you can expect to receive decent marks. \n",
"* If your score is higher than the 75th percentile, you are likely to receive good marks. \n",
"* If you achieve a score above the 90th percentile (top 10%), you will likely receive full marks.\n",
"\n",
"Throughout the contest, we will provide periodic updates on the distribution of the score of student submissions in the official forum thread (see Overview) based on the **public test case**, which test the performance of the model on **a small subset of data from the hidden test data**. You can use these updates to estimate your relative standing, compared to your peers. \n",
"\n",
"<b>*) Macro F1 Score:</b> F1 score for multi-class classification computed by taking the average of all the per-class F1 score"
]
},
{
"cell_type": "markdown",
"id": "44c79c17",
"metadata": {},
"source": [
"### Task 2: Scratch Pad (20% Marks)\n",
"\n",
"Fill up the `scratchpad.ipynb` with your working. \n",
"\n",
"In the **\"Report\" section**, write a report that explain the thought process behind your solution, and convince us that you have understood the concepts taught in class and can apply them. The report should cover data exploration and preparation, data preprocessing, modeling, and evaluation. The final solution and any alternative approaches that were tried but did not work may also be documented. The length of the report should be approximately equivalent to **1-2 pages of A4 paper (up to 1,000 words)**.\n",
"\n",
"#### Grading Scheme\n",
"\n",
"The report will be graded based on the reasonability and soundness of the approach you take, your understanding of the data, and your final solution. If you do not make any errors in your approach, reasoning/understanding, and conclusion, you can expect to receive full marks. This part is meant to be \"standard\", and is only for us to do a quick sanity check that you actually did the work required to come up with the model you submitted."
]
},
{
"cell_type": "markdown",
"id": "28b658b4",
"metadata": {},
"source": [
"### Submission\n",
"\n",
"Once you are done, please submit your work to Coursemology, by copying the right snippets of code into the corresponding box that says 'Model Implementation', and click 'Save Draft'. You can still make changes after you save your submission.\n",
"\n",
"When submitting your model, the `fit` function will be called to train your model with the **data that we have provided you**. Due to the inherent stochasticity of the training process, **your model's performance may vary across different runs**. To ensure deterministic results, you can set a fixed random seed in your code. After the training is completed, the `predict` function will be used to evaluate your model. The evaluation of your model will be based on two test cases: \n",
"1. **Public test cases, containing a small portion of the test data**, that allows you to **estimate** your score. \n",
"2. **Evaluation test cases containing the remaining test data** (which you will not be able to see) by which we will evaluate your model. \n",
"\n",
"Your score in the public test case may not reflect your actual score. **Note that running all test cases can take up to 3 minutes to complete, and you have a maximum of 20 attempts.** We only provide you with a limited number of tries because we do not want you to spam our autograder. \n",
"\n",
"Finally, when you are satisfied with your submission, you can finalize it by clicking \"Finalize submission.\". <span style=\"color:red\">**Note that once you have finalized your submission, it is considered to be submitted for grading, and no further changes can be made**.</span>\n",
"\n",
"When nearing the submission deadline for our final assessment, the Coursemology server may experience overloads due to high request volumes. Here are some guidelines for you if you encounter issues close to the deadline:\n",
"\n",
"1. If you notice that no result is returned after running your code, please refrain from rerunning it. Instead, simply refresh the page and await the result.\n",
"\n",
"2. If your code continues running and you cannot finalize your submission, \n",
"* You should be able to finalize it by refreshing the submission page. Please be aware that any input made after running your code will be lost upon refreshing the page. \n",
"* If you didn't make any changes after running your code, your running code will be considered the final submission.\n",
"* If you make changes to your code after refreshing and then press finalize, your final code will be the one you typed just before finalizing.\n",
"\n",
"We highly recommend <span style=\"color:red\">**not waiting until the last moment to submit your final assessment**.</span>"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.18"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

846
cs2109s/labs/final 2/scratchpad.ipynb Normal file
View File

@ -0,0 +1,846 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "7d017333",
"metadata": {},
"source": [
"# Final Assessment Scratch Pad"
]
},
{
"cell_type": "markdown",
"id": "d3d00386",
"metadata": {},
"source": [
"## Instructions"
]
},
{
"cell_type": "markdown",
"id": "ea516aa7",
"metadata": {},
"source": [
"1. Please use only this Jupyter notebook to work on your model, and **do not use any extra files**. If you need to define helper classes or functions, feel free to do so in this notebook.\n",
"2. This template is intended to be general, but it may not cover every use case. The sections are given so that it will be easier for us to grade your submission. If your specific use case isn't addressed, **you may add new Markdown or code blocks to this notebook**. However, please **don't delete any existing blocks**.\n",
"3. If you don't think a particular section of this template is necessary for your work, **you may skip it**. Be sure to explain clearly why you decided to do so."
]
},
{
"cell_type": "markdown",
"id": "022cb4cd",
"metadata": {},
"source": [
"## Report"
]
},
{
"cell_type": "markdown",
"id": "9c14a2d8",
"metadata": {},
"source": [
"**[TODO]**\n",
"\n",
"Please provide a summary of the ideas and steps that led you to your final model. Someone reading this summary should understand why you chose to approach the problem in a particular way and able to replicate your final model at a high level. Please ensure that your summary is detailed enough to provide an overview of your thought process and approach but also concise enough to be easily understandable. Also, please follow the guidelines given in the `main.ipynb`.\n",
"\n",
"This report should not be longer than **1-2 pages of A4 paper (up to around 1,000 words)**. Marks will be deducted if you do not follow instructions and you include too many words here. \n",
"\n",
"**[DELETE EVERYTHING FROM THE PREVIOUS TODO TO HERE BEFORE SUBMISSION]**\n",
"\n",
"##### Overview\n",
"**[TODO]**\n",
"\n",
"##### 1. Descriptive Analysis\n",
"**[TODO]**\n",
"\n",
"##### 2. Detection and Handling of Missing Values\n",
"**[TODO]**\n",
"\n",
"##### 3. Detection and Handling of Outliers\n",
"**[TODO]**\n",
"\n",
"##### 4. Detection and Handling of Class Imbalance \n",
"**[TODO]**\n",
"\n",
"##### 5. Understanding Relationship Between Variables\n",
"**[TODO]**\n",
"\n",
"##### 6. Data Visualization\n",
"**[TODO]** \n",
"##### 7. General Preprocessing\n",
"**[TODO]**\n",
" \n",
"##### 8. Feature Selection \n",
"**[TODO]**\n",
"\n",
"##### 9. Feature Engineering\n",
"**[TODO]**\n",
"\n",
"##### 10. Creating Models\n",
"**[TODO]**\n",
"\n",
"##### 11. Model Evaluation\n",
"**[TODO]**\n",
"\n",
"##### 12. Hyperparameters Search\n",
"**[TODO]**\n",
"\n",
"##### Conclusion\n",
"**[TODO]**"
]
},
{
"cell_type": "markdown",
"id": "49dcaf29",
"metadata": {},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"id": "27103374",
"metadata": {},
"source": [
"# Workings (Not Graded)\n",
"\n",
"You will do your working below. Note that anything below this section will not be graded, but we might counter-check what you wrote in the report above with your workings to make sure that you actually did what you claimed to have done. "
]
},
{
"cell_type": "markdown",
"id": "0f4c6cd4",
"metadata": {},
"source": [
"## Import Packages\n",
"\n",
"Here, we import some packages necessary to run this notebook. In addition, you may import other packages as well. Do note that when submitting your model, you may only use packages that are available in Coursemology (see `main.ipynb`)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "cded1ed6",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-28T02:23:08.720475Z",
"start_time": "2024-04-28T02:23:08.235724Z"
}
},
"outputs": [],
"source": [
"import pandas as pd\n",
"import os\n",
"import numpy as np"
]
},
{
"cell_type": "markdown",
"id": "748c35d7",
"metadata": {},
"source": [
"## Load Dataset\n",
"\n",
"The dataset `data.npy` consists of $N$ grayscale videos and their corresponding labels. Each video has a shape of (L, H, W). L represents the length of the video, which may vary between videos. H and W represent the height and width, which are consistent across all videos. \n",
"\n",
"A code snippet that loads the data is provided below."
]
},
{
"cell_type": "markdown",
"id": "c09da291",
"metadata": {},
"source": [
"### Load Data"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "6297e25a",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-28T02:23:10.018019Z",
"start_time": "2024-04-28T02:23:09.989783Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of data sample: 2500\n",
"Shape of the first data sample: (10, 16, 16)\n",
"Shape of the third data sample: (8, 16, 16)\n"
]
}
],
"source": [
"with open('data.npy', 'rb') as f:\n",
" data = np.load(f, allow_pickle=True).item()\n",
" X = data['data']\n",
" y = data['label']\n",
" \n",
"print('Number of data sample:', len(X))\n",
"print('Shape of the first data sample:', X[0].shape)\n",
"print('Shape of the third data sample:', X[2].shape)"
]
},
{
"cell_type": "markdown",
"id": "cbe832b6",
"metadata": {},
"source": [
"## Data Exploration & Preparation"
]
},
{
"cell_type": "markdown",
"id": "2f6a464c",
"metadata": {},
"source": [
"### 1. Descriptive Analysis"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3b1f62dd",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "adb61967",
"metadata": {},
"source": [
"### 2. Detection and Handling of Missing Values"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4bb9cdfb",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "8adcb9cd",
"metadata": {},
"source": [
"### 3. Detection and Handling of Outliers"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ed1c17a1",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "d4916043",
"metadata": {},
"source": [
"### 4. Detection and Handling of Class Imbalance"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ad3ab20e",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "2552a795",
"metadata": {},
"source": [
"### 5. Understanding Relationship Between Variables"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "29ddbbcf",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "757fb315",
"metadata": {},
"source": [
"### 6. Data Visualization"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "93f82e42",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "2a7eebcf",
"metadata": {},
"source": [
"## Data Preprocessing"
]
},
{
"cell_type": "markdown",
"id": "ae3e3383",
"metadata": {},
"source": [
"### 7. General Preprocessing"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "19174365",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "fb3aa527",
"metadata": {},
"source": [
"### 8. Feature Selection"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a85808bf",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "4921e8ca",
"metadata": {},
"source": [
"### 9. Feature Engineering"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dbcde626",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "fa676c3f",
"metadata": {},
"source": [
"## Modeling & Evaluation"
]
},
{
"cell_type": "markdown",
"id": "589b37e4",
"metadata": {},
"source": [
"### 10. Creating models"
]
},
{
"cell_type": "code",
"outputs": [],
"source": [
"import torch\n",
"from torch import nn\n",
"class CNN(nn.Module):\n",
" def __init__(self, num_classes):\n",
" super(CNN, self).__init__()\n",
"\n",
" self.conv1 = nn.Conv2d(1,32,3,stride=1,padding=0)\n",
" self.conv2 = nn.Conv2d(32,64,3,stride=1,padding=0)\n",
" self.relu = nn.ReLU()\n",
" self.maxpool = nn.MaxPool2d(2)\n",
" self.fc1 = nn.Linear(256, 128) # Calculate input size based on output from conv2 and pooling\n",
" self.fc2 = nn.Linear(128, num_classes)\n",
" self.flatten = nn.Flatten()\n",
"\n",
" def forward(self, x):\n",
" x = self.conv1(x)\n",
" x = self.relu(x)\n",
" x = self.maxpool(x)\n",
" x = self.conv2(x)\n",
" x = self.relu(x)\n",
" x = self.maxpool(x)\n",
" x = self.flatten(x)\n",
" x = self.fc1(x)\n",
" x = self.relu(x)\n",
" x = self.fc2(x)\n",
" return x\n",
"\n",
"# video is a numpy array of shape (L, H, W)\n",
"def clean_batch(batch):\n",
" batch = np.array(batch)\n",
" print(batch.shape)\n",
" temp_x = batch.reshape(-1, 256)\n",
" np.nan_to_num(temp_x, copy=False)\n",
" col_mean = np.nanmean(temp_x, axis=0)\n",
" inds = np.where(np.isnan(temp_x))\n",
" temp_x[inds] = np.take(col_mean, inds[1])\n",
" temp_x = np.clip(temp_x, 1, 255)\n",
" batch = temp_x.reshape(-1, 1, 16,16)\n",
" return torch.tensor(batch, dtype=torch.float32)\n",
"def flatten_data(X, y):\n",
" not_nan_indices = np.argwhere(~np.isnan(np.array(y))).squeeze()\n",
" y = [y[i] for i in not_nan_indices]\n",
" X = [X[i] for i in not_nan_indices]\n",
" flattened_x = []\n",
" flattened_y = []\n",
" for idx, video in enumerate(X):\n",
" for frame in video:\n",
" flattened_x.append(frame)\n",
" flattened_y.append(y[idx])\n",
" flattened_x = clean_batch(flattened_x)\n",
" return flattened_x, torch.Tensor(np.array(flattened_y, dtype=np.int64)).long()\n",
"\n",
"class Model():\n",
" def __init__(self):\n",
" self.cnn = CNN(6)\n",
" def fit(self, X, y):\n",
" self.cnn.train()\n",
" X, y = flatten_data(X, y)\n",
" train_dataset = torch.utils.data.TensorDataset(X, y)\n",
" train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=320, shuffle=True)\n",
" criterion = nn.CrossEntropyLoss()\n",
" optimizer = torch.optim.Adam(self.cnn.parameters(), lr=0.001)\n",
" for epoch in range(70):\n",
" for idx, (inputs, labels) in enumerate(train_loader):\n",
" optimizer.zero_grad()\n",
" outputs = self.cnn(inputs)\n",
" loss = criterion(outputs, labels)\n",
" loss.backward()\n",
" optimizer.step()\n",
" print(f'Epoch {epoch}, Loss: {loss.item()}')\n",
" return self\n",
" def predict(self, X):\n",
" self.cnn.eval()\n",
" results = []\n",
" for idx, batch in enumerate(X):\n",
" batch = clean_batch(batch)\n",
" pred = self.cnn(batch)\n",
" result = torch.argmax(pred, axis=1)\n",
" results.append(torch.max(result))\n",
" return results"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-04-28T04:15:35.410374Z",
"start_time": "2024-04-28T04:15:35.390070Z"
}
},
"id": "d8dffd7d",
"execution_count": 190
},
{
"cell_type": "markdown",
"id": "495bf3c0",
"metadata": {},
"source": [
"### 11. Model Evaluation"
]
},
{
"cell_type": "code",
"execution_count": 191,
"id": "9245ab47",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-28T04:15:44.130692Z",
"start_time": "2024-04-28T04:15:37.604561Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(16186, 16, 16)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/numpy/core/fromnumeric.py:88: RuntimeWarning: overflow encountered in reduce\n",
" return ufunc.reduce(obj, axis, dtype, out, **passkwargs)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 0, Loss: 1.2299271821975708\n",
"Epoch 1, Loss: 1.1530401706695557\n",
"Epoch 2, Loss: 1.0396554470062256\n"
]
},
{
"ename": "KeyboardInterrupt",
"evalue": "",
"output_type": "error",
"traceback": [
"\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
"\u001B[0;31mKeyboardInterrupt\u001B[0m Traceback (most recent call last)",
"Cell \u001B[0;32mIn[191], line 9\u001B[0m\n\u001B[1;32m 6\u001B[0m X_test \u001B[38;5;241m=\u001B[39m [X_test[i] \u001B[38;5;28;01mfor\u001B[39;00m i \u001B[38;5;129;01min\u001B[39;00m not_nan_indices]\n\u001B[1;32m 8\u001B[0m model \u001B[38;5;241m=\u001B[39m Model()\n\u001B[0;32m----> 9\u001B[0m \u001B[43mmodel\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mfit\u001B[49m\u001B[43m(\u001B[49m\u001B[43mX_train\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43my_train\u001B[49m\u001B[43m)\u001B[49m\n",
"Cell \u001B[0;32mIn[190], line 66\u001B[0m, in \u001B[0;36mModel.fit\u001B[0;34m(self, X, y)\u001B[0m\n\u001B[1;32m 64\u001B[0m \u001B[38;5;28;01mfor\u001B[39;00m idx, (inputs, labels) \u001B[38;5;129;01min\u001B[39;00m \u001B[38;5;28menumerate\u001B[39m(train_loader):\n\u001B[1;32m 65\u001B[0m optimizer\u001B[38;5;241m.\u001B[39mzero_grad()\n\u001B[0;32m---> 66\u001B[0m outputs \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mcnn\u001B[49m\u001B[43m(\u001B[49m\u001B[43minputs\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 67\u001B[0m loss \u001B[38;5;241m=\u001B[39m criterion(outputs, labels)\n\u001B[1;32m 68\u001B[0m loss\u001B[38;5;241m.\u001B[39mbackward()\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/module.py:1511\u001B[0m, in \u001B[0;36mModule._wrapped_call_impl\u001B[0;34m(self, *args, **kwargs)\u001B[0m\n\u001B[1;32m 1509\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_compiled_call_impl(\u001B[38;5;241m*\u001B[39margs, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwargs) \u001B[38;5;66;03m# type: ignore[misc]\u001B[39;00m\n\u001B[1;32m 1510\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[0;32m-> 1511\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43m_call_impl\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/module.py:1520\u001B[0m, in \u001B[0;36mModule._call_impl\u001B[0;34m(self, *args, **kwargs)\u001B[0m\n\u001B[1;32m 1515\u001B[0m \u001B[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001B[39;00m\n\u001B[1;32m 1516\u001B[0m \u001B[38;5;66;03m# this function, and just call forward.\u001B[39;00m\n\u001B[1;32m 1517\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m (\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_backward_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_backward_pre_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_forward_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_forward_pre_hooks\n\u001B[1;32m 1518\u001B[0m \u001B[38;5;129;01mor\u001B[39;00m _global_backward_pre_hooks \u001B[38;5;129;01mor\u001B[39;00m _global_backward_hooks\n\u001B[1;32m 1519\u001B[0m \u001B[38;5;129;01mor\u001B[39;00m _global_forward_hooks \u001B[38;5;129;01mor\u001B[39;00m _global_forward_pre_hooks):\n\u001B[0;32m-> 1520\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mforward_call\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 1522\u001B[0m \u001B[38;5;28;01mtry\u001B[39;00m:\n\u001B[1;32m 1523\u001B[0m result \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;01mNone\u001B[39;00m\n",
"Cell \u001B[0;32mIn[190], line 21\u001B[0m, in \u001B[0;36mCNN.forward\u001B[0;34m(self, x)\u001B[0m\n\u001B[1;32m 19\u001B[0m x \u001B[38;5;241m=\u001B[39m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mconv2(x)\n\u001B[1;32m 20\u001B[0m x \u001B[38;5;241m=\u001B[39m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mrelu(x)\n\u001B[0;32m---> 21\u001B[0m x \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mmaxpool\u001B[49m\u001B[43m(\u001B[49m\u001B[43mx\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 22\u001B[0m x \u001B[38;5;241m=\u001B[39m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mflatten(x)\n\u001B[1;32m 23\u001B[0m x \u001B[38;5;241m=\u001B[39m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mfc1(x)\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/module.py:1511\u001B[0m, in \u001B[0;36mModule._wrapped_call_impl\u001B[0;34m(self, *args, **kwargs)\u001B[0m\n\u001B[1;32m 1509\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_compiled_call_impl(\u001B[38;5;241m*\u001B[39margs, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwargs) \u001B[38;5;66;03m# type: ignore[misc]\u001B[39;00m\n\u001B[1;32m 1510\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[0;32m-> 1511\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43m_call_impl\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/module.py:1520\u001B[0m, in \u001B[0;36mModule._call_impl\u001B[0;34m(self, *args, **kwargs)\u001B[0m\n\u001B[1;32m 1515\u001B[0m \u001B[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001B[39;00m\n\u001B[1;32m 1516\u001B[0m \u001B[38;5;66;03m# this function, and just call forward.\u001B[39;00m\n\u001B[1;32m 1517\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m (\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_backward_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_backward_pre_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_forward_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_forward_pre_hooks\n\u001B[1;32m 1518\u001B[0m \u001B[38;5;129;01mor\u001B[39;00m _global_backward_pre_hooks \u001B[38;5;129;01mor\u001B[39;00m _global_backward_hooks\n\u001B[1;32m 1519\u001B[0m \u001B[38;5;129;01mor\u001B[39;00m _global_forward_hooks \u001B[38;5;129;01mor\u001B[39;00m _global_forward_pre_hooks):\n\u001B[0;32m-> 1520\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mforward_call\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 1522\u001B[0m \u001B[38;5;28;01mtry\u001B[39;00m:\n\u001B[1;32m 1523\u001B[0m result \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;01mNone\u001B[39;00m\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/pooling.py:164\u001B[0m, in \u001B[0;36mMaxPool2d.forward\u001B[0;34m(self, input)\u001B[0m\n\u001B[1;32m 163\u001B[0m \u001B[38;5;28;01mdef\u001B[39;00m \u001B[38;5;21mforward\u001B[39m(\u001B[38;5;28mself\u001B[39m, \u001B[38;5;28minput\u001B[39m: Tensor):\n\u001B[0;32m--> 164\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mF\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mmax_pool2d\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;28;43minput\u001B[39;49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mkernel_size\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mstride\u001B[49m\u001B[43m,\u001B[49m\n\u001B[1;32m 165\u001B[0m \u001B[43m \u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mpadding\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mdilation\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mceil_mode\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mceil_mode\u001B[49m\u001B[43m,\u001B[49m\n\u001B[1;32m 166\u001B[0m \u001B[43m \u001B[49m\u001B[43mreturn_indices\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mreturn_indices\u001B[49m\u001B[43m)\u001B[49m\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/_jit_internal.py:499\u001B[0m, in \u001B[0;36mboolean_dispatch.<locals>.fn\u001B[0;34m(*args, **kwargs)\u001B[0m\n\u001B[1;32m 497\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m if_true(\u001B[38;5;241m*\u001B[39margs, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwargs)\n\u001B[1;32m 498\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[0;32m--> 499\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mif_false\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/functional.py:796\u001B[0m, in \u001B[0;36m_max_pool2d\u001B[0;34m(input, kernel_size, stride, padding, dilation, ceil_mode, return_indices)\u001B[0m\n\u001B[1;32m 794\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m stride \u001B[38;5;129;01mis\u001B[39;00m \u001B[38;5;28;01mNone\u001B[39;00m:\n\u001B[1;32m 795\u001B[0m stride \u001B[38;5;241m=\u001B[39m torch\u001B[38;5;241m.\u001B[39mjit\u001B[38;5;241m.\u001B[39mannotate(List[\u001B[38;5;28mint\u001B[39m], [])\n\u001B[0;32m--> 796\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mtorch\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mmax_pool2d\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;28;43minput\u001B[39;49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mkernel_size\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mstride\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mpadding\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mdilation\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mceil_mode\u001B[49m\u001B[43m)\u001B[49m\n",
"\u001B[0;31mKeyboardInterrupt\u001B[0m: "
]
}
],
"source": [
"from sklearn.model_selection import train_test_split\n",
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1)\n",
"\n",
"not_nan_indices = np.argwhere(~np.isnan(np.array(y_test))).squeeze()\n",
"y_test = [y_test[i] for i in not_nan_indices]\n",
"X_test = [X_test[i] for i in not_nan_indices]\n",
"\n",
"model = Model()\n",
"model.fit(X_train, y_train)\n",
"# predictions = model.predict(X_train)\n",
"# print(predictions[0])\n",
"# print(y_train[0])"
]
},
{
"cell_type": "code",
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(9, 16, 16)\n",
"(9, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(10, 16, 16)\n",
"(7, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(8, 16, 16)\n",
"(9, 16, 16)\n",
"(6, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(6, 16, 16)\n",
"(8, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(6, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(6, 16, 16)\n",
"(10, 16, 16)\n",
"(6, 16, 16)\n",
"(9, 16, 16)\n",
"(9, 16, 16)\n",
"(9, 16, 16)\n",
"(7, 16, 16)\n",
"(10, 16, 16)\n",
"(7, 16, 16)\n",
"(10, 16, 16)\n",
"(7, 16, 16)\n",
"(7, 16, 16)\n",
"(8, 16, 16)\n",
"(7, 16, 16)\n",
"(8, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(7, 16, 16)\n",
"(10, 16, 16)\n",
"(9, 16, 16)\n",
"(7, 16, 16)\n",
"(7, 16, 16)\n",
"(9, 16, 16)\n",
"(7, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(10, 16, 16)\n",
"(9, 16, 16)\n",
"(6, 16, 16)\n",
"(7, 16, 16)\n",
"(9, 16, 16)\n",
"(6, 16, 16)\n",
"(7, 16, 16)\n",
"(6, 16, 16)\n",
"(9, 16, 16)\n",
"(6, 16, 16)\n",
"(9, 16, 16)\n",
"(8, 16, 16)\n",
"(7, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(7, 16, 16)\n",
"(6, 16, 16)\n",
"(9, 16, 16)\n",
"(6, 16, 16)\n",
"(6, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(10, 16, 16)\n",
"(10, 16, 16)\n",
"(6, 16, 16)\n",
"(9, 16, 16)\n",
"(7, 16, 16)\n",
"(8, 16, 16)\n",
"(7, 16, 16)\n",
"(9, 16, 16)\n",
"(7, 16, 16)\n",
"(7, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(7, 16, 16)\n",
"(6, 16, 16)\n",
"(9, 16, 16)\n",
"(9, 16, 16)\n",
"(9, 16, 16)\n",
"(6, 16, 16)\n",
"(10, 16, 16)\n",
"(6, 16, 16)\n",
"(9, 16, 16)\n",
"(7, 16, 16)\n",
"(6, 16, 16)\n",
"(10, 16, 16)\n",
"(7, 16, 16)\n",
"(7, 16, 16)\n",
"(7, 16, 16)\n",
"(7, 16, 16)\n",
"(7, 16, 16)\n",
"(7, 16, 16)\n",
"(10, 16, 16)\n",
"(10, 16, 16)\n",
"(7, 16, 16)\n",
"(9, 16, 16)\n",
"(6, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(6, 16, 16)\n",
"(7, 16, 16)\n",
"(6, 16, 16)\n",
"(6, 16, 16)\n",
"(6, 16, 16)\n",
"(8, 16, 16)\n",
"(6, 16, 16)\n",
"(10, 16, 16)\n",
"(9, 16, 16)\n",
"(10, 16, 16)\n",
"(9, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(8, 16, 16)\n",
"(7, 16, 16)\n",
"(7, 16, 16)\n",
"(7, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(10, 16, 16)\n",
"(10, 16, 16)\n",
"(10, 16, 16)\n",
"(6, 16, 16)\n",
"(7, 16, 16)\n",
"(7, 16, 16)\n",
"(9, 16, 16)\n",
"(7, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(6, 16, 16)\n",
"(6, 16, 16)\n",
"(8, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(9, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(9, 16, 16)\n",
"(8, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(9, 16, 16)\n",
"(10, 16, 16)\n",
"(7, 16, 16)\n",
"(9, 16, 16)\n",
"(10, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(8, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(7, 16, 16)\n",
"(7, 16, 16)\n",
"(9, 16, 16)\n",
"(9, 16, 16)\n",
"(6, 16, 16)\n",
"(10, 16, 16)\n",
"(10, 16, 16)\n",
"(10, 16, 16)\n",
"(7, 16, 16)\n",
"(9, 16, 16)\n",
"(6, 16, 16)\n",
"(7, 16, 16)\n",
"(8, 16, 16)\n",
"(9, 16, 16)\n",
"(9, 16, 16)\n",
"(8, 16, 16)\n",
"(7, 16, 16)\n",
"(10, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(9, 16, 16)\n",
"(8, 16, 16)\n",
"(6, 16, 16)\n",
"(7, 16, 16)\n",
"(7, 16, 16)\n",
"(6, 16, 16)\n",
"(8, 16, 16)\n",
"(9, 16, 16)\n",
"(7, 16, 16)\n",
"(6, 16, 16)\n",
"(6, 16, 16)\n",
"(10, 16, 16)\n",
"(9, 16, 16)\n",
"(7, 16, 16)\n",
"(8, 16, 16)\n",
"(8, 16, 16)\n",
"(8, 16, 16)\n",
"(9, 16, 16)\n",
"(8, 16, 16)\n",
"(9, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(9, 16, 16)\n",
"(10, 16, 16)\n",
"(8, 16, 16)\n",
"(10, 16, 16)\n",
"(7, 16, 16)\n",
"(8, 16, 16)\n",
"(7, 16, 16)\n",
"(6, 16, 16)\n",
"(9, 16, 16)\n",
"(6, 16, 16)\n",
"tensor([[-0.7872, 0.9621, -1.1426, -1.4650, -0.4242, -0.4840],\n",
" [-0.4881, 1.5047, -1.0677, -0.8765, -1.2379, -1.1973],\n",
" [-0.7944, 0.7633, -0.8908, -1.2021, -0.1398, -0.1900],\n",
" [-0.2103, 2.2481, -2.2957, -2.3898, 0.0291, -0.8567],\n",
" [ 0.4143, 2.8888, -1.1756, -0.8911, -1.5052, -1.1943],\n",
" [ 0.5421, 2.6672, -1.5965, -1.4607, -2.0967, -1.3973]],\n",
" grad_fn=<AddmmBackward0>)\n",
"tensor([1, 1, 1, 1, 1, 1])\n",
"F1 Score (macro): 0.36\n"
]
}
],
"source": [
"from sklearn.metrics import f1_score\n",
"\n",
"y_pred = model.predict(X_test)\n",
"result = model.cnn(clean_batch(X_train[0]))\n",
"print(result)\n",
"print(torch.argmax(result, axis=1))\n",
"# print(y_train[0])\n",
"print(\"F1 Score (macro): {0:.2f}\".format(f1_score(y_test, y_pred, average='macro'))) # You may encounter errors, you are expected to figure out what's the issue.\n"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-04-28T04:08:22.419351Z",
"start_time": "2024-04-28T04:08:22.241362Z"
}
},
"id": "dd595539230499dd",
"execution_count": 189
},
{
"cell_type": "markdown",
"id": "8aa31404",
"metadata": {},
"source": [
"### 12. Hyperparameters Search"
]
},
{
"cell_type": "code",
"outputs": [],
"source": [],
"metadata": {
"collapsed": false
},
"id": "7728c49ea8a0bacf"
},
{
"cell_type": "code",
"execution_count": null,
"id": "81addd51",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.18"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

110
cs2109s/labs/final-mock/main.ipynb
View File

@ -299,16 +299,25 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": 70,
"id": "a44b7aa4", "id": "a44b7aa4",
"metadata": {}, "metadata": {
"ExecuteTime": {
"end_time": "2024-04-27T07:45:57.664982Z",
"start_time": "2024-04-27T07:45:57.652624Z"
}
},
"outputs": [], "outputs": [],
"source": [ "source": [
"from sklearn.preprocessing import OrdinalEncoder\n",
"from sklearn.linear_model import LinearRegression\n",
"import sklearn.ensemble\n",
"\n",
"\n",
"class Model: \n", "class Model: \n",
" \"\"\"\n", " \"\"\"\n",
" This class represents an AI model.\n", " This class represents an AI model.\n",
" \"\"\"\n", " \"\"\"\n",
" \n",
" def __init__(self):\n", " def __init__(self):\n",
" \"\"\"\n", " \"\"\"\n",
" Constructor for Model class.\n", " Constructor for Model class.\n",
@ -318,9 +327,26 @@
" self : object\n", " self : object\n",
" The instance of the object passed by Python.\n", " The instance of the object passed by Python.\n",
" \"\"\"\n", " \"\"\"\n",
" # TODO: Replace the following code with your own initialization code.\n", " self.model = LinearRegression()\n",
" pass\n", "\n",
" \n", " def process_input(self, X):\n",
" images = X['images'].reshape(X['images'].shape[0], -1)\n",
" X = X['tabular']\n",
" X = \n",
" def object_columns(X):\n",
" return X.dtypes[X.dtypes == 'object'].index\n",
"\n",
" def convert_to_ordinal(X, columns):\n",
" encoder = OrdinalEncoder()\n",
" return encoder.fit_transform(X[columns])\n",
"\n",
" obj_cols = object_columns(X)\n",
" ordinal_columns = convert_to_ordinal(X, obj_cols)\n",
" X[obj_cols] = ordinal_columns\n",
" columns_to_drop = ['V40', 'V20', 'V39', 'V15', 'V10', 'V35', 'V2', 'V52', 'V45', 'V7', 'V48', 'V49', 'V43', 'V44', 'V26', 'V41', 'V11', 'V53', 'V42', 'V38']\n",
" X = X.drop(columns_to_drop, axis=1)\n",
" X = X.fillna(X.mean())\n",
" return X\n",
" def fit(self, X_dict, y):\n", " def fit(self, X_dict, y):\n",
" \"\"\"\n", " \"\"\"\n",
" Train the model using the input data.\n", " Train the model using the input data.\n",
@ -339,9 +365,11 @@
" self : object\n", " self : object\n",
" Returns an instance of the trained model.\n", " Returns an instance of the trained model.\n",
" \"\"\"\n", " \"\"\"\n",
" # TODO: Add your training code.\n", " X = X_dict['tabular']\n",
" X = self.process_input(X)\n",
" self.model.fit(X, y)\n",
" return self\n", " return self\n",
" \n", " \n",
" def predict(self, X_dict):\n", " def predict(self, X_dict):\n",
" \"\"\"\n", " \"\"\"\n",
" Use the trained model to make predictions.\n", " Use the trained model to make predictions.\n",
@ -359,8 +387,9 @@
" Predicted target values per element in X_dict.\n", " Predicted target values per element in X_dict.\n",
" \n", " \n",
" \"\"\"\n", " \"\"\"\n",
" # TODO: Replace the following code with your own prediction code.\n", " X = self.process_input(X_dict['tabular'])\n",
" return [0 for _ in range(len(X_dict['tabular']))]" " return self.model.predict(X)\n",
" # return [0 for _ in range(len(X_dict['tabular']))]"
] ]
}, },
{ {
@ -375,9 +404,14 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": 51,
"id": "4f4dd489", "id": "4f4dd489",
"metadata": {}, "metadata": {
"ExecuteTime": {
"end_time": "2024-04-27T07:23:39.732051Z",
"start_time": "2024-04-27T07:23:39.725818Z"
}
},
"outputs": [], "outputs": [],
"source": [ "source": [
"# Import packages\n", "# Import packages\n",
@ -390,9 +424,14 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": 52,
"id": "3064e0ff", "id": "3064e0ff",
"metadata": {}, "metadata": {
"ExecuteTime": {
"end_time": "2024-04-27T07:23:42.216498Z",
"start_time": "2024-04-27T07:23:40.676178Z"
}
},
"outputs": [], "outputs": [],
"source": [ "source": [
"# Load data\n", "# Load data\n",
@ -413,10 +452,47 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": 71,
"id": "27c9fd10", "id": "27c9fd10",
"metadata": {}, "metadata": {
"outputs": [], "ExecuteTime": {
"end_time": "2024-04-27T07:46:01.374238Z",
"start_time": "2024-04-27T07:45:59.640013Z"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/var/folders/zd/9vyg32393qncxwt_3r_873mh0000gn/T/ipykernel_29080/3308836053.py:43: SettingWithCopyWarning: \n",
"A value is trying to be set on a copy of a slice from a DataFrame.\n",
"Try using .loc[row_indexer,col_indexer] = value instead\n",
"\n",
"See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
" X[obj_cols] = ordinal_columns\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"MSE: 5352.19\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/var/folders/zd/9vyg32393qncxwt_3r_873mh0000gn/T/ipykernel_29080/3308836053.py:43: SettingWithCopyWarning: \n",
"A value is trying to be set on a copy of a slice from a DataFrame.\n",
"Try using .loc[row_indexer,col_indexer] = value instead\n",
"\n",
"See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
" X[obj_cols] = ordinal_columns\n"
]
}
],
"source": [ "source": [
"# Split train and test\n", "# Split train and test\n",
"X_dict_train, y_train, X_dict_test, y_test = dict_train_test_split(X_dict, y, ratio=0.9)\n", "X_dict_train, y_train, X_dict_test, y_test = dict_train_test_split(X_dict, y, ratio=0.9)\n",

385
cs2109s/labs/final-mock/scratchpad.ipynb
View File

File diff suppressed because one or more lines are too long

255
cs2109s/labs/final/main.ipynb
View File

@ -314,94 +314,163 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 224, "execution_count": 65,
"id": "a44b7aa4", "id": "a44b7aa4",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-27T15:21:54.599035Z", "end_time": "2024-04-28T07:13:16.866724Z",
"start_time": "2024-04-27T15:21:54.587252Z" "start_time": "2024-04-28T07:13:16.841471Z"
} }
}, },
"outputs": [], "outputs": [],
"source": [ "source": [
"from torch.utils.data import TensorDataset, DataLoader\n",
"import torch\n", "import torch\n",
"from torch import nn\n", "from torch import nn\n",
"class tinyCNN(nn.Module):\n",
" def __init__(self, num_classes):\n",
" super(tinyCNN, self).__init__()\n",
"\n", "\n",
"class Model: \n", " self.conv1 = nn.Conv2d(1,32,3,stride=1,padding=0)\n",
" \"\"\"\n", " self.conv2 = nn.Conv2d(32,32,3,stride=1,padding=0)\n",
" This class represents an AI model.\n", " self.conv3 = nn.Conv2d(32,64,3,stride=1,padding=0)\n",
" \"\"\"\n", " self.conv4 = nn.Conv2d(64,64,3,stride=1,padding=0)\n",
" \n",
" self.relu = nn.ReLU()\n",
" self.maxpool = nn.MaxPool2d(2)\n",
" self.batchnorm1 = nn.BatchNorm2d(32)\n",
" self.batchnorm2 = nn.BatchNorm2d(64)\n",
" self.fc = nn.Linear(64, num_classes)\n",
" self.flatten = nn.Flatten()\n",
"\n",
" def forward(self, x):\n",
" x = self.conv1(x)\n",
" x = self.relu(x)\n",
" \n",
" x = self.batchnorm1(x)\n",
" x = self.maxpool(x)\n",
" \n",
" x = self.conv2(x)\n",
" x = self.relu(x)\n",
"\n",
" x = self.conv3(x)\n",
" x = self.relu(x)\n",
"\n",
" x = self.batchnorm2(x)\n",
" x = self.maxpool(x)\n",
" x = self.flatten(x)\n",
" x = self.fc(x)\n",
" return x\n",
" \n", " \n",
" def __init__(self):\n", "class CIFARCNN(nn.Module):\n",
" def __init__(self, classes):\n",
" super().__init__()\n",
" \"\"\"\n", " \"\"\"\n",
" Constructor for Model class.\n", " classes: integer that corresponds to the number of classes for CIFAR-10\n",
" \n",
" Parameters\n",
" ----------\n",
" self : object\n",
" The instance of the object passed by Python.\n",
" \"\"\"\n", " \"\"\"\n",
" # TODO: Replace the following code with your own initialization code.\n", " self.flatten = nn.Flatten()\n",
" self.model = nn.Sequential(\n", " self.conv = nn.Sequential(\n",
" nn.LSTM(256, 128, 2, batch_first=True),\n", " nn.Conv2d(1, 32, 3),\n",
" nn.Linear(128, 6),\n", " nn.MaxPool2d(2),\n",
" nn.LeakyReLU(0.1),\n",
" nn.Conv2d(32, 64, (3, 3)),\n",
" nn.MaxPool2d(2),\n",
" nn.LeakyReLU(0.1),\n",
" )\n",
"\n",
" self.fc = nn.Sequential(\n",
" nn.Linear(256, 256),\n",
" nn.LeakyReLU(0.1),\n",
" nn.Linear(256, 128),\n",
" nn.LeakyReLU(0.1),\n",
" nn.Linear(128, classes)\n",
" )\n", " )\n",
" \n", " \n",
" def process(self, X):\n", " def forward(self, x):\n",
" X_array = np.zeros((10, 16, 16))\n", " # YOUR CODE HERE\n",
" for i, video in enumerate(X):\n", " x = self.conv(x)\n",
" X_array[i, :, :] = video\n", " x = self.flatten(x)\n",
" X_array = X_array.reshape((10, 1, 256))\n", " x = self.fc(x)\n",
" print(X_array.shape)\n", " return x\n",
" \n",
" return torch.from_numpy(X_array).float()\n",
"\n", "\n",
"# video is a numpy array of shape (L, H, W)\n",
"def clean_batch(batch):\n",
" batch = np.array(batch)\n",
" temp_x = batch.reshape(-1, 256)\n",
" np.nan_to_num(temp_x, copy=False)\n",
" col_mean = np.nanmean(temp_x, axis=0)\n",
" inds = np.where(np.isnan(temp_x))\n",
" temp_x[inds] = np.take(col_mean, inds[1])\n",
" temp_x = np.clip(temp_x, 1, 255)\n",
" batch = temp_x.reshape(-1, 1, 16,16)\n",
" return torch.tensor(batch, dtype=torch.float32)\n",
"def flatten_data(X, y):\n",
" not_nan_indices = np.argwhere(~np.isnan(np.array(y))).squeeze()\n",
" # Remove non y columns\n",
" y = [y[i] for i in not_nan_indices]\n",
" X = [X[i] for i in not_nan_indices]\n",
" flattened_x = [video[i] for video in X for i in range(video.shape[0])]\n",
" flattened_y = np.repeat(y, [video.shape[0] for video in X])\n",
" flattened_x = clean_batch(flattened_x)\n",
" return flattened_x, torch.Tensor(np.array(flattened_y, dtype=np.int64)).long()\n",
"\n",
"def train(model, criterion, optimizer, loader, epochs = 10):\n",
" for epoch in range(epochs):\n",
" for idx, (inputs, labels) in enumerate(loader):\n",
" optimizer.zero_grad()\n",
" outputs = model(inputs)\n",
" loss = criterion(outputs, labels)\n",
" loss.backward()\n",
" optimizer.step()\n",
" print(f'Epoch {epoch}, Loss: {loss.item()}')\n",
" return model\n",
"def process_data(X, y):\n",
" y = np.array(y)\n",
" X = np.array([video[:6] for video in X])\n",
" tensor_videos = torch.tensor(X, dtype=torch.float32)\n",
" # Clip values to 0 and 255\n",
" tensor_videos = np.clip(tensor_videos, 0, 255)\n",
" # Replace NaNs in each frame, with the average of the frame. This was generated with GPT\n",
" for i in range(tensor_videos.shape[0]):\n",
" for j in range(tensor_videos.shape[1]):\n",
" tensor_videos[i][j][torch.isnan(tensor_videos[i][j])] = torch.mean(tensor_videos[i][j][~torch.isnan(tensor_videos[i][j])])\n",
" \n",
" # Undersample the data for each of the 6 classes. Select max of 300 samples for each class\n",
" # Very much generated with the assitance of chatGPT with some modifications\n",
" # Get the indices of each class\n",
" indices = [np.argwhere(y == i).squeeze(1) for i in range(6)]\n",
" # Get the number of samples to take for each class\n",
" num_samples_to_take = 300\n",
" # Get the indices of the samples to take\n",
" indices_to_take = [np.random.choice(indices[i], num_samples_to_take, replace=True) for i in range(6)]\n",
" # Concatenate the indices\n",
" indices_to_take = np.concatenate(indices_to_take)\n",
" # Select the samples\n",
" tensor_videos = tensor_videos[indices_to_take]\n",
" y = y[indices_to_take]\n",
" return torch.Tensor(tensor_videos), torch.Tensor(y)\n",
"\n",
"class Model():\n",
" def __init__(self):\n",
" self.cnn = CIFARCNN(6)\n",
" def fit(self, X, y):\n", " def fit(self, X, y):\n",
" \"\"\"\n", " self.cnn.train()\n",
" Train the model using the input data.\n", " X, y = process_data(X, y)\n",
" \n", " print(X.shape, y.shape)\n",
" Parameters\n", " train_dataset = torch.utils.data.TensorDataset(X, y)\n",
" ----------\n", " train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=32, shuffle=True)\n",
" X : list of size (n_samples)\n", " criterion = nn.CrossEntropyLoss()\n",
" Each item in the list is a grayscale video of shape (L, H, W).\n", " optimizer = torch.optim.Adam(self.cnn.parameters(), lr=0.001)\n",
" L represents the length of the video, which may vary between videos. \n", " self.model = train(self.cnn, criterion, optimizer, train_loader)\n",
" H and W represent the height and width, which are consistent across all videos. \n",
" y : list of size (n_samples)\n",
" Class labels for videos\n",
" \n",
" Returns\n",
" -------\n",
" self : object\n",
" Returns an instance of the trained model.\n",
" \"\"\"\n",
" X = self.process(X)\n",
" train_dataset = TensorDataset(X, torch.tensor(y))\n",
" train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)\n",
"\n",
" return self\n", " return self\n",
" \n",
" def predict(self, X):\n", " def predict(self, X):\n",
" \"\"\"\n", " self.cnn.eval()\n",
" Use the trained model to make predictions.\n", " results = []\n",
" \n", " for idx, batch in enumerate(X):\n",
" Parameters\n", " batch = clean_batch(batch)\n",
" ----------\n", " pred = self.cnn(batch)\n",
" X : list of size (n_samples)\n", " result = torch.argmax(pred, axis=1)\n",
" Each item in the list is a grayscale video of shape (L, H, W).\n", " results.append(torch.max(result))\n",
" L represents the length of the video, which may vary between videos. \n", " return results\n"
" H and W represent the height and width, which are consistent across all videos. \n",
" \n",
" Returns\n",
" -------\n",
" ndarray of shape (n_samples,)\n",
" Predicted target values per element in X.\n",
" \n",
" \"\"\"\n",
" result = []\n",
" for video in X:\n",
" result.append(self.model(self.process(video)).argmax(dim=1).detach().numpy())\n",
" return result"
] ]
}, },
{ {
@ -416,12 +485,12 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 2, "execution_count": 49,
"id": "4f4dd489", "id": "4f4dd489",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-27T13:36:15.463930Z", "end_time": "2024-04-28T07:10:17.037378Z",
"start_time": "2024-04-27T13:36:15.298430Z" "start_time": "2024-04-28T07:10:17.031404Z"
} }
}, },
"outputs": [], "outputs": [],
@ -436,12 +505,12 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 4, "execution_count": 50,
"id": "3064e0ff", "id": "3064e0ff",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-27T13:36:18.535709Z", "end_time": "2024-04-28T07:10:18.316631Z",
"start_time": "2024-04-27T13:36:18.509684Z" "start_time": "2024-04-28T07:10:18.289375Z"
} }
}, },
"outputs": [], "outputs": [],
@ -455,12 +524,12 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 225, "execution_count": 67,
"id": "27c9fd10", "id": "27c9fd10",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-27T15:21:56.847320Z", "end_time": "2024-04-28T07:14:48.901477Z",
"start_time": "2024-04-27T15:21:56.787331Z" "start_time": "2024-04-28T07:14:48.343775Z"
} }
}, },
"outputs": [ "outputs": [
@ -468,31 +537,37 @@
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"(10, 1, 256)\n" "torch.Size([1800, 6, 16, 16]) torch.Size([1800])\n"
] ]
}, },
{ {
"ename": "TypeError", "ename": "RuntimeError",
"evalue": "linear(): argument 'input' (position 1) must be Tensor, not tuple", "evalue": "Given groups=1, weight of size [32, 1, 3, 3], expected input[32, 6, 16, 16] to have 1 channels, but got 6 channels instead",
"output_type": "error", "output_type": "error",
"traceback": [ "traceback": [
"\u001B[0;31m---------------------------------------------------------------------------\u001B[0m", "\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
"\u001B[0;31mTypeError\u001B[0m Traceback (most recent call last)", "\u001B[0;31mRuntimeError\u001B[0m Traceback (most recent call last)",
"Cell \u001B[0;32mIn[225], line 13\u001B[0m\n\u001B[1;32m 11\u001B[0m model \u001B[38;5;241m=\u001B[39m Model()\n\u001B[1;32m 12\u001B[0m \u001B[38;5;66;03m# model.fit(X_train, y_train)\u001B[39;00m\n\u001B[0;32m---> 13\u001B[0m y_pred \u001B[38;5;241m=\u001B[39m \u001B[43mmodel\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mpredict\u001B[49m\u001B[43m(\u001B[49m\u001B[43mX_test\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 14\u001B[0m \u001B[38;5;28mprint\u001B[39m(y_pred[\u001B[38;5;241m0\u001B[39m])\n\u001B[1;32m 15\u001B[0m \u001B[38;5;28mprint\u001B[39m(y[\u001B[38;5;241m0\u001B[39m])\n", "File \u001B[0;32m<timed exec>:12\u001B[0m\n",
"Cell \u001B[0;32mIn[224], line 77\u001B[0m, in \u001B[0;36mModel.predict\u001B[0;34m(self, X)\u001B[0m\n\u001B[1;32m 75\u001B[0m result \u001B[38;5;241m=\u001B[39m []\n\u001B[1;32m 76\u001B[0m \u001B[38;5;28;01mfor\u001B[39;00m video \u001B[38;5;129;01min\u001B[39;00m X:\n\u001B[0;32m---> 77\u001B[0m result\u001B[38;5;241m.\u001B[39mappend(\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mmodel\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mprocess\u001B[49m\u001B[43m(\u001B[49m\u001B[43mvideo\u001B[49m\u001B[43m)\u001B[49m\u001B[43m)\u001B[49m\u001B[38;5;241m.\u001B[39margmax(dim\u001B[38;5;241m=\u001B[39m\u001B[38;5;241m1\u001B[39m)\u001B[38;5;241m.\u001B[39mdetach()\u001B[38;5;241m.\u001B[39mnumpy())\n\u001B[1;32m 78\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m result\n", "Cell \u001B[0;32mIn[65], line 137\u001B[0m, in \u001B[0;36mModel.fit\u001B[0;34m(self, X, y)\u001B[0m\n\u001B[1;32m 135\u001B[0m criterion \u001B[38;5;241m=\u001B[39m nn\u001B[38;5;241m.\u001B[39mCrossEntropyLoss()\n\u001B[1;32m 136\u001B[0m optimizer \u001B[38;5;241m=\u001B[39m torch\u001B[38;5;241m.\u001B[39moptim\u001B[38;5;241m.\u001B[39mAdam(\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mcnn\u001B[38;5;241m.\u001B[39mparameters(), lr\u001B[38;5;241m=\u001B[39m\u001B[38;5;241m0.001\u001B[39m)\n\u001B[0;32m--> 137\u001B[0m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mmodel \u001B[38;5;241m=\u001B[39m \u001B[43mtrain\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mcnn\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mcriterion\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43moptimizer\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mtrain_loader\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 138\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28mself\u001B[39m\n",
"Cell \u001B[0;32mIn[65], line 94\u001B[0m, in \u001B[0;36mtrain\u001B[0;34m(model, criterion, optimizer, loader, epochs)\u001B[0m\n\u001B[1;32m 92\u001B[0m \u001B[38;5;28;01mfor\u001B[39;00m idx, (inputs, labels) \u001B[38;5;129;01min\u001B[39;00m \u001B[38;5;28menumerate\u001B[39m(loader):\n\u001B[1;32m 93\u001B[0m optimizer\u001B[38;5;241m.\u001B[39mzero_grad()\n\u001B[0;32m---> 94\u001B[0m outputs \u001B[38;5;241m=\u001B[39m \u001B[43mmodel\u001B[49m\u001B[43m(\u001B[49m\u001B[43minputs\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 95\u001B[0m loss \u001B[38;5;241m=\u001B[39m criterion(outputs, labels)\n\u001B[1;32m 96\u001B[0m loss\u001B[38;5;241m.\u001B[39mbackward()\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/module.py:1511\u001B[0m, in \u001B[0;36mModule._wrapped_call_impl\u001B[0;34m(self, *args, **kwargs)\u001B[0m\n\u001B[1;32m 1509\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_compiled_call_impl(\u001B[38;5;241m*\u001B[39margs, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwargs) \u001B[38;5;66;03m# type: ignore[misc]\u001B[39;00m\n\u001B[1;32m 1510\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[0;32m-> 1511\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43m_call_impl\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/module.py:1520\u001B[0m, in \u001B[0;36mModule._call_impl\u001B[0;34m(self, *args, **kwargs)\u001B[0m\n\u001B[1;32m 1515\u001B[0m \u001B[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001B[39;00m\n\u001B[1;32m 1516\u001B[0m \u001B[38;5;66;03m# this function, and just call forward.\u001B[39;00m\n\u001B[1;32m 1517\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m (\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_backward_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_backward_pre_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_forward_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_forward_pre_hooks\n\u001B[1;32m 1518\u001B[0m \u001B[38;5;129;01mor\u001B[39;00m _global_backward_pre_hooks \u001B[38;5;129;01mor\u001B[39;00m _global_backward_hooks\n\u001B[1;32m 1519\u001B[0m \u001B[38;5;129;01mor\u001B[39;00m _global_forward_hooks \u001B[38;5;129;01mor\u001B[39;00m _global_forward_pre_hooks):\n\u001B[0;32m-> 1520\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mforward_call\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 1522\u001B[0m \u001B[38;5;28;01mtry\u001B[39;00m:\n\u001B[1;32m 1523\u001B[0m result \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;01mNone\u001B[39;00m\n",
"Cell \u001B[0;32mIn[65], line 64\u001B[0m, in \u001B[0;36mCIFARCNN.forward\u001B[0;34m(self, x)\u001B[0m\n\u001B[1;32m 62\u001B[0m \u001B[38;5;28;01mdef\u001B[39;00m \u001B[38;5;21mforward\u001B[39m(\u001B[38;5;28mself\u001B[39m, x):\n\u001B[1;32m 63\u001B[0m \u001B[38;5;66;03m# YOUR CODE HERE\u001B[39;00m\n\u001B[0;32m---> 64\u001B[0m x \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mconv\u001B[49m\u001B[43m(\u001B[49m\u001B[43mx\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 65\u001B[0m x \u001B[38;5;241m=\u001B[39m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mflatten(x)\n\u001B[1;32m 66\u001B[0m x \u001B[38;5;241m=\u001B[39m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mfc(x)\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/module.py:1511\u001B[0m, in \u001B[0;36mModule._wrapped_call_impl\u001B[0;34m(self, *args, **kwargs)\u001B[0m\n\u001B[1;32m 1509\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_compiled_call_impl(\u001B[38;5;241m*\u001B[39margs, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwargs) \u001B[38;5;66;03m# type: ignore[misc]\u001B[39;00m\n\u001B[1;32m 1510\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[0;32m-> 1511\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43m_call_impl\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n", "File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/module.py:1511\u001B[0m, in \u001B[0;36mModule._wrapped_call_impl\u001B[0;34m(self, *args, **kwargs)\u001B[0m\n\u001B[1;32m 1509\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_compiled_call_impl(\u001B[38;5;241m*\u001B[39margs, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwargs) \u001B[38;5;66;03m# type: ignore[misc]\u001B[39;00m\n\u001B[1;32m 1510\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[0;32m-> 1511\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43m_call_impl\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/module.py:1520\u001B[0m, in \u001B[0;36mModule._call_impl\u001B[0;34m(self, *args, **kwargs)\u001B[0m\n\u001B[1;32m 1515\u001B[0m \u001B[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001B[39;00m\n\u001B[1;32m 1516\u001B[0m \u001B[38;5;66;03m# this function, and just call forward.\u001B[39;00m\n\u001B[1;32m 1517\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m (\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_backward_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_backward_pre_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_forward_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_forward_pre_hooks\n\u001B[1;32m 1518\u001B[0m \u001B[38;5;129;01mor\u001B[39;00m _global_backward_pre_hooks \u001B[38;5;129;01mor\u001B[39;00m _global_backward_hooks\n\u001B[1;32m 1519\u001B[0m \u001B[38;5;129;01mor\u001B[39;00m _global_forward_hooks \u001B[38;5;129;01mor\u001B[39;00m _global_forward_pre_hooks):\n\u001B[0;32m-> 1520\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mforward_call\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 1522\u001B[0m \u001B[38;5;28;01mtry\u001B[39;00m:\n\u001B[1;32m 1523\u001B[0m result \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;01mNone\u001B[39;00m\n", "File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/module.py:1520\u001B[0m, in \u001B[0;36mModule._call_impl\u001B[0;34m(self, *args, **kwargs)\u001B[0m\n\u001B[1;32m 1515\u001B[0m \u001B[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001B[39;00m\n\u001B[1;32m 1516\u001B[0m \u001B[38;5;66;03m# this function, and just call forward.\u001B[39;00m\n\u001B[1;32m 1517\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m (\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_backward_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_backward_pre_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_forward_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_forward_pre_hooks\n\u001B[1;32m 1518\u001B[0m \u001B[38;5;129;01mor\u001B[39;00m _global_backward_pre_hooks \u001B[38;5;129;01mor\u001B[39;00m _global_backward_hooks\n\u001B[1;32m 1519\u001B[0m \u001B[38;5;129;01mor\u001B[39;00m _global_forward_hooks \u001B[38;5;129;01mor\u001B[39;00m _global_forward_pre_hooks):\n\u001B[0;32m-> 1520\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mforward_call\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 1522\u001B[0m \u001B[38;5;28;01mtry\u001B[39;00m:\n\u001B[1;32m 1523\u001B[0m result \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;01mNone\u001B[39;00m\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/container.py:217\u001B[0m, in \u001B[0;36mSequential.forward\u001B[0;34m(self, input)\u001B[0m\n\u001B[1;32m 215\u001B[0m \u001B[38;5;28;01mdef\u001B[39;00m \u001B[38;5;21mforward\u001B[39m(\u001B[38;5;28mself\u001B[39m, \u001B[38;5;28minput\u001B[39m):\n\u001B[1;32m 216\u001B[0m \u001B[38;5;28;01mfor\u001B[39;00m module \u001B[38;5;129;01min\u001B[39;00m \u001B[38;5;28mself\u001B[39m:\n\u001B[0;32m--> 217\u001B[0m \u001B[38;5;28minput\u001B[39m \u001B[38;5;241m=\u001B[39m \u001B[43mmodule\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;28;43minput\u001B[39;49m\u001B[43m)\u001B[49m\n\u001B[1;32m 218\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28minput\u001B[39m\n", "File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/container.py:217\u001B[0m, in \u001B[0;36mSequential.forward\u001B[0;34m(self, input)\u001B[0m\n\u001B[1;32m 215\u001B[0m \u001B[38;5;28;01mdef\u001B[39;00m \u001B[38;5;21mforward\u001B[39m(\u001B[38;5;28mself\u001B[39m, \u001B[38;5;28minput\u001B[39m):\n\u001B[1;32m 216\u001B[0m \u001B[38;5;28;01mfor\u001B[39;00m module \u001B[38;5;129;01min\u001B[39;00m \u001B[38;5;28mself\u001B[39m:\n\u001B[0;32m--> 217\u001B[0m \u001B[38;5;28minput\u001B[39m \u001B[38;5;241m=\u001B[39m \u001B[43mmodule\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;28;43minput\u001B[39;49m\u001B[43m)\u001B[49m\n\u001B[1;32m 218\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28minput\u001B[39m\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/module.py:1511\u001B[0m, in \u001B[0;36mModule._wrapped_call_impl\u001B[0;34m(self, *args, **kwargs)\u001B[0m\n\u001B[1;32m 1509\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_compiled_call_impl(\u001B[38;5;241m*\u001B[39margs, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwargs) \u001B[38;5;66;03m# type: ignore[misc]\u001B[39;00m\n\u001B[1;32m 1510\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[0;32m-> 1511\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43m_call_impl\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n", "File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/module.py:1511\u001B[0m, in \u001B[0;36mModule._wrapped_call_impl\u001B[0;34m(self, *args, **kwargs)\u001B[0m\n\u001B[1;32m 1509\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_compiled_call_impl(\u001B[38;5;241m*\u001B[39margs, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwargs) \u001B[38;5;66;03m# type: ignore[misc]\u001B[39;00m\n\u001B[1;32m 1510\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[0;32m-> 1511\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43m_call_impl\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/module.py:1520\u001B[0m, in \u001B[0;36mModule._call_impl\u001B[0;34m(self, *args, **kwargs)\u001B[0m\n\u001B[1;32m 1515\u001B[0m \u001B[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001B[39;00m\n\u001B[1;32m 1516\u001B[0m \u001B[38;5;66;03m# this function, and just call forward.\u001B[39;00m\n\u001B[1;32m 1517\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m (\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_backward_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_backward_pre_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_forward_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_forward_pre_hooks\n\u001B[1;32m 1518\u001B[0m \u001B[38;5;129;01mor\u001B[39;00m _global_backward_pre_hooks \u001B[38;5;129;01mor\u001B[39;00m _global_backward_hooks\n\u001B[1;32m 1519\u001B[0m \u001B[38;5;129;01mor\u001B[39;00m _global_forward_hooks \u001B[38;5;129;01mor\u001B[39;00m _global_forward_pre_hooks):\n\u001B[0;32m-> 1520\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mforward_call\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 1522\u001B[0m \u001B[38;5;28;01mtry\u001B[39;00m:\n\u001B[1;32m 1523\u001B[0m result \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;01mNone\u001B[39;00m\n", "File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/module.py:1520\u001B[0m, in \u001B[0;36mModule._call_impl\u001B[0;34m(self, *args, **kwargs)\u001B[0m\n\u001B[1;32m 1515\u001B[0m \u001B[38;5;66;03m# If we don't have any hooks, we want to skip the rest of the logic in\u001B[39;00m\n\u001B[1;32m 1516\u001B[0m \u001B[38;5;66;03m# this function, and just call forward.\u001B[39;00m\n\u001B[1;32m 1517\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m (\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_backward_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_backward_pre_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_forward_hooks \u001B[38;5;129;01mor\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_forward_pre_hooks\n\u001B[1;32m 1518\u001B[0m \u001B[38;5;129;01mor\u001B[39;00m _global_backward_pre_hooks \u001B[38;5;129;01mor\u001B[39;00m _global_backward_hooks\n\u001B[1;32m 1519\u001B[0m \u001B[38;5;129;01mor\u001B[39;00m _global_forward_hooks \u001B[38;5;129;01mor\u001B[39;00m _global_forward_pre_hooks):\n\u001B[0;32m-> 1520\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mforward_call\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43margs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[38;5;241;43m*\u001B[39;49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n\u001B[1;32m 1522\u001B[0m \u001B[38;5;28;01mtry\u001B[39;00m:\n\u001B[1;32m 1523\u001B[0m result \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;01mNone\u001B[39;00m\n",
"File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/linear.py:116\u001B[0m, in \u001B[0;36mLinear.forward\u001B[0;34m(self, input)\u001B[0m\n\u001B[1;32m 115\u001B[0m \u001B[38;5;28;01mdef\u001B[39;00m \u001B[38;5;21mforward\u001B[39m(\u001B[38;5;28mself\u001B[39m, \u001B[38;5;28minput\u001B[39m: Tensor) \u001B[38;5;241m-\u001B[39m\u001B[38;5;241m>\u001B[39m Tensor:\n\u001B[0;32m--> 116\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mF\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mlinear\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;28;43minput\u001B[39;49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mweight\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mbias\u001B[49m\u001B[43m)\u001B[49m\n", "File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/conv.py:460\u001B[0m, in \u001B[0;36mConv2d.forward\u001B[0;34m(self, input)\u001B[0m\n\u001B[1;32m 459\u001B[0m \u001B[38;5;28;01mdef\u001B[39;00m \u001B[38;5;21mforward\u001B[39m(\u001B[38;5;28mself\u001B[39m, \u001B[38;5;28minput\u001B[39m: Tensor) \u001B[38;5;241m-\u001B[39m\u001B[38;5;241m>\u001B[39m Tensor:\n\u001B[0;32m--> 460\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43m_conv_forward\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;28;43minput\u001B[39;49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mweight\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mbias\u001B[49m\u001B[43m)\u001B[49m\n",
"\u001B[0;31mTypeError\u001B[0m: linear(): argument 'input' (position 1) must be Tensor, not tuple" "File \u001B[0;32m/nix/store/4mv9lb8b1vjx88y2i7px1r2s8p3xlr7d-python3-3.11.9-env/lib/python3.11/site-packages/torch/nn/modules/conv.py:456\u001B[0m, in \u001B[0;36mConv2d._conv_forward\u001B[0;34m(self, input, weight, bias)\u001B[0m\n\u001B[1;32m 452\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mpadding_mode \u001B[38;5;241m!=\u001B[39m \u001B[38;5;124m'\u001B[39m\u001B[38;5;124mzeros\u001B[39m\u001B[38;5;124m'\u001B[39m:\n\u001B[1;32m 453\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m F\u001B[38;5;241m.\u001B[39mconv2d(F\u001B[38;5;241m.\u001B[39mpad(\u001B[38;5;28minput\u001B[39m, \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_reversed_padding_repeated_twice, mode\u001B[38;5;241m=\u001B[39m\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mpadding_mode),\n\u001B[1;32m 454\u001B[0m weight, bias, \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mstride,\n\u001B[1;32m 455\u001B[0m _pair(\u001B[38;5;241m0\u001B[39m), \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mdilation, \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mgroups)\n\u001B[0;32m--> 456\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mF\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mconv2d\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;28;43minput\u001B[39;49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mweight\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mbias\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mstride\u001B[49m\u001B[43m,\u001B[49m\n\u001B[1;32m 457\u001B[0m \u001B[43m \u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mpadding\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mdilation\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mgroups\u001B[49m\u001B[43m)\u001B[49m\n",
"\u001B[0;31mRuntimeError\u001B[0m: Given groups=1, weight of size [32, 1, 3, 3], expected input[32, 6, 16, 16] to have 1 channels, but got 6 channels instead"
] ]
} }
], ],
"source": [ "source": [
"%%time\n",
"# Split train and test\n", "# Split train and test\n",
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, random_state=2)\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1)\n",
"\n", "\n",
"# Filter test data that contains no labels\n", "# Filter test data that contains no labels\n",
"# In Coursemology, the test data is guaranteed to have labels\n", "# In Coursemology, the test data is guaranteed to have labels\n",
@ -502,14 +577,12 @@
"\n", "\n",
"# Train and predict\n", "# Train and predict\n",
"model = Model()\n", "model = Model()\n",
"# model.fit(X_train, y_train)\n", "model.fit(X_train, y_train)\n",
"y_pred = model.predict(X_test)\n", "y_pred = model.predict(X_test)\n",
"print(y_pred[0])\n",
"print(y[0])\n",
"\n", "\n",
"# Evaluate model predition\n", "# Evaluate model predition\n",
"# Learn more: https://scikit-learn.org/stable/modules/classes.html#module-sklearn.metrics\n", "# Learn more: https://scikit-learn.org/stable/modules/classes.html#module-sklearn.metrics\n",
"# print(\"F1 Score (macro): {0:.2f}\".format(f1_score(y_test, y_pred, average='macro'))) # You may encounter errors, you are expected to figure out what's the issue." "print(\"F1 Score (macro): {0:.2f}\".format(f1_score(y_test, y_pred, average='macro'))) # You may encounter errors, you are expected to figure out what's the issue."
] ]
}, },
{ {

564
cs2109s/labs/final/scratchpad.ipynb
View File

@ -51,11 +51,7 @@
"**[TODO]**\n", "**[TODO]**\n",
"\n", "\n",
"##### 1. Descriptive Analysis\n", "##### 1. Descriptive Analysis\n",
"First step: Look at the target values. The target values are floats and NAs, which is interesting. NAs in the target data is a bit suspicious. However, despite being floats, the target values are actually ordinal. I'll convert them to ordinal values by just `Y.fillna(-1).astype(int).`. Now, I can do value counts and see that there are only 7 distinct values, including NaN. I will regard this as a classification problem with 7 classes.\n", "**[TODO]**\n",
"\n",
"Looking at the `X`, I realise each entry in the list is an `n` by 16 by 16 matrix. 16 by 16 matrix, my first idea is to look at them like images. Plotting the images showed no relevant info. `6 <= n <= 10`.\n",
"\n",
"I just realised this is a video dataset. I'll pad all the frames to be of size 10. so that i'll have a 2500 x 10 x 16 x 16 video datset. \n",
"\n", "\n",
"##### 2. Detection and Handling of Missing Values\n", "##### 2. Detection and Handling of Missing Values\n",
"**[TODO]**\n", "**[TODO]**\n",
@ -123,21 +119,19 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 1, "execution_count": 42,
"id": "cded1ed6", "id": "cded1ed6",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-27T16:15:03.602644Z", "end_time": "2024-04-28T06:46:52.407375Z",
"start_time": "2024-04-27T16:15:03.179277Z" "start_time": "2024-04-28T06:46:52.405317Z"
} }
}, },
"outputs": [], "outputs": [],
"source": [ "source": [
"import pandas\n",
"import pandas as pd\n", "import pandas as pd\n",
"import os\n", "import os\n",
"import numpy as np\n", "import numpy as np"
"import matplotlib.pyplot as plt"
] ]
}, },
{ {
@ -162,12 +156,12 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 2, "execution_count": 43,
"id": "6297e25a", "id": "6297e25a",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-27T16:15:06.411332Z", "end_time": "2024-04-28T06:46:52.453152Z",
"start_time": "2024-04-27T16:15:06.392391Z" "start_time": "2024-04-28T06:46:52.428539Z"
} }
}, },
"outputs": [ "outputs": [
@ -186,7 +180,8 @@
" data = np.load(f, allow_pickle=True).item()\n", " data = np.load(f, allow_pickle=True).item()\n",
" X = data['data']\n", " X = data['data']\n",
" y = data['label']\n", " y = data['label']\n",
" \n", "\n",
"\n",
"print('Number of data sample:', len(X))\n", "print('Number of data sample:', len(X))\n",
"print('Shape of the first data sample:', X[0].shape)\n", "print('Shape of the first data sample:', X[0].shape)\n",
"print('Shape of the third data sample:', X[2].shape)" "print('Shape of the third data sample:', X[2].shape)"
@ -200,49 +195,6 @@
"## Data Exploration & Preparation" "## Data Exploration & Preparation"
] ]
}, },
{
"cell_type": "code",
"outputs": [],
"source": [
"from sklearn.preprocessing import OrdinalEncoder\n",
"# Some Helper Functions\n",
"def show_images(images, n_row=5, n_col=5, figsize=[12,12]):\n",
" _, axs = plt.subplots(n_row, n_col, figsize=figsize)\n",
" axs = axs.flatten()\n",
" for img, ax in zip(images, axs):\n",
" ax.imshow(img, cmap='gray')\n",
" plt.show()\n",
"def nan_columns(X, threshold=0.5):\n",
" count = X.shape[0] * threshold\n",
" nan_columns = X.isna().sum()\n",
" return nan_columns[nan_columns >= count].index\n",
"def zero_columns(X, threshold=0.5):\n",
" count = X.shape[0] * threshold\n",
" zero_cols = (X == 0).sum()\n",
" return zero_cols[zero_cols >= count].index\n",
"\n",
"def object_columns(X):\n",
" return X.dtypes[X.dtypes == 'object'].index\n",
"\n",
"def convert_to_ordinal(X, columns):\n",
" encoder = OrdinalEncoder()\n",
" return encoder.fit_transform(X[columns])\n",
"\n",
"def correlated_columns(X, threshold=0.99):\n",
" corr = X.corr()\n",
" upper = corr.where(np.triu(np.ones(corr.shape), k=1).astype(bool))\n",
" return [column for column in upper.columns if any(upper[column] > threshold)]"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-04-27T16:15:12.963319Z",
"start_time": "2024-04-27T16:15:12.025487Z"
}
},
"id": "f68b8b1c21eae6d6",
"execution_count": 3
},
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "2f6a464c", "id": "2f6a464c",
@ -253,79 +205,52 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 44,
"id": "3b1f62dd",
"metadata": {
"ExecuteTime": {
"end_time": "2024-04-28T06:46:52.472306Z",
"start_time": "2024-04-28T06:46:52.454360Z"
}
},
"outputs": [ "outputs": [
{ {
"name": "stdout", "ename": "ValueError",
"output_type": "stream", "evalue": "setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2250,) + inhomogeneous part.",
"text": [ "output_type": "error",
"2250\n" "traceback": [
"\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
"\u001B[0;31mValueError\u001B[0m Traceback (most recent call last)",
"Cell \u001B[0;32mIn[44], line 10\u001B[0m\n\u001B[1;32m 8\u001B[0m X6 \u001B[38;5;241m=\u001B[39m np\u001B[38;5;241m.\u001B[39marray([video[:\u001B[38;5;241m6\u001B[39m] \u001B[38;5;28;01mfor\u001B[39;00m video \u001B[38;5;129;01min\u001B[39;00m X])\n\u001B[1;32m 9\u001B[0m \u001B[38;5;66;03m# Now that they are consistent, we can convert them to a numpy array\u001B[39;00m\n\u001B[0;32m---> 10\u001B[0m X6 \u001B[38;5;241m=\u001B[39m \u001B[43mnp\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43marray\u001B[49m\u001B[43m(\u001B[49m\u001B[43mX\u001B[49m\u001B[43m)\u001B[49m\n",
"\u001B[0;31mValueError\u001B[0m: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2250,) + inhomogeneous part."
] ]
} }
], ],
"source": [ "source": [
"# Remove nans from the input. This needs to be done in the model for training data as well\n",
"not_nan_indices = np.argwhere(~np.isnan(np.array(y))).squeeze()\n", "not_nan_indices = np.argwhere(~np.isnan(np.array(y))).squeeze()\n",
"print(len(not_nan_indices))\n", "y = [y[i] for i in not_nan_indices]\n",
"y_filtered = [y[i] for i in not_nan_indices]\n", "X = [X[i] for i in not_nan_indices]\n",
"x_filtered = [X[i] for i in not_nan_indices]\n", "y = np.array(y).astype(int)\n",
"X = x_filtered\n", "\n",
"y = y_filtered\n", "# Since each video varies in length, we will take the min length, 6, for each video\n",
"# show_images(X[0], 2, 5, [16, 16])\n", "X6 = np.array([video[:6] for video in X])\n",
"Y = pd.DataFrame(y)\n", "# Now that they are consistent, we can convert them to a numpy array\n",
"# show_images(X[0], 1, 10, [10, 1])\n", "X6 = np.array(X)\n"
"# show_images(X[1], 1, 10, [10, 1])\n", ]
"# show_images(X[2], 1, 10, [10, 1])\n",
"# show_images(X[3], 1, 10, [10, 1])\n",
"# Y[:10].T\n",
"# print(type(X[0]))"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-04-27T16:15:14.719386Z",
"start_time": "2024-04-27T16:15:14.712849Z"
}
},
"id": "3b1f62dd",
"execution_count": 4
}, },
{ {
"cell_type": "code", "cell_type": "code",
"outputs": [ "outputs": [],
{
"name": "stdout",
"output_type": "stream",
"text": [
"(2250, 10, 256)\n"
]
}
],
"source": [ "source": [
"# We can now try to pad the videos to be of size 10\n", "pd.DataFrame(y).value_counts()\n",
"\n", "# From this, we know that we need to undersample or upsample the data. We will pick understampling as the data is quite large, and understampling will reduce the training time."
"def process_video(video):\n",
" L = video.shape[0]\n",
" if L < 10:\n",
" return np.concatenate([video, np.zeros((10 - L, 16, 16))]).reshape(10, -1)\n",
" return video.reshape(10, -1).astype(np.float32)\n",
"\n",
"L_max = 10\n",
"X_array = np.zeros((len(X), 10, 256))\n",
"for i, video in enumerate(X):\n",
" X_array[i] = process_video(video)\n",
"np.expand_dims(X_array, axis=2).shape\n",
"print(X_array.shape)\n",
"X_array = np.reshape(X_array, (X_array.shape[0], X_array.shape[1], 256)).shape\n",
"# flattened_data = print(flattened_data)"
], ],
"metadata": { "metadata": {
"collapsed": false, "collapsed": false
"ExecuteTime": {
"end_time": "2024-04-27T16:15:17.056545Z",
"start_time": "2024-04-27T16:15:17.014489Z"
}
}, },
"id": "558f2d74562bc7c8", "id": "dd66bb1efa4e602c",
"execution_count": 5 "execution_count": null
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
@ -337,16 +262,13 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": null,
"id": "4bb9cdfb", "id": "4bb9cdfb",
"metadata": { "metadata": {},
"ExecuteTime": {
"end_time": "2024-04-27T16:12:36.843437Z",
"start_time": "2024-04-27T16:12:36.842009Z"
}
},
"outputs": [], "outputs": [],
"source": [] "source": [
"np.isnan(X6).sum() # We know that there is quite a few NaNs in the data. However, I will not be figuring out which column / nan has this value. Instead we can just take the average of each image, adn use that as the input to the nan"
]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
@ -358,16 +280,16 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": null,
"id": "ed1c17a1", "id": "ed1c17a1",
"metadata": { "metadata": {},
"ExecuteTime": {
"end_time": "2024-04-27T16:12:36.845318Z",
"start_time": "2024-04-27T16:12:36.843930Z"
}
},
"outputs": [], "outputs": [],
"source": [] "source": [
"# Check if there are outliers\n",
"# We can check if there are outliers by checking the max and min values of each video\n",
"np.max(X6, axis=3)\n",
"# From this we can see that there are values whic exceed 255, and thus, we can clip that."
]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
@ -379,16 +301,29 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": 83,
"id": "ad3ab20e", "id": "ad3ab20e",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-27T16:12:36.847266Z", "end_time": "2024-04-28T06:59:16.949196Z",
"start_time": "2024-04-27T16:12:36.845985Z" "start_time": "2024-04-28T06:59:16.943398Z"
} }
}, },
"outputs": [], "outputs": [
"source": [] {
"data": {
"text/plain": "0\n0 300\n1 300\n2 300\n3 300\n4 300\n5 300\nName: count, dtype: int64"
},
"execution_count": 83,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Handling Undersampling\n",
"pd.DataFrame(y).value_counts()\n",
"# There is a class imbalance, and we will need to undersample the data"
]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
@ -400,12 +335,12 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": 44,
"id": "29ddbbcf", "id": "29ddbbcf",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-27T16:12:36.849483Z", "end_time": "2024-04-28T06:46:52.478781Z",
"start_time": "2024-04-27T16:12:36.848012Z" "start_time": "2024-04-28T06:46:52.477156Z"
} }
}, },
"outputs": [], "outputs": [],
@ -421,12 +356,12 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": 44,
"id": "93f82e42", "id": "93f82e42",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-27T16:12:36.852862Z", "end_time": "2024-04-28T06:46:52.483551Z",
"start_time": "2024-04-27T16:12:36.851617Z" "start_time": "2024-04-28T06:46:52.482068Z"
} }
}, },
"outputs": [], "outputs": [],
@ -450,16 +385,76 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "outputs": [
"id": "19174365", {
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([1800, 6, 16, 16])\n",
"(1800,)\n",
"<class 'numpy.ndarray'>\n"
]
}
],
"source": [
"import torch\n",
"\n",
"# Reduce the data to 6 frames\n",
"X = np.array([video[:6] for video in X])\n",
"tensor_videos = torch.tensor(X, dtype=torch.float32)\n",
"# Clip values to 0 and 255\n",
"tensor_videos = np.clip(tensor_videos, 0, 255)\n",
"# Replace NaNs in each frame, with the average of the frame. This was generated with GPT\n",
"for i in range(tensor_videos.shape[0]):\n",
" for j in range(tensor_videos.shape[1]):\n",
" tensor_videos[i][j][torch.isnan(tensor_videos[i][j])] = torch.mean(tensor_videos[i][j][~torch.isnan(tensor_videos[i][j])])\n",
" \n",
"# Undersample the data for each of the 6 classes. Select max of 300 samples for each class\n",
"# Very much generated with the assitance of chatGPT with some modifications\n",
"# Get the indices of each class\n",
"indices = [np.argwhere(y == i).squeeze(1) for i in range(6)]\n",
"# Get the number of samples to take for each class\n",
"num_samples_to_take = 300\n",
"# Get the indices of the samples to take\n",
"indices_to_take = [np.random.choice(indices[i], num_samples_to_take, replace=True) for i in range(6)]\n",
"# Concatenate the indices\n",
"indices_to_take = np.concatenate(indices_to_take)\n",
"# Select the samples\n",
"tensor_videos = tensor_videos[indices_to_take]\n",
"y = y[indices_to_take]\n"
],
"metadata": { "metadata": {
"collapsed": false
},
"id": "19174365",
"execution_count": 82
},
{
"cell_type": "code",
"outputs": [
{
"data": {
"text/plain": "torch.Size([1800, 1, 6, 16, 16])"
},
"execution_count": 85,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# This is the extra channel dimention to work with the conv3d\n",
"tensor_videos = tensor_videos.unsqueeze(1)\n",
"tensor_videos.shape"
],
"metadata": {
"collapsed": false,
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-27T16:12:36.854714Z", "end_time": "2024-04-28T07:01:44.496557Z",
"start_time": "2024-04-27T16:12:36.853430Z" "start_time": "2024-04-28T07:01:44.492973Z"
} }
}, },
"outputs": [], "id": "8b6bcf332c355e9d",
"source": [] "execution_count": 85
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
@ -471,14 +466,9 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": null,
"id": "a85808bf", "id": "a85808bf",
"metadata": { "metadata": {},
"ExecuteTime": {
"end_time": "2024-04-27T16:12:36.856476Z",
"start_time": "2024-04-27T16:12:36.855157Z"
}
},
"outputs": [], "outputs": [],
"source": [] "source": []
}, },
@ -492,14 +482,9 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": null,
"id": "dbcde626", "id": "dbcde626",
"metadata": { "metadata": {},
"ExecuteTime": {
"end_time": "2024-04-27T16:12:36.858522Z",
"start_time": "2024-04-27T16:12:36.857080Z"
}
},
"outputs": [], "outputs": [],
"source": [] "source": []
}, },
@ -521,18 +506,95 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 6, "execution_count": 230,
"id": "d8dffd7d", "id": "d8dffd7d",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-27T16:15:21.497276Z", "end_time": "2024-04-28T07:57:09.790124Z",
"start_time": "2024-04-27T16:15:20.501754Z" "start_time": "2024-04-28T07:57:09.780591Z"
} }
}, },
"outputs": [], "outputs": [],
"source": [ "source": [
"import torch\n", "from torch import nn\n",
"from torch import nn" "class CNN3D(nn.Module):\n",
" def __init__(self):\n",
" super(CNN3D, self).__init__()\n",
" self.conv1 = nn.Conv3d(1, 12, 2, 1,2)\n",
" self.mp = nn.AvgPool3d(2)\n",
" self.relu = nn.LeakyReLU()\n",
" self.fc1 = nn.Linear(3888, 6)\n",
" self.fc2 = nn.Linear(128, 6)\n",
" self.flatten = nn.Flatten()\n",
" def forward(self, x):\n",
" x = self.conv1(x)\n",
" x = self.mp(x)\n",
" x = self.relu(x)\n",
" \n",
" # print(x.shape)\n",
" \n",
" x = x.view(-1, 3888)\n",
" x = self.fc1(x)\n",
" # x = self.fc2(x)\n",
" return x\n",
" \n",
"def train(model, criterion, optimizer, loader, epochs = 10):\n",
" for epoch in range(epochs):\n",
" for idx, (inputs, labels) in enumerate(loader):\n",
" optimizer.zero_grad()\n",
" outputs = model(inputs)\n",
" loss = criterion(outputs, labels)\n",
" loss.backward()\n",
" optimizer.step()\n",
" print(f'Epoch {epoch}, Loss: {loss.item()}')\n",
" return model\n",
"def process_data(X, y):\n",
" y = np.array(y)\n",
" X = np.array([video[:6] for video in X])\n",
" tensor_videos = torch.tensor(X, dtype=torch.float32)\n",
" # Clip values to 0 and 255\n",
" tensor_videos = np.clip(tensor_videos, 0, 255)\n",
" # Replace NaNs in each frame, with the average of the frame. This was generated with GPT\n",
" for i in range(tensor_videos.shape[0]):\n",
" for j in range(tensor_videos.shape[1]):\n",
" tensor_videos[i][j][torch.isnan(tensor_videos[i][j])] = torch.mean(tensor_videos[i][j][~torch.isnan(tensor_videos[i][j])])\n",
" # Undersample the data for each of the 6 classes. Select max of 300 samples for each class\n",
" # Very much generated with the assitance of chatGPT with some modifications\n",
" # Get the indices of each class\n",
" indices = [np.argwhere(y == i).squeeze(1) for i in range(6)]\n",
" # Get the number of samples to take for each class\n",
" num_samples_to_take = 300\n",
" # Get the indices of the samples to take\n",
" indices_to_take = [np.random.choice(indices[i], num_samples_to_take, replace=True) for i in range(6)]\n",
" # Concatenate the indices\n",
" indices_to_take = np.concatenate(indices_to_take)\n",
" # Select the samples\n",
" tensor_videos = tensor_videos[indices_to_take].unsqueeze(1)\n",
" y = y[indices_to_take]\n",
" return torch.Tensor(tensor_videos), torch.Tensor(y).long()\n",
"class Model():\n",
" def __init__(self):\n",
" self.model = CNN3D()\n",
" self.criterion = nn.CrossEntropyLoss()\n",
" self.optimizer = torch.optim.Adam(self.model.parameters(), lr=0.001)\n",
" def fit(self, X, y):\n",
" X, y = process_data(X, y)\n",
" train_dataset = torch.utils.data.TensorDataset(X, y)\n",
" train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=32, shuffle=True)\n",
" train(self.model, self.criterion, self.optimizer, train_loader)\n",
" def predict(self, X):\n",
" self.model.eval()\n",
"\n",
" X = np.array([video[:6] for video in X])\n",
" tensor_videos = torch.tensor(X, dtype=torch.float32)\n",
" # Clip values to 0 and 255\n",
" tensor_videos = np.clip(tensor_videos, 0, 255)\n",
" # Replace NaNs in each frame, with the average of the frame. This was generated with GPT\n",
" for i in range(tensor_videos.shape[0]):\n",
" for j in range(tensor_videos.shape[1]):\n",
" tensor_videos[i][j][torch.isnan(tensor_videos[i][j])] = torch.mean(tensor_videos[i][j][~torch.isnan(tensor_videos[i][j])])\n",
" X = torch.Tensor(tensor_videos.unsqueeze(1))\n",
" return np.argmax(self.model(X).detach().numpy(), axis=1)\n"
] ]
}, },
{ {
@ -545,106 +607,32 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 19, "execution_count": 217,
"id": "9245ab47", "id": "9245ab47",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-27T16:19:38.194596Z", "end_time": "2024-04-28T07:55:53.563103Z",
"start_time": "2024-04-27T16:19:37.776094Z" "start_time": "2024-04-28T07:55:53.544134Z"
} }
}, },
"outputs": [], "outputs": [],
"source": [ "source": [
"from sklearn.model_selection import train_test_split\n", "from sklearn.model_selection import train_test_split\n",
"# Split train and test\n", "\n",
"with open('data.npy', 'rb') as f:\n",
" data = np.load(f, allow_pickle=True).item()\n",
" X = data['data']\n",
" y = data['label']\n",
"\n",
"\n",
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1)\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1)\n",
"X_train = [process_video(video) for video in X_train]\n",
"X_test = [process_video(video) for video in X_test]\n",
"\n", "\n",
"y_train = np.array(y_train).astype(np.int64)\n", "not_nan_indices = np.argwhere(~np.isnan(np.array(y_test))).squeeze()\n",
"\n", "y_test = [y_test[i] for i in not_nan_indices]\n",
"X_tensor = torch.tensor(X_train, dtype=torch.float32)\n", "X_test = [X_test[i] for i in not_nan_indices]\n",
"y_tensor = torch.tensor(y_train, dtype=torch.long)\n", "\n"
"\n",
"train_dataset = torch.utils.data.TensorDataset(X_tensor, y_tensor)\n",
"train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=32, shuffle=True)"
] ]
}, },
{
"cell_type": "code",
"outputs": [],
"source": [
"class Model(nn.Module):\n",
" def __init__(self):\n",
" super(Model, self).__init__()\n",
" self.input_size = 256\n",
" self.hidden_layers = 64\n",
" self.num_layers = 1\n",
" self.num_classes = 6\n",
" \n",
" self.lstm = nn.LSTM(self.input_size, self.hidden_layers, self.num_layers, batch_first=True)\n",
" self.fc = nn.Linear(self.hidden_layers, self.num_classes)\n",
" def forward(self, x):\n",
" h0 = torch.zeros(self.num_layers, x.size(0), self.hidden_layers).to(x.device)\n",
" c0 = torch.zeros(self.num_layers, x.size(0), self.hidden_layers).to(x.device)\n",
"\n",
" # Forward propagate LSTM\n",
" out, _ = self.lstm(x, (h0, c0))\n",
" \n",
" out = self.fc(out[:, -1, :])\n",
" return out "
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-04-27T16:20:46.738811Z",
"start_time": "2024-04-27T16:20:46.734583Z"
}
},
"id": "7396b295037aa70f",
"execution_count": 25
},
{
"cell_type": "code",
"outputs": [],
"source": [],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-04-27T16:12:37.998501Z",
"start_time": "2024-04-27T16:12:37.997472Z"
}
},
"id": "9057629fbaaa8571",
"execution_count": 8
},
{
"cell_type": "code",
"outputs": [],
"source": [
"def train_model(model, loss_fn, optimizer, train_loader, num_epochs=10):\n",
" model.train()\n",
" for epoch in range(num_epochs):\n",
" running_loss = 0.0\n",
" for inputs, labels in train_loader:\n",
" optimizer.zero_grad()\n",
" outputs = model(inputs)\n",
" loss = loss_fn(outputs, labels)\n",
" loss.backward()\n",
" optimizer.step()\n",
" running_loss += loss.item()\n",
" print(f\"Epoch {epoch + 1}, Loss: {running_loss / len(train_loader)}\")\n"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-04-27T16:19:57.048696Z",
"start_time": "2024-04-27T16:19:57.045181Z"
}
},
"id": "c3901cf56e12eade",
"execution_count": 21
},
{ {
"cell_type": "code", "cell_type": "code",
"outputs": [ "outputs": [
@ -652,34 +640,62 @@
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"Epoch 1, Loss: nan\n", "Epoch 0, Loss: 4.225716590881348\n",
"Epoch 2, Loss: nan\n", "Epoch 1, Loss: 0.9198675155639648\n",
"Epoch 3, Loss: nan\n", "Epoch 2, Loss: 1.7365752458572388\n",
"Epoch 4, Loss: nan\n", "Epoch 3, Loss: 0.4570190906524658\n",
"Epoch 5, Loss: nan\n", "Epoch 4, Loss: 0.11014104634523392\n",
"Epoch 6, Loss: nan\n", "Epoch 5, Loss: 0.24420055747032166\n",
"Epoch 7, Loss: nan\n", "Epoch 6, Loss: 0.03079795092344284\n",
"Epoch 8, Loss: nan\n", "Epoch 7, Loss: 0.07790327817201614\n",
"Epoch 9, Loss: nan\n", "Epoch 8, Loss: 0.07603466510772705\n",
"Epoch 10, Loss: nan\n" "Epoch 9, Loss: 0.04154537618160248\n",
"F1 Score (macro): 0.51\n"
] ]
} }
], ],
"source": [ "source": [
"model = Model()\n", "model = Model()\n",
"lossFn = nn.CrossEntropyLoss()\n", "model.fit(X_train, y_train)\n",
"optimizer = torch.optim.Adam(model.parameters(), lr=0.001)\n", "\n",
"train_model(model, lossFn, optimizer, train_loader, num_epochs=10)" "from sklearn.metrics import f1_score\n",
"\n",
"y_pred = model.predict(X_test)\n",
"print(\"F1 Score (macro): {0:.2f}\".format(f1_score(y_test, y_pred, average='macro'))) # You may encounter errors, you are expected to figure out what's the issue.\n"
], ],
"metadata": { "metadata": {
"collapsed": false, "collapsed": false,
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-27T16:20:49.798810Z", "end_time": "2024-04-28T07:57:38.644155Z",
"start_time": "2024-04-27T16:20:48.477326Z" "start_time": "2024-04-28T07:57:35.958882Z"
} }
}, },
"id": "dbb00fef60449a02", "id": "abb2d957f4a15bd2",
"execution_count": 26 "execution_count": 235
},
{
"cell_type": "code",
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"F1 Score (macro): 0.60\n"
]
}
],
"source": [
"\n"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-04-28T07:57:16.355215Z",
"start_time": "2024-04-28T07:57:16.281540Z"
}
},
"id": "37ff28a8da9dba6c",
"execution_count": 232
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",

8
cs2109s/labs/flake.lock
View File

@ -20,16 +20,16 @@
}, },
"nixpkgs": { "nixpkgs": {
"locked": { "locked": {
"lastModified": 1712310679, "lastModified": 1713714899,
"narHash": "sha256-XgC/a/giEeNkhme/AV1ToipoZ/IVm1MV2ntiK4Tm+pw=", "narHash": "sha256-+z/XjO3QJs5rLE5UOf015gdVauVRQd2vZtsFkaXBq2Y=",
"owner": "nixos", "owner": "nixos",
"repo": "nixpkgs", "repo": "nixpkgs",
"rev": "72da83d9515b43550436891f538ff41d68eecc7f", "rev": "6143fc5eeb9c4f00163267708e26191d1e918932",
"type": "github" "type": "github"
}, },
"original": { "original": {
"owner": "nixos", "owner": "nixos",
"ref": "nixos-23.11", "ref": "nixos-unstable",
"repo": "nixpkgs", "repo": "nixpkgs",
"type": "github" "type": "github"
} }

15
cs2109s/labs/flake.nix
View File

@ -1,5 +1,5 @@
{ {
description = "A very basic flake"; description = "Flake to setup CS2109S";
inputs = { inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable"; nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
@ -16,23 +16,26 @@
overlays = [(final: prev: { overlays = [(final: prev: {
pythonPackagesExtensions = [(py-final: py-prev: { pythonPackagesExtensions = [(py-final: py-prev: {
torch = py-final.torch-bin; torch = py-final.torch-bin;
torchvision = py-final.torchvision-bin;
torchaudio = py-final.torchaudio-bin;
})]; })];
})]; })];
devShell = pkgs.mkShell { devShell = pkgs.mkShell {
buildInputs = with pkgs; [ buildInputs = with pkgs; [
(python3.withPackages(ps: with ps; (python3.withPackages(ps: with ps;
[ let myoptuna = optuna.override {
torch = python3.pkgs.torch-bin;
torchaudio = python3.pkgs.torchaudio-bin;
torchvision = python3.pkgs.torchvision-bin;
};
in [
ipython ipython
jupyter jupyter
numpy numpy
pandas pandas
matplotlib matplotlib
torch torch-bin
scikit-learn scikit-learn
timeout-decorator timeout-decorator
torchvision torchvision-bin
seaborn seaborn
])) ]))
]; ];

16
cs2109s/labs/ps5/ps5.ipynb
View File

@ -30,7 +30,9 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {
"jp-MarkdownHeadingCollapsed": true
},
"source": [ "source": [
"## Important\n", "## Important\n",
"\n", "\n",
@ -43,7 +45,9 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {
"jp-MarkdownHeadingCollapsed": true
},
"source": [ "source": [
"## Introduction\n", "## Introduction\n",
"\n", "\n",
@ -54,7 +58,9 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {
"jp-MarkdownHeadingCollapsed": true
},
"source": [ "source": [
"## Getting Started\n", "## Getting Started\n",
"\n", "\n",
@ -2382,7 +2388,7 @@
"name": "python", "name": "python",
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython3", "pygments_lexer": "ipython3",
"version": "3.9.9" "version": "3.11.8"
}, },
"vscode": { "vscode": {
"interpreter": { "interpreter": {
@ -2391,5 +2397,5 @@
} }
}, },
"nbformat": 4, "nbformat": 4,
"nbformat_minor": 2 "nbformat_minor": 4
} }

116
cs2109s/labs/ps7/ps7.ipynb
View File

@ -23,21 +23,26 @@
"execution_count": 1, "execution_count": 1,
"id": "adfd1c67", "id": "adfd1c67",
"metadata": { "metadata": {
"ExecuteTime": {
"end_time": "2024-04-11T04:02:54.316718Z",
"start_time": "2024-04-11T04:02:53.604913Z"
},
"collapsed": false, "collapsed": false,
"jupyter": { "ExecuteTime": {
"outputs_hidden": false "end_time": "2024-04-27T17:03:32.690848Z",
"start_time": "2024-04-27T17:03:30.945851Z"
} }
}, },
"outputs": [ "outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/tmp/ipykernel_95268/3863716717.py:17: UserWarning: 'has_mps' is deprecated, please use 'torch.backends.mps.is_built()'\n",
" device = torch.device(\"mps\") if torch.has_mps else torch.device(\"cpu\")\n",
"/tmp/ipykernel_95268/3863716717.py:18: UserWarning: 'has_mps' is deprecated, please use 'torch.backends.mps.is_built()'\n",
" torch.has_mps\n"
]
},
{ {
"data": { "data": {
"text/plain": [ "text/plain": "False"
"True"
]
}, },
"execution_count": 1, "execution_count": 1,
"metadata": {}, "metadata": {},
@ -139,22 +144,19 @@
"execution_count": null, "execution_count": null,
"id": "a63a577557da6e87", "id": "a63a577557da6e87",
"metadata": { "metadata": {
"collapsed": false, "collapsed": false
"jupyter": {
"outputs_hidden": false
}
}, },
"outputs": [], "outputs": [],
"source": [] "source": []
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 11, "execution_count": 2,
"id": "7873ce10", "id": "7873ce10",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-03T08:51:30.782923Z", "end_time": "2024-04-27T17:03:40.731264Z",
"start_time": "2024-04-03T08:51:30.778415Z" "start_time": "2024-04-27T17:03:40.726572Z"
} }
}, },
"outputs": [], "outputs": [],
@ -417,12 +419,12 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 3, "execution_count": 4,
"id": "c78ee5e1", "id": "c78ee5e1",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-11T04:03:03.689966Z", "end_time": "2024-04-27T17:04:36.032494Z",
"start_time": "2024-04-11T04:03:03.667782Z" "start_time": "2024-04-27T17:04:35.981931Z"
} }
}, },
"outputs": [], "outputs": [],
@ -477,12 +479,12 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 86, "execution_count": 5,
"id": "4e0fdf18", "id": "4e0fdf18",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-07T03:15:42.540657Z", "end_time": "2024-04-27T17:04:39.560907Z",
"start_time": "2024-04-07T03:15:42.436629Z" "start_time": "2024-04-27T17:04:39.424591Z"
} }
}, },
"outputs": [ "outputs": [
@ -496,10 +498,8 @@
}, },
{ {
"data": { "data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAGFCAYAAAASI+9IAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAI1UlEQVR4nO3cr2uVfx/H8evczm8YTJSDwWEzajE5jCZ1oCzIBH+AMpBFwSKiCHNBsBks/gVOsGwWwxBElrZkGEuiIgxN0yXx3OG+71e6w3lfut+PRz4vrmt65pNP8NPp9Xq9BgCapvnXVr8AANuHKAAQogBAiAIAIQoAhCgAEKIAQIgCADHQ7wc7nc5GvgcAG6yf/6vspABAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEANb/QLsHVevXm21O3ny5F9+k/9vYmKivDlw4EB58/nz5/KmaZpmenq6vHn+/Hl58+vXr/KG3cNJAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACA6vV6v19cHO52Nfhd2kOHh4fLm06dPrZ7V51d0S7T5vdjMn+fRo0flzcOHD//+i7At9PPdc1IAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQAiIGtfgF2pjaXus3MzLR61tzcXHmzsrLS6lmb4dq1a612t27dKm+OHTvW6lnsXU4KAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCANHp9XmzWafT2eh3gT1hZGSk1e7du3flzY8fP8qbgwcPljfsDP38c++kAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAEAMbPULwE7W5kbRe/futXpWm5uKr1+/3upZ7F1OCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgDhQjz4r+Hh4fLmyZMn5c25c+fKm6Zpmm/fvpU3CwsLrZ7F3uWkAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABAuxGPb63a75c2ZM2fKm/v375c3x48fL2/aXGzXNE1z6dKl8mZ1dbXVs9i7nBQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAwoV4bJrbt2+32o2NjZU3p0+fbvWszTA0NNRqNzo6Wt4sLS2VN2tra+UNu4eTAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgDR6fV6vb4+2Ols9LuwRQYHB8ublZWV8ubIkSPlTdM0TZ9f0S3R5vdiM3+excXF8ubOnTvlzdu3b8sbNl8/3z0nBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAY2OoXYOudOnWqvDl06FB50/YiuOXl5fLm2bNn5c3Xr1/Lm800Pj5e3oyOjpY3U1NT5c3Y2Fh58/379/KGjeekAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCdXp+3lHU6nY1+F3aQiYmJ8mb//v2tnvXixYvyxmVr//Hx48fy5ujRo+VNm8v6Xr58Wd7wZ/r5595JAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBciAe72KtXr8qbCxculDdv3rwpb86ePVve8GdciAdAiSgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAMbDVLwBsnKWlpfLm4sWL5c3Q0FB5w/bkpABAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAuCUVdohut1veTE5Olje9Xq+8mZ2dLW/YnpwUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAMKFeLDJTpw40Wo3MzNT3hw+fLi8WV9fL2/m5+fLG7YnJwUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAcCEe/IEHDx6UN5OTk62e1eZyuzYuX75c3iwsLGzAm7AVnBQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAwoV4RU+fPi1v/vnnn/Lm9evX5U3TNM3i4mJ5s7q6Wt50u93yZt++feVN0zTNkSNHypvx8fHyZmJiorwZGhoqb3q9XnnTNE2zvr5e3rS53G5ubq68YfdwUgAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFACITq/P27k6nc5Gv8uOcPfu3fJmamqqvGn7593msrX5+fnyZmRkpLwZHBwsb5qm/QVym6HN39Py8nKrZ928ebO8WVhYaPUsdqd+fpecFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIt6QWdbvd8ubx48flzY0bN8qbptl9N4o2zfb+md6/f1/eTE5OtnrWhw8fWu3gf9ySCkCJKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgDhQrxt6sqVK61258+fL2/Gx8dbPavq58+frXazs7Plze/fv8ub6enp8ubLly/lzdraWnkDf4ML8QAoEQUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgXIgHsEe4EA+AElEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAGKg3w/2er2NfA8AtgEnBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAOLfZHAnDT21VNIAAAAASUVORK5CYII=", "text/plain": "<Figure size 640x480 with 1 Axes>",
"text/plain": [ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAGFCAYAAAASI+9IAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAJbklEQVR4nO3cMWieVR/G4ZM2CDHYGCrUCCo2GAex0EEq6FLo1kExDkZLzKAg1CVrwaK4BjsKSqBQiJkK3TNkqLabmkUMVixCFEEFNRmC4XUQbj6ww/t/vuRtTK9rzs15oE1/nsEz1Ov1eg0AWmuH7vYHALB/iAIAIQoAhCgAEKIAQIgCACEKAIQoABDD/f7g0NDQXn4HAHusn/9X2U0BgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBi+2x/A7jp8+HB5c/Xq1fJmamqqvHn22WfLm9Za+/PPPzvtgDo3BQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQDCK6kHzPBw/Y/0yJEj5c2TTz5Z3oyMjJQ3rXklFQbJTQGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgPIh3wJw6daq8OXHixB58CfBf5KYAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEB7EO2Dee++98mZsbKy8WVtbK282NzfLG2Cw3BQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAwoN4+9Tp06c77Z5//vld/pI7u3TpUnmztbW1B18C7CY3BQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYDwIN4+NTk52Wl3+PDhXf6SO1tfXx/IOcBguSkAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEF5JpW1sbAxkA+x/bgoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIA4UG8fero0aMDO+uHH34YyIZ/vPDCC512s7Ozu/wld/bTTz+VN5988kl54+/Q/uSmAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABAexNunXnvttYGd9d133w3srIPm/Pnz5c3Fixc7nTXIRxKr3nzzzfJmcXGx01nvvvtupx39cVMAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQAiKFer9fr6weHhvb6W/gfX331Vafd008/Xd6cO3euvFleXi5vDqLffvutvHnggQf24Evu7PPPPy9vHnroofJmamqqvNnc3CxvWmttbGys047W+vnn3k0BgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBi+2x9wL+jyguQjjzzS6awuL09+8cUXnc7az0ZHR8ubpaWl8ubBBx8sb7755pvyprXW3nnnnfJmZWWlvJmYmChvurzG+thjj5U3rbU2Pz9f3ly6dKnTWfciNwUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGA8CDeADz66KPlzfj4eKezfv755/Km6wNt+9nc3Fx5c/bs2fJma2urvHn//ffLm9a6PW7XxY8//jiQTZffi9Zae+ONN8obD+L1z00BgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIDyIx4H00ksvDeSchYWF8ubTTz/dgy/ZPTMzM+XNM888swdfcmdLS0sDO+te5KYAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEB7EG4DV1dXy5uuvv+501uOPP17ePPfcc+XNzZs3y5uuHn744fLm+PHje/Al//brr78O5JyuDh2q/3ffK6+8Ut6MjIyUNysrK+VNa619+OGHnXb0x00BgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIDyINwA7OzvlzZdfftnprKeeeqq8uXbtWnkzPT1d3ly/fr28aa210dHR8mZ8fLzTWVXHjh0byDldLSwslDcvvvhiebO+vl7eXLhwobxprbW//vqr047+uCkAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEEO9Xq/X1w8ODe31t7ALfv/99/Lm/vvvL29u3LhR3pw/f768aa21tbW18uatt94qbz766KPyZnt7u7z54IMPypvWuv05zc3NlTcTExPlze3bt8ubkydPljetdfs7zj/6+efeTQGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFAAIUQAgPIh3wLz66qvlzeXLl8ub4eHh8uaXX34pb1pr7YknnihvdnZ2ypsrV66UN9PT0+VNn79yd83GxkZ5c+bMmfJmfX29vOH/40E8AEpEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAIj6q2bsa8vLy+VNl8cOP/744/Lm6NGj5U1rrd2+fbu8mZ2dLW9u3bpV3ux333//fXkzPz9f3njc7uBwUwAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQBCFACIoV6v1+vrBzs8msbBNTMzU94sLi52Ouu+++7rtBuELr8Xff7K/cvq6mp58/bbb5c33377bXnDf0M/f/fcFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACBEAYAQBQDCg3gMzOTkZKfdxYsXy5vXX3+901lVf/zxR3nz8ssvdzrrs88+K2+2t7c7ncXB5EE8AEpEAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgBAFAEIUAAhRACC8kgpwj/BKKgAlogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQIgCACEKAIQoABCiAECIAgAhCgCEKAAQogBAiAIAIQoAhCgAEKIAQAz3+4O9Xm8vvwOAfcBNAYAQBQBCFAAIUQAgRAGAEAUAQhQACFEAIEQBgPgbLvku86SCoJIAAAAASUVORK5CYII="
"<Figure size 640x480 with 1 Axes>"
]
}, },
"metadata": {}, "metadata": {},
"output_type": "display_data" "output_type": "display_data"
@ -508,7 +508,7 @@
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"Label: 3\n" "Label: 6\n"
] ]
} }
], ],
@ -561,12 +561,12 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 7, "execution_count": 6,
"id": "6dd33b95", "id": "6dd33b95",
"metadata": { "metadata": {
"ExecuteTime": { "ExecuteTime": {
"end_time": "2024-04-11T04:03:51.459649Z", "end_time": "2024-04-27T17:04:45.792747Z",
"start_time": "2024-04-11T04:03:51.434499Z" "start_time": "2024-04-27T17:04:45.419030Z"
} }
}, },
"outputs": [ "outputs": [
@ -808,43 +808,6 @@
"__Tip:__ Don't be worried if your model takes a while to train. Your mileage may also vary depending on your CPU. But if you would like to speed things up, you can consider making use of your device's GPU to parallelize the matrix computations." "__Tip:__ Don't be worried if your model takes a while to train. Your mileage may also vary depending on your CPU. But if you would like to speed things up, you can consider making use of your device's GPU to parallelize the matrix computations."
] ]
}, },
{
"cell_type": "code",
"execution_count": 1,
"id": "e95b10e89d6cfe43",
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [],
"source": [
" self.conv = nn.Sequential(\n",
" nn.Conv2d(3, 32, (3, 3)),\n",
" nn.MaxPool2d((2, 2)),\n",
" nn.LeakyReLU(0.1),\n",
" nn.Conv2d(32, 64, (3, 3)),\n",
" nn.MaxPool2d((2, 2)),\n",
" nn.LeakyReLU(0.1),\n",
" )\n",
"\n",
" self.fc = nn.Sequential(\n",
" nn.Linear(64, 256),\n",
" nn.LeakyReLU(0.1),\n",
" nn.Linear(256, 128),\n",
" nn.LeakyReLU(0.1),\n",
" nn.Linear(128, classes)\n",
" )\n",
" \n",
" def forward(self, x):\n",
" # YOUR CODE HERE\n",
" x = self.conv(x)\n",
" x = x.view(x.shape[0], 64, 6*6).mean(2) # GAP – do not remove this line\n",
" x = self.fc(x)\n",
" return x\n"
]
},
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 8, "execution_count": 8,
@ -882,13 +845,13 @@
"evalue": "", "evalue": "",
"output_type": "error", "output_type": "error",
"traceback": [ "traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
"\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)", "\u001B[0;31mKeyboardInterrupt\u001B[0m Traceback (most recent call last)",
"File \u001b[0;32m<timed exec>:33\u001b[0m\n", "File \u001B[0;32m<timed exec>:33\u001B[0m\n",
"File \u001b[0;32m<timed exec>:21\u001b[0m, in \u001b[0;36mtrain_model\u001b[0;34m(loader, model, device)\u001b[0m\n", "File \u001B[0;32m<timed exec>:21\u001B[0m, in \u001B[0;36mtrain_model\u001B[0;34m(loader, model, device)\u001B[0m\n",
"File \u001b[0;32m/opt/homebrew/anaconda3/envs/cs2109s-ay2223s1/lib/python3.9/site-packages/torch/_tensor.py:396\u001b[0m, in \u001b[0;36mTensor.backward\u001b[0;34m(self, gradient, retain_graph, create_graph, inputs)\u001b[0m\n\u001b[1;32m 387\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m has_torch_function_unary(\u001b[38;5;28mself\u001b[39m):\n\u001b[1;32m 388\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m handle_torch_function(\n\u001b[1;32m 389\u001b[0m Tensor\u001b[38;5;241m.\u001b[39mbackward,\n\u001b[1;32m 390\u001b[0m (\u001b[38;5;28mself\u001b[39m,),\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 394\u001b[0m create_graph\u001b[38;5;241m=\u001b[39mcreate_graph,\n\u001b[1;32m 395\u001b[0m inputs\u001b[38;5;241m=\u001b[39minputs)\n\u001b[0;32m--> 396\u001b[0m \u001b[43mtorch\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mautograd\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mbackward\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mgradient\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mretain_graph\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcreate_graph\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43minputs\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43minputs\u001b[49m\u001b[43m)\u001b[49m\n", "File \u001B[0;32m/opt/homebrew/anaconda3/envs/cs2109s-ay2223s1/lib/python3.9/site-packages/torch/_tensor.py:396\u001B[0m, in \u001B[0;36mTensor.backward\u001B[0;34m(self, gradient, retain_graph, create_graph, inputs)\u001B[0m\n\u001B[1;32m 387\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m has_torch_function_unary(\u001B[38;5;28mself\u001B[39m):\n\u001B[1;32m 388\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m handle_torch_function(\n\u001B[1;32m 389\u001B[0m Tensor\u001B[38;5;241m.\u001B[39mbackward,\n\u001B[1;32m 390\u001B[0m (\u001B[38;5;28mself\u001B[39m,),\n\u001B[0;32m (...)\u001B[0m\n\u001B[1;32m 394\u001B[0m create_graph\u001B[38;5;241m=\u001B[39mcreate_graph,\n\u001B[1;32m 395\u001B[0m inputs\u001B[38;5;241m=\u001B[39minputs)\n\u001B[0;32m--> 396\u001B[0m \u001B[43mtorch\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mautograd\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mbackward\u001B[49m\u001B[43m(\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mgradient\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mretain_graph\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mcreate_graph\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43minputs\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[43minputs\u001B[49m\u001B[43m)\u001B[49m\n",
"File \u001b[0;32m/opt/homebrew/anaconda3/envs/cs2109s-ay2223s1/lib/python3.9/site-packages/torch/autograd/__init__.py:173\u001b[0m, in \u001b[0;36mbackward\u001b[0;34m(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)\u001b[0m\n\u001b[1;32m 168\u001b[0m retain_graph \u001b[38;5;241m=\u001b[39m create_graph\n\u001b[1;32m 170\u001b[0m \u001b[38;5;66;03m# The reason we repeat same the comment below is that\u001b[39;00m\n\u001b[1;32m 171\u001b[0m \u001b[38;5;66;03m# some Python versions print out the first line of a multi-line function\u001b[39;00m\n\u001b[1;32m 172\u001b[0m \u001b[38;5;66;03m# calls in the traceback and some print out the last line\u001b[39;00m\n\u001b[0;32m--> 173\u001b[0m \u001b[43mVariable\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_execution_engine\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrun_backward\u001b[49m\u001b[43m(\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;66;43;03m# Calls into the C++ engine to run the backward pass\u001b[39;49;00m\n\u001b[1;32m 174\u001b[0m \u001b[43m \u001b[49m\u001b[43mtensors\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mgrad_tensors_\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mretain_graph\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcreate_graph\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43minputs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 175\u001b[0m \u001b[43m \u001b[49m\u001b[43mallow_unreachable\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mTrue\u001b[39;49;00m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43maccumulate_grad\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mTrue\u001b[39;49;00m\u001b[43m)\u001b[49m\n", "File \u001B[0;32m/opt/homebrew/anaconda3/envs/cs2109s-ay2223s1/lib/python3.9/site-packages/torch/autograd/__init__.py:173\u001B[0m, in \u001B[0;36mbackward\u001B[0;34m(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)\u001B[0m\n\u001B[1;32m 168\u001B[0m retain_graph \u001B[38;5;241m=\u001B[39m create_graph\n\u001B[1;32m 170\u001B[0m \u001B[38;5;66;03m# The reason we repeat same the comment below is that\u001B[39;00m\n\u001B[1;32m 171\u001B[0m \u001B[38;5;66;03m# some Python versions print out the first line of a multi-line function\u001B[39;00m\n\u001B[1;32m 172\u001B[0m \u001B[38;5;66;03m# calls in the traceback and some print out the last line\u001B[39;00m\n\u001B[0;32m--> 173\u001B[0m \u001B[43mVariable\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43m_execution_engine\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mrun_backward\u001B[49m\u001B[43m(\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;66;43;03m# Calls into the C++ engine to run the backward pass\u001B[39;49;00m\n\u001B[1;32m 174\u001B[0m \u001B[43m \u001B[49m\u001B[43mtensors\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mgrad_tensors_\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mretain_graph\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mcreate_graph\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43minputs\u001B[49m\u001B[43m,\u001B[49m\n\u001B[1;32m 175\u001B[0m \u001B[43m \u001B[49m\u001B[43mallow_unreachable\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[38;5;28;43;01mTrue\u001B[39;49;00m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43maccumulate_grad\u001B[49m\u001B[38;5;241;43m=\u001B[39;49m\u001B[38;5;28;43;01mTrue\u001B[39;49;00m\u001B[43m)\u001B[49m\n",
"\u001b[0;31mKeyboardInterrupt\u001b[0m: " "\u001B[0;31mKeyboardInterrupt\u001B[0m: "
] ]
} }
], ],
@ -1444,10 +1407,7 @@
"execution_count": 1, "execution_count": 1,
"id": "bd138177d2e4877", "id": "bd138177d2e4877",
"metadata": { "metadata": {
"collapsed": false, "collapsed": false
"jupyter": {
"outputs_hidden": false
}
}, },
"outputs": [], "outputs": [],
"source": [] "source": []

75
cs2109s/labs/ps8/ps8.ipynb
View File

File diff suppressed because one or more lines are too long

BIN
cs3230/main.pdf
View File

Binary file not shown.

16
cs3230/main.typ
View File

@ -211,3 +211,19 @@ Proof by contradiction
- Therefore $"LCS"(n, m) = "LCS"(n-1, m-1)::a_n$ - Therefore $"LCS"(n, m) = "LCS"(n-1, m-1)::a_n$
If $a_n != b_m, "LCS"(n, m) = max("LCS"(n-1,m), "LCS"(n, m-1))$ If $a_n != b_m, "LCS"(n, m) = max("LCS"(n-1,m), "LCS"(n, m-1))$
= Tutorial 6
Greedy vs DP
= Lecture 7 Greedy
== DP Recap
- Express solutions recursively
- Small number(polynomial) of subproblems
- Huge overlap among subproblems, so recursive may take exponential time
- Compute recursive iteratively in bottom up fashion
== Greedy
Recast the problem so that only 1 subproblem needs to be solved at each step.
= Lecture 8 Amortized Analysis

BIN
cs3230/pa3.pdf
View File

Binary file not shown.

303
cs3234/labs/midterm_project.v
View File

@ -956,13 +956,34 @@ Proof.
fold_unfold_tactic list_reverse. fold_unfold_tactic list_reverse.
Qed. Qed.
Theorem about_list_reverse_acc_and_list_append :
forall (V: Type)
(v : V)
(vs acc : list V),
list_reverse_acc V vs (list_append V acc (v :: nil)) =
list_append V (list_reverse_acc V vs acc) (v :: nil).
Proof.
intros V v vs.
induction vs as [ | v' vs' IHvs'].
- intro acc.
rewrite -> (fold_unfold_list_reverse_acc_nil V (list_append V acc (v :: nil))).
rewrite -> (fold_unfold_list_reverse_acc_nil V acc).
reflexivity.
- intro acc.
rewrite -> (fold_unfold_list_reverse_acc_cons V v' vs' (list_append V acc (v :: nil))).
rewrite -> (fold_unfold_list_reverse_acc_cons V v' vs' acc).
rewrite <- (IHvs' (v' :: acc)).
rewrite -> (fold_unfold_list_append_cons V v' acc).
reflexivity.
Qed.
Theorem list_reverse_alt_satisfies_the_specification_of_list_reverse : Theorem list_reverse_alt_satisfies_the_specification_of_list_reverse :
specification_of_list_reverse list_reverse_alt. specification_of_list_reverse list_reverse_alt.
Proof. Proof.
unfold specification_of_list_reverse. unfold specification_of_list_reverse.
intros append. intros append.
unfold specification_of_list_append. unfold specification_of_list_append.
intros [S_append_nil S_append_cons]. intro S_append.
split. split.
- intros V. - intros V.
unfold list_reverse_alt. unfold list_reverse_alt.
@ -970,59 +991,108 @@ Proof.
- intros V v vs. - intros V v vs.
unfold list_reverse_alt. unfold list_reverse_alt.
rewrite -> (fold_unfold_list_reverse_acc_cons V v vs nil). rewrite -> (fold_unfold_list_reverse_acc_cons V v vs nil).
induction vs as [ | v' vs' IHvs' ]. rewrite <- (fold_unfold_list_append_nil V (v :: nil)).
+ rewrite -> (fold_unfold_list_reverse_acc_nil V (v :: nil)). rewrite -> (about_list_reverse_acc_and_list_append V v vs nil).
rewrite -> (fold_unfold_list_reverse_acc_nil V nil). (* FIXME: This feels pretty sketchy and I don't fully pass in arguments, so there might be some Coq magic here. *)
rewrite -> (S_append_nil V (v :: nil)). Check (there_is_at_most_one_list_append_function V append list_append S_append
reflexivity. list_append_satisfies_the_specification_of_list_append).
+ rewrite -> (fold_unfold_list_reverse_acc_cons V v' vs' (v :: nil)). rewrite -> (there_is_at_most_one_list_append_function V append list_append S_append
rewrite -> (fold_unfold_list_reverse_acc_cons V v' vs' nil). list_append_satisfies_the_specification_of_list_append
Search list_append. (list_reverse_acc V vs nil) (list_append V nil (v :: nil))).
Abort. rewrite -> (nil_is_left_neutral_wrt_list_append V).
reflexivity.
Qed.
(* {END} *)
(* Theorem about_list_reverse_acc_and_list_append : *) (* {task_5_h_4} *)
(* forall append : forall W : Type, list W -> list W -> list W, *) Theorem there_is_at_most_one_list_reverse_function :
(* specification_of_list_append append -> *) forall (V: Type)
(* forall (V: Type) *) (list_reverse_1 list_reverse_2 : forall V : Type, list V -> list V),
(* (acc acc' vs : list V), *) specification_of_list_reverse list_reverse_1 ->
(* list_reverse_acc V vs (append V acc acc') = append V (list_reverse_acc V vs acc) acc'. *) specification_of_list_reverse list_reverse_2 ->
(* Proof. *) forall vs : list V,
(* intros append [S_append_nil S_append_cons]. *) list_reverse_1 V vs = list_reverse_2 V vs.
(* intros V acc acc' vs. *) Proof.
(* revert acc' vs. *) intros V list_reverse_1 list_reverse_2.
(* induction acc as [ | v acc'' IHacc'']. *) intros S_list_reverse_1 S_list_reverse_2.
(* - intros acc' vs. *) unfold specification_of_list_reverse in S_list_reverse_1.
(* rewrite -> (S_append_nil V acc'). *) assert (specification_of_list_append list_append) as S_list_append.
(* Check (fold_unfold_list_reverse_acc_nil). *) { exact list_append_satisfies_the_specification_of_list_append. }
assert (S_list_reverse_1 := (S_list_reverse_1 list_append S_list_append)).
assert (S_list_reverse_2 := (S_list_reverse_2 list_append S_list_append)).
destruct S_list_reverse_1 as [S_list_reverse_1_nil S_list_reverse_1_cons].
destruct S_list_reverse_2 as [S_list_reverse_2_nil S_list_reverse_2_cons].
(* reverse vs (append acc acc') = append (reverse vs acc) acc' *) intro vs.
(* reverse vs (a :: b :: nil) = [reverse vs (a :: nil)] (b :: nil) *) induction vs as [ | v' vs' IHvs' ].
(* reverse abc nil - rewrite -> (S_list_reverse_2_nil V).
= reverse bc [a, nil] exact (S_list_reverse_1_nil V).
= reverse c [b, a, nil] - rewrite -> (S_list_reverse_2_cons V v' vs').
= reverse nil [c, b, a, nil] rewrite -> (S_list_reverse_1_cons V v' vs').
*) rewrite -> IHvs'.
reflexivity.
Qed.
(* {END} *)
(* TODO The name of this isn't that great *)
(* {task_5_h_5} *)
Proposition list_reverse_and_list_reverse_alt_equiv :
forall (V : Type)
(vs : list V),
list_reverse V vs = list_reverse_alt V vs.
Proof.
intro V.
exact (there_is_at_most_one_list_reverse_function V list_reverse list_reverse_alt
list_reverse_satisfies_the_specification_of_list_reverse
list_reverse_alt_satisfies_the_specification_of_list_reverse).
Qed.
(* {END} *)
(* Lemma about_list_append : *) (* TODO The name of this isn't that great *)
(* forall append : forall W : list W -> list W -> list W, *)
(* specification_of_list_append append -> *) (* {task_5_h_6} *)
(* (V : Type) *) Proposition list_reverse_alt_and_list_append_commute_with_each_other :
(* (v v' : V) *) forall (V : Type)
(* append V *) (v1s v2s : list V),
list_append V (list_reverse_alt V v2s) (list_reverse_alt V v1s) = list_reverse_alt V (list_append V v1s v2s).
Proof.
intros V v1s v2s.
rewrite <- (list_reverse_and_list_reverse_alt_equiv V v2s).
rewrite <- (list_reverse_and_list_reverse_alt_equiv V v1s).
rewrite <- (list_reverse_and_list_reverse_alt_equiv V (list_append V v1s v2s)).
exact (list_reverse_and_list_append_commute_with_each_other V v1s v2s).
Qed.
(* {END} *)
(* TODO The name of this isn't that great *)
(* {task_5_h_7} *)
Proposition list_reverse_alt_and_list_length_commute_with_each_other :
forall (V : Type)
(vs : list V),
list_length V (list_reverse_alt V vs) = list_length V vs.
Proof.
intros V vs.
rewrite <- (list_reverse_and_list_reverse_alt_equiv V vs).
exact (list_reverse_and_list_length_commute_with_each_other V vs).
Qed.
(* {END} *)
(* TODO The name of this isn't that great *)
(* {task_5_h_8} *)
Proposition list_reverse_alt_is_involutory :
forall (V : Type)
(vs : list V),
list_reverse_alt V (list_reverse_alt V vs) = vs.
Proof.
intros V vs.
rewrite <- (list_reverse_and_list_reverse_alt_equiv V vs).
rewrite <- (list_reverse_and_list_reverse_alt_equiv V (list_reverse V vs)).
exact (list_reverse_is_involutory V vs).
Qed.
(* Definition there_is_at_most_one_list_reverse_function : *)
(* forall (V : Type) *)
(* (f g : forall V : Type, list V -> list V), *)
(* specification_of_list_reverse f -> *)
(* specification_of_list_reverse g -> *)
(* forall vs : list V, *)
(* f V vs = g V vs. *)
(* Proof. *)
(* intros V. *)
(* intros f g S_f S_g. *)
(* intros vs. *)
(* ********** *) (* ********** *)
@ -1133,7 +1203,8 @@ Proof.
rewrite -> (fold_unfold_list_map_nil V V (fun v => v)). rewrite -> (fold_unfold_list_map_nil V V (fun v => v)).
reflexivity. reflexivity.
- intros V v vs. - intros V v vs.
rewrite -> (fold_unfold_list_map_cons V V (fun v => v) v vs). unfold list_copy_as_list_map.
rewrite -> (fold_unfold_list_map_cons V V (fun a => a) v vs).
reflexivity. reflexivity.
Qed. Qed.
@ -1152,7 +1223,7 @@ Proposition list_map_and_list_length_commute_with_each_other :
Proof. Proof.
intros V W f vs. intros V W f vs.
induction vs as [ | v' vs' IHvs' ]. induction vs as [ | v' vs' IHvs' ].
- rewrite ->(fold_unfold_list_map_nil V W f). - rewrite ->(fold_unfold_list_map_nil V W f).
rewrite -> (fold_unfold_list_length_nil V). rewrite -> (fold_unfold_list_length_nil V).
exact (fold_unfold_list_length_nil W). exact (fold_unfold_list_length_nil W).
- rewrite -> (fold_unfold_list_map_cons V W f v' vs'). - rewrite -> (fold_unfold_list_map_cons V W f v' vs').
@ -1192,15 +1263,72 @@ Proof.
h. Do list_map and list_reverse commute with each other and if so how? h. Do list_map and list_reverse commute with each other and if so how?
*) *)
Proposition list_map_and_list_reverse_commute_with_each_other :
forall (V W : Type)
(f : V -> W)
(vs : list V),
list_reverse W (list_map V W f vs) = list_map V W f (list_reverse V vs).
Proof.
intros V W f vs.
induction vs as [ | v vs' IHvs' ].
- rewrite -> (fold_unfold_list_map_nil V W f).
rewrite -> (fold_unfold_list_reverse_nil V).
rewrite -> (fold_unfold_list_reverse_nil W).
rewrite -> (fold_unfold_list_map_nil V W f).
reflexivity.
- rewrite -> (fold_unfold_list_map_cons V W f v vs').
rewrite -> (fold_unfold_list_reverse_cons W (f v) (list_map V W f vs')).
rewrite -> IHvs'.
rewrite -> (fold_unfold_list_reverse_cons V v vs').
rewrite <- (fold_unfold_list_map_nil V W f).
rewrite <- (fold_unfold_list_map_cons V W f v nil).
Check list_map_and_list_append_commute_with_each_other.
rewrite -> (list_map_and_list_append_commute_with_each_other V W f (list_reverse V vs') (v :: nil)).
reflexivity.
Qed.
(* (*
i. Do list_map and list_reverse_alt commute with each other and if so how? i. Do list_map and list_reverse_alt commute with each other and if so how?
*) *)
Proposition list_map_and_list_reverse_alt_commute_with_each_other :
forall (V W : Type)
(f : V -> W)
(vs : list V),
list_reverse_alt W (list_map V W f vs) = list_map V W f (list_reverse_alt V vs).
Proof.
intros V W f vs.
Check list_reverse_and_list_reverse_alt_equiv.
rewrite <- (list_reverse_and_list_reverse_alt_equiv W (list_map V W f vs)).
rewrite <- (list_reverse_and_list_reverse_alt_equiv V vs).
exact (list_map_and_list_reverse_commute_with_each_other V W f vs).
Qed.
(* (*
j. Define a unit-test function for the map function j. Define a unit-test function for the map function
and verify that your implementation satisfies it. and verify that your implementation satisfies it.
*) *)
Fixpoint evenp (n : nat): bool :=
match n with
| 0 => true
| S n' => match n' with
| 0 => false
| S n'' => evenp n''
end
end.
Definition test_list_map (candidate : forall V W : Type, (V -> W) -> list V -> list W) :=
(eqb_list nat Nat.eqb (candidate nat nat (fun v => v) nil) nil) &&
(eqb_list nat Nat.eqb (candidate nat nat (fun v => v) (1 :: 2:: 3 :: nil)) (1 :: 2 :: 3 :: nil)) &&
(eqb_list nat Nat.eqb (candidate nat nat (fun v => S v) (1 :: 2:: 3 :: nil)) (2 :: 3 :: 4 :: nil)) &&
(eqb_list bool Bool.eqb (candidate nat bool evenp (1 :: 2:: 3 :: nil)) (false :: true :: false :: nil)).
Compute test_list_map list_map.
(* ********** *) (* ********** *)
(* A study of the polymorphic fold-right and fold-left functions: *) (* A study of the polymorphic fold-right and fold-left functions: *)
@ -1242,6 +1370,7 @@ Definition specification_of_list_fold_left (fold_left : forall V W : Type, W ->
a. Implement the fold-right function recursively. a. Implement the fold-right function recursively.
*) *)
Fixpoint list_fold_right (V W : Type) (nil_case : W) (cons_case : V -> W -> W) (vs : list V) : W := Fixpoint list_fold_right (V W : Type) (nil_case : W) (cons_case : V -> W -> W) (vs : list V) : W :=
match vs with match vs with
nil => nil =>
@ -1314,44 +1443,110 @@ Qed.
d. Prove that each of your implementations satisfies the corresponding specification. d. Prove that each of your implementations satisfies the corresponding specification.
*) *)
Theorem list_fold_left_satisfies_the_specification_of_fold_left :
specification_of_list_fold_left list_fold_left.
Proof.
unfold specification_of_list_fold_left.
split.
- exact fold_unfold_list_fold_left_nil.
- exact fold_unfold_list_fold_left_cons.
Qed.
Theorem list_fold_right_satisfies_the_specification_of_fold_right :
specification_of_list_fold_right list_fold_right.
Proof.
unfold specification_of_list_fold_right.
split.
- exact fold_unfold_list_fold_right_nil.
- exact fold_unfold_list_fold_right_cons.
Qed.
(* (*
e. Which function do foo and bar (defined just below) compute? e. Which function do foo and bar (defined just below) compute?
*) *)
(*
Definition foo (V : Type) (vs : list V) := Definition foo (V : Type) (vs : list V) :=
list_fold_right V (list V) nil (fun v vs => v :: vs) vs. list_fold_right V (list V) nil (fun v vs => v :: vs) vs.
*)
(* Compute test_list_copy foo.
Definition bar (V : Type) (vs : list V) := Definition bar (V : Type) (vs : list V) :=
list_fold_left V (list V) nil (fun v vs => v :: vs) vs. list_fold_left V (list V) nil (fun v vs => v :: vs) vs.
*)
Compute test_list_reverse bar.
(* (*
f. Implement the length function either as an instance of list_fold_right or as an instance of list_fold_left, and justify your choice. f. Implement the length function either as an instance of list_fold_right or as an instance of list_fold_left, and justify your choice.
*) *)
Definition list_length_using_list_fold_left (V: Type) (vs : list V): nat :=
list_fold_left V nat 0 (fun _ length => S (length)) vs.
Compute test_list_length list_length_using_list_fold_left.
(* (*
g. Implement the copy function either as an instance of list_fold_right or as an instance of list_fold_left, and justify your choice. g. Implement the copy function either as an instance of list_fold_right or as an instance of list_fold_left, and justify your choice.
*) *)
Definition list_copy_using_list_fold_right (V: Type) (vs : list V): (list V) :=
list_fold_right V (list V) nil (fun v acc => v :: acc) vs.
Compute test_list_copy list_copy_using_list_fold_right.
(* (*
h. Implement the append function either as an instance of list_fold_right or as an instance of list_fold_left, and justify your choice. h. Implement the append function either as an instance of list_fold_right or as an instance of list_fold_left, and justify your choice.
*) *)
Definition list_append_using_list_fold_right (V: Type) (v1s v2s : list V): (list V) :=
list_fold_right V (list V) v2s (fun v acc => v :: acc) v1s.
Compute test_list_append list_append_using_list_fold_right.
(* (*
i. Implement the reverse function either as an instance of list_fold_right or as an instance of list_fold_left, and justify your choice. i. Implement the reverse function either as an instance of list_fold_right or as an instance of list_fold_left, and justify your choice.
*) *)
Definition list_reverse_using_list_fold_left (V: Type) (vs : list V): (list V) :=
list_fold_left V (list V) nil (fun v acc => v :: acc) vs.
Compute test_list_reverse list_reverse_using_list_fold_left.
(* (*
j. Implement the map function either as an instance of list_fold_right or as an instance of list_fold_left, and justify your choice. j. Implement the map function either as an instance of list_fold_right or as an instance of list_fold_left, and justify your choice.
*) *)
Definition list_map_using_list_fold_right (V W: Type) (f : V -> W) (vs : list V): (list W) :=
list_fold_right V (list W) nil (fun v w => (f v) :: w) vs.
Compute test_list_map list_map_using_list_fold_right.
(* (*
k. Implement eqb_list either as an instance of list_fold_right or as an instance of list_fold_left, and justify your choice. k. Implement eqb_list either as an instance of list_fold_right or as an instance of list_fold_left, and justify your choice.
*) *)
(* FIXME: This feels very hacky, because of the match result with, but I have no idea how to consider if the first list finishes iteration,
then the result should be false without the match case outside of the fold function
*)
Definition eqb_list_using_list_fold_right (V : Type) (eqb_V: V -> V -> bool) (v1s v2s : list V) :=
let result := list_fold_left V
(bool * list V)
(true, v2s)
(fun v1 acc =>
match acc with
| (false, v2::v2s') => (false, v2s')
| (false, nil) => (false, nil)
| (true, v2::v2s') => (eqb_V v1 v2, v2s')
| (true, nil) => (false, nil)
end)
v1s in
match result with
| (false, _) => false
| (true, nil) => true
| _ => false
end.
(* TODO Write testcases *)
(* (*
l. Implement list_fold_right as an instance of list_fold_left, using list_reverse. l. Implement list_fold_right as an instance of list_fold_left, using list_reverse.
*) *)

20
cs3234/labs/term-project.v
View File

@ -724,6 +724,26 @@ Definition specification_of_run (run : target_program -> expressible_value) :=
fetch_decode_execute_loop bcis nil = KO s -> fetch_decode_execute_loop bcis nil = KO s ->
run (Target_program bcis) = Expressible_msg s). run (Target_program bcis) = Expressible_msg s).
Theorem there_is_at_most_one_run_function :
forall (f g : target_program -> expressible_value),
specification_of_run f ->
specification_of_run g ->
forall (t: target_program),
f t = g t.
Proof.
intros f g.
unfold specification_of_run.
intros S_f S_g [bcis].
case (fetch_decode_execute_loop bcis nil) as [ds | s] eqn:H_fdel.
- destruct (S_f fetch_decode_execute_loop fetch_decode_execute_loop_satifies_the_specification) as [S_f_nil _].
destruct (S_g fetch_decode_execute_loop fetch_decode_execute_loop_satifies_the_specification) as [S_g_nil _].
case ds as [ | n ds' ] eqn:H_ds.
+ rewrite -> (S_g_nil bcis H_fdel).
exact (S_f_nil bcis H_fdel).
+ Check (S_f_nil bcis).
(* Task 4: (* Task 4:
a. time permitting, prove that the definition above specifies at most one function; a. time permitting, prove that the definition above specifies at most one function;
b. implement this function; and b. implement this function; and

BIN
ma1522/1522 Notes.pdf
View File

Binary file not shown.

2
ma1522/1522 Notes.tex
View File

@ -29,6 +29,8 @@
% ------------------------------------------------------------------------------ % ------------------------------------------------------------------------------
\layout*
\section{Linear Systems} \section{Linear Systems}
\hr \hr
\input{ch_01.tex} \input{ch_01.tex}

448
ma1522/ch_03.tex
View File

@ -3,10 +3,456 @@
\begin{defn}[Vector Definitions]\ \\ \begin{defn}[Vector Definitions]\ \\
\begin{itemize} \begin{itemize}
\item $n$-vector : $v = (v_1, v_2, ..., v_n)$ \item $n$-vector : $v = (v_1, v_2, \dots, v_n)$
\item $\vec{PQ} // \vec{P'Q'} \implies \vec{PQ} = \vec{P'Q'}$ \item $\vec{PQ} // \vec{P'Q'} \implies \vec{PQ} = \vec{P'Q'}$
\item $|| \vec{PQ} || = \sqrt{(a_2 - a_1)^2 + (b_2 - b_1)^2}$ \item $|| \vec{PQ} || = \sqrt{(a_2 - a_1)^2 + (b_2 - b_1)^2}$
\item $u + v = (u_1 + v_1, u_2 + v_2), u = (u_1, u_2), v = (v_1, v_2)$ \item $u + v = (u_1 + v_1, u_2 + v_2), u = (u_1, u_2), v = (v_1, v_2)$
\item $n$-vector can be viewed as a row matrix / column matrix \item $n$-vector can be viewed as a row matrix / column matrix
\item $\mathbb{R}^n = \left\{ (v_1, v_2, \dots, v_n) | v_1, v_2, \dots, v_n \right\} \in \mathbb{R}$, Euclidean $n$-space
\end{itemize} \end{itemize}
\end{defn} \end{defn}
A linear system is given in implicit form.
$\begin{cases}
a_{11}x_1 + a_{12}x_2 + \dots + a_{1n}x_n = b_1 \\
a_{21}x_1 + a_{22}x_2 + \dots + a_{2n}x_n = b_2 \\
\vdots \\
a_{m1}x_1 + a_{m2}x_2 + \dots + a_{mn}x_n = b_m \\
\end{cases}$
and its general solution is in the explicit form
\begin{defn} Straight lines in $\mathbb{R}^2$
\begin{itemize}
\item Implicit: $\left\{(x,y)|ax+by=c\right\}$
\item Explicit: (Equation Form)
\begin{itemize}
\item If $a \neq 0$, then $\left\{\left(\dfrac{c-bt}{a},t\right)| t \in \mathbb{R}\right\}$
\item If $b \neq 0$, then $\left\{\left(s,\dfrac{c-as}{b}\right)| s \in \mathbb{R}\right\}$
\end{itemize}
\item Explicit: (Vector form)
\begin{itemize}
\item A point on the line $(x_0, y_0)$ and its direction vector $(a,b)\neq 0$
\item $(x_0, y_0) + t(a,b)$
\item $\left\{(x_0 + ta, y_0 + tb|t \in \mathbb{R}\right\}$
\end{itemize}
\end{itemize}
\end{defn}
\begin{defn} Planes in $\mathbb{R}^3$
\begin{itemize}
\item Implicit: $\left\{(x,y,z)|ax+by+cz=d\right\}$
\item Explicit: (Equation Form)
\begin{itemize}
\item If $a \neq 0$, then $\left\{\left(\dfrac{c-bs-ct}{a},s,t\right)| s,t \in \mathbb{R}\right\}$
\item If $b \neq 0$, then $\left\{\left(s,\dfrac{d-as-ct}{b},t\right)| s,t \in \mathbb{R}\right\}$
\item If $c \neq 0$, then $\left\{\left(s,t,\dfrac{d-as-bt}{c}\right)| s,t \in \mathbb{R}\right\}$
\end{itemize}
\item Explicit: (Vector Form)
\begin{itemize}
\item $\left\{(x_0, y_0, z_0) + s(a_1, b_1,c_1) + t(a_2,b_2,c_2)|s,t \in \mathbb{R}\right\}$
\item $(a_1,b_1,c_1)$ and $(a_2, b_2, c_2)$ are non-parallel vectors, parallel to the plane
\end{itemize}
\end{itemize}
\end{defn}
Example: Plane is given by $\left\{(1+s-t, 2+s-2t,4-s-3t)|s,t\in\mathbb{R}\right\}$
\begin{itemize}
\item Let $x=1+s-t,y=2+s-2t,z=4-s-3t$
\item $\begin{amatrix}{2}
1&-1&x-1\\
1&-2&y-2\\
-1&-3&z-4
\end{amatrix} \to \begin{amatrix}{2}
1&-1&x-1\\
0&-1&-x + y-1\\
0&0&5x-4y+z-1
\end{amatrix}$
\item For system to be consistent, $5x-4y+z = 1$
\item Implicit: $\left\{(x,y,z)|5x-4y+z=1\right\}$
\end{itemize}
\begin{defn} Lines in $\mathbb{R}^3$ is the intersection of 2 non-parallel planes
\begin{itemize}
\item Implicit: $\left\{(x,y,z) | a_1x + b_1y + c_1z = d_1 \text{ and } a_2x+b_2y+c_2z=d_2\right\}$
\item Explicit $\left\{(x_0 + ta, y_0 + tb, z_0 + tc) | t \in \mathbb{R}\right\}$
\end{itemize}
\end{defn}
It is easy to go from implicit to explicit form, by just solving the linear equation. To have an implicit form of line, we need to find 2 non parallel planes $a_ix + b_iy+c_iz = d_i (i=1,2)$ containing the line
Example: Line is $\{(t-2, -2t+3,t+1) | t \in \mathbb{R}\}$.
\begin{itemize}
\item $t=x+2, -2t = y-3, t=z-1$
\item $\begin{amatrix}{1}
1&x+2\\
-2&y-3\\
1&z-1
\end{amatrix} \to \begin{amatrix}{1}
1&x+2\\
0&2x + y + 1\\
0&-x + z - 3
\end{amatrix}$
\item Implicit Form: $\{(x,y,z) | 2x+y+1 = 0 \text{ and } -x+z-3 = 0$
\end{itemize}
\subsection{Linear Combinations and Linear Spans}
\begin{defn} Linear Combination
\begin{itemize}
\item Linear combination of $v_1, v_2, \dots, v_k$ has the form
\item $c_1v_1 + c_2v_2 + \dots + c_kv_k, c_1,c_2,\dots,c_k \in \mathbb{R}$
\item $0$ is always a linear combination of $v_1, v_2, \dots, v_k$
\item to check if $v$ is a linear combination of $v_1, v_2, v_3$, solve for $(v_1, v_2, v_3 | v)$ and check if the REF is consistent
\end{itemize}
\end{defn}
\begin{defn} Linear Span
\begin{itemize}
\item Let $S = \{v_1, v_2, \dots, v_k\}$ be a subset of $\mathbb{R}^n$
\item Set of all linear combinations of $v_1, v_2,\dots,v_k$
\item $\{c_1v_1+ c_2v_2+\dots+c_kv_k | c_1,c_2,\dots,c_k \in \mathbb{R}$
\item is called the Span of $S$, $\text{Span}(S)$
\end{itemize}
\end{defn}
Example:
\begin{itemize}
\item Let $S = \{(2,1,3), (1,-1,2),(3,0,5)\}$
\subitem $(3,3,4) \in \text{Span}(S), (1,2,4) \not\in \text{Span}(S)$
\item Let $S = \{(1,0,0), (0,1,0),(0,0,1)\}$
\subitem for any $(x, y, z) \in \mathbb{R}^3, (x,y,z) = x(1,0,0) + y(0,1,0)+z(0,0,1)$
\subitem Therefore, $\text{Span}(S) = \mathbb{R}^3$
\end{itemize}
More Examples:
\begin{itemize}
\item Let $S = \{(1,0,0,-1),(0,1,1,0)\}$ be subset of $\mathbb{R}^4$
\begin{itemize}
\item $a(1,0,0-1) + b(0,1,1,0) = (a,b,b,-a), (a,b \in \mathbb{R})$
\item span(S) = $\{(a,b,b,-a) | a,b \in \mathbb{R}$
\end{itemize}
\item Let $V = \{(2a+b,a,3b-a) | a,b \in \mathbb{R} \} \subseteq \mathbb{R}^3$
\begin{itemize}
\item $a(2a+b,a,3b-a) = a(2,1,-1) + b(1,0,3), (a,b \in \mathbb{R})$
\item span(V) = $\{(a,b,b,-a) | a,b \in \mathbb{R}$
\item $V = \text{span}\{(2,1,-1),(1,0,3)\}$
\end{itemize}
\item Prove that span$\{(1,0,1),(1,1,0),(0,1,1)\} = \mathbb{R}^3$
\begin{itemize}
\item It is clear span$\{(1,0,1),(1,1,0),(0,1,1)\} \subseteq \mathbb{R}^3$
\item let $(x,y,z) \in \mathbb{R}^3$. Show that there exists $a,b,c \in \mathbb{R}$ s.t.
\begin{itemize}
\item $(x,y,z) = a(1,0,1) + b(1,1,0) + c(0,1,1)$
\item Do gaussian Elimination on $(1,0,1),(1,1,0),(0,1,1) | (x,y,z)$
\item if the system is always consistent then span$\{\dots\} = \mathbb{R}$
\item IF the system is consistent $\iff$ condition, then $\not \subseteq \mathbb{R}^3$
\end{itemize}
\end{itemize}
\end{itemize}
\begin{defn} Criterion for Span$(S) = \mathbb{R}^n$
\begin{itemize}
\item Let $S = \{v_1,v_2,\dots,\v_k\} \subseteq \mathbb{R}^n$
\item for an arbitrary $v \in \mathbb{R}^n$, we shall check the consistency of the equation $c_1v_1 + v_2v_2 + \dots + c_kv_k = v$
\item View $v_j$ as column vectors, $A = \left(v_1\ v_2\ \dots\ v_k\right)$
\subitem The equation is $Ax=v$
\item Let $R$ be a REF of $A$
\subitem $(A | v) \to (R | v')$
\subitem Since $v \in \mathbb{R}^n$ is arbitrary, $v' \in \mathbb{R}^n$ is also arbitrary
\subitem span$(S) = \mathbb{R}^n \iff Ax = v$ is consistent for every $v \in \mathbb{R}^n$
\subitem span$(S) = \mathbb{R}^n \iff Rx = v'$ is consistent for every $v' \in \mathbb{R}^n$
\subitem span$(S) = \mathbb{R}^n \iff$ rightmost column of $(R | v')$ is non pivot for any $v' \in \mathbb{R}^n$
\subitem span$(S) = \mathbb{R}^n \iff$ All rows in $R$ are nonzero
\end{itemize}
TLDR:
\begin{enumerate}
\item Let $S = \{v_1,v_2,\dots,\v_k\} \subseteq \mathbb{R}^n$
\item View $v_j$ as column vectors, $A = \left(v_1\ v_2\ \dots\ v_k\right)$
\item Find REF $R$ of $A$
\subitem If $R$ has zero row, then span$(S) \neq \mathbb{R}^n$
\subitem If $R$ has no zero row, then span$(S) = \mathbb{R}^n$
\end{enumerate}
Other rules
\begin{itemize}
\item $\mathbb{R}^n$ cannot be spanned by $n-1$ vectors
\subitem $\mathbb{R}^3$ cannot be spanned by 2 vectors
\end{itemize}
\end{defn}
\begin{defn} Properties of Linear Spans
\begin{itemize}
\item $0 \in $span(S), span(S) $\neq \emptyset$
\item $v \in $span(S) and $c \in \mathbb{R} \to cv \in$ span(S).
\item $u \in$ span(S) and $v \in$ span(S) $\to u+v \in$ span(S).
\end{itemize}
Check if $\text{span}(S_1) \subseteq \text{span}S_2$
\begin{itemize}
\item Let $S = \{v_1,v_2,\dots,\v_k\} \subseteq \mathbb{R}^n$
\item View $v_j$ as column vectors, $A = \left(v_1\ v_2\ \dots\ v_k\right)$
\item Check whether $Ax = u$, where $u$ is one of the vectors in $S_1$
\subitem If $Ax=u$ is consistent, u $\subseteq$ span(S)
\subitem If $Ax=u$ is inconsistent, u $\not\subseteq$ span(S)
\end{itemize}
\end{defn}
\subsection{Subspaces}
\begin{defn} Subspaces
\begin{itemize}
\item Let $V \subseteq \mathbb{R}^n$. Then $V$ is the subspace of $\mathbb{R}^n$
\item If there exists $v_1, \dots, v_k \in \mathbb{R}^n$, then V is the subspace spanned by $S = \{v_1, \dots, v_k\}$.
\end{itemize}
To validate if $V$ is subspace of $\mathbb{R}^n$
\begin{itemize}
\item $0 \in V$
\item $c \in \mathbb{R}$ and $v \in V \to cv \in V$
\item $u \in V$ and $v \in V \to u+v \in V$
\end{itemize}
\end{defn}
\begin{defn} Subspaces of $\mathbb{R}^1,\mathbb{R}^2,\mathbb{R}^3$
\end{defn}
\begin{defn} Solution Space
\end{defn}
\subsection{Linear Independence}
\begin{defn} Linear Independence
\begin{itemize}
\item Let $S=\{v_1,\dots,v_k\}$ be a subset of $\mathbb{R}^n$
\item Equation $c_1v_1 + \dots + c_kv_k = 0$ has trivial solution $c_1 = \dots = c_k = 0$
\item If equation has non-trivial solution, then
\begin{itemize}
\item $S$ is a linearly dependent set
\item $v_1, \dots, v_k$ is a linearly dependent set
\item Exists $c_1,\dots,c_k \in \mathbb{R}$ not all zero s.t. $c_1v_1 + \dots + c_kv_k = 0$
\end{itemize}
\item If equation has only the trivial solution, then
\begin{itemize}
\item S is linearly independent set
\item $v_1,\dots,v_k$ are linearly independent
\end{itemize}
\end{itemize}
How do you calculate whether trivial or non trivial? Solve for $Ax = 0$, perform gaussian elimination and identify if non-pivot columns exist. If there are non-pivot columns, then there are infintely many solutions, and thus, linearly dependent. If all columns are pivot, then system has only trivial solution, and thus, linearly independent set.
\end{defn}
\begin{defn} Properties of Linear Independence \\
Let $S_1$ and $S_2$ be finite subsets of $\mathbb{R}^n$ s.t. $S_1 \subseteq S_2$
\begin{itemize}
\item $S_1$ linearly dependent $\to$ $S_2$ linearly dependent
\item $S_2$ linearly independent $\to$ $S_1$ linearly independent
\end{itemize}
Let $S = \{v_1, v_2, \dots, v_k \} \subseteq \mathbb{R}^n, k \geq 2$
\begin{itemize}
\item $S$ is linearly dependent $\iff v_i$ is a linear combination of other vectors in $S$
\item S is linearly independent $\iff$ no vector in $S$ can be written as a linear combination of other vectors
\end{itemize}
Suppose $S = \{v_1, v_2, \dots, v_k\}$ is linearly dependent
\begin{itemize}
\item Let $V = \text{span}(S)$
\item If $v_i \in S$ is a linear combination of other vectors, remove $v_i$ from $S$.
\item Repeat until we obtain linearly independent set $S'$.
\item span$(S') = V$ and $S'$ has no redundant vector to span $V$.
\end{itemize}
Let $S = \{v_1, v_2, \dots, v_k \} \subseteq \mathbb{R}^n$ be linearly independent
\begin{enumerate}
\item Suppose span$(S) \neq \mathbb{R}^n$
\item pick $v_{k+1} \in \mathbb{R}^n$ but $v_{k+1} \not\in$ span$(S) \neq \mathbb{R}^n$
\item $\{v_1, ..., v_k, v_{k+1}\}$ is linearly independent
\item Repeat until $\{v_1, ..., v_k, ..., v_m\}$ is linearly independent and span$(S') = \mathbb{R}^n$
\end{enumerate}
\begin{itemize}
\item If $m > n$ then $S$ is linearly dependent
\item If $m < n$, then $S$ cannot span $\mathbb{R}^n$
\item If $m = n$, then $S$ is linearly independent and spans $\mathbb{R}^n$
\end{itemize}
\end{defn}
\begin{defn} Vector Spaces
\begin{itemize}
\item $V$ is vector space if $V$ is subspace of $\mathbb{R}^n$
\item $W$ and $V$ are vector space such that $W \subseteq V$, $W$ is a subspace of $V$
\end{itemize}
\end{defn}
\subsection{Bases}
\subsubsection{Definition}
$S$ is basis for $V$ if $S$ is
\begin{enumerate}
\item Linearly Independent
\item Span$(S) = V$
\end{enumerate}
\begin{note}
To show that Vector $S$ is a basis vector for $\mathbb{R}^n$, show that $S$ is linearly independent.
$S \xrightarrow[\text{Elimination}]{\text{Gaussian}} R$
\begin{enumerate}
\item Linear Independence
\begin{enumerate}
\item Show All columns are pivot. $\therefore$ system has only trivial solution
\item $S$ is linearly independent
\end{enumerate}
\item Span$(S) = \mathbb{R}^n$
\begin{enumerate}
\item REF has no zero row
\item span$(S) = \mathbb{R}^n$
\end{enumerate}
\item We can conclude $S$ is basis for $\mathbb{R}^n$
\end{enumerate}
\end{note}
Basis for Vector space $V$ contains
\begin{itemize}
\item Smallest possible number of vectors that spans $V$
\item largest possible number of vectors that is linearly independent $V$
\end{itemize}
\subsubsection{Coordinate Vector}
\begin{theorem} Coordinate Vectors
\begin{itemize}
\item Let $S = \{v_1, \dots, v_k \}$ be a subset of vector space $V$
$S$ is basis for $V$ $\iff$ every vector in $V$ can be written as $v = c_1v_1 + \dots + c_kv_k$
\item Let $S = \{v_1, \dots, v_k \}$ be a basis for vector space $V$
For every $v \in V$, there exists a unique $c_1, \dots, c_k \in \mathbb{R}$ such that $v = c_1v_1 + \dots + c_kv_k$
\end{itemize}
$(v)_S = (c_1, \dots, c_k)$ is the coordinate vector of $v$ relative to $S$.
Column vector $[v]_S = \begin{pmatrix}c_1\\c_2\\\vdots\\c_k\end{pmatrix}$ is also coordinate vector
Let $a = (v_1 \; \dots \; v_k)$. Then $[v]_S$ is the unique solution to $Ax = v$. We can write $A[v]_S = v$
\end{theorem}
\noindent To calculate Coordinate vector for $v$ relative to $S$, then view each vector in $S$ as a column vector, and let $A = (v_1 \: \dots \: v_k)$ and solve for $Ax=v$
\begin{note} Criterion for bases
Let $T = \{v_1, \dots, v_k\}$ be subset of $\mathbb{R}^n$
\begin{itemize}
\item $k > n$, then $T$ is linearly dependent
\item $k < n$, then span$(T) \neq \mathbb{R}^n$
\end{itemize}
If $T$ is basis for $\mathbb{R}^n$, then $k=n$
Let $V$ be a vector space having basis $S$ with $|S| = N$
\begin{itemize}
\item Let $T = \{v_1, \dots, v_k\}$be subset of $V$
\item if $k > n$, then $\{(v_1)_S, \dots, (v_k)_S\}$ is linearly dependent on $\mathbb{R}^n \therefore T$ is linearly dependent on $V$
\item if $k < n$, then span$(\{(v_1)_S, \dots, (v_k)_S\}) \neq \mathbb{R}^n \therefore$ span$(T) \neq V$
\end{itemize}
If $T$ is a basis for $V$, then $|T| = n = |S|$. If $S$ and $T$ are bases for vector space $V$ then $|S| = |T|$
\end{note}
\subsection{Dimensions}
Let $V$ be a vector space and $S$ be basis for $V$. $\dim(V) = |S|$
\subsubsection{Examples}
\begin{itemize}
\item $\varnothing$ is basis for $\{0\}$, $\dim(\{0\}) = |\varnothing| = 0$
\item $\mathbb{R}^n$ has standard basis $E = \{e_1,\dots,e_n\}, \dim(\mathbb{R}^n) = n$
\end{itemize}
\subsubsection{Dimension of Solution Space}
Let $Ax=0$ be a homogeneous linear system.
Solution set of $Ax = 0$ is a vector space $V$.
Let $R$ be REF of $A$. The \# non pivot columns = \# arbitrary params = dimension of $V$.
\subsubsection{Properties of Dimensions}
\begin{theorem} Dimensions
Let $S$ be a subset of vector space $V$, the following are equivalent
\begin{itemize}
\item $S$ is basis for $V$
\item $S$ is linearly independent and $|S| = \dim(V)$
\item $S$ spans $V$ and $|S| = \dim(V)$
\end{itemize}
Let $U$ be subspace of $V$. Then $\dim(U) \leq \dim(V)$
\begin{itemize}
\item $U = V \iff \dim(U) = \dim(V)$
\item $U \neq V \iff \dim(U) < \dim(V)$
\end{itemize}
Let $A$ be a square matrix of order $n$
\begin{itemize}
\item $A$ is invertible
\item $Ax=b$ has unique solution
\item $Ax=0$ has only trivial solution
\item RREF of $A$ is $I_n$
\item $\det(A) \neq 0$
\item rows of $A$ form basis for $\mathbb{R}^n$
\item columns of $A$ form basis for $\mathbb{R}^n$
\end{itemize}
\end{theorem}
\subsection{Transition Matrices}
\begin{defn} Let $V$ be vector space and
$S = \{u_1, \dots, u_k\}$ and $T$ be bases for $V$.
\begin{itemize}
\item $P = ([u_1]_T \dots [u_k]_T)$ is the transition matrix from $S$ to $T$
\item $P[w]_S = [w]_T,$ $\forall w \in V$
\end{itemize}
Let $S_1, S_2, S_3$ be bases for vector space $V$
\begin{itemize}
\item $P$ be transition matrix from $S_1$ to $S_2$
\item $Q$ be transition matrix from $S_2$ to $S_3$
\item $[v]_{S_1} \xrightarrow{P} [v]_{S_2} \xrightarrow{Q} [v]_{S_3}$
\item $[v]_{S_3} = Q[v]_{S_2} = QP[v]_{S_1}$
\item $QP$ is transition matrix from $S_1$ to $S_3$
\end{itemize}
Let $S, T$ be bases for vector space $V$
\begin{itemize}
\item $P$ be transition matrix from $S$ to $T$
\item $P$ is invertible
\item $P^{-1}$ is transition matrix from $T$ to $S$
\end{itemize}
\end{defn}
To calc transition matrices for $S$ and $T$, given that $S = \{(1, 1), (1, -1)\} = \{u_1, u_2\}, T = \{(1, 0), (1, 1) \} = \{v_1, v_2\}$
$ (v_1 v_2 | u_1 u_2) =
\left(\begin{array}{@{}*{2}{c}|c|c@{}}
1 & 1 & 1 & 1 \\
0 & 1 & 1 & 0 \\
\end{array}\right) \xrightarrow{R_1 - R_2}
\left(\begin{array}{@{}*{2}{c}|c|c@{}}
1 & 0 & 0 & 2 \\
0 & 1 & 1 & -1 \\
\end{array}\right) $
Transition matrix from $S$ to $T$: $P = \begin{pmatrix}0 & 2 \\ 1 & -1\end{pmatrix}$

9
ma1522/preamble.tex
View File

@ -3,6 +3,7 @@
\usepackage{graphicx} \usepackage{graphicx}
\usepackage{setspace} \usepackage{setspace}
\usepackage{geometry} \usepackage{geometry}
\usepackage{layout}
\usepackage{float} \usepackage{float}
\usepackage{hyperref} \usepackage{hyperref}
\usepackage[utf8]{inputenc} \usepackage[utf8]{inputenc}
@ -48,12 +49,12 @@
\setstretch{1.2} \setstretch{1.2}
\geometry{ \geometry{
textheight=9in, top=1cm,
textwidth=5.5in,
top=1in,
headheight=12pt, headheight=12pt,
headsep=25pt, headsep=25pt,
footskip=30pt footskip=30pt,
left=1.5cm,
right=1.5cm
} }
\newcommand\hr{ \newcommand\hr{