Data Powered Software - Solutions (IT3212 - NTNU)

This repository contains the code and resources for the Data Powered Software Solutions course (IT3212) at the Norwegian University of Science and Technology (NTNU) during Fall 2024. The course focuses on leveraging data to build efficient and effective software solutions. The projects have been developed as a group effort.

Navigation

Under Report_All_Assignments, you will find comprehensive reports for all assignments completed during the course. These reports detail the methodologies, analyses, and results obtained from various data-driven software projects. Those projects include:

1. A Data Exploration and Preprocessing Pipeline for Temperature Forecasting.
2. Classical Image Processing.
3. Basic and Advanced Modelling.
4. Deep Learning and Clustering.

Regarding the code implementations, this repository mainly includes my individual contributions to the assignments. While the reports reflect the collaborative work of the group, the following code files represent my personal implementations and solutions to the tasks assigned:

• Temperature forecasting (Assignment 1 & 3):
  • preprocessing_data.ipynb: Some contributions to data preprocessing (Data Exploration, Identifying data issues, handling outliers with Z-score method).
  • preprocessing.py: Data preprocessing pipeline for temperature forecasting.
• PCA (Assignment 2):
  • main.py: Implementation of Principal Component Analysis and Main script to run PCA experiments.
• Image Classification (Assignment 4):
  • main.py: Code for image classification tasks.
  • cnn.py: Model definitions for image classification.

Installation

• Create a virtual environment.
• Installing dependencies: pip install -r requirements.txt.
• Run main.py files in respective folders to execute the code.

Data

• Temperature forecasting dataset is included in the temperature forecasting folder.
• The image classification dataset is not included due to size constraints.
• Please refer to the dataset source mentioned in the report for downloading instructions and details about the different datasets used.

Experiments & Results

The experiments conducted are mainly comparison studies between different models and techniques applied to the datasets. The results and analyses of these experiments are detailed in the reports under the Report_All_Assignments.pdf file.

Project status

This repository contains working implementations for the course assignments and is suitable for exploration and reproduction of the presented experiments. Expect the code to be maintained for clarity and educational use rather than production deployment.

Getting help

Open an issue for bugs, feature requests, or questions. Include logs, commands used, and minimal reproduction steps to speed up triage. Or send me an email at arthur.testard.pro@gmail.com.

Acknowledgements & References

Special thanks to my project group members for their collaboration and contributions throughout the course. Their insights and efforts have been invaluable in completing the assignments successfully.

Also, always nice to aknowledge python open-source libraries such as NumPy, Pandas, Scikit-learn, PyTorch, and others that have been instrumental in implementing the solutions.

References are given under the respective reports in the Report_All_Assignments.pdf file.

License & Citation

The code here is free to use, as it has been developed for educational purposes. However, you should cite appropriately respecting works if used in academic work.