Week 10

16/02/26

In our final week of development, we worked on performing unit tests on both our scrapers as well as our react pages. As this was worth a big part of our third year project we wanted to ensure we were as thorough as possible when conducting testing for both our grade and the integrity of our application.

These tests were written using the pytest framework. The main area we wanted to test for our caroutlet scraper was structure detection, as the caroutlet website changed structure during our development process and we wanted to be ready in case of any unexpected changes. We made 2 structures and made sure the scraper could identify them based on key HTML elements. We also ensured the website stayed solid on bad HTTP responses and edge cases such as pages missing certain sections

Aside from identifying different structures, the same trend ran for our ebay scraper.

We carried out unit tests on the frontend by using Vitest, which is made for Vite applications, covering things such as core UI components and user flow like URL submission on the homepage, form rendering and field validation, the quote page displaying the correct details and the history page listing quotes correctly. We have x amount that covers the essential components, such as the user form and quote estimation page, which are vital to our system.
They are stored in the tests/ folder in our front end; we have 4, each one testing each page.

We used Vitest to perform unit tests on our frontend and it helped make sure everything is flowing and being displayed correctly. We performed unit tests on the homepage, the quoteform page and the view all quotes page.

We also performed integration tests on our backend which covered all the core functions of our program. The difference with integration testing compared to unit testing is that it validates how the main backend application behaves when its API endpoints are used together as a complete workflow. The tests simulated real HTTP requests and ensured they were being handled with the correct status codes and validation handling.

Machine Learning

With our Machine Learning model finished, the final thing left to do was to test it. We discussed which tests to carry out with our mentor in our final meeting together and he suggested we do 3 tests which consists of a T-test which checks whether the means of two groups (or a sample vs. a known value) are statistically different from each other, a skewness test which ssesses whether a dataset is asymmetrical (leans left or right) compared to a normal distribution and finally an Omnibus test which evaluates whether there are any overall differences among multiple groups or parameters, without specifying where those differences occur.

These were the results of our tests

Skewness (2.15)

T-test (mean residual = 13.25€, t-statistic = 28.5925)

Omnibus (21216.2533)