Week 7

26/01/26

Our main objective for the web application in week 7 was to implement our second scraper that allows users to scrape data from eBay listings. The first step to implementing this was authenticating using eBay’s OAuth API as eBay requires an application-level OAuth token using the Client Credentials flow. We send a post request to /identity/v1/oauth2/token and the script requests the scope  https://api.ebay.com/oauth/api_scope which returns a short lived access token that is reused for all subsequent API calls. 
Every ebay listing has a legacy item ID inside the /itm/ path. For example, https://www.ebay.com/itm/166123456789  where the highlighted number is the item ID. We use the url_to_item_id(url) function to strip the query params and return the ID. As ebays modern Browse API does not accept legacy IDs directly, we have to convert it to a RESTful item ID. This is a necessary step as it is required to connect old web URLs with the newer API ecosystem. With this REST compatible ID we can finally request the actual listing and extracting data.

Machine Learning

This week we were tasked with building our Machine learning model and building features to help make the predictions as accurate as possible:

The things we had to consider before we started is what values are we taking from the user in order to come up with a prediction, we also considered using our previous data and information on variables that have a strong correlation or a clear relation with something to help us with prediction. We also used heuristics on to decided what factors are common in the real world that we know affects insurance like power and age,

Taking all these into consideration and after a lot of experimenting we came up with 59 features w can think of:

Imgs

These features were ranging from raw values, to created rations, to log values , to bins ect..

In order to fine the usefulness of the we did some things like p values ect: This part is documented already