DIMMiN

Created at: April 29, 2026, noon Updated at: April 30, 2026, 4:27 a.m.

2026-04-29-Wednesday

Extracted all numeric values from the competition pages, including price_per_unit, total_lot_value
Extracted total lot size (in lbs) using total_lot_value and price_per_unit
Completed update of 1_incorporate_competition_data.py

Created at: April 25, 2026, noon Updated at: April 25, 2026, 11:10 a.m.

Re-worked 1_incorporate_competition_data.py to extract relevant data row-by-row for each table instead of trying to designate a price or auction table. This resulted in much higher fidelity data, however there are still edge cases resulting in about ~10% of the data not including a field for the price_per_unit feature.
Began converting competition columns to numeric features (prices, sizes in kg, etc.)
Visualized the distribution of the target variable (price_per_unit), looks like it follows an almost perfect Log Normal Distribution

Created at: April 24, 2026, noon Updated at: April 24, 2026, 11 a.m.

Began updating the second script in the pipeline, 1_incorporate_competition_data.py by switching away from a pandas processing job to using raw Python Dictionarys

Created at: April 24, 2026, noon Updated at: April 24, 2026, 2:50 a.m.

Started building out a validator using Python pytest to validate each of the steps involved in processing the cup of excellence data
Caught a bug where some files were incorrectly named, re-collected these files and fixed them
Explicitly handled cases where competition pages did not have links to individual lots

Created at: April 12, 2026, noon Updated at: April 22, 2026, 11:32 p.m.

Built and deployed an interactive A/B testing demo here to help secure a potential client contract.

Created at: April 11, 2026, noon Updated at: April 11, 2026, 9:46 a.m.

Resolved issue 4 and fixed the error page on the website, allowing errors to surface without giving an ugly raw html response
Added a basic maintenance page I can use while developing / updating parts of the DIMMiN App

Created at: April 10, 2026, noon Updated at: April 10, 2026, 3:20 p.m.

Created a dimmin-public bucket for public assets that don't need to be pre-signed
Created a new IAM User account for the django app specifically to help scope its permissions
Solved optimization issues for loading js libraries and loading image assets

Created at: April 10, 2026, noon Updated at: April 10, 2026, 4:52 a.m.

Created at: April 10, 2026, noon Updated at: Jan. 28, 2026, 10:23 a.m.

Created the required Django Views and Django Routes to test the endpoints
Made sure that a user needs to be logged in to be able to make POST requests to the DIMMiN App.

Created at: April 10, 2026, noon Updated at: Jan. 28, 2026, 8:50 a.m.

Started by creating the Django Serializer necessary to Serialize the data for ingestion to the DIMMiN App