| title | price | reviews | star_rating | url | |
|---|---|---|---|---|---|
| 0 | 63" L Shaped Office Executive Standing Desk wi... | $231.99 | (239) | 4.0 | https://www.wayfair.com/furniture/pdp/hokku-de... |
| 1 | Dediu Farmhouse Computer Desk With Charging St... | $183.99 | (173) | 4.5 | https://www.wayfair.com/furniture/pdp/august-g... |
| 2 | Koree Standing & Height-Adjustable Desks | $131.99 | (64) | 5.0 | https://www.wayfair.com/furniture/pdp/inbox-ze... |
| 3 | Caidence Computer Desk | $699.99 | (11) | 3.5 | https://www.wayfair.com/furniture/pdp/george-o... |
| 4 | Channay Writing Desk | $309.99 | (2) | 4.0 | https://www.wayfair.com/furniture/pdp/wade-log... |
Wayfair Product Price Tracking Analysis
Step 1: Load and clean product data from CSV
Scrape product data from Wayfair
Pre-process the data
| title | price | reviews | star_rating | url | page | |
|---|---|---|---|---|---|---|
| 0 | 63" L Shaped Office Executive Standing Desk wi... | 231.99 | 239 | 4.0 | https://www.wayfair.com/furniture/pdp/hokku-de... | 1 |
| 1 | Dediu Farmhouse Computer Desk With Charging St... | 183.99 | 173 | 4.5 | https://www.wayfair.com/furniture/pdp/august-g... | 1 |
| 2 | Koree Standing & Height-Adjustable Desks | 131.99 | 64 | 5.0 | https://www.wayfair.com/furniture/pdp/inbox-ze... | 1 |
| 3 | Caidence Computer Desk | 699.99 | 11 | 3.5 | https://www.wayfair.com/furniture/pdp/george-o... | 1 |
| 4 | Channay Writing Desk | 309.99 | 2 | 4.0 | https://www.wayfair.com/furniture/pdp/wade-log... | 1 |
Step 2: Visualize the data
The price distribution of Wayfair products is heavily right-skewed, with most items priced below $500. A few high-end products range up to $7000, but they are rare. The compact boxplot and long tail confirm affordability as a key focus, while still offering premium options for a broader customer base.
The star rating distribution is strongly skewed toward higher ratings, with most products rated between 4 and 5 stars. Ratings below 3 are extremely rare, suggesting high overall customer satisfaction. The peak around 4.5 stars indicates that Wayfair products are generally well-received and positively reviewed by buyers.
The review count distribution is highly right-skewed, with most products receiving fewer than 300 reviews. A small number of items have very high review counts, reaching over 4000, indicating a few popular or best-selling products. This suggests that while most items get limited customer feedback, a few standout products drive significant engagement.
The average price per page plot shows significant fluctuation across Wayfair’s product pages. While early pages mostly feature lower-priced items (around $200–$400), later pages show sharp spikes, with some averages exceeding $1500–$1900. This suggests Wayfair lists budget-friendly products first, followed by a more scattered distribution of mid- to high-end items on later pages.
The average star rating per page remains consistently high, mostly between 4.2 and 4.6, across all product pages. Occasional dips suggest a few pages contain lower-rated items, but these are rare. Overall, customer satisfaction appears strong and uniformly distributed, indicating reliable product quality throughout the Wayfair catalog.
Step 3: Merge product data with price history
Load the data
Loaded 100 rows from 10 files.| timestamp | url | price | time_spent_sec | source_file | |
|---|---|---|---|---|---|
| 0 | 2025-04-29 15:46:10 | https://www.wayfair.com/furniture/pdp/hokku-de... | $539.99 | 27.46 | wayfair_price_tracking_20250429_1602.csv |
| 1 | 2025-04-29 15:46:37 | https://www.wayfair.com/furniture/pdp/august-g... | $259.99 | 28.91 | wayfair_price_tracking_20250429_1602.csv |
| 2 | 2025-04-29 15:47:06 | https://www.wayfair.com/furniture/pdp/inbox-ze... | NaN | 290.83 | wayfair_price_tracking_20250429_1602.csv |
| 3 | 2025-04-29 15:51:57 | https://www.wayfair.com/furniture/pdp/george-o... | NaN | 314.16 | wayfair_price_tracking_20250429_1602.csv |
| 4 | 2025-04-29 15:57:11 | https://www.wayfair.com/furniture/pdp/wade-log... | NaN | 174.94 | wayfair_price_tracking_20250429_1602.csv |
Merge the data and clean it so it is ready for analysis
| timestamp | url | price | time_spent_sec | source_file | title | |
|---|---|---|---|---|---|---|
| 0 | 2025-04-29 15:46:10 | https://www.wayfair.com/furniture/pdp/hokku-de... | 539.99 | 27.46 | wayfair_price_tracking_20250429_1602.csv | 63" L Shaped Office Executive Standing Desk wi... |
| 1 | 2025-04-29 15:46:37 | https://www.wayfair.com/furniture/pdp/august-g... | 259.99 | 28.91 | wayfair_price_tracking_20250429_1602.csv | Dediu Farmhouse Computer Desk With Charging St... |
| 2 | 2025-04-29 15:47:06 | https://www.wayfair.com/furniture/pdp/inbox-ze... | NaN | 290.83 | wayfair_price_tracking_20250429_1602.csv | Koree Standing & Height-Adjustable Desks |
| 3 | 2025-04-29 15:51:57 | https://www.wayfair.com/furniture/pdp/george-o... | NaN | 314.16 | wayfair_price_tracking_20250429_1602.csv | Caidence Computer Desk |
| 4 | 2025-04-29 15:57:11 | https://www.wayfair.com/furniture/pdp/wade-log... | NaN | 174.94 | wayfair_price_tracking_20250429_1602.csv | Channay Writing Desk |
Step 4: Visualize the data
Time spent on each product to scrape the data
This graph tracks the time taken to scrape data for individual products using a dropdown menu. Most scraping tasks were completed in 20–40 seconds, indicating efficient extraction. One notable spike near 300 seconds suggests a delay, likely due to a loading issue or website lag. Overall, the scraping process was consistently fast with rare exceptions.
Success rate of scraping the data
We were able to successfully scrape 98% of the products, with only 2% failing due to various issues. This high success rate indicates that the scraping process was generally effective and reliable, allowing us to gather a comprehensive dataset for analysis.
Thank you
Group Members: - Shang-Yi Lin - Prem Anandsharan Thakkar - Wenliang Ding