A comprehensive web scraper for extracting and categorizing products from multiple Indian e-commerce and grocery delivery platforms.

• Zepto - 10-minute grocery delivery
• Swiggy Instamart - Quick commerce platform
• Blinkit - Instant delivery service
• BigBasket - Online grocery store
• Flipkart Minutes - Hyperlocal delivery
• DealShare - Social commerce platform

• ✅ Multi-site scraping support
• ✅ Automatic product categorization
• ✅ Price extraction and discount calculation
• ✅ Product metadata (rating, reviews, images)
• ✅ Data export in multiple formats (JSON, CSV, Excel)
• ✅ Category-based organization in Excel
• ✅ Deduplication of products
• ✅ Statistics and analytics

1. Clone or download this repository

2. Install Python 3.8 or higher

3. Install required packages:

pip install -r requirements.txt

4. Ensure Chrome browser is installed (for Selenium WebDriver)

Run the main scraper:

python main.py

The scraper will:

1. Visit each site
2. Extract product information
3. Categorize products automatically
4. Save data to output/ directory in multiple formats

After scraping, you'll find:

• products.json - Complete data in JSON format with metadata
• products.csv - CSV file for easy import into databases/spreadsheets
• products.xlsx - Excel file with:
  • All Products sheet
  • Separate sheets for each category
  • Summary statistics sheet

Each product contains:

• Name
• Price
• Original Price (if discounted)
• Discount amount/percentage
• Category (automatically assigned)
• Description
• Brand
• Rating & Review Count
• Image URL
• Product URL
• Source Site
• Stock Status

Products are automatically categorized into:

• Fruits & Vegetables
• Dairy & Eggs
• Bakery & Bread
• Grocery Staples (Rice, Wheat, Pulses, Oil, Spices, etc.)
• Beverages
• Snacks & Munchies
• Frozen Food
• Meat & Seafood
• Breakfast & Cereals
• Sauces & Condiments
• Personal Care
• Home & Cleaning
• Baby Care
• Health & Wellness
• Pet Care
• Electronics & Accessories
• Beauty & Makeup

Edit config.py to customize:

• Site URLs
• Request delays
• Selenium options
• Output file paths
• User agents

1. Create a new scraper class in scraper.py inheriting from ProductScraper
2. Implement the scrape() method
3. Add site URL to config.py
4. Update main.py to include the new site

Edit categories.py to:

• Add new categories
• Add keywords for categorization
• Modify categorization logic

If you encounter ChromeDriver errors:

1. Ensure Chrome browser is installed
2. The script will auto-download ChromeDriver using webdriver-manager
3. Alternatively, manually install ChromeDriver and update the path

Some sites may:

• Load slowly - increase timeout in config.py
• Require login - you may need to implement authentication
• Block requests - try adjusting user agents or adding delays

If no products are extracted:

• Sites may have changed their HTML structure
• JavaScript rendering may be required (already handled)
• Rate limiting may be active (increase delays)

⚠️ Important:

• Always respect robots.txt files
• Don't overload servers with requests
• Check terms of service before scraping
• Use scraped data responsibly
• Consider using official APIs when available

• Python 3.8+
• Chrome browser
• Internet connection

This project is for educational purposes. Please use responsibly and in accordance with the terms of service of the websites being scraped.

• Scraping may take time depending on site load and number of products
• Some sites may implement anti-scraping measures
• Product availability and prices are subject to change
• The scraper extracts publicly available information only