🚀 Go Scraper

Extract Product Data from E-Commerce Platforms with Ease

Lightning Fast

Built in Go for optimal performance. Extract product data in milliseconds.

🌐

Multi-Platform Support

Works with Shopify, WooCommerce, Jumia, Amazon, and generic e-commerce sites.

📊

Structured Data

Returns clean JSON with product details including prices, variants, and images.

🔍

Smart Extraction

Uses JSON-LD, Open Graph, and HTML selectors for accurate data extraction.

📝

Auto Logging

All requests are logged to terminal and file for monitoring and debugging.

🛠️

Easy Deployment

Deploy with systemd, Docker, or Nginx reverse proxy in minutes.

Supported Platforms

🛍️ Shopify
🏪 WooCommerce
🌍 Jumia
📦 Amazon
✨ Generic Sites

API Usage

GET /scrape - For simple URLs (no query parameters)
Best for simple URLs without complex query parameters
http://localhost:8080/scrape?url=https://www.jumia.com.eg/product-name.html
POST /scrape - For complex URLs (Recommended)
Handles URLs with query parameters (e.g., Amazon wishlist parameters)
curl -X POST http://localhost:8080/scrape \
  -H "Content-Type: application/json" \
  -d '{"url": "https://www.amazon.com/product/dp/B0BMN7DFRR?th=1&psc=1"}'

⚡ Quick Start

1. Requirements

Go 1.21 or higher

2. Clone Repository
git clone https://github.com/mostafaabdelrazek/go-scrap
cd go-scrap
3. Install Dependencies
go mod tidy
4. Run Development Server
go run .
5. Build for Production
go build -o scrapper
./scrapper
6. Test the API
curl "http://localhost:8080/scrape?url=https://example.com/product"

Server runs on port 8080

📊 Response Format

The API returns structured JSON with the following fields:

{
  "name": "Product Name",
  "description": "Product description...",
  "price": 100,
  "original_price": 150,
  "discount_percentage": 33,
  "currency": "USD",
  "sku": "SKU123",
  "images": ["url1", "url2"],
  "variants": [{"name": "Size", "value": "M"}],
  "platform": "shopify",
  "source_url": "https://..."
}