$ show gosom/google-maps-scraper
This repository implements a production-grade Google Maps business data extraction platform with multiple execution modes: CLI, web UI, REST API, AWS Lambda, and a self-hosted SaaS edition. The core scraping engine uses Playwright for browser automation and also supports a stealth HTTP fast mode. It extracts 36+ structured fields per business (name, address, phone, website, reviews, ratings, emails, coordinates, hours, etc.) and provides deduplication, grid-based area coverage, email extraction via website visits, and review parsing. The architecture is modular with packages for scraping jobs, runners (file, database, web, Lambda, SaaS worker), output writers (CSV, JSON, PostgreSQL, S3, custom plugins), job queuing via River, and a comprehensive admin dashboard with cloud provisioning on DigitalOcean and Hetzner. The system is designed for both single-user CLI use and multi-tenant SaaS deployments with authentication, 2FA, API key management, and scalable worker pools. The codebase demonstrates strong Go engineering practices with well-defined interfaces, dependency injection, and test coverage. Notable reusable components include cryptography utilities, HTTP server middleware, rate limiting, and a CLI prompt library. However, the repository has some maintenance concerns: two separate database migration sets for legacy and SaaS modes, in-memory deduplication that doesn't persist across restarts, committed test SSH keys in the `local/` directory, and a large dependency surface that could introduce version conflicts. The scraper logic is inherently brittle due to reliance on Google Maps internal data structures, which are subject to change without notice. For AI agent consumption: This is a powerful off-the-shelf scraping tool that can be deployed as a Docker container, AWS Lambda, or managed SaaS. It provides a complete API and web interface for managing scraping jobs, monitoring progress, and exporting data. The SaaS edition includes worker auto-provisioning, making it suitable for large-scale lead generation operations. The code is actively maintained with recent commits addressing performance and feature enhancements.
The LLM identified 5 files worth highlighting for reuse:
A self-contained Go package providing AES-GCM encryption/decryption, bcrypt password hashing, SHA-256 hashing, SSH key pair generation, TOTP backup code generation, and QR code encoding for TOTP secret keys. No external dependencies beyond standard library and 'rsc.io/qr'. Can be extracted and used in any Go project requiring these security primitives.
An HTTP server wrapper that creates a properly configured http.Server with configurable read/write timeouts, graceful shutdown via signal handling, and a middleware stack including request ID injection (X-Request-Id), structured logging, and panic recovery. Provides New() and ListenAndServe() functions. Reusable as a foundation for any Go web service.
A generic rate limiter interface with a PostgreSQL backend implementation. The interface defines Check(key string) error that returns a structured error with retry-after duration if rate limited. The PostgreSQL implementation uses a sliding window approach with atomic counters. Cleanly abstracted and testable; can be reused for any Go service needing rate limiting with database-backed state.
A simple in-memory deduplicator using an FNV-1a 64-bit hash map. Provides Visit(url string) bool to check if a URL has been seen. Thread-safe with sync.Mutex. Used by the scraper to avoid processing duplicate place URLs across multiple jobs. Lightweight and easily extractable for any deduplication need.
A Go package providing interactive CLI prompts for user input: Confirm (yes/no), Select (choose from options), and Input (free text with default). Uses ANSI escape codes for styling and os.Stdin for input. Designed for the SaaS provisioning wizard but can be extracted for any Go CLI tool requiring interactive prompts.