Download the PHP package iprodev/sitemap-generator-pro without Composer

PHP XML Sitemap Generator (Library + CLI)

A professional, production-ready PHP sitemap generator by iProDev (Hemn Chawroka) — supports concurrency, robots.txt, gzip compression, sitemap index files, and comprehensive error handling.

🚀 What's New in v3.0

• ✨ Database Storage with change detection and historical tracking
• 🔄 Resume Capability with checkpoint system
• 🎯 SEO Analysis and content quality checking
• 📊 Performance Metrics and detailed analytics
• 🖼️ Multi-format Sitemaps (Images, Videos, News)
• 🌐 JavaScript Rendering support for SPAs
• 🔐 Proxy Support with rotation
• 🔔 Webhook Notifications for events
• 📅 Scheduled Crawling with cron integration
• 🎨 Interactive Mode for easy configuration
• ⚡ Caching System (File & Redis)
• 🎛️ Smart Filtering with priority rules
• 📈 Rate Limiting with retry handling

📋 Table of Contents

• Features
• Installation
• Quick Start
• CLI Usage
• Advanced Features
• Programmatic Usage
• Examples
• Configuration
• API Reference
• Testing
• Docker
• Troubleshooting
• Contributing

✨ Features

Core Features

• 🚀 High Performance - Concurrent HTTP requests
• 🤖 Robots.txt Compliant - Respects crawling rules
• 📦 Gzip Compression - Automatic compression
• 📊 Sitemap Index - Multiple sitemap files
• 🛡️ Error Handling - Comprehensive error management
• 📝 PSR-3 Logging - Standard logging interface

Advanced Features

• 💾 Database Storage - SQLite/MySQL/PostgreSQL support
• 🔄 Change Detection - Track URL changes over time
• 📈 SEO Analysis - Analyze pages for SEO issues
• 🔍 Quality Checks - Find duplicates, broken links
• 🎯 Smart Filtering - Include/exclude patterns
• ⚡ Caching - File and Redis cache support
• 📍 Resume Support - Continue interrupted crawls
• 🔔 Webhooks - Real-time notifications
• 📅 Scheduling - Automated periodic crawls
• 🌐 JavaScript - Render SPAs with headless Chrome
• 🔐 Proxy Support - HTTP/SOCKS proxies with rotation
• 🎨 Interactive Mode - User-friendly configuration

Sitemap Types

• 📄 Standard XML Sitemap
• 🖼️ Image Sitemap
• 🎬 Video Sitemap
• 📰 News Sitemap

📥 Installation

Requirements

• PHP >= 8.0
• Extensions: curl, xml, mbstring, zlib, pdo
• Optional: redis, posix

🚀 Quick Start

Basic Usage

Interactive Mode

With All Features

🖥️ CLI Usage

Basic Options

Caching

Database & Change Detection

Resume Support

Rate Limiting

Filtering

SEO & Analysis

Advanced Sitemaps

JavaScript Rendering

Proxy Support

Webhooks

🎯 Advanced Features

1. Database Storage & Change Detection

Track changes over time:

The system will:

• Store all URLs in database
• Compare with previous crawl
• Generate change report (new, modified, deleted)
• Track SEO metrics over time

2. Resume Interrupted Crawls

Large crawls can be resumed:

3. SEO Analysis

Analyze pages for SEO issues:

Reports include:

• Missing title/meta descriptions
• Duplicate content
• Broken links
• Page load times
• Mobile optimization
• Structured data

4. JavaScript Rendering

For SPAs (React, Vue, Angular):

5. Scheduled Crawling

Setup automated crawls:

6. Webhooks

Get notified of events:

Webhook payload:

7. Proxy Support

Use proxies for crawling:

Proxy file format:

8. Smart Filtering

Control what gets crawled:

💻 Programmatic Usage

Basic Example

With Database & Change Detection

With SEO Analysis

With Caching

⚙️ Configuration

Configuration File

Create sitemap.config.php:

Use config file:

📂 Examples

The examples/ directory contains ready-to-use code examples:

Basic Example (examples/basic.php)

Simple sitemap generation with minimal configuration:

Advanced Example (examples/advanced.php)

Includes caching, filtering, and rate limiting.

Comprehensive Example (examples/comprehensive.php)

Full-featured example demonstrating all capabilities:

• Database storage with change detection
• SEO analysis and quality checks
• Performance metrics
• Webhook notifications
• Image sitemap generation
• Resume capability

Run the comprehensive example:

📖 API Reference

Core Classes

Advanced Classes

Sitemap Writers

🧪 Testing

🐳 Docker

📊 Performance Tips

1. Increase Concurrency: For faster crawling

2. Enable Caching: Reduce duplicate requests

3. Use Database: Track changes efficiently

4. Smart Filtering: Reduce unnecessary pages

5. Resume Support: Handle large sites

🔧 Troubleshooting

Common Issues

1. Memory Exhausted Error

2. Too Many Open Connections

3. Crawl Takes Too Long

4. Rate Limited by Server

5. JavaScript-rendered Content Not Captured

6. SSL Certificate Errors

Debug Mode

Enable verbose output for debugging:

Logs

When using database storage, check crawl history:

🔒 Security

• Path traversal prevention
• URL validation and sanitization
• Safe XML generation
• Proxy authentication support
• Rate limiting to prevent blocking

📝 License

MIT License - see LICENSE.md

🤝 Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Make your changes
4. Run tests (composer check)
5. Commit your changes (git commit -m 'Add amazing feature')
6. Push to the branch (git push origin feature/amazing-feature)
7. Open a Pull Request

Development Setup

🙏 Credits

Created by iProDev (Hemn Chawroka) - https://github.com/iprodev

📞 Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Documentation: Wiki

Made with ❤️ by iProDev (Hemn Chawroka)