v2.1.2 - 2026-03-25

HeadlessX v2.1.2

Released on March 25, 2026

HeadlessX v2.1.2 is the current CLI-first release for self-hosted operators, dashboard workflows, and agent-ready automation.

What's new

CLI-first setup and updates

• published lifecycle flow around headlessx init, start, stop, restart, status, doctor, logs, and init update
• modern interactive prompts with @clack/prompts
• improved health checks, loopback fallback, Docker log handling, and VPS operator UX
• workspace updates now reconcile missing env keys instead of only pulling the repo

Better self-hosting defaults

• rare default public host ports to avoid local conflicts:
  • web 34872
  • api 38473
  • postgres 35432
  • redis 36379
  • html-to-md 38081
  • yt-engine 38090
• new infra/domain-setup scaffold with Caddy for domain-based production deployment
• headlessx restart rebuilds Docker for self-host and production installs

Docker and browser runtime hardening

• fixed Docker startup issues around native dependencies and browser runtime boot
• switched API browser runtime to headfox-js
• stabilized browser release fetching for Linux Docker builds
• restored yt-engine in the full Docker stack
• web now waits for a healthy API before startup

Google AI Search improvements

• added shared-profile Google cookie bootstrap
• new Build Cookies and Stop Browser flow in the dashboard
• first Google run now requires one manual bootstrap session on the shared persistent profile
• improved structured errors and runtime setup messaging

Dashboard and operator UX

• improved operators catalog startup recovery
• clarified operator availability errors, especially for YouTube
• improved website crawl results with page overlay inspection
• hardened crawl SSE behavior and reconnect flow

Important changes

• older installs should update with:
  • headlessx init update
  • headlessx restart
• Google AI Search now requires a one-time cookie bootstrap before first real use
• full Docker includes yt-engine by default
• the tracked seeded browser profile has been removed; runtime profile data is now created and persisted on demand

Recommended update commands

Existing install on main

npm install -g @headlessx-cli/core@latest
headlessx init update
headlessx restart
headlessx status
headlessx doctor

v2.1.1 - 2026-03-20

HeadlessX v2.1.1

Released: 2026-03-20

Highlights

HeadlessX v2.1.1 focuses on operator-first API cleanup, Google AI Search improvements, a published CLI flow, Docker completeness, and runtime stability across the dashboard and backend.

What Changed

Operator-first routing

• reorganized public API routes under /api/operators/*
• added operator status at /api/operators/status
• reorganized playground routes under /playground/operators/*
• stabilized Website Scrape as a single frontend route with format switching inside the UI

Persistent browser runtime

• added one shared persistent Camoufox profile in the API
• kept cookies, storage, and browser state on disk
• made runtime prefer apps/api/data first, then apps/api/default-data
• removed the old forced 1920x1080 browser size and improved viewport behavior

Google AI Search

• renamed the old Google flow to Google AI Search
• added region, language, and time-filter targeting
• kept plain google.com as the default start flow unless targeting is explicitly set
• improved SSE stability and AI overview rendering
• made CAPTCHA progress messaging appear only when a real CAPTCHA is detected
• aligned the Google workbench UI with the Website UI

CLI

• published the CLI as @headlessx-cli/core
• standardized the command as headlessx
• made CLI output markdown-first by default
• updated login, auth, operator, and Google AI Search command flows

Crawl and queue reliability

• fixed crawl cancel behavior
• fixed the enqueue/cancel race window
• improved worker-side cancellation handling
• fixed crawl page budgeting
• made Website Crawl use the advanced timeout value correctly

Docker and self-hosting

• added yt-engine to the full Docker stack
• made full Docker closer to one-command setup
• aligned setup and API docs with the current runtime model
• documented that Website Crawl specifically requires Redis and the worker

Settings and UX cleanup

• simplified proxy settings UI
• replaced browser-native alerts with in-app feedback
• fixed operator Stop button behavior across workbenches
• cleaned settings layout and section structure

Docs and skills

• refreshed setup, API, CLI, and docs-site content for the current release
• added CLI and Agent Skills documentation
• added the HeadlessX CLI skill install flow for supported AI agents

Included Surfaces

• Website
• Google AI Search
• Tavily
• Exa
• YouTube
• Jobs
• Remote MCP
• Published CLI

Notes

• full Docker now includes yt-engine
• headlessx uses the same backend API and browser state as the dashboard
• Website Crawl remains the only website operator flow that requires Redis and the queue worker

v2.1.0 - 2026-03-16

HeadlessX v2.1.0

Released on March 16, 2026

Summary

HeadlessX v2.1.0 expands the platform with new scraper workspaces, queue-backed website workflows, remote MCP support, updated setup and API docs, refreshed branding assets, and Docker fixes for browser runtime dependencies.

Highlights

Platform and Dashboard

• removed profile-based runtime and session flows
• simplified the dashboard around a global browser/runtime model
• improved settings, sidebar behavior, overview, logs, API keys, and playground UX
• added loading skeletons and aligned scraper workbench surfaces
• updated sidebar resource links for hosted docs and changelog
• refreshed version surfaces to v2.1.0

Website Workflows and Jobs

• added queue-backed website workflows for scrape, crawl, map, and extraction
• improved worker and Redis handling for async jobs
• tightened stream progress and queue status behavior
• returned clearer queue-unavailable responses when Redis is missing

New Scraper Workspaces

• added Tavily workspace
• added Exa workspace
• added YouTube engine integration and workspace
• removed unsupported YouTube web and tv fallback behavior

MCP

• added remote MCP endpoint at /mcp
• MCP now uses normal dashboard-created API keys
• kept DASHBOARD_INTERNAL_API_KEY out of MCP access
• improved job access scoping for queue and stream status

Docs and Setup

• added setup guide
• added API reference
• refreshed README for v2.1.0
• added CONTRIBUTING guide
• aligned hosted docs links
• expanded env documentation to use the full current .env.example
• removed unused public env settings from docs

Docker

• Prisma migrations now run automatically on API and worker startup
• added missing browser libraries for Docker runtime:
  • libgtk-3-0
  • libx11-xcb1

Live Scrapers

• Website
• Google SERP
• Tavily
• Exa
• YouTube

Coming Soon

• Google Maps
• Twitter / X
• LinkedIn
• Instagram
• Amazon
• Facebook
• Reddit

Important Changes

• profile-based flows were removed
• dashboard runtime is now global rather than profile-scoped
• MCP uses user-created API keys, not the dashboard internal key
• YouTube web and tv fallback behavior was removed

Docs

• Setup Guide: https://headlessx.saify.me/docs/self-hosting/overview
• API Reference: https://headlessx.saify.me/docs/api-reference/overview
• MCP Setup: https://headlessx.saify.me/docs/get-started/mcp-setup
• Changelog: https://headlessx.saify.me/docs/changelog/

Included Work

• feat: add tavily playground and repo cleanup
• feat: add exa search workspace and unify playground shells
• feat: add youtube engine and unify playground workflows
• feat: add playground loading skeletons and simplify youtube clients
• docs: add setup and API guides and align scraper run actions
• feat(api): add remote MCP server and secure job access
• docs: refresh scraper visuals in README
• docs: refresh v2.1.0 setup and contributor guidance
• assets: update horizontal logo
• docs: remove unused env settings
• fix(docker): add missing browser libraries

Deployment Notes

• PostgreSQL and Redis are required for full production use
• website crawl requires Redis plus the worker
• Docker users should follow the setup guide
• Docker browser runtime now includes the missing Linux libraries required by the API container

v2.0.4 - 2026-03-07

🖥️ UI & Dashboard UX Refactor

• Flatten Dashboard Chrome: Flattened the shared dashboard surface system by removing shadow-heavy styling, disabling blur and smooth transitions, and strengthening borders across cards, dialogs, lists, and controls.
• Normalize Layout Rhythm: Unified the dashboard shell around a single content width and flatter bordered surface system so page headers, cards, and scraper workspaces align on the same design rhythm.
• Unify Scraper Workspace UX: Aligned the Website Scraper workspace with the Google SERP workspace by matching the header shell, page framing, and configuration card structure. Converted output and config options into dropdown-style controls.
• Header Structure: Turned page headers into bordered section shells for a clearer screen structure.
• Logs Route Redesign: Added a full-width /logs request workspace with a denser table, compact filter toolbar, and an overlay log details dialog to prevent compressing the table view.
• Sidebar Adjustments: Kept Headless Mode enabled by default in settings, moved the collapse control into the header, and updated the primary logo text color to black and blue.

🛠️ Backend API Enhancements

• Automated Cloudflare Challenge Solver: Added automated Cloudflare turnstile bypass logic with simulated human mouse movements directly to the API, alongside retry logic and SSE progress rendering for the frontend.
• Markdown Improvements: Integrated Mozilla Readability & GitHub Flavored Markdown for improved markdown extraction, along with expanding/collapsing results panel UI reliability enhancements.
• Documentation: Added demo GIF and Cloudflare bypass proof images to the README.

🔗 Related GitHub Commits

• refactor(web): flatten dashboard chrome and unify scraper workspace UX (@saifyxpro)
• refactor(web): normalize dashboard layout and improve logs workspace UX (@saifyxpro)
• feat(api): add automated cloudflare challenge solver (@saifyxpro)
• feat(api): enhance cloudflare challenge solver with retry and SSE progress (@saifyxpro)
• fix(website-scraper): improve markdown extraction and results panel expand button reliability (@saifyxpro)
• chore(web): update version to v2.0.4 (@saifyxpro)
• style(web): update primary logo text color to black and blue (@saifyxpro)
• docs(readme): add demo gif and cloudflare bypass proof images (@saifyxpro)

v2.0.3 - 2026-03-07

🔒 Security

• Dashboard Auth Hardening: Removed the public dashboard-internal fallback key and localhost trust path from the backend auth layer.
• Protected Admin Routes: Added API key protection to dashboard, profile, proxy, and job management endpoints that were previously exposed without auth.
• Secret Storage Improvements:
  • API keys are now stored as hashes instead of plaintext.
  • Stored proxy and profile passwords are now encrypted at rest.
  • Dashboard and API responses now redact credential values instead of returning raw secrets.
• Server-Side Dashboard Proxy: The dashboard now forwards /api/* requests through a Next.js server route so browser clients no longer send reusable internal keys.

🛠️ Reliability & Behavior

• Cloudflare Handling: Website scraping now fails fast with explicit Cloudflare challenge metadata instead of returning challenge pages as successful output.
• REST/SSE Output Parity: Aligned html-js behavior between the REST API and streaming endpoint to reduce timing drift on dynamic pages like Shopee.
• DOM Stability Waiting: Added a DOM stability wait before HTML extraction and screenshot capture so selector matches do not return partially rendered documents.
• Streaming Job Cancellation: Job cancellation now interrupts active scraping work instead of only updating job status in the UI.
• PDF Capability Cleanup: Removed unsupported PDF scraping/export paths from the API contract and dashboard UI.

🖥️ Developer Experience

• Fresh Install Fixes: Fixed a frontend startup failure caused by an invalid .npmrc encoding and resolved web/API port conflicts during local development.
• Docker Fix: Corrected the web Dockerfile path mismatch so Compose builds now target the expected file.
• Security Environment Variables: Added and documented required DASHBOARD_INTERNAL_API_KEY and CREDENTIAL_ENCRYPTION_KEY values for secure local and Docker deployments.
• Browser Profile Stability: Fixed Firefox preference merging so timezone overrides no longer wipe other stealth-related preferences.

📚 Documentation & UI

• Documentation Navigation: Updated dashboard and docs links to point to the introduction page and removed obsolete sidebar references.
• Landing Page Messaging: Removed outdated PDF export claims so the marketing copy matches the current product surface.
• Changelog Coverage: This release now links the supporting GitHub issues and commit history directly for easier auditability.

🐛 GitHub Issues

• #34 Can't run Headless:
  • Fixed fresh-install startup failures in the frontend dev workflow.
  • Resolved .npmrc encoding problems and port collisions between web and API processes.
• #32 Frontend response is different from backend:
  • Fixed REST and streaming output drift for JS-rendered pages.
  • Added DOM stabilization so API responses now match dashboard-rendered results more closely.

🔗 Related GitHub Commits

• a9e2367 chore: remove obsolete files and update documentation links in Sidebar component
• e1b91d9 feat: update docs link to point to introduction
• 7eb4a45 feat(api,web): improve scrape reliability and remove unsupported PDF flow
• 27bd3e3 fix(web): resolve dev startup failures on fresh installs
• 0abfb20 fix(api): align REST and SSE HTML rendering for dynamic pages
• 3c70c7d fix(security): harden dashboard auth and secret handling

v2.0.2 - Monorepo Restructure & Docker Support

🔧 Chore & Infrastructure

• Monorepo Restructure: Updated project structure to modern standards (apps/web, apps/api)
• Docker Support: Added comprehensive Docker infrastructure (infra/docker) with docker-compose.yml for easy deployment
• Legacy Cleanup: Removed deprecated OS-specific scripts (install.sh, start.bat, etc.) in favor of mise/docker
• GitHub Templates: Updated issue and PR templates for the new structure
• Documentation: Added docs/docker_setup.md and updated README.md with new port (8000) and deployment instructions

HeadlessX v2.0.1⚡ PERFORMANCE FIX

[2.0.1] - 2026-01-31 ⚡ PERFORMANCE FIX

🚀 Performance Improvements

Major performance optimizations for profile-based scraping with stealth mode disabled.

Mode	Before	After
API with profile + stealth OFF	25-35s	1.7s
Frontend with profile + stealth OFF	25-35s	2.8s

✨ What's Fixed

• Profile Context Reuse - Profile browser contexts now stay alive between requests
• Page Reuse - Existing pages are reused instead of creating new ones (~15s saved)
• Stealth Toggle Pipeline - stealth: false now properly skips all humanize delays
• Streaming Endpoint - Fixed stealth parameter passing from frontend to backend

📦 Update Instructions

cd HeadlessX
git pull origin main
cd backend
pnpm install
pnpm exec prisma generate
pnpm exec prisma db push

🔧 How to Get Maximum Speed

1. Create & Launch Profile - Go to Profiles → Create → Launch
2. Disable Stealth Mode - In Advanced Config, toggle OFF "Stealth Mode"
3. API Usage - Pass "options": { "stealth": false } for fastest scraping

curl -X POST http://localhost:3001/api/website/html \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -d '{"url":"https://example.com","profileId":"your-profile","options":{"stealth":false}}'

Note: First request after launching a profile takes ~15-20s (Camoufox initialization). Subsequent requests are under 3 seconds.

HeadlessX v2.0.0 - 🦊 CAMOUFOX RELEASE

[2.0.0] - 2026-01-28 🦊 CAMOUFOX RELEASE

🎉 Major Release: Complete Browser Engine Rewrite

HeadlessX V2.0 represents a complete rewrite of the browser engine, achieving 0% detection rate across all major anti-bot systems by replacing Chromium-based stealth with Camoufox (Firefox with C++ level fingerprint spoofing).

✨ Added

Browser Engine

• Camoufox Browser Engine - Firefox with C++ level fingerprint spoofing
• 0% Detection Rate - Passes CreepJS, Sannysoft, BrowserScan, and all major bot detection
• WebRTC Protection - Built-in IP leak prevention
• Binary-Level Stealth - Canvas, WebGL, and AudioContext spoofing at C++ level
• Persistent Context Pooling - 3x faster browser launches

New Scrapers

• Google SERP Scraper - Extract search results with zero detection
  • Organic results with position tracking
  • Featured snippets extraction
  • People Also Ask questions
  • Related searches
• Website Scraper - Full-featured web scraping
  • Raw HTML extraction (fast)
  • JavaScript-rendered content
  • Markdown conversion for LLMs
  • Full-page screenshots (PNG/JPEG)
  • Real-time SSE streaming

API Endpoints

• /api/website/html - Fast HTML extraction without JavaScript
• /api/website/html-js - JavaScript rendering with wait options
• /api/website/content - Clean markdown conversion
• /api/website/screenshot - Screenshot capture with quality control
• /api/website/stream - Real-time SSE stream with progress
• /api/google-serp/search - Google SERP extraction
• /api/google-serp/stream - Real-time SERP streaming

Documentation

• Complete API Reference - All endpoints documented with curl examples
• N8N Integration Guide - Workflow automation with examples
• Zapier Integration Guide - No-code automation patterns
• Make.com Integration Guide - Visual automation scenarios
• Configuration Guide - Dashboard and environment settings
• Installation Guide - Platform-specific setup (Windows, Linux, macOS)

Dashboard Features

• Modern UI - Next.js 16 with React 19 and Turbopack
• Live Configuration - Change settings without restart via Dashboard
• Request Logs - Full history with screenshots and timing
• Playground - Real-time testing interface for both scrapers
• Profile Management - Persistent browser sessions and cookies
• Proxy Management - Automatic rotation and validation

🚀 Improved

• Performance - 3x faster browser initialization with context pooling
• Stealth - From 67% detection (V1.3) to 0% detection (V2.0)
• Reliability - Better error handling and automatic recovery
• Documentation - Comprehensive guides with visual examples
• Developer Experience - Simplified installation with one-command setup

🗑️ Removed

• playwright-extra - No longer needed with Camoufox native stealth
• puppeteer-extra-plugin-stealth - Replaced by C++ level patches
• PDF endpoint (/api/website/pdf) - Removed for optimization
• Generic component previews from docs - Focused on documentation clarity
• All JavaScript-based fingerprint protection - Replaced by binary-level

🔧 Changed

• Browser Engine - Migrated from Chromium to Firefox (Camoufox)
• Config Management - Moved most settings from .env to Dashboard UI
• Frontend - Upgraded to Next.js 16 with Turbopack and React 19
• TypeScript - Upgraded to 5.9+ with stricter type checking
• Node.js - Now requires Node.js 22+ for optimal performance

🐛 Fixed

• Hydration mismatch in mobile navigation components
• Invalid SVG attributes (fill-rule → fillRule) in icon components
• Component preview build errors after library removal
• Missing module imports after Badtz-UI refactoring
• File system access errors in component library stubs
• Emoji in heading IDs causing invalid CSS selectors

📦 Dependencies

Added:

• camoufox - Stealth browser engine with Firefox
• playwright 1.58+ - Browser automation framework

Updated:

• next → 16.0.7 (with Turbopack)
• react → 19.x
• typescript → 5.9+
• node → 22+
• pnpm → 9+

Removed:

• playwright-extra and all stealth plugins
• puppeteer-extra-plugin-stealth
• Legacy fingerprinting libraries

🚨 Breaking Changes

1. Browser Engine: Chromium → Firefox (Camoufox)
2. API Response Format: All responses wrapped in { success, data, error } structure
3. PDF Endpoint Removed: /api/website/pdf no longer available
4. Configuration: Most settings moved from .env to Dashboard UI
5. Node.js Version: Requires Node.js 22+ (was 18+)

🔄 Migration Guide from V1.3

1. Update Dependencies

cd HeadlessX
pnpm install

2. Download Camoufox

pnpm camoufox:fetch

3. Update Environment Variables

# Remove old stealth configs
# Add new required vars
NEXT_PUBLIC_API_URL="http://localhost:3001"

4. Update API Integrations

// Old format
const html = response.html

// New format (V2.0)
const html = response.data.html

5. Remove PDF Endpoint Usage

// ❌ No longer available
POST /api/website/pdf

// ✅ Use screenshot instead
POST /api/website/screenshot

No Database Migration Required:

• Database schema unchanged
• Existing profiles remain compatible
• API keys remain valid

🚀 HeadlessX v1.2.0

Release Date: September 15, 2025
Version: 1.2.0
Codename: "Modular Architecture Revolution"

---

🎯 Major Breakthrough: Complete Modular Refactor

🏗️ Revolutionary Architecture Change

HeadlessX v1.2.0 represents a complete transformation from a monolithic to a modular architecture:

Before (v1.1.0):

• Single massive server.js file (3079 lines)
• Mixed concerns and responsibilities
• Difficult to maintain and extend

After (v1.2.0):

• 20+ focused modules with clear responsibilities
• Separation of concerns across logical layers
• Enterprise-grade maintainability and scalability

📦 New Modular Structure

src/
├── config/         # Configuration management
│   ├── index.js    # Main config with environment loading
│   └── browser.js  # Browser-specific settings
├── utils/          # Utility functions
│   ├── errors.js   # Error handling & categorization
│   ├── logger.js   # Structured logging
│   └── helpers.js  # Common utilities
├── services/       # Business logic services
│   ├── browser.js  # Browser lifecycle management
│   ├── stealth.js  # Anti-detection techniques
│   ├── interaction.js  # Human-like behavior
│   └── rendering.js    # Core rendering logic
├── middleware/     # Express middleware
│   ├── auth.js     # Authentication
│   └── error.js    # Error handling
├── controllers/    # Request handlers
│   ├── system.js   # Health & status endpoints
│   ├── rendering.js # Main rendering endpoints
│   ├── batch.js    # Batch processing
│   └── get.js      # GET endpoints & docs
├── routes/         # Route definitions
│   ├── api.js      # API route mappings
│   └── static.js   # Static file serving
├── app.js          # Main application setup
├── server.js       # Entry point for PM2
└── rate-limiter.js # Rate limiting implementation

---

✨ Major Features & Improvements

🔧 Enhanced Error Handling

• Structured Error Responses: Categorized errors with proper HTTP status codes
• Correlation IDs: Request tracing for debugging and monitoring
• Graceful Fallbacks: Better error recovery mechanisms
• Detailed Logging: Context-aware error logging with stack traces

🛡️ Advanced Security & Rate Limiting

• Intelligent Rate Limiting: Memory-based with automatic cleanup
• Enhanced Authentication: Improved token validation and middleware
• Security Headers: Comprehensive security header management
• Request Validation: Better input validation and sanitization

🚀 Performance Optimizations

• Browser Management: Optimized browser lifecycle with resource monitoring
• Memory Efficiency: Better memory usage and automatic cleanup
• Concurrent Processing: Improved handling of multiple requests
• Resource Monitoring: Real-time monitoring of system resources

📊 Monitoring & Observability

• Structured Logging: JSON-formatted logs with correlation IDs
• Health Monitoring: Comprehensive health checks and status reporting
• Performance Metrics: Detailed performance and resource metrics
• Request Tracing: Full request lifecycle tracking

---

💥 Breaking Changes

🔧 Configuration Updates

• Environment Variable: TOKEN → AUTH_TOKEN
• PM2 Configuration: Moved from config/ecosystem.config.js to ecosystem.config.js
• Enhanced .env: Additional configuration options with validation

📁 File Structure Changes

• Modular Architecture: Complete reorganization of source code
• Import Paths: Updated internal module imports
• Script Updates: Setup and deployment scripts updated

🔄 Migration Guide

# 1. Update environment variables
sed -i 's/TOKEN=/AUTH_TOKEN=/g' .env

# 2. Move PM2 configuration
mv config/ecosystem.config.js ./ecosystem.config.js

# 3. Update dependencies
npm install

# 4. Restart services
npm run pm2:restart

---

🛠️ Developer Experience

📖 Enhanced Documentation

• Modular Architecture Guide: Comprehensive MODULAR_ARCHITECTURE.md
• Updated README: Complete rewrite for v1.2.0
• API Documentation: Updated endpoint documentation
• Setup Guides: Enhanced deployment and development guides

🧪 Better Testing & Development

• Module Independence: Each module can be tested separately
• Clear Dependencies: Explicit dependency injection
• Better Debugging: Enhanced error messages and logging
• IDE Support: Improved code organization for better IntelliSense

🔧 Development Workflow

• Hot Reload: Faster development with modular reloading
• Clear Separation: Easy to understand and modify specific features
• Extensibility: Simple to add new features without affecting others
• Type Safety: Better code organization for type checking

---

🐳 Infrastructure & Deployment

📦 Updated Docker Support

• Modular Dockerfile: Updated for new architecture
• Optimized Builds: Better layer caching and build times
• Environment Integration: Enhanced environment variable handling

⚙️ PM2 Enhancements

• Root Configuration: Simplified PM2 setup in root directory
• Enhanced Monitoring: Better process monitoring and management
• Scaling Support: Improved scaling configuration
• Log Management: Enhanced log rotation and management

🔄 Deployment Scripts

• Updated Setup: Enhanced setup.sh for modular architecture
• Server Updates: Improved update_server.sh with validation
• Integration Tests: Updated test scripts for new structure

---

📊 Performance Improvements

🚀 Resource Optimization

• Memory Usage: 30% reduction in memory footprint
• CPU Efficiency: Better CPU utilization with modular loading
• Response Times: Improved response times for API endpoints
• Concurrent Handling: Better handling of multiple simultaneous requests

🔋 Browser Management

• Instance Reuse: Optimized browser instance lifecycle
• Resource Cleanup: Automatic cleanup of unused resources
• Memory Monitoring: Real-time memory usage monitoring
• Graceful Shutdowns: Better handling of browser shutdown sequences

---

🔮 Future-Ready Architecture

🏗️ Scalability

• Microservice Ready: Architecture prepared for microservice transition
• Horizontal Scaling: Better support for scaling across multiple instances
• Load Balancing: Improved load balancing capabilities
• Service Isolation: Services can be deployed independently if needed

🔧 Maintainability

• Clear Boundaries: Well-defined module boundaries and responsibilities
• Easy Updates: Individual modules can be updated independently
• Testing: Comprehensive testing strategy for each module
• Documentation: Living documentation with each module

---

🎉 Getting Started with v1.2.0

🔄 Upgrading from v1.1.0

# 1. Backup your current setup
cp .env .env.backup

# 2. Pull latest changes
git pull origin main

# 3. Update environment variables
sed -i 's/TOKEN=/AUTH_TOKEN=/g' .env

# 4. Install dependencies
npm install

# 5. Restart services
npm run pm2:restart

🆕 Fresh Installation

# 1. Clone repository
git clone https://github.com/SaifyXPRO/HeadlessX.git
cd HeadlessX

# 2. Configure environment
cp .env.example .env
nano .env  # Set AUTH_TOKEN and domain

# 3. Run automated setup
chmod +x scripts/setup.sh
sudo ./scripts/setup.sh

# 4. Access your HeadlessX
# Website: https://your-subdomain.yourdomain.com
# API: https://your-subdomain.yourdomain.com/api/health

---

📚 Resources

• 📖 Modular Architecture Guide: MODULAR_ARCHITECTURE.md
• 🚀 Deployment Guide: README.md
• 🔧 API Documentation: Visit /api/docs on your instance
• 💬 Support: GitHub Issues
• 🌟 GitHub: SaifyXPRO/HeadlessX

---

Thank you for using HeadlessX v1.2.0! This modular architecture sets the foundation for even more exciting features to come.

Built with ❤️ by SaifyXPRO

# 🚀 HeadlessX v1.1.0

Release Date: September 12, 2025
Version: 1.1.0
Codename: "Unified Architecture"

---

🎯 What's New

🌐 Unified Architecture

• Single Domain Solution: Website and API now run on the same domain
• Integrated Server: One Node.js server handles both frontend and backend
• Seamless User Experience: No more separate domains for website and API

🧠 Advanced Human-like Behavior

• 40+ Anti-Detection Techniques: Enhanced stealth capabilities
• Realistic User Agent Rotation: Windows-focused browser agents
• Natural Mouse Movements: Human-like interaction patterns
• Smart Scrolling: Behavioral randomization for better detection avoidance
• Browser-Specific Headers: Chrome, Edge, and Firefox header simulation

🔒 Production-Ready Infrastructure

• Docker Support: Complete containerization with docker-compose
• PM2 Integration: Professional process management
• Nginx Configuration: Load balancing and rate limiting
• SSL Ready: Automatic HTTPS setup with certbot
• Rate Limiting: API protection with zone-based limits

📊 Enhanced API Capabilities

• Multiple Output Formats: HTML, text, screenshots, PDFs
• Batch Processing: Handle multiple URLs efficiently
• Advanced Rendering Options: Wait conditions, timeouts, user agents
• Cookie Support: Session management and authentication
• Comprehensive Error Handling: Detailed error responses

---

✨ Key Features

🌐 Unified Web Architecture

Before v1.1.0:
├── Website: yourdomain.com (separate)
└── API: api.yourdomain.com (separate)

After v1.1.0:
└── All-in-One: headlessx.yourdomain.com
    ├── / (website)
    └── /api/* (API endpoints)

🛡️ Enhanced Security

• Token-based Authentication: Secure API access
• Rate Limiting: Multiple zones (health, API, website)
• Security Headers: X-Frame-Options, CSP, XSS protection
• Input Validation: Comprehensive request validation
• Error Sanitization: Safe error responses

🚀 Deployment Options

• 🐳 Docker: docker-compose up -d
• 🔧 Node.js + PM2: sudo ./scripts/setup.sh
• 💻 Development: npm start

---

📋 Complete Feature List

🔧 API Endpoints

Endpoint	Method	Description
/api/health	GET	Health check (no auth)
/api/status	GET	Server status
/api/render	POST	Full page rendering (JSON)
/api/html	GET/POST	Raw HTML extraction
/api/content	GET/POST	Clean text extraction
/api/screenshot	GET	Screenshot generation
/api/pdf	GET	PDF generation
/api/batch	POST	Batch URL processing

🧠 Human Behavior Features

• User Agent Rotation: Chrome 126-128, Edge 127-128, Firefox 128-129
• Realistic Headers: Browser-specific header sets
• Mouse Movements: Natural cursor patterns
• Scroll Simulation: Human-like page navigation
• Timing Randomization: Variable delays and timeouts
• Viewport Simulation: Multiple screen resolutions

🔒 Security & Performance

• Rate Limiting: 10r/s API, 100r/s health, 30r/s website
• Request Validation: URL, timeout, and parameter validation
• Memory Management: Automatic browser cleanup
• Error Handling: Graceful degradation
• Logging: Comprehensive request/error logging

---

🛠️ Installation & Setup

Quick Start

# Clone repository
git clone https://github.com/SaifyXPRO/HeadlessX.git
cd HeadlessX

# Quick setup
chmod +x scripts/quick-setup.sh && ./scripts/quick-setup.sh
nano .env  # Configure DOMAIN, SUBDOMAIN, TOKEN

# Choose deployment method:
docker-compose up -d              # Docker (recommended)
sudo ./scripts/setup.sh          # Auto setup
npm install && npm start         # Development

Environment Configuration

# Required .env variables
TOKEN=your_secure_token_here
DOMAIN=yourdomain.com
SUBDOMAIN=headlessx

# Optional settings
PORT=3000
NODE_ENV=production
BROWSER_TIMEOUT=60000
MAX_CONCURRENT_BROWSERS=5

Domain Setup

# DNS Record (A record)
headlessx.yourdomain.com → YOUR_SERVER_IP

# Automatic nginx configuration
sudo ./scripts/setup.sh

# Manual nginx setup
sudo sed -i 's/SUBDOMAIN.DOMAIN.COM/headlessx.yourdomain.com/g' /etc/nginx/sites-available/headlessx

---

🔄 Migration from v1.0.x

Breaking Changes

• Unified Architecture: Website and API now share the same domain
• New Environment Variables: DOMAIN and SUBDOMAIN required
• Updated Nginx Config: New configuration with rate limiting
• PM2 Configuration: Enhanced process management setup

Migration Steps

1. Update Environment:

   cp .env.example .env
   # Add DOMAIN and SUBDOMAIN variables

2. Update Nginx:

   sudo cp nginx/headlessx.conf /etc/nginx/sites-available/headlessx
   sudo sed -i 's/SUBDOMAIN.DOMAIN.COM/your.domain.com/g' /etc/nginx/sites-available/headlessx

3. Restart Services:

   pm2 restart all
   sudo systemctl reload nginx

---

🧪 Testing & Validation

Health Checks

# Basic health check
curl http://headlessx.yourdomain.com/api/health

# Server status (requires token)
curl "http://headlessx.yourdomain.com/api/status?token=YOUR_TOKEN"

# Website access
curl http://headlessx.yourdomain.com

API Testing

# HTML extraction
curl -X POST "http://headlessx.yourdomain.com/api/html?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

# Screenshot generation
curl "http://headlessx.yourdomain.com/api/screenshot?url=https://example.com&token=YOUR_TOKEN" \
  --output screenshot.png

---

📊 Performance & Specifications

System Requirements

• OS: Ubuntu 20.04+ (recommended), Ubuntu 24.04+, Ubuntu 25.04+
• Node.js: 20.x LTS
• RAM: 2GB minimum, 4GB recommended
• Storage: 2GB free space
• Network: Public IP with domain pointing

Performance Metrics

• Response Time: <500ms for simple requests
• Throughput: 100+ requests/minute
• Memory Usage: ~200MB base, +50MB per browser instance
• Browser Startup: <2 seconds
• Screenshot Generation: <3 seconds

Scalability

• Concurrent Browsers: 5 (configurable)
• Rate Limits: Configurable per zone
• Load Balancing: Nginx reverse proxy
• Process Management: PM2 cluster mode ready

---

🐛 Bug Fixes

v1.1.0 Fixes

• Fixed nginx configuration conflicts with rate limiting
• Resolved Ubuntu 25.04 package dependency issues
• Fixed duplicate proxy settings in nginx config
• Corrected Playwright browser installation on modern Ubuntu
• Fixed PM2 auto-start configuration
• Resolved script permission issues
• Fixed environment variable loading

Known Issues

• Playwright may require manual dependency installation on some Ubuntu versions
• SSL certificate renewal requires manual certbot setup
• Large PDF generation may timeout (use higher timeout values)

---

🔧 Scripts & Tools

Available Scripts

scripts/setup.sh              # Complete automated setup
scripts/setup-playwright.sh   # Playwright-specific setup
scripts/quick-setup.sh         # Initial setup (permissions + .env)
scripts/test-routing.sh        # Integration testing
scripts/verify-domain.sh       # Domain verification
scripts/validate-setup.sh      # Installation validation
scripts/generate-lockfiles.sh  # Package lock generation

NPM Scripts

npm start                # Development server
npm run build           # Build website
npm run pm2:start       # Start with PM2
npm run pm2:stop        # Stop PM2 processes
npm run pm2:restart     # Restart PM2 processes
npm run pm2:logs        # View PM2 logs
npm run pm2:status      # Check PM2 status
npm run setup-playwright # Setup Playwright browsers
npm run validate        # Validate installation

---

📚 Documentation

Available Documentation

• README.md - Main documentation and setup guide
• docs/GET_ENDPOINTS.md - GET API endpoints reference
• docs/POST_ENDPOINTS.md - POST API endpoints reference
• docs/HUMAN_BEHAVIOR_UPDATE.md - Human behavior features
• DEPLOYMENT.md - Detailed deployment guide
• CONTRIBUTING.md - Contribution guidelines
• PROJECT_STRUCTURE.md - Project architecture

Configuration Files

• nginx/headlessx.conf - Nginx configuration template
• config/ecosystem.config.js - PM2 configuration
• docker/docker-compose.yml - Docker setup
• .env.example - Environment template

---

🤝 Contributing

We welcome contributions! Please see:

• CONTRIBUTING.md for contribution guidelines
• GitHub Issues for bug reports
• GitHub Discussions for feature requests

Development Setup

git clone https://github.com/SaifyXPRO/HeadlessX.git
cd HeadlessX
npm install
cd website && npm install && npm run build && cd ..
cp .env.example .env
npm start

---

📄 License

HeadlessX v1.1.0 is released under the MIT License.

---

🙏 Acknowledgments

• Playwright Team - For the excellent browser automation framework
• Next.js Team - For the powerful React framework
• Express.js Community - For the robust web framework
• PM2 Team - For process management excellence
• Community Contributors - For feedback and testing

---

📞 Support

• GitHub Issues: Report bugs
• **Documentat...