# Changelog ## Version 1.8 ### Added - **`view_snapshots.sh` Enhancements**: - Introduced support for a new `all` tag to display all snapshots regardless of specific tags. - Added functionality to dynamically fetch and display all available tags in the repository, improving usability and flexibility. - Enhanced error handling for empty repositories or inaccessible snapshots with detailed log messages. - **Improved Snapshot Display**: - For the `all` tag, all snapshots are displayed in a formatted output including `short_id`, `time`, and associated tags. - Enhanced the script to log raw snapshot data during errors for easier debugging. ### Fixed - Resolved issues with fetching available tags when no snapshots existed in the repository, ensuring consistent script behavior. - Fixed a missing password issue when executing `restic -r /mnt/backups snapshots --json`, ensuring `RESTIC_PASSWORD` is securely passed in all cases. - Corrected formatting and display of available tags to ensure no duplicate or unnecessary entries are shown. ### Updated - **Logging and Usability**: - Improved log formatting to include more descriptive timestamps and detailed error or success messages for each step. - Standardized output for snapshot retrieval by tag, including snapshot ID, timestamp, and tags in a clean and readable format. - **Environment Validation**: - Updated `view_snapshots.sh` to ensure all necessary environment variables (`RESTIC_REPOSITORY`, `RESTIC_PASSWORD`) are set before executing Restic commands. - Validated repository access and password file existence at the start of the script to avoid runtime errors. ### Improved - **Modularity and Robustness**: - Refactored `view_snapshots.sh` to handle dynamic tags more effectively, making it easier to extend or customize in the future. - Centralized dependency validation (`restic`, `jq`) and repository checks to reduce redundancy across script operations. - **Error Handling**: - Added detailed error messages when repository access fails or tags cannot be fetched, ensuring issues are easy to debug. - Ensured graceful script exits with appropriate logs when critical validation checks fail. - **Code Maintenance**: - Reduced redundancy by consolidating snapshot and tag-related logic into reusable functions. - Enhanced maintainability by improving script readability, modularity, and inline documentation. ## Version 1.7 ### Added - **`restore_backup_direct.sh` Enhancements**: - Improved restoration script to handle specific backup types (core files, media files, database backups) based on tags. - Integrated functionality to restore databases directly by piping `.sql` files from Restic to MySQL. - Added automatic detection of database credentials from `wp-config.php` with fallback to default values for `DB_HOST`. - Ensured proper environment setup with `RESTIC_PASSWORD` retrieved securely from `/etc/restic-password`. - **Automatic Backup Tagging**: - Modified `backup_all.sh` to differentiate between manual (`manual-backup-YYYY-MM-DD_HH-MM-SS`) and automated (`auto-backup-YYYY-MM-DD_HH-MM-SS`) backups. - Ensured backups triggered by cron jobs are tagged appropriately for easy identification. - **`manage_backup_schedule.sh` Enhancements**: - Ensured `backup_all.sh` runs in `auto` mode when added to cron, guaranteeing automated backups have the correct tags. - Validated cron job syntax and dependencies to avoid misconfigurations. - Added explicit logging of cron actions, including schedule additions, updates, removals, and listing. - **Improved Error Messaging**: - Enhanced restoration script to provide detailed error messages when database restoration fails, including incorrect credentials or insufficient privileges. ### Fixed - Resolved issues with incorrect handling of database restoration paths, ensuring `.sql` files are restored directly into MySQL when detected. - Addressed potential confusion in tagging by clearly segregating manual and automated backups in logs and tags. - Corrected empty `DB_HOST` values in the restoration script by setting a default to `localhost` when no value is provided in `wp-config.php`. ### Updated - **Logging and Traceability**: - Enhanced all relevant scripts (`restore_backup_direct.sh`, `backup_all.sh`, `manage_backup_schedule.sh`) to log detailed timestamps and descriptive messages. - Improved consistency in log formatting across scripts for better traceability. - Added detailed logs for all cron operations in `manage_backup_schedule.sh` to track automated backups effectively. ### Improved - **Backup and Restore Modularity**: - Streamlined the orchestration of backups and restores to ensure modularity and easier debugging. - Improved robustness of `restore_backup_direct.sh` by handling different backup types dynamically using snapshot tags. - Simplified management of cron schedules with better error handling and validation in `manage_backup_schedule.sh`. - **Error Handling**: - Ensured all scripts exit gracefully on errors with detailed logs to pinpoint issues. - Enhanced dependency validation to check for all required tools (`restic`, `mysql`, `jq`, `crontab`, etc.) before execution. - **Code Maintenance**: - Centralized key operations such as password retrieval, lock handling, and backup tagging across scripts for easier maintenance and fewer redundancies. ## Version 1.6 ### Added - **`check_repo_stats.sh`**: - Introduced a new script to automate the retrieval of repository statistics and perform maintenance tasks. - Logs repository stats, retention policy application, and integrity checks to dynamically named log files (`repo_stats_YYYY-MM-DD.log`). - Automatically applies a retention policy (`--keep-last 7 --prune`) to manage repository size. - Includes an integrity check (`restic check --read-data-subset=5%`) to validate repository consistency and data reliability. - Detects and removes stale locks before performing operations. - Ensures dependencies (`restic`) and repository access are validated before execution. - Handles errors gracefully with detailed logging for each step. - **Enhanced `backup_repo_check` Functionality**: - Created a standalone script `check_backup_repo.sh` to validate repository integrity and initialize repositories if empty. - Logs actions and errors to `/home/litespeed/logs/backup_repo_check.log` for better traceability. - Improved stale lock handling by detecting and removing locks when necessary. - Validates or creates a password file (`/etc/restic-password`) to ensure smooth operation without prompts. - Automatically initializes a new repository if no files are present in the backup path. ### Fixed - Addressed permission issues in `check_backup_repo.sh` by ensuring the log directory and repository paths are writable by the correct user. - Corrected potential errors in handling stale locks by adding proper validation before removing them. ### Improved - Enhanced logging across `check_repo_stats.sh` and `check_backup_repo.sh` for better monitoring and traceability. - Optimized script robustness with dependency validation, error handling, and reduced redundancy. - Streamlined repository maintenance by combining retention policy application, stale lock removal, and integrity checks into `check_repo_stats.sh`. ## Version 1.5 ### Added - Simplified `backup_all.sh` by delegating backup tasks to individual scripts (`backup_core_files.sh`, `backup_database.sh`, `backup_media.sh`). - Introduced centralized orchestration of all backups in `backup_all.sh` with a unified `manual-backup` tag for consistency. - Ensured individual scripts can be reused independently or called as part of a larger workflow. ### Fixed - Corrected `trap` command in `backup_all.sh` to ensure errors are properly logged to `backup_error.log`. - Addressed issues with redundant logic in `backup_all.sh` by consolidating common functionality into the individual backup scripts. - Resolved potential inconsistencies in tag usage by applying a standard `manual-backup` tag with the current server date across all backup scripts. ### Updated - Refactored `backup_all.sh` to dynamically call individual backup scripts instead of duplicating logic for core, database, and media backups. - Adjusted YAML configuration for `backupnow` to call `backup_all.sh` as the primary entry point for the backup process. ### Improved - Centralized the backup process to enhance modularity and maintainability by separating core, database, and media backup logic. - Ensured consistent tagging and logging across all backup types for easier monitoring and debugging. - Streamlined the orchestration script (`backup_all.sh`) for improved readability and reduced duplication. ## Version 1.4 ### Added - Introduced efficient lock management in all backup scripts to prevent conflicts during simultaneous Restic operations. - Automated removal of stale locks before each Restic operation. - Added global lock mechanism using `flock` to serialize backup operations across multiple processes. - Implemented dynamic inclusion of Restic password handling by reading it directly from the `/etc/restic-password` file. - Added automatic validation of Restic repository access before performing backups. ### Fixed - Fixed issues with manual Restic prompts for the repository password by ensuring passwords are passed via environment variables. - Resolved potential conflicts caused by simultaneous Restic processes with the introduction of serialized operations. - Corrected permissions for Restic lock directories to avoid permission-denied errors during backup and restore processes. - Enhanced password validation logic to ensure backups fail gracefully if the provided password is incorrect. ### Updated - Updated core, media, and database backup scripts to handle dynamic exclusion paths using a loop-based approach for `--exclude` options. - Revised logging mechanisms across all scripts to include detailed timestamps and step-specific logs for better traceability. - Improved script robustness by validating the Restic repository's accessibility upfront using `restic snapshots`. ### Improved - Optimized all backup scripts to use efficient lock and unlock handling, ensuring smooth operation during concurrent backups and restores. - Standardized backup script flow across media, database, and core backups, including consistent use of environment variables and error handling. - Improved log formatting with more descriptive log messages and clear delineation of errors, warnings, and successes. - Reduced redundancy in password handling by centralizing Restic password retrieval logic.