mb-backup-manager/changelogs.md

# Changelog

## Version 1.7

### Added
- **`restore_backup_direct.sh` Enhancements**:
  - Improved restoration script to handle specific backup types (core files, media files, database backups) based on tags.
  - Integrated functionality to restore databases directly by piping `.sql` files from Restic to MySQL.
  - Added automatic detection of database credentials from `wp-config.php` with fallback to default values for `DB_HOST`.
  - Ensured proper environment setup with `RESTIC_PASSWORD` retrieved securely from `/etc/restic-password`.

- **Automatic Backup Tagging**:
  - Modified `backup_all.sh` to differentiate between manual (`manual-backup-YYYY-MM-DD_HH-MM-SS`) and automated (`auto-backup-YYYY-MM-DD_HH-MM-SS`) backups.
  - Ensured backups triggered by cron jobs are tagged appropriately for easy identification.

- **`manage_backup_schedule.sh` Enhancements**:
  - Ensured `backup_all.sh` runs in `auto` mode when added to cron, guaranteeing automated backups have the correct tags.
  - Validated cron job syntax and dependencies to avoid misconfigurations.
  - Added explicit logging of cron actions, including schedule additions, updates, removals, and listing.

- **Improved Error Messaging**:
  - Enhanced restoration script to provide detailed error messages when database restoration fails, including incorrect credentials or insufficient privileges.

### Fixed
- Resolved issues with incorrect handling of database restoration paths, ensuring `.sql` files are restored directly into MySQL when detected.
- Addressed potential confusion in tagging by clearly segregating manual and automated backups in logs and tags.
- Corrected empty `DB_HOST` values in the restoration script by setting a default to `localhost` when no value is provided in `wp-config.php`.

### Updated
- **Logging and Traceability**:
  - Enhanced all relevant scripts (`restore_backup_direct.sh`, `backup_all.sh`, `manage_backup_schedule.sh`) to log detailed timestamps and descriptive messages.
  - Improved consistency in log formatting across scripts for better traceability.
  - Added detailed logs for all cron operations in `manage_backup_schedule.sh` to track automated backups effectively.

### Improved
- **Backup and Restore Modularity**:
  - Streamlined the orchestration of backups and restores to ensure modularity and easier debugging.
  - Improved robustness of `restore_backup_direct.sh` by handling different backup types dynamically using snapshot tags.
  - Simplified management of cron schedules with better error handling and validation in `manage_backup_schedule.sh`.

- **Error Handling**:
  - Ensured all scripts exit gracefully on errors with detailed logs to pinpoint issues.
  - Enhanced dependency validation to check for all required tools (`restic`, `mysql`, `jq`, `crontab`, etc.) before execution.

- **Code Maintenance**:
  - Centralized key operations such as password retrieval, lock handling, and backup tagging across scripts for easier maintenance and fewer redundancies.
  
## Version 1.6

### Added
- **`check_repo_stats.sh`**:
  - Introduced a new script to automate the retrieval of repository statistics and perform maintenance tasks.
  - Logs repository stats, retention policy application, and integrity checks to dynamically named log files (`repo_stats_YYYY-MM-DD.log`).
  - Automatically applies a retention policy (`--keep-last 7 --prune`) to manage repository size.
  - Includes an integrity check (`restic check --read-data-subset=5%`) to validate repository consistency and data reliability.
  - Detects and removes stale locks before performing operations.
  - Ensures dependencies (`restic`) and repository access are validated before execution.
  - Handles errors gracefully with detailed logging for each step.

- **Enhanced `backup_repo_check` Functionality**:
  - Created a standalone script `check_backup_repo.sh` to validate repository integrity and initialize repositories if empty.
  - Logs actions and errors to `/home/litespeed/logs/backup_repo_check.log` for better traceability.
  - Improved stale lock handling by detecting and removing locks when necessary.
  - Validates or creates a password file (`/etc/restic-password`) to ensure smooth operation without prompts.
  - Automatically initializes a new repository if no files are present in the backup path.

### Fixed
- Addressed permission issues in `check_backup_repo.sh` by ensuring the log directory and repository paths are writable by the correct user.
- Corrected potential errors in handling stale locks by adding proper validation before removing them.

### Improved
- Enhanced logging across `check_repo_stats.sh` and `check_backup_repo.sh` for better monitoring and traceability.
- Optimized script robustness with dependency validation, error handling, and reduced redundancy.
- Streamlined repository maintenance by combining retention policy application, stale lock removal, and integrity checks into `check_repo_stats.sh`.

## Version 1.5

### Added
- Simplified `backup_all.sh` by delegating backup tasks to individual scripts (`backup_core_files.sh`, `backup_database.sh`, `backup_media.sh`).
- Introduced centralized orchestration of all backups in `backup_all.sh` with a unified `manual-backup` tag for consistency.
- Ensured individual scripts can be reused independently or called as part of a larger workflow.

### Fixed
- Corrected `trap` command in `backup_all.sh` to ensure errors are properly logged to `backup_error.log`.
- Addressed issues with redundant logic in `backup_all.sh` by consolidating common functionality into the individual backup scripts.
- Resolved potential inconsistencies in tag usage by applying a standard `manual-backup` tag with the current server date across all backup scripts.

### Updated
- Refactored `backup_all.sh` to dynamically call individual backup scripts instead of duplicating logic for core, database, and media backups.
- Adjusted YAML configuration for `backupnow` to call `backup_all.sh` as the primary entry point for the backup process.

### Improved
- Centralized the backup process to enhance modularity and maintainability by separating core, database, and media backup logic.
- Ensured consistent tagging and logging across all backup types for easier monitoring and debugging.
- Streamlined the orchestration script (`backup_all.sh`) for improved readability and reduced duplication.

## Version 1.4

### Added
- Introduced efficient lock management in all backup scripts to prevent conflicts during simultaneous Restic operations.
- Automated removal of stale locks before each Restic operation.
- Added global lock mechanism using `flock` to serialize backup operations across multiple processes.
- Implemented dynamic inclusion of Restic password handling by reading it directly from the `/etc/restic-password` file.
- Added automatic validation of Restic repository access before performing backups.

### Fixed
- Fixed issues with manual Restic prompts for the repository password by ensuring passwords are passed via environment variables.
- Resolved potential conflicts caused by simultaneous Restic processes with the introduction of serialized operations.
- Corrected permissions for Restic lock directories to avoid permission-denied errors during backup and restore processes.
- Enhanced password validation logic to ensure backups fail gracefully if the provided password is incorrect.

### Updated
- Updated core, media, and database backup scripts to handle dynamic exclusion paths using a loop-based approach for `--exclude` options.
- Revised logging mechanisms across all scripts to include detailed timestamps and step-specific logs for better traceability.
- Improved script robustness by validating the Restic repository's accessibility upfront using `restic snapshots`.

### Improved
- Optimized all backup scripts to use efficient lock and unlock handling, ensuring smooth operation during concurrent backups and restores.
- Standardized backup script flow across media, database, and core backups, including consistent use of environment variables and error handling.
- Improved log formatting with more descriptive log messages and clear delineation of errors, warnings, and successes.
- Reduced redundancy in password handling by centralizing Restic password retrieval logic.