mb-backup-manager/readme.md

# MB Backup Manager Documentation

## Overview

The **MB Backup Manager** is a comprehensive backup solution designed for managing automated backups of web applications, particularly those running on the Virtuozzo Application Platform. This tool leverages Restic for efficient backup storage and management, with support for automated scheduling through cron jobs.

## Features

- **Automated Backups**: Schedule and manage backups using `manage_backup_schedule.sh`
- **Backup Types**: 
  - Core files backup (`backup_core_files.sh`)
  - Media files backup (`backup_media.sh`)
  - Database backup (`backup_database.sh`)
- **Full Backup Sessions**: Complete backup orchestration with session-based restoration
- **Backup Management**:
  - View snapshots with tag filtering (`view_snapshots.sh`)
  - View backup sessions as logical units (`view_backup_sessions.sh`)
  - Direct backup restoration (`restore_backup_direct.sh`)
  - **Full backup session restoration** (`restore_full_backup_session.sh`)
  - Repository statistics and maintenance (`check_repo_stats.sh`)
  - Repository validation (`check_backup_repo.sh`)
- **Logging**: Comprehensive logging system with separate directories for automated and manual backups
- **Backup Orchestration**: Centralized backup management through `backup_all.sh`

## New: Full Backup Session Restore

### Overview
The system now supports **hyper-accurate full backup session restoration** that ensures all components (database, core files, and media files) are restored from the same backup session, maintaining data consistency and integrity.

### Key Features
- **Session Validation**: Automatically validates that all required components are present
- **Timestamp Consistency**: Ensures all snapshots belong to the same backup session
- **Component Verification**: Confirms presence of core_files, media_themes, and wordpress_db tags
- **Pre-restoration Backup**: Creates a backup of current state before restoration
- **Comprehensive Logging**: Detailed logs for troubleshooting and audit trails
- **Error Recovery**: Graceful handling of failures with cleanup procedures

### How It Works
1. **Backup Session Creation**: When you run a full backup, three separate snapshots are created:
   - Core files snapshot (tagged with `core_files`)
   - Media files snapshot (tagged with `media_themes`) 
   - Database snapshot (tagged with `wordpress_db`)
   - All snapshots share a common session tag (e.g., `manual-backup-2024-01-15_10-30-45`)

2. **Session Restoration**: The full restore process:
   - Identifies all snapshots with the specified session tag
   - Validates exactly 3 snapshots are present
   - Confirms all required component tags are present
   - Checks timestamp consistency (within 30 minutes)
   - Creates pre-restoration backup
   - Restores components in proper order (database → core files → media files)
   - Verifies restoration integrity

### Usage

#### Via Web Interface
1. **View Backup Sessions**: Use "View Backup Sessions" to see all complete backup sessions
2. **Full Restore**: Use "Full Backup Session Restore" and enter the session tag (e.g., `manual-backup-2024-01-15_10-30-45`)

#### Via Command Line
```bash
# View complete backup sessions
./view_backup_sessions.sh sessions

# View incomplete backup sessions
./view_backup_sessions.sh incomplete

# View all sessions with details
./view_backup_sessions.sh all

# Validate a specific backup session
./view_backup_sessions.sh validate manual-backup-2024-01-15_10-30-45

# Restore a complete backup session
./restore_full_backup_session.sh manual-backup-2024-01-15_10-30-45
```

### Safety Features
- **Format Validation**: Ensures session tag follows expected pattern
- **Component Count Validation**: Requires exactly 3 snapshots per session
- **Tag Validation**: Verifies all required component tags are present
- **Timestamp Validation**: Checks for reasonable time differences between snapshots
- **Pre-restoration Backup**: Creates backup of current state before restoration
- **Database Connectivity Test**: Validates database access before restoration
- **Comprehensive Error Handling**: Detailed error messages and cleanup procedures

### Example Output
```
=== COMPLETE BACKUP SESSIONS ===
Format: [Session Tag] [Date/Time] [Status]
========================================
manual-backup-2024-01-15_10-30-45    2024-01-15 10:30:45 [COMPLETE]
automated-backup-2024-01-15_02-00-30 2024-01-15 02:00:30 [COMPLETE]
========================================
Total complete backup sessions: 2

To restore a complete session, use:
  ./restore_full_backup_session.sh <session_tag>
```

## Installation

1. **Clone the Repository**:
   Clone the repository containing the backup manager scripts to your local machine or server.
   [MB Backup Manager Repository](https://deploy.mightybox.io/addons/mb-backup-manager)

2. **Install Dependencies**:
   Ensure that `Restic` and `cron` are installed on your system:

   ```bash
   sudo dnf install -y cronie
   sudo dnf install -y restic
   ```

3. **Set Up Directory Structure**:
   Create the necessary directories for logs:

   ```bash
   mkdir -p /home/litespeed/mb-backups/logs/auto
   mkdir -p /home/litespeed/mb-backups/logs/manual
   mkdir -p /home/litespeed/mb-backups/logs/restore
   ```

4. **Configure Environment**:
   Ensure the Restic password is stored in `/etc/restic-password`.

## Core Scripts and Usage

### Backup Management

1. **Viewing Snapshots** (`view_snapshots.sh`):
   - View all snapshots or filter by specific tags
   - Displays snapshot IDs, timestamps, and associated tags
   - Supports an `all` tag to show all snapshots regardless of tags

2. **Viewing Backup Sessions** (`view_backup_sessions.sh`):
   - View backup sessions as logical units
   - Identify complete vs incomplete backup sessions
   - Validate specific backup sessions before restoration
   - Shows session timestamps and missing components

3. **Full Backup Session Restoration** (`restore_full_backup_session.sh`):
   - Restores complete backup sessions with all components
   - Validates session integrity before restoration
   - Creates pre-restoration backup for safety
   - Comprehensive error handling and logging

4. **Direct Backup Restoration** (`restore_backup_direct.sh`):
   - Restores specific backup types based on tags
   - Handles direct database restoration by piping SQL files from Restic to MySQL
   - Automatically detects database credentials from `wp-config.php`

5. **Backup Scheduling** (`manage_backup_schedule.sh`):
   - Add or update backup schedules with cron
   - Remove existing backup schedules
   - Validates cron syntax and dependencies
   - Logs all scheduling actions

6. **Repository Maintenance** (`check_repo_stats.sh`):
   - Retrieves detailed repository statistics.
   - Applies retention policies to manage snapshot storage:
     - **Default Retention**: Keeps only the last 7 snapshots to balance storage usage and recovery needs.
     - Forgotten snapshots are pruned to reclaim storage space.
   - Performs repository integrity checks to ensure data consistency and reliability.
   - Automatically detects and removes stale locks to prevent repository conflicts.
   - Logs all actions and results to `repo_stats_YYYY-MM-DD.log` for traceability.

7. **Repository Validation** (`check_backup_repo.sh`):
   - Validates repository integrity
   - Initializes empty repositories
   - Handles stale locks
   - Logs to `/home/litespeed/logs/backup_repo_check.log`

### Backup Components

1. **Complete Backup** (`backup_all.sh`):
   - Orchestrates the entire backup process
   - Calls individual backup scripts for core, media, and database
   - Applies consistent tagging (`manual-backup` or `auto-backup` with timestamps)
   - Handles error logging and reporting

2. **Individual Backup Scripts**:
   - `backup_core_files.sh`: Backs up essential application files
   - `backup_media.sh`: Handles media file backups
   - `backup_database.sh`: Manages database backups
   - Each script can run independently or as part of `backup_all.sh`

### Schedule Monitoring

**Check Scheduled Backups** (`check_sched.sh`):
```bash
./check_sched.sh
```
Displays current backup-related cron jobs and automation status.

## Available Commands

### Backup Management Commands

1. **Schedule Management** (`manage_backup_schedule.sh`):
   ```bash
   # Add or update a backup schedule
   ./manage_backup_schedule.sh add "0 0 * * *" "your_restic_password"
   
   # Remove all backup schedules
   ./manage_backup_schedule.sh remove
   
   # List current backup schedules
   ./manage_backup_schedule.sh list
   ```

2. **Manual Backup Operations** (`backup_all.sh`):
   ```bash
   # Run a manual backup
   ./backup_all.sh manual
   
   # Run an automated backup (used by cron)
   ./backup_all.sh auto
   ```

3. **Individual Backup Components**:
   ```bash
   # Core files backup
   ./backup_core_files.sh "restic_password" "backup_tag"
   
   # Database backup
   ./backup_database.sh "restic_password" "backup_tag"
   
   # Media files backup
   ./backup_media.sh "restic_password" "backup_tag"
   ```

4. **Repository Management**:
   ```bash
   # Check repository status
   ./check_repo_stats.sh
   
   # Validate repository
   ./check_backup_repo.sh
   
   # View snapshots (all or by tag)
   ./view_snapshots.sh
   ```

5. **Backup Session Management**:
   ```bash
   # View complete backup sessions
   ./view_backup_sessions.sh sessions
   
   # View incomplete backup sessions
   ./view_backup_sessions.sh incomplete
   
   # View all sessions with details
   ./view_backup_sessions.sh all
   
   # Validate specific backup session
   ./view_backup_sessions.sh validate manual-backup-2024-01-15_10-30-45
   
   # Restore complete backup session
   ./restore_full_backup_session.sh manual-backup-2024-01-15_10-30-45
   ```

6. **Schedule Monitoring**:
   ```bash
   # Check current backup schedules
   ./check_sched.sh
   ```

7. **Backup Restoration**:
   ```bash
   # Restore individual backup
   ./restore_backup_direct.sh <snapshot_id>
   
   # Restore complete backup session
   ./restore_full_backup_session.sh <session_tag>
   ```

### Environment Variables

The following environment variables are used by the scripts:

- `RESTIC_PASSWORD`: Stored in `/etc/restic-password`
- `RESTIC_REPOSITORY`: Default path `/mnt/backup`

### Log File Locations

- Automated backup logs: `/home/litespeed/mb-backups/logs/auto/`
- Manual backup logs: `/home/litespeed/mb-backups/logs/manual/`
- Restoration logs: `/home/litespeed/mb-backups/logs/restore/`
- Repository stats: `repo_stats_YYYY-MM-DD.log`
- Repository check logs: `/home/litespeed/logs/backup_repo_check.log`
- Schedule actions log: `/home/litespeed/mb-backups/logs/auto/schedule_actions.log`
- Backup addon log: `/var/log/backup_addon.log`

## Troubleshooting

1. **Cron Service Issues**:
   ```bash
   sudo systemctl status crond
   sudo systemctl start crond
   sudo systemctl enable crond
   ```

2. **Log Locations**:
   - Automated backup logs: `/home/litespeed/mb-backups/logs/auto/`
   - Manual backup logs: `/home/litespeed/mb-backups/logs/manual/`
   - Restoration logs: `/home/litespeed/mb-backups/logs/restore/`
   - Repository stats: `repo_stats_YYYY-MM-DD.log`
   - Repository check logs: `/home/litespeed/logs/backup_repo_check.log`

3. **Common Issues**:
   - Stale locks: Automatically handled by scripts
   - Repository access: Verify Restic password in `/etc/restic-password`
   - Backup failures: Check corresponding log files for error messages
   - Cron job issues: Verify cron service status and job configuration
   - Full restore failures: Check session completeness with `view_backup_sessions.sh validate`

4. **Full Restore Troubleshooting**:
   - **Incomplete Session**: Use `view_backup_sessions.sh incomplete` to see missing components
   - **Invalid Session Tag**: Ensure tag format matches `manual-backup-YYYY-MM-DD_HH-MM-SS`
   - **Timestamp Issues**: Check for large time differences between snapshots
   - **Database Connection**: Verify MySQL service and credentials before restoration

## Repository Information

For the latest updates and commit history:
[MB Backup Manager Repository](https://deploy.mightybox.io/addons/mb-backup-manager)