addons
/
mb-backup-manager
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
mb-backup-manager/readme.md

12 KiB
Raw Permalink Blame History

MB Backup Manager Documentation

Overview

The MB Backup Manager is a comprehensive backup solution designed for managing automated backups of web applications, particularly those running on the Virtuozzo Application Platform. This tool leverages Restic for efficient backup storage and management, with support for automated scheduling through cron jobs.

Features

  • Automated Backups: Schedule and manage backups using manage_backup_schedule.sh
  • Backup Types:
    • Core files backup (backup_core_files.sh)
    • Media files backup (backup_media.sh)
    • Database backup (backup_database.sh)
  • Full Backup Sessions: Complete backup orchestration with session-based restoration
  • Backup Management:
    • View snapshots with tag filtering (view_snapshots.sh)
    • View backup sessions as logical units (view_backup_sessions.sh)
    • Direct backup restoration (restore_backup_direct.sh)
    • Full backup session restoration (restore_full_backup_session.sh)
    • Repository statistics and maintenance (check_repo_stats.sh)
    • Repository validation (check_backup_repo.sh)
  • Logging: Comprehensive logging system with separate directories for automated and manual backups
  • Backup Orchestration: Centralized backup management through backup_all.sh

New: Full Backup Session Restore

Overview

The system now supports hyper-accurate full backup session restoration that ensures all components (database, core files, and media files) are restored from the same backup session, maintaining data consistency and integrity.

Key Features

  • Session Validation: Automatically validates that all required components are present
  • Timestamp Consistency: Ensures all snapshots belong to the same backup session
  • Component Verification: Confirms presence of core_files, media_themes, and wordpress_db tags
  • Pre-restoration Backup: Creates a backup of current state before restoration
  • Comprehensive Logging: Detailed logs for troubleshooting and audit trails
  • Error Recovery: Graceful handling of failures with cleanup procedures

How It Works

  1. Backup Session Creation: When you run a full backup, three separate snapshots are created:

    • Core files snapshot (tagged with core_files)
    • Media files snapshot (tagged with media_themes)
    • Database snapshot (tagged with wordpress_db)
    • All snapshots share a common session tag (e.g., manual-backup-2024-01-15_10-30-45)

  2. Session Restoration: The full restore process:

    • Identifies all snapshots with the specified session tag
    • Validates exactly 3 snapshots are present
    • Confirms all required component tags are present
    • Checks timestamp consistency (within 30 minutes)
    • Creates pre-restoration backup
    • Restores components in proper order (database → core files → media files)
    • Verifies restoration integrity

Usage

Via Web Interface

  1. View Backup Sessions: Use "View Backup Sessions" to see all complete backup sessions
  2. Full Restore: Use "Full Backup Session Restore" and enter the session tag (e.g., manual-backup-2024-01-15_10-30-45)

Via Command Line

# View complete backup sessions
./view_backup_sessions.sh sessions

# View incomplete backup sessions
./view_backup_sessions.sh incomplete

# View all sessions with details
./view_backup_sessions.sh all

# Validate a specific backup session
./view_backup_sessions.sh validate manual-backup-2024-01-15_10-30-45

# Restore a complete backup session
./restore_full_backup_session.sh manual-backup-2024-01-15_10-30-45

Safety Features

  • Format Validation: Ensures session tag follows expected pattern
  • Component Count Validation: Requires exactly 3 snapshots per session
  • Tag Validation: Verifies all required component tags are present
  • Timestamp Validation: Checks for reasonable time differences between snapshots
  • Pre-restoration Backup: Creates backup of current state before restoration
  • Database Connectivity Test: Validates database access before restoration
  • Comprehensive Error Handling: Detailed error messages and cleanup procedures

Example Output

=== COMPLETE BACKUP SESSIONS ===
Format: [Session Tag] [Date/Time] [Status]
========================================
manual-backup-2024-01-15_10-30-45    2024-01-15 10:30:45 [COMPLETE]
automated-backup-2024-01-15_02-00-30 2024-01-15 02:00:30 [COMPLETE]
========================================
Total complete backup sessions: 2

To restore a complete session, use:
  ./restore_full_backup_session.sh <session_tag>

Installation

  1. Clone the Repository: Clone the repository containing the backup manager scripts to your local machine or server. MB Backup Manager Repository

  2. Install Dependencies: Ensure that Restic and cron are installed on your system:

    sudo dnf install -y cronie
sudo dnf install -y restic

  3. Set Up Directory Structure: Create the necessary directories for logs:

    mkdir -p /home/litespeed/mb-backups/logs/auto
mkdir -p /home/litespeed/mb-backups/logs/manual
mkdir -p /home/litespeed/mb-backups/logs/restore

  4. Configure Environment: Ensure the Restic password is stored in /etc/restic-password.

Core Scripts and Usage

Backup Management

  1. Viewing Snapshots (view_snapshots.sh):

    • View all snapshots or filter by specific tags
    • Displays snapshot IDs, timestamps, and associated tags
    • Supports an all tag to show all snapshots regardless of tags

  2. Viewing Backup Sessions (view_backup_sessions.sh):

    • View backup sessions as logical units
    • Identify complete vs incomplete backup sessions
    • Validate specific backup sessions before restoration
    • Shows session timestamps and missing components

  3. Full Backup Session Restoration (restore_full_backup_session.sh):

    • Restores complete backup sessions with all components
    • Validates session integrity before restoration
    • Creates pre-restoration backup for safety
    • Comprehensive error handling and logging

  4. Direct Backup Restoration (restore_backup_direct.sh):

    • Restores specific backup types based on tags
    • Handles direct database restoration by piping SQL files from Restic to MySQL
    • Automatically detects database credentials from wp-config.php

  5. Backup Scheduling (manage_backup_schedule.sh):

    • Add or update backup schedules with cron
    • Remove existing backup schedules
    • Validates cron syntax and dependencies
    • Logs all scheduling actions

  6. Repository Maintenance (check_repo_stats.sh):

    • Retrieves detailed repository statistics.
    • Applies retention policies to manage snapshot storage:
      • Default Retention: Keeps only the last 7 snapshots to balance storage usage and recovery needs.
      • Forgotten snapshots are pruned to reclaim storage space.
    • Performs repository integrity checks to ensure data consistency and reliability.
    • Automatically detects and removes stale locks to prevent repository conflicts.
    • Logs all actions and results to repo_stats_YYYY-MM-DD.log for traceability.

  7. Repository Validation (check_backup_repo.sh):

    • Validates repository integrity
    • Initializes empty repositories
    • Handles stale locks
    • Logs to /home/litespeed/logs/backup_repo_check.log

Backup Components

  1. Complete Backup (backup_all.sh):

    • Orchestrates the entire backup process
    • Calls individual backup scripts for core, media, and database
    • Applies consistent tagging (manual-backup or auto-backup with timestamps)
    • Handles error logging and reporting

  2. Individual Backup Scripts:

    • backup_core_files.sh: Backs up essential application files
    • backup_media.sh: Handles media file backups
    • backup_database.sh: Manages database backups
    • Each script can run independently or as part of backup_all.sh

Schedule Monitoring

Check Scheduled Backups (check_sched.sh):

./check_sched.sh

Displays current backup-related cron jobs and automation status.

Available Commands

Backup Management Commands

  1. Schedule Management (manage_backup_schedule.sh):

    # Add or update a backup schedule
./manage_backup_schedule.sh add "0 0 * * *" "your_restic_password"

# Remove all backup schedules
./manage_backup_schedule.sh remove

# List current backup schedules
./manage_backup_schedule.sh list

  2. Manual Backup Operations (backup_all.sh):

    # Run a manual backup
./backup_all.sh manual

# Run an automated backup (used by cron)
./backup_all.sh auto

  3. Individual Backup Components:

    # Core files backup
./backup_core_files.sh "restic_password" "backup_tag"

# Database backup
./backup_database.sh "restic_password" "backup_tag"

# Media files backup
./backup_media.sh "restic_password" "backup_tag"

  4. Repository Management:

    # Check repository status
./check_repo_stats.sh

# Validate repository
./check_backup_repo.sh

# View snapshots (all or by tag)
./view_snapshots.sh

  5. Backup Session Management:

    # View complete backup sessions
./view_backup_sessions.sh sessions

# View incomplete backup sessions
./view_backup_sessions.sh incomplete

# View all sessions with details
./view_backup_sessions.sh all

# Validate specific backup session
./view_backup_sessions.sh validate manual-backup-2024-01-15_10-30-45

# Restore complete backup session
./restore_full_backup_session.sh manual-backup-2024-01-15_10-30-45

  6. Schedule Monitoring:

    # Check current backup schedules
./check_sched.sh

  7. Backup Restoration:

    # Restore individual backup
./restore_backup_direct.sh <snapshot_id>

# Restore complete backup session
./restore_full_backup_session.sh <session_tag>

Environment Variables

The following environment variables are used by the scripts:

  • RESTIC_PASSWORD: Stored in /etc/restic-password
  • RESTIC_REPOSITORY: Default path /mnt/backup

Log File Locations

  • Automated backup logs: /home/litespeed/mb-backups/logs/auto/
  • Manual backup logs: /home/litespeed/mb-backups/logs/manual/
  • Restoration logs: /home/litespeed/mb-backups/logs/restore/
  • Repository stats: repo_stats_YYYY-MM-DD.log
  • Repository check logs: /home/litespeed/logs/backup_repo_check.log
  • Schedule actions log: /home/litespeed/mb-backups/logs/auto/schedule_actions.log
  • Backup addon log: /var/log/backup_addon.log

Troubleshooting

  1. Cron Service Issues:

    sudo systemctl status crond
sudo systemctl start crond
sudo systemctl enable crond

  2. Log Locations:

    • Automated backup logs: /home/litespeed/mb-backups/logs/auto/
    • Manual backup logs: /home/litespeed/mb-backups/logs/manual/
    • Restoration logs: /home/litespeed/mb-backups/logs/restore/
    • Repository stats: repo_stats_YYYY-MM-DD.log
    • Repository check logs: /home/litespeed/logs/backup_repo_check.log

  3. Common Issues:

    • Stale locks: Automatically handled by scripts
    • Repository access: Verify Restic password in /etc/restic-password
    • Backup failures: Check corresponding log files for error messages
    • Cron job issues: Verify cron service status and job configuration
    • Full restore failures: Check session completeness with view_backup_sessions.sh validate

  4. Full Restore Troubleshooting:

    • Incomplete Session: Use view_backup_sessions.sh incomplete to see missing components
    • Invalid Session Tag: Ensure tag format matches manual-backup-YYYY-MM-DD_HH-MM-SS
    • Timestamp Issues: Check for large time differences between snapshots
    • Database Connection: Verify MySQL service and credentials before restoration

Repository Information

For the latest updates and commit history: MB Backup Manager Repository