NomadBackup/README.md

Infrastructure Backup with GitHub Actions

This project provides automated backup solutions for HashiCorp Nomad job specifications and Consul KV store using GitHub Actions. Instead of creating timestamped backup folders, it uses Git's version control to track changes to your infrastructure configurations.

Features

• Automated Backups: Runs daily via GitHub Actions
• Git Version Control: Uses Git commits to track configuration changes
• Dual Format Support: Backs up both HCL and JSON job specifications
• Full Consul KV Backup: Complete Consul key-value store backup and restore
• Secure: Uses GitHub Secrets for API credentials
• Manual Trigger: Can be run on-demand via GitHub UI

Setup Instructions

1. Repository Setup

1. Push this code to a GitHub repository
2. Ensure the repository has write permissions for GitHub Actions

2. GitHub Secrets Configuration

Add the following secrets to your GitHub repository:

1. Go to your repository → Settings → Secrets and variables → Actions
2. Add these secrets:
   • NOMAD_ADDR: Your Nomad API address (e.g., https://nomad.example.com:4646)
   • NOMAD_TOKEN: Your Nomad ACL token (if authentication is enabled)
   • CONSUL_HTTP_ADDR: Your Consul API address (e.g., https://consul.example.com:8500)
   • CONSUL_HTTP_TOKEN: Your Consul ACL token (if authentication is enabled)

3. Schedule Configuration

The workflow is configured to run daily at 2 AM UTC (6 PM PST). To modify the schedule, edit the cron expression in .github/workflows/backup.yml:

schedule:
  - cron: '0 2 * * *'  # Daily at 2 AM UTC

4. Manual Execution

You can manually trigger the backup from the GitHub UI:

1. Go to your repository → Actions → "Infrastructure Backup" workflow
2. Click "Run workflow" → "Run workflow"

How It Works

1. Checkout: The workflow checks out your repository
2. Setup: Installs Python and required dependencies
3. Nomad Backup: Runs the backup.py script to fetch Nomad job specifications
4. Consul Backup: Runs the consul_backup.py script to backup Consul KV store
5. Commit: If changes are detected, commits them to the repository
6. Push: Pushes the changes back to the remote repository

File Structure

├── .github/
│   └── workflows/
│       └── backup.yml          # GitHub Actions workflow
├── backup.py                   # Nomad backup script
├── consul_backup.py            # Consul KV backup script
├── consul_restore.py           # Consul KV restore script
├── nomad_backup/               # Nomad backup directory
│   ├── job1.hcl               # Job specifications in HCL format
│   ├── job2.json              # Job specifications in JSON format
│   └── ...
├── consul_backup/              # Consul backup directory
│   ├── kv/                    # Key-value store backups
│   │   ├── key1.json
│   │   ├── key2.json
│   │   └── nested/
│   │       └── key3.json
│   └── metadata.json          # Backup metadata
└── README.md                   # This file

Local Usage

Nomad Backup

# Basic usage (connects to local Nomad)
python backup.py

# Connect to remote Nomad with token
python backup.py --addr https://nomad.example.com:4646 --token your-token

# Custom output directory
python backup.py --output /path/to/backups

Consul Backup

# Basic usage (connects to local Consul)
python consul_backup.py

# Connect to remote Consul with token
python consul_backup.py --addr https://consul.example.com:8500 --token your-token

# Custom output directory
python consul_backup.py --output /path/to/consul-backups

Consul Restore

# Restore from backup (dry run first)
python consul_restore.py --dry-run

# Actual restore
python consul_restore.py

# Restore to remote Consul
python consul_restore.py --addr https://consul.example.com:8500 --token your-token

Environment Variables

Nomad Backup

• NOMAD_ADDR: Nomad API address (default: http://localhost:4646)
• NOMAD_TOKEN: Nomad ACL token for authentication

Consul Backup/Restore

• CONSUL_HTTP_ADDR: Consul API address (default: http://localhost:8500)
• CONSUL_HTTP_TOKEN: Consul ACL token for authentication

Security Notes

• Never commit sensitive data or tokens to the repository
• Use GitHub Secrets for all credentials
• Review job specifications before committing to ensure no sensitive data is included

Troubleshooting

Common Issues

1. Connection Timeout: Ensure API addresses are correct and accessible
2. Authentication Failure: Verify tokens are valid and have read permissions
3. No Changes Committed: This is normal when configurations haven't changed
4. Consul Backup Fails: Check if Consul KV store is empty or inaccessible

Debug Mode

To debug the GitHub Actions workflow, you can:

1. Add -v flag to the backup command in the workflow
2. Check the Actions logs for detailed error messages

License

This project is open source and available under the MIT License.