diff --git a/Design.md b/Design.md index b5d49df..2483fbe 100644 --- a/Design.md +++ b/Design.md @@ -9,13 +9,13 @@ ## **Core Features** -### **CLI Mode (Current)** +### **CLI Mode (Implemented)** 1. List all activities (`garminsync list --all`) 2. List activities that have not been downloaded (`garminsync list --missing`) 3. List activities that have been downloaded (`garminsync list --downloaded`) 4. Download all missing activities (`garminsync download --missing`) -### **Enhanced Features (New)** +### **Enhanced Features (Implemented)** 5. **Offline Mode**: List activities without polling Garmin Connect (`garminsync list --missing --offline`) 6. **Daemon Mode**: Run as background service with scheduled downloads (`garminsync daemon --start`) 7. **Web UI**: Browser-based interface for daemon monitoring and configuration (`http://localhost:8080`) @@ -24,7 +24,7 @@ ## **Tech Stack 🐍** -* **Frontend:** CLI (**Python**) +* **Frontend:** CLI (**Python** with Typer) + Web UI (FastAPI + Jinja2) * **Backend:** **Python** * **Database:** SQLite (`garmin.db`) * **Hosting:** Docker container @@ -85,23 +85,23 @@ class SyncLog(Base): ## **User Flow** -### **CLI Mode (Existing)** +### **CLI Mode (Implemented)** 1. User sets up credentials in `.env` file with `GARMIN_EMAIL` and `GARMIN_PASSWORD` 2. User launches the container: `docker run -it --env-file .env -v $(pwd)/data:/app/data garminsync` 3. User runs commands like `garminsync download --missing` 4. Application syncs with Garmin Connect, shows progress bars, and downloads activities -### **Offline Mode (New)** +### **Offline Mode (Implemented)** 1. User runs `garminsync list --missing --offline` to view cached data without API calls 2. Application queries local database only, showing last known state 3. Useful for checking status without network connectivity or API rate limits -### **Daemon Mode (New)** -1. User starts daemon: `garminsync daemon --start` -2. Daemon runs in background, scheduling automatic sync/download operations +### **Daemon Mode (Implemented)** +1. User starts daemon: `garminsync daemon` (runs continuously in foreground) +2. Daemon automatically starts web UI and background scheduler 3. User accesses web UI at `http://localhost:8080` for monitoring and configuration 4. Web UI provides real-time status, logs, and schedule management -5. Daemon can be stopped with `garminsync daemon --stop` or through web UI +5. Daemon can be stopped with `Ctrl+C` or through web UI stop functionality ----- @@ -116,24 +116,25 @@ class SyncLog(Base): │ ├── database.py # SQLAlchemy models and database operations │ ├── garmin.py # Garmin Connect client wrapper with robust download logic │ ├── daemon.py # Daemon mode implementation with APScheduler -│ ├── web/ # Web UI components -│ │ ├── __init__.py -│ │ ├── app.py # FastAPI application setup -│ │ ├── routes.py # API endpoints for web UI -│ │ ├── static/ # CSS, JavaScript, images -│ │ │ ├── style.css -│ │ │ └── app.js -│ │ └── templates/ # Jinja2 HTML templates -│ │ ├── base.html -│ │ ├── dashboard.html -│ │ └── config.html -│ └── utils.py # Shared utilities and helpers +│ ├── utils.py # Shared utilities and helpers +│ └── web/ # Web UI components +│ ├── __init__.py +│ ├── app.py # FastAPI application setup +│ ├── routes.py # API endpoints for web UI +│ ├── static/ # CSS, JavaScript, images +│ │ ├── style.css +│ │ └── app.js +│ └── templates/ # Jinja2 HTML templates +│ ├── base.html +│ ├── dashboard.html +│ └── config.html ├── data/ # Directory for downloaded .fit files and SQLite DB ├── .env # Stores GARMIN_EMAIL/GARMIN_PASSWORD (gitignored) -├── .gitignore # Excludes .env file +├── .gitignore # Excludes .env file and data directory ├── Dockerfile # Production-ready container configuration ├── Design.md # This design document -└── requirements.txt # Pinned Python dependencies (updated) +├── plan.md # Implementation notes and fixes +└── requirements.txt # Python dependencies with compatibility fixes ``` ----- @@ -202,432 +203,38 @@ class SyncLog(Base): - [x] **Database Updates**: Proper status tracking and file path storage - [x] **File Management**: Safe filename generation and directory creation -#### **Phase 4: Polish** -- [x] **Progress Bars**: Comprehensive tqdm implementation across all operations -- [x] **Error Handling**: Graceful error handling with informative messages -- [x] **Container Optimization**: Efficient Docker build with proper dependency management +#### **Phase 4: Enhanced Features** +- [x] **Offline Mode**: List activities without API calls using cached data +- [x] **Daemon Mode**: Background service with APScheduler for automatic sync +- [x] **Web UI**: FastAPI-based dashboard with real-time monitoring +- [x] **Schedule Configuration**: Configurable cron-based sync schedules +- [x] **Activity Logs**: Comprehensive logging of sync operations -### **🚧 New Features Implementation Guide** +#### **Phase 5: Web Interface** +- [x] **Dashboard**: Real-time statistics and daemon status monitoring +- [x] **API Routes**: RESTful endpoints for configuration and control +- [x] **Templates**: Responsive HTML templates with Bootstrap styling +- [x] **JavaScript Integration**: Auto-refreshing status and interactive controls +- [x] **Configuration Management**: Web-based daemon settings and schedule updates -#### **Feature 1: Offline Mode** +### **🔧 Recent Fixes and Improvements** -**Implementation Steps:** -1. **CLI Enhancement** (`cli.py`): - ```python - @app.command("list") - def list_activities( - all_activities: bool = False, - missing: bool = False, - downloaded: bool = False, - offline: Annotated[bool, typer.Option("--offline", help="Work offline without syncing")] = False - ): - if not offline: - # Existing sync logic - sync_database(client) - else: - typer.echo("Working in offline mode - using cached data") - - # Rest of listing logic remains the same - ``` +#### **Dependency Management** +- [x] **Pydantic Compatibility**: Fixed version constraints to avoid conflicts with `garth` +- [x] **Requirements Lock**: Updated to `pydantic>=2.0.0,<2.5.0` for stability +- [x] **Package Versions**: Verified compatibility across all dependencies -2. **Database Enhancements** (`database.py`): - - Add `last_sync` column to Activity table - - Add utility functions for offline status checking - ```python - def get_offline_stats(): - """Return statistics about cached data without API calls""" - session = get_session() - total = session.query(Activity).count() - downloaded = session.query(Activity).filter_by(downloaded=True).count() - missing = total - downloaded - last_sync = session.query(Activity).order_by(Activity.last_sync.desc()).first() - return { - 'total': total, - 'downloaded': downloaded, - 'missing': missing, - 'last_sync': last_sync.last_sync if last_sync else 'Never' - } - ``` +#### **Code Quality Fixes** +- [x] **Missing Fields**: Added `created_at` field to Activity model and sync operations +- [x] **Import Issues**: Resolved circular import problems in daemon module +- [x] **Error Handling**: Improved exception handling and logging throughout +- [x] **Method Names**: Corrected method calls and parameter names -#### **Feature 2: Daemon Mode** - -**Implementation Steps:** -1. **New Daemon Module** (`daemon.py`): - ```python - from apscheduler.schedulers.background import BackgroundScheduler - from apscheduler.triggers.cron import CronTrigger - import signal - import sys - import time - import threading - from datetime import datetime - - class GarminSyncDaemon: - def __init__(self): - self.scheduler = BackgroundScheduler() - self.running = False - self.web_server = None - - def start(self, web_port=8080): - """Start daemon with scheduler and web UI""" - # Load configuration from database - config = self.load_config() - - # Setup scheduled job - if config.enabled: - self.scheduler.add_job( - func=self.sync_and_download, - trigger=CronTrigger.from_crontab(config.schedule_cron), - id='sync_job', - replace_existing=True - ) - - # Start scheduler - self.scheduler.start() - self.running = True - - # Start web UI in separate thread - self.start_web_ui(web_port) - - # Setup signal handlers for graceful shutdown - signal.signal(signal.SIGINT, self.signal_handler) - signal.signal(signal.SIGTERM, self.signal_handler) - - print(f"Daemon started. Web UI available at http://localhost:{web_port}") - - # Keep daemon running - try: - while self.running: - time.sleep(1) - except KeyboardInterrupt: - self.stop() - - def sync_and_download(self): - """Scheduled job function""" - try: - self.log_operation("sync", "started") - - # Perform sync and download - from .garmin import GarminClient - from .database import sync_database - - client = GarminClient() - activities_before = self.count_missing() - - sync_database(client) - - # Download missing activities - downloaded_count = self.download_missing_activities(client) - - self.log_operation("sync", "success", - f"Downloaded {downloaded_count} new activities") - - except Exception as e: - self.log_operation("sync", "error", str(e)) - - def load_config(self): - """Load daemon configuration from database""" - session = get_session() - config = session.query(DaemonConfig).first() - if not config: - # Create default configuration - config = DaemonConfig() - session.add(config) - session.commit() - session.close() - return config - ``` - -2. **CLI Integration** (`cli.py`): - ```python - @app.command("daemon") - def daemon_mode( - start: Annotated[bool, typer.Option("--start", help="Start daemon")] = False, - stop: Annotated[bool, typer.Option("--stop", help="Stop daemon")] = False, - status: Annotated[bool, typer.Option("--status", help="Show daemon status")] = False, - port: Annotated[int, typer.Option("--port", help="Web UI port")] = 8080 - ): - """Daemon mode operations""" - from .daemon import GarminSyncDaemon - - if start: - daemon = GarminSyncDaemon() - daemon.start(web_port=port) - elif stop: - # Implementation for stopping daemon (PID file or signal) - pass - elif status: - # Show current daemon status - pass - ``` - -#### **Feature 3: Web UI** - -**Implementation Steps:** -1. **FastAPI Application** (`web/app.py`): - ```python - from fastapi import FastAPI, Request - from fastapi.staticfiles import StaticFiles - from fastapi.templating import Jinja2Templates - from .routes import router - - app = FastAPI(title="GarminSync Dashboard") - - # Mount static files and templates - app.mount("/static", StaticFiles(directory="garminsync/web/static"), name="static") - templates = Jinja2Templates(directory="garminsync/web/templates") - - # Include API routes - app.include_router(router) - - @app.get("/") - async def dashboard(request: Request): - # Get current statistics - from ..database import get_offline_stats - stats = get_offline_stats() - - return templates.TemplateResponse("dashboard.html", { - "request": request, - "stats": stats - }) - ``` - -2. **API Routes** (`web/routes.py`): - ```python - from fastapi import APIRouter, HTTPException - from pydantic import BaseModel - from ..database import get_session, DaemonConfig, SyncLog - - router = APIRouter(prefix="/api") - - class ScheduleConfig(BaseModel): - enabled: bool - cron_schedule: str - - @router.get("/status") - async def get_status(): - """Get current daemon status""" - session = get_session() - config = session.query(DaemonConfig).first() - - # Get recent logs - logs = session.query(SyncLog).order_by(SyncLog.timestamp.desc()).limit(10).all() - - return { - "daemon": { - "running": config.status == "running" if config else False, - "next_run": config.next_run if config else None, - "schedule": config.schedule_cron if config else None - }, - "recent_logs": [ - { - "timestamp": log.timestamp, - "operation": log.operation, - "status": log.status, - "message": log.message - } for log in logs - ] - } - - @router.post("/schedule") - async def update_schedule(config: ScheduleConfig): - """Update daemon schedule configuration""" - session = get_session() - daemon_config = session.query(DaemonConfig).first() - - if not daemon_config: - daemon_config = DaemonConfig() - session.add(daemon_config) - - daemon_config.enabled = config.enabled - daemon_config.schedule_cron = config.cron_schedule - session.commit() - - return {"message": "Configuration updated successfully"} - - @router.post("/sync/trigger") - async def trigger_sync(): - """Manually trigger a sync operation""" - # Implementation to trigger immediate sync - pass - ``` - -3. **HTML Templates** (`web/templates/dashboard.html`): - ```html - {% extends "base.html" %} - - {% block content %} -
-

GarminSync Dashboard

- -
-
-
-
Statistics
-
-

Total Activities: {{ stats.total }}

-

Downloaded: {{ stats.downloaded }}

-

Missing: {{ stats.missing }}

-

Last Sync: {{ stats.last_sync }}

-
-
-
- -
-
-
Daemon Status
-
- -
-
-
- -
-
-
Quick Actions
-
- - -
-
-
-
- -
-
-
-
Recent Activity
-
- -
-
-
-
- -
-
-
-
Schedule Configuration
-
-
-
- - -
-
- - -
- -
-
-
-
-
-
- {% endblock %} - ``` - -4. **JavaScript for Interactivity** (`web/static/app.js`): - ```javascript - // Auto-refresh dashboard data - setInterval(updateStatus, 30000); // Every 30 seconds - - async function updateStatus() { - try { - const response = await fetch('/api/status'); - const data = await response.json(); - - // Update daemon status - document.getElementById('daemon-status').innerHTML = ` -

Status: - ${data.daemon.running ? 'Running' : 'Stopped'} -

-

Next Run: ${data.daemon.next_run || 'Not scheduled'}

-

Schedule: ${data.daemon.schedule || 'Not configured'}

- `; - - // Update recent logs - const logsHtml = data.recent_logs.map(log => ` -
- ${log.timestamp} - - ${log.status} - - ${log.operation}: ${log.message || ''} -
- `).join(''); - - document.getElementById('recent-logs').innerHTML = logsHtml; - - } catch (error) { - console.error('Failed to update status:', error); - } - } - - async function triggerSync() { - try { - await fetch('/api/sync/trigger', { method: 'POST' }); - alert('Sync triggered successfully'); - updateStatus(); - } catch (error) { - alert('Failed to trigger sync'); - } - } - - // Initialize on page load - document.addEventListener('DOMContentLoaded', updateStatus); - ``` - -### **Updated Requirements** (`requirements.txt`): -``` -typer==0.9.0 -click==8.1.7 -python-dotenv==1.0.0 -garminconnect==0.2.28 -sqlalchemy==2.0.23 -tqdm==4.66.1 -fastapi==0.104.1 -uvicorn[standard]==0.24.0 -apscheduler==3.10.4 -pydantic==2.5.0 -jinja2==3.1.2 -python-multipart==0.0.6 -aiofiles==23.2.1 -``` - -### **Docker Updates**: -```dockerfile -# Expose web UI port -EXPOSE 8080 - -# Update entrypoint to support daemon mode -ENTRYPOINT ["python", "-m", "garminsync.cli"] -CMD ["--help"] -``` - -### **Usage Examples**: - -**Offline Mode:** -```bash -# List missing activities without network calls -docker run --env-file .env -v $(pwd)/data:/app/data garminsync list --missing --offline -``` - -**Daemon Mode:** -```bash -# Start daemon with web UI on port 8080 -docker run -d --env-file .env -v $(pwd)/data:/app/data -p 8080:8080 garminsync daemon --start - -# Access web UI at http://localhost:8080 -``` +#### **Web UI Enhancements** +- [x] **Template Safety**: Added fallback handling for missing template files +- [x] **API Error Handling**: Improved error responses and status codes +- [x] **JavaScript Functions**: Added missing daemon control functions +- [x] **Status Updates**: Real-time status updates with proper data formatting ----- @@ -648,8 +255,14 @@ docker run -it --env-file .env -v $(pwd)/data:/app/data garminsync --help # List all activities docker run -it --env-file .env -v $(pwd)/data:/app/data garminsync list --all +# List missing activities offline +docker run -it --env-file .env -v $(pwd)/data:/app/data garminsync list --missing --offline + # Download missing activities docker run -it --env-file .env -v $(pwd)/data:/app/data garminsync download --missing + +# Start daemon with web UI +docker run -it --env-file .env -v $(pwd)/data:/app/data -p 8080:8080 garminsync daemon ``` ----- @@ -689,21 +302,87 @@ def sync_database(garmin_client): # Only add new activities, preserve existing download status existing = session.query(Activity).filter_by(activity_id=activity_id).first() if not existing: - new_activity = Activity(...) + new_activity = Activity( + activity_id=activity_id, + start_time=start_time, + downloaded=False, + created_at=datetime.now().isoformat(), + last_sync=datetime.now().isoformat() + ) session.add(new_activity) ``` -### **CLI Integration** -Clean separation between CLI interface and business logic with proper type annotations: +### **Daemon Implementation** +The daemon uses APScheduler for reliable background task execution: ```python -def list_activities( - all_activities: Annotated[bool, typer.Option("--all", help="List all activities")] = False, - missing: Annotated[bool, typer.Option("--missing", help="List missing activities")] = False, - downloaded: Annotated[bool, typer.Option("--downloaded", help="List downloaded activities")] = False -): +class GarminSyncDaemon: + def __init__(self): + self.scheduler = BackgroundScheduler() + self.running = False + self.web_server = None + + def start(self, web_port=8080): + config_data = self.load_config() + if config_data['enabled']: + self.scheduler.add_job( + func=self.sync_and_download, + trigger=CronTrigger.from_crontab(config_data['schedule_cron']), + id='sync_job', + replace_existing=True + ) ``` +### **Web API Integration** +FastAPI provides RESTful endpoints for daemon control and monitoring: + +```python +@router.get("/status") +async def get_status(): + """Get current daemon status with logs""" + config = session.query(DaemonConfig).first() + logs = session.query(SyncLog).order_by(SyncLog.timestamp.desc()).limit(10).all() + return { + "daemon": {"running": config.status == "running"}, + "recent_logs": [{"timestamp": log.timestamp, "status": log.status} for log in logs] + } +``` + +----- + +## **Known Issues & Limitations** + +### **Current Limitations** +1. **Web Interface**: Some components need completion (detailed below) +2. **Error Recovery**: Limited automatic retry logic for failed downloads +3. **Batch Processing**: No support for selective activity date range downloads +4. **Authentication**: No support for two-factor authentication (2FA) + +### **Dependency Issues Resolved** +- ✅ **Pydantic Conflicts**: Fixed version constraints to avoid `garth` compatibility issues +- ✅ **Missing Fields**: Added all required database fields +- ✅ **Import Errors**: Resolved circular import problems + +----- + +## **Performance Considerations** + +- **Rate Limiting**: 2-second delays between API requests prevent server overload +- **Batch Processing**: Fetches up to 1000 activities per sync operation +- **Efficient Queries**: Database queries optimized for filtering operations +- **Memory Management**: Proper session cleanup and resource management +- **Docker Optimization**: Layer caching and minimal base image for faster builds +- **Background Processing**: Daemon mode prevents blocking CLI operations + +----- + +## **Security Considerations** + +- **Credential Storage**: Environment variables prevent hardcoded credentials +- **File Permissions**: Docker container runs with appropriate user permissions +- **API Rate Limiting**: Respects Garmin Connect rate limits to prevent account restrictions +- **Error Logging**: Sensitive information excluded from logs and error messages + ----- ## **Documentation 📚** @@ -715,13 +394,433 @@ Here are links to the official documentation for the key libraries used: * **Environment Variables:** [python-dotenv](https://github.com/theskumar/python-dotenv) * **Database ORM:** [SQLAlchemy](https://docs.sqlalchemy.org/en/20/) * **Progress Bars:** [tqdm](https://github.com/tqdm/tqdm) +* **Web Framework:** [FastAPI](https://fastapi.tiangolo.com/) +* **Task Scheduler:** [APScheduler](https://apscheduler.readthedocs.io/) ----- -## **Performance Considerations** +## **Web Interface Implementation Steps** -- **Rate Limiting**: 2-second delays between API requests prevent server overload -- **Batch Processing**: Fetches up to 1000 activities per sync operation -- **Efficient Queries**: Database queries optimized for filtering operations -- **Memory Management**: Proper session cleanup and resource management -- **Docker Optimization**: Layer caching and minimal base image for faster builds \ No newline at end of file +### **🎯 Missing Components to Complete** + +#### **1. Enhanced Dashboard Components** + +**A. Real-time Activity Counter** +- **File:** `garminsync/web/templates/dashboard.html` +- **Implementation:** + ```html +
+
+
+

Idle

+

Current Operation

+
+
+
+ ``` +- **JavaScript Update:** Add WebSocket or periodic updates for sync status + +**B. Activity Progress Charts** +- **File:** Add Chart.js to `garminsync/web/static/charts.js` +- **Implementation:** + ```javascript + // Add to dashboard + const ctx = document.getElementById('activityChart').getContext('2d'); + const chart = new Chart(ctx, { + type: 'doughnut', + data: { + labels: ['Downloaded', 'Missing'], + datasets: [{ + data: [downloaded, missing], + backgroundColor: ['#28a745', '#dc3545'] + }] + } + }); + ``` + +#### **2. Enhanced Configuration Page** + +**A. Advanced Schedule Options** +- **File:** `garminsync/web/templates/config.html` +- **Add Preset Schedules:** + ```html +
+ + +
+ ``` + +**B. Notification Settings** +- **New Model in `database.py`:** + ```python + class NotificationConfig(Base): + __tablename__ = 'notification_config' + + id = Column(Integer, primary_key=True) + email_enabled = Column(Boolean, default=False) + email_address = Column(String, nullable=True) + webhook_enabled = Column(Boolean, default=False) + webhook_url = Column(String, nullable=True) + notify_on_success = Column(Boolean, default=True) + notify_on_error = Column(Boolean, default=True) + ``` + +#### **3. Comprehensive Logs Page** + +**A. Create Dedicated Logs Page** +- **File:** `garminsync/web/templates/logs.html` +- **Implementation:** + ```html + {% extends "base.html" %} + + {% block content %} +
+
+

Sync Logs

+
+ + +
+
+ + +
+
Filters
+
+
+
+ +
+
+ +
+
+ +
+
+ +
+
+
+
+ + +
+
Log Entries
+
+
+ + + + + + + + + + + + + +
TimestampOperationStatusMessageActivities
+
+ + + +
+
+
+ {% endblock %} + ``` + +**B. Enhanced Logs API** +- **File:** `garminsync/web/routes.py` +- **Add Filtering and Pagination:** + ```python + @router.get("/logs") + async def get_logs( + limit: int = 50, + offset: int = 0, + status: str = None, + operation: str = None, + date: str = None + ): + """Get logs with filtering and pagination""" + session = get_session() + try: + query = session.query(SyncLog) + + # Apply filters + if status: + query = query.filter(SyncLog.status == status) + if operation: + query = query.filter(SyncLog.operation == operation) + if date: + # Filter by date (assuming ISO format) + query = query.filter(SyncLog.timestamp.like(f"{date}%")) + + # Get total count for pagination + total = query.count() + + # Apply pagination + logs = query.order_by(SyncLog.timestamp.desc()).offset(offset).limit(limit).all() + + return { + "logs": [log_to_dict(log) for log in logs], + "total": total, + "limit": limit, + "offset": offset + } + finally: + session.close() + + def log_to_dict(log): + return { + "id": log.id, + "timestamp": log.timestamp, + "operation": log.operation, + "status": log.status, + "message": log.message, + "activities_processed": log.activities_processed, + "activities_downloaded": log.activities_downloaded + } + ``` + +#### **4. Activity Management Page** + +**A. Create Activities Page** +- **File:** `garminsync/web/templates/activities.html` +- **Features:** + - List all activities with status + - Filter by date range, status, activity type + - Bulk download options + - Individual activity details modal + +**B. Activity Details API** +- **File:** `garminsync/web/routes.py` +- **Implementation:** + ```python + @router.get("/activities") + async def get_activities( + limit: int = 100, + offset: int = 0, + downloaded: bool = None, + start_date: str = None, + end_date: str = None + ): + """Get activities with filtering and pagination""" + session = get_session() + try: + query = session.query(Activity) + + if downloaded is not None: + query = query.filter(Activity.downloaded == downloaded) + if start_date: + query = query.filter(Activity.start_time >= start_date) + if end_date: + query = query.filter(Activity.start_time <= end_date) + + total = query.count() + activities = query.order_by(Activity.start_time.desc()).offset(offset).limit(limit).all() + + return { + "activities": [activity_to_dict(a) for a in activities], + "total": total, + "limit": limit, + "offset": offset + } + finally: + session.close() + + @router.post("/activities/{activity_id}/download") + async def download_single_activity(activity_id: int): + """Download a specific activity""" + # Implementation to download single activity + pass + ``` + +#### **5. System Status Page** + +**A. Create System Status Template** +- **File:** `garminsync/web/templates/system.html` +- **Show:** + - Database statistics + - Disk usage + - Memory usage + - API rate limiting status + - Last errors + +**B. System Status API** +- **File:** `garminsync/web/routes.py` +- **Implementation:** + ```python + @router.get("/system/status") + async def get_system_status(): + """Get comprehensive system status""" + import psutil + import os + from pathlib import Path + + # Database stats + session = get_session() + try: + db_stats = { + "total_activities": session.query(Activity).count(), + "downloaded_activities": session.query(Activity).filter_by(downloaded=True).count(), + "total_logs": session.query(SyncLog).count(), + "database_size": get_database_size() + } + finally: + session.close() + + # System stats + data_dir = Path(os.getenv("DATA_DIR", "data")) + disk_usage = psutil.disk_usage(str(data_dir)) + + return { + "database": db_stats, + "system": { + "cpu_percent": psutil.cpu_percent(), + "memory": psutil.virtual_memory()._asdict(), + "disk_usage": { + "total": disk_usage.total, + "used": disk_usage.used, + "free": disk_usage.free + } + }, + "garmin_api": { + "last_successful_call": get_last_successful_api_call(), + "rate_limit_remaining": get_rate_limit_status() + } + } + ``` + +#### **6. Enhanced Navigation and Layout** + +**A. Update Base Template** +- **File:** `garminsync/web/templates/base.html` +- **Add Complete Navigation:** + ```html + + ``` + +**B. Add FontAwesome Icons** +- **Update base template with:** + ```html + + ``` + +### **🔄 Implementation Order** + +1. **Week 1: Enhanced Dashboard** + - Add real-time counters and charts + - Implement activity progress visualization + - Add sync status indicators + +2. **Week 2: Logs Page** + - Create comprehensive logs template + - Implement filtering and pagination APIs + - Add log management features + +3. **Week 3: Activities Management** + - Build activities listing page + - Add filtering and search capabilities + - Implement individual activity actions + +4. **Week 4: System Status & Configuration** + - Create system monitoring page + - Enhanced configuration options + - Notification system setup + +5. **Week 5: Polish & Testing** + - Improve responsive design + - Add error handling and loading states + - Performance optimization + +### **📁 New Files Needed** + +``` +garminsync/web/ +├── templates/ +│ ├── activities.html # New: Activity management +│ ├── logs.html # New: Enhanced logs page +│ └── system.html # New: System status +├── static/ +│ ├── charts.js # New: Chart.js integration +│ ├── activities.js # New: Activity management JS +│ └── system.js # New: System monitoring JS +``` + +### **🛠️ Required Dependencies** + +Add to `requirements.txt`: +``` +psutil==5.9.6 # For system monitoring +python-dateutil==2.8.2 # For date parsing +``` + +This comprehensive implementation plan will transform the basic web interface into a full-featured dashboard for managing GarminSync operations. + +### **Planned Features** +- **Authentication**: Support for two-factor authentication +- **Selective Sync**: Date range and activity type filtering +- **Export Options**: Support for additional export formats (GPX, TCX) +- **Notification System**: Email/webhook notifications for sync completion +- **Activity Analysis**: Basic statistics and activity summary features +- **Multi-user Support**: Support for multiple Garmin accounts +- **Cloud Storage**: Integration with cloud storage providers +- **Mobile Interface**: Responsive design improvements for mobile devices + +### **Technical Improvements** +- **Health Checks**: Comprehensive health monitoring endpoints +- **Metrics**: Prometheus metrics for monitoring and alerting +- **Database Migrations**: Automatic schema migration support +- **Configuration Validation**: Enhanced validation for cron expressions and settings +- **Logging Enhancement**: Structured logging with configurable levels +- **Test Coverage**: Comprehensive unit and integration tests +- **CI/CD Pipeline**: Automated testing and deployment workflows \ No newline at end of file diff --git a/garminsync/daemon.py b/garminsync/daemon.py index b7b095b..daf7173 100644 --- a/garminsync/daemon.py +++ b/garminsync/daemon.py @@ -23,12 +23,30 @@ class GarminSyncDaemon: # Setup scheduled job if config_data['enabled']: - self.scheduler.add_job( - func=self.sync_and_download, - trigger=CronTrigger.from_crontab(config_data['schedule_cron']), - id='sync_job', - replace_existing=True - ) + cron_str = config_data['schedule_cron'] + try: + # Validate cron string + if not cron_str or len(cron_str.strip().split()) != 5: + logger.error(f"Invalid cron schedule: '{cron_str}'. Using default '0 */6 * * *'") + cron_str = "0 */6 * * *" + + self.scheduler.add_job( + func=self.sync_and_download, + trigger=CronTrigger.from_crontab(cron_str), + id='sync_job', + replace_existing=True + ) + logger.info(f"Scheduled job created with cron: '{cron_str}'") + except Exception as e: + logger.error(f"Failed to create scheduled job: {str(e)}") + # Fallback to default schedule + self.scheduler.add_job( + func=self.sync_and_download, + trigger=CronTrigger.from_crontab("0 */6 * * *"), + id='sync_job', + replace_existing=True + ) + logger.info("Using default schedule '0 */6 * * *'") # Start scheduler self.scheduler.start() @@ -123,8 +141,12 @@ class GarminSyncDaemon: try: config = session.query(DaemonConfig).first() if not config: - # Create default configuration - config = DaemonConfig() + # Create default configuration with explicit cron schedule + config = DaemonConfig( + schedule_cron="0 */6 * * *", + enabled=True, + status="stopped" + ) session.add(config) session.commit() session.refresh(config) # Ensure we have the latest data @@ -223,4 +245,4 @@ class GarminSyncDaemon: try: return session.query(Activity).filter_by(downloaded=False).count() finally: - session.close() \ No newline at end of file + session.close() diff --git a/garminsync/web/routes.py b/garminsync/web/routes.py index 2c3b3ff..1d405ef 100644 --- a/garminsync/web/routes.py +++ b/garminsync/web/routes.py @@ -126,11 +126,35 @@ async def get_activity_stats(): return get_offline_stats() @router.get("/logs") -async def get_logs(limit: int = 50): - """Get recent sync logs""" +async def get_logs( + status: str = None, + operation: str = None, + date: str = None, + page: int = 1, + per_page: int = 20 +): + """Get sync logs with filtering and pagination""" session = get_session() try: - logs = session.query(SyncLog).order_by(SyncLog.timestamp.desc()).limit(limit).all() + query = session.query(SyncLog) + + # Apply filters + if status: + query = query.filter(SyncLog.status == status) + if operation: + query = query.filter(SyncLog.operation == operation) + if date: + # Filter by date (assuming ISO format) + query = query.filter(SyncLog.timestamp.like(f"{date}%")) + + # Get total count for pagination + total = query.count() + + # Apply pagination + logs = query.order_by(SyncLog.timestamp.desc()) \ + .offset((page - 1) * per_page) \ + .limit(per_page) \ + .all() log_data = [] for log in logs: @@ -144,7 +168,12 @@ async def get_logs(limit: int = 50): "activities_downloaded": log.activities_downloaded }) - return {"logs": log_data} + return { + "logs": log_data, + "total": total, + "page": page, + "per_page": per_page + } finally: session.close() diff --git a/garminsync/web/static/app.js b/garminsync/web/static/app.js index 9103358..7b7a6a8 100644 --- a/garminsync/web/static/app.js +++ b/garminsync/web/static/app.js @@ -6,11 +6,17 @@ async function updateStatus() { const response = await fetch('/api/status'); const data = await response.json(); + // Update sync status + const syncStatus = document.getElementById('sync-status'); + const statusBadge = data.daemon.running ? + 'Running' : + 'Stopped'; + + syncStatus.innerHTML = `${statusBadge}`; + // Update daemon status document.getElementById('daemon-status').innerHTML = ` -

Status: - ${data.daemon.running ? 'Running' : 'Stopped'} -

+

Status: ${statusBadge}

Last Run: ${data.daemon.last_run || 'Never'}

Next Run: ${data.daemon.next_run || 'Not scheduled'}

Schedule: ${data.daemon.schedule || 'Not configured'}

diff --git a/garminsync/web/static/charts.js b/garminsync/web/static/charts.js new file mode 100644 index 0000000..97ae710 --- /dev/null +++ b/garminsync/web/static/charts.js @@ -0,0 +1,37 @@ +// Initialize the activity progress chart +document.addEventListener('DOMContentLoaded', function() { + // Fetch activity stats from the API + fetch('/api/activities/stats') + .then(response => response.json()) + .then(data => { + // Create doughnut chart + const ctx = document.getElementById('activityChart').getContext('2d'); + const chart = new Chart(ctx, { + type: 'doughnut', + data: { + labels: ['Downloaded', 'Missing'], + datasets: [{ + data: [data.downloaded, data.missing], + backgroundColor: ['#28a745', '#dc3545'], + borderWidth: 1 + }] + }, + options: { + responsive: true, + maintainAspectRatio: false, + plugins: { + legend: { + position: 'top', + }, + title: { + display: true, + text: 'Activity Status' + } + } + } + }); + }) + .catch(error => { + console.error('Error fetching activity stats:', error); + }); +}); diff --git a/garminsync/web/static/logs.js b/garminsync/web/static/logs.js new file mode 100644 index 0000000..f11a5f1 --- /dev/null +++ b/garminsync/web/static/logs.js @@ -0,0 +1,121 @@ +// Global variables for pagination and filtering +let currentPage = 1; +const logsPerPage = 20; +let totalLogs = 0; +let currentFilters = {}; + +// Initialize logs page +document.addEventListener('DOMContentLoaded', function() { + loadLogs(); +}); + +async function loadLogs() { + try { + // Build query string from filters + const params = new URLSearchParams({ + page: currentPage, + perPage: logsPerPage, + ...currentFilters + }).toString(); + + const response = await fetch(`/api/logs?${params}`); + if (!response.ok) { + throw new Error('Failed to fetch logs'); + } + + const data = await response.json(); + totalLogs = data.total; + renderLogs(data.logs); + renderPagination(); + } catch (error) { + console.error('Error loading logs:', error); + alert('Failed to load logs: ' + error.message); + } +} + +function renderLogs(logs) { + const tbody = document.getElementById('logs-tbody'); + tbody.innerHTML = ''; + + logs.forEach(log => { + const row = document.createElement('tr'); + + row.innerHTML = ` + ${log.timestamp} + ${log.operation} + ${log.status} + ${log.message || ''} + ${log.activities_processed} + ${log.activities_downloaded} + `; + + tbody.appendChild(row); + }); +} + +function renderPagination() { + const totalPages = Math.ceil(totalLogs / logsPerPage); + const pagination = document.getElementById('pagination'); + pagination.innerHTML = ''; + + // Previous button + const prevLi = document.createElement('li'); + prevLi.className = `page-item ${currentPage === 1 ? 'disabled' : ''}`; + prevLi.innerHTML = `Previous`; + pagination.appendChild(prevLi); + + // Page numbers + for (let i = 1; i <= totalPages; i++) { + const li = document.createElement('li'); + li.className = `page-item ${i === currentPage ? 'active' : ''}`; + li.innerHTML = `${i}`; + pagination.appendChild(li); + } + + // Next button + const nextLi = document.createElement('li'); + nextLi.className = `page-item ${currentPage === totalPages ? 'disabled' : ''}`; + nextLi.innerHTML = `Next`; + pagination.appendChild(nextLi); +} + +function changePage(page) { + if (page < 1 || page > Math.ceil(totalLogs / logsPerPage)) return; + currentPage = page; + loadLogs(); +} + +function refreshLogs() { + currentPage = 1; + loadLogs(); +} + +function applyFilters() { + currentFilters = { + status: document.getElementById('status-filter').value, + operation: document.getElementById('operation-filter').value, + date: document.getElementById('date-filter').value + }; + + currentPage = 1; + loadLogs(); +} + +async function clearLogs() { + if (!confirm('Are you sure you want to clear all logs? This cannot be undone.')) return; + + try { + const response = await fetch('/api/logs', { method: 'DELETE' }); + if (response.ok) { + alert('Logs cleared successfully'); + refreshLogs(); + } else { + throw new Error('Failed to clear logs'); + } + } catch (error) { + console.error('Error clearing logs:', error); + alert('Failed to clear logs: ' + error.message); + } +} diff --git a/garminsync/web/templates/base.html b/garminsync/web/templates/base.html index e91a55c..2a34e5a 100644 --- a/garminsync/web/templates/base.html +++ b/garminsync/web/templates/base.html @@ -6,6 +6,8 @@ GarminSync Dashboard + +