mirror of
https://github.com/sstent/GarminSync.git
synced 2026-01-25 08:35:02 +00:00
python v2 - added feartures 1 and 3 - no errors2
This commit is contained in:
727
Design.md
Normal file
727
Design.md
Normal file
@@ -0,0 +1,727 @@
|
||||
# **GarminSync Application Design (Python Version)**
|
||||
|
||||
## **Basic Info**
|
||||
|
||||
**App Name:** GarminSync
|
||||
**What it does:** A CLI application that downloads `.fit` files for every activity in Garmin Connect.
|
||||
|
||||
-----
|
||||
|
||||
## **Core Features**
|
||||
|
||||
### **CLI Mode (Current)**
|
||||
1. List all activities (`garminsync list --all`)
|
||||
2. List activities that have not been downloaded (`garminsync list --missing`)
|
||||
3. List activities that have been downloaded (`garminsync list --downloaded`)
|
||||
4. Download all missing activities (`garminsync download --missing`)
|
||||
|
||||
### **Enhanced Features (New)**
|
||||
5. **Offline Mode**: List activities without polling Garmin Connect (`garminsync list --missing --offline`)
|
||||
6. **Daemon Mode**: Run as background service with scheduled downloads (`garminsync daemon --start`)
|
||||
7. **Web UI**: Browser-based interface for daemon monitoring and configuration (`http://localhost:8080`)
|
||||
|
||||
-----
|
||||
|
||||
## **Tech Stack 🐍**
|
||||
|
||||
* **Frontend:** CLI (**Python**)
|
||||
* **Backend:** **Python**
|
||||
* **Database:** SQLite (`garmin.db`)
|
||||
* **Hosting:** Docker container
|
||||
* **Key Libraries:**
|
||||
* **`python-garminconnect`**: The library for Garmin Connect API communication.
|
||||
* **`typer`**: A modern and easy-to-use CLI framework (built on `click`).
|
||||
* **`python-dotenv`**: For loading credentials from a `.env` file.
|
||||
* **`sqlalchemy`**: A robust ORM for database interaction and schema management.
|
||||
* **`tqdm`**: For creating user-friendly progress bars.
|
||||
* **`fastapi`**: Modern web framework for the daemon web UI.
|
||||
* **`uvicorn`**: ASGI server for running the FastAPI web interface.
|
||||
* **`apscheduler`**: Advanced Python Scheduler for daemon mode scheduling.
|
||||
* **`pydantic`**: Data validation and settings management for configuration.
|
||||
* **`jinja2`**: Template engine for web UI rendering.
|
||||
|
||||
-----
|
||||
|
||||
## **Data Structure**
|
||||
|
||||
The application uses SQLAlchemy ORM with expanded models for daemon functionality:
|
||||
|
||||
**SQLAlchemy Models (`database.py`):**
|
||||
|
||||
```python
|
||||
class Activity(Base):
|
||||
__tablename__ = 'activities'
|
||||
|
||||
activity_id = Column(Integer, primary_key=True)
|
||||
start_time = Column(String, nullable=False)
|
||||
filename = Column(String, unique=True, nullable=True)
|
||||
downloaded = Column(Boolean, default=False, nullable=False)
|
||||
created_at = Column(String, nullable=False) # When record was added
|
||||
last_sync = Column(String, nullable=True) # Last successful sync
|
||||
|
||||
class DaemonConfig(Base):
|
||||
__tablename__ = 'daemon_config'
|
||||
|
||||
id = Column(Integer, primary_key=True, default=1)
|
||||
enabled = Column(Boolean, default=True, nullable=False)
|
||||
schedule_cron = Column(String, default="0 */6 * * *", nullable=False) # Every 6 hours
|
||||
last_run = Column(String, nullable=True)
|
||||
next_run = Column(String, nullable=True)
|
||||
status = Column(String, default="stopped", nullable=False) # stopped, running, error
|
||||
|
||||
class SyncLog(Base):
|
||||
__tablename__ = 'sync_logs'
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
timestamp = Column(String, nullable=False)
|
||||
operation = Column(String, nullable=False) # sync, download, daemon_start, daemon_stop
|
||||
status = Column(String, nullable=False) # success, error, partial
|
||||
message = Column(String, nullable=True)
|
||||
activities_processed = Column(Integer, default=0, nullable=False)
|
||||
activities_downloaded = Column(Integer, default=0, nullable=False)
|
||||
```
|
||||
|
||||
-----
|
||||
|
||||
## **User Flow**
|
||||
|
||||
### **CLI Mode (Existing)**
|
||||
1. User sets up credentials in `.env` file with `GARMIN_EMAIL` and `GARMIN_PASSWORD`
|
||||
2. User launches the container: `docker run -it --env-file .env -v $(pwd)/data:/app/data garminsync`
|
||||
3. User runs commands like `garminsync download --missing`
|
||||
4. Application syncs with Garmin Connect, shows progress bars, and downloads activities
|
||||
|
||||
### **Offline Mode (New)**
|
||||
1. User runs `garminsync list --missing --offline` to view cached data without API calls
|
||||
2. Application queries local database only, showing last known state
|
||||
3. Useful for checking status without network connectivity or API rate limits
|
||||
|
||||
### **Daemon Mode (New)**
|
||||
1. User starts daemon: `garminsync daemon --start`
|
||||
2. Daemon runs in background, scheduling automatic sync/download operations
|
||||
3. User accesses web UI at `http://localhost:8080` for monitoring and configuration
|
||||
4. Web UI provides real-time status, logs, and schedule management
|
||||
5. Daemon can be stopped with `garminsync daemon --stop` or through web UI
|
||||
|
||||
-----
|
||||
|
||||
## **File Structure**
|
||||
|
||||
```
|
||||
/garminsync
|
||||
├── garminsync/ # Main application package
|
||||
│ ├── __init__.py # Empty package file
|
||||
│ ├── cli.py # Typer CLI commands and main entrypoint
|
||||
│ ├── config.py # Configuration and environment variable loading
|
||||
│ ├── database.py # SQLAlchemy models and database operations
|
||||
│ ├── garmin.py # Garmin Connect client wrapper with robust download logic
|
||||
│ ├── daemon.py # Daemon mode implementation with APScheduler
|
||||
│ ├── web/ # Web UI components
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── app.py # FastAPI application setup
|
||||
│ │ ├── routes.py # API endpoints for web UI
|
||||
│ │ ├── static/ # CSS, JavaScript, images
|
||||
│ │ │ ├── style.css
|
||||
│ │ │ └── app.js
|
||||
│ │ └── templates/ # Jinja2 HTML templates
|
||||
│ │ ├── base.html
|
||||
│ │ ├── dashboard.html
|
||||
│ │ └── config.html
|
||||
│ └── utils.py # Shared utilities and helpers
|
||||
├── data/ # Directory for downloaded .fit files and SQLite DB
|
||||
├── .env # Stores GARMIN_EMAIL/GARMIN_PASSWORD (gitignored)
|
||||
├── .gitignore # Excludes .env file
|
||||
├── Dockerfile # Production-ready container configuration
|
||||
├── Design.md # This design document
|
||||
└── requirements.txt # Pinned Python dependencies (updated)
|
||||
```
|
||||
|
||||
-----
|
||||
|
||||
## **Technical Implementation Details**
|
||||
|
||||
### **Architecture**
|
||||
- **CLI Framework**: Uses Typer with proper type hints and validation
|
||||
- **Module Separation**: Clear separation between CLI commands, database operations, and Garmin API interactions
|
||||
- **Error Handling**: Comprehensive exception handling with user-friendly error messages
|
||||
- **Session Management**: Proper SQLAlchemy session management with cleanup
|
||||
|
||||
### **Authentication & Configuration**
|
||||
- Credentials loaded via `python-dotenv` from environment variables
|
||||
- Configuration validation ensures required credentials are present
|
||||
- Garmin client handles authentication automatically with session persistence
|
||||
|
||||
### **Database Operations**
|
||||
- SQLite database with SQLAlchemy ORM for type safety
|
||||
- Database initialization creates tables automatically
|
||||
- Sync functionality reconciles local database with Garmin Connect activities
|
||||
- Proper transaction management with rollback on errors
|
||||
|
||||
### **File Management**
|
||||
- Files named with pattern: `activity_{activity_id}_{timestamp}.fit`
|
||||
- Timestamp sanitized for filesystem compatibility (colons and spaces replaced)
|
||||
- Downloads saved to configurable data directory
|
||||
- Database tracks both download status and file paths
|
||||
|
||||
### **API Integration**
|
||||
- **Rate Limiting**: 2-second delays between API requests to respect Garmin's servers
|
||||
- **Robust Downloads**: Multiple fallback methods for downloading FIT files:
|
||||
1. Default download method
|
||||
2. Explicit 'fit' format parameter
|
||||
3. Alternative parameter names and formats
|
||||
4. Graceful fallback with detailed error reporting
|
||||
- **Activity Fetching**: Configurable batch sizes (currently 1000 activities per sync)
|
||||
|
||||
### **User Experience**
|
||||
- **Progress Indicators**: tqdm progress bars for all long-running operations
|
||||
- **Informative Output**: Clear status messages and operation summaries
|
||||
- **Input Validation**: Prevents invalid command combinations
|
||||
- **Exit Codes**: Proper exit codes for script integration
|
||||
|
||||
-----
|
||||
|
||||
## **Development Status ✅**
|
||||
|
||||
### **✅ Completed Features**
|
||||
|
||||
#### **Phase 1: Core Infrastructure**
|
||||
- [x] **Dockerfile**: Production-ready Python 3.10 container with proper layer caching
|
||||
- [x] **Environment Configuration**: `python-dotenv` integration with validation
|
||||
- [x] **CLI Framework**: Complete Typer implementation with type hints and help text
|
||||
- [x] **Garmin Integration**: Robust `python-garminconnect` wrapper with authentication
|
||||
|
||||
#### **Phase 2: Activity Listing**
|
||||
- [x] **Database Schema**: SQLAlchemy models with proper relationships
|
||||
- [x] **Database Operations**: Session management, initialization, and sync functionality
|
||||
- [x] **List Commands**: All filter options (`--all`, `--missing`, `--downloaded`) implemented
|
||||
- [x] **Progress Display**: tqdm integration for user feedback during operations
|
||||
|
||||
#### **Phase 3: Download Pipeline**
|
||||
- [x] **FIT File Downloads**: Multi-method download approach with fallback strategies
|
||||
- [x] **Idempotent Operations**: Prevents re-downloading existing files
|
||||
- [x] **Database Updates**: Proper status tracking and file path storage
|
||||
- [x] **File Management**: Safe filename generation and directory creation
|
||||
|
||||
#### **Phase 4: Polish**
|
||||
- [x] **Progress Bars**: Comprehensive tqdm implementation across all operations
|
||||
- [x] **Error Handling**: Graceful error handling with informative messages
|
||||
- [x] **Container Optimization**: Efficient Docker build with proper dependency management
|
||||
|
||||
### **🚧 New Features Implementation Guide**
|
||||
|
||||
#### **Feature 1: Offline Mode**
|
||||
|
||||
**Implementation Steps:**
|
||||
1. **CLI Enhancement** (`cli.py`):
|
||||
```python
|
||||
@app.command("list")
|
||||
def list_activities(
|
||||
all_activities: bool = False,
|
||||
missing: bool = False,
|
||||
downloaded: bool = False,
|
||||
offline: Annotated[bool, typer.Option("--offline", help="Work offline without syncing")] = False
|
||||
):
|
||||
if not offline:
|
||||
# Existing sync logic
|
||||
sync_database(client)
|
||||
else:
|
||||
typer.echo("Working in offline mode - using cached data")
|
||||
|
||||
# Rest of listing logic remains the same
|
||||
```
|
||||
|
||||
2. **Database Enhancements** (`database.py`):
|
||||
- Add `last_sync` column to Activity table
|
||||
- Add utility functions for offline status checking
|
||||
```python
|
||||
def get_offline_stats():
|
||||
"""Return statistics about cached data without API calls"""
|
||||
session = get_session()
|
||||
total = session.query(Activity).count()
|
||||
downloaded = session.query(Activity).filter_by(downloaded=True).count()
|
||||
missing = total - downloaded
|
||||
last_sync = session.query(Activity).order_by(Activity.last_sync.desc()).first()
|
||||
return {
|
||||
'total': total,
|
||||
'downloaded': downloaded,
|
||||
'missing': missing,
|
||||
'last_sync': last_sync.last_sync if last_sync else 'Never'
|
||||
}
|
||||
```
|
||||
|
||||
#### **Feature 2: Daemon Mode**
|
||||
|
||||
**Implementation Steps:**
|
||||
1. **New Daemon Module** (`daemon.py`):
|
||||
```python
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
from apscheduler.triggers.cron import CronTrigger
|
||||
import signal
|
||||
import sys
|
||||
import time
|
||||
import threading
|
||||
from datetime import datetime
|
||||
|
||||
class GarminSyncDaemon:
|
||||
def __init__(self):
|
||||
self.scheduler = BackgroundScheduler()
|
||||
self.running = False
|
||||
self.web_server = None
|
||||
|
||||
def start(self, web_port=8080):
|
||||
"""Start daemon with scheduler and web UI"""
|
||||
# Load configuration from database
|
||||
config = self.load_config()
|
||||
|
||||
# Setup scheduled job
|
||||
if config.enabled:
|
||||
self.scheduler.add_job(
|
||||
func=self.sync_and_download,
|
||||
trigger=CronTrigger.from_crontab(config.schedule_cron),
|
||||
id='sync_job',
|
||||
replace_existing=True
|
||||
)
|
||||
|
||||
# Start scheduler
|
||||
self.scheduler.start()
|
||||
self.running = True
|
||||
|
||||
# Start web UI in separate thread
|
||||
self.start_web_ui(web_port)
|
||||
|
||||
# Setup signal handlers for graceful shutdown
|
||||
signal.signal(signal.SIGINT, self.signal_handler)
|
||||
signal.signal(signal.SIGTERM, self.signal_handler)
|
||||
|
||||
print(f"Daemon started. Web UI available at http://localhost:{web_port}")
|
||||
|
||||
# Keep daemon running
|
||||
try:
|
||||
while self.running:
|
||||
time.sleep(1)
|
||||
except KeyboardInterrupt:
|
||||
self.stop()
|
||||
|
||||
def sync_and_download(self):
|
||||
"""Scheduled job function"""
|
||||
try:
|
||||
self.log_operation("sync", "started")
|
||||
|
||||
# Perform sync and download
|
||||
from .garmin import GarminClient
|
||||
from .database import sync_database
|
||||
|
||||
client = GarminClient()
|
||||
activities_before = self.count_missing()
|
||||
|
||||
sync_database(client)
|
||||
|
||||
# Download missing activities
|
||||
downloaded_count = self.download_missing_activities(client)
|
||||
|
||||
self.log_operation("sync", "success",
|
||||
f"Downloaded {downloaded_count} new activities")
|
||||
|
||||
except Exception as e:
|
||||
self.log_operation("sync", "error", str(e))
|
||||
|
||||
def load_config(self):
|
||||
"""Load daemon configuration from database"""
|
||||
session = get_session()
|
||||
config = session.query(DaemonConfig).first()
|
||||
if not config:
|
||||
# Create default configuration
|
||||
config = DaemonConfig()
|
||||
session.add(config)
|
||||
session.commit()
|
||||
session.close()
|
||||
return config
|
||||
```
|
||||
|
||||
2. **CLI Integration** (`cli.py`):
|
||||
```python
|
||||
@app.command("daemon")
|
||||
def daemon_mode(
|
||||
start: Annotated[bool, typer.Option("--start", help="Start daemon")] = False,
|
||||
stop: Annotated[bool, typer.Option("--stop", help="Stop daemon")] = False,
|
||||
status: Annotated[bool, typer.Option("--status", help="Show daemon status")] = False,
|
||||
port: Annotated[int, typer.Option("--port", help="Web UI port")] = 8080
|
||||
):
|
||||
"""Daemon mode operations"""
|
||||
from .daemon import GarminSyncDaemon
|
||||
|
||||
if start:
|
||||
daemon = GarminSyncDaemon()
|
||||
daemon.start(web_port=port)
|
||||
elif stop:
|
||||
# Implementation for stopping daemon (PID file or signal)
|
||||
pass
|
||||
elif status:
|
||||
# Show current daemon status
|
||||
pass
|
||||
```
|
||||
|
||||
#### **Feature 3: Web UI**
|
||||
|
||||
**Implementation Steps:**
|
||||
1. **FastAPI Application** (`web/app.py`):
|
||||
```python
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from fastapi.templating import Jinja2Templates
|
||||
from .routes import router
|
||||
|
||||
app = FastAPI(title="GarminSync Dashboard")
|
||||
|
||||
# Mount static files and templates
|
||||
app.mount("/static", StaticFiles(directory="garminsync/web/static"), name="static")
|
||||
templates = Jinja2Templates(directory="garminsync/web/templates")
|
||||
|
||||
# Include API routes
|
||||
app.include_router(router)
|
||||
|
||||
@app.get("/")
|
||||
async def dashboard(request: Request):
|
||||
# Get current statistics
|
||||
from ..database import get_offline_stats
|
||||
stats = get_offline_stats()
|
||||
|
||||
return templates.TemplateResponse("dashboard.html", {
|
||||
"request": request,
|
||||
"stats": stats
|
||||
})
|
||||
```
|
||||
|
||||
2. **API Routes** (`web/routes.py`):
|
||||
```python
|
||||
from fastapi import APIRouter, HTTPException
|
||||
from pydantic import BaseModel
|
||||
from ..database import get_session, DaemonConfig, SyncLog
|
||||
|
||||
router = APIRouter(prefix="/api")
|
||||
|
||||
class ScheduleConfig(BaseModel):
|
||||
enabled: bool
|
||||
cron_schedule: str
|
||||
|
||||
@router.get("/status")
|
||||
async def get_status():
|
||||
"""Get current daemon status"""
|
||||
session = get_session()
|
||||
config = session.query(DaemonConfig).first()
|
||||
|
||||
# Get recent logs
|
||||
logs = session.query(SyncLog).order_by(SyncLog.timestamp.desc()).limit(10).all()
|
||||
|
||||
return {
|
||||
"daemon": {
|
||||
"running": config.status == "running" if config else False,
|
||||
"next_run": config.next_run if config else None,
|
||||
"schedule": config.schedule_cron if config else None
|
||||
},
|
||||
"recent_logs": [
|
||||
{
|
||||
"timestamp": log.timestamp,
|
||||
"operation": log.operation,
|
||||
"status": log.status,
|
||||
"message": log.message
|
||||
} for log in logs
|
||||
]
|
||||
}
|
||||
|
||||
@router.post("/schedule")
|
||||
async def update_schedule(config: ScheduleConfig):
|
||||
"""Update daemon schedule configuration"""
|
||||
session = get_session()
|
||||
daemon_config = session.query(DaemonConfig).first()
|
||||
|
||||
if not daemon_config:
|
||||
daemon_config = DaemonConfig()
|
||||
session.add(daemon_config)
|
||||
|
||||
daemon_config.enabled = config.enabled
|
||||
daemon_config.schedule_cron = config.cron_schedule
|
||||
session.commit()
|
||||
|
||||
return {"message": "Configuration updated successfully"}
|
||||
|
||||
@router.post("/sync/trigger")
|
||||
async def trigger_sync():
|
||||
"""Manually trigger a sync operation"""
|
||||
# Implementation to trigger immediate sync
|
||||
pass
|
||||
```
|
||||
|
||||
3. **HTML Templates** (`web/templates/dashboard.html`):
|
||||
```html
|
||||
{% extends "base.html" %}
|
||||
|
||||
{% block content %}
|
||||
<div class="container">
|
||||
<h1>GarminSync Dashboard</h1>
|
||||
|
||||
<div class="row">
|
||||
<div class="col-md-4">
|
||||
<div class="card">
|
||||
<div class="card-header">Statistics</div>
|
||||
<div class="card-body">
|
||||
<p>Total Activities: {{ stats.total }}</p>
|
||||
<p>Downloaded: {{ stats.downloaded }}</p>
|
||||
<p>Missing: {{ stats.missing }}</p>
|
||||
<p>Last Sync: {{ stats.last_sync }}</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="col-md-4">
|
||||
<div class="card">
|
||||
<div class="card-header">Daemon Status</div>
|
||||
<div class="card-body" id="daemon-status">
|
||||
<!-- Populated by JavaScript -->
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="col-md-4">
|
||||
<div class="card">
|
||||
<div class="card-header">Quick Actions</div>
|
||||
<div class="card-body">
|
||||
<button class="btn btn-primary" onclick="triggerSync()">
|
||||
Sync Now
|
||||
</button>
|
||||
<button class="btn btn-secondary" onclick="toggleDaemon()">
|
||||
Toggle Daemon
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row mt-4">
|
||||
<div class="col-12">
|
||||
<div class="card">
|
||||
<div class="card-header">Recent Activity</div>
|
||||
<div class="card-body" id="recent-logs">
|
||||
<!-- Populated by JavaScript -->
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row mt-4">
|
||||
<div class="col-12">
|
||||
<div class="card">
|
||||
<div class="card-header">Schedule Configuration</div>
|
||||
<div class="card-body">
|
||||
<form id="schedule-form">
|
||||
<div class="form-group">
|
||||
<label for="schedule-enabled">Enable Scheduled Sync</label>
|
||||
<input type="checkbox" id="schedule-enabled">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="cron-schedule">Cron Schedule</label>
|
||||
<input type="text" class="form-control" id="cron-schedule"
|
||||
placeholder="0 */6 * * *" title="Every 6 hours">
|
||||
</div>
|
||||
<button type="submit" class="btn btn-primary">
|
||||
Update Schedule
|
||||
</button>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
```
|
||||
|
||||
4. **JavaScript for Interactivity** (`web/static/app.js`):
|
||||
```javascript
|
||||
// Auto-refresh dashboard data
|
||||
setInterval(updateStatus, 30000); // Every 30 seconds
|
||||
|
||||
async function updateStatus() {
|
||||
try {
|
||||
const response = await fetch('/api/status');
|
||||
const data = await response.json();
|
||||
|
||||
// Update daemon status
|
||||
document.getElementById('daemon-status').innerHTML = `
|
||||
<p>Status: <span class="badge ${data.daemon.running ? 'badge-success' : 'badge-danger'}">
|
||||
${data.daemon.running ? 'Running' : 'Stopped'}
|
||||
</span></p>
|
||||
<p>Next Run: ${data.daemon.next_run || 'Not scheduled'}</p>
|
||||
<p>Schedule: ${data.daemon.schedule || 'Not configured'}</p>
|
||||
`;
|
||||
|
||||
// Update recent logs
|
||||
const logsHtml = data.recent_logs.map(log => `
|
||||
<div class="log-entry">
|
||||
<small class="text-muted">${log.timestamp}</small>
|
||||
<span class="badge badge-${log.status === 'success' ? 'success' : 'danger'}">
|
||||
${log.status}
|
||||
</span>
|
||||
${log.operation}: ${log.message || ''}
|
||||
</div>
|
||||
`).join('');
|
||||
|
||||
document.getElementById('recent-logs').innerHTML = logsHtml;
|
||||
|
||||
} catch (error) {
|
||||
console.error('Failed to update status:', error);
|
||||
}
|
||||
}
|
||||
|
||||
async function triggerSync() {
|
||||
try {
|
||||
await fetch('/api/sync/trigger', { method: 'POST' });
|
||||
alert('Sync triggered successfully');
|
||||
updateStatus();
|
||||
} catch (error) {
|
||||
alert('Failed to trigger sync');
|
||||
}
|
||||
}
|
||||
|
||||
// Initialize on page load
|
||||
document.addEventListener('DOMContentLoaded', updateStatus);
|
||||
```
|
||||
|
||||
### **Updated Requirements** (`requirements.txt`):
|
||||
```
|
||||
typer==0.9.0
|
||||
click==8.1.7
|
||||
python-dotenv==1.0.0
|
||||
garminconnect==0.2.28
|
||||
sqlalchemy==2.0.23
|
||||
tqdm==4.66.1
|
||||
fastapi==0.104.1
|
||||
uvicorn[standard]==0.24.0
|
||||
apscheduler==3.10.4
|
||||
pydantic==2.5.0
|
||||
jinja2==3.1.2
|
||||
python-multipart==0.0.6
|
||||
aiofiles==23.2.1
|
||||
```
|
||||
|
||||
### **Docker Updates**:
|
||||
```dockerfile
|
||||
# Expose web UI port
|
||||
EXPOSE 8080
|
||||
|
||||
# Update entrypoint to support daemon mode
|
||||
ENTRYPOINT ["python", "-m", "garminsync.cli"]
|
||||
CMD ["--help"]
|
||||
```
|
||||
|
||||
### **Usage Examples**:
|
||||
|
||||
**Offline Mode:**
|
||||
```bash
|
||||
# List missing activities without network calls
|
||||
docker run --env-file .env -v $(pwd)/data:/app/data garminsync list --missing --offline
|
||||
```
|
||||
|
||||
**Daemon Mode:**
|
||||
```bash
|
||||
# Start daemon with web UI on port 8080
|
||||
docker run -d --env-file .env -v $(pwd)/data:/app/data -p 8080:8080 garminsync daemon --start
|
||||
|
||||
# Access web UI at http://localhost:8080
|
||||
```
|
||||
|
||||
-----
|
||||
|
||||
## **Docker Usage**
|
||||
|
||||
### **Build the Container**
|
||||
```bash
|
||||
docker build -t garminsync .
|
||||
```
|
||||
|
||||
### **Run with Environment File**
|
||||
```bash
|
||||
docker run -it --env-file .env -v $(pwd)/data:/app/data garminsync --help
|
||||
```
|
||||
|
||||
### **Example Commands**
|
||||
```bash
|
||||
# List all activities
|
||||
docker run -it --env-file .env -v $(pwd)/data:/app/data garminsync list --all
|
||||
|
||||
# Download missing activities
|
||||
docker run -it --env-file .env -v $(pwd)/data:/app/data garminsync download --missing
|
||||
```
|
||||
|
||||
-----
|
||||
|
||||
## **Environment Setup**
|
||||
|
||||
Create a `.env` file in the project root:
|
||||
```
|
||||
GARMIN_EMAIL=your_email@example.com
|
||||
GARMIN_PASSWORD=your_password
|
||||
```
|
||||
|
||||
-----
|
||||
|
||||
## **Key Implementation Highlights**
|
||||
|
||||
### **Robust Download Logic**
|
||||
The `garmin.py` module implements a sophisticated download strategy that tries multiple methods to handle variations in the Garmin Connect API:
|
||||
|
||||
```python
|
||||
methods_to_try = [
|
||||
lambda: self.client.download_activity(activity_id),
|
||||
lambda: self.client.download_activity(activity_id, fmt='fit'),
|
||||
lambda: self.client.download_activity(activity_id, format='fit'),
|
||||
# ... additional fallback methods
|
||||
]
|
||||
```
|
||||
|
||||
### **Database Synchronization**
|
||||
The sync process efficiently updates the local database with new activities from Garmin Connect:
|
||||
|
||||
```python
|
||||
def sync_database(garmin_client):
|
||||
"""Sync local database with Garmin Connect activities"""
|
||||
activities = garmin_client.get_activities(0, 1000)
|
||||
for activity in activities:
|
||||
# Only add new activities, preserve existing download status
|
||||
existing = session.query(Activity).filter_by(activity_id=activity_id).first()
|
||||
if not existing:
|
||||
new_activity = Activity(...)
|
||||
session.add(new_activity)
|
||||
```
|
||||
|
||||
### **CLI Integration**
|
||||
Clean separation between CLI interface and business logic with proper type annotations:
|
||||
|
||||
```python
|
||||
def list_activities(
|
||||
all_activities: Annotated[bool, typer.Option("--all", help="List all activities")] = False,
|
||||
missing: Annotated[bool, typer.Option("--missing", help="List missing activities")] = False,
|
||||
downloaded: Annotated[bool, typer.Option("--downloaded", help="List downloaded activities")] = False
|
||||
):
|
||||
```
|
||||
|
||||
-----
|
||||
|
||||
## **Documentation 📚**
|
||||
|
||||
Here are links to the official documentation for the key libraries used:
|
||||
|
||||
* **Garmin API:** [python-garminconnect](https://github.com/cyberjunky/python-garminconnect)
|
||||
* **CLI Framework:** [Typer](https://typer.tiangolo.com/)
|
||||
* **Environment Variables:** [python-dotenv](https://github.com/theskumar/python-dotenv)
|
||||
* **Database ORM:** [SQLAlchemy](https://docs.sqlalchemy.org/en/20/)
|
||||
* **Progress Bars:** [tqdm](https://github.com/tqdm/tqdm)
|
||||
|
||||
-----
|
||||
|
||||
## **Performance Considerations**
|
||||
|
||||
- **Rate Limiting**: 2-second delays between API requests prevent server overload
|
||||
- **Batch Processing**: Fetches up to 1000 activities per sync operation
|
||||
- **Efficient Queries**: Database queries optimized for filtering operations
|
||||
- **Memory Management**: Proper session cleanup and resource management
|
||||
- **Docker Optimization**: Layer caching and minimal base image for faster builds
|
||||
36
Dockerfile
Normal file
36
Dockerfile
Normal file
@@ -0,0 +1,36 @@
|
||||
# Use official Python base image
|
||||
FROM python:3.10-slim
|
||||
|
||||
# Set environment variables
|
||||
ENV PYTHONDONTWRITEBYTECODE=1
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
|
||||
# Set work directory
|
||||
WORKDIR /app
|
||||
|
||||
# Install system dependencies
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
build-essential \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Copy and install Python dependencies
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir --upgrade pip && \
|
||||
pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
# Copy application code
|
||||
COPY garminsync/ ./garminsync/
|
||||
|
||||
# Create data directory
|
||||
RUN mkdir -p /app/data
|
||||
|
||||
# Set environment variables from .env file
|
||||
ENV ENV_FILE=/app/.env
|
||||
ENV DATA_DIR=/app/data
|
||||
|
||||
# Expose web UI port
|
||||
EXPOSE 8080
|
||||
|
||||
# Update entrypoint to support daemon mode
|
||||
ENTRYPOINT ["python", "-m", "garminsync.cli"]
|
||||
CMD ["--help"]
|
||||
0
garminsync/__init__.py
Normal file
0
garminsync/__init__.py
Normal file
166
garminsync/cli.py
Normal file
166
garminsync/cli.py
Normal file
@@ -0,0 +1,166 @@
|
||||
import os
|
||||
import typer
|
||||
from typing_extensions import Annotated
|
||||
from .config import load_config
|
||||
|
||||
# Initialize environment variables
|
||||
load_config()
|
||||
|
||||
app = typer.Typer(help="GarminSync - Download Garmin Connect activities", rich_markup_mode=None)
|
||||
|
||||
@app.command("list")
|
||||
def list_activities(
|
||||
all_activities: Annotated[bool, typer.Option("--all", help="List all activities")] = False,
|
||||
missing: Annotated[bool, typer.Option("--missing", help="List missing activities")] = False,
|
||||
downloaded: Annotated[bool, typer.Option("--downloaded", help="List downloaded activities")] = False,
|
||||
offline: Annotated[bool, typer.Option("--offline", help="Work offline without syncing")] = False
|
||||
):
|
||||
"""List activities based on specified filters"""
|
||||
from tqdm import tqdm
|
||||
from .database import get_session, Activity, get_offline_stats, sync_database
|
||||
from .garmin import GarminClient
|
||||
|
||||
# Validate input
|
||||
if not any([all_activities, missing, downloaded]):
|
||||
typer.echo("Error: Please specify at least one filter option (--all, --missing, --downloaded)")
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
try:
|
||||
client = GarminClient()
|
||||
session = get_session()
|
||||
|
||||
if not offline:
|
||||
# Sync database with latest activities
|
||||
typer.echo("Syncing activities from Garmin Connect...")
|
||||
sync_database(client)
|
||||
else:
|
||||
# Show offline status with last sync info
|
||||
stats = get_offline_stats()
|
||||
typer.echo(f"Working in offline mode - using cached data (last sync: {stats['last_sync']})")
|
||||
|
||||
# Build query based on filters
|
||||
query = session.query(Activity)
|
||||
|
||||
if all_activities:
|
||||
pass # Return all activities
|
||||
elif missing:
|
||||
query = query.filter_by(downloaded=False)
|
||||
elif downloaded:
|
||||
query = query.filter_by(downloaded=True)
|
||||
|
||||
# Execute query and display results
|
||||
activities = query.all()
|
||||
if not activities:
|
||||
typer.echo("No activities found matching your criteria")
|
||||
return
|
||||
|
||||
# Display results with progress bar
|
||||
typer.echo(f"Found {len(activities)} activities:")
|
||||
for activity in tqdm(activities, desc="Listing activities"):
|
||||
status = "Downloaded" if activity.downloaded else "Missing"
|
||||
typer.echo(f"- ID: {activity.activity_id}, Start: {activity.start_time}, Status: {status}")
|
||||
|
||||
except Exception as e:
|
||||
typer.echo(f"Error: {str(e)}")
|
||||
raise typer.Exit(code=1)
|
||||
finally:
|
||||
if 'session' in locals():
|
||||
session.close()
|
||||
|
||||
@app.command("download")
|
||||
def download(
|
||||
missing: Annotated[bool, typer.Option("--missing", help="Download missing activities")] = False
|
||||
):
|
||||
"""Download activities based on specified filters"""
|
||||
from tqdm import tqdm
|
||||
from pathlib import Path
|
||||
from .database import get_session, Activity
|
||||
from .garmin import GarminClient
|
||||
|
||||
# Validate input
|
||||
if not missing:
|
||||
typer.echo("Error: Currently only --missing downloads are supported")
|
||||
raise typer.Exit(code=1)
|
||||
|
||||
try:
|
||||
client = GarminClient()
|
||||
session = get_session()
|
||||
|
||||
# Sync database with latest activities
|
||||
typer.echo("Syncing activities from Garmin Connect...")
|
||||
from .database import sync_database
|
||||
sync_database(client)
|
||||
|
||||
# Get missing activities
|
||||
activities = session.query(Activity).filter_by(downloaded=False).all()
|
||||
if not activities:
|
||||
typer.echo("No missing activities found")
|
||||
return
|
||||
|
||||
# Create data directory if it doesn't exist
|
||||
data_dir = Path(os.getenv("DATA_DIR", "data"))
|
||||
data_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Download activities with progress bar
|
||||
typer.echo(f"Downloading {len(activities)} missing activities...")
|
||||
for activity in tqdm(activities, desc="Downloading"):
|
||||
try:
|
||||
# Download FIT data
|
||||
fit_data = client.download_activity_fit(activity.activity_id)
|
||||
|
||||
# Create filename-safe timestamp
|
||||
timestamp = activity.start_time.replace(":", "-").replace(" ", "_")
|
||||
filename = f"activity_{activity.activity_id}_{timestamp}.fit"
|
||||
filepath = data_dir / filename
|
||||
|
||||
# Save file
|
||||
with open(filepath, "wb") as f:
|
||||
f.write(fit_data)
|
||||
|
||||
# Update database
|
||||
activity.filename = str(filepath)
|
||||
activity.downloaded = True
|
||||
session.commit()
|
||||
|
||||
except Exception as e:
|
||||
typer.echo(f"Error downloading activity {activity.activity_id}: {str(e)}")
|
||||
session.rollback()
|
||||
|
||||
typer.echo("Download completed successfully")
|
||||
|
||||
except Exception as e:
|
||||
typer.echo(f"Error: {str(e)}")
|
||||
raise typer.Exit(code=1)
|
||||
finally:
|
||||
if 'session' in locals():
|
||||
session.close()
|
||||
|
||||
@app.command("daemon")
|
||||
def daemon_mode(
|
||||
start: Annotated[bool, typer.Option("--start", help="Start daemon")] = False,
|
||||
stop: Annotated[bool, typer.Option("--stop", help="Stop daemon")] = False,
|
||||
status: Annotated[bool, typer.Option("--status", help="Show daemon status")] = False,
|
||||
port: Annotated[int, typer.Option("--port", help="Web UI port")] = 8080
|
||||
):
|
||||
"""Daemon mode operations"""
|
||||
from .daemon import GarminSyncDaemon
|
||||
|
||||
if start:
|
||||
daemon = GarminSyncDaemon()
|
||||
daemon.start(web_port=port)
|
||||
elif stop:
|
||||
# Implementation for stopping daemon (PID file or signal)
|
||||
typer.echo("Stopping daemon...")
|
||||
# TODO: Implement stop (we can use a PID file to stop the daemon)
|
||||
typer.echo("Daemon stop not implemented yet")
|
||||
elif status:
|
||||
# Show current daemon status
|
||||
typer.echo("Daemon status not implemented yet")
|
||||
else:
|
||||
typer.echo("Please specify one of: --start, --stop, --status")
|
||||
|
||||
def main():
|
||||
app()
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
15
garminsync/config.py
Normal file
15
garminsync/config.py
Normal file
@@ -0,0 +1,15 @@
|
||||
from dotenv import load_dotenv
|
||||
import os
|
||||
|
||||
def load_config():
|
||||
"""Load environment variables from .env file"""
|
||||
load_dotenv()
|
||||
|
||||
class Config:
|
||||
GARMIN_EMAIL = os.getenv("GARMIN_EMAIL")
|
||||
GARMIN_PASSWORD = os.getenv("GARMIN_PASSWORD")
|
||||
|
||||
@classmethod
|
||||
def validate(cls):
|
||||
if not cls.GARMIN_EMAIL or not cls.GARMIN_PASSWORD:
|
||||
raise ValueError("Missing GARMIN_EMAIL or GARMIN_PASSWORD in environment")
|
||||
226
garminsync/daemon.py
Normal file
226
garminsync/daemon.py
Normal file
@@ -0,0 +1,226 @@
|
||||
import signal
|
||||
import sys
|
||||
import time
|
||||
import threading
|
||||
from datetime import datetime
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
from apscheduler.triggers.cron import CronTrigger
|
||||
from .database import get_session, Activity, DaemonConfig, SyncLog
|
||||
from .garmin import GarminClient
|
||||
from .utils import logger
|
||||
|
||||
class GarminSyncDaemon:
|
||||
def __init__(self):
|
||||
self.scheduler = BackgroundScheduler()
|
||||
self.running = False
|
||||
self.web_server = None
|
||||
|
||||
def start(self, web_port=8080):
|
||||
"""Start daemon with scheduler and web UI"""
|
||||
try:
|
||||
# Load configuration from database
|
||||
config_data = self.load_config()
|
||||
|
||||
# Setup scheduled job
|
||||
if config_data['enabled']:
|
||||
self.scheduler.add_job(
|
||||
func=self.sync_and_download,
|
||||
trigger=CronTrigger.from_crontab(config_data['schedule_cron']),
|
||||
id='sync_job',
|
||||
replace_existing=True
|
||||
)
|
||||
|
||||
# Start scheduler
|
||||
self.scheduler.start()
|
||||
self.running = True
|
||||
|
||||
# Update daemon status to running
|
||||
self.update_daemon_status("running")
|
||||
|
||||
# Start web UI in separate thread
|
||||
self.start_web_ui(web_port)
|
||||
|
||||
# Setup signal handlers for graceful shutdown
|
||||
signal.signal(signal.SIGINT, self.signal_handler)
|
||||
signal.signal(signal.SIGTERM, self.signal_handler)
|
||||
|
||||
logger.info(f"Daemon started. Web UI available at http://localhost:{web_port}")
|
||||
|
||||
# Keep daemon running
|
||||
while self.running:
|
||||
time.sleep(1)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to start daemon: {str(e)}")
|
||||
self.update_daemon_status("error")
|
||||
self.stop()
|
||||
|
||||
def sync_and_download(self):
|
||||
"""Scheduled job function"""
|
||||
session = None
|
||||
try:
|
||||
self.log_operation("sync", "started")
|
||||
|
||||
# Import here to avoid circular imports
|
||||
from .garmin import GarminClient
|
||||
from .database import sync_database
|
||||
|
||||
# Perform sync and download
|
||||
client = GarminClient()
|
||||
|
||||
# Sync database first
|
||||
sync_database(client)
|
||||
|
||||
# Download missing activities
|
||||
downloaded_count = 0
|
||||
session = get_session()
|
||||
missing_activities = session.query(Activity).filter_by(downloaded=False).all()
|
||||
|
||||
for activity in missing_activities:
|
||||
try:
|
||||
# Use the correct method name
|
||||
fit_data = client.download_activity_fit(activity.activity_id)
|
||||
|
||||
# Save the file
|
||||
import os
|
||||
from pathlib import Path
|
||||
data_dir = Path(os.getenv("DATA_DIR", "data"))
|
||||
data_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
timestamp = activity.start_time.replace(":", "-").replace(" ", "_")
|
||||
filename = f"activity_{activity.activity_id}_{timestamp}.fit"
|
||||
filepath = data_dir / filename
|
||||
|
||||
with open(filepath, "wb") as f:
|
||||
f.write(fit_data)
|
||||
|
||||
activity.filename = str(filepath)
|
||||
activity.downloaded = True
|
||||
activity.last_sync = datetime.now().isoformat()
|
||||
downloaded_count += 1
|
||||
session.commit()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to download activity {activity.activity_id}: {e}")
|
||||
session.rollback()
|
||||
|
||||
self.log_operation("sync", "success",
|
||||
f"Downloaded {downloaded_count} new activities")
|
||||
|
||||
# Update last run time
|
||||
self.update_daemon_last_run()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Sync failed: {e}")
|
||||
self.log_operation("sync", "error", str(e))
|
||||
finally:
|
||||
if session:
|
||||
session.close()
|
||||
|
||||
def load_config(self):
|
||||
"""Load daemon configuration from database and return dict"""
|
||||
session = get_session()
|
||||
try:
|
||||
config = session.query(DaemonConfig).first()
|
||||
if not config:
|
||||
# Create default configuration
|
||||
config = DaemonConfig()
|
||||
session.add(config)
|
||||
session.commit()
|
||||
session.refresh(config) # Ensure we have the latest data
|
||||
|
||||
# Return configuration as dictionary to avoid session issues
|
||||
return {
|
||||
'id': config.id,
|
||||
'enabled': config.enabled,
|
||||
'schedule_cron': config.schedule_cron,
|
||||
'last_run': config.last_run,
|
||||
'next_run': config.next_run,
|
||||
'status': config.status
|
||||
}
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def update_daemon_status(self, status):
|
||||
"""Update daemon status in database"""
|
||||
session = get_session()
|
||||
try:
|
||||
config = session.query(DaemonConfig).first()
|
||||
if not config:
|
||||
config = DaemonConfig()
|
||||
session.add(config)
|
||||
|
||||
config.status = status
|
||||
session.commit()
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def update_daemon_last_run(self):
|
||||
"""Update daemon last run timestamp"""
|
||||
session = get_session()
|
||||
try:
|
||||
config = session.query(DaemonConfig).first()
|
||||
if config:
|
||||
config.last_run = datetime.now().isoformat()
|
||||
session.commit()
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def start_web_ui(self, port):
|
||||
"""Start FastAPI web server in a separate thread"""
|
||||
try:
|
||||
from .web.app import app
|
||||
import uvicorn
|
||||
|
||||
def run_server():
|
||||
try:
|
||||
uvicorn.run(app, host="0.0.0.0", port=port, log_level="info")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to start web server: {e}")
|
||||
|
||||
web_thread = threading.Thread(target=run_server, daemon=True)
|
||||
web_thread.start()
|
||||
self.web_server = web_thread
|
||||
except ImportError as e:
|
||||
logger.warning(f"Could not start web UI: {e}")
|
||||
|
||||
def signal_handler(self, signum, frame):
|
||||
"""Handle shutdown signals"""
|
||||
logger.info("Received shutdown signal, stopping daemon...")
|
||||
self.stop()
|
||||
|
||||
def stop(self):
|
||||
"""Stop daemon and clean up resources"""
|
||||
if self.scheduler.running:
|
||||
self.scheduler.shutdown()
|
||||
self.running = False
|
||||
self.update_daemon_status("stopped")
|
||||
self.log_operation("daemon", "stopped", "Daemon shutdown completed")
|
||||
logger.info("Daemon stopped")
|
||||
|
||||
def log_operation(self, operation, status, message=None):
|
||||
"""Log sync operation to database"""
|
||||
session = get_session()
|
||||
try:
|
||||
log = SyncLog(
|
||||
timestamp=datetime.now().isoformat(),
|
||||
operation=operation,
|
||||
status=status,
|
||||
message=message,
|
||||
activities_processed=0, # Can be updated later if needed
|
||||
activities_downloaded=0 # Can be updated later if needed
|
||||
)
|
||||
session.add(log)
|
||||
session.commit()
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to log operation: {e}")
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def count_missing(self):
|
||||
"""Count missing activities"""
|
||||
session = get_session()
|
||||
try:
|
||||
return session.query(Activity).filter_by(downloaded=False).count()
|
||||
finally:
|
||||
session.close()
|
||||
105
garminsync/database.py
Normal file
105
garminsync/database.py
Normal file
@@ -0,0 +1,105 @@
|
||||
import os
|
||||
from sqlalchemy import create_engine, Column, Integer, String, Boolean
|
||||
from sqlalchemy.orm import declarative_base, sessionmaker
|
||||
from sqlalchemy.exc import SQLAlchemyError
|
||||
|
||||
Base = declarative_base()
|
||||
|
||||
class Activity(Base):
|
||||
__tablename__ = 'activities'
|
||||
|
||||
activity_id = Column(Integer, primary_key=True)
|
||||
start_time = Column(String, nullable=False)
|
||||
filename = Column(String, unique=True, nullable=True)
|
||||
downloaded = Column(Boolean, default=False, nullable=False)
|
||||
created_at = Column(String, nullable=False) # Add this line
|
||||
last_sync = Column(String, nullable=True) # ISO timestamp of last sync
|
||||
|
||||
class DaemonConfig(Base):
|
||||
__tablename__ = 'daemon_config'
|
||||
|
||||
id = Column(Integer, primary_key=True, default=1)
|
||||
enabled = Column(Boolean, default=True, nullable=False)
|
||||
schedule_cron = Column(String, default="0 */6 * * *", nullable=False) # Every 6 hours
|
||||
last_run = Column(String, nullable=True)
|
||||
next_run = Column(String, nullable=True)
|
||||
status = Column(String, default="stopped", nullable=False) # stopped, running, error
|
||||
|
||||
class SyncLog(Base):
|
||||
__tablename__ = 'sync_logs'
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
timestamp = Column(String, nullable=False)
|
||||
operation = Column(String, nullable=False) # sync, download, daemon_start, daemon_stop
|
||||
status = Column(String, nullable=False) # success, error, partial
|
||||
message = Column(String, nullable=True)
|
||||
activities_processed = Column(Integer, default=0, nullable=False)
|
||||
activities_downloaded = Column(Integer, default=0, nullable=False)
|
||||
|
||||
def init_db():
|
||||
"""Initialize database connection and create tables"""
|
||||
db_path = os.path.join(os.getenv("DATA_DIR", "data"), "garmin.db")
|
||||
engine = create_engine(f"sqlite:///{db_path}")
|
||||
Base.metadata.create_all(engine)
|
||||
return engine
|
||||
|
||||
def get_session():
|
||||
"""Create a new database session"""
|
||||
engine = init_db()
|
||||
Session = sessionmaker(bind=engine)
|
||||
return Session()
|
||||
|
||||
def sync_database(garmin_client):
|
||||
"""Sync local database with Garmin Connect activities"""
|
||||
from datetime import datetime
|
||||
session = get_session()
|
||||
try:
|
||||
# Fetch activities from Garmin Connect
|
||||
activities = garmin_client.get_activities(0, 1000)
|
||||
|
||||
# Process activities and update database
|
||||
for activity in activities:
|
||||
activity_id = activity["activityId"]
|
||||
start_time = activity["startTimeLocal"]
|
||||
|
||||
# Check if activity exists in database
|
||||
existing = session.query(Activity).filter_by(activity_id=activity_id).first()
|
||||
if not existing:
|
||||
new_activity = Activity(
|
||||
activity_id=activity_id,
|
||||
start_time=start_time,
|
||||
downloaded=False,
|
||||
created_at=datetime.now().isoformat(), # Add this line
|
||||
last_sync=datetime.now().isoformat()
|
||||
)
|
||||
session.add(new_activity)
|
||||
|
||||
session.commit()
|
||||
except SQLAlchemyError as e:
|
||||
session.rollback()
|
||||
raise e
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
def get_offline_stats():
|
||||
"""Return statistics about cached data without API calls"""
|
||||
session = get_session()
|
||||
try:
|
||||
total = session.query(Activity).count()
|
||||
downloaded = session.query(Activity).filter_by(downloaded=True).count()
|
||||
missing = total - downloaded
|
||||
# Get most recent sync timestamp
|
||||
last_sync = session.query(Activity).order_by(Activity.last_sync.desc()).first()
|
||||
return {
|
||||
'total': total,
|
||||
'downloaded': downloaded,
|
||||
'missing': missing,
|
||||
'last_sync': last_sync.last_sync if last_sync else 'Never synced'
|
||||
}
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
# Example usage:
|
||||
# from .garmin import GarminClient
|
||||
# client = GarminClient()
|
||||
# sync_database(client)
|
||||
123
garminsync/garmin.py
Normal file
123
garminsync/garmin.py
Normal file
@@ -0,0 +1,123 @@
|
||||
import os
|
||||
import time
|
||||
from garminconnect import Garmin
|
||||
|
||||
class GarminClient:
|
||||
def __init__(self):
|
||||
self.client = None
|
||||
|
||||
def authenticate(self):
|
||||
"""Authenticate using credentials from environment variables"""
|
||||
email = os.getenv("GARMIN_EMAIL")
|
||||
password = os.getenv("GARMIN_PASSWORD")
|
||||
|
||||
if not email or not password:
|
||||
raise ValueError("Garmin credentials not found in environment variables")
|
||||
|
||||
self.client = Garmin(email, password)
|
||||
self.client.login()
|
||||
return self.client
|
||||
|
||||
def get_activities(self, start=0, limit=10):
|
||||
"""Get list of activities with rate limiting"""
|
||||
if not self.client:
|
||||
self.authenticate()
|
||||
|
||||
activities = self.client.get_activities(start, limit)
|
||||
time.sleep(2) # Rate limiting
|
||||
return activities
|
||||
|
||||
def download_activity_fit(self, activity_id):
|
||||
"""Download .fit file for a specific activity"""
|
||||
if not self.client:
|
||||
self.authenticate()
|
||||
|
||||
print(f"Attempting to download activity {activity_id}")
|
||||
|
||||
# Try multiple methods to download FIT file
|
||||
methods_to_try = [
|
||||
# Method 1: No format parameter (most likely to work)
|
||||
lambda: self.client.download_activity(activity_id),
|
||||
|
||||
# Method 2: Use 'fmt' instead of 'dl_fmt'
|
||||
lambda: self.client.download_activity(activity_id, fmt='fit'),
|
||||
|
||||
# Method 3: Use 'format' parameter
|
||||
lambda: self.client.download_activity(activity_id, format='fit'),
|
||||
|
||||
# Method 4: Try original parameter name with different values
|
||||
lambda: self.client.download_activity(activity_id, dl_fmt='FIT'),
|
||||
lambda: self.client.download_activity(activity_id, dl_fmt='tcx'), # Fallback format
|
||||
]
|
||||
|
||||
last_exception = None
|
||||
|
||||
for i, method in enumerate(methods_to_try, 1):
|
||||
try:
|
||||
print(f"Trying download method {i}...")
|
||||
fit_data = method()
|
||||
|
||||
if fit_data:
|
||||
print(f"Successfully downloaded {len(fit_data)} bytes using method {i}")
|
||||
time.sleep(2) # Rate limiting
|
||||
return fit_data
|
||||
else:
|
||||
print(f"Method {i} returned empty data")
|
||||
|
||||
except Exception as e:
|
||||
print(f"Method {i} failed: {type(e).__name__}: {e}")
|
||||
last_exception = e
|
||||
continue
|
||||
|
||||
# If all methods failed, raise the last exception
|
||||
raise RuntimeError(f"All download methods failed. Last error: {last_exception}")
|
||||
|
||||
def get_activity_details(self, activity_id):
|
||||
"""Get detailed information about a specific activity"""
|
||||
if not self.client:
|
||||
self.authenticate()
|
||||
|
||||
try:
|
||||
activity_details = self.client.get_activity_by_id(activity_id)
|
||||
time.sleep(2) # Rate limiting
|
||||
return activity_details
|
||||
except Exception as e:
|
||||
print(f"Failed to get activity details for {activity_id}: {e}")
|
||||
return None
|
||||
|
||||
# Example usage and testing function
|
||||
def test_download(activity_id):
|
||||
"""Test function to verify download functionality"""
|
||||
client = GarminClient()
|
||||
try:
|
||||
fit_data = client.download_activity_fit(activity_id)
|
||||
|
||||
# Verify the data looks like a FIT file
|
||||
if fit_data and len(fit_data) > 14:
|
||||
# FIT files start with specific header
|
||||
header = fit_data[:14]
|
||||
if b'.FIT' in header or header[8:12] == b'.FIT':
|
||||
print("✅ Downloaded data appears to be a valid FIT file")
|
||||
return fit_data
|
||||
else:
|
||||
print("⚠️ Downloaded data may not be a FIT file")
|
||||
print(f"Header: {header}")
|
||||
return fit_data
|
||||
else:
|
||||
print("❌ Downloaded data is empty or too small")
|
||||
return None
|
||||
|
||||
except Exception as e:
|
||||
print(f"❌ Test failed: {e}")
|
||||
return None
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Test with a sample activity ID if provided
|
||||
import sys
|
||||
if len(sys.argv) > 1:
|
||||
test_activity_id = sys.argv[1]
|
||||
print(f"Testing download for activity ID: {test_activity_id}")
|
||||
test_download(test_activity_id)
|
||||
else:
|
||||
print("Usage: python garmin.py <activity_id>")
|
||||
print("This will test the download functionality with the provided activity ID")
|
||||
85
garminsync/utils.py
Normal file
85
garminsync/utils.py
Normal file
@@ -0,0 +1,85 @@
|
||||
import logging
|
||||
import sys
|
||||
from datetime import datetime
|
||||
|
||||
# Configure logging
|
||||
def setup_logger(name="garminsync", level=logging.INFO):
|
||||
"""Setup logger with consistent formatting"""
|
||||
logger = logging.getLogger(name)
|
||||
|
||||
# Prevent duplicate handlers
|
||||
if logger.handlers:
|
||||
return logger
|
||||
|
||||
logger.setLevel(level)
|
||||
|
||||
# Create console handler
|
||||
handler = logging.StreamHandler(sys.stdout)
|
||||
handler.setLevel(level)
|
||||
|
||||
# Create formatter
|
||||
formatter = logging.Formatter(
|
||||
'%(asctime)s - %(name)s - %(levelname)s - %(message)s'
|
||||
)
|
||||
handler.setFormatter(formatter)
|
||||
|
||||
# Add handler to logger
|
||||
logger.addHandler(handler)
|
||||
|
||||
return logger
|
||||
|
||||
# Create default logger instance
|
||||
logger = setup_logger()
|
||||
|
||||
def format_timestamp(timestamp_str=None):
|
||||
"""Format timestamp string for display"""
|
||||
if not timestamp_str:
|
||||
return "Never"
|
||||
|
||||
try:
|
||||
# Parse ISO format timestamp
|
||||
dt = datetime.fromisoformat(timestamp_str.replace('Z', '+00:00'))
|
||||
return dt.strftime("%Y-%m-%d %H:%M:%S")
|
||||
except (ValueError, AttributeError):
|
||||
return timestamp_str
|
||||
|
||||
def safe_filename(filename):
|
||||
"""Make filename safe for filesystem"""
|
||||
import re
|
||||
# Replace problematic characters
|
||||
safe_name = re.sub(r'[<>:"/\\|?*]', '_', filename)
|
||||
# Replace spaces and colons commonly found in timestamps
|
||||
safe_name = safe_name.replace(':', '-').replace(' ', '_')
|
||||
return safe_name
|
||||
|
||||
def bytes_to_human_readable(bytes_count):
|
||||
"""Convert bytes to human readable format"""
|
||||
if bytes_count == 0:
|
||||
return "0 B"
|
||||
|
||||
for unit in ['B', 'KB', 'MB', 'GB']:
|
||||
if bytes_count < 1024.0:
|
||||
return f"{bytes_count:.1f} {unit}"
|
||||
bytes_count /= 1024.0
|
||||
return f"{bytes_count:.1f} TB"
|
||||
|
||||
def validate_cron_expression(cron_expr):
|
||||
"""Basic validation of cron expression"""
|
||||
try:
|
||||
from apscheduler.triggers.cron import CronTrigger
|
||||
# Try to create a CronTrigger with the expression
|
||||
CronTrigger.from_crontab(cron_expr)
|
||||
return True
|
||||
except (ValueError, TypeError):
|
||||
return False
|
||||
|
||||
# Utility function for error handling
|
||||
def handle_db_error(func):
|
||||
"""Decorator for database operations with error handling"""
|
||||
def wrapper(*args, **kwargs):
|
||||
try:
|
||||
return func(*args, **kwargs)
|
||||
except Exception as e:
|
||||
logger.error(f"Database operation failed in {func.__name__}: {e}")
|
||||
raise
|
||||
return wrapper
|
||||
1
garminsync/web/__init__.py
Normal file
1
garminsync/web/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
# Empty file to mark this directory as a Python package
|
||||
88
garminsync/web/app.py
Normal file
88
garminsync/web/app.py
Normal file
@@ -0,0 +1,88 @@
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from fastapi.templating import Jinja2Templates
|
||||
from fastapi.responses import JSONResponse
|
||||
import os
|
||||
from pathlib import Path
|
||||
from .routes import router
|
||||
|
||||
app = FastAPI(title="GarminSync Dashboard")
|
||||
|
||||
# Get the current directory path
|
||||
current_dir = Path(__file__).parent
|
||||
|
||||
# Mount static files and templates with error handling
|
||||
static_dir = current_dir / "static"
|
||||
templates_dir = current_dir / "templates"
|
||||
|
||||
if static_dir.exists():
|
||||
app.mount("/static", StaticFiles(directory=str(static_dir)), name="static")
|
||||
|
||||
if templates_dir.exists():
|
||||
templates = Jinja2Templates(directory=str(templates_dir))
|
||||
else:
|
||||
templates = None
|
||||
|
||||
# Include API routes
|
||||
app.include_router(router)
|
||||
|
||||
@app.get("/")
|
||||
async def dashboard(request: Request):
|
||||
"""Dashboard route with fallback for missing templates"""
|
||||
if not templates:
|
||||
# Return JSON response if templates are not available
|
||||
from garminsync.database import get_offline_stats
|
||||
stats = get_offline_stats()
|
||||
return JSONResponse({
|
||||
"message": "GarminSync Dashboard",
|
||||
"stats": stats,
|
||||
"note": "Web UI templates not found, showing JSON response"
|
||||
})
|
||||
|
||||
try:
|
||||
# Get current statistics
|
||||
from garminsync.database import get_offline_stats
|
||||
stats = get_offline_stats()
|
||||
|
||||
return templates.TemplateResponse("dashboard.html", {
|
||||
"request": request,
|
||||
"stats": stats
|
||||
})
|
||||
except Exception as e:
|
||||
return JSONResponse({
|
||||
"error": f"Failed to load dashboard: {str(e)}",
|
||||
"message": "Dashboard unavailable, API endpoints still functional"
|
||||
})
|
||||
|
||||
@app.get("/health")
|
||||
async def health_check():
|
||||
"""Health check endpoint"""
|
||||
return {"status": "healthy", "service": "GarminSync Dashboard"}
|
||||
|
||||
@app.get("/config")
|
||||
async def config_page(request: Request):
|
||||
"""Configuration page"""
|
||||
if not templates:
|
||||
return JSONResponse({
|
||||
"message": "Configuration endpoint",
|
||||
"note": "Use /api/schedule endpoints for configuration"
|
||||
})
|
||||
|
||||
return templates.TemplateResponse("config.html", {
|
||||
"request": request
|
||||
})
|
||||
|
||||
# Error handlers
|
||||
@app.exception_handler(404)
|
||||
async def not_found_handler(request: Request, exc):
|
||||
return JSONResponse(
|
||||
status_code=404,
|
||||
content={"error": "Not found", "path": str(request.url.path)}
|
||||
)
|
||||
|
||||
@app.exception_handler(500)
|
||||
async def server_error_handler(request: Request, exc):
|
||||
return JSONResponse(
|
||||
status_code=500,
|
||||
content={"error": "Internal server error", "detail": str(exc)}
|
||||
)
|
||||
212
garminsync/web/routes.py
Normal file
212
garminsync/web/routes.py
Normal file
@@ -0,0 +1,212 @@
|
||||
from fastapi import APIRouter, HTTPException
|
||||
from pydantic import BaseModel
|
||||
from garminsync.database import get_session, DaemonConfig, SyncLog
|
||||
|
||||
router = APIRouter(prefix="/api")
|
||||
|
||||
class ScheduleConfig(BaseModel):
|
||||
enabled: bool
|
||||
cron_schedule: str
|
||||
|
||||
@router.get("/status")
|
||||
async def get_status():
|
||||
"""Get current daemon status"""
|
||||
session = get_session()
|
||||
try:
|
||||
config = session.query(DaemonConfig).first()
|
||||
|
||||
# Get recent logs
|
||||
logs = session.query(SyncLog).order_by(SyncLog.timestamp.desc()).limit(10).all()
|
||||
|
||||
# Convert to dictionaries to avoid session issues
|
||||
daemon_data = {
|
||||
"running": config.status == "running" if config else False,
|
||||
"next_run": config.next_run if config else None,
|
||||
"schedule": config.schedule_cron if config else None,
|
||||
"last_run": config.last_run if config else None,
|
||||
"enabled": config.enabled if config else False
|
||||
}
|
||||
|
||||
log_data = []
|
||||
for log in logs:
|
||||
log_data.append({
|
||||
"timestamp": log.timestamp,
|
||||
"operation": log.operation,
|
||||
"status": log.status,
|
||||
"message": log.message,
|
||||
"activities_processed": log.activities_processed,
|
||||
"activities_downloaded": log.activities_downloaded
|
||||
})
|
||||
|
||||
return {
|
||||
"daemon": daemon_data,
|
||||
"recent_logs": log_data
|
||||
}
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
@router.post("/schedule")
|
||||
async def update_schedule(config: ScheduleConfig):
|
||||
"""Update daemon schedule configuration"""
|
||||
session = get_session()
|
||||
try:
|
||||
daemon_config = session.query(DaemonConfig).first()
|
||||
|
||||
if not daemon_config:
|
||||
daemon_config = DaemonConfig()
|
||||
session.add(daemon_config)
|
||||
|
||||
daemon_config.enabled = config.enabled
|
||||
daemon_config.schedule_cron = config.cron_schedule
|
||||
session.commit()
|
||||
|
||||
return {"message": "Configuration updated successfully"}
|
||||
except Exception as e:
|
||||
session.rollback()
|
||||
raise HTTPException(status_code=500, detail=f"Failed to update configuration: {str(e)}")
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
@router.post("/sync/trigger")
|
||||
async def trigger_sync():
|
||||
"""Manually trigger a sync operation"""
|
||||
try:
|
||||
# Import here to avoid circular imports
|
||||
from garminsync.garmin import GarminClient
|
||||
from garminsync.database import sync_database, Activity
|
||||
from datetime import datetime
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
# Create client and sync
|
||||
client = GarminClient()
|
||||
sync_database(client)
|
||||
|
||||
# Download missing activities
|
||||
session = get_session()
|
||||
try:
|
||||
missing_activities = session.query(Activity).filter_by(downloaded=False).all()
|
||||
downloaded_count = 0
|
||||
|
||||
data_dir = Path(os.getenv("DATA_DIR", "data"))
|
||||
data_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
for activity in missing_activities:
|
||||
try:
|
||||
fit_data = client.download_activity_fit(activity.activity_id)
|
||||
|
||||
timestamp = activity.start_time.replace(":", "-").replace(" ", "_")
|
||||
filename = f"activity_{activity.activity_id}_{timestamp}.fit"
|
||||
filepath = data_dir / filename
|
||||
|
||||
with open(filepath, "wb") as f:
|
||||
f.write(fit_data)
|
||||
|
||||
activity.filename = str(filepath)
|
||||
activity.downloaded = True
|
||||
activity.last_sync = datetime.now().isoformat()
|
||||
downloaded_count += 1
|
||||
session.commit()
|
||||
|
||||
except Exception as e:
|
||||
print(f"Failed to download activity {activity.activity_id}: {e}")
|
||||
session.rollback()
|
||||
|
||||
return {"message": f"Sync completed successfully. Downloaded {downloaded_count} activities."}
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
except Exception as e:
|
||||
raise HTTPException(status_code=500, detail=f"Sync failed: {str(e)}")
|
||||
|
||||
@router.get("/activities/stats")
|
||||
async def get_activity_stats():
|
||||
"""Get activity statistics"""
|
||||
from garminsync.database import get_offline_stats
|
||||
return get_offline_stats()
|
||||
|
||||
@router.get("/logs")
|
||||
async def get_logs(limit: int = 50):
|
||||
"""Get recent sync logs"""
|
||||
session = get_session()
|
||||
try:
|
||||
logs = session.query(SyncLog).order_by(SyncLog.timestamp.desc()).limit(limit).all()
|
||||
|
||||
log_data = []
|
||||
for log in logs:
|
||||
log_data.append({
|
||||
"id": log.id,
|
||||
"timestamp": log.timestamp,
|
||||
"operation": log.operation,
|
||||
"status": log.status,
|
||||
"message": log.message,
|
||||
"activities_processed": log.activities_processed,
|
||||
"activities_downloaded": log.activities_downloaded
|
||||
})
|
||||
|
||||
return {"logs": log_data}
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
@router.post("/daemon/start")
|
||||
async def start_daemon():
|
||||
"""Start the daemon process"""
|
||||
from garminsync.daemon import daemon_instance
|
||||
try:
|
||||
# Start the daemon in a separate thread to avoid blocking
|
||||
import threading
|
||||
daemon_thread = threading.Thread(target=daemon_instance.start)
|
||||
daemon_thread.daemon = True
|
||||
daemon_thread.start()
|
||||
|
||||
# Update daemon status in database
|
||||
session = get_session()
|
||||
config = session.query(DaemonConfig).first()
|
||||
if not config:
|
||||
config = DaemonConfig()
|
||||
session.add(config)
|
||||
config.status = "running"
|
||||
session.commit()
|
||||
|
||||
return {"message": "Daemon started successfully"}
|
||||
except Exception as e:
|
||||
session.rollback()
|
||||
raise HTTPException(status_code=500, detail=f"Failed to start daemon: {str(e)}")
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
@router.post("/daemon/stop")
|
||||
async def stop_daemon():
|
||||
"""Stop the daemon process"""
|
||||
from garminsync.daemon import daemon_instance
|
||||
try:
|
||||
# Stop the daemon
|
||||
daemon_instance.stop()
|
||||
|
||||
# Update daemon status in database
|
||||
session = get_session()
|
||||
config = session.query(DaemonConfig).first()
|
||||
if config:
|
||||
config.status = "stopped"
|
||||
session.commit()
|
||||
|
||||
return {"message": "Daemon stopped successfully"}
|
||||
except Exception as e:
|
||||
session.rollback()
|
||||
raise HTTPException(status_code=500, detail=f"Failed to stop daemon: {str(e)}")
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
@router.delete("/logs")
|
||||
async def clear_logs():
|
||||
"""Clear all sync logs"""
|
||||
session = get_session()
|
||||
try:
|
||||
session.query(SyncLog).delete()
|
||||
session.commit()
|
||||
return {"message": "Logs cleared successfully"}
|
||||
except Exception as e:
|
||||
session.rollback()
|
||||
raise HTTPException(status_code=500, detail=f"Failed to clear logs: {str(e)}")
|
||||
finally:
|
||||
session.close()
|
||||
98
garminsync/web/static/app.js
Normal file
98
garminsync/web/static/app.js
Normal file
@@ -0,0 +1,98 @@
|
||||
// Auto-refresh dashboard data
|
||||
setInterval(updateStatus, 30000); // Every 30 seconds
|
||||
|
||||
async function updateStatus() {
|
||||
try {
|
||||
const response = await fetch('/api/status');
|
||||
const data = await response.json();
|
||||
|
||||
// Update daemon status
|
||||
document.getElementById('daemon-status').innerHTML = `
|
||||
<p>Status: <span class="badge ${data.daemon.running ? 'badge-success' : 'badge-danger'}">
|
||||
${data.daemon.running ? 'Running' : 'Stopped'}
|
||||
</span></p>
|
||||
<p>Last Run: ${data.daemon.last_run || 'Never'}</p>
|
||||
<p>Next Run: ${data.daemon.next_run || 'Not scheduled'}</p>
|
||||
<p>Schedule: ${data.daemon.schedule || 'Not configured'}</p>
|
||||
`;
|
||||
|
||||
// Update recent logs
|
||||
const logsHtml = data.recent_logs.map(log => `
|
||||
<div class="log-entry">
|
||||
<small class="text-muted">${log.timestamp}</small>
|
||||
<span class="badge badge-${log.status === 'success' ? 'success' : 'danger'}">
|
||||
${log.status}
|
||||
</span>
|
||||
${log.operation}: ${log.message || ''}
|
||||
${log.activities_downloaded > 0 ? `Downloaded ${log.activities_downloaded} activities` : ''}
|
||||
</div>
|
||||
`).join('');
|
||||
|
||||
document.getElementById('recent-logs').innerHTML = logsHtml;
|
||||
|
||||
} catch (error) {
|
||||
console.error('Failed to update status:', error);
|
||||
}
|
||||
}
|
||||
|
||||
async function triggerSync() {
|
||||
try {
|
||||
await fetch('/api/sync/trigger', { method: 'POST' });
|
||||
alert('Sync triggered successfully');
|
||||
updateStatus();
|
||||
} catch (error) {
|
||||
alert('Failed to trigger sync');
|
||||
}
|
||||
}
|
||||
|
||||
async function toggleDaemon() {
|
||||
try {
|
||||
const statusResponse = await fetch('/api/status');
|
||||
const statusData = await statusResponse.json();
|
||||
const isRunning = statusData.daemon.running;
|
||||
|
||||
if (isRunning) {
|
||||
await fetch('/api/daemon/stop', { method: 'POST' });
|
||||
alert('Daemon stopped successfully');
|
||||
} else {
|
||||
await fetch('/api/daemon/start', { method: 'POST' });
|
||||
alert('Daemon started successfully');
|
||||
}
|
||||
|
||||
updateStatus();
|
||||
} catch (error) {
|
||||
alert('Failed to toggle daemon: ' + error.message);
|
||||
}
|
||||
}
|
||||
|
||||
// Schedule form handling
|
||||
document.getElementById('schedule-form')?.addEventListener('submit', async function(e) {
|
||||
e.preventDefault();
|
||||
|
||||
const enabled = document.getElementById('schedule-enabled').checked;
|
||||
const cronSchedule = document.getElementById('cron-schedule').value;
|
||||
|
||||
try {
|
||||
const response = await fetch('/api/schedule', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
enabled: enabled,
|
||||
cron_schedule: cronSchedule
|
||||
})
|
||||
});
|
||||
|
||||
if (response.ok) {
|
||||
alert('Schedule updated successfully');
|
||||
updateStatus();
|
||||
} else {
|
||||
const error = await response.json();
|
||||
alert(`Error: ${error.detail}`);
|
||||
}
|
||||
} catch (error) {
|
||||
alert('Failed to update schedule: ' + error.message);
|
||||
}
|
||||
});
|
||||
|
||||
// Initialize on page load
|
||||
document.addEventListener('DOMContentLoaded', updateStatus);
|
||||
32
garminsync/web/static/style.css
Normal file
32
garminsync/web/static/style.css
Normal file
@@ -0,0 +1,32 @@
|
||||
body {
|
||||
font-family: Arial, sans-serif;
|
||||
background-color: #f8f9fa;
|
||||
}
|
||||
|
||||
.card {
|
||||
margin-bottom: 20px;
|
||||
box-shadow: 0 2px 10px rgba(0,0,0,0.1);
|
||||
}
|
||||
|
||||
.card-header {
|
||||
font-weight: bold;
|
||||
background-color: #f1f1f1;
|
||||
}
|
||||
|
||||
.btn {
|
||||
margin-right: 5px;
|
||||
}
|
||||
|
||||
.log-entry {
|
||||
margin-bottom: 10px;
|
||||
padding: 5px;
|
||||
border-left: 3px solid #ddd;
|
||||
}
|
||||
|
||||
.log-entry .badge-success {
|
||||
background-color: #28a745;
|
||||
}
|
||||
|
||||
.log-entry .badge-error {
|
||||
background-color: #dc3545;
|
||||
}
|
||||
37
garminsync/web/templates/base.html
Normal file
37
garminsync/web/templates/base.html
Normal file
@@ -0,0 +1,37 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>GarminSync Dashboard</title>
|
||||
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/css/bootstrap.min.css" rel="stylesheet">
|
||||
<link href="/static/style.css" rel="stylesheet">
|
||||
</head>
|
||||
<body>
|
||||
<nav class="navbar navbar-expand-lg navbar-dark bg-dark">
|
||||
<div class="container">
|
||||
<a class="navbar-brand" href="/">GarminSync</a>
|
||||
<button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarNav">
|
||||
<span class="navbar-toggler-icon"></span>
|
||||
</button>
|
||||
<div class="collapse navbar-collapse" id="navbarNav">
|
||||
<ul class="navbar-nav">
|
||||
<li class="nav-item">
|
||||
<a class="nav-link active" href="/">Dashboard</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a class="nav-link" href="/config">Configuration</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<div class="container mt-4">
|
||||
{% block content %}{% endblock %}
|
||||
</div>
|
||||
|
||||
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/js/bootstrap.bundle.min.js"></script>
|
||||
<script src="/static/app.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
141
garminsync/web/templates/config.html
Normal file
141
garminsync/web/templates/config.html
Normal file
@@ -0,0 +1,141 @@
|
||||
{% extends "base.html" %}
|
||||
|
||||
{% block content %}
|
||||
<div class="container">
|
||||
<h1>GarminSync Configuration</h1>
|
||||
|
||||
<div class="row">
|
||||
<div class="col-12">
|
||||
<div class="card">
|
||||
<div class="card-header">Daemon Settings</div>
|
||||
<div class="card-body">
|
||||
<form id="daemon-config-form">
|
||||
<div class="form-group">
|
||||
<label for="daemon-enabled">Enable Daemon</label>
|
||||
<input type="checkbox" id="daemon-enabled" {% if config.enabled %}checked{% endif %}>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="cron-schedule">Synchronization Schedule</label>
|
||||
<input type="text" class="form-control" id="cron-schedule"
|
||||
value="{{ config.schedule_cron }}"
|
||||
placeholder="0 */6 * * *"
|
||||
title="Cron expression (every 6 hours by default)">
|
||||
<small class="form-text text-muted">
|
||||
Cron format: minute hour day(month) month day(week)
|
||||
</small>
|
||||
</div>
|
||||
<button type="submit" class="btn btn-primary">Save Settings</button>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row mt-4">
|
||||
<div class="col-12">
|
||||
<div class="card">
|
||||
<div class="card-header">Daemon Status</div>
|
||||
<div class="card-body">
|
||||
<p>Current Status: <span id="daemon-status-text">{{ config.status|capitalize }}</span></p>
|
||||
<p>Last Run: <span id="daemon-last-run">{{ config.last_run or 'Never' }}</span></p>
|
||||
<p>Next Run: <span id="daemon-next-run">{{ config.next_run or 'Not scheduled' }}</span></p>
|
||||
|
||||
<div class="mt-3">
|
||||
<button id="start-daemon-btn" class="btn btn-success mr-2">
|
||||
Start Daemon
|
||||
</button>
|
||||
<button id="stop-daemon-btn" class="btn btn-danger">
|
||||
Stop Daemon
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
document.addEventListener('DOMContentLoaded', function() {
|
||||
// Form submission handler
|
||||
document.getElementById('daemon-config-form').addEventListener('submit', async function(e) {
|
||||
e.preventDefault();
|
||||
|
||||
const enabled = document.getElementById('daemon-enabled').checked;
|
||||
const cronSchedule = document.getElementById('cron-schedule').value;
|
||||
|
||||
try {
|
||||
const response = await fetch('/api/schedule', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
enabled: enabled,
|
||||
cron_schedule: cronSchedule
|
||||
})
|
||||
});
|
||||
|
||||
if (response.ok) {
|
||||
alert('Configuration saved successfully');
|
||||
updateStatus();
|
||||
} else {
|
||||
const error = await response.json();
|
||||
alert(`Error: ${error.detail}`);
|
||||
}
|
||||
} catch (error) {
|
||||
alert('Failed to save configuration: ' + error.message);
|
||||
}
|
||||
});
|
||||
|
||||
// Daemon control buttons
|
||||
document.getElementById('start-daemon-btn').addEventListener('click', async function() {
|
||||
try {
|
||||
const response = await fetch('/api/daemon/start', { method: 'POST' });
|
||||
if (response.ok) {
|
||||
alert('Daemon started successfully');
|
||||
updateStatus();
|
||||
} else {
|
||||
const error = await response.json();
|
||||
alert(`Error: ${error.detail}`);
|
||||
}
|
||||
} catch (error) {
|
||||
alert('Failed to start daemon: ' + error.message);
|
||||
}
|
||||
});
|
||||
|
||||
document.getElementById('stop-daemon-btn').addEventListener('click', async function() {
|
||||
try {
|
||||
const response = await fetch('/api/daemon/stop', { method: 'POST' });
|
||||
if (response.ok) {
|
||||
alert('Daemon stopped successfully');
|
||||
updateStatus();
|
||||
} else {
|
||||
const error = await response.json();
|
||||
alert(`Error: ${error.detail}`);
|
||||
}
|
||||
} catch (error) {
|
||||
alert('Failed to stop daemon: ' + error.message);
|
||||
}
|
||||
});
|
||||
|
||||
// Initial status update
|
||||
updateStatus();
|
||||
|
||||
async function updateStatus() {
|
||||
try {
|
||||
const response = await fetch('/api/status');
|
||||
const data = await response.json();
|
||||
|
||||
// Update status display
|
||||
document.getElementById('daemon-status-text').textContent =
|
||||
data.daemon.running ? 'Running' : 'Stopped';
|
||||
document.getElementById('daemon-last-run').textContent =
|
||||
data.daemon.last_run || 'Never';
|
||||
document.getElementById('daemon-next-run').textContent =
|
||||
data.daemon.next_run || 'Not scheduled';
|
||||
|
||||
} catch (error) {
|
||||
console.error('Failed to update status:', error);
|
||||
}
|
||||
}
|
||||
});
|
||||
</script>
|
||||
{% endblock %}
|
||||
79
garminsync/web/templates/dashboard.html
Normal file
79
garminsync/web/templates/dashboard.html
Normal file
@@ -0,0 +1,79 @@
|
||||
{% extends "base.html" %}
|
||||
|
||||
{% block content %}
|
||||
<div class="container">
|
||||
<h1>GarminSync Dashboard</h1>
|
||||
|
||||
<div class="row">
|
||||
<div class="col-md-4">
|
||||
<div class="card">
|
||||
<div class="card-header">Statistics</div>
|
||||
<div class="card-body">
|
||||
<p>Total Activities: {{ stats.total }}</p>
|
||||
<p>Downloaded: {{ stats.downloaded }}</p>
|
||||
<p>Missing: {{ stats.missing }}</p>
|
||||
<p>Last Sync: {{ stats.last_sync }}</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="col-md-4">
|
||||
<div class="card">
|
||||
<div class="card-header">Daemon Status</div>
|
||||
<div class="card-body" id="daemon-status">
|
||||
<!-- Populated by JavaScript -->
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="col-md-4">
|
||||
<div class="card">
|
||||
<div class="card-header">Quick Actions</div>
|
||||
<div class="card-body">
|
||||
<button class="btn btn-primary" onclick="triggerSync()">
|
||||
Sync Now
|
||||
</button>
|
||||
<button class="btn btn-secondary" onclick="toggleDaemon()">
|
||||
Toggle Daemon
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row mt-4">
|
||||
<div class="col-12">
|
||||
<div class="card">
|
||||
<div class="card-header">Recent Activity</div>
|
||||
<div class="card-body" id="recent-logs">
|
||||
<!-- Populated by JavaScript -->
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row mt-4">
|
||||
<div class="col-12">
|
||||
<div class="card">
|
||||
<div class="card-header">Schedule Configuration</div>
|
||||
<div class="card-body">
|
||||
<form id="schedule-form">
|
||||
<div class="form-group">
|
||||
<label for="schedule-enabled">Enable Scheduled Sync</label>
|
||||
<input type="checkbox" id="schedule-enabled">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="cron-schedule">Cron Schedule</label>
|
||||
<input type="text" class="form-control" id="cron-schedule"
|
||||
placeholder="0 */6 * * *" title="Every 6 hours">
|
||||
</div>
|
||||
<button type="submit" class="btn btn-primary">
|
||||
Update Schedule
|
||||
</button>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
203
plan.md
Normal file
203
plan.md
Normal file
@@ -0,0 +1,203 @@
|
||||
# GarminSync Fixes and Updated Requirements
|
||||
|
||||
## Primary Issue: Dependency Conflicts
|
||||
|
||||
The main error you're encountering is a dependency conflict between `pydantic` and `garth` (a dependency of `garminconnect`). Here's the solution:
|
||||
|
||||
### Updated requirements.txt
|
||||
```
|
||||
typer==0.9.0
|
||||
click==8.1.7
|
||||
python-dotenv==1.0.0
|
||||
garminconnect==0.2.29
|
||||
sqlalchemy==2.0.23
|
||||
tqdm==4.66.1
|
||||
fastapi==0.104.1
|
||||
uvicorn[standard]==0.24.0
|
||||
apscheduler==3.10.4
|
||||
pydantic>=2.0.0,<2.5.0
|
||||
jinja2==3.1.2
|
||||
python-multipart==0.0.6
|
||||
aiofiles==23.2.1
|
||||
```
|
||||
|
||||
**Key Change**: Changed `pydantic==2.5.0` to `pydantic>=2.0.0,<2.5.0` to avoid the compatibility issue with `garth`.
|
||||
|
||||
## Code Issues Found and Fixes
|
||||
|
||||
### 1. Missing utils.py File
|
||||
Your `daemon.py` imports `from .utils import logger` but this file doesn't exist.
|
||||
|
||||
**Fix**: Create `garminsync/utils.py`:
|
||||
```python
|
||||
import logging
|
||||
|
||||
# Configure logging
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
|
||||
)
|
||||
|
||||
logger = logging.getLogger('garminsync')
|
||||
```
|
||||
|
||||
### 2. Daemon.py Import Issues
|
||||
The `daemon.py` file has several import and method call issues.
|
||||
|
||||
**Fix for garminsync/daemon.py** (line 56-75):
|
||||
```python
|
||||
def sync_and_download(self):
|
||||
"""Scheduled job function"""
|
||||
try:
|
||||
self.log_operation("sync", "started")
|
||||
|
||||
# Import here to avoid circular imports
|
||||
from .garmin import GarminClient
|
||||
from .database import sync_database
|
||||
|
||||
# Perform sync and download
|
||||
client = GarminClient()
|
||||
|
||||
# Sync database first
|
||||
sync_database(client)
|
||||
|
||||
# Download missing activities
|
||||
downloaded_count = 0
|
||||
session = get_session()
|
||||
missing_activities = session.query(Activity).filter_by(downloaded=False).all()
|
||||
|
||||
for activity in missing_activities:
|
||||
try:
|
||||
# Use the correct method name
|
||||
fit_data = client.download_activity_fit(activity.activity_id)
|
||||
|
||||
# Save the file
|
||||
import os
|
||||
from pathlib import Path
|
||||
data_dir = Path(os.getenv("DATA_DIR", "data"))
|
||||
data_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
timestamp = activity.start_time.replace(":", "-").replace(" ", "_")
|
||||
filename = f"activity_{activity.activity_id}_{timestamp}.fit"
|
||||
filepath = data_dir / filename
|
||||
|
||||
with open(filepath, "wb") as f:
|
||||
f.write(fit_data)
|
||||
|
||||
activity.filename = str(filepath)
|
||||
activity.downloaded = True
|
||||
activity.last_sync = datetime.now().isoformat()
|
||||
downloaded_count += 1
|
||||
session.commit()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to download activity {activity.activity_id}: {e}")
|
||||
session.rollback()
|
||||
|
||||
session.close()
|
||||
self.log_operation("sync", "success",
|
||||
f"Downloaded {downloaded_count} new activities")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Sync failed: {e}")
|
||||
self.log_operation("sync", "error", str(e))
|
||||
```
|
||||
|
||||
### 3. Missing created_at Field in Database Sync
|
||||
The `sync_database` function in `database.py` doesn't set the `created_at` field.
|
||||
|
||||
**Fix for garminsync/database.py** (line 64-75):
|
||||
```python
|
||||
def sync_database(garmin_client):
|
||||
"""Sync local database with Garmin Connect activities"""
|
||||
from datetime import datetime
|
||||
session = get_session()
|
||||
try:
|
||||
# Fetch activities from Garmin Connect
|
||||
activities = garmin_client.get_activities(0, 1000)
|
||||
|
||||
# Process activities and update database
|
||||
for activity in activities:
|
||||
activity_id = activity["activityId"]
|
||||
start_time = activity["startTimeLocal"]
|
||||
|
||||
# Check if activity exists in database
|
||||
existing = session.query(Activity).filter_by(activity_id=activity_id).first()
|
||||
if not existing:
|
||||
new_activity = Activity(
|
||||
activity_id=activity_id,
|
||||
start_time=start_time,
|
||||
downloaded=False,
|
||||
created_at=datetime.now().isoformat(), # Add this line
|
||||
last_sync=datetime.now().isoformat()
|
||||
)
|
||||
session.add(new_activity)
|
||||
|
||||
session.commit()
|
||||
except SQLAlchemyError as e:
|
||||
session.rollback()
|
||||
raise e
|
||||
finally:
|
||||
session.close()
|
||||
```
|
||||
|
||||
### 4. Add Missing created_at Field to Database Model
|
||||
The `Activity` model is missing the `created_at` field that's used in the daemon.
|
||||
|
||||
**Fix for garminsync/database.py** (line 12):
|
||||
```python
|
||||
class Activity(Base):
|
||||
__tablename__ = 'activities'
|
||||
|
||||
activity_id = Column(Integer, primary_key=True)
|
||||
start_time = Column(String, nullable=False)
|
||||
filename = Column(String, unique=True, nullable=True)
|
||||
downloaded = Column(Boolean, default=False, nullable=False)
|
||||
created_at = Column(String, nullable=False) # Add this line
|
||||
last_sync = Column(String, nullable=True) # ISO timestamp of last sync
|
||||
```
|
||||
|
||||
### 5. JavaScript Function Missing in Dashboard
|
||||
The dashboard template calls `toggleDaemon()` but this function doesn't exist in the JavaScript.
|
||||
|
||||
**Fix for garminsync/web/static/app.js** (add this function):
|
||||
```javascript
|
||||
async function toggleDaemon() {
|
||||
// TODO: Implement daemon toggle functionality
|
||||
alert('Daemon toggle functionality not yet implemented');
|
||||
}
|
||||
```
|
||||
|
||||
## Testing the Fixes
|
||||
|
||||
After applying these fixes:
|
||||
|
||||
1. **Rebuild the Docker image**:
|
||||
```bash
|
||||
docker build -t garminsync .
|
||||
```
|
||||
|
||||
2. **Test the daemon mode**:
|
||||
```bash
|
||||
docker run -d --env-file .env -v $(pwd)/data:/app/data -p 8080:8080 garminsync daemon --start
|
||||
```
|
||||
|
||||
3. **Check the logs**:
|
||||
```bash
|
||||
docker logs <container_id>
|
||||
```
|
||||
|
||||
4. **Access the web UI**:
|
||||
Open http://localhost:8080 in your browser
|
||||
|
||||
## Additional Recommendations
|
||||
|
||||
1. **Add error handling for missing directories**: The daemon should create the data directory if it doesn't exist.
|
||||
|
||||
2. **Improve logging**: Add more detailed logging throughout the application.
|
||||
|
||||
3. **Add health checks**: Implement health check endpoints for the daemon.
|
||||
|
||||
4. **Database migrations**: Consider adding database migration support for schema changes.
|
||||
|
||||
The primary fix for your immediate issue is updating the `pydantic` version constraint in `requirements.txt`. The other fixes address various code quality and functionality issues I found during the review.
|
||||
13
requirements.txt
Normal file
13
requirements.txt
Normal file
@@ -0,0 +1,13 @@
|
||||
typer==0.9.0
|
||||
click==8.1.7
|
||||
python-dotenv==1.0.0
|
||||
garminconnect==0.2.28
|
||||
sqlalchemy==2.0.23
|
||||
tqdm==4.66.1
|
||||
fastapi==0.104.1
|
||||
uvicorn[standard]==0.24.0
|
||||
apscheduler==3.10.4
|
||||
pydantic>=2.0.0,<2.5.0
|
||||
jinja2==3.1.2
|
||||
python-multipart==0.0.6
|
||||
aiofiles==23.2.1
|
||||
Reference in New Issue
Block a user