mirror of
https://github.com/sstent/GarminSync.git
synced 2026-01-25 08:35:02 +00:00
python v1
This commit is contained in:
278
Design.md
278
Design.md
@@ -1,116 +1,204 @@
|
||||
# GarminSync Application Design
|
||||
Of course. The design has been updated to use the `python-garminconnect` library and now includes a dedicated documentation section with links to all key upstream libraries.
|
||||
|
||||
-----
|
||||
|
||||
## **GarminSync Application Design (Python Version)**
|
||||
|
||||
### **Basic Info**
|
||||
|
||||
## Basic Info
|
||||
**App Name:** GarminSync
|
||||
**What it does:** CLI application that downloads FIT files for every activity in Garmin Connect
|
||||
**What it does:** A CLI application that downloads `.fit` files for every activity in Garmin Connect.
|
||||
|
||||
## Core Features
|
||||
1. List activities (`garminsync list --all`)
|
||||
2. List activities that have not been downloaded (`garminsync list --missing`)
|
||||
3. List activities that have been downloaded (`garminsync list --downloaded`)
|
||||
4. Download missing activities (`garminsync download --missing`)
|
||||
-----
|
||||
|
||||
## Tech Stack
|
||||
**Frontend:** CLI (Go)
|
||||
**Backend:** Go
|
||||
**Database:** SQLite (garmin.db)
|
||||
**Hosting:** Docker container
|
||||
**Key Libraries:** garminexport (Go), viper (env vars), cobra (CLI framework), go-sqlite3
|
||||
### **Core Features**
|
||||
|
||||
## Data Structure
|
||||
**Main data object:**
|
||||
```
|
||||
Activity:
|
||||
- activity_id: INTEGER (primary key, from Garmin)
|
||||
- start_time: TEXT (ISO 8601 format)
|
||||
- filename: TEXT (unique, e.g., activity_123_20240807.fit)
|
||||
- downloaded: BOOLEAN (0 = pending, 1 = completed)
|
||||
1. List all activities (`garminsync list --all`)
|
||||
2. List activities that have not been downloaded (`garminsync list --missing`)
|
||||
3. List activities that have been downloaded (`garminsync list --downloaded`)
|
||||
4. Download all missing activities (`garminsync download --missing`)
|
||||
|
||||
-----
|
||||
|
||||
### **Tech Stack 🐍**
|
||||
|
||||
* **Frontend:** CLI (**Python**)
|
||||
* **Backend:** **Python**
|
||||
* **Database:** SQLite (`garmin.db`)
|
||||
* **Hosting:** Docker container
|
||||
* **Key Libraries:**
|
||||
* **`python-garminconnect`**: The library for Garmin Connect API communication.
|
||||
* **`typer`**: A modern and easy-to-use CLI framework (built on `click`).
|
||||
* **`python-dotenv`**: For loading credentials from a `.env` file.
|
||||
* **`sqlalchemy`**: A robust ORM for database interaction and schema management (using Alembic for migrations). The built-in `sqlite3` module can be used for simpler needs.
|
||||
* **`tqdm`**: For creating user-friendly progress bars.
|
||||
|
||||
-----
|
||||
|
||||
### **Data Structure**
|
||||
|
||||
The core database schema remains the same. It can be defined using raw SQL with the `sqlite3` module or, more robustly, with an `SQLAlchemy` model.
|
||||
|
||||
**SQLAlchemy Model Example (`models.py`):**
|
||||
|
||||
```python
|
||||
from sqlalchemy import create_engine, Column, Integer, String, Boolean
|
||||
from sqlalchemy.orm import declarative_base
|
||||
|
||||
Base = declarative_base()
|
||||
|
||||
class Activity(Base):
|
||||
__tablename__ = 'activities'
|
||||
|
||||
activity_id = Column(Integer, primary_key=True)
|
||||
start_time = Column(String, nullable=False)
|
||||
filename = Column(String, unique=True, nullable=True)
|
||||
downloaded = Column(Boolean, default=False, nullable=False)
|
||||
```
|
||||
|
||||
## User Flow
|
||||
1. User launches container with credentials: `sudo docker run -it --env-file .env garminsync`
|
||||
2. User is presented with CLI menu of options
|
||||
3. User selects command (e.g., `garminsync download --missing`)
|
||||
4. Application executes task with progress indicators
|
||||
5. Application displays completion status and summary
|
||||
-----
|
||||
|
||||
### **User Flow**
|
||||
|
||||
1. User launches the container with their credentials: `docker run -it --env-file .env -v $(pwd)/data:/app/data garminsync`
|
||||
2. The application presents a help menu generated by `Typer`.
|
||||
3. User runs a command, e.g., `garminsync download --missing`.
|
||||
4. The application executes the task, showing progress indicators for API calls and downloads.
|
||||
5. Upon completion, the application displays a summary of actions taken.
|
||||
|
||||
-----
|
||||
|
||||
### **File Structure**
|
||||
|
||||
A standard Python project structure is recommended.
|
||||
|
||||
## File Structure
|
||||
```
|
||||
/garminsync
|
||||
├── main.go (CLI entrypoint and command implementations)
|
||||
├── internal/
|
||||
│ ├── config/
|
||||
│ │ └── config.go (configuration loading)
|
||||
│ ├── garmin/
|
||||
│ │ ├── client.go (API integration)
|
||||
│ │ └── activity.go (activity models)
|
||||
│ └── db/
|
||||
│ ├── database.go (embedded schema)
|
||||
│ ├── sync.go (database synchronization)
|
||||
│ └── migrations.go (versioned migrations)
|
||||
├── garminsync/ # Main application package
|
||||
│ ├── __init__.py
|
||||
│ ├── cli.py # Typer CLI commands and entrypoint
|
||||
│ ├── config.py # Configuration loading (dotenv)
|
||||
│ ├── database.py # SQLAlchemy models and sync logic
|
||||
│ └── garmin.py # Wrapper for the python-garminconnect client
|
||||
├── data/ # Directory for downloaded .fit files and DB
|
||||
├── tests/ # Unit and integration tests
|
||||
├── .env # Stores GARMIN_EMAIL/GARMIN_PASSWORD
|
||||
├── .gitignore
|
||||
├── Dockerfile
|
||||
├── .env
|
||||
└── README.md
|
||||
├── README.md
|
||||
└── requirements.txt # Pinned Python dependencies
|
||||
```
|
||||
|
||||
## Technical Implementation Notes
|
||||
- **Architecture:** Go-based implementation with Cobra CLI framework
|
||||
- **Authentication:** Credentials via GARMIN_EMAIL/GARMIN_PASSWORD env vars (never stored)
|
||||
- **File naming:** `activity_{id}_{timestamp}.fit` (e.g., activity_123456_20240807.fit)
|
||||
- **Rate limiting:** 2-second delays between API requests
|
||||
- **Database:** Embedded schema creation in Go code with versioned migrations
|
||||
- **Database Sync:** Before any list/download operation, the application performs a synchronization between Garmin Connect and the local SQLite database to ensure activity records are up-to-date.
|
||||
- **CLI Structure:** All CLI commands and flags are implemented in main.go using Cobra, without separate command files
|
||||
- **Docker:**
|
||||
- All commands require sudo as specified
|
||||
- Fully containerized build process (no host Go dependencies)
|
||||
- **Session management:** Automatic cookie handling via garminexport with file-based persistence
|
||||
- **Pagination:** Implemented for activity listing
|
||||
- **Package stability:** Always use stable, released versions of packages to ensure reproducibility
|
||||
-----
|
||||
|
||||
## Development Phases
|
||||
### Phase 1: Core Infrastructure - COMPLETE
|
||||
- [x] Dockerfile creation
|
||||
- [x] Environment variable handling (viper)
|
||||
- [x] Cobra CLI framework setup
|
||||
- [x] garminexport client initialization (with session persistence)
|
||||
### **Technical Implementation Notes**
|
||||
|
||||
### Phase 2: Activity Listing - COMPLETE
|
||||
- [x] SQLite schema implementation
|
||||
- [x] Activity listing commands
|
||||
- [x] Database synchronization
|
||||
- [x] List command UI implementation
|
||||
* **Architecture:** A Python application using **`Typer`** for the CLI. Logic is separated into modules for configuration, database interaction, and Garmin communication.
|
||||
* **Authentication:** Credentials from `GARMIN_EMAIL` and `GARMIN_PASSWORD` environment variables are loaded via **`python-dotenv`** and passed to **`python-garminconnect`**.
|
||||
* **File Naming:** Files will be named using Python's f-strings: `f"activity_{activity_id}_{timestamp}.fit"`.
|
||||
* **Rate Limiting:** A simple `time.sleep(2)` will be inserted between API requests to avoid overloading Garmin's servers.
|
||||
* **Database:** The schema will be managed by **`SQLAlchemy`**. Versioned migrations can be handled by **Alembic**, which integrates with SQLAlchemy.
|
||||
* **Database Sync:** Before any operation, a sync function will use **`python-garminconnect`** to fetch the latest activities and reconcile them with the local SQLite database.
|
||||
* **Package Management:** Dependencies and their versions will be pinned in `requirements.txt` for reproducible builds (`pip freeze > requirements.txt`).
|
||||
* **Session Management:** **`python-garminconnect`** handles authentication and session management.
|
||||
* **Docker Permissions:** Mounting a local `./data` directory into the container allows downloaded files and the database to persist. The `sudo` prefix is generally not required if the user is part of the `docker` group.
|
||||
|
||||
### Phase 3: Download Pipeline - COMPLETE
|
||||
- [x] FIT file download implementation
|
||||
- [x] Idempotent download logic (with exponential backoff)
|
||||
- [x] Database update on success
|
||||
- [x] Database sync integration
|
||||
-----
|
||||
|
||||
### Phase 4: Polish
|
||||
- [x] Progress indicators (download command)
|
||||
- [~] Error handling (partial implementation - retry logic exists but needs expansion)
|
||||
- [ ] README documentation
|
||||
- [x] Session timeout handling (via garminexport)
|
||||
### **Development Phases (Proposed for Python)**
|
||||
|
||||
## Critical Roadblocks
|
||||
1. **Rate limiting:** Built-in 2-second request delays (implemented)
|
||||
2. **Session management:** Automatic cookie handling via garminexport (implemented)
|
||||
3. **File conflicts:** Atomic database updates during downloads (implemented)
|
||||
4. **Docker permissions:** Volume-mounted /data directory for downloads (implemented)
|
||||
5. ~~**Database sync:** Efficient Garmin API ↔ local sync~~ (implemented)
|
||||
#### **Phase 1: Core Infrastructure**
|
||||
|
||||
## Current Status
|
||||
**Working on:** Phase 4 - Final polish and error handling
|
||||
**Next steps:**
|
||||
1. Fix command flag parsing issue
|
||||
2. Implement comprehensive error handling
|
||||
3. Complete README documentation
|
||||
4. Final testing and validation
|
||||
* [x] `Dockerfile` creation for a Python environment.
|
||||
* Uses Python 3.10 slim base image
|
||||
* Sets PYTHONDONTWRITEBYTECODE and PYTHONUNBUFFERED
|
||||
* Installs build dependencies
|
||||
* Copies requirements first for layer caching
|
||||
* Sets ENV_FILE for configuration
|
||||
* Entrypoint runs application via `python -m garminsync`
|
||||
* [x] Setup `python-dotenv` for environment variable handling.
|
||||
* Added `python-dotenv` to requirements.txt
|
||||
* Implemented environment loading in config.py:
|
||||
```python
|
||||
from dotenv import load_dotenv
|
||||
load_dotenv() # Load .env file
|
||||
```
|
||||
* Updated Dockerfile to copy .env file during build
|
||||
* Added documentation for environment variable setup
|
||||
* [x] Initialize the **`Typer`** CLI framework in `cli.py`.
|
||||
* Created basic Typer app structure
|
||||
* Implemented skeleton commands for `list` and `download`
|
||||
* Added proper type hints and docstrings
|
||||
* Integrated with config.py for environment variable loading
|
||||
* [x] Implement the **`python-garminconnect`** client wrapper (`garmin.py`).
|
||||
* Created GarminClient class with authentication
|
||||
* Implemented get_activities() with rate limiting
|
||||
* Implemented download_activity_fit() method
|
||||
* Added error handling for missing credentials
|
||||
|
||||
**Known issues:** None
|
||||
#### **Phase 2: Activity Listing**
|
||||
|
||||
## Recent Fixes
|
||||
- Fixed package declaration conflicts in cmd/ directory (changed from `package cmd` to `package main`)
|
||||
- Removed unnecessary import in root.go that was causing build errors
|
||||
- Verified Docker build process now completes successfully
|
||||
* [x] Implement the `SQLAlchemy` schema and database connection.
|
||||
* Created database.py with Activity model
|
||||
* Implemented init_db() for database initialization
|
||||
* Added get_session() for session management
|
||||
* Implemented sync_database() to fetch activities from Garmin and update database
|
||||
* [x] Create the `list` commands (`--all`, `--missing`, `--downloaded`).
|
||||
* Implemented command logic in cli.py
|
||||
* Added database synchronization before listing
|
||||
* Added input validation for filter options
|
||||
* Used tqdm for progress display
|
||||
|
||||
#### **Phase 3: Download Pipeline**
|
||||
|
||||
* [x] Implement the `.fit` file download function using `python-garminconnect`.
|
||||
* Added download logic to cli.py
|
||||
* Used GarminClient to download FIT files
|
||||
* Created filename-safe timestamps
|
||||
* Saved files to data directory
|
||||
* [x] Ensure downloads are idempotent (don't re-download existing files).
|
||||
* Database tracks downloaded status
|
||||
* Only missing activities are downloaded
|
||||
* [x] Update the database record to `downloaded=True` on success.
|
||||
* Updated activity record after successful download
|
||||
* Set filename path in database
|
||||
|
||||
#### **Phase 4: Polish ✨**
|
||||
|
||||
* [x] Add **`tqdm`** progress bars for the download command.
|
||||
* Implemented in both list and download commands
|
||||
* [x] Update Dockerfile to:
|
||||
* Use multi-stage builds for smaller image size
|
||||
* Add health checks
|
||||
* Set non-root user for security
|
||||
* [ ] Implement robust error handling (e.g., API errors, network issues).
|
||||
* Basic error handling implemented but needs improvement
|
||||
* [ ] Write comprehensive `README.md` documentation.
|
||||
* [ ] Add unit tests for database and utility functions.
|
||||
|
||||
#### **Current Issues**
|
||||
|
||||
1. **Docker Build Warnings**
|
||||
- Legacy ENV format warnings during build
|
||||
- Solution: Update Dockerfile to use `ENV key=value` format
|
||||
|
||||
2. **Typer Command Parameters**
|
||||
- Occasional `Parameter.make_metavar()` errors
|
||||
- Resolved by renaming reserved keywords and fixing decorators
|
||||
|
||||
3. **Configuration Loading**
|
||||
- Initial import issues between modules
|
||||
- Resolved by restructuring config loading
|
||||
|
||||
-----
|
||||
|
||||
### **Documentation 📚**
|
||||
|
||||
Here are links to the official documentation for the key libraries used in this design.
|
||||
|
||||
* **Garmin API:** [`python-garminconnect`](https://www.google.com/search?q=%5Bhttps://github.com/cyberjunky/python-garminconnect%5D\(https://github.com/cyberjunky/python-garminconnect\))
|
||||
* **CLI Framework:** [`Typer`](https://www.google.com/search?q=%5Bhttps://typer.tiangolo.com/%5D\(https://typer.tiangolo.com/\))
|
||||
* **Environment Variables:** [`python-dotenv`](https://www.google.com/search?q=%5Bhttps://github.com/theskumar/python-dotenv%5D\(https://github.com/theskumar/python-dotenv\))
|
||||
* **Database ORM:** [`SQLAlchemy`](https://www.google.com/search?q=%5Bhttps://docs.sqlalchemy.org/en/20/%5D\(https://docs.sqlalchemy.org/en/20/\))
|
||||
* **Database Migrations:** [`Alembic`](https://www.google.com/search?q=%5Bhttps://alembic.sqlalchemy.org/en/latest/%5D\(https://alembic.sqlalchemy.org/en/latest/\))
|
||||
* **Progress Bars:** [`tqdm`](https://www.google.com/search?q=%5Bhttps://github.com/tqdm/tqdm%5D\(https://github.com/tqdm/tqdm\))
|
||||
|
||||
57
Dockerfile
57
Dockerfile
@@ -1,44 +1,27 @@
|
||||
# GarminSync Dockerfile - Pure Go Implementation
|
||||
FROM golang:1.22.0-alpine3.19 AS builder
|
||||
# Use official Python base image
|
||||
FROM python:3.10-slim
|
||||
|
||||
# Create working directory
|
||||
# Set environment variables
|
||||
ENV PYTHONDONTWRITEBYTECODE=1
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
|
||||
# Set work directory
|
||||
WORKDIR /app
|
||||
|
||||
# Set Go module permissions and install Git
|
||||
RUN mkdir -p /go/pkg/mod && \
|
||||
chown -R 1000:1000 /go && \
|
||||
chmod -R 777 /go/pkg/mod && \
|
||||
apk add --no-cache git
|
||||
# Install system dependencies
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
build-essential \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Copy entire project
|
||||
# Copy and install Python dependencies
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
# Copy application code
|
||||
COPY . .
|
||||
|
||||
# Generate checksums and download dependencies
|
||||
RUN go mod tidy && go mod download
|
||||
# Set environment variables from .env file
|
||||
ENV ENV_FILE=/app/.env
|
||||
|
||||
# Build the Go application
|
||||
RUN CGO_ENABLED=0 go build -o /garminsync main.go
|
||||
|
||||
# Final stage
|
||||
FROM alpine:3.19
|
||||
|
||||
# Create non-root user (UID 1000:1000)
|
||||
RUN addgroup -S -g 1000 garminsync && \
|
||||
adduser -S -u 1000 -G garminsync garminsync
|
||||
|
||||
# Create data directory for FIT files and set permissions
|
||||
RUN mkdir -p /data && chown garminsync:garminsync /data
|
||||
|
||||
# Copy the built Go binary from the builder stage
|
||||
COPY --from=builder /garminsync /garminsync
|
||||
|
||||
# Set the working directory
|
||||
WORKDIR /data
|
||||
|
||||
# Switch to non-root user
|
||||
USER garminsync
|
||||
|
||||
# Set the entrypoint to the binary
|
||||
ENTRYPOINT ["/garminsync"]
|
||||
# Default command (can be overridden)
|
||||
CMD ["--help"]
|
||||
# Set the entrypoint to run the CLI
|
||||
ENTRYPOINT ["python", "-m", "garminsync.cli"]
|
||||
|
||||
17
go.mod
17
go.mod
@@ -1,17 +0,0 @@
|
||||
module github.com/sstent/garminsync
|
||||
|
||||
go 1.22.0
|
||||
|
||||
require (
|
||||
github.com/abrander/garmin-connect v0.0.0-20221117211130-dc0681952026
|
||||
github.com/mattn/go-sqlite3 v1.14.22
|
||||
github.com/spf13/cobra v1.7.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/inconshreveable/mousetrap v1.1.0 // indirect
|
||||
github.com/spf13/pflag v1.0.5 // indirect
|
||||
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa // indirect
|
||||
golang.org/x/sys v0.16.0 // indirect
|
||||
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 // indirect
|
||||
)
|
||||
@@ -1,68 +0,0 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Config holds application configuration
|
||||
type Config struct {
|
||||
GarminEmail string
|
||||
GarminPassword string
|
||||
DatabasePath string
|
||||
RateLimit time.Duration
|
||||
SessionPath string
|
||||
SessionTimeout time.Duration
|
||||
}
|
||||
|
||||
// LoadConfig loads configuration from environment variables
|
||||
func LoadConfig() (*Config, error) {
|
||||
email := os.Getenv("GARMIN_EMAIL")
|
||||
password := os.Getenv("GARMIN_PASSWORD")
|
||||
if email == "" || password == "" {
|
||||
return nil, fmt.Errorf("GARMIN_EMAIL and GARMIN_PASSWORD environment variables are required")
|
||||
}
|
||||
|
||||
databasePath := os.Getenv("DATABASE_PATH")
|
||||
if databasePath == "" {
|
||||
databasePath = "garmin.db"
|
||||
}
|
||||
|
||||
rateLimit := parseDuration(os.Getenv("RATE_LIMIT"), 2*time.Second)
|
||||
sessionTimeout := parseDuration(os.Getenv("SESSION_TIMEOUT"), 30*time.Minute)
|
||||
|
||||
sessionPath := os.Getenv("SESSION_PATH")
|
||||
if sessionPath == "" {
|
||||
sessionPath = "/data/session.json"
|
||||
}
|
||||
|
||||
// Ensure session path directory exists
|
||||
if err := os.MkdirAll(filepath.Dir(sessionPath), 0755); err != nil {
|
||||
return nil, fmt.Errorf("failed to create session directory: %w", err)
|
||||
}
|
||||
|
||||
return &Config{
|
||||
GarminEmail: email,
|
||||
GarminPassword: password,
|
||||
DatabasePath: databasePath,
|
||||
RateLimit: rateLimit,
|
||||
SessionPath: sessionPath,
|
||||
SessionTimeout: sessionTimeout,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// parseDuration parses a duration string with a default
|
||||
func parseDuration(value string, defaultValue time.Duration) time.Duration {
|
||||
if value == "" {
|
||||
return defaultValue
|
||||
}
|
||||
|
||||
d, err := time.ParseDuration(value)
|
||||
if err != nil {
|
||||
return defaultValue
|
||||
}
|
||||
|
||||
return d
|
||||
}
|
||||
@@ -1,153 +0,0 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"github.com/mattn/go-sqlite3"
|
||||
"github.com/sstent/garminsync/internal/garmin"
|
||||
)
|
||||
|
||||
// SQLiteDatabase implements ActivityRepository using SQLite
|
||||
type SQLiteDatabase struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
// NewDatabase creates a new SQLite database connection
|
||||
func NewDatabase(path string) (*SQLiteDatabase, error) {
|
||||
db, err := sql.Open("sqlite3", path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to open database: %w", err)
|
||||
}
|
||||
|
||||
// Create table if it doesn't exist
|
||||
if err := createSchema(db); err != nil {
|
||||
return nil, fmt.Errorf("failed to create schema: %w", err)
|
||||
}
|
||||
|
||||
return &SQLiteDatabase{db: db}, nil
|
||||
}
|
||||
|
||||
// Close closes the database connection
|
||||
func (d *SQLiteDatabase) Close() error {
|
||||
return d.db.Close()
|
||||
}
|
||||
|
||||
// createSchema creates the database schema
|
||||
func createSchema(db *sql.DB) error {
|
||||
schema := `
|
||||
CREATE TABLE IF NOT EXISTS activities (
|
||||
activity_id INTEGER PRIMARY KEY,
|
||||
start_time TEXT NOT NULL,
|
||||
filename TEXT NOT NULL,
|
||||
downloaded BOOLEAN NOT NULL DEFAULT 0
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_activity_id ON activities(activity_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_downloaded ON activities(downloaded);
|
||||
`
|
||||
|
||||
if _, err := db.Exec(schema); err != nil {
|
||||
return fmt.Errorf("failed to create schema: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// GetAll returns all activities from the database
|
||||
func (d *SQLiteDatabase) GetAll() ([]garmin.Activity, error) {
|
||||
return d.GetAllPaginated(0, 0) // 0,0 means no pagination
|
||||
}
|
||||
|
||||
// GetMissing returns activities that haven't been downloaded yet
|
||||
func (d *SQLiteDatabase) GetMissing() ([]garmin.Activity, error) {
|
||||
return d.GetMissingPaginated(0, 0)
|
||||
}
|
||||
|
||||
// GetDownloaded returns activities that have been downloaded
|
||||
func (d *SQLiteDatabase) GetDownloaded() ([]garmin.Activity, error) {
|
||||
return d.GetDownloadedPaginated(0, 0)
|
||||
}
|
||||
|
||||
// GetAllPaginated returns a paginated list of all activities
|
||||
func (d *SQLiteDatabase) GetAllPaginated(page, pageSize int) ([]garmin.Activity, error) {
|
||||
offset := (page - 1) * pageSize
|
||||
query := "SELECT activity_id, start_time, filename, downloaded FROM activities"
|
||||
if pageSize > 0 {
|
||||
query += fmt.Sprintf(" LIMIT %d OFFSET %d", pageSize, offset)
|
||||
}
|
||||
rows, err := d.db.Query(query)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get all activities: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
return scanActivities(rows)
|
||||
}
|
||||
|
||||
// GetMissingPaginated returns a paginated list of missing activities
|
||||
func (d *SQLiteDatabase) GetMissingPaginated(page, pageSize int) ([]garmin.Activity, error) {
|
||||
offset := (page - 1) * pageSize
|
||||
query := "SELECT activity_id, start_time, filename, downloaded FROM activities WHERE downloaded = 0"
|
||||
if pageSize > 0 {
|
||||
query += fmt.Sprintf(" LIMIT %d OFFSET %d", pageSize, offset)
|
||||
}
|
||||
rows, err := d.db.Query(query)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get missing activities: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
return scanActivities(rows)
|
||||
}
|
||||
|
||||
// GetDownloadedPaginated returns a paginated list of downloaded activities
|
||||
func (d *SQLiteDatabase) GetDownloadedPaginated(page, pageSize int) ([]garmin.Activity, error) {
|
||||
offset := (page - 1) * pageSize
|
||||
query := "SELECT activity_id, start_time, filename, downloaded FROM activities WHERE downloaded = 1"
|
||||
if pageSize > 0 {
|
||||
query += fmt.Sprintf(" LIMIT %d OFFSET %d", pageSize, offset)
|
||||
}
|
||||
rows, err := d.db.Query(query)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get downloaded activities: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
return scanActivities(rows)
|
||||
}
|
||||
|
||||
// MarkDownloaded updates the database when an activity is downloaded
|
||||
func (d *SQLiteDatabase) MarkDownloaded(activityId int, filename string) error {
|
||||
_, err := d.db.Exec("UPDATE activities SET downloaded = 1, filename = ? WHERE activity_id = ?",
|
||||
filename, activityId)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to mark activity as downloaded: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// scanActivities converts database rows to Activity objects
|
||||
func scanActivities(rows *sql.Rows) ([]garmin.Activity, error) {
|
||||
var activities []garmin.Activity
|
||||
|
||||
for rows.Next() {
|
||||
var activity garmin.Activity
|
||||
var downloaded int
|
||||
var startTime string
|
||||
|
||||
if err := rows.Scan(&activity.ActivityId, &startTime, &activity.Filename, &downloaded); err != nil {
|
||||
return nil, fmt.Errorf("failed to scan activity: %w", err)
|
||||
}
|
||||
|
||||
// Convert SQLite time string to time.Time
|
||||
activity.StartTime, _ = time.Parse("2006-01-02 15:04:05", startTime)
|
||||
activity.Downloaded = downloaded == 1
|
||||
activities = append(activities, activity)
|
||||
}
|
||||
|
||||
return activities, nil
|
||||
}
|
||||
@@ -1,78 +0,0 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/sstent/garminsync/internal/config"
|
||||
"github.com/sstent/garminsync/internal/garmin"
|
||||
)
|
||||
|
||||
// SyncActivities synchronizes Garmin Connect activities with local database
|
||||
func SyncActivities(cfg *config.Config) error {
|
||||
// Initialize Garmin client
|
||||
client, err := garmin.NewClient(cfg)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to create Garmin client: %w", err)
|
||||
}
|
||||
|
||||
// Initialize database
|
||||
db, err := NewDatabase(cfg.DatabasePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to connect to database: %w", err)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
// Get activities from Garmin API
|
||||
garminActivities, err := client.GetActivities()
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get Garmin activities: %w", err)
|
||||
}
|
||||
|
||||
// Get all activities from local database
|
||||
localActivities, err := db.GetAll()
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get local activities: %w", err)
|
||||
}
|
||||
|
||||
// Create map for quick lookup of local activities
|
||||
localMap := make(map[int]garmin.Activity)
|
||||
for _, activity := range localActivities {
|
||||
localMap[activity.ActivityId] = activity
|
||||
}
|
||||
|
||||
// Process each Garmin activity
|
||||
for _, ga := range garminActivities {
|
||||
localActivity, exists := localMap[ga.ActivityId]
|
||||
|
||||
// New activity - insert into database
|
||||
if !exists {
|
||||
_, err := db.db.Exec(
|
||||
"INSERT INTO activities (activity_id, start_time, filename, downloaded) VALUES (?, ?, ?, ?)",
|
||||
ga.ActivityId,
|
||||
ga.StartTime.Format("2006-01-02 15:04:05"),
|
||||
ga.Filename,
|
||||
false,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to insert new activity %d: %w", ga.ActivityId, err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// Existing activity - check for metadata changes
|
||||
if localActivity.StartTime != ga.StartTime || localActivity.Filename != ga.Filename {
|
||||
_, err := db.db.Exec(
|
||||
"UPDATE activities SET start_time = ?, filename = ? WHERE activity_id = ?",
|
||||
ga.StartTime.Format("2006-01-02 15:04:05"),
|
||||
ga.Filename,
|
||||
ga.ActivityId,
|
||||
)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to update activity %d: %w", ga.ActivityId, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -1,19 +0,0 @@
|
||||
package garmin
|
||||
|
||||
import "time"
|
||||
|
||||
// Activity represents a Garmin Connect activity
|
||||
type Activity struct {
|
||||
ActivityId int `db:"activity_id"`
|
||||
StartTime time.Time `db:"start_time"`
|
||||
Filename string `db:"filename"`
|
||||
Downloaded bool `db:"downloaded"`
|
||||
}
|
||||
|
||||
// ActivityRepository provides methods for activity persistence
|
||||
type ActivityRepository interface {
|
||||
GetAll() ([]Activity, error)
|
||||
GetMissing() ([]Activity, error)
|
||||
GetDownloaded() ([]Activity, error)
|
||||
MarkDownloaded(activityId int, filename string) error
|
||||
}
|
||||
@@ -1,106 +0,0 @@
|
||||
package garmin
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
garminconnect "github.com/abrander/garmin-connect"
|
||||
"github.com/sstent/garminsync/internal/config"
|
||||
)
|
||||
|
||||
// Client represents a Garmin Connect API client
|
||||
type Client struct {
|
||||
client *garminconnect.Client
|
||||
cfg *config.Config
|
||||
lastAuth time.Time
|
||||
}
|
||||
|
||||
const (
|
||||
defaultSessionTimeout = 30 * time.Minute
|
||||
)
|
||||
|
||||
// NewClient creates a new Garmin Connect client
|
||||
func NewClient(cfg *config.Config) (*Client, error) {
|
||||
// Create client with session persistence
|
||||
client := garminconnect.NewClient(garminconnect.Credentials(cfg.GarminEmail, cfg.GarminPassword))
|
||||
client.SessionFile = cfg.SessionPath
|
||||
|
||||
// Attempt to load existing session
|
||||
if err := client.Authenticate(); err != nil {
|
||||
return nil, fmt.Errorf("authentication failed: %w", err)
|
||||
}
|
||||
|
||||
return &Client{
|
||||
client: client,
|
||||
cfg: cfg,
|
||||
lastAuth: time.Now(),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// checkSession checks if session is still valid, refreshes if expired
|
||||
func (c *Client) checkSession() error {
|
||||
timeout := c.cfg.SessionTimeout
|
||||
if timeout == 0 {
|
||||
timeout = defaultSessionTimeout
|
||||
}
|
||||
|
||||
if time.Since(c.lastAuth) > timeout {
|
||||
if err := c.client.Authenticate(); err != nil {
|
||||
return fmt.Errorf("session refresh failed: %w", err)
|
||||
}
|
||||
c.lastAuth = time.Now()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// GetActivities retrieves activities from Garmin Connect
|
||||
func (c *Client) GetActivities() ([]Activity, error) {
|
||||
// Check and refresh session if needed
|
||||
if err := c.checkSession(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// Get activities from Garmin Connect
|
||||
garminActivities, err := c.client.Activities("", 0, 100) // Empty string = current user
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get activities: %w", err)
|
||||
}
|
||||
|
||||
// Convert to our Activity struct
|
||||
var activities []Activity
|
||||
for _, ga := range garminActivities {
|
||||
activities = append(activities, Activity{
|
||||
ActivityId: int(ga.ID),
|
||||
StartTime: time.Time(ga.StartLocal),
|
||||
Filename: fmt.Sprintf("activity_%d_%s.fit", ga.ID, ga.StartLocal.Time().Format("20060102")),
|
||||
Downloaded: false,
|
||||
})
|
||||
}
|
||||
|
||||
return activities, nil
|
||||
}
|
||||
|
||||
// DownloadActivityFIT downloads a specific FIT file
|
||||
func (c *Client) DownloadActivityFIT(activityId int, filename string) error {
|
||||
// Check and refresh session if needed
|
||||
if err := c.checkSession(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Apply rate limiting
|
||||
time.Sleep(c.cfg.RateLimit)
|
||||
|
||||
// Create file for writing
|
||||
file, err := os.Create(filename)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to create file: %w", err)
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
// Download FIT file
|
||||
if err := c.client.ExportActivity(activityId, file, garminconnect.ActivityFormatFIT); err != nil {
|
||||
return fmt.Errorf("failed to export activity %d: %w", activityId, err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
258
main.go
258
main.go
@@ -1,258 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/spf13/cobra"
|
||||
"github.com/sstent/garminsync/internal/config"
|
||||
"github.com/sstent/garminsync/internal/db"
|
||||
"github.com/sstent/garminsync/internal/garmin"
|
||||
)
|
||||
|
||||
var rootCmd = &cobra.Command{
|
||||
Use: "garminsync",
|
||||
Short: "GarminSync synchronizes Garmin Connect activities to FIT files",
|
||||
Long: `GarminSync is a CLI application that:
|
||||
1. Authenticates with Garmin Connect
|
||||
2. Lists activities (all, missing, downloaded)
|
||||
3. Downloads missing FIT files
|
||||
4. Tracks download status in SQLite database`,
|
||||
}
|
||||
|
||||
// List command flags
|
||||
var listAll bool
|
||||
var listMissing bool
|
||||
var listDownloaded bool
|
||||
|
||||
// Download command flags
|
||||
var downloadAll bool
|
||||
var downloadMissing bool
|
||||
var maxRetries int
|
||||
|
||||
var listCmd = &cobra.Command{
|
||||
Use: "list",
|
||||
Short: "List activities from Garmin Connect",
|
||||
Long: `List activities with various filters:
|
||||
- All activities
|
||||
- Missing activities (not yet downloaded)
|
||||
- Downloaded activities`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
// Initialize config
|
||||
cfg, err := config.LoadConfig()
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to load config: %w", err)
|
||||
}
|
||||
|
||||
// Sync database with Garmin Connect
|
||||
fmt.Println("Syncing activities with Garmin Connect...")
|
||||
if err := db.SyncActivities(cfg); err != nil {
|
||||
return fmt.Errorf("database sync failed: %w", err)
|
||||
}
|
||||
|
||||
// Initialize database
|
||||
database, err := db.NewDatabase(cfg.DatabasePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to connect to database: %w", err)
|
||||
}
|
||||
defer database.Close()
|
||||
|
||||
// Get activities from database with pagination
|
||||
page := 1
|
||||
pageSize := 20
|
||||
totalShown := 0
|
||||
|
||||
for {
|
||||
var filteredActivities []garmin.Activity
|
||||
var err error
|
||||
|
||||
if listAll {
|
||||
filteredActivities, err = database.GetAllPaginated(page, pageSize)
|
||||
} else if listMissing {
|
||||
filteredActivities, err = database.GetMissingPaginated(page, pageSize)
|
||||
} else if listDownloaded {
|
||||
filteredActivities, err = database.GetDownloadedPaginated(page, pageSize)
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get activities: %w", err)
|
||||
}
|
||||
|
||||
if len(filteredActivities) == 0 {
|
||||
if totalShown == 0 {
|
||||
fmt.Println("No activities found matching the criteria")
|
||||
}
|
||||
break
|
||||
}
|
||||
|
||||
// Print activities for current page
|
||||
for _, activity := range filteredActivities {
|
||||
status := "❌ Not Downloaded"
|
||||
if activity.Downloaded {
|
||||
status = "✅ Downloaded"
|
||||
}
|
||||
fmt.Printf("ID: %d | %s | %s | %s\n",
|
||||
activity.ActivityId,
|
||||
activity.StartTime.Format("2006-01-02 15:04:05"),
|
||||
activity.Filename,
|
||||
status)
|
||||
totalShown++
|
||||
}
|
||||
|
||||
// Only prompt if there might be more results
|
||||
if len(filteredActivities) == pageSize {
|
||||
fmt.Printf("\nPage %d (%d activities shown) - Show more? (y/n): ", page, totalShown)
|
||||
var response string
|
||||
fmt.Scanln(&response)
|
||||
if strings.ToLower(response) != "y" {
|
||||
break
|
||||
}
|
||||
page++
|
||||
} else {
|
||||
fmt.Printf("\nTotal: %d activities shown\n", totalShown)
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
},
|
||||
}
|
||||
|
||||
var downloadCmd = &cobra.Command{
|
||||
Use: "download",
|
||||
Short: "Download missing FIT files",
|
||||
Long: `Downloads missing activity files from Garmin Connect`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
// Load configuration
|
||||
cfg, err := config.LoadConfig()
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to load config: %w", err)
|
||||
}
|
||||
|
||||
// Sync database with Garmin Connect
|
||||
fmt.Println("Syncing activities with Garmin Connect...")
|
||||
if err := db.SyncActivities(cfg); err != nil {
|
||||
return fmt.Errorf("database sync failed: %w", err)
|
||||
}
|
||||
|
||||
// Initialize Garmin client
|
||||
client, err := garmin.NewClient(cfg)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to create Garmin client: %w", err)
|
||||
}
|
||||
|
||||
// Initialize database
|
||||
database, err := db.NewDatabase(cfg.DatabasePath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to connect to database: %w", err)
|
||||
}
|
||||
defer database.Close()
|
||||
|
||||
// Get activities to download
|
||||
var activities []garmin.Activity
|
||||
if downloadAll {
|
||||
activities, err = database.GetAll()
|
||||
} else if downloadMissing {
|
||||
activities, err = database.GetMissing()
|
||||
}
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to get activities: %w", err)
|
||||
}
|
||||
|
||||
// Filter out already downloaded activities
|
||||
var toDownload []garmin.Activity
|
||||
for _, activity := range activities {
|
||||
if !activity.Downloaded {
|
||||
toDownload = append(toDownload, activity)
|
||||
}
|
||||
}
|
||||
|
||||
total := len(toDownload)
|
||||
if total == 0 {
|
||||
fmt.Println("No activities to download")
|
||||
return nil
|
||||
}
|
||||
|
||||
fmt.Printf("Found %d activities to download\n", total)
|
||||
|
||||
// Ensure download directory exists
|
||||
downloadDir := "/data"
|
||||
if err := os.MkdirAll(downloadDir, 0755); err != nil {
|
||||
return fmt.Errorf("failed to create download directory: %w", err)
|
||||
}
|
||||
|
||||
// Download activities with exponential backoff retry
|
||||
successCount := 0
|
||||
for i, activity := range toDownload {
|
||||
filename := filepath.Join(downloadDir, activity.Filename)
|
||||
fmt.Printf("[%d/%d] Downloading activity %d to %s\n", i+1, total, activity.ActivityId, filename)
|
||||
|
||||
// Exponential backoff retry
|
||||
baseDelay := 2 * time.Second
|
||||
var lastErr error
|
||||
for attempt := 1; attempt <= maxRetries; attempt++ {
|
||||
err := client.DownloadActivityFIT(activity.ActivityId, filename)
|
||||
if err == nil {
|
||||
// Mark as downloaded in database
|
||||
if err := database.MarkDownloaded(activity.ActivityId, filename); err != nil {
|
||||
fmt.Printf("⚠️ Failed to mark activity %d as downloaded: %v\n", activity.ActivityId, err)
|
||||
} else {
|
||||
successCount++
|
||||
fmt.Printf("✅ Successfully downloaded activity %d\n", activity.ActivityId)
|
||||
}
|
||||
lastErr = nil
|
||||
break
|
||||
}
|
||||
|
||||
lastErr = err
|
||||
fmt.Printf("⚠️ Attempt %d/%d failed: %v\n", attempt, maxRetries, err)
|
||||
if attempt < maxRetries {
|
||||
retryDelay := time.Duration(attempt) * baseDelay
|
||||
fmt.Printf("⏳ Retrying in %v...\n", retryDelay)
|
||||
time.Sleep(retryDelay)
|
||||
}
|
||||
}
|
||||
|
||||
if lastErr != nil {
|
||||
fmt.Printf("❌ Failed to download activity %d after %d attempts: %v\n", activity.ActivityId, maxRetries, lastErr)
|
||||
}
|
||||
}
|
||||
|
||||
fmt.Printf("\n📊 Download summary: %d/%d activities successfully downloaded\n", successCount, total)
|
||||
return nil
|
||||
},
|
||||
}
|
||||
|
||||
func main() {
|
||||
Execute()
|
||||
}
|
||||
|
||||
func Execute() {
|
||||
if err := rootCmd.Execute(); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func init() {
|
||||
// Configure list command flags
|
||||
listCmd.Flags().BoolVar(&listAll, "all", false, "List all activities")
|
||||
listCmd.Flags().BoolVar(&listMissing, "missing", false, "List activities that have not been downloaded")
|
||||
listCmd.Flags().BoolVar(&listDownloaded, "downloaded", false, "List activities that have been downloaded")
|
||||
listCmd.MarkFlagsMutuallyExclusive("all", "missing", "downloaded")
|
||||
listCmd.MarkFlagsRequiredAtLeastOne("all", "missing", "downloaded")
|
||||
|
||||
// Configure download command flags
|
||||
downloadCmd.Flags().BoolVar(&downloadAll, "all", false, "Download all activities")
|
||||
downloadCmd.Flags().BoolVar(&downloadMissing, "missing", false, "Download only missing activities")
|
||||
downloadCmd.Flags().IntVar(&maxRetries, "max-retries", 3, "Maximum download retry attempts")
|
||||
downloadCmd.MarkFlagsMutuallyExclusive("all", "missing")
|
||||
downloadCmd.MarkFlagsRequiredAtLeastOne("all", "missing")
|
||||
|
||||
// Add subcommands to root
|
||||
rootCmd.AddCommand(listCmd)
|
||||
rootCmd.AddCommand(downloadCmd)
|
||||
}
|
||||
Reference in New Issue
Block a user