sstent/GarminSync
1
0
Fork 0
mirror of https://github.com/sstent/GarminSync.git synced 2026-01-26 00:52:32 +00:00
Code Issues Packages Projects Releases Wiki Activity

python v1

Browse Source
This commit is contained in:
sstent
2025-08-08 12:33:11 -07:00
parent 7b9f0a7178
commit 0c13b92e5a
9 changed files with 203 additions and 831 deletions
Download Patch File Download Diff File Expand all files Collapse all files

278
Design.md
View File

@@ -1,116 +1,204 @@
# GarminSync Application Design
Of course. The design has been updated to use the `python-garminconnect` library and now includes a dedicated documentation section with links to all key upstream libraries.
-----
## **GarminSync Application Design (Python Version)**
### **Basic Info**
## Basic Info
**App Name:** GarminSync
**What it does:** CLI application that downloads FIT files for every activity in Garmin Connect
**What it does:** A CLI application that downloads `.fit` files for every activity in Garmin Connect.
## Core Features
1. List activities (`garminsync list --all`)
2. List activities that have not been downloaded (`garminsync list --missing`)
3. List activities that have been downloaded (`garminsync list --downloaded`)
4. Download missing activities (`garminsync download --missing`)
-----
## Tech Stack
**Frontend:** CLI (Go)
**Backend:** Go
**Database:** SQLite (garmin.db)
**Hosting:** Docker container
**Key Libraries:** garminexport (Go), viper (env vars), cobra (CLI framework), go-sqlite3
### **Core Features**
## Data Structure
**Main data object:**
```
Activity:
- activity_id: INTEGER (primary key, from Garmin)
- start_time: TEXT (ISO 8601 format)
- filename: TEXT (unique, e.g., activity_123_20240807.fit)
- downloaded: BOOLEAN (0 = pending, 1 = completed)
1. List all activities (`garminsync list --all`)
2. List activities that have not been downloaded (`garminsync list --missing`)
3. List activities that have been downloaded (`garminsync list --downloaded`)
4. Download all missing activities (`garminsync download --missing`)
-----
### **Tech Stack 🐍**
* **Frontend:** CLI (**Python**)
* **Backend:** **Python**
* **Database:** SQLite (`garmin.db`)
* **Hosting:** Docker container
* **Key Libraries:**
* **`python-garminconnect`**: The library for Garmin Connect API communication.
* **`typer`**: A modern and easy-to-use CLI framework (built on `click`).
* **`python-dotenv`**: For loading credentials from a `.env` file.
* **`sqlalchemy`**: A robust ORM for database interaction and schema management (using Alembic for migrations). The built-in `sqlite3` module can be used for simpler needs.
* **`tqdm`**: For creating user-friendly progress bars.
-----
### **Data Structure**
The core database schema remains the same. It can be defined using raw SQL with the `sqlite3` module or, more robustly, with an `SQLAlchemy` model.
**SQLAlchemy Model Example (`models.py`):**
```python
from sqlalchemy import create_engine, Column, Integer, String, Boolean
from sqlalchemy.orm import declarative_base
Base = declarative_base()
class Activity(Base):
__tablename__ = 'activities'
activity_id = Column(Integer, primary_key=True)
start_time = Column(String, nullable=False)
filename = Column(String, unique=True, nullable=True)
downloaded = Column(Boolean, default=False, nullable=False)
```
## User Flow
1. User launches container with credentials: `sudo docker run -it --env-file .env garminsync`
2. User is presented with CLI menu of options
3. User selects command (e.g., `garminsync download --missing`)
4. Application executes task with progress indicators
5. Application displays completion status and summary
-----
### **User Flow**
1. User launches the container with their credentials: `docker run -it --env-file .env -v $(pwd)/data:/app/data garminsync`
2. The application presents a help menu generated by `Typer`.
3. User runs a command, e.g., `garminsync download --missing`.
4. The application executes the task, showing progress indicators for API calls and downloads.
5. Upon completion, the application displays a summary of actions taken.
-----
### **File Structure**
A standard Python project structure is recommended.
## File Structure
```
/garminsync
├── main.go (CLI entrypoint and command implementations)
├── internal/
│ ├── config/
│ │ └── config.go (configuration loading)
│ ├── garmin/
│ │ ├── client.go (API integration)
│ │ └── activity.go (activity models)
│ └── db/
│ ├── database.go (embedded schema)
│ ├── sync.go (database synchronization)
│ └── migrations.go (versioned migrations)
├── garminsync/ # Main application package
│ ├── __init__.py
│ ├── cli.py # Typer CLI commands and entrypoint
│ ├── config.py # Configuration loading (dotenv)
│ ├── database.py # SQLAlchemy models and sync logic
│ └── garmin.py # Wrapper for the python-garminconnect client
├── data/ # Directory for downloaded .fit files and DB
├── tests/ # Unit and integration tests
├── .env # Stores GARMIN_EMAIL/GARMIN_PASSWORD
├── .gitignore
├── Dockerfile
├── .env
└── README.md
├── README.md
└── requirements.txt # Pinned Python dependencies
```
## Technical Implementation Notes
- **Architecture:** Go-based implementation with Cobra CLI framework
- **Authentication:** Credentials via GARMIN_EMAIL/GARMIN_PASSWORD env vars (never stored)
- **File naming:** `activity_{id}_{timestamp}.fit` (e.g., activity_123456_20240807.fit)
- **Rate limiting:** 2-second delays between API requests
- **Database:** Embedded schema creation in Go code with versioned migrations
- **Database Sync:** Before any list/download operation, the application performs a synchronization between Garmin Connect and the local SQLite database to ensure activity records are up-to-date.
- **CLI Structure:** All CLI commands and flags are implemented in main.go using Cobra, without separate command files
- **Docker:**
- All commands require sudo as specified
- Fully containerized build process (no host Go dependencies)
- **Session management:** Automatic cookie handling via garminexport with file-based persistence
- **Pagination:** Implemented for activity listing
- **Package stability:** Always use stable, released versions of packages to ensure reproducibility
-----
## Development Phases
### Phase 1: Core Infrastructure - COMPLETE
- [x] Dockerfile creation
- [x] Environment variable handling (viper)
- [x] Cobra CLI framework setup
- [x] garminexport client initialization (with session persistence)
### **Technical Implementation Notes**
### Phase 2: Activity Listing - COMPLETE
- [x] SQLite schema implementation
- [x] Activity listing commands
- [x] Database synchronization
- [x] List command UI implementation
* **Architecture:** A Python application using **`Typer`** for the CLI. Logic is separated into modules for configuration, database interaction, and Garmin communication.
* **Authentication:** Credentials from `GARMIN_EMAIL` and `GARMIN_PASSWORD` environment variables are loaded via **`python-dotenv`** and passed to **`python-garminconnect`**.
* **File Naming:** Files will be named using Python's f-strings: `f"activity_{activity_id}_{timestamp}.fit"`.
* **Rate Limiting:** A simple `time.sleep(2)` will be inserted between API requests to avoid overloading Garmin's servers.
* **Database:** The schema will be managed by **`SQLAlchemy`**. Versioned migrations can be handled by **Alembic**, which integrates with SQLAlchemy.
* **Database Sync:** Before any operation, a sync function will use **`python-garminconnect`** to fetch the latest activities and reconcile them with the local SQLite database.
* **Package Management:** Dependencies and their versions will be pinned in `requirements.txt` for reproducible builds (`pip freeze > requirements.txt`).
* **Session Management:** **`python-garminconnect`** handles authentication and session management.
* **Docker Permissions:** Mounting a local `./data` directory into the container allows downloaded files and the database to persist. The `sudo` prefix is generally not required if the user is part of the `docker` group.
### Phase 3: Download Pipeline - COMPLETE
- [x] FIT file download implementation
- [x] Idempotent download logic (with exponential backoff)
- [x] Database update on success
- [x] Database sync integration
-----
### Phase 4: Polish
- [x] Progress indicators (download command)
- [~] Error handling (partial implementation - retry logic exists but needs expansion)
- [ ] README documentation
- [x] Session timeout handling (via garminexport)
### **Development Phases (Proposed for Python)**
## Critical Roadblocks
1. **Rate limiting:** Built-in 2-second request delays (implemented)
2. **Session management:** Automatic cookie handling via garminexport (implemented)
3. **File conflicts:** Atomic database updates during downloads (implemented)
4. **Docker permissions:** Volume-mounted /data directory for downloads (implemented)
5. ~~**Database sync:** Efficient Garmin API ↔ local sync~~ (implemented)
#### **Phase 1: Core Infrastructure**
## Current Status
**Working on:** Phase 4 - Final polish and error handling
**Next steps:**
1. Fix command flag parsing issue
2. Implement comprehensive error handling
3. Complete README documentation
4. Final testing and validation
* [x] `Dockerfile` creation for a Python environment.
* Uses Python 3.10 slim base image
* Sets PYTHONDONTWRITEBYTECODE and PYTHONUNBUFFERED
* Installs build dependencies
* Copies requirements first for layer caching
* Sets ENV_FILE for configuration
* Entrypoint runs application via `python -m garminsync`
* [x] Setup `python-dotenv` for environment variable handling.
* Added `python-dotenv` to requirements.txt
* Implemented environment loading in config.py:
```python
from dotenv import load_dotenv
load_dotenv() # Load .env file
```
* Updated Dockerfile to copy .env file during build
* Added documentation for environment variable setup
* [x] Initialize the **`Typer`** CLI framework in `cli.py`.
* Created basic Typer app structure
* Implemented skeleton commands for `list` and `download`
* Added proper type hints and docstrings
* Integrated with config.py for environment variable loading
* [x] Implement the **`python-garminconnect`** client wrapper (`garmin.py`).
* Created GarminClient class with authentication
* Implemented get_activities() with rate limiting
* Implemented download_activity_fit() method
* Added error handling for missing credentials
**Known issues:** None
#### **Phase 2: Activity Listing**
## Recent Fixes
- Fixed package declaration conflicts in cmd/ directory (changed from `package cmd` to `package main`)
- Removed unnecessary import in root.go that was causing build errors
- Verified Docker build process now completes successfully
* [x] Implement the `SQLAlchemy` schema and database connection.
* Created database.py with Activity model
* Implemented init_db() for database initialization
* Added get_session() for session management
* Implemented sync_database() to fetch activities from Garmin and update database
* [x] Create the `list` commands (`--all`, `--missing`, `--downloaded`).
* Implemented command logic in cli.py
* Added database synchronization before listing
* Added input validation for filter options
* Used tqdm for progress display
#### **Phase 3: Download Pipeline**
* [x] Implement the `.fit` file download function using `python-garminconnect`.
* Added download logic to cli.py
* Used GarminClient to download FIT files
* Created filename-safe timestamps
* Saved files to data directory
* [x] Ensure downloads are idempotent (don't re-download existing files).
* Database tracks downloaded status
* Only missing activities are downloaded
* [x] Update the database record to `downloaded=True` on success.
* Updated activity record after successful download
* Set filename path in database
#### **Phase 4: Polish ✨**
* [x] Add **`tqdm`** progress bars for the download command.
* Implemented in both list and download commands
* [x] Update Dockerfile to:
* Use multi-stage builds for smaller image size
* Add health checks
* Set non-root user for security
* [ ] Implement robust error handling (e.g., API errors, network issues).
* Basic error handling implemented but needs improvement
* [ ] Write comprehensive `README.md` documentation.
* [ ] Add unit tests for database and utility functions.
#### **Current Issues**
1. **Docker Build Warnings**
- Legacy ENV format warnings during build
- Solution: Update Dockerfile to use `ENV key=value` format
2. **Typer Command Parameters**
- Occasional `Parameter.make_metavar()` errors
- Resolved by renaming reserved keywords and fixing decorators
3. **Configuration Loading**
- Initial import issues between modules
- Resolved by restructuring config loading
-----
### **Documentation 📚**
Here are links to the official documentation for the key libraries used in this design.
* **Garmin API:** [`python-garminconnect`](https://www.google.com/search?q=%5Bhttps://github.com/cyberjunky/python-garminconnect%5D\(https://github.com/cyberjunky/python-garminconnect\))
* **CLI Framework:** [`Typer`](https://www.google.com/search?q=%5Bhttps://typer.tiangolo.com/%5D\(https://typer.tiangolo.com/\))
* **Environment Variables:** [`python-dotenv`](https://www.google.com/search?q=%5Bhttps://github.com/theskumar/python-dotenv%5D\(https://github.com/theskumar/python-dotenv\))
* **Database ORM:** [`SQLAlchemy`](https://www.google.com/search?q=%5Bhttps://docs.sqlalchemy.org/en/20/%5D\(https://docs.sqlalchemy.org/en/20/\))
* **Database Migrations:** [`Alembic`](https://www.google.com/search?q=%5Bhttps://alembic.sqlalchemy.org/en/latest/%5D\(https://alembic.sqlalchemy.org/en/latest/\))
* **Progress Bars:** [`tqdm`](https://www.google.com/search?q=%5Bhttps://github.com/tqdm/tqdm%5D\(https://github.com/tqdm/tqdm\))