sync
All checks were successful
Build and Push Docker Image / build-and-push (push) Successful in 3m2s
All checks were successful
Build and Push Docker Image / build-and-push (push) Successful in 3m2s
This commit is contained in:
58
README.md
58
README.md
@@ -16,32 +16,33 @@ docker build -t fitbit-garmin-sync .
|
|||||||
|
|
||||||
## Running the Application
|
## Running the Application
|
||||||
|
|
||||||
The application requires persistent storage for configuration, database, logs, and session data. You should create a local directory to store this data and mount it as a volume when running the container.
|
The application is configured entirely via Consul. You can specify the Consul agent's location using environment variables.
|
||||||
|
|
||||||
1. **Create a data directory on your host machine:**
|
- `CONSUL_HOST`: The hostname or IP address of your Consul agent (defaults to `consul.service.dc1.consul`).
|
||||||
|
- `CONSUL_PORT`: The port of your Consul agent (defaults to `8500`).
|
||||||
|
|
||||||
```bash
|
The application can be run in several modes. The default command is `schedule` to run the sync on a schedule.
|
||||||
mkdir fitbit_garmin_data
|
|
||||||
```
|
|
||||||
|
|
||||||
2. **Run the Docker container with a mounted volume:**
|
```bash
|
||||||
|
docker run -d --name fitbit-sync \
|
||||||
|
-e CONSUL_HOST=your-consul-host \
|
||||||
|
-e CONSUL_PORT=8500 \
|
||||||
|
fitbit-garmin-sync
|
||||||
|
```
|
||||||
|
|
||||||
The application can be run in several modes. The default command is `schedule` to run the sync on a schedule.
|
This will start the container in detached mode (`-d`) and run the scheduled sync.
|
||||||
|
|
||||||
```bash
|
|
||||||
docker run -d --name fitbit-sync -v "$(pwd)/fitbit_garmin_data":/app/data fitbit-garmin-sync
|
|
||||||
```
|
|
||||||
|
|
||||||
This will start the container in detached mode (`-d`) and run the scheduled sync.
|
|
||||||
|
|
||||||
### Interactive Setup
|
### Interactive Setup
|
||||||
|
|
||||||
The first time you run the application, you will need to perform an interactive setup to provide your Fitbit and Garmin credentials.
|
The first time you run the application, you will need to perform an interactive setup to provide your Fitbit and Garmin credentials. These will be stored securely in Consul.
|
||||||
|
|
||||||
1. **Run the container with the `setup` command:**
|
1. **Run the container with the `setup` command:**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker run -it --rm -v "$(pwd)/fitbit_garmin_data":/app/data fitbit-garmin-sync setup
|
docker run -it --rm \
|
||||||
|
-e CONSUL_HOST=your-consul-host \
|
||||||
|
-e CONSUL_PORT=8500 \
|
||||||
|
fitbit-garmin-sync setup
|
||||||
```
|
```
|
||||||
|
|
||||||
- `-it` allows you to interact with the container's terminal.
|
- `-it` allows you to interact with the container's terminal.
|
||||||
@@ -49,33 +50,28 @@ The first time you run the application, you will need to perform an interactive
|
|||||||
|
|
||||||
2. **Follow the on-screen prompts** to enter your Fitbit and Garmin credentials. The application will guide you through the OAuth process for Fitbit, which requires you to copy and paste a URL into your browser.
|
2. **Follow the on-screen prompts** to enter your Fitbit and Garmin credentials. The application will guide you through the OAuth process for Fitbit, which requires you to copy and paste a URL into your browser.
|
||||||
|
|
||||||
After the setup is complete, the necessary configuration and session files will be saved in your `fitbit_garmin_data` directory.
|
After the setup is complete, the necessary configuration and session data will be saved in Consul.
|
||||||
|
|
||||||
### Other Commands
|
### Other Commands
|
||||||
|
|
||||||
You can run other commands by specifying them when you run the container. For example, to run a manual sync:
|
You can run other commands by specifying them when you run the container. For example, to run a manual sync:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker run -it --rm -v "$(pwd)/fitbit_garmin_data":/app/data fitbit-garmin-sync sync
|
docker run -it --rm \
|
||||||
|
-e CONSUL_HOST=your-consul-host \
|
||||||
|
-e CONSUL_PORT=8500 \
|
||||||
|
fitbit-garmin-sync sync
|
||||||
```
|
```
|
||||||
|
|
||||||
To check the status:
|
To check the status:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker run -it --rm -v "$(pwd)/fitbit_garmin_data":/app/data fitbit-garmin-sync status
|
docker run -it --rm \
|
||||||
|
-e CONSUL_HOST=your-consul-host \
|
||||||
|
-e CONSUL_PORT=8500 \
|
||||||
|
fitbit-garmin-sync status
|
||||||
```
|
```
|
||||||
|
|
||||||
## Data Persistence
|
## Configuration in Consul
|
||||||
|
|
||||||
The following files will be stored in the mounted data volume (`fitbit_garmin_data`):
|
All application state, including credentials, tokens, and sync status, is stored in Consul under a configurable prefix (default: `fitbit-garmin-sync`).
|
||||||
|
|
||||||
- `config.json`: Application configuration, including API keys.
|
|
||||||
- `weight_sync.db`: SQLite database for storing sync state.
|
|
||||||
- `weight_sync.log`: Log file.
|
|
||||||
- `garmin_session.json`: Garmin session data.
|
|
||||||
|
|
||||||
By using a volume, this data will persist even if the container is stopped or removed.
|
|
||||||
|
|
||||||
## Managing Credentials
|
|
||||||
|
|
||||||
Your Fitbit and Garmin credentials are an essential part of the `config.json` file, which is stored in the data volume. Be sure to treat this data as sensitive. It is recommended to restrict permissions on the `fitbit_garmin_data` directory.
|
|
||||||
@@ -9,7 +9,7 @@ job "fitbit-garmin-sync" {
|
|||||||
driver = "docker"
|
driver = "docker"
|
||||||
|
|
||||||
config {
|
config {
|
||||||
image = "gitea.service.dc1.fbleagh.duckdns.org/sstent/fitbit-garmin-sync:latest"
|
image = "gitea.service.dc1.fbleagh.duckdns.org/sstent/fitbit_garmin_sync:latest"
|
||||||
volumes = [
|
volumes = [
|
||||||
"/mnt/Public/configs/fitbit-garmin-sync:/app/data"
|
"/mnt/Public/configs/fitbit-garmin-sync:/app/data"
|
||||||
]
|
]
|
||||||
@@ -26,4 +26,4 @@ job "fitbit-garmin-sync" {
|
|||||||
# The command to run is defined in the Dockerfile ENTRYPOINT and CMD.
|
# The command to run is defined in the Dockerfile ENTRYPOINT and CMD.
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
456
fitbitsync.py
456
fitbitsync.py
@@ -4,7 +4,6 @@
|
|||||||
import asyncio
|
import asyncio
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
import sqlite3
|
|
||||||
from datetime import datetime, timedelta, timezone
|
from datetime import datetime, timedelta, timezone
|
||||||
from typing import List, Dict, Optional, Tuple
|
from typing import List, Dict, Optional, Tuple
|
||||||
from dataclasses import dataclass, asdict
|
from dataclasses import dataclass, asdict
|
||||||
@@ -33,6 +32,12 @@ try:
|
|||||||
except ImportError:
|
except ImportError:
|
||||||
GARMINCONNECT_LIBRARY = False
|
GARMINCONNECT_LIBRARY = False
|
||||||
|
|
||||||
|
try:
|
||||||
|
import consul
|
||||||
|
CONSUL_LIBRARY = True
|
||||||
|
except ImportError:
|
||||||
|
CONSUL_LIBRARY = False
|
||||||
|
|
||||||
import schedule
|
import schedule
|
||||||
|
|
||||||
# Configure logging
|
# Configure logging
|
||||||
@@ -112,8 +117,10 @@ class ConfigManager:
|
|||||||
"max_retries": 3,
|
"max_retries": 3,
|
||||||
"read_only_mode": False # Set to True to prevent uploads to Garmin
|
"read_only_mode": False # Set to True to prevent uploads to Garmin
|
||||||
},
|
},
|
||||||
"database": {
|
"consul": {
|
||||||
"path": "weight_sync.db"
|
"host": "consul.service.dc1.consul",
|
||||||
|
"port": 8500,
|
||||||
|
"prefix": "fitbit-garmin-sync"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
# Don't automatically save here, let the caller decide
|
# Don't automatically save here, let the caller decide
|
||||||
@@ -166,114 +173,208 @@ class ConfigManager:
|
|||||||
elif service == "fitbit":
|
elif service == "fitbit":
|
||||||
return self.config.get("fitbit", {}).get(field)
|
return self.config.get("fitbit", {}).get(field)
|
||||||
|
|
||||||
class DatabaseManager:
|
class ConsulStateManager:
|
||||||
"""Manages SQLite database for sync state and records"""
|
"""Manages sync state and records using Consul K/V store"""
|
||||||
|
|
||||||
def __init__(self, db_path: str):
|
|
||||||
self.db_path = db_path
|
|
||||||
self._init_database()
|
|
||||||
|
|
||||||
def _init_database(self):
|
|
||||||
"""Initialize database tables"""
|
|
||||||
with sqlite3.connect(self.db_path) as conn:
|
|
||||||
conn.execute('''
|
|
||||||
CREATE TABLE IF NOT EXISTS weight_records (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
||||||
sync_id TEXT UNIQUE NOT NULL,
|
|
||||||
timestamp TEXT NOT NULL,
|
|
||||||
weight_kg REAL NOT NULL,
|
|
||||||
source TEXT NOT NULL,
|
|
||||||
synced_to_garmin BOOLEAN DEFAULT FALSE,
|
|
||||||
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
updated_at TEXT DEFAULT CURRENT_TIMESTAMP
|
|
||||||
)
|
|
||||||
''')
|
|
||||||
|
|
||||||
conn.execute('''
|
|
||||||
CREATE TABLE IF NOT EXISTS sync_log (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
||||||
sync_type TEXT NOT NULL,
|
|
||||||
status TEXT NOT NULL,
|
|
||||||
message TEXT,
|
|
||||||
records_processed INTEGER DEFAULT 0,
|
|
||||||
timestamp TEXT DEFAULT CURRENT_TIMESTAMP
|
|
||||||
)
|
|
||||||
''')
|
|
||||||
|
|
||||||
# Create indexes separately
|
|
||||||
conn.execute('CREATE INDEX IF NOT EXISTS idx_weight_timestamp ON weight_records(timestamp)')
|
|
||||||
conn.execute('CREATE INDEX IF NOT EXISTS idx_weight_sync_id ON weight_records(sync_id)')
|
|
||||||
conn.execute('CREATE INDEX IF NOT EXISTS idx_sync_log_timestamp ON sync_log(timestamp)')
|
|
||||||
|
|
||||||
|
def __init__(self, config: ConfigManager):
|
||||||
|
if not CONSUL_LIBRARY:
|
||||||
|
raise ImportError("python-consul library not installed. Please install it with: pip install python-consul")
|
||||||
|
|
||||||
|
consul_config = config.get('consul')
|
||||||
|
self.client = consul.Consul(
|
||||||
|
host=consul_config.get('host', 'localhost'),
|
||||||
|
port=consul_config.get('port', 8500)
|
||||||
|
)
|
||||||
|
self.prefix = consul_config.get('prefix', 'fitbit-garmin-sync').strip('/')
|
||||||
|
self.records_prefix = f"{self.prefix}/records/"
|
||||||
|
self.logs_prefix = f"{self.prefix}/logs/"
|
||||||
|
|
||||||
|
logger.info(f"Using Consul for state management at {consul_config.get('host')}:{consul_config.get('port')} with prefix '{self.prefix}'")
|
||||||
|
|
||||||
def save_weight_record(self, record: WeightRecord) -> bool:
|
def save_weight_record(self, record: WeightRecord) -> bool:
|
||||||
"""Save weight record to database"""
|
"""Save weight record to Consul if it doesn't exist."""
|
||||||
|
key = f"{self.records_prefix}{record.sync_id}"
|
||||||
try:
|
try:
|
||||||
with sqlite3.connect(self.db_path) as conn:
|
# Check if record already exists
|
||||||
conn.execute('''
|
index, data = self.client.kv.get(key)
|
||||||
INSERT OR REPLACE INTO weight_records
|
if data is not None:
|
||||||
(sync_id, timestamp, weight_kg, source, updated_at)
|
# Record already exists, no need to save again
|
||||||
VALUES (?, ?, ?, ?, ?)
|
return False
|
||||||
''', (
|
|
||||||
record.sync_id,
|
# Record doesn't exist, save it with synced_to_garmin=False
|
||||||
record.timestamp.isoformat(),
|
record_data = asdict(record)
|
||||||
record.weight_kg,
|
record_data['timestamp'] = record.timestamp.isoformat()
|
||||||
record.source,
|
record_data['synced_to_garmin'] = False
|
||||||
datetime.now().isoformat()
|
|
||||||
))
|
self.client.kv.put(key, json.dumps(record_data))
|
||||||
return True
|
return True
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Error saving weight record: {e}")
|
logger.error(f"Error saving weight record to Consul: {e}")
|
||||||
return False
|
return False
|
||||||
|
|
||||||
def get_unsynced_records(self) -> List[WeightRecord]:
|
def get_unsynced_records(self) -> List[WeightRecord]:
|
||||||
"""Get records that haven't been synced to Garmin"""
|
"""Get records from Consul that haven't been synced to Garmin."""
|
||||||
records = []
|
records = []
|
||||||
try:
|
try:
|
||||||
with sqlite3.connect(self.db_path) as conn:
|
# This is inefficient and not recommended for large datasets
|
||||||
cursor = conn.execute('''
|
index, keys = self.client.kv.get(self.records_prefix, keys=True)
|
||||||
SELECT sync_id, timestamp, weight_kg, source
|
if not keys:
|
||||||
FROM weight_records
|
return []
|
||||||
WHERE synced_to_garmin = FALSE
|
|
||||||
ORDER BY timestamp DESC
|
logger.info(f"Scanning {len(keys)} records from Consul to find unsynced items. This may be slow.")
|
||||||
''')
|
|
||||||
|
for key in keys:
|
||||||
for row in cursor.fetchall():
|
index, data = self.client.kv.get(key)
|
||||||
record = WeightRecord(
|
if data and data.get('Value'):
|
||||||
sync_id=row[0],
|
try:
|
||||||
timestamp=datetime.fromisoformat(row[1]),
|
record_data = json.loads(data['Value'])
|
||||||
weight_kg=row[2],
|
if not record_data.get('synced_to_garmin'):
|
||||||
source=row[3]
|
record = WeightRecord(
|
||||||
)
|
sync_id=record_data['sync_id'],
|
||||||
records.append(record)
|
timestamp=datetime.fromisoformat(record_data['timestamp']),
|
||||||
|
weight_kg=record_data['weight_kg'],
|
||||||
|
source=record_data['source']
|
||||||
|
)
|
||||||
|
records.append(record)
|
||||||
|
except (json.JSONDecodeError, KeyError) as e:
|
||||||
|
logger.warning(f"Could not parse record from Consul at key {key}: {e}")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Error getting unsynced records: {e}")
|
logger.error(f"Error getting unsynced records from Consul: {e}")
|
||||||
|
|
||||||
|
# Sort by timestamp descending to sync newest records first
|
||||||
|
records.sort(key=lambda r: r.timestamp, reverse=True)
|
||||||
return records
|
return records
|
||||||
|
|
||||||
def mark_synced(self, sync_id: str) -> bool:
|
def mark_synced(self, sync_id: str) -> bool:
|
||||||
"""Mark a record as synced to Garmin"""
|
"""Mark a record as synced to Garmin in Consul."""
|
||||||
|
key = f"{self.records_prefix}{sync_id}"
|
||||||
try:
|
try:
|
||||||
with sqlite3.connect(self.db_path) as conn:
|
# Use a Check-And-Set (CAS) loop for safe updates
|
||||||
conn.execute('''
|
for _ in range(5): # Max 5 retries
|
||||||
UPDATE weight_records
|
index, data = self.client.kv.get(key)
|
||||||
SET synced_to_garmin = TRUE, updated_at = ?
|
if data is None:
|
||||||
WHERE sync_id = ?
|
logger.warning(f"Cannot mark sync_id {sync_id} as synced: record not found in Consul.")
|
||||||
''', (datetime.now().isoformat(), sync_id))
|
return False
|
||||||
return True
|
|
||||||
except Exception as e:
|
record_data = json.loads(data['Value'])
|
||||||
logger.error(f"Error marking record as synced: {e}")
|
record_data['synced_to_garmin'] = True
|
||||||
|
|
||||||
|
# The 'cas' parameter ensures we only update if the key hasn't changed
|
||||||
|
success = self.client.kv.put(key, json.dumps(record_data), cas=data['ModifyIndex'])
|
||||||
|
if success:
|
||||||
|
return True
|
||||||
|
time.sleep(0.1) # Wait a bit before retrying
|
||||||
|
|
||||||
|
logger.error(f"Failed to mark record {sync_id} as synced after multiple retries.")
|
||||||
return False
|
return False
|
||||||
|
|
||||||
def log_sync(self, sync_type: str, status: str, message: str = "", records_processed: int = 0):
|
|
||||||
"""Log sync operation"""
|
|
||||||
try:
|
|
||||||
with sqlite3.connect(self.db_path) as conn:
|
|
||||||
conn.execute('''
|
|
||||||
INSERT INTO sync_log (sync_type, status, message, records_processed)
|
|
||||||
VALUES (?, ?, ?, ?)
|
|
||||||
''', (sync_type, status, message, records_processed))
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Error logging sync: {e}")
|
logger.error(f"Error marking record as synced in Consul: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
def log_sync(self, sync_type: str, status: str, message: str = "", records_processed: int = 0):
|
||||||
|
"""Log sync operation to Consul."""
|
||||||
|
log_entry = {
|
||||||
|
"sync_type": sync_type,
|
||||||
|
"status": status,
|
||||||
|
"message": message,
|
||||||
|
"records_processed": records_processed,
|
||||||
|
"timestamp": datetime.now(timezone.utc).isoformat()
|
||||||
|
}
|
||||||
|
key = f"{self.logs_prefix}{log_entry['timestamp']}"
|
||||||
|
try:
|
||||||
|
self.client.kv.put(key, json.dumps(log_entry))
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Error logging sync to Consul: {e}")
|
||||||
|
|
||||||
|
def reset_sync_status(self) -> int:
|
||||||
|
"""Reset all records to unsynced status in Consul."""
|
||||||
|
affected_rows = 0
|
||||||
|
try:
|
||||||
|
index, keys = self.client.kv.get(self.records_prefix, keys=True)
|
||||||
|
if not keys:
|
||||||
|
return 0
|
||||||
|
|
||||||
|
logger.info(f"Resetting sync status for {len(keys)} records in Consul...")
|
||||||
|
|
||||||
|
for key in keys:
|
||||||
|
try:
|
||||||
|
# Use CAS loop for safety
|
||||||
|
for _ in range(3):
|
||||||
|
index, data = self.client.kv.get(key)
|
||||||
|
if data and data.get('Value'):
|
||||||
|
record_data = json.loads(data['Value'])
|
||||||
|
if record_data.get('synced_to_garmin'):
|
||||||
|
record_data['synced_to_garmin'] = False
|
||||||
|
success = self.client.kv.put(key, json.dumps(record_data), cas=data['ModifyIndex'])
|
||||||
|
if success:
|
||||||
|
affected_rows += 1
|
||||||
|
break
|
||||||
|
else:
|
||||||
|
break # Already unsynced
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Failed to reset sync status for key {key}: {e}")
|
||||||
|
return affected_rows
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Error resetting sync status in Consul: {e}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
def get_status_info(self) -> Dict:
|
||||||
|
"""Get status info from Consul."""
|
||||||
|
status_info = {
|
||||||
|
"total_records": 0,
|
||||||
|
"synced_records": 0,
|
||||||
|
"unsynced_records": 0,
|
||||||
|
"recent_syncs": [],
|
||||||
|
"recent_records": []
|
||||||
|
}
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Get record counts
|
||||||
|
index, keys = self.client.kv.get(self.records_prefix, keys=True)
|
||||||
|
if keys:
|
||||||
|
status_info['total_records'] = len(keys)
|
||||||
|
synced_count = 0
|
||||||
|
all_records = []
|
||||||
|
for key in keys:
|
||||||
|
index, data = self.client.kv.get(key)
|
||||||
|
if data and data.get('Value'):
|
||||||
|
record_data = json.loads(data['Value'])
|
||||||
|
all_records.append(record_data)
|
||||||
|
if record_data.get('synced_to_garmin'):
|
||||||
|
synced_count += 1
|
||||||
|
|
||||||
|
status_info['synced_records'] = synced_count
|
||||||
|
status_info['unsynced_records'] = status_info['total_records'] - synced_count
|
||||||
|
|
||||||
|
# Get recent records
|
||||||
|
all_records.sort(key=lambda r: r.get('timestamp', ''), reverse=True)
|
||||||
|
for record in all_records[:5]:
|
||||||
|
status_info['recent_records'].append((
|
||||||
|
record['timestamp'],
|
||||||
|
record['weight_kg'],
|
||||||
|
record['source'],
|
||||||
|
record['synced_to_garmin']
|
||||||
|
))
|
||||||
|
|
||||||
|
# Get recent sync logs
|
||||||
|
index, log_keys = self.client.kv.get(self.logs_prefix, keys=True)
|
||||||
|
if log_keys:
|
||||||
|
log_keys.sort(reverse=True) # Sort by timestamp descending
|
||||||
|
for key in log_keys[:5]:
|
||||||
|
index, data = self.client.kv.get(key)
|
||||||
|
if data and data.get('Value'):
|
||||||
|
log_data = json.loads(data['Value'])
|
||||||
|
status_info['recent_syncs'].append((
|
||||||
|
log_data['timestamp'],
|
||||||
|
log_data['status'],
|
||||||
|
log_data['message'],
|
||||||
|
log_data['records_processed']
|
||||||
|
))
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Error getting status info from Consul: {e}")
|
||||||
|
|
||||||
|
return status_info
|
||||||
|
|
||||||
class FitbitClient:
|
class FitbitClient:
|
||||||
"""Client for Fitbit API using python-fitbit"""
|
"""Client for Fitbit API using python-fitbit"""
|
||||||
@@ -881,12 +982,7 @@ class WeightSyncApp:
|
|||||||
|
|
||||||
def __init__(self, config_file: str = "data/config.json"):
|
def __init__(self, config_file: str = "data/config.json"):
|
||||||
self.config = ConfigManager(config_file)
|
self.config = ConfigManager(config_file)
|
||||||
|
self.state = ConsulStateManager(self.config)
|
||||||
# Construct full paths for data files
|
|
||||||
data_dir = self.config.config_file.parent
|
|
||||||
db_path = data_dir / self.config.get('database.path', 'weight_sync.db')
|
|
||||||
|
|
||||||
self.db = DatabaseManager(db_path)
|
|
||||||
self.fitbit = FitbitClient(self.config)
|
self.fitbit = FitbitClient(self.config)
|
||||||
self.garmin = GarminClient(self.config)
|
self.garmin = GarminClient(self.config)
|
||||||
|
|
||||||
@@ -933,20 +1029,17 @@ class WeightSyncApp:
|
|||||||
logger.info(f"Found {len(fitbit_records)} weight records from Fitbit")
|
logger.info(f"Found {len(fitbit_records)} weight records from Fitbit")
|
||||||
print(f"📊 Found {len(fitbit_records)} weight records from Fitbit")
|
print(f"📊 Found {len(fitbit_records)} weight records from Fitbit")
|
||||||
|
|
||||||
# Save new records to database
|
# Save new records to state manager
|
||||||
new_records = 0
|
new_records = 0
|
||||||
updated_records = 0
|
|
||||||
for record in fitbit_records:
|
for record in fitbit_records:
|
||||||
if self.db.save_weight_record(record):
|
if self.state.save_weight_record(record):
|
||||||
new_records += 1
|
new_records += 1
|
||||||
else:
|
|
||||||
updated_records += 1
|
|
||||||
|
|
||||||
logger.info(f"Processed {new_records} new weight records, {updated_records} updated records")
|
logger.info(f"Processed {new_records} new weight records")
|
||||||
print(f"💾 Processed {new_records} new records, {updated_records} updated records")
|
print(f"💾 Found {new_records} new records to potentially sync")
|
||||||
|
|
||||||
# Get unsynced records
|
# Get unsynced records
|
||||||
unsynced_records = self.db.get_unsynced_records()
|
unsynced_records = self.state.get_unsynced_records()
|
||||||
|
|
||||||
if not unsynced_records:
|
if not unsynced_records:
|
||||||
logger.info("No unsynced records found")
|
logger.info("No unsynced records found")
|
||||||
@@ -960,15 +1053,17 @@ class WeightSyncApp:
|
|||||||
|
|
||||||
# Mark successful uploads as synced
|
# Mark successful uploads as synced
|
||||||
synced_count = 0
|
synced_count = 0
|
||||||
for record in unsynced_records[:success_count]:
|
# Iterate over the original list but only up to the number of successes
|
||||||
if self.db.mark_synced(record.sync_id):
|
for i in range(success_count):
|
||||||
|
record_to_mark = unsynced_records[i]
|
||||||
|
if self.state.mark_synced(record_to_mark.sync_id):
|
||||||
synced_count += 1
|
synced_count += 1
|
||||||
|
|
||||||
# Log results
|
# Log results
|
||||||
mode_prefix = "(Read-only) " if read_only_mode else ""
|
mode_prefix = "(Read-only) " if read_only_mode else ""
|
||||||
message = f"{mode_prefix}Full sync: {synced_count} records synced, {failed_count} failed"
|
message = f"{mode_prefix}Full sync: {synced_count} records synced, {failed_count} failed"
|
||||||
status = "success" if failed_count == 0 else "partial"
|
status = "success" if failed_count == 0 else "partial"
|
||||||
self.db.log_sync("full_sync", status, message, synced_count)
|
self.state.log_sync("full_sync", status, message, synced_count)
|
||||||
|
|
||||||
logger.info(f"Full sync completed: {message}")
|
logger.info(f"Full sync completed: {message}")
|
||||||
print(f"✅ Full sync completed: {synced_count} synced, {failed_count} failed")
|
print(f"✅ Full sync completed: {synced_count} synced, {failed_count} failed")
|
||||||
@@ -977,7 +1072,7 @@ class WeightSyncApp:
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
error_msg = f"Full sync failed: {e}"
|
error_msg = f"Full sync failed: {e}"
|
||||||
logger.error(error_msg)
|
logger.error(error_msg)
|
||||||
self.db.log_sync("full_sync", "error", error_msg, 0)
|
self.state.log_sync("full_sync", "error", error_msg, 0)
|
||||||
print(f"❌ Full sync failed: {e}")
|
print(f"❌ Full sync failed: {e}")
|
||||||
return False
|
return False
|
||||||
|
|
||||||
@@ -1036,17 +1131,11 @@ class WeightSyncApp:
|
|||||||
def reset_sync_status(self):
|
def reset_sync_status(self):
|
||||||
"""Reset all records to unsynced status"""
|
"""Reset all records to unsynced status"""
|
||||||
try:
|
try:
|
||||||
with sqlite3.connect(self.db.db_path) as conn:
|
affected_rows = self.state.reset_sync_status()
|
||||||
result = conn.execute('''
|
logger.info(f"Reset sync status for {affected_rows} records")
|
||||||
UPDATE weight_records
|
print(f"🔄 Reset sync status for {affected_rows} records")
|
||||||
SET synced_to_garmin = FALSE, updated_at = ?
|
print(" All records will be synced again on next sync")
|
||||||
''', (datetime.now().isoformat(),))
|
return True
|
||||||
|
|
||||||
affected_rows = result.rowcount
|
|
||||||
logger.info(f"Reset sync status for {affected_rows} records")
|
|
||||||
print(f"🔄 Reset sync status for {affected_rows} records")
|
|
||||||
print(" All records will be synced again on next sync")
|
|
||||||
return True
|
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Error resetting sync status: {e}")
|
logger.error(f"Error resetting sync status: {e}")
|
||||||
@@ -1071,20 +1160,20 @@ class WeightSyncApp:
|
|||||||
# Fetch data from Fitbit
|
# Fetch data from Fitbit
|
||||||
fitbit_records = await self.fitbit.get_weight_data(start_date, end_date)
|
fitbit_records = await self.fitbit.get_weight_data(start_date, end_date)
|
||||||
|
|
||||||
# Save new records to database
|
# Save new records to state manager
|
||||||
new_records = 0
|
new_records = 0
|
||||||
for record in fitbit_records:
|
for record in fitbit_records:
|
||||||
if self.db.save_weight_record(record):
|
if self.state.save_weight_record(record):
|
||||||
new_records += 1
|
new_records += 1
|
||||||
|
|
||||||
logger.info(f"Processed {new_records} new weight records from Fitbit")
|
logger.info(f"Processed {new_records} new weight records from Fitbit")
|
||||||
|
|
||||||
# Get unsynced records
|
# Get unsynced records
|
||||||
unsynced_records = self.db.get_unsynced_records()
|
unsynced_records = self.state.get_unsynced_records()
|
||||||
|
|
||||||
if not unsynced_records:
|
if not unsynced_records:
|
||||||
logger.info("No unsynced records found")
|
logger.info("No unsynced records found")
|
||||||
self.db.log_sync("weight_sync", "success", "No records to sync", 0)
|
self.state.log_sync("weight_sync", "success", "No records to sync", 0)
|
||||||
return True
|
return True
|
||||||
|
|
||||||
# Upload to Garmin (or simulate in read-only mode)
|
# Upload to Garmin (or simulate in read-only mode)
|
||||||
@@ -1092,15 +1181,17 @@ class WeightSyncApp:
|
|||||||
|
|
||||||
# Mark successful uploads as synced (even in read-only mode for simulation)
|
# Mark successful uploads as synced (even in read-only mode for simulation)
|
||||||
synced_count = 0
|
synced_count = 0
|
||||||
for record in unsynced_records[:success_count]:
|
# Iterate over the original list but only up to the number of successes
|
||||||
if self.db.mark_synced(record.sync_id):
|
for i in range(success_count):
|
||||||
|
record_to_mark = unsynced_records[i]
|
||||||
|
if self.state.mark_synced(record_to_mark.sync_id):
|
||||||
synced_count += 1
|
synced_count += 1
|
||||||
|
|
||||||
# Log results
|
# Log results
|
||||||
mode_prefix = "(Read-only) " if read_only_mode else ""
|
mode_prefix = "(Read-only) " if read_only_mode else ""
|
||||||
message = f"{mode_prefix}Synced {synced_count} records, {failed_count} failed"
|
message = f"{mode_prefix}Synced {synced_count} records, {failed_count} failed"
|
||||||
status = "success" if failed_count == 0 else "partial"
|
status = "success" if failed_count == 0 else "partial"
|
||||||
self.db.log_sync("weight_sync", status, message, synced_count)
|
self.state.log_sync("weight_sync", status, message, synced_count)
|
||||||
|
|
||||||
logger.info(f"Sync completed: {message}")
|
logger.info(f"Sync completed: {message}")
|
||||||
return True
|
return True
|
||||||
@@ -1108,7 +1199,7 @@ class WeightSyncApp:
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
error_msg = f"Sync failed: {e}"
|
error_msg = f"Sync failed: {e}"
|
||||||
logger.error(error_msg)
|
logger.error(error_msg)
|
||||||
self.db.log_sync("weight_sync", "error", error_msg, 0)
|
self.state.log_sync("weight_sync", "error", error_msg, 0)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
def start_scheduler(self):
|
def start_scheduler(self):
|
||||||
@@ -1142,65 +1233,45 @@ class WeightSyncApp:
|
|||||||
"""Show application status"""
|
"""Show application status"""
|
||||||
try:
|
try:
|
||||||
read_only_mode = self.config.get('sync.read_only_mode', False)
|
read_only_mode = self.config.get('sync.read_only_mode', False)
|
||||||
|
status_info = self.state.get_status_info()
|
||||||
|
|
||||||
|
print("\n📊 Weight Sync Status")
|
||||||
|
print("=" * 50)
|
||||||
|
print(f"Mode: {'Read-only (No Garmin uploads)' if read_only_mode else 'Full sync mode'}")
|
||||||
|
print(f"Backend: {'Consul' if CONSUL_LIBRARY else 'Unknown'}")
|
||||||
|
print(f"Fitbit Library: {'Available' if FITBIT_LIBRARY else 'Not Available'}")
|
||||||
|
print(f"Garmin Library: {'Available' if GARMINCONNECT_LIBRARY else 'Not Available'}")
|
||||||
|
print(f"Total weight records: {status_info['total_records']}")
|
||||||
|
print(f"Synced to Garmin: {status_info['synced_records']}")
|
||||||
|
print(f"Pending sync: {status_info['unsynced_records']}")
|
||||||
|
|
||||||
with sqlite3.connect(self.db.db_path) as conn:
|
print(f"\n📝 Recent Sync History:")
|
||||||
# Get record counts
|
if status_info['recent_syncs']:
|
||||||
total_records = conn.execute("SELECT COUNT(*) FROM weight_records").fetchone()[0]
|
for sync in status_info['recent_syncs']:
|
||||||
synced_records = conn.execute("SELECT COUNT(*) FROM weight_records WHERE synced_to_garmin = TRUE").fetchone()[0]
|
status_emoji = "✅" if sync[1] == "success" else "⚠️" if sync[1] == "partial" else "❌"
|
||||||
unsynced_records = total_records - synced_records
|
print(f" {status_emoji} {sync[0]} - {sync[1]} - {sync[2]} ({sync[3]} records)")
|
||||||
|
else:
|
||||||
# Get recent sync logs
|
print(" No sync history found")
|
||||||
recent_syncs = conn.execute('''
|
|
||||||
SELECT timestamp, status, message, records_processed
|
# Show recent Garmin weights if available and not in read-only mode
|
||||||
FROM sync_log
|
if not read_only_mode:
|
||||||
ORDER BY timestamp DESC
|
try:
|
||||||
LIMIT 5
|
recent_weights = self.garmin.get_recent_weights(7)
|
||||||
''').fetchall()
|
if recent_weights:
|
||||||
|
print(f"\n⚖️ Recent Garmin Weights:")
|
||||||
print("\n📊 Weight Sync Status")
|
for weight in recent_weights[:5]: # Show last 5
|
||||||
print("=" * 50)
|
date = weight.get('calendarDate', 'Unknown')
|
||||||
print(f"Mode: {'Read-only (No Garmin uploads)' if read_only_mode else 'Full sync mode'}")
|
weight_kg = weight.get('weight', 0) / 1000 if weight.get('weight') else 'Unknown'
|
||||||
print(f"Fitbit Library: {'Available' if FITBIT_LIBRARY else 'Not Available'}")
|
print(f" 📅 {date}: {weight_kg}kg")
|
||||||
print(f"Garmin Library: {GARMIN_LIBRARY or 'Not Available'}")
|
except Exception as e:
|
||||||
print(f"Total weight records: {total_records}")
|
logger.debug(f"Could not fetch recent Garmin weights: {e}")
|
||||||
print(f"Synced to Garmin: {synced_records}")
|
|
||||||
print(f"Pending sync: {unsynced_records}")
|
if status_info['recent_records']:
|
||||||
|
print(f"\n📈 Recent Weight Records (from Consul):")
|
||||||
print(f"\n📝 Recent Sync History:")
|
for record in status_info['recent_records']:
|
||||||
if recent_syncs:
|
sync_status = "✅" if record[3] else "⏳"
|
||||||
for sync in recent_syncs:
|
timestamp = datetime.fromisoformat(record[0])
|
||||||
status_emoji = "✅" if sync[1] == "success" else "⚠️" if sync[1] == "partial" else "❌"
|
print(f" {sync_status} {timestamp.strftime('%Y-%m-%d %H:%M')}: {record[1]}kg ({record[2]})")
|
||||||
print(f" {status_emoji} {sync[0]} - {sync[1]} - {sync[2]} ({sync[3]} records)")
|
|
||||||
else:
|
|
||||||
print(" No sync history found")
|
|
||||||
|
|
||||||
# Show recent Garmin weights if available and not in read-only mode
|
|
||||||
if not read_only_mode:
|
|
||||||
try:
|
|
||||||
recent_weights = self.garmin.get_recent_weights(7)
|
|
||||||
if recent_weights:
|
|
||||||
print(f"\n⚖️ Recent Garmin Weights:")
|
|
||||||
for weight in recent_weights[:5]: # Show last 5
|
|
||||||
date = weight.get('calendarDate', 'Unknown')
|
|
||||||
weight_kg = weight.get('weight', 0) / 1000 if weight.get('weight') else 'Unknown'
|
|
||||||
print(f" 📅 {date}: {weight_kg}kg")
|
|
||||||
except Exception as e:
|
|
||||||
logger.debug(f"Could not fetch recent Garmin weights: {e}")
|
|
||||||
|
|
||||||
# Show recent database records
|
|
||||||
recent_records = conn.execute('''
|
|
||||||
SELECT timestamp, weight_kg, source, synced_to_garmin
|
|
||||||
FROM weight_records
|
|
||||||
ORDER BY timestamp DESC
|
|
||||||
LIMIT 5
|
|
||||||
''').fetchall()
|
|
||||||
|
|
||||||
if recent_records:
|
|
||||||
print(f"\n📈 Recent Weight Records:")
|
|
||||||
for record in recent_records:
|
|
||||||
sync_status = "✅" if record[3] else "⏳"
|
|
||||||
timestamp = datetime.fromisoformat(record[0])
|
|
||||||
print(f" {sync_status} {timestamp.strftime('%Y-%m-%d %H:%M')}: {record[1]}kg ({record[2]})")
|
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print(f"❌ Error getting status: {e}")
|
print(f"❌ Error getting status: {e}")
|
||||||
@@ -1281,9 +1352,10 @@ async def main():
|
|||||||
|
|
||||||
elif command == "config":
|
elif command == "config":
|
||||||
read_only_mode = app.config.get('sync.read_only_mode', False)
|
read_only_mode = app.config.get('sync.read_only_mode', False)
|
||||||
|
consul_config = app.config.get('consul')
|
||||||
print(f"📁 Configuration file: {app.config.config_file}")
|
print(f"📁 Configuration file: {app.config.config_file}")
|
||||||
print(f"📁 Database file: {app.config.get('database.path')}")
|
print(f"🔗 Consul K/V Prefix: {consul_config.get('prefix')} at {consul_config.get('host')}:{consul_config.get('port')}")
|
||||||
print(f"📁 Log file: weight_sync.log")
|
print(f"📁 Log file: data/weight_sync.log")
|
||||||
print(f"🔒 Read-only mode: {'Enabled' if read_only_mode else 'Disabled'}")
|
print(f"🔒 Read-only mode: {'Enabled' if read_only_mode else 'Disabled'}")
|
||||||
|
|
||||||
elif command == "readonly":
|
elif command == "readonly":
|
||||||
|
|||||||
1159
fitbitsync_debug.py
1159
fitbitsync_debug.py
File diff suppressed because it is too large
Load Diff
@@ -2,3 +2,4 @@ fitbit==0.3.1
|
|||||||
garminconnect==0.2.30
|
garminconnect==0.2.30
|
||||||
garth==0.5.17
|
garth==0.5.17
|
||||||
schedule==1.2.2
|
schedule==1.2.2
|
||||||
|
python-consul
|
||||||
|
|||||||
Reference in New Issue
Block a user