conductor(setup): Add conductor setup files and initial track

This commit is contained in:
2026-02-07 17:49:57 -08:00
parent 29c9c697ee
commit 9b2b09b02a
13 changed files with 542 additions and 0 deletions

View File

@@ -0,0 +1,23 @@
# General Code Style Principles
This document outlines general coding principles that apply across all languages and frameworks used in this project.
## Readability
- Code should be easy to read and understand by humans.
- Avoid overly clever or obscure constructs.
## Consistency
- Follow existing patterns in the codebase.
- Maintain consistent formatting, naming, and structure.
## Simplicity
- Prefer simple solutions over complex ones.
- Break down complex problems into smaller, manageable parts.
## Maintainability
- Write code that is easy to modify and extend.
- Minimize dependencies and coupling.
## Documentation
- Document *why* something is done, not just *what*.
- Keep documentation up-to-date with code changes.

View File

@@ -0,0 +1,48 @@
# Effective Go Style Guide Summary
This document summarizes key rules and best practices from the official "Effective Go" guide for writing idiomatic Go code.
## 1. Formatting
- **`gofmt`:** All Go code **must** be formatted with `gofmt` (or `go fmt`). This is a non-negotiable, automated standard.
- **Indentation:** Use tabs for indentation (`gofmt` handles this).
- **Line Length:** Go has no strict line length limit. Let `gofmt` handle line wrapping.
## 2. Naming
- **`MixedCaps`:** Use `MixedCaps` or `mixedCaps` for multi-word names. Do not use underscores.
- **Exported vs. Unexported:** Names starting with an uppercase letter are exported (public). Names starting with a lowercase letter are not exported (private).
- **Package Names:** Short, concise, single-word, lowercase names.
- **Getters:** Do not name getters with a `Get` prefix. A getter for a field named `owner` should be named `Owner()`.
- **Interface Names:** One-method interfaces are named by the method name plus an `-er` suffix (e.g., `Reader`, `Writer`).
## 3. Control Structures
- **`if`:** No parentheses around the condition. Braces are mandatory. Can include an initialization statement (e.g., `if err := file.Chmod(0664); err != nil`).
- **`for`:** Go's only looping construct. Unifies `for` and `while`. Use `for...range` to iterate over slices, maps, strings, and channels.
- **`switch`:** More general than in C. Cases do not fall through by default (use `fallthrough` explicitly). Can be used without an expression to function as a cleaner `if-else-if` chain.
## 4. Functions
- **Multiple Returns:** Functions can return multiple values. This is the standard way to return a result and an error (e.g., `value, err`).
- **Named Result Parameters:** Return parameters can be named. This can make code clearer and more concise.
- **`defer`:** Schedules a function call to be run immediately before the function executing `defer` returns. Use it for cleanup tasks like closing files.
## 5. Data
- **`new` vs. `make`:**
- `new(T)`: Allocates memory for a new item of type `T`, zeroes it, and returns a pointer (`*T`).
- `make(T, ...)`: Creates and initializes slices, maps, and channels only. Returns an initialized value of type `T` (not a pointer).
- **Slices:** The preferred way to work with sequences. They are more flexible than arrays.
- **Maps:** Use the "comma ok" idiom to check for the existence of a key: `value, ok := myMap[key]`.
## 6. Interfaces
- **Implicit Implementation:** A type implements an interface by implementing its methods. No `implements` keyword is needed.
- **Small Interfaces:** Prefer many small interfaces over one large one. The standard library is full of single-method interfaces (e.g., `io.Reader`).
## 7. Concurrency
- **Share Memory By Communicating:** This is the core philosophy. Do not communicate by sharing memory; instead, share memory by communicating.
- **Goroutines:** Lightweight, concurrently executing functions. Start one with the `go` keyword.
- **Channels:** Typed conduits for communication between goroutines. Use `make` to create them.
## 8. Errors
- **`error` type:** The built-in `error` interface is the standard way to handle errors.
- **Explicit Error Handling:** Do not discard errors with the blank identifier (`_`). Check for errors explicitly.
- **`panic`:** Reserved for truly exceptional, unrecoverable situations. Generally, libraries should not panic.
*Source: [Effective Go](https://go.dev/doc/effective_go)*

14
conductor/index.md Normal file
View File

@@ -0,0 +1,14 @@
# Project Context
## Definition
- [Product Definition](./product.md)
- [Product Guidelines](./product-guidelines.md)
- [Tech Stack](./tech-stack.md)
## Workflow
- [Workflow](./workflow.md)
- [Code Style Guides](./code_styleguides/)
## Management
- [Tracks Registry](./tracks.md)
- [Tracks Directory](./tracks/)

View File

@@ -0,0 +1,17 @@
# Product Guidelines - JuiceNavidrome Infrastructure
## Vision
To provide a rock-solid, self-healing, and highly available deployment model for Navidrome, treating the infrastructure as a first-class product.
## Operational Principles
- **Infrastructure as Code:** All cluster configurations, job definitions, and container builds must be versioned and managed through Git.
- **Failover-First Design:** The primary measure of success is the system's ability to automatically recover from node failures without manual intervention.
- **Minimal State on Host:** Local host data should be limited to necessary LiteFS state and caches; all critical application data must be replicated or reside on shared storage.
## Automation Standards
- **Automated Rebuilds:** Any change to the base configuration or Dockerfile must trigger an automatic build and push to the local registry via Gitea Actions.
- **Safe Rollouts:** Use sequential update strategies (max_parallel=1) to ensure the cluster remains healthy during version transitions.
## Reliability & Monitoring
- **Health-Aware Orchestration:** Leverage Consul health checks to provide Traefik with accurate routing information, ensuring traffic only hits ready nodes.
- **Consistent Initialization:** Use clean bootstrap procedures to avoid common distributed system pitfalls like stale cluster IDs or locking conflicts.

16
conductor/product.md Normal file
View File

@@ -0,0 +1,16 @@
# Initial Concept\n\nDeploy and manage a highly available Navidrome music server with replicated SQLite storage using LiteFS on a Nomad cluster.
## Overview
A highly available and durable personal music streaming service built on Navidrome and LiteFS, orchestrated by Nomad.
## Target Audience
- Personal use for a single music enthusiast who demands constant access to their library.
## Key Goals
- **High Availability:** Ensure the music server remains accessible even if a cluster node fails, utilizing automatic failover.
- **Data Durability:** Maintain multiple synchronous copies of the SQLite database across different physical nodes to prevent data loss.
## Core Features
- **High-Quality Streaming:** Support for advanced audio formats and on-the-fly transcoding (Opus/FLAC) to ensure the best possible listening experience.
- **Universal Compatibility:** Full support for the Subsonic API to allow connection from a wide variety of mobile and desktop music clients.
- **Automated Infrastructure:** Managed by Nomad and Consul for seamless cluster operations and service discovery.

View File

@@ -0,0 +1 @@
{"last_successful_step": "3.3_initial_track_generated"}

18
conductor/tech-stack.md Normal file
View File

@@ -0,0 +1,18 @@
# Technology Stack - JuiceNavidrome
## Core Infrastructure
- **Nomad:** Orchestrates the deployment of the Navidrome service across the cluster nodes.
- **Consul:** Handles service discovery, health monitoring, and the distributed locking required for LiteFS leader election.
- **Docker:** Provides the container runtime environment for Navidrome and LiteFS.
## Storage & Database
- **SQLite:** The primary relational database used by Navidrome for metadata and state.
- **LiteFS:** A FUSE-based filesystem that provides synchronous replication of the SQLite database across the cluster.
## Automation & Delivery
- **Gitea Actions:** Automates the multi-arch (AMD64/ARM64) building and pushing of the custom supervised container image.
- **Git:** Source control for all infrastructure-as-code artifacts.
## Networking
- **Traefik:** Acts as the cluster's ingress controller, routing traffic based on Consul tags.
- **LiteFS Proxy:** Handles transparent write-forwarding to the cluster leader.

8
conductor/tracks.md Normal file
View File

@@ -0,0 +1,8 @@
# Project Tracks
This file tracks all major tracks for the project. Each track has its own detailed plan in its respective folder.
---
- [ ] **Track: fix routing - use litefs to register the navidrome service with consul. the serivce should point to the master and avoid the litefs proxy (it breaks navidrome)**
*Link: [./tracks/fix_routing_20260207/](./tracks/fix_routing_20260207/)*

View File

@@ -0,0 +1,5 @@
# Track fix_routing_20260207 Context
- [Specification](./spec.md)
- [Implementation Plan](./plan.md)
- [Metadata](./metadata.json)

View File

@@ -0,0 +1,8 @@
{
"track_id": "fix_routing_20260207",
"type": "bug",
"status": "new",
"created_at": "2026-02-07T17:36:00Z",
"updated_at": "2026-02-07T17:36:00Z",
"description": "fix routing - use litefs to register the navidrome service with consul. the serivce should point to the master and avoid the litefs proxy (it breaks navidrome)"
}

View File

@@ -0,0 +1,25 @@
# Implementation Plan: Direct Primary Routing for Navidrome-LiteFS
This plan outlines the steps to reconfigure the Navidrome-LiteFS cluster to bypass the LiteFS write-forwarding proxy and use direct primary node routing for improved reliability and performance.
## Phase 1: Infrastructure Configuration Update
In this phase, we will modify the Nomad job and LiteFS configuration to support direct port access and primary-aware health checks.
- [ ] Task: Update `navidrome-litefs-v2.nomad` to point service directly to Navidrome port
- [ ] Modify `service` block to use port 4533 instead of dynamic mapped port.
- [ ] Replace HTTP health check with a script check running `litefs is-primary`.
- [ ] Task: Update `litefs.yml` to ensure consistent internal API binding (if needed)
- [ ] Task: Conductor - User Manual Verification 'Infrastructure Configuration Update' (Protocol in workflow.md)
## Phase 2: Deployment and Validation
In this phase, we will deploy the changes and verify that the cluster correctly handles primary election and routing.
- [ ] Task: Deploy updated Nomad job
- [ ] Execute `nomad job run navidrome-litefs-v2.nomad`.
- [ ] Task: Verify Consul health status
- [ ] Confirm that only the LiteFS primary node is marked as `passing`.
- [ ] Confirm that replica nodes are marked as `critical`.
- [ ] Task: Verify Ingress Routing
- [ ] Confirm Traefik correctly routes traffic only to the primary node.
- [ ] Verify that Navidrome is accessible and functional.
- [ ] Task: Conductor - User Manual Verification 'Deployment and Validation' (Protocol in workflow.md)

View File

@@ -0,0 +1,26 @@
# Specification: Direct Primary Routing for Navidrome-LiteFS
## Overview
This track aims to fix routing issues caused by the LiteFS proxy. We will reconfigure the Nomad service registration to point directly to the Navidrome process (port 4533) on the primary node, bypassing the LiteFS write-forwarding proxy (port 8080). To ensure Traefik only routes traffic to the node capable of writes, we will implement a "Primary-only" health check.
## Functional Requirements
- **Direct Port Mapping:** Update the Nomad `service` block to use the host port `4533` directly instead of the LiteFS proxy port.
- **Primary-Aware Health Check:** Replace the standard HTTP health check with a script check.
- **Check Logic:** The script will execute `litefs is-primary`.
- If the node is the primary, the command exits with `0` (Passing).
- If the node is a replica, the command exits with a non-zero code (Critical).
- **Service Tags:** Retain all existing Traefik tags so ingress routing continues to work.
## Non-Functional Requirements
- **Failover Reliability:** In the event of a leader election, the old primary must become unhealthy and the new primary must become healthy in Consul, allowing Traefik to update its backends automatically.
- **Minimal Latency:** Bypassing the proxy eliminates the extra network hop for reads and potential compatibility issues with Navidrome's connection handling.
## Acceptance Criteria
- [ ] Consul reports the service as `passing` only on the node currently holding the LiteFS primary lease.
- [ ] Consul reports the service as `critical` on all replica nodes.
- [ ] Traefik correctly routes traffic to the primary node.
- [ ] Navidrome is accessible and functions correctly without the LiteFS proxy intermediary.
## Out of Scope
- Modifying Navidrome internal logic.
- Implementing an external health-check responder.

333
conductor/workflow.md Normal file
View File

@@ -0,0 +1,333 @@
# Project Workflow
## Guiding Principles
1. **The Plan is the Source of Truth:** All work must be tracked in `plan.md`
2. **The Tech Stack is Deliberate:** Changes to the tech stack must be documented in `tech-stack.md` *before* implementation
3. **Test-Driven Development:** Write unit tests before implementing functionality
4. **High Code Coverage:** Aim for >80% code coverage for all modules
5. **User Experience First:** Every decision should prioritize user experience
6. **Non-Interactive & CI-Aware:** Prefer non-interactive commands. Use `CI=true` for watch-mode tools (tests, linters) to ensure single execution.
## Task Workflow
All tasks follow a strict lifecycle:
### Standard Task Workflow
1. **Select Task:** Choose the next available task from `plan.md` in sequential order
2. **Mark In Progress:** Before beginning work, edit `plan.md` and change the task from `[ ]` to `[~]`
3. **Write Failing Tests (Red Phase):**
- Create a new test file for the feature or bug fix.
- Write one or more unit tests that clearly define the expected behavior and acceptance criteria for the task.
- **CRITICAL:** Run the tests and confirm that they fail as expected. This is the "Red" phase of TDD. Do not proceed until you have failing tests.
4. **Implement to Pass Tests (Green Phase):**
- Write the minimum amount of application code necessary to make the failing tests pass.
- Run the test suite again and confirm that all tests now pass. This is the "Green" phase.
5. **Refactor (Optional but Recommended):**
- With the safety of passing tests, refactor the implementation code and the test code to improve clarity, remove duplication, and enhance performance without changing the external behavior.
- Rerun tests to ensure they still pass after refactoring.
6. **Verify Coverage:** Run coverage reports using the project's chosen tools. For example, in a Python project, this might look like:
```bash
pytest --cov=app --cov-report=html
```
Target: >80% coverage for new code. The specific tools and commands will vary by language and framework.
7. **Document Deviations:** If implementation differs from tech stack:
- **STOP** implementation
- Update `tech-stack.md` with new design
- Add dated note explaining the change
- Resume implementation
8. **Commit Code Changes:**
- Stage all code changes related to the task.
- Propose a clear, concise commit message e.g, `feat(ui): Create basic HTML structure for calculator`.
- Perform the commit.
9. **Attach Task Summary with Git Notes:**
- **Step 9.1: Get Commit Hash:** Obtain the hash of the *just-completed commit* (`git log -1 --format="%H"`).
- **Step 9.2: Draft Note Content:** Create a detailed summary for the completed task. This should include the task name, a summary of changes, a list of all created/modified files, and the core "why" for the change.
- **Step 9.3: Attach Note:** Use the `git notes` command to attach the summary to the commit.
```bash
# The note content from the previous step is passed via the -m flag.
git notes add -m "<note content>" <commit_hash>
```
10. **Get and Record Task Commit SHA:**
- **Step 10.1: Update Plan:** Read `plan.md`, find the line for the completed task, update its status from `[~]` to `[x]`, and append the first 7 characters of the *just-completed commit's* commit hash.
- **Step 10.2: Write Plan:** Write the updated content back to `plan.md`.
11. **Commit Plan Update:**
- **Action:** Stage the modified `plan.md` file.
- **Action:** Commit this change with a descriptive message (e.g., `conductor(plan): Mark task 'Create user model' as complete`).
### Phase Completion Verification and Checkpointing Protocol
**Trigger:** This protocol is executed immediately after a task is completed that also concludes a phase in `plan.md`.
1. **Announce Protocol Start:** Inform the user that the phase is complete and the verification and checkpointing protocol has begun.
2. **Ensure Test Coverage for Phase Changes:**
- **Step 2.1: Determine Phase Scope:** To identify the files changed in this phase, you must first find the starting point. Read `plan.md` to find the Git commit SHA of the *previous* phase's checkpoint. If no previous checkpoint exists, the scope is all changes since the first commit.
- **Step 2.2: List Changed Files:** Execute `git diff --name-only <previous_checkpoint_sha> HEAD` to get a precise list of all files modified during this phase.
- **Step 2.3: Verify and Create Tests:** For each file in the list:
- **CRITICAL:** First, check its extension. Exclude non-code files (e.g., `.json`, `.md`, `.yaml`).
- For each remaining code file, verify a corresponding test file exists.
- If a test file is missing, you **must** create one. Before writing the test, **first, analyze other test files in the repository to determine the correct naming convention and testing style.** The new tests **must** validate the functionality described in this phase's tasks (`plan.md`).
3. **Execute Automated Tests with Proactive Debugging:**
- Before execution, you **must** announce the exact shell command you will use to run the tests.
- **Example Announcement:** "I will now run the automated test suite to verify the phase. **Command:** `CI=true npm test`"
- Execute the announced command.
- If tests fail, you **must** inform the user and begin debugging. You may attempt to propose a fix a **maximum of two times**. If the tests still fail after your second proposed fix, you **must stop**, report the persistent failure, and ask the user for guidance.
4. **Propose a Detailed, Actionable Manual Verification Plan:**
- **CRITICAL:** To generate the plan, first analyze `product.md`, `product-guidelines.md`, and `plan.md` to determine the user-facing goals of the completed phase.
- You **must** generate a step-by-step plan that walks the user through the verification process, including any necessary commands and specific, expected outcomes.
- The plan you present to the user **must** follow this format:
**For a Frontend Change:**
```
The automated tests have passed. For manual verification, please follow these steps:
**Manual Verification Steps:**
1. **Start the development server with the command:** `npm run dev`
2. **Open your browser to:** `http://localhost:3000`
3. **Confirm that you see:** The new user profile page, with the user's name and email displayed correctly.
```
**For a Backend Change:**
```
The automated tests have passed. For manual verification, please follow these steps:
**Manual Verification Steps:**
1. **Ensure the server is running.**
2. **Execute the following command in your terminal:** `curl -X POST http://localhost:8080/api/v1/users -d '{"name": "test"}'`
3. **Confirm that you receive:** A JSON response with a status of `201 Created`.
```
5. **Await Explicit User Feedback:**
- After presenting the detailed plan, ask the user for confirmation: "**Does this meet your expectations? Please confirm with yes or provide feedback on what needs to be changed.**"
- **PAUSE** and await the user's response. Do not proceed without an explicit yes or confirmation.
6. **Create Checkpoint Commit:**
- Stage all changes. If no changes occurred in this step, proceed with an empty commit.
- Perform the commit with a clear and concise message (e.g., `conductor(checkpoint): Checkpoint end of Phase X`).
7. **Attach Auditable Verification Report using Git Notes:**
- **Step 7.1: Draft Note Content:** Create a detailed verification report including the automated test command, the manual verification steps, and the user's confirmation.
- **Step 7.2: Attach Note:** Use the `git notes` command and the full commit hash from the previous step to attach the full report to the checkpoint commit.
8. **Get and Record Phase Checkpoint SHA:**
- **Step 8.1: Get Commit Hash:** Obtain the hash of the *just-created checkpoint commit* (`git log -1 --format="%H"`).
- **Step 8.2: Update Plan:** Read `plan.md`, find the heading for the completed phase, and append the first 7 characters of the commit hash in the format `[checkpoint: <sha>]`.
- **Step 8.3: Write Plan:** Write the updated content back to `plan.md`.
9. **Commit Plan Update:**
- **Action:** Stage the modified `plan.md` file.
- **Action:** Commit this change with a descriptive message following the format `conductor(plan): Mark phase '<PHASE NAME>' as complete`.
10. **Announce Completion:** Inform the user that the phase is complete and the checkpoint has been created, with the detailed verification report attached as a git note.
### Quality Gates
Before marking any task complete, verify:
- [ ] All tests pass
- [ ] Code coverage meets requirements (>80%)
- [ ] Code follows project's code style guidelines (as defined in `code_styleguides/`)
- [ ] All public functions/methods are documented (e.g., docstrings, JSDoc, GoDoc)
- [ ] Type safety is enforced (e.g., type hints, TypeScript types, Go types)
- [ ] No linting or static analysis errors (using the project's configured tools)
- [ ] Works correctly on mobile (if applicable)
- [ ] Documentation updated if needed
- [ ] No security vulnerabilities introduced
## Development Commands
**AI AGENT INSTRUCTION: This section should be adapted to the project's specific language, framework, and build tools.**
### Setup
```bash
# Example: Commands to set up the development environment (e.g., install dependencies, configure database)
# e.g., for a Node.js project: npm install
# e.g., for a Go project: go mod tidy
```
### Daily Development
```bash
# Example: Commands for common daily tasks (e.g., start dev server, run tests, lint, format)
# e.g., for a Node.js project: npm run dev, npm test, npm run lint
# e.g., for a Go project: go run main.go, go test ./..., go fmt ./...
```
### Before Committing
```bash
# Example: Commands to run all pre-commit checks (e.g., format, lint, type check, run tests)
# e.g., for a Node.js project: npm run check
# e.g., for a Go project: make check (if a Makefile exists)
```
## Testing Requirements
### Unit Testing
- Every module must have corresponding tests.
- Use appropriate test setup/teardown mechanisms (e.g., fixtures, beforeEach/afterEach).
- Mock external dependencies.
- Test both success and failure cases.
### Integration Testing
- Test complete user flows
- Verify database transactions
- Test authentication and authorization
- Check form submissions
### Mobile Testing
- Test on actual iPhone when possible
- Use Safari developer tools
- Test touch interactions
- Verify responsive layouts
- Check performance on 3G/4G
## Code Review Process
### Self-Review Checklist
Before requesting review:
1. **Functionality**
- Feature works as specified
- Edge cases handled
- Error messages are user-friendly
2. **Code Quality**
- Follows style guide
- DRY principle applied
- Clear variable/function names
- Appropriate comments
3. **Testing**
- Unit tests comprehensive
- Integration tests pass
- Coverage adequate (>80%)
4. **Security**
- No hardcoded secrets
- Input validation present
- SQL injection prevented
- XSS protection in place
5. **Performance**
- Database queries optimized
- Images optimized
- Caching implemented where needed
6. **Mobile Experience**
- Touch targets adequate (44x44px)
- Text readable without zooming
- Performance acceptable on mobile
- Interactions feel native
## Commit Guidelines
### Message Format
```
<type>(<scope>): <description>
[optional body]
[optional footer]
```
### Types
- `feat`: New feature
- `fix`: Bug fix
- `docs`: Documentation only
- `style`: Formatting, missing semicolons, etc.
- `refactor`: Code change that neither fixes a bug nor adds a feature
- `test`: Adding missing tests
- `chore`: Maintenance tasks
### Examples
```bash
git commit -m "feat(auth): Add remember me functionality"
git commit -m "fix(posts): Correct excerpt generation for short posts"
git commit -m "test(comments): Add tests for emoji reaction limits"
git commit -m "style(mobile): Improve button touch targets"
```
## Definition of Done
A task is complete when:
1. All code implemented to specification
2. Unit tests written and passing
3. Code coverage meets project requirements
4. Documentation complete (if applicable)
5. Code passes all configured linting and static analysis checks
6. Works beautifully on mobile (if applicable)
7. Implementation notes added to `plan.md`
8. Changes committed with proper message
9. Git note with task summary attached to the commit
## Emergency Procedures
### Critical Bug in Production
1. Create hotfix branch from main
2. Write failing test for bug
3. Implement minimal fix
4. Test thoroughly including mobile
5. Deploy immediately
6. Document in plan.md
### Data Loss
1. Stop all write operations
2. Restore from latest backup
3. Verify data integrity
4. Document incident
5. Update backup procedures
### Security Breach
1. Rotate all secrets immediately
2. Review access logs
3. Patch vulnerability
4. Notify affected users (if any)
5. Document and update security procedures
## Deployment Workflow
### Pre-Deployment Checklist
- [ ] All tests passing
- [ ] Coverage >80%
- [ ] No linting errors
- [ ] Mobile testing complete
- [ ] Environment variables configured
- [ ] Database migrations ready
- [ ] Backup created
### Deployment Steps
1. Merge feature branch to main
2. Tag release with version
3. Push to deployment service
4. Run database migrations
5. Verify deployment
6. Test critical paths
7. Monitor for errors
### Post-Deployment
1. Monitor analytics
2. Check error logs
3. Gather user feedback
4. Plan next iteration
## Continuous Improvement
- Review workflow weekly
- Update based on pain points
- Document lessons learned
- Optimize for user happiness
- Keep things simple and maintainable