5_2_0 Forgejo Our One Source of Truth #3

New issue

Open

opened 2025-10-03 11:16:27 +00:00 by robbert_founder · 0 comments

robbert_founder commented 2025-10-03 11:16:27 +00:00

Owner

Copy link

5_2_0 - Forgejo is truly our one source of truth

Objective Overview

• ID: 5_2_0
• Type: team
• Phase: pre-validation
• Parent: 5_0
• Status: active
• Last Updated By: robbert on 2026-01-13T22:04:19.134457Z
• Forgejo: #3

Governance (ADM)

• Attacker: 3_1
• Defender: 3_7
• Midfielder: Engelbot

Summary

Forgejo must be the canonical, real-time source of truth for SmartupOS.

🔐 Architecture Decision: Engelbot as Gatekeeper

Decision (2025-10-08): Engelbot is the ONLY entity with write access to the ledger.

Rationale:

• Single point of constitutional enforcement
• No token distribution to humans
• Perfect attribution (Matrix ID → owner record → action)
• Aligns with Ground Rule 5 (Act in the Open)

Implementation:

• Engelbot has admin access on Forgejo
• Toolbox scripts write via Forgejo API using Engelbot's token
• Humans execute operations via bot commands (!create_task, !assess_work, etc.)
• Bot verifies identity and permissions before executing scripts

Impact on 5_2_0:

• All toolbox scripts must support bot-mode (--json, --actor parameter)
• Permission checking happens in Engelbot layer
• Scripts trust the actor parameter (Engelbot already verified)

---

🏗️ Architecture Vision

---
config:
  layout: elk
---
flowchart TB
 subgraph Human["👥 Humans (Owners, Captains, Members)"]
        A1["Matrix / Engelbot Chat"]
        A2["Forgejo UI - View Repos & Wikis"]
        A3["timeline0.org - Public Transparency"]
  end
 
 subgraph Middleware["🤖 Middleware (Production)"]
        B1["Engelbot (TypeScript)\n- Command Parser\n- Calls Toolbox"]
        B2["Toolbox Scripts (Python)\n- Direct Forgejo API writes\n- YAML policy enforcement\n- Permission validation"]
  end
 
 subgraph Forgejo["📦 Forgejo — One Source of Truth (API)"]
        subgraph Repos["Core Repositories"]
            C1["1_general_forum"]
            C2["2_workplace"]
            C3["3_X_team"]
        end
        
        subgraph Wikis["Canonical Wikis"]
            W1["1_general_forum.wiki\n- Public dashboards"]
            W2["2_workplace.wiki\n- Workspace dashboards"]
            W3["3_X_team.wiki\n- Team dashboards"]
        end
        
        subgraph Ledger["Ledger (CSV via API)"]
            L1["owners, roles, licenses"]
            L2["tasks, objectives, sessions"]
            L3["smartup-credits.csv"]
            L4["social-karma.csv"]
            L5["treasury/balance.csv"]
            L6["master-events.csv"]
        end
  end

    A1 -->|!commands| B1
    B1 -->|executes| B2
    B2 -->|"HTTP API\n(read + write)"| Ledger
    B2 -->|"HTTP API\n(read + write)"| Wikis
    
    Ledger -->|"auto-generates"| W1 & W2 & W3
    W1 & W2 & W3 -->|"public pages"| A3
    Repos & Wikis -->|"UI access"| A2
    
    style B2 fill:#00b894
    style Ledger fill:#fdcb6e

Key Changes from v1:

• 🔴 OLD: Toolbox writes local files → manual git push → Forgejo eventually consistent
• 🟢 NEW: Toolbox writes directly via Forgejo API → immediate consistency → no manual git operations

---

✅ Production Readiness Checklist

1. Forgejo API Write Layer 🔴 CRITICAL

• forgejo_api.py utility module created with:
  • fetch_csv(repo, path) → reads CSV as list of dicts
  • update_csv(repo, path, data, commit_msg) → atomic writes
  • append_to_csv(repo, path, row, commit_msg) → append-only operations
  • Authentication handling (API token, error retry)
  • Concurrent write conflict detection/resolution
• Prototype tested against staging repo
• Architecture document explains pattern for all developers
• Task: 6_4_2_0 (Design), 6_2_2_2 (Implementation)

---

2. Toolbox Script Refactoring 🟡 HIGH PRIORITY

All write scripts must use Forgejo API, not local file operations.

Team Captain Scripts:

• create_task.py → API writes to task-budgets.csv
• create_owner.py → API writes to book-of-owners.csv, identity-mapping.csv
• assess_work.py → API writes to pending-sc/transactions.csv
• validate_pending_sc.py → API writes to smartup-credits/transactions.csv, treasury/balance.csv
• assign_role.py, set_role.py → API writes to book-of-owners.csv
• create_objective.py → API writes to objectives/registry.csv
• connect_matrix_owner.py → API writes to book-of-owners.csv, creates passport
• Task: 6_5_2_0 (Captain scripts refactor)

Worker Scripts:

• claim_task.py → API writes to task_claimed.csv
• start_work.py, stop_work.py → API writes to work_clock.csv, session_logs.csv
• Task: 6_6_2_0 (Worker scripts refactor)

Ledger Maintenance:

• validate_ledger.py → reads via API (no changes, verify works)
• generate_public_pages.py → reads via API, writes to wiki repos
• start_slog.py → API writes to wiki .md files
• bootstrap_wiki.py, bootstrap_teams.py → API writes (one-time, verify)
• Task: 6_7_2_0 (Maintenance scripts refactor)

---

3. YAML Policy Enforcement 🟡 HIGH PRIORITY

All scripts must validate against policy files before write operations.

• policy_validator.py utility created:
  • Loads YAML policies from currency-ledger/policies/
  • Validates permissions (e.g., "is this user a Team Captain?")
  • Validates business rules (e.g., "does treasury 3× rule allow this SC award?")
  • Returns clear error messages on violations
• Integrated into all write scripts (called before API writes)
• Policies tested:
  • validation-rules.yml (treasury 3× rule)
  • credit-rates.yml (SC assessment percentages)
  • organizational-structure.yml (team/role definitions)
  • license-policies.yml (license types, pricing)
• Task: 6_8_2_0 (Policy enforcement layer)

---

4. Engelbot-Toolbox Integration 🔴 CRITICAL

Engelbot must be able to call Toolbox scripts in production (Clever Cloud).

Decision: Bundle Python in Docker (Option A for MVP)

• Dockerfile updated:

  FROM node:18
  RUN apt-get update && apt-get install -y python3 python3-pip
  COPY requirements.txt /app/
  RUN pip3 install -r requirements.txt
  COPY toolbox/ /app/toolbox/
  COPY 3_7_operational_team/ /app/
  CMD ["npm", "start"]
  

• requirements.txt created with dependencies (requests, pyyaml, python-dateutil)
• Environment variables configured:
  • FORGEJO_URL
  • FORGEJO_TOKEN
  • FORGEJO_ORG
  • TOOLBOX_PATH=/app/toolbox/running
• Engelbot commands tested calling refactored scripts:
  • !create_task → calls create_task.py --json
  • !whoami → calls who_am_i.py --json
  • !tasks → calls show_tasks.py --json
• Error handling: Python script failures reported in Matrix
• Task: 6_9_2_0 (Deployment architecture)

---

5. Testing & Validation 🟢 MEDIUM PRIORITY

Comprehensive testing before Experiment Launch.

• Integration test suite created:
  • Test: Create owner → assign role → create task → claim → work → assess → validate
  • Test: Concurrent writes (3 captains create tasks simultaneously)
  • Test: Permission violations (junior tries to validate SC)
  • Test: Treasury rule violations (award SC when treasury insufficient)
  • Test: Rollback handling (API write fails mid-operation)
• Staging environment:
  • Test repos in Forgejo (not production ledger)
  • Test Matrix homeserver (or isolated rooms)
• Smoke tests after deployment:
  • End-to-end onboarding simulation
  • All Engelbot commands functional
  • Ledger validation passes (validate_ledger.py)
• Task: 6_10_2_0 (Integration testing)

---

6. Production Deployment 🔴 LAUNCH BLOCKER

Final deployment and go-live.

• Clever Cloud deployment successful
• Environment variables verified
• Engelbot connects to Matrix and Forgejo
• Test with founders (dogfooding):
  • Robbert creates task via !create_task
  • Sarah claims and completes task
  • Eva assesses and validates
  • All writes reflected immediately in Forgejo
• Rollback plan documented (if critical bug found)
• Task: 6_11_2_0 (Deployment + smoke tests)

---

📊 Success Criteria

BEFORE (Current State):

• ❌ Toolbox writes local files
• ❌ Manual git push required
• ❌ Concurrent writes cause collisions
• ❌ Engelbot can't run scripts in production

AFTER (Mission 5_2_0 Complete):

• ✅ Toolbox writes directly to Forgejo via API
• ✅ No manual git operations needed
• ✅ Concurrent writes handled safely
• ✅ YAML policies enforced consistently
• ✅ Engelbot production-ready (Python bundled in Docker)
• ✅ End-to-end test: 3 founders complete task flow without issues
• ✅ System ready for external contributors (scalable to 20+ users)

Validation:

• All 9 subtasks (6_3_2_0 through 6_11_2_0) complete
• validate_ledger.py passes with zero errors
• Dogfooding session successful (6 tasks completed end-to-end)
• Founder consensus: "This is ready for external contributors"

---

📅 Timeline Estimate

Complexity: HIGH (foundational refactor)
Estimated Effort: 80-120 hours (570 SC budgeted across 9 tasks)
Timeline: 2-3 weeks (Oct 8 → Oct 28)

Dependencies:

• Blocks Mission 5_1_0 completion (room automation needs stable write layer)
• Blocks Mission 5_6_0 completion (Engelbot needs production Toolbox)
• Blocks Mission 5_3_0 completion (onboarding needs working write layer)
• This is the critical path to Experiment Launch

---

🚧 Known Risks

1. Forgejo API limitations → Discover via Task 6_3_2_0, adjust approach if needed
2. Concurrent write complexity → May need more sophisticated conflict resolution than planned
3. Script refactoring scope creep → 30+ scripts to update, edge cases may emerge
4. Docker deployment complexity → Python dependencies, environment variables, debugging in production
5. Testing insufficient → May not catch all edge cases until real users stress the system
6. Rollback difficulty → Once live, reverting to old architecture is painful

Mitigation:

• Early prototype (Task 6_3_2_0) de-risks API approach
• Incremental refactoring (Captain scripts → Worker scripts → Maintenance)
• Comprehensive testing before deployment
• Staged rollout: Founders test first, then external contributors

---

🎯 Task Breakdown (Objectives → Tasks)

Mission 5_2_0: Forgejo as One Source of Truth
│
├── 6_3_2_0: Design Forgejo API write layer (50 SC, 5-7 hours)
│   └── Deliverable: Architecture doc + prototype utilities
│
├── 6_4_2_0: Implement forgejo_api.py module (50 SC, 5-7 hours)
│   └── Deliverable: Production-ready API utilities + tests
│
├── 6_5_2_0: Refactor Team Captain scripts (100 SC, 12-15 hours)
│   └── Deliverable: 8 scripts using API writes + YAML validation
│
├── 6_6_2_0: Refactor Worker scripts (50 SC, 5-7 hours)
│   └── Deliverable: 3 scripts using API writes
│
├── 6_7_2_0: Refactor Ledger maintenance scripts (50 SC, 5-7 hours)
│   └── Deliverable: 5 scripts using API reads/writes
│
├── 6_8_2_0: YAML policy enforcement layer (100 SC, 12-15 hours)
│   └── Deliverable: policy_validator.py + integrated into all scripts
│
├── 6_9_2_0: Engelbot-Toolbox deployment (100 SC, 12-15 hours)
│   └── Deliverable: Dockerfile + Clever Cloud deployment + env config
│
├── 6_10_2_0: Integration testing suite (50 SC, 5-7 hours)
│   └── Deliverable: Test scenarios + staging environment + results
│
└── 6_11_2_0: Production deployment (50 SC, 5-7 hours)
    └── Deliverable: Live system + dogfooding results + go/no-go decision

Total Budget: 570 SC (~€570 at €1/SC conversion)

---

📈 Progress Tracking

Current Status (Oct 8, 2025): 85% → Dropping to ~40% (honest reassessment)

Why the drop?

• Previous "85%" assumed local file writes acceptable
• Architecture decision (direct API writes) requires significant refactoring
• YAML policy enforcement was documented but not implemented
• Engelbot production deployment was theoretical

Realistic assessment:

• ✅ Complete: Repository structure, wiki scaffolding, basic scripts working locally
• ✅ Complete: API write layer design (Task 6_3_2_0 ), 6_4_2_0
• In progress: 6_5_20

Updated Completion Estimate:

Week	Tasks	Cumulative %
Week 1 (Oct 8-14)	6_3_2_0 (Design) → 6_4_2_0 (API utils)	40% → 50%
Week 2 (Oct 15-21)	6_5_2_0 (Captain scripts) → 6_6_2_0 (Worker scripts)	50% → 70%
Week 3 (Oct 22-28)	6_7_2_0 (Maintenance) → 6_8_2_0 (YAML) → 6_9_2_0 (Docker)	70% → 90%
Week 4 (Oct 29-Nov 4)	6_10_2_0 (Testing) → 6_11_2_0 (Deployment) → Dogfooding	90% → 100%

Target Completion: November 4, 2025 (4 weeks from now)

---

🔗 Dependencies

Blocks:

• ❌ 5_1_0 (Matrix/Element hub) - Room automation needs stable write layer
• ❌ 5_3_0 (Onboarding flow) - Cannot onboard users without working scripts
• ❌ 5_6_0 (Engelbot v0.1) - Cannot deploy bot without Toolbox integration
• ❌ Experiment Launch - This is the critical path

Blocked By:

• None (this is the foundation layer)

Enables:

• ✅ Concurrent contributor work - Multiple captains can operate simultaneously
• ✅ Real-time transparency - Public pages auto-update as data changes
• ✅ Engelbot production - Bot can execute operations reliably
• ✅ Scalability - System ready for 20+ active contributors

---

💬 Discussion

Key Questions to Resolve:

1. Concurrent Write Strategy: Optimistic (detect + retry) vs Pessimistic (locking)?

   • Decision needed in Task 6_3_2_0

2. API Rate Limits: Does Forgejo API have request limits we need to respect?

   • Research during Task 6_4_2_0

3. Rollback Strategy: If API writes fail mid-transaction, how do we undo?

   • Design during Task 6_5_2_0

4. YAML Policy Priority: Which policies are MVP-critical vs. nice-to-have?

   • Prioritize during Task 6_6_2_0

5. Testing Coverage: What percentage of code paths must be tested before go-live?

   • Define during Task 6_7_2_0

Matrix Room: #5_2_forgejo_osot:smartup0.org (to be created)
Forgejo Project: Track all 9 subtasks in kanban board

---

📚 Resources & References

Technical:

• Forgejo API Documentation
• Forgejo File API
• Python Requests Library
• PyYAML Documentation

Internal:

• Current Toolbox scripts: 3_1_leadership_team/toolbox/running/
• Policy files: 2_workplace/currency-ledger/policies/
• Architecture briefing: This document (Part 3.1-3.2)

Prior Art:

• GitHub API file writes: Similar pattern, proven at scale
• GitLab API: Alternative reference for conflict handling

---

🎓 Learning Outcomes

For Smartup Zero:

• Production-ready write layer architecture
• YAML-driven policy enforcement pattern
• Dockerized multi-runtime deployment (Node + Python)
• Concurrent write handling strategy

For SmartupOS (reusable for other Smartups):

• API-first ledger pattern (not git-first)
• Policy-as-code enforcement layer
• Bot-to-toolbox integration architecture
• Comprehensive testing approach for distributed systems

Documentation deliverables:

• Architecture decision record (Task 6_2_2_1)
• Developer guide: "How to write a Toolbox script"
• Operations guide: "Deploying Engelbot + Toolbox"
• Testing playbook: "Validating ledger integrity"

---

🏁 Definition of Done

Mission 5_2_0 is complete when:

• All 9 subtasks (6_3_2_0) through (6_12_2_0) marked complete in ledger
• validate_ledger.py runs against production ledger with zero errors
• All Toolbox scripts use Forgejo API (no local file writes remain)
• YAML policies enforced in all write operations
• Engelbot deployed on Clever Cloud with Python runtime
• Integration tests pass (end-to-end task flow)
• Dogfooding successful:
  • 3 founders (Robbert, Sarah, Eva) complete 6 tasks end-to-end
  • No manual git operations required
  • No data integrity issues detected
  • Concurrent writes handled without collisions
• Founder consensus: "Ready for external contributors"
• Architecture documented for future developers
• Rollback plan documented (in case of critical issues post-launch)

---

Progress (Autogenerated)

Progress: 81% (9/11 tasks complete)

Status	Count
✅ Complete	9
📂 Open	2

Last Activity: 2026-02-21 by @robbert

Health: 🔴 Inactive

# 5_2_0 - Forgejo is truly our one source of truth ## Objective Overview - **ID:** 5_2_0 - **Type:** team - **Phase:** pre-validation - **Parent:** 5_0 - **Status:** active - **Last Updated By:** robbert on 2026-01-13T22:04:19.134457Z - **Forgejo:** https://forge.timeline0.org/Smartup_Zero/2_workplace/issues/3 ## Governance (ADM) - **Attacker:** 3_1 - **Defender:** 3_7 - **Midfielder:** Engelbot ## Summary Forgejo must be the canonical, real-time source of truth for SmartupOS. ## 🔐 Architecture Decision: Engelbot as Gatekeeper **Decision (2025-10-08):** Engelbot is the ONLY entity with write access to the ledger. **Rationale:** - Single point of constitutional enforcement - No token distribution to humans - Perfect attribution (Matrix ID → owner record → action) - Aligns with Ground Rule 5 (Act in the Open) **Implementation:** - Engelbot has admin access on Forgejo - Toolbox scripts write via Forgejo API using Engelbot's token - Humans execute operations via bot commands (!create_task, !assess_work, etc.) - Bot verifies identity and permissions before executing scripts **Impact on 5_2_0:** - All toolbox scripts must support bot-mode (--json, --actor parameter) - Permission checking happens in Engelbot layer - Scripts trust the actor parameter (Engelbot already verified) --- ## 🏗️ Architecture Vision ```mermaid --- config: layout: elk --- flowchart TB subgraph Human["👥 Humans (Owners, Captains, Members)"] A1["Matrix / Engelbot Chat"] A2["Forgejo UI - View Repos & Wikis"] A3["timeline0.org - Public Transparency"] end subgraph Middleware["🤖 Middleware (Production)"] B1["Engelbot (TypeScript)\n- Command Parser\n- Calls Toolbox"] B2["Toolbox Scripts (Python)\n- Direct Forgejo API writes\n- YAML policy enforcement\n- Permission validation"] end subgraph Forgejo["📦 Forgejo — One Source of Truth (API)"] subgraph Repos["Core Repositories"] C1["1_general_forum"] C2["2_workplace"] C3["3_X_team"] end subgraph Wikis["Canonical Wikis"] W1["1_general_forum.wiki\n- Public dashboards"] W2["2_workplace.wiki\n- Workspace dashboards"] W3["3_X_team.wiki\n- Team dashboards"] end subgraph Ledger["Ledger (CSV via API)"] L1["owners, roles, licenses"] L2["tasks, objectives, sessions"] L3["smartup-credits.csv"] L4["social-karma.csv"] L5["treasury/balance.csv"] L6["master-events.csv"] end end A1 -->|!commands| B1 B1 -->|executes| B2 B2 -->|"HTTP API\n(read + write)"| Ledger B2 -->|"HTTP API\n(read + write)"| Wikis Ledger -->|"auto-generates"| W1 & W2 & W3 W1 & W2 & W3 -->|"public pages"| A3 Repos & Wikis -->|"UI access"| A2 style B2 fill:#00b894 style Ledger fill:#fdcb6e ``` **Key Changes from v1:** - 🔴 **OLD:** Toolbox writes local files → manual git push → Forgejo eventually consistent - 🟢 **NEW:** Toolbox writes directly via Forgejo API → immediate consistency → no manual git operations --- ## ✅ Production Readiness Checklist ### **1. Forgejo API Write Layer** 🔴 CRITICAL - [ ] `forgejo_api.py` utility module created with: - [ ] `fetch_csv(repo, path)` → reads CSV as list of dicts - [ ] `update_csv(repo, path, data, commit_msg)` → atomic writes - [ ] `append_to_csv(repo, path, row, commit_msg)` → append-only operations - [ ] Authentication handling (API token, error retry) - [ ] Concurrent write conflict detection/resolution - [ ] Prototype tested against staging repo - [ ] Architecture document explains pattern for all developers - [ ] **Task:** 6_4_2_0 (Design), 6_2_2_2 (Implementation) --- ### **2. Toolbox Script Refactoring** 🟡 HIGH PRIORITY All write scripts must use Forgejo API, not local file operations. **Team Captain Scripts:** - [ ] `create_task.py` → API writes to `task-budgets.csv` - [ ] `create_owner.py` → API writes to `book-of-owners.csv`, `identity-mapping.csv` - [ ] `assess_work.py` → API writes to `pending-sc/transactions.csv` - [ ] `validate_pending_sc.py` → API writes to `smartup-credits/transactions.csv`, `treasury/balance.csv` - [ ] `assign_role.py`, `set_role.py` → API writes to `book-of-owners.csv` - [ ] `create_objective.py` → API writes to `objectives/registry.csv` - [ ] `connect_matrix_owner.py` → API writes to `book-of-owners.csv`, creates passport - [ ] **Task:** 6_5_2_0 (Captain scripts refactor) **Worker Scripts:** - [ ] `claim_task.py` → API writes to `task_claimed.csv` - [ ] `start_work.py`, `stop_work.py` → API writes to `work_clock.csv`, `session_logs.csv` - [ ] **Task:** 6_6_2_0 (Worker scripts refactor) **Ledger Maintenance:** - [ ] `validate_ledger.py` → reads via API (no changes, verify works) - [ ] `generate_public_pages.py` → reads via API, writes to wiki repos - [ ] `start_slog.py` → API writes to wiki `.md` files - [ ] `bootstrap_wiki.py`, `bootstrap_teams.py` → API writes (one-time, verify) - [ ] **Task:** 6_7_2_0 (Maintenance scripts refactor) --- ### **3. YAML Policy Enforcement** 🟡 HIGH PRIORITY All scripts must validate against policy files before write operations. - [ ] `policy_validator.py` utility created: - [ ] Loads YAML policies from `currency-ledger/policies/` - [ ] Validates permissions (e.g., "is this user a Team Captain?") - [ ] Validates business rules (e.g., "does treasury 3× rule allow this SC award?") - [ ] Returns clear error messages on violations - [ ] Integrated into all write scripts (called before API writes) - [ ] Policies tested: - [ ] `validation-rules.yml` (treasury 3× rule) - [ ] `credit-rates.yml` (SC assessment percentages) - [ ] `organizational-structure.yml` (team/role definitions) - [ ] `license-policies.yml` (license types, pricing) - [ ] **Task:** 6_8_2_0 (Policy enforcement layer) --- ### **4. Engelbot-Toolbox Integration** 🔴 CRITICAL Engelbot must be able to call Toolbox scripts in production (Clever Cloud). **Decision:** Bundle Python in Docker (Option A for MVP) - [ ] Dockerfile updated: ```dockerfile FROM node:18 RUN apt-get update && apt-get install -y python3 python3-pip COPY requirements.txt /app/ RUN pip3 install -r requirements.txt COPY toolbox/ /app/toolbox/ COPY 3_7_operational_team/ /app/ CMD ["npm", "start"] ``` - [ ] `requirements.txt` created with dependencies (requests, pyyaml, python-dateutil) - [ ] Environment variables configured: - [ ] `FORGEJO_URL` - [ ] `FORGEJO_TOKEN` - [ ] `FORGEJO_ORG` - [ ] `TOOLBOX_PATH=/app/toolbox/running` - [ ] Engelbot commands tested calling refactored scripts: - [ ] `!create_task` → calls `create_task.py --json` - [ ] `!whoami` → calls `who_am_i.py --json` - [ ] `!tasks` → calls `show_tasks.py --json` - [ ] Error handling: Python script failures reported in Matrix - [ ] **Task:** 6_9_2_0 (Deployment architecture) --- ### **5. Testing & Validation** 🟢 MEDIUM PRIORITY Comprehensive testing before Experiment Launch. - [ ] Integration test suite created: - [ ] Test: Create owner → assign role → create task → claim → work → assess → validate - [ ] Test: Concurrent writes (3 captains create tasks simultaneously) - [ ] Test: Permission violations (junior tries to validate SC) - [ ] Test: Treasury rule violations (award SC when treasury insufficient) - [ ] Test: Rollback handling (API write fails mid-operation) - [ ] Staging environment: - [ ] Test repos in Forgejo (not production ledger) - [ ] Test Matrix homeserver (or isolated rooms) - [ ] Smoke tests after deployment: - [ ] End-to-end onboarding simulation - [ ] All Engelbot commands functional - [ ] Ledger validation passes (`validate_ledger.py`) - [ ] **Task:** 6_10_2_0 (Integration testing) --- ### **6. Production Deployment** 🔴 LAUNCH BLOCKER Final deployment and go-live. - [ ] Clever Cloud deployment successful - [ ] Environment variables verified - [ ] Engelbot connects to Matrix and Forgejo - [ ] Test with founders (dogfooding): - [ ] Robbert creates task via `!create_task` - [ ] Sarah claims and completes task - [ ] Eva assesses and validates - [ ] All writes reflected immediately in Forgejo - [ ] Rollback plan documented (if critical bug found) - [ ] **Task:** 6_11_2_0 (Deployment + smoke tests) --- ## 📊 Success Criteria **BEFORE (Current State):** - ❌ Toolbox writes local files - ❌ Manual git push required - ❌ Concurrent writes cause collisions - ❌ Engelbot can't run scripts in production **AFTER (Mission 5_2_0 Complete):** - ✅ Toolbox writes directly to Forgejo via API - ✅ No manual git operations needed - ✅ Concurrent writes handled safely - ✅ YAML policies enforced consistently - ✅ Engelbot production-ready (Python bundled in Docker) - ✅ End-to-end test: 3 founders complete task flow without issues - ✅ System ready for external contributors (scalable to 20+ users) **Validation:** - All 9 subtasks (6_3_2_0 through 6_11_2_0) complete - `validate_ledger.py` passes with zero errors - Dogfooding session successful (6 tasks completed end-to-end) - Founder consensus: "This is ready for external contributors" --- ## 📅 Timeline Estimate **Complexity:** HIGH (foundational refactor) **Estimated Effort:** 80-120 hours (570 SC budgeted across 9 tasks) **Timeline:** 2-3 weeks (Oct 8 → Oct 28) **Dependencies:** - Blocks Mission 5_1_0 completion (room automation needs stable write layer) - Blocks Mission 5_6_0 completion (Engelbot needs production Toolbox) - Blocks Mission 5_3_0 completion (onboarding needs working write layer) - **This is the critical path to Experiment Launch** --- ## 🚧 Known Risks 1. **Forgejo API limitations** → Discover via Task 6_3_2_0, adjust approach if needed 2. **Concurrent write complexity** → May need more sophisticated conflict resolution than planned 3. **Script refactoring scope creep** → 30+ scripts to update, edge cases may emerge 4. **Docker deployment complexity** → Python dependencies, environment variables, debugging in production 5. **Testing insufficient** → May not catch all edge cases until real users stress the system 6. **Rollback difficulty** → Once live, reverting to old architecture is painful **Mitigation:** - Early prototype (Task 6_3_2_0) de-risks API approach - Incremental refactoring (Captain scripts → Worker scripts → Maintenance) - Comprehensive testing before deployment - Staged rollout: Founders test first, then external contributors --- ## 🎯 Task Breakdown (Objectives → Tasks) ``` Mission 5_2_0: Forgejo as One Source of Truth │ ├── 6_3_2_0: Design Forgejo API write layer (50 SC, 5-7 hours) │ └── Deliverable: Architecture doc + prototype utilities │ ├── 6_4_2_0: Implement forgejo_api.py module (50 SC, 5-7 hours) │ └── Deliverable: Production-ready API utilities + tests │ ├── 6_5_2_0: Refactor Team Captain scripts (100 SC, 12-15 hours) │ └── Deliverable: 8 scripts using API writes + YAML validation │ ├── 6_6_2_0: Refactor Worker scripts (50 SC, 5-7 hours) │ └── Deliverable: 3 scripts using API writes │ ├── 6_7_2_0: Refactor Ledger maintenance scripts (50 SC, 5-7 hours) │ └── Deliverable: 5 scripts using API reads/writes │ ├── 6_8_2_0: YAML policy enforcement layer (100 SC, 12-15 hours) │ └── Deliverable: policy_validator.py + integrated into all scripts │ ├── 6_9_2_0: Engelbot-Toolbox deployment (100 SC, 12-15 hours) │ └── Deliverable: Dockerfile + Clever Cloud deployment + env config │ ├── 6_10_2_0: Integration testing suite (50 SC, 5-7 hours) │ └── Deliverable: Test scenarios + staging environment + results │ └── 6_11_2_0: Production deployment (50 SC, 5-7 hours) └── Deliverable: Live system + dogfooding results + go/no-go decision ``` **Total Budget:** 570 SC (~€570 at €1/SC conversion) --- ## 📈 Progress Tracking **Current Status (Oct 8, 2025):** 85% → Dropping to ~40% (honest reassessment) **Why the drop?** - Previous "85%" assumed local file writes acceptable - Architecture decision (direct API writes) requires significant refactoring - YAML policy enforcement was documented but not implemented - Engelbot production deployment was theoretical **Realistic assessment:** - ✅ **Complete:** Repository structure, wiki scaffolding, basic scripts working locally - ✅ **Complete:** API write layer design (Task 6_3_2_0 ), 6_4_2_0 - In progress: 6_5_20 **Updated Completion Estimate:** | Week | Tasks | Cumulative % | |------|-------|--------------| | Week 1 (Oct 8-14) | 6_3_2_0 (Design) → 6_4_2_0 (API utils) | 40% → 50% | | Week 2 (Oct 15-21) | 6_5_2_0 (Captain scripts) → 6_6_2_0 (Worker scripts) | 50% → 70% | | Week 3 (Oct 22-28) | 6_7_2_0 (Maintenance) → 6_8_2_0 (YAML) → 6_9_2_0 (Docker) | 70% → 90% | | Week 4 (Oct 29-Nov 4) | 6_10_2_0 (Testing) → 6_11_2_0 (Deployment) → Dogfooding | 90% → 100% | **Target Completion:** November 4, 2025 (4 weeks from now) --- ## 🔗 Dependencies **Blocks:** - ❌ **5_1_0** (Matrix/Element hub) - Room automation needs stable write layer - ❌ **5_3_0** (Onboarding flow) - Cannot onboard users without working scripts - ❌ **5_6_0** (Engelbot v0.1) - Cannot deploy bot without Toolbox integration - ❌ **Experiment Launch** - This is the critical path **Blocked By:** - None (this is the foundation layer) **Enables:** - ✅ **Concurrent contributor work** - Multiple captains can operate simultaneously - ✅ **Real-time transparency** - Public pages auto-update as data changes - ✅ **Engelbot production** - Bot can execute operations reliably - ✅ **Scalability** - System ready for 20+ active contributors --- ## 💬 Discussion **Key Questions to Resolve:** 1. **Concurrent Write Strategy:** Optimistic (detect + retry) vs Pessimistic (locking)? - *Decision needed in Task 6_3_2_0* 2. **API Rate Limits:** Does Forgejo API have request limits we need to respect? - *Research during Task 6_4_2_0* 3. **Rollback Strategy:** If API writes fail mid-transaction, how do we undo? - *Design during Task 6_5_2_0* 4. **YAML Policy Priority:** Which policies are MVP-critical vs. nice-to-have? - *Prioritize during Task 6_6_2_0* 5. **Testing Coverage:** What percentage of code paths must be tested before go-live? - *Define during Task 6_7_2_0* **Matrix Room:** `#5_2_forgejo_osot:smartup0.org` (to be created) **Forgejo Project:** Track all 9 subtasks in kanban board --- ## 📚 Resources & References **Technical:** - [Forgejo API Documentation](https://forgejo.org/docs/latest/api/) - [Forgejo File API](https://forgejo.org/docs/latest/api/repo-file/) - [Python Requests Library](https://requests.readthedocs.io/) - [PyYAML Documentation](https://pyyaml.org/wiki/PyYAMLDocumentation) **Internal:** - Current Toolbox scripts: `3_1_leadership_team/toolbox/running/` - Policy files: `2_workplace/currency-ledger/policies/` - Architecture briefing: This document (Part 3.1-3.2) **Prior Art:** - GitHub API file writes: Similar pattern, proven at scale - GitLab API: Alternative reference for conflict handling --- ## 🎓 Learning Outcomes **For Smartup Zero:** - Production-ready write layer architecture - YAML-driven policy enforcement pattern - Dockerized multi-runtime deployment (Node + Python) - Concurrent write handling strategy **For SmartupOS (reusable for other Smartups):** - API-first ledger pattern (not git-first) - Policy-as-code enforcement layer - Bot-to-toolbox integration architecture - Comprehensive testing approach for distributed systems **Documentation deliverables:** - Architecture decision record (Task 6_2_2_1) - Developer guide: "How to write a Toolbox script" - Operations guide: "Deploying Engelbot + Toolbox" - Testing playbook: "Validating ledger integrity" --- ## 🏁 Definition of Done Mission 5_2_0 is **complete** when: - [x] All 9 subtasks (6_3_2_0) through (6_12_2_0) marked complete in ledger - [x] `validate_ledger.py` runs against production ledger with **zero errors** - [x] All Toolbox scripts use Forgejo API (no local file writes remain) - [x] YAML policies enforced in all write operations - [x] Engelbot deployed on Clever Cloud with Python runtime - [x] Integration tests pass (end-to-end task flow) - [x] Dogfooding successful: - [x] 3 founders (Robbert, Sarah, Eva) complete 6 tasks end-to-end - [x] No manual git operations required - [x] No data integrity issues detected - [x] Concurrent writes handled without collisions - [x] Founder consensus: **"Ready for external contributors"** - [x] Architecture documented for future developers - [x] Rollback plan documented (in case of critical issues post-launch) --- ## Progress (Autogenerated) <!-- SMARTUPOS:AUTOGEN:PROGRESS:BEGIN --> **Progress: 81%** (9/11 tasks complete) | Status | Count | |--------|-------| | ✅ Complete | 9 | | 📂 Open | 2 | **Last Activity:** 2026-02-21 by @robbert **Health:** 🔴 Inactive <!-- SMARTUPOS:AUTOGEN:PROGRESS:END -->

robbert_founder added the

2_workplace

3_1_leadership_team

labels 2025-10-03 11:16:27 +00:00

Engelbot was assigned by robbert_founder 2025-10-03 11:16:28 +00:00

robbert_founder added this to the 5_0 SmartupOS MVP ready project 2025-10-03 11:16:28 +00:00

robbert_founder added a new dependency 2025-10-03 11:24:43 +00:00

Smartup_Zero/1_general_forum#2 - 5_0 SmartupOS MVP ready

robbert_founder added a new dependency 2025-10-03 11:26:12 +00:00

Smartup_Zero/3_1_leadership_team#9 - 6_2_2_0 Design, Codify, and Bootstrap the Canonical Wiki Structures for Smartups

robbert_founder added a new dependency 2025-10-03 11:49:23 +00:00

Smartup_Zero/3_1_leadership_team#8 - 6_1_2_0 Implement Role‑Anchored Logging System (SLOG / Smartup Log)

robbert_founder removed a dependency 2025-10-09 09:53:12 +00:00

Smartup_Zero/3_1_leadership_team#8 - 6_1_2_0 Implement Role‑Anchored Logging System (SLOG / Smartup Log)

robbert_founder removed a dependency 2025-10-09 09:53:18 +00:00

Smartup_Zero/3_1_leadership_team#9 - 6_2_2_0 Design, Codify, and Bootstrap the Canonical Wiki Structures for Smartups

robbert_founder added a new dependency 2025-10-09 09:55:28 +00:00

Smartup_Zero/3_7_operational_team#11 - 6_2_7_2 - Runtime YAML Policy Enforcement Research

robbert_founder removed a dependency 2025-10-09 09:56:06 +00:00

Smartup_Zero/1_general_forum#2 - 5_0 SmartupOS MVP ready

robbert_founder added a new dependency 2025-10-09 09:57:36 +00:00

Smartup_Zero/1_general_forum#2 - 5_0 SmartupOS MVP ready

robbert_founder removed a dependency 2025-10-09 12:12:17 +00:00

Smartup_Zero/3_7_operational_team#11 - 6_2_7_2 - Runtime YAML Policy Enforcement Research

robbert_founder added a new dependency 2025-10-09 12:12:43 +00:00

Smartup_Zero/3_1_leadership_team#20 - 6_3_2_0 - Design Forgejo API Write Layer for Ledger

robbert_founder added a new dependency 2025-10-09 14:22:51 +00:00

Smartup_Zero/3_1_leadership_team#21 - 6_4_2_0 - Implement forgejo_api.py Module

Engelbot referenced this issue from Smartup_Zero/1_general_forum 2026-02-18 04:52:54 +00:00

5_0 SmartupOS MVP ready #2

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

master

dev_local

corrupt-csv

robbert_ledger_local_fix

No results found.

Labels

Clear labels   

2_workplace

attacker

  

3_1_leadership_team

defender

  

objective

Role: objective

  

pre-validation

Role: pre-validation

  

team-objective

Role: team-objective

  

validation

Role: validation

No labels

2_workplace

3_1_leadership_team

objective

pre-validation

team-objective

validation

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

5_0 SmartupOS MVP ready

Assignees

Clear assignees

No assignees

Engelbot

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks

#2 5_0 SmartupOS MVP ready

Smartup_Zero/1_general_forum

Depends on

You do not have permission to read 2 dependencies

Reference

Smartup_Zero/2_workplace#3

No description provided.