DAW/furt

Fork 0

Update Installation.md for helper scripts system #102

New issue

Closed

opened 2025-09-03 22:36:33 +02:00 by michael · 5 comments

michael commented

2025-09-03 22:36:33 +02:00

Owner

Purpose

Update Installation.md to include the new helper scripts system from Issue #87.

Current Gap

Installation.md currently describes manual step-by-step installation. With the new helper scripts system (install.sh + 6 helper scripts), users have multiple installation paths but the documentation doesn't reflect this.

Required Documentation Updates

1. Installation Methods Section

Add overview of available installation methods:

Automated Installation: ./install.sh (recommended)
Manual Installation: Step-by-step (current documentation)
Update Installation: ./install.sh --upgrade

2. Helper Scripts Documentation

Document each helper script with purpose and usage:

install.sh - Main orchestrator with --upgrade support
scripts/setup-user.sh - System user creation
scripts/setup-directories.sh - Directory structure
scripts/sync-files.sh - Source file installation
scripts/create-service.sh - Service creation from templates
scripts/validate-config.sh - Configuration validation
scripts/health-check.sh - Health check functionality

3. Update Workflow Documentation

Add section for updating existing installations:

# Git-based updates
cd /path/to/furt-source
git pull
./install.sh --upgrade
sudo systemctl restart furt  # Linux
doas rcctl restart furt      # OpenBSD

4. Troubleshooting Updates

Add troubleshooting section for helper scripts:

Permission issues with scripts
Failed validation steps
Service creation problems
Platform-specific issues

Structure Changes

Before (current)

# Manual step-by-step installation only
1. System Requirements
2. Step-by-step Installation (manual)
3. Configuration
4. Service Integration

After (updated)

# Multiple installation paths
1. System Requirements
2. Installation Methods Overview
   2.1 Quick Installation (./install.sh)
   2.2 Manual Installation (existing steps)
   2.3 Update Installation (--upgrade)
3. Helper Scripts Reference
4. Configuration (unchanged)
5. Service Integration (updated)
6. Update Workflows

Success Criteria

Users can choose between automated and manual installation
Update process is clearly documented
Helper scripts are explained with examples
Troubleshooting covers common script issues
Both Git and Package workflows are documented

Dependencies

Issue #87 helper scripts implementation (completed)
Current Installation.md as baseline

Target Audience

System administrators installing furt
Developers setting up test environments
Users updating existing installations

Issue #87: Helper Scripts (foundation)
Issue #55: Installation Scripts (resolved by #87)
Issue #90: Git-based Deployment (for advanced workflows)

## Purpose Update Installation.md to include the new helper scripts system from Issue #87. ## Current Gap Installation.md currently describes manual step-by-step installation. With the new helper scripts system (install.sh + 6 helper scripts), users have multiple installation paths but the documentation doesn't reflect this. ## Required Documentation Updates ### 1. Installation Methods Section Add overview of available installation methods: - **Automated Installation**: `./install.sh` (recommended) - **Manual Installation**: Step-by-step (current documentation) - **Update Installation**: `./install.sh --upgrade` ### 2. Helper Scripts Documentation Document each helper script with purpose and usage: - `install.sh` - Main orchestrator with --upgrade support - `scripts/setup-user.sh` - System user creation - `scripts/setup-directories.sh` - Directory structure - `scripts/sync-files.sh` - Source file installation - `scripts/create-service.sh` - Service creation from templates - `scripts/validate-config.sh` - Configuration validation - `scripts/health-check.sh` - Health check functionality ### 3. Update Workflow Documentation Add section for updating existing installations: ```bash # Git-based updates cd /path/to/furt-source git pull ./install.sh --upgrade sudo systemctl restart furt # Linux doas rcctl restart furt # OpenBSD ``` ### 4. Troubleshooting Updates Add troubleshooting section for helper scripts: - Permission issues with scripts - Failed validation steps - Service creation problems - Platform-specific issues ## Structure Changes ### Before (current) ``` # Manual step-by-step installation only 1. System Requirements 2. Step-by-step Installation (manual) 3. Configuration 4. Service Integration ``` ### After (updated) ``` # Multiple installation paths 1. System Requirements 2. Installation Methods Overview 2.1 Quick Installation (./install.sh) 2.2 Manual Installation (existing steps) 2.3 Update Installation (--upgrade) 3. Helper Scripts Reference 4. Configuration (unchanged) 5. Service Integration (updated) 6. Update Workflows ``` ## Success Criteria - [ ] Users can choose between automated and manual installation - [ ] Update process is clearly documented - [ ] Helper scripts are explained with examples - [ ] Troubleshooting covers common script issues - [ ] Both Git and Package workflows are documented ## Dependencies - Issue #87 helper scripts implementation (completed) - Current Installation.md as baseline ## Target Audience - System administrators installing furt - Developers setting up test environments - Users updating existing installations ## Related Issues - Issue #87: Helper Scripts (foundation) - Issue #55: Installation Scripts (resolved by #87) - Issue #90: Git-based Deployment (for advanced workflows)

michael added the

labels 2025-09-03 22:36:33 +02:00

michael added this to the v0.1.2 - Gateway Basics milestone 2025-09-03 22:36:56 +02:00

michael commented

2025-09-04 20:07:04 +02:00

Author

Owner

Update: Successful walter Installation Test + Documentation Restructure Plan

walter Test Results - Helper Scripts Work Perfectly

Test Environment: walter (OpenBSD VM)
Method: Package installation with ./install.sh
Result: ✅ Complete Success

Installation Flow:

# Download & extract package
curl -OJ https://smida.dragons-at-work.de/api/packages/DAW/generic/furt-api-gateway/0.1.1/furt-api-gateway-v0.1.1.tar.gz
tar -xzf furt-api-gateway-v0.1.1.tar.gz
cd furt-api-gateway-v0.1.1

# Automated installation with helper scripts
doas ./install.sh
# Result: furt(ok) - service running immediately

All 6 Helper Scripts executed successfully:

✅ setup-user.sh - Created _furt user
✅ setup-directories.sh - Created /usr/local/etc/furt, /usr/local/share/furt, /var/log/furt
✅ sync-files.sh - Copied source files correctly
✅ create-service.sh - Created /etc/rc.d/furt from template
✅ validate-config.sh - Config validation (expected warning for missing config)
✅ health-check.sh - Service health verification

API Testing successful:

curl http://127.0.0.1:7811/health
# {"status":"healthy","version":"0.1.1+25a29c32",...}

curl -H "X-API-Key: monitoring-health-key" http://127.0.0.1:7811/v1/auth/status
# {"authenticated":true,"permissions":["health:check"],...}

Documentation Restructure Plan

Current Problem: Installation.md has grown to 600+ lines - violates DAW minimalist principles.

DAW Philosophy for Documentation:

UNDERSTAND WHY - Explain reasoning behind each step
MANUAL STEPS FIRST - Detailed manual process for understanding
AUTOMATION AS SHORTCUT - Helper scripts after understanding is gained

Proposed Structure:

1. Quick-Install.md (NEW) - 50-100 lines max

# furt Quick Installation

## Prerequisites Check
[Why these dependencies matter]

## Automated Installation (Recommended)
```bash
# Download package
curl -OJ [package-url]
tar -xzf [package]
cd [package-dir]

# Install with helper scripts
doas ./install.sh

Manual Installation

[Link to Manual-Install.md for full understanding]

Next Steps

[Config & Testing - minimal essential only]


#### 2. Manual-Install.md (Existing content refactored)

Manual Installation Guide

Understanding the Installation Process

[Why each step is necessary - educational focus]

Step-by-Step Manual Process

[Current detailed content, but structured for learning]

What the Helper Scripts Do

[Mapping manual steps to automated scripts]


#### 3. Configuration.md (Extracted from Installation.md)

furt Configuration Guide

Understanding Multi-Tenant Architecture

[Why API keys, SMTP routing, etc.]

Configuration Examples

[Real-world scenarios]

Security Considerations

[File permissions, network access, etc.]


#### 4. Helper-Scripts.md (NEW)

Helper Scripts Reference

Script Overview

[What each script does and why]

Manual Equivalent

[For each script, show the manual commands it replaces]

Troubleshooting Scripts

[When scripts fail, how to debug]


#### 5. Update Home.md
Remove lengthy feature lists, focus on:
- Minimal description
- Quick start (link to Quick-Install.md)
- Navigation to detailed docs

### Implementation Strategy

**Phase 1: Extract Configuration**
- Move config examples from Installation.md to Configuration.md
- Update Installation.md to reference Configuration.md

**Phase 2: Create Quick-Install**
- Extract automated installation flow to Quick-Install.md
- Focus on getting furt running in <10 minutes

**Phase 3: Refactor Manual Process**
- Keep educational manual process in Manual-Install.md
- Emphasize understanding over speed

**Phase 4: Helper Scripts Documentation**
- Document what each script replaces
- Show manual equivalent for transparency

**Phase 5: Streamline Home**
- Remove redundant information
- Clear navigation structure

### Success Criteria Updated

**Original Issue #102:**
- [x] Helper scripts system documented
- [x] Update workflow documented
- [x] Installation methods overview

**Extended Scope:**
- [ ] Documentation split into focused, minimal files
- [ ] "Understanding first, automation second" approach implemented
- [ ] Each document under 200 lines (exception: Manual-Install.md)
- [ ] Clear navigation between related docs
- [ ] Helper scripts mapped to manual equivalents

### Dependencies
- walter test proves helper scripts work (✅ Complete)
- Current Installation.md as content source
- Need to preserve all existing information while reorganizing

This restructure aligns with DAW principles: provide understanding first, then efficient tools for those who understand the process.

## Update: Successful walter Installation Test + Documentation Restructure Plan ### walter Test Results - Helper Scripts Work Perfectly **Test Environment:** walter (OpenBSD VM) **Method:** Package installation with `./install.sh` **Result:** ✅ Complete Success **Installation Flow:** ```bash # Download & extract package curl -OJ https://smida.dragons-at-work.de/api/packages/DAW/generic/furt-api-gateway/0.1.1/furt-api-gateway-v0.1.1.tar.gz tar -xzf furt-api-gateway-v0.1.1.tar.gz cd furt-api-gateway-v0.1.1 # Automated installation with helper scripts doas ./install.sh # Result: furt(ok) - service running immediately ``` **All 6 Helper Scripts executed successfully:** - ✅ `setup-user.sh` - Created `_furt` user - ✅ `setup-directories.sh` - Created `/usr/local/etc/furt`, `/usr/local/share/furt`, `/var/log/furt` - ✅ `sync-files.sh` - Copied source files correctly - ✅ `create-service.sh` - Created `/etc/rc.d/furt` from template - ✅ `validate-config.sh` - Config validation (expected warning for missing config) - ✅ `health-check.sh` - Service health verification **API Testing successful:** ```bash curl http://127.0.0.1:7811/health # {"status":"healthy","version":"0.1.1+25a29c32",...} curl -H "X-API-Key: monitoring-health-key" http://127.0.0.1:7811/v1/auth/status # {"authenticated":true,"permissions":["health:check"],...} ``` ### Documentation Restructure Plan **Current Problem:** Installation.md has grown to 600+ lines - violates DAW minimalist principles. **DAW Philosophy for Documentation:** 1. **UNDERSTAND WHY** - Explain reasoning behind each step 2. **MANUAL STEPS FIRST** - Detailed manual process for understanding 3. **AUTOMATION AS SHORTCUT** - Helper scripts after understanding is gained **Proposed Structure:** #### 1. Quick-Install.md (NEW) - 50-100 lines max ``` # furt Quick Installation ## Prerequisites Check [Why these dependencies matter] ## Automated Installation (Recommended) ```bash # Download package curl -OJ [package-url] tar -xzf [package] cd [package-dir] # Install with helper scripts doas ./install.sh ``` ## Manual Installation [Link to Manual-Install.md for full understanding] ## Next Steps [Config & Testing - minimal essential only] ``` #### 2. Manual-Install.md (Existing content refactored) ``` # Manual Installation Guide ## Understanding the Installation Process [Why each step is necessary - educational focus] ## Step-by-Step Manual Process [Current detailed content, but structured for learning] ## What the Helper Scripts Do [Mapping manual steps to automated scripts] ``` #### 3. Configuration.md (Extracted from Installation.md) ``` # furt Configuration Guide ## Understanding Multi-Tenant Architecture [Why API keys, SMTP routing, etc.] ## Configuration Examples [Real-world scenarios] ## Security Considerations [File permissions, network access, etc.] ``` #### 4. Helper-Scripts.md (NEW) ``` # Helper Scripts Reference ## Script Overview [What each script does and why] ## Manual Equivalent [For each script, show the manual commands it replaces] ## Troubleshooting Scripts [When scripts fail, how to debug] ``` #### 5. Update Home.md Remove lengthy feature lists, focus on: - Minimal description - Quick start (link to Quick-Install.md) - Navigation to detailed docs ### Implementation Strategy **Phase 1: Extract Configuration** - Move config examples from Installation.md to Configuration.md - Update Installation.md to reference Configuration.md **Phase 2: Create Quick-Install** - Extract automated installation flow to Quick-Install.md - Focus on getting furt running in <10 minutes **Phase 3: Refactor Manual Process** - Keep educational manual process in Manual-Install.md - Emphasize understanding over speed **Phase 4: Helper Scripts Documentation** - Document what each script replaces - Show manual equivalent for transparency **Phase 5: Streamline Home** - Remove redundant information - Clear navigation structure ### Success Criteria Updated **Original Issue #102:** - [x] Helper scripts system documented - [x] Update workflow documented - [x] Installation methods overview **Extended Scope:** - [ ] Documentation split into focused, minimal files - [ ] "Understanding first, automation second" approach implemented - [ ] Each document under 200 lines (exception: Manual-Install.md) - [ ] Clear navigation between related docs - [ ] Helper scripts mapped to manual equivalents ### Dependencies - walter test proves helper scripts work (✅ Complete) - Current Installation.md as content source - Need to preserve all existing information while reorganizing This restructure aligns with DAW principles: provide understanding first, then efficient tools for those who understand the process.

michael commented

2025-09-05 15:37:00 +02:00

Author

Owner

Multi-Platform Testing Update (3/4 Platforms Complete)

Successful Platform Tests

OpenBSD (walter): Helper Scripts + rc.d functional
Debian (johann): Helper Scripts + systemd functional after fixes
Arch Linux (klaus): Helper Scripts + systemd functional after fixes

Critical Issues Identified

1. Config Path Inconsistency

Problem: Helper Scripts create /etc/furt/, start.sh searches /usr/local/etc/furt/
Impact: All Linux distributions affected
Workaround: Manual cp config/furt.conf.example /usr/local/etc/furt/furt.conf

2. systemd Service Type Issue

Problem: Repository template uses Type=simple but start.sh runs with & (forking)
Impact: Linux distributions (Debian, Arch) - service fails
Fix: Type=forking in systemd template

3. JSON Library Platform Differences

Problem:

OpenBSD/Debian: lua-cjson available
Arch Linux: Only lua51-dkjson available
furt hardcodes require('cjson')
Fix Applied: Manual sed replacement for Arch testing
Needed: Flexible JSON library detection in code

4. Missing SSL/TLS Dependency Check

Problem: start.sh checks lua-socket and lua-cjson but not lua-sec
Impact: Service starts but fails silently on SMTP SSL connections

Documentation Restructure Requirements

Based on testing, Installation.md needs platform-specific sections:

Quick-Install.md - Automated installation (updated helper scripts)
Manual-Install.md - Step-by-step with platform differences
Platform-Notes.md - JSON libraries, systemd vs rc.d, config paths
Troubleshooting.md - Common multi-platform issues

Remaining Testing

FreeBSD (franz): Expected similar to OpenBSD with pkg vs pkg_add differences

Implementation Priority

Fix config path inconsistency (affects all Linux)
Update systemd template (Type=forking)
Add flexible JSON library support
Add SSL dependency check
Split Installation.md per findings

## Multi-Platform Testing Update (3/4 Platforms Complete) ### Successful Platform Tests - **OpenBSD (walter):** Helper Scripts + rc.d functional - **Debian (johann):** Helper Scripts + systemd functional after fixes - **Arch Linux (klaus):** Helper Scripts + systemd functional after fixes ### Critical Issues Identified #### 1. Config Path Inconsistency **Problem:** Helper Scripts create `/etc/furt/`, start.sh searches `/usr/local/etc/furt/` **Impact:** All Linux distributions affected **Workaround:** Manual `cp config/furt.conf.example /usr/local/etc/furt/furt.conf` #### 2. systemd Service Type Issue **Problem:** Repository template uses `Type=simple` but start.sh runs with `&` (forking) **Impact:** Linux distributions (Debian, Arch) - service fails **Fix:** `Type=forking` in systemd template #### 3. JSON Library Platform Differences **Problem:** - OpenBSD/Debian: `lua-cjson` available - Arch Linux: Only `lua51-dkjson` available - furt hardcodes `require('cjson')` **Fix Applied:** Manual sed replacement for Arch testing **Needed:** Flexible JSON library detection in code #### 4. Missing SSL/TLS Dependency Check **Problem:** start.sh checks lua-socket and lua-cjson but not lua-sec **Impact:** Service starts but fails silently on SMTP SSL connections ### Documentation Restructure Requirements Based on testing, Installation.md needs platform-specific sections: **Quick-Install.md** - Automated installation (updated helper scripts) **Manual-Install.md** - Step-by-step with platform differences **Platform-Notes.md** - JSON libraries, systemd vs rc.d, config paths **Troubleshooting.md** - Common multi-platform issues ### Remaining Testing - **FreeBSD (franz):** Expected similar to OpenBSD with pkg vs pkg_add differences ### Implementation Priority 1. Fix config path inconsistency (affects all Linux) 2. Update systemd template (Type=forking) 3. Add flexible JSON library support 4. Add SSL dependency check 5. Split Installation.md per findings

michael commented

2025-09-05 16:13:07 +02:00

Author

Owner

Scope-Korrektur: FreeBSD entfernt aus Multi-Platform-Testing

Grund für Scope-Änderung

Nach Session-Analyse: FreeBSD ist nicht Teil der DAW-Plattform-Strategie.

DAW-Blog unterstützt nur 3 Plattformen:

✅ OpenBSD - Primary (DAW-Standard)
✅ Debian - Linux-Stabilität
✅ Arch - Linux-Rolling-Release

FreeBSD-Probleme:

Nicht in DAW-Dokumentations-Strategie
Corporate Foundation (weniger authentisch als OpenBSD)
Scope-Creep von 3→4 Plattformen ohne strategische Begründung

Multi-Platform-Testing-Ergebnis (Final)

walter (OpenBSD): ✅ Helper Scripts funktional
johann (Debian): ✅ Funktional nach systemd Type=forking Fix
klaus (Arch): ✅ Funktional nach dkjson-Kompatibilität Fix
franz (FreeBSD): ❌ Entfernt aus Scope - nicht DAW-Plattform

Installation.md-Restructure (Updated Scope)

Quick-Install.md + Platform-Notes.md: Nur OpenBSD/Debian/Arch
Manual-Install.md: OpenBSD-focused mit Linux-Hinweisen

3-Platform-Support bleibt DAW-konform ohne Corporate-Multi-Platform-Versprechen.

## Scope-Korrektur: FreeBSD entfernt aus Multi-Platform-Testing ### Grund für Scope-Änderung Nach Session-Analyse: FreeBSD ist **nicht** Teil der DAW-Plattform-Strategie. **DAW-Blog unterstützt nur 3 Plattformen:** - ✅ **OpenBSD** - Primary (DAW-Standard) - ✅ **Debian** - Linux-Stabilität - ✅ **Arch** - Linux-Rolling-Release **FreeBSD-Probleme:** - Nicht in DAW-Dokumentations-Strategie - Corporate Foundation (weniger authentisch als OpenBSD) - Scope-Creep von 3→4 Plattformen ohne strategische Begründung ### Multi-Platform-Testing-Ergebnis (Final) - **walter (OpenBSD):** ✅ Helper Scripts funktional - **johann (Debian):** ✅ Funktional nach systemd Type=forking Fix - **klaus (Arch):** ✅ Funktional nach dkjson-Kompatibilität Fix - **franz (FreeBSD):** ❌ **Entfernt aus Scope** - nicht DAW-Plattform ### Installation.md-Restructure (Updated Scope) **Quick-Install.md + Platform-Notes.md:** Nur OpenBSD/Debian/Arch **Manual-Install.md:** OpenBSD-focused mit Linux-Hinweisen **3-Platform-Support** bleibt DAW-konform ohne Corporate-Multi-Platform-Versprechen.

michael commented

2025-09-05 22:28:31 +02:00

Author

Owner

Additional Service-Management Features Impact

Update zu Issue #100 Scope-Erweiterung:

Zusätzliche Documentation-Requirements

Service-Management-Features (Issue #100)

PID-File Management - /var/run/furt.pid Handling
Graceful Shutdown - rcctl stop mit Timeout-Verhalten
Config-Reload - doas rcctl reload furt / sudo systemctl reload furt
Process-Validation - Startup-Verification

Updated Documentation Structure

Quick-Install.md: Service-Commands erweitern

# Service-Management
doas rcctl start furt      # OpenBSD
doas rcctl reload furt     # Config-Reload ohne Restart
sudo systemctl start furt  # Linux
sudo systemctl reload furt # Config-Reload

Troubleshooting.md: PID-File spezifische Issues

Stale PID-Files nach Crashes
Permission-Probleme mit /var/run/
Service-Status-Verification

Platform-Notes.md: rc.d vs systemd Service-Command-Unterschiede
Testing-Coordination: Issue #100 Implementation bereit - Documentation kann parallel aktualisiert werden

## Additional Service-Management Features Impact **Update zu Issue #100 Scope-Erweiterung:** ### Zusätzliche Documentation-Requirements #### Service-Management-Features (Issue #100) - **PID-File Management** - /var/run/furt.pid Handling - **Graceful Shutdown** - rcctl stop mit Timeout-Verhalten - **Config-Reload** - `doas rcctl reload furt` / `sudo systemctl reload furt` - **Process-Validation** - Startup-Verification #### Updated Documentation Structure **Quick-Install.md:** Service-Commands erweitern ```bash # Service-Management doas rcctl start furt # OpenBSD doas rcctl reload furt # Config-Reload ohne Restart sudo systemctl start furt # Linux sudo systemctl reload furt # Config-Reload ``` Troubleshooting.md: PID-File spezifische Issues Stale PID-Files nach Crashes Permission-Probleme mit /var/run/ Service-Status-Verification Platform-Notes.md: rc.d vs systemd Service-Command-Unterschiede Testing-Coordination: Issue #100 Implementation bereit - Documentation kann parallel aktualisiert werden

michael referenced this issue

2025-09-05 22:30:10 +02:00

systemd Service Hardening: Security-Optionen für Production #110

michael commented

2025-09-10 12:25:19 +02:00

Author

Owner

Dokumentation im wiki erstellt

michael

2025-09-10 12:25:19 +02:00