diff --git a/.clinerules/satware-ai-dev.md b/.clinerules/satware-ai-dev.md new file mode 100644 index 0000000..f0102a2 --- /dev/null +++ b/.clinerules/satware-ai-dev.md @@ -0,0 +1,301 @@ +# satware.ai Development Rules (MANDATORY) + +**Project:** satware.ai - Public Documentation Website +**Repository:** https://github.com/satwareAG/satware.ai +**Enforcement:** ABSOLUTE - No exceptions + +--- + +## πŸ”’ Rule #1: Internal Documentation Placement (CRITICAL) + +### MANDATORY Policy + +**ALL internal documentation MUST be placed in `docs/internal/`** + +This is **NON-NEGOTIABLE** and **ABSOLUTE**. The `docs/internal/` directory is excluded from publication via `mkdocs.yml` configuration. + +### ⚠️ CRITICAL: Correct Glob Pattern (TESTED & VERIFIED) + +**The mkdocs-exclude plugin requires specific glob syntax:** + +```yaml +plugins: + - exclude: + glob: + - "internal/*" # βœ… WORKS - Excludes docs/internal/ + - "templates/*" # βœ… WORKS - Excludes docs/templates/ +``` + +**❌ WRONG - Does NOT work:** +```yaml +plugins: + - exclude: + glob: + - "internal/**/*" # ❌ FAILS - Does not exclude! + - "templates/**/*" # ❌ FAILS - Does not exclude! +``` + +**Why:** mkdocs-exclude patterns are relative to `docs/` directory. The `**/*` subdirectory wildcard does NOT work for top-level exclusion. Use `directory/*` pattern only. + +**Verification (2025-11-09):** +- βœ… Pattern `"internal/*"` - site/internal/ does NOT exist (SUCCESS) +- ❌ Pattern `"internal/**/*"` - site/internal/ DOES exist (FAILURE) +- πŸ“Š Performance: Build time improved 17.51s β†’ 7.07s (59% faster) + +### What MUST Go in `docs/internal/` + +βœ… **ALWAYS place these in `docs/internal/`:** +- Development analysis documents +- CI/CD configuration notes +- Architecture decision records (internal) +- Testing strategies and results +- Migration guides (internal processes) +- Performance analysis and benchmarks +- Security audit notes (internal) +- Team process documentation +- Technical debt tracking +- Research and investigation notes +- Planning documents for future features +- Any documentation NOT intended for public consumption + +### What Goes in `docs/` (Public) + +βœ… **Public documentation only:** +- Team member profiles (`docs/team/*.md`) +- Public blog posts (`docs/blog/posts/*.md`) +- FAQ pages (`docs/faq/index.md`) +- Feature documentation for users +- Webinar/workshop information + +### Verification Checklist (MANDATORY Before Committing) + +Before committing ANY new `.md` file in `docs/`: + +1. **Is this intended for public consumption on satware.ai?** + - βœ… YES β†’ Can go in `docs/` + - ❌ NO β†’ **MUST** go in `docs/internal/` + +2. **Does this contain internal notes, analysis, or development information?** + - βœ… YES β†’ **MUST** go in `docs/internal/` + - ❌ NO β†’ Can go in `docs/` if public-facing + +3. **Would I be comfortable with this being on the live website?** + - βœ… YES β†’ Can go in `docs/` + - ❌ NO β†’ **MUST** go in `docs/internal/` + +**When in doubt:** PUT IT IN `docs/internal/` + +### Testing Requirements (MANDATORY - VERIFIED PROCEDURES) + +**Before pushing commits with new documentation:** + +```bash +# 1. Start local development server +./mkdocs.sh + +# 2. Build site explicitly (dev server doesn't generate site/) +docker exec nostalgic_bohr mkdocs build --clean + +# 3. CRITICAL TEST: Verify site/internal/ does NOT exist +docker exec nostalgic_bohr ls -la /docs/site/internal/ 2>&1 +# Expected output: "ls: cannot access '/docs/site/internal/': No such file or directory" βœ… + +# 4. Verify no internal files in built site +docker exec nostalgic_bohr find /docs/site/ -name "*internal*" 2>&1 +# Expected output: (empty - no results) βœ… + +# 5. List site structure (should NOT include internal/) +docker exec nostalgic_bohr ls -1 /docs/site/ | grep internal +# Expected output: (empty - no match) βœ… +``` + +**Success Criteria:** +- βœ… `site/internal/` directory does NOT exist +- βœ… No files with "internal" in name found in site/ +- βœ… Build completes without internal document warnings +- βœ… Build time ~7s (not ~17s indicating excluded files) + +--- + +## πŸ€– Rule #2: AI Assistant Behavior (MANDATORY) + +### Automatic Enforcement + +When working on this project, AI assistants **MUST**: + +1. **Always check file destination** before creating `.md` files in `docs/` +2. **Default to `docs/internal/`** for any development-related documentation +3. **Ask for confirmation** if uncertain about public vs. internal placement +4. **Refuse to create** internal docs outside `docs/internal/` without explicit override +5. **Suggest moving** existing internal docs found outside `docs/internal/` + +### Red Flags (Trigger `docs/internal/` Placement) + +Keywords indicating internal documentation: +- "analysis" / "analyze" / "investigation" +- "CI/CD" / "GitHub Actions" / "workflow" / "pipeline" +- "dev" / "development" / "developer" / "debug" +- "internal" / "private" / "confidential" +- "architecture decision" / "technical debt" +- "testing strategy" / "performance benchmark" +- "migration guide" (internal) / "planning" / "roadmap" (internal) + +--- + +## πŸ“‹ Rule #3: File Naming Conventions + +### Internal Documentation + +**Pattern:** `docs/internal/.md` + +**Examples:** +- `docs/internal/dev-ci-parity-analysis.md` +- `docs/internal/architecture-microservices-evaluation.md` +- `docs/internal/testing-strategy-2025.md` +- `docs/internal/performance-optimization-notes.md` + +**Avoid:** Vague names like `notes.md`, `temp.md`, `draft.md`, `doc1.md` + +### Public Documentation + +**Follow existing patterns:** +- Team profiles: `docs/team/.md` +- Blog posts: `docs/blog/posts/YYYY-MM-DD-.md` +- FAQ: `docs/faq/index.md` +- Features: `docs/anwendungen/index.md` + +--- + +## πŸ” Rule #4: Code Review Checklist + +### For ALL Pull Requests + +Before approving any PR that adds/modifies `.md` files: + +- [ ] All internal docs are in `docs/internal/` +- [ ] No development notes in public `docs/` directories +- [ ] `mkdocs.yml` uses correct glob pattern: `"internal/*"` (not `"internal/**/*"`) +- [ ] Local build tested with verification commands (Rule #1) +- [ ] Verified internal docs not visible in built site +- [ ] No sensitive information in ANY committed files + +--- + +## 🚨 Rule #5: Incident Response + +### If Internal Docs Are Accidentally Published + +**Immediate Actions:** + +1. **Remove from source:** + ```bash + git rm docs/<accidental-file>.md + git commit -m "fix: Remove accidentally published internal documentation" + git push + ``` + +2. **Wait for redeployment** (GitHub Actions will rebuild) + +3. **Verify removal:** + - Check https://satware.ai after deployment + - Use Google Search Console to request re-crawl + - Monitor search results for leaked content + +4. **Document incident** in `docs/internal/incidents/YYYY-MM-DD-doc-leak.md` + +--- + +## πŸ“– Rule #6: Documentation Standards + +### All Documentation + +1. **Markdown:** Proper syntax with frontmatter where appropriate +2. **Headers:** Proper hierarchy (H1 β†’ H6) +3. **Links:** Relative links for internal references +4. **Images:** Place in `docs/assets/images/` subdirectories +5. **Code blocks:** Use syntax highlighting (```language) + +### Internal Documentation Specific + +**Example frontmatter:** + +```yaml +--- +title: "CI/CD Pipeline Analysis (INTERNAL)" +date: 2025-11-09 +author: Development Team +status: Active +confidentiality: Internal Use Only +--- +``` + +--- + +## 🎯 Rule #7: Enforcement Priority + +### Priority Levels + +1. **P0 - CRITICAL:** Never publish internal docs (Rule #1) + Use correct glob pattern +2. **P1 - HIGH:** Always test before committing (Rule #1) +3. **P2 - MEDIUM:** Follow naming conventions (Rule #3) +4. **P3 - LOW:** Documentation standards (Rule #6) + +### Non-Compliance Consequences + +**P0 Violation:** Immediate rollback, incident report required +**P1 Violation:** PR rejected, must fix before merge +**P2 Violation:** Warning, fix in next commit +**P3 Violation:** Suggestion for improvement + +--- + +## πŸ“š Quick Reference + +### Essential Commands + +```bash +# Start local server +./mkdocs.sh + +# Build and test (MANDATORY before commit) +docker exec nostalgic_bohr mkdocs build --clean +docker exec nostalgic_bohr ls -la /docs/site/internal/ 2>&1 # Should error + +# Create new internal document +touch docs/internal/<descriptive-name>.md + +# Check for accidental internal content +grep -r "INTERNAL" docs/ --exclude-dir=internal +``` + +### Key Files + +- `mkdocs.yml` lines 48-52 - Exclusion configuration (use `"internal/*"` pattern) +- `README.md` - Internal documentation policy section +- `.clinerules/satware-ai-dev.md` - This file (enforcement rules) + +### Performance Benchmarks (2025-11-09) + +- Build time WITHOUT exclusion: ~17.51s +- Build time WITH exclusion: ~7.07s +- **Improvement:** 59% faster (10.44s saved per build) + +--- + +## βœ… Summary + +**The ONE rule to remember:** + +> **Internal docs β†’ `docs/internal/`** +> **Public docs β†’ `docs/`** +> **When in doubt β†’ `docs/internal/`** +> +> **Use glob pattern: `"internal/*"` NOT `"internal/**/*"`** + +**This is MANDATORY and ABSOLUTE. No exceptions.** + +--- + +**Last Updated:** 2025-11-09 +**Version:** 1.1 (Updated with verified glob pattern and testing procedures) +**Status:** Active and Enforced diff --git a/.gitignore b/.gitignore index df2cc75..524c6df 100644 --- a/.gitignore +++ b/.gitignore @@ -1,8 +1,15 @@ .idea .cache /site +/_site /docs/assets/css/custom.css /package-lock.json # Add readme.md to ensure case sensitivity is enforced # This commit includes an attempt to delete readme.md /docs/assets/css/custom.css.map + +# Ruby/Bundler (orphaned files - this is a Python/MkDocs project) +Gemfile.lock +Gemfile +.bundle/ +vendor/bundle/ diff --git a/README.md b/README.md index 6f0461a..d26777c 100644 --- a/README.md +++ b/README.md @@ -43,6 +43,8 @@ Developing locally is streamlined using Docker, which encapsulates all necessary ``` . +β”œβ”€β”€ .clinerules/ # Project-specific AI assistant rules (MANDATORY) +β”‚ └── satware-ai-dev.md # Development rules for this project β”œβ”€β”€ .github/ # GitHub Actions workflows for CI/CD β”‚ └── workflows/ # Contains deploy-live.yml and deploy-preview.yml β”œβ”€β”€ .gitignore # Specifies intentionally untracked files to ignore @@ -50,6 +52,7 @@ Developing locally is streamlined using Docker, which encapsulates all necessary β”œβ”€β”€ docker/ # Docker-related files β”‚ └── mkdocs-material/ # Dockerfile for the MkDocs Material environment β”œβ”€β”€ docs/ # Markdown source files for the documentation +β”‚ β”œβ”€β”€ internal/ # ⚠️ INTERNAL DOCS - NOT PUBLISHED (excluded via mkdocs.yml) β”‚ β”œβ”€β”€ index.md # Homepage content β”‚ └── ... # Other documentation pages and assets β”œβ”€β”€ mkdocs.sh # Script for local development with Docker @@ -63,6 +66,36 @@ Developing locally is streamlined using Docker, which encapsulates all necessary └── README.md # This file ``` +### πŸ“ Internal Documentation Policy + +**⚠️ CRITICAL: All internal documentation MUST be placed in `docs/internal/`** + +The `docs/internal/` directory is **automatically excluded** from the published website via the `mkdocs-exclude` plugin configured in `mkdocs.yml`. This ensures that internal development notes, analysis documents, and sensitive information are **never published** to satware.ai. + +**Rules:** +- βœ… **DO**: Place all internal docs in `docs/internal/` +- βœ… **DO**: Use descriptive filenames (e.g., `dev-ci-parity-analysis.md`, `architecture-notes.md`) +- ❌ **DON'T**: Place internal docs anywhere else in `docs/` +- ❌ **DON'T**: Commit sensitive information even to `docs/internal/` + +**Verification:** +```bash +# Test locally before pushing +./mkdocs.sh +# Navigate to http://localhost:8000 +# Verify internal docs are NOT listed in navigation or search +``` + +**What belongs in `docs/internal/`:** +- Development analysis and planning documents +- Architecture decision records (internal) +- CI/CD configuration notes +- Testing strategies and results +- Migration guides (internal) +- Any documentation NOT intended for public consumption + +**Enforcement:** The `.clinerules/satware-ai-dev.md` file contains **MANDATORY rules** for AI assistants working on this project to enforce this policy automatically. + ## πŸ› οΈ Technologies Used * **Static Site Generator**: MkDocs @@ -121,4 +154,4 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file ## πŸ“§ Contact -For any inquiries or support, please contact [info@satware.ai](mailto:info@satware.ai). \ No newline at end of file +For any inquiries or support, please contact [info@satware.ai](mailto:info@satware.ai). diff --git a/docs/internal/dev-ci-parity-analysis.md b/docs/internal/dev-ci-parity-analysis.md new file mode 100644 index 0000000..e5942b3 --- /dev/null +++ b/docs/internal/dev-ci-parity-analysis.md @@ -0,0 +1,349 @@ +# Development-CI Environment Parity Analysis + +**Date:** 2025-11-09 +**Repository:** satware.ai +**Analyzed by:** Jane Alesi + +## Executive Summary + +The local Docker development environment (`mkdocs.sh` + `docker/mkdocs-material/Dockerfile`) and GitHub Actions CI/CD workflows (`.github/workflows/deploy-live.yml` and `deploy-preview.yml`) are **functionally identical** but suffer from **critical maintainability issues** due to **dependency duplication** and lack of a **single source of truth**. + +### Critical Finding +Both environments use **identical Python packages** (15 total, including `mkdocs-material==9.6.14`), but dependencies are **hardcoded in 3 separate locations**: +1. `docker/mkdocs-material/Dockerfile` +2. `.github/workflows/deploy-live.yml` +3. `.github/workflows/deploy-preview.yml` + +This creates **high risk of drift** when updating dependencies. + +--- + +## Detailed Comparison Matrix + +| Aspect | Local Development | GitHub Actions CI/CD | Match? | Risk Level | +|--------|-------------------|----------------------|--------|------------| +| **Base OS** | Debian Bullseye (via python:3.13-slim-bullseye) | Ubuntu 22.04 | ⚠️ Different | Low | +| **Python Version** | 3.13 | 3.13 | βœ… Identical | None | +| **Python Packages** | 15 packages (see below) | 15 packages (see below) | βœ… Identical | None | +| **mkdocs-material Version** | 9.6.14 (pinned) | 9.6.14 (pinned) | βœ… Identical | None | +| **SCSS Compilation** | pysassc (libsass) | pysassc (libsass) | βœ… Identical | None | +| **System Dependencies** | build-essential, cairo, gdk-pixbuf, pango, etc. | Not explicitly installed (bundled in ubuntu-22.04) | ⚠️ Implicit | Low | +| **Build Process** | mkdocs serve (development server) | mkdocs build β†’ gh-deploy | ⚠️ Different purpose | None | +| **Dependency Management** | Hardcoded in Dockerfile | Hardcoded in workflow YAML | ❌ **DUPLICATED** | **HIGH** | +| **Version Locking** | Inline pip install | Inline pip install | ❌ **NO requirements.txt** | **HIGH** | + +### Python Packages (Identical in Both Environments) +``` +cairosvg +libsass +mkdocs-exclude +mkdocs-git-revision-date-localized-plugin +mkdocs-glightbox +mkdocs-include-markdown-plugin +mkdocs-literate-nav +mkdocs-macros-plugin +mkdocs-material==9.6.14 +mkdocs-material[imaging] +mkdocs-minify-plugin +mkdocs-redirects +mkdocs-rss-plugin +mkdocs-snippets +mkdocs-video +watchdog +``` + +--- + +## Critical Issues Identified + +### πŸ”΄ Issue 1: Dependency Duplication (HIGH RISK) +**Problem:** Dependencies listed in 3 separate files means updating package versions requires changing 3 locations. Missing one creates immediate drift. + +**Current State:** +- `docker/mkdocs-material/Dockerfile` (lines 20-36) +- `.github/workflows/deploy-live.yml` (lines 25-39) +- `.github/workflows/deploy-preview.yml` (lines 25-39) + +**Risk:** Developer updates Dockerfile but forgets workflow β†’ local works, CI fails β†’ "works on my machine" syndrome. + +### πŸ”΄ Issue 2: No Requirements File (HIGH RISK) +**Problem:** Industry best practice is `requirements.txt` with pinned versions for reproducibility, audit trail, and dependency locking. + +**Current Risk:** +- No version control history of dependency changes +- No easy rollback if package update breaks build +- Manual coordination required for updates + +### 🟑 Issue 3: Base OS Mismatch (LOW RISK) +**Problem:** Local uses Debian Bullseye, CI uses Ubuntu 22.04. Both work but creates slight differences. + +**Current Impact:** Minimal - Python packages abstract most OS differences. System dependencies (cairo, pango) are handled differently but both work. + +### 🟒 Issue 4: Build Process Difference (NO RISK) +**Status:** Expected and correct behavior: +- Local: `mkdocs serve` for live development with hot reload +- CI: `mkdocs build` + `mkdocs gh-deploy` for static site generation + +This is **intentional and appropriate**. + +--- + +## Best Practices Research Findings + +Based on research of GitHub Actions + Docker best practices (2024-2025): + +### 1. **Single Source of Truth** +βœ… **Recommendation:** Create `requirements.txt` with all Python dependencies +- Eliminates duplication +- Enables version locking with `pip freeze` +- Provides audit trail in git history +- Industry standard for Python projects + +### 2. **Dockerfile Reuse in CI** +βœ… **Recommendation:** GitHub Actions should build and use the same Dockerfile +- Guarantees environment parity +- Reduces configuration drift +- "Use same Dockerfile everywhere" principle + +### 3. **Multi-Stage Builds** +βœ… **Recommendation:** Separate build and runtime stages +- Already partially implemented (dependencies vs runtime) +- Could optimize further for production + +### 4. **Cache Optimization** +βœ… **Recommendation:** Layer caching for faster builds +- Docker layer caching in GitHub Actions +- Cache Python packages between runs + +--- + +## Proposed Solutions + +### 🎯 Option A: Requirements.txt with Dockerfile Reuse (RECOMMENDED) + +**Implementation Steps:** + +1. **Create `requirements.txt`** + ```bash + # Extract from current Dockerfile + cairosvg + libsass + mkdocs-exclude + mkdocs-git-revision-date-localized-plugin + mkdocs-glightbox + mkdocs-include-markdown-plugin + mkdocs-literate-nav + mkdocs-macros-plugin + mkdocs-material==9.6.14 + mkdocs-material[imaging] + mkdocs-minify-plugin + mkdocs-redirects + mkdocs-rss-plugin + mkdocs-snippets + mkdocs-video + watchdog + ``` + +2. **Update Dockerfile** + ```dockerfile + FROM python:3.13-slim-bullseye + WORKDIR /docs + + # Install system dependencies + RUN apt-get update && apt-get install -y --no-install-recommends \ + build-essential curl git \ + libcairo2 libffi-dev libgdk-pixbuf2.0-0 \ + libpango-1.0-0 libpangocairo-1.0-0 shared-mime-info \ + && rm -rf /var/lib/apt/lists/* + + # Copy and install Python dependencies + COPY requirements.txt . + RUN pip install --no-cache-dir -r requirements.txt + ``` + +3. **Update GitHub Actions Workflows** + ```yaml + - name: Setup Python + uses: actions/setup-python@v5 + with: + python-version: '3.13' + + - name: Install dependencies + run: | + pip install --upgrade pip + pip install -r requirements.txt + ``` + +4. **Update `mkdocs.sh`** + ```bash + # Build Docker image from Dockerfile + docker build -t squidfunk/mkdocs-material-custom ${PWD}/docker/mkdocs-material + + # Run container (unchanged) + docker run --rm -it --user $(id -u):$(id -g) -p 8000:8000 \ + -v ${PWD}:/docs --entrypoint sh \ + squidfunk/mkdocs-material-custom -c "..." + ``` + +**Benefits:** +- βœ… Single source of truth (`requirements.txt`) +- βœ… Guaranteed parity (same file used everywhere) +- βœ… Easy dependency updates (one file) +- βœ… Git history of dependency changes +- βœ… Industry standard approach + +**Trade-offs:** +- Requires updating 3 files initially (one-time cost) +- Dockerfile must `COPY requirements.txt` (adds build context dependency) + +--- + +### 🎯 Option B: Docker-First CI/CD (ALTERNATIVE) + +**Make GitHub Actions use the Docker container:** + +```yaml +jobs: + deploy: + runs-on: ubuntu-22.04 + steps: + - uses: actions/checkout@v4 + + - name: Build Docker image + run: docker build -t mkdocs-builder ./docker/mkdocs-material + + - name: Build SCSS + run: | + docker run --rm -v ${PWD}:/docs mkdocs-builder \ + pysassc overrides/assets/css/custom.scss docs/assets/css/custom.css + + - name: Build site + run: docker run --rm -v ${PWD}:/docs mkdocs-builder mkdocs build + + - name: Deploy + run: docker run --rm -v ${PWD}:/docs mkdocs-builder mkdocs gh-deploy --force +``` + +**Benefits:** +- βœ… 100% identical environments (same container) +- βœ… True "build once, run anywhere" + +**Trade-offs:** +- ❌ Slower CI builds (Docker build overhead) +- ❌ More complex workflow +- ❌ Still needs requirements.txt for maintainability + +--- + +### 🎯 Option C: Hybrid Approach (BALANCED) + +**Combine best of both:** + +1. Create `requirements.txt` (single source of truth) +2. Keep lightweight GitHub Actions (faster CI) +3. Docker image references `requirements.txt` +4. Both use same dependency file + +**This is Option A** - recommended approach. + +--- + +## Implementation Roadmap + +### Phase 1: Create Single Source of Truth (IMMEDIATE) +- [ ] Create `requirements.txt` in repository root +- [ ] Copy exact package list from Dockerfile +- [ ] Commit with message: `build: create requirements.txt for dependency management` + +### Phase 2: Update Dockerfile (IMMEDIATE) +- [ ] Modify `docker/mkdocs-material/Dockerfile` to use `requirements.txt` +- [ ] Test local build: `docker build -t test ./docker/mkdocs-material` +- [ ] Test local server: `./mkdocs.sh` +- [ ] Commit with message: `build: update Dockerfile to use requirements.txt` + +### Phase 3: Update GitHub Actions (IMMEDIATE) +- [ ] Update `.github/workflows/deploy-live.yml` to use `requirements.txt` +- [ ] Update `.github/workflows/deploy-preview.yml` to use `requirements.txt` +- [ ] Test on feature branch before merging +- [ ] Commit with message: `ci: use requirements.txt for Python dependencies` + +### Phase 4: Validation (BEFORE MERGE) +- [ ] Local development server works +- [ ] GitHub Actions preview deployment succeeds +- [ ] No regressions in site build + +### Phase 5: Documentation (POST-MERGE) +- [ ] Update README.md with dependency update process +- [ ] Document: "To update dependencies, edit requirements.txt only" +- [ ] Add comments in Dockerfile and workflows pointing to requirements.txt + +--- + +## Dependency Update Process (After Implementation) + +**Current (BROKEN):** +```bash +# Must update 3 files manually - high error risk +1. Edit docker/mkdocs-material/Dockerfile +2. Edit .github/workflows/deploy-live.yml +3. Edit .github/workflows/deploy-preview.yml +``` + +**Future (RECOMMENDED):** +```bash +# Single source of truth +1. Edit requirements.txt +2. Test locally: docker build && ./mkdocs.sh +3. Commit and push - CI automatically uses new versions +``` + +--- + +## Risk Assessment + +| Risk | Current | After Implementation | +|------|---------|---------------------| +| Dependency drift | HIGH | LOW | +| "Works on my machine" | MEDIUM | LOW | +| Update complexity | HIGH | LOW | +| Audit trail | NONE | FULL (git history) | +| Rollback difficulty | HIGH | LOW (git revert) | + +--- + +## Conclusion + +**Current Status:** βœ… Environments are functionally identical but ❌ maintained unsafely + +**Recommended Action:** Implement **Option A (Requirements.txt with Dockerfile Reuse)** immediately + +**Effort:** ~30 minutes implementation, ~15 minutes testing + +**Impact:** +- Eliminates high-risk dependency duplication +- Establishes industry-standard Python dependency management +- Prevents future "works on my machine" issues +- Makes dependency updates 10x easier and safer + +**Next Step:** Create `requirements.txt` file and begin Phase 1 implementation + +--- + +## References + +### Files Analyzed +- `mkdocs.sh` - Local development container launcher +- `docker/mkdocs-material/Dockerfile` - Custom Docker image definition +- `.github/workflows/deploy-live.yml` - Production deployment workflow +- `.github/workflows/deploy-preview.yml` - Preview deployment workflow + +### Research Sources +- "GitHub Actions Docker development environment parity best practices 2024 2025" (Perplexity search) +- "MkDocs Material Docker GitHub Actions best practices requirements.txt vs inline pip install" (Perplexity search) + +### Key Best Practices Applied +1. Use same Dockerfile in both local and CI +2. Single source of truth for dependencies (requirements.txt) +3. Version locking for reproducibility +4. Layer caching optimization +5. Clear documentation and audit trails diff --git a/docs/internal/project-improvement-backlog.md b/docs/internal/project-improvement-backlog.md new file mode 100644 index 0000000..71b0f53 --- /dev/null +++ b/docs/internal/project-improvement-backlog.md @@ -0,0 +1,408 @@ +--- +title: "satware.ai Documentation Project - Improvement Backlog (INTERNAL)" +date: 2025-11-09 +author: Development Team +status: Active +confidentiality: Internal Use Only +last_reviewed: 2025-11-09 +--- + +# satware.ai Documentation Project - Improvement Backlog + +**Status:** Active development backlog +**Purpose:** Track improvement tasks for quality, performance, and maintainability +**Last Updated:** 2025-11-09 + +--- + +## Recently Completed (2025-11-09) + +These tasks have been completed as part of recent development work: + +- [x] **Internal Documentation Protection System** + - Created `docs/internal/` directory for internal documentation + - Configured mkdocs-exclude plugin with correct glob pattern + - Added `.clinerules/satware-ai-dev.md` for AI assistant enforcement + - Documented policy in README.md + - Verified exclusion works (build testing completed) + +- [x] **CI/CD Development Environment Analysis** + - Analyzed local development environment (mkdocs.sh + Dockerfile) + - Compared with GitHub Actions workflow + - Documented findings in `docs/internal/dev-ci-parity-analysis.md` + +--- + +## Code Organization and Structure + +### High Priority + +[ ] 1. **Reorganize SCSS files into a more modular structure** + - Split large SCSS files into smaller, purpose-specific files + - Create a consistent naming convention for all SCSS files + - Document the purpose of each SCSS module + - **Status:** Pending + +[ ] 2. **Implement a consistent file naming convention across the project** + - Standardize on kebab-case or snake_case for all filenames + - Ensure all filenames clearly indicate their purpose + - **Status:** Pending + +[ ] 3. **Add missing dependencies to requirements.txt** + - Add libsass/sass package which is used in compile_scss.py + - Verify all dependencies are properly versioned + - **Status:** Pending - Note: Python dependencies may not be needed if using Docker + +[ ] 4. **Create a proper project structure documentation** + - Document the purpose of each directory + - Explain the relationship between different components + - **Status:** Pending - Consider adding to docs/internal/ + +### Medium Priority + +[ ] 5. **Implement Git hooks for pre-commit validation** + - Add linting for Markdown files + - Add validation for SCSS/CSS files + - Ensure proper formatting before commits + - **Status:** Partial - .clinerules exists but no git hooks yet + +--- + +## Documentation Quality + +### High Priority + +[ ] 6. **Establish consistent documentation standards** + - Create a style guide for documentation content + - Define standards for headings, lists, code blocks, etc. + - Implement templates for common documentation types + - **Status:** Pending + +[ ] 7. **Review and improve all documentation content** + - Check for spelling and grammar issues + - Ensure consistent terminology throughout + - Verify all links are working correctly + - **Status:** Pending + +### Medium Priority + +[ ] 8. **Add proper documentation for custom components** + - Document all custom HTML/CSS components + - Provide usage examples for each component + - Create a component showcase page + - **Status:** Pending + +[ ] 9. **Improve code documentation** + - Add docstrings to all Python functions + - Document CSS/SCSS classes and their purposes + - Add comments to complex code sections + - **Status:** Pending + +[ ] 10. **Create contributor guidelines** + - Document the process for contributing to the project + - Provide setup instructions for new contributors + - Define code review and merge processes + - **Status:** Pending + +--- + +## Build Process + +### High Priority + +[ ] 12. **Create a proper build pipeline** + - Implement a CI/CD workflow using GitHub Actions or similar + - Add automated testing for the build process + - Create staging and production deployment workflows + - **Status:** Partial - CI/CD analysis done in `docs/internal/dev-ci-parity-analysis.md` + - **Next Step:** Implement improvements based on analysis + +[ ] 13. **Optimize the development workflow** + - Add hot reloading for all file types + - Improve error reporting during development + - Create a unified development command + - **Status:** Pending + +### Medium Priority + +[ ] 11. **Improve the SCSS compilation process** + - Add source maps for easier debugging + - Implement proper error reporting + - Add autoprefixing for better browser compatibility + - **Status:** Pending + +[ ] 14. **Add build validation steps** + - Implement link checking + - Add HTML validation + - Check for accessibility issues during build + - **Status:** Pending + +[ ] 15. **Create a proper release process** + - Document version numbering scheme + - Implement changelog generation + - Create release tagging process + - **Status:** Pending + +--- + +## Performance Optimizations + +### High Priority + +[ ] 16. **Optimize image assets** + - Implement proper image compression + - Convert images to modern formats (WebP, AVIF) + - Add responsive image handling + - **Status:** Pending - Many images already in AVIF format + +[ ] 20. **Optimize page load performance** + - Reduce time to first contentful paint + - Implement lazy loading for below-the-fold content + - Optimize third-party script loading + - **Status:** Pending + +### Medium Priority + +[ ] 17. **Improve JavaScript performance** + - Minify and bundle JavaScript files + - Implement lazy loading for non-critical scripts + - Add proper error handling and logging + - **Status:** Pending + +[ ] 18. **Enhance CSS performance** + - Remove unused CSS + - Optimize CSS delivery + - Implement critical CSS loading + - **Status:** Pending + +[ ] 19. **Implement proper caching strategies** + - Add cache headers for static assets + - Implement service worker for offline support + - Use content hashing for cache busting + - **Status:** Pending + +--- + +## Accessibility Improvements + +### High Priority + +[ ] 21. **Conduct a comprehensive accessibility audit** + - Test with screen readers + - Check keyboard navigation + - Verify color contrast ratios + - **Status:** Pending + +[ ] 22. **Implement proper ARIA attributes** + - Add appropriate ARIA roles + - Ensure all interactive elements have proper labels + - Implement proper focus management + - **Status:** Pending + +### Medium Priority + +[ ] 23. **Improve form accessibility** + - Add proper labels for all form fields + - Implement error messaging for form validation + - Ensure keyboard accessibility for all forms + - **Status:** Pending + +[ ] 24. **Enhance content readability** + - Implement proper heading hierarchy + - Ensure sufficient text contrast + - Add alt text for all images + - **Status:** Pending + +[ ] 25. **Create an accessibility statement page** + - Document the accessibility standards followed + - Provide contact information for accessibility issues + - List known accessibility limitations + - **Status:** Pending + +--- + +## SEO Enhancements + +### High Priority + +[ ] 26. **Implement proper meta tags** + - Add OpenGraph tags for social sharing + - Implement Twitter card metadata + - Ensure all pages have unique meta descriptions + - **Status:** Pending - Some OG tags may already exist + +[ ] 29. **Implement XML sitemap** + - Generate a comprehensive sitemap + - Add sitemap to robots.txt + - Submit sitemap to search engines + - **Status:** Pending - MkDocs may generate sitemap automatically + +### Medium Priority + +[ ] 27. **Improve URL structure** + - Create SEO-friendly URLs + - Implement proper redirects for changed URLs + - Add canonical URLs where appropriate + - **Status:** Pending - REDIRECTS.md exists with redirect config + +[ ] 28. **Enhance content for SEO** + - Optimize heading structure for keywords + - Improve content readability scores + - Add structured data where appropriate + - **Status:** Pending + +[ ] 30. **Add analytics and monitoring** + - Implement privacy-friendly analytics + - Set up performance monitoring + - Create SEO performance dashboards + - **Status:** Pending + +--- + +## Content Structure and Organization + +### High Priority + +[ ] 31. **Review and improve navigation structure** + - Optimize main navigation for usability + - Implement breadcrumbs for better orientation + - Create a logical content hierarchy + - **Status:** Pending + +[ ] 33. **Improve search functionality** + - Enhance search result relevance + - Add search filters and facets + - Implement search analytics + - **Status:** Pending + +### Medium Priority + +[ ] 32. **Standardize content templates** + - Create consistent page templates + - Implement standard sections for similar content + - Ensure consistent formatting across pages + - **Status:** Pending - docs/templates/ exists + +[ ] 34. **Create a proper content strategy** + - Define target audiences and their needs + - Map content to user journeys + - Establish content update processes + - **Status:** Pending + +[ ] 35. **Implement content versioning** + - Add version indicators for documentation + - Create an archive for older versions + - Implement version switching functionality + - **Status:** Pending + +--- + +## Internationalization and Localization + +### Low Priority (Future) + +[ ] 36. **Prepare for multi-language support** + - Implement proper language selection + - Extract all UI strings for translation + - Create a translation workflow + - **Status:** Pending - Currently German language site + +[ ] 37. **Add language-specific SEO** + - Implement hreflang tags + - Create language-specific sitemaps + - Optimize metadata for each language + - **Status:** Pending + +[ ] 38. **Implement right-to-left (RTL) support** + - Add RTL stylesheets + - Test UI components in RTL mode + - Ensure proper text rendering for all languages + - **Status:** Pending - Not needed for German + +[ ] 39. **Create localization guidelines** + - Document translation processes + - Define terminology glossaries + - Establish quality control for translations + - **Status:** Pending + +[ ] 40. **Implement region-specific content** + - Add region detection + - Create region-specific examples + - Implement locale-aware formatting + - **Status:** Pending + +--- + +## Testing and Quality Assurance + +### High Priority + +[ ] 41. **Implement automated testing** + - Add unit tests for JavaScript functionality + - Create visual regression tests + - Implement end-to-end testing + - **Status:** Pending + +[ ] 42. **Create a cross-browser testing strategy** + - Define supported browsers and versions + - Implement browser-specific fixes + - Document browser compatibility issues + - **Status:** Pending + +### Medium Priority + +[ ] 43. **Add mobile device testing** + - Test on various device sizes + - Implement device-specific optimizations + - Create a responsive design testing process + - **Status:** Pending + +[ ] 44. **Implement content quality checks** + - Add spelling and grammar checking + - Implement readability scoring + - Create a content review process + - **Status:** Pending + +[ ] 45. **Create a user feedback mechanism** + - Add page rating functionality + - Implement user feedback forms + - Create a process for addressing user feedback + - **Status:** Pending + +--- + +## Priority Summary + +### Immediate Next Steps (High Priority) + +1. Fix YAML duplicate key warning in mkdocs.yml (line 102) +2. Implement CI/CD improvements based on analysis +3. Conduct accessibility audit +4. Optimize image assets (many already in AVIF) +5. Implement automated testing + +### Short-term (Medium Priority) + +1. Reorganize SCSS files +2. Improve build pipeline +3. Enhance documentation quality +4. Optimize performance + +### Long-term (Low Priority) + +1. Internationalization (if needed) +2. Advanced SEO enhancements +3. Content versioning + +--- + +## Notes + +- This backlog is maintained in `docs/internal/` because it contains development plans not intended for public consumption +- Tasks should be moved to GitLab Issues when ready for implementation +- Progress should be tracked in GitLab Milestones +- Many tasks may already be partially implemented - requires investigation + +**Original file:** `tasks.md` (root directory) +**Moved to:** `docs/internal/project-improvement-backlog.md` on 2025-11-09 +**Reason:** Internal planning document, not public-facing content diff --git a/docs/internal/session-2025-11-09-end-of-day.md b/docs/internal/session-2025-11-09-end-of-day.md new file mode 100644 index 0000000..fb7c7f8 --- /dev/null +++ b/docs/internal/session-2025-11-09-end-of-day.md @@ -0,0 +1,120 @@ +# Session End: 2025-11-09 (Saturday) + +**Branch:** `feature/dev-env-inspection` +**Status:** Ready for next session +**Working Tree:** Clean βœ… + +## Today's Accomplishments + +### 1. Internal Documentation Protection System βœ… +- **Commit:** `ff766f8` - feat: Implement internal documentation protection system +- Verified and implemented `docs/internal/` exclusion from public site +- Tested mkdocs-exclude plugin with correct glob pattern: `"internal/*"` +- Performance improvement: 17.51s β†’ 7.07s (59% faster builds) + +### 2. .clinerules Documentation Updates βœ… +- **Commit:** `853b86a` - docs: Update .clinerules with verified glob pattern and testing procedures +- Documented verified glob pattern: `"internal/*"` NOT `"internal/**/*"` +- Added comprehensive testing procedures +- Included verification commands and success criteria + +### 3. Project Organization βœ… +- **Commit:** `f2386e1` - docs: Move and organize tasks.md to internal backlog +- Reorganized project improvement backlog +- Moved tasks.md to `docs/internal/project-improvement-backlog.md` + +### 4. Repository Cleanup βœ… +- **Commit:** `e9c6734` - chore: Add Gemfile.lock to .gitignore (orphaned file) +- Added Gemfile.lock to .gitignore (orphaned Jekyll file) + +### 5. Earlier Today: User Experience Improvements βœ… +- **Commit:** `0a25d32` - Add custom 404 page and preload optimizations +- Custom 404 page implementation +- Preload optimizations for enhanced user experience + +### 6. Profile Integration βœ… +- **Commit:** `53dff3c` - Merge feature/add-john-alesi-profile +- Integrated John Alesi profile into dev-env-inspection branch + +## Current State + +### Repository Status +- **Local Branch:** `feature/dev-env-inspection` (up to date with origin) +- **Remote:** Synchronized with `origin/feature/dev-env-inspection` +- **Working Tree:** Clean (no uncommitted changes) +- **Last Commit:** e9c6734 (chore: Add Gemfile.lock to .gitignore) + +### Running Services +- **MkDocs Container:** `nostalgic_bohr` (running on port 8000) + - Container ID: b2fa71c840fd + - Image: squidfunk/mkdocs-material-custom + - Started: ~1 hour ago + - Status: Up and serving + +### GitHub Actions Status +- **Note:** GitHub Actions pipeline status not checked (requires GitHub MCP or manual browser check) +- **Recommendation:** Check https://github.com/jane-alesi/satware.ai/actions manually + +## What's Complete vs In-Progress + +### βœ… Complete +1. Internal documentation protection system fully implemented and tested +2. .clinerules updated with verified patterns and procedures +3. Project organization improved (backlog moved to internal/) +4. Repository cleanup (Gemfile.lock handling) +5. All changes committed and pushed to origin +6. Local development environment verified + +### πŸ”„ In-Progress +- None (all work committed) + +### ⏸️ Paused / Future Considerations +- Additional testing of internal documentation exclusion on live deployment +- Review GitHub Actions pipeline status +- Consider additional items from project-improvement-backlog.md + +## Blockers + +**None identified.** All planned work completed successfully. + +## Next Session Action Items + +### High Priority +- [ ] Review GitHub Actions pipeline status for recent commits +- [ ] Test internal documentation exclusion on live site (verify site/internal/ doesn't exist after deployment) +- [ ] Review and prioritize items from `docs/internal/project-improvement-backlog.md` + +### Medium Priority +- [ ] Consider merging `feature/dev-env-inspection` to main-mkdocs (if all tests pass) +- [ ] Document any findings from CI/CD pipeline review +- [ ] Plan next development iteration + +### Low Priority +- [ ] Cleanup: Remove exited Docker containers (dokuwiki_satware, satwareai-jekyll-1) +- [ ] Review and update team profiles if needed + +## Key Learnings + +1. **Glob Pattern Discovery:** mkdocs-exclude requires `"directory/*"` NOT `"directory/**/*"` for top-level exclusion +2. **Performance Impact:** Proper exclusion dramatically improves build times (59% faster) +3. **Testing Importance:** Always verify exclusions work with explicit container checks +4. **Documentation Value:** Comprehensive .clinerules prevent future mistakes + +## Environment Details + +- **OS:** Manjaro Linux +- **IDE:** IntelliJ IDEA Ultimate +- **Docker:** Multiple containers (1 running, 2 exited) +- **Time:** Saturday, 2025-11-09, 12:30 PM (Europe/Berlin, UTC+1) + +## Repository Links + +- **Fork (Origin):** https://github.com/jane-alesi/satware.ai.git +- **Upstream:** https://github.com/satwareAG/satware.ai.git +- **Branch:** feature/dev-env-inspection + +--- + +**Session Duration:** Full development day +**Commits Made:** 6 commits today +**Status:** βœ… Safe to end session - all work preserved remotely diff --git a/mkdocs.yml b/mkdocs.yml index 4e4775b..c3c3a6d 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -54,6 +54,10 @@ theme: plugins: - meta + - exclude: + glob: + - "internal/*" + - "templates/*" - blog: blog_dir: blog blog_toc: true diff --git a/tasks.md b/tasks.md deleted file mode 100644 index 5e1d68b..0000000 --- a/tasks.md +++ /dev/null @@ -1,243 +0,0 @@ -# Improvement Tasks for satware.ai Documentation - -This document contains a detailed list of actionable improvement tasks for the satware.ai documentation project. Each task is designed to enhance the quality, performance, and maintainability of the documentation site. - -## Code Organization and Structure - -[ ] 1. Reorganize SCSS files into a more modular structure - - Split large SCSS files into smaller, purpose-specific files - - Create a consistent naming convention for all SCSS files - - Document the purpose of each SCSS module - -[ ] 2. Implement a consistent file naming convention across the project - - Standardize on kebab-case or snake_case for all filenames - - Ensure all filenames clearly indicate their purpose - -[ ] 3. Add missing dependencies to requirements.txt - - Add libsass/sass package which is used in compile_scss.py - - Verify all dependencies are properly versioned - -[ ] 4. Create a proper project structure documentation - - Document the purpose of each directory - - Explain the relationship between different components - -[ ] 5. Implement Git hooks for pre-commit validation - - Add linting for Markdown files - - Add validation for SCSS/CSS files - - Ensure proper formatting before commits - -## Documentation Quality - -[ ] 6. Establish consistent documentation standards - - Create a style guide for documentation content - - Define standards for headings, lists, code blocks, etc. - - Implement templates for common documentation types - -[ ] 7. Review and improve all documentation content - - Check for spelling and grammar issues - - Ensure consistent terminology throughout - - Verify all links are working correctly - -[ ] 8. Add proper documentation for custom components - - Document all custom HTML/CSS components - - Provide usage examples for each component - - Create a component showcase page - -[ ] 9. Improve code documentation - - Add docstrings to all Python functions - - Document CSS/SCSS classes and their purposes - - Add comments to complex code sections - -[ ] 10. Create contributor guidelines - - Document the process for contributing to the project - - Provide setup instructions for new contributors - - Define code review and merge processes - -## Build Process - -[ ] 11. Improve the SCSS compilation process - - Add source maps for easier debugging - - Implement proper error reporting - - Add autoprefixing for better browser compatibility - -[ ] 12. Create a proper build pipeline - - Implement a CI/CD workflow using GitHub Actions or similar - - Add automated testing for the build process - - Create staging and production deployment workflows - -[ ] 13. Optimize the development workflow - - Add hot reloading for all file types - - Improve error reporting during development - - Create a unified development command - -[ ] 14. Add build validation steps - - Implement link checking - - Add HTML validation - - Check for accessibility issues during build - -[ ] 15. Create a proper release process - - Document version numbering scheme - - Implement changelog generation - - Create release tagging process - -## Performance Optimizations - -[ ] 16. Optimize image assets - - Implement proper image compression - - Convert images to modern formats (WebP, AVIF) - - Add responsive image handling - -[ ] 17. Improve JavaScript performance - - Minify and bundle JavaScript files - - Implement lazy loading for non-critical scripts - - Add proper error handling and logging - -[ ] 18. Enhance CSS performance - - Remove unused CSS - - Optimize CSS delivery - - Implement critical CSS loading - -[ ] 19. Implement proper caching strategies - - Add cache headers for static assets - - Implement service worker for offline support - - Use content hashing for cache busting - -[ ] 20. Optimize page load performance - - Reduce time to first contentful paint - - Implement lazy loading for below-the-fold content - - Optimize third-party script loading - -## Accessibility Improvements - -[ ] 21. Conduct a comprehensive accessibility audit - - Test with screen readers - - Check keyboard navigation - - Verify color contrast ratios - -[ ] 22. Implement proper ARIA attributes - - Add appropriate ARIA roles - - Ensure all interactive elements have proper labels - - Implement proper focus management - -[ ] 23. Improve form accessibility - - Add proper labels for all form fields - - Implement error messaging for form validation - - Ensure keyboard accessibility for all forms - -[ ] 24. Enhance content readability - - Implement proper heading hierarchy - - Ensure sufficient text contrast - - Add alt text for all images - -[ ] 25. Create an accessibility statement page - - Document the accessibility standards followed - - Provide contact information for accessibility issues - - List known accessibility limitations - -## SEO Enhancements - -[ ] 26. Implement proper meta tags - - Add OpenGraph tags for social sharing - - Implement Twitter card metadata - - Ensure all pages have unique meta descriptions - -[ ] 27. Improve URL structure - - Create SEO-friendly URLs - - Implement proper redirects for changed URLs - - Add canonical URLs where appropriate - -[ ] 28. Enhance content for SEO - - Optimize heading structure for keywords - - Improve content readability scores - - Add structured data where appropriate - -[ ] 29. Implement XML sitemap - - Generate a comprehensive sitemap - - Add sitemap to robots.txt - - Submit sitemap to search engines - -[ ] 30. Add analytics and monitoring - - Implement privacy-friendly analytics - - Set up performance monitoring - - Create SEO performance dashboards - -## Content Structure and Organization - -[ ] 31. Review and improve navigation structure - - Optimize main navigation for usability - - Implement breadcrumbs for better orientation - - Create a logical content hierarchy - -[ ] 32. Standardize content templates - - Create consistent page templates - - Implement standard sections for similar content - - Ensure consistent formatting across pages - -[ ] 33. Improve search functionality - - Enhance search result relevance - - Add search filters and facets - - Implement search analytics - -[ ] 34. Create a proper content strategy - - Define target audiences and their needs - - Map content to user journeys - - Establish content update processes - -[ ] 35. Implement content versioning - - Add version indicators for documentation - - Create an archive for older versions - - Implement version switching functionality - -## Internationalization and Localization - -[ ] 36. Prepare for multi-language support - - Implement proper language selection - - Extract all UI strings for translation - - Create a translation workflow - -[ ] 37. Add language-specific SEO - - Implement hreflang tags - - Create language-specific sitemaps - - Optimize metadata for each language - -[ ] 38. Implement right-to-left (RTL) support - - Add RTL stylesheets - - Test UI components in RTL mode - - Ensure proper text rendering for all languages - -[ ] 39. Create localization guidelines - - Document translation processes - - Define terminology glossaries - - Establish quality control for translations - -[ ] 40. Implement region-specific content - - Add region detection - - Create region-specific examples - - Implement locale-aware formatting - -## Testing and Quality Assurance - -[ ] 41. Implement automated testing - - Add unit tests for JavaScript functionality - - Create visual regression tests - - Implement end-to-end testing - -[ ] 42. Create a cross-browser testing strategy - - Define supported browsers and versions - - Implement browser-specific fixes - - Document browser compatibility issues - -[ ] 43. Add mobile device testing - - Test on various device sizes - - Implement device-specific optimizations - - Create a responsive design testing process - -[ ] 44. Implement content quality checks - - Add spelling and grammar checking - - Implement readability scoring - - Create a content review process - -[ ] 45. Create a user feedback mechanism - - Add page rating functionality - - Implement user feedback forms - - Create a process for addressing user feedback \ No newline at end of file