Workpackage System Overview

Workpackage Developer Guide

This guide is for developers who want to extend, customize, or contribute to the Sigao CLI Workpackage Automation system.

Workpackage Developer Guide

This guide is for developers who want to extend, customize, or contribute to the Sigao CLI Workpackage Automation system.

Architecture Overview

The workpackage system follows a modular, event-driven architecture with clear separation of concerns.

System Architecture

graph TD
    A[CLI Interface] --> B[WorkpackageModule]
    B --> C[Validator]
    B --> D[Orchestrator]
    B --> E[UI System]
    D --> F[Context Engine]
    D --> G[Claude Interface]
    D --> H[Task Executor]
    F --> I[Logging System]
    G --> I
    H --> I

Core Components

1. WorkpackageModule

Main entry point extending BaseInstaller
Handles CLI integration and option parsing
Exports executeWorkpackage function

2. WorkpackageOrchestrator

WorkpackageOrchestrator.js - Main execution controller
DependencyResolver.js - Handles workpackage dependencies
TaskExecutor.js - Individual task execution
InterventionHandler.js - Manual intervention points

3. Context System

ContextManager.js - Central context management
GlobalContext.js - Project-wide configuration
UowContext.js - Unit of work (workpackage) context
CumulativeSummary.js - Task output aggregation

4. Claude Integration

ClaudeInterface.js - Claude CLI wrapper
PromptBuilder.js - Template-based prompt generation
SubprocessManager.js - Process execution with retry

5. UI Components

UIComponent.js - Base UI class (singleton)
WorkpackageUI.js - Main UI controller
ProcessIndicator.js - Real-time process tracking
ProgressBar.js - Progress visualization

Extension Points

The system provides several extension points for customization:

1. Custom Validators

Create custom validators by extending the validation system:

// src/modules/workpackage/validators/custom-validator.js
export class CustomValidator {
  constructor(schema) {
    this.schema = schema;
  }

  validate(data) {
    // Your validation logic
    const errors = [];
    
    // Example: Check custom field
    if (data.customField && !this.isValidCustomField(data.customField)) {
      errors.push({
        path: '/customField',
        message: 'Invalid custom field format'
      });
    }
    
    return {
      valid: errors.length === 0,
      errors
    };
  }
  
  isValidCustomField(field) {
    // Custom validation logic
    return /^CUSTOM-\d+$/.test(field);
  }
}

// Register with WorkpackageValidator
import { WorkpackageValidator } from '../validator.js';

WorkpackageValidator.addCustomValidator('custom', new CustomValidator());

2. Custom UI Components

Extend the UI system with custom components:

// src/modules/workpackage/ui/CustomIndicator.js
import { UIComponent } from './UIComponent.js';

export class CustomIndicator extends UIComponent {
  constructor(name) {
    super();
    this.name = name;
    this.state = 'idle';
  }

  display() {
    const icon = this.getStateIcon();
    const color = this.getStateColor();
    
    process.stdout.write(
      `\r${this.chalk[color](icon)} ${this.name}: ${this.state}`
    );
  }

  getStateIcon() {
    const icons = {
      idle: '⏸',
      running: '▶',
      success: '✓',
      error: '✗'
    };
    return icons[this.state] || '?';
  }

  getStateColor() {
    const colors = {
      idle: 'gray',
      running: 'yellow',
      success: 'green',
      error: 'red'
    };
    return colors[this.state] || 'white';
  }

  setState(state) {
    this.state = state;
    this.display();
  }
}

3. Custom Test Runners

Implement custom test execution strategies:

// src/modules/workpackage/testing/CustomTestRunner.js
import { TestRunner } from './TestRunner.js';

export class CustomTestRunner extends TestRunner {
  constructor(options = {}) {
    super(options);
    this.framework = options.framework || 'custom';
  }

  async runTest(command, options = {}) {
    // Pre-process command for your framework
    const processedCommand = this.preprocessCommand(command);
    
    // Run with custom logic
    const result = await super.runTest(processedCommand, {
      ...options,
      env: {
        ...process.env,
        CUSTOM_TEST_MODE: 'true'
      }
    });
    
    // Post-process results
    return this.postprocessResult(result);
  }

  preprocessCommand(command) {
    // Add framework-specific flags
    if (this.framework === 'jest') {
      return `${command} --json --outputFile=test-results.json`;
    }
    return command;
  }

  postprocessResult(result) {
    // Parse framework-specific output
    if (this.framework === 'jest' && result.stdout) {
      try {
        const jsonResult = JSON.parse(result.stdout);
        return {
          ...result,
          passed: jsonResult.success,
          testCount: jsonResult.numTotalTests,
          failures: jsonResult.numFailedTests
        };
      } catch (e) {
        // Fallback to standard result
      }
    }
    return result;
  }
}

4. Custom Logging Formatters

Create custom output formats:

// src/modules/workpackage/logging/CustomFormatter.js
export class CustomFormatter {
  constructor(options = {}) {
    this.format = options.format || 'custom';
    this.includeTimestamps = options.includeTimestamps !== false;
  }

  formatTask(task, result) {
    switch (this.format) {
      case 'json':
        return this.formatJSON(task, result);
      case 'xml':
        return this.formatXML(task, result);
      default:
        return this.formatCustom(task, result);
    }
  }

  formatJSON(task, result) {
    return JSON.stringify({
      timestamp: new Date().toISOString(),
      task: task.task,
      result: {
        success: result.success,
        output: result.output,
        duration: result.duration
      }
    }, null, 2);
  }

  formatXML(task, result) {
    return `
<task>
  <timestamp>${new Date().toISOString()}</timestamp>
  <description>${this.escapeXML(task.task)}</description>
  <result>
    <success>${result.success}</success>
    <output>${this.escapeXML(result.output)}</output>
    <duration>${result.duration}</duration>
  </result>
</task>`;
  }

  formatCustom(task, result) {
    // Your custom format
    return `[${new Date().toLocaleTimeString()}] Task: ${task.task}
Status: ${result.success ? 'PASSED' : 'FAILED'}
Duration: ${result.duration}ms
Output: ${result.output}
${'='.repeat(80)}`;
  }

  escapeXML(str) {
    return str.replace(/[<>&'"]/g, c => ({
      '<': '&lt;',
      '>': '&gt;',
      '&': '&amp;',
      "'": '&apos;',
      '"': '&quot;'
    })[c]);
  }
}

Context System Deep Dive

The context system provides state management throughout workpackage execution.

Context Hierarchy

GlobalContext (Project-wide)
    │
    ├─> UowContext (Per Workpackage)
    │       │
    │       └─> Task Context (Per Task)
    │
    └─> CumulativeContext (Aggregated)

GlobalContext Usage

// Setting global context
const contextManager = new ContextManager();

contextManager.setGlobalContext({
  project: {
    name: 'my-app',
    version: '1.0.0',
    type: 'web-application'
  },
  environment: {
    node: process.version,
    platform: process.platform,
    cwd: process.cwd()
  },
  config: {
    testFramework: 'jest',
    linter: 'eslint',
    packageManager: 'npm'
  }
});

// Accessing in prompt templates
const context = contextManager.getGlobalContext();
const prompt = `
Project: ${context.project.name}
Framework: ${context.config.testFramework}
`;

UowContext Management

// Initialize UoW context for a workpackage
contextManager.setUowContext(workpackageId, {
  startTime: new Date(),
  tasks: workpackage.tasks,
  currentTaskIndex: 0,
  outputs: [],
  status: 'in_progress'
});

// Update after each task
contextManager.updateUowContext(workpackageId, {
  currentTaskIndex: taskIndex + 1,
  outputs: [...previousOutputs, taskOutput],
  lastTaskDuration: endTime - startTime
});

// Access cumulative information
const summary = contextManager.getCumulativeSummary();
// Returns last 10 task summaries with outputs

Context in Prompt Building

// src/modules/workpackage/claude/custom-prompt.js
export class CustomPromptBuilder extends PromptBuilder {
  buildTaskPrompt(task, context) {
    const template = this.loadTemplate('custom-task');
    
    return this.renderTemplate(template, {
      task: task.task,
      acceptance: task.acceptance,
      globalContext: this.formatContext(context.global),
      uowContext: this.formatContext(context.uow),
      previousOutputs: this.formatPreviousOutputs(context.cumulative),
      customInstructions: this.getCustomInstructions(task)
    });
  }

  getCustomInstructions(task) {
    // Add custom instructions based on task type
    if (task.type === 'database') {
      return `
Special considerations for database tasks:
- Ensure migrations are reversible
- Include proper indexes
- Consider data integrity constraints
`;
    }
    return '';
  }

  formatPreviousOutputs(cumulative) {
    // Format previous task outputs for context
    return cumulative.tasks
      .slice(-5) // Last 5 tasks
      .map(t => `Task: ${t.task}\nOutput: ${t.output}`)
      .join('\n---\n');
  }
}

Creating Custom Workpackage Templates

Template Structure

Create reusable workpackage templates for common scenarios:

// templates/web-api-template.js
export function createWebAPITemplate(options) {
  const { name, database, auth } = options;
  
  return [
    {
      id: `WP-01-${name.toUpperCase()}-SETUP`,
      title: `${name} API Setup`,
      depends_on: [],
      global_context: {
        project: name,
        type: 'rest-api',
        framework: 'express',
        database: database || 'postgresql',
        auth: auth || 'jwt'
      },
      tasks: [
        {
          task: 'Initialize Express application with middleware',
          acceptance: 'Server starts and responds to health check',
          test: 'curl -f http://localhost:3000/health'
        },
        {
          task: `Set up ${database} connection with connection pooling`,
          acceptance: 'Database connection established',
          test: 'npm run db:test'
        }
      ],
      deliverables: [
        'src/server.js',
        'src/config/database.js',
        'package.json'
      ],
      agent: 'claude',
      status: 'ready'
    },
    // Additional workpackages...
  ];
}

// Usage
const template = createWebAPITemplate({
  name: 'user-service',
  database: 'mongodb',
  auth: 'oauth2'
});

Template Registry

// src/modules/workpackage/templates/registry.js
export class TemplateRegistry {
  constructor() {
    this.templates = new Map();
    this.loadBuiltinTemplates();
  }

  register(name, templateFunction) {
    this.templates.set(name, templateFunction);
  }

  get(name, options = {}) {
    const templateFn = this.templates.get(name);
    if (!templateFn) {
      throw new Error(`Template '${name}' not found`);
    }
    return templateFn(options);
  }

  list() {
    return Array.from(this.templates.keys()).map(name => ({
      name,
      description: this.getDescription(name)
    }));
  }

  loadBuiltinTemplates() {
    this.register('web-api', createWebAPITemplate);
    this.register('cli-tool', createCLIToolTemplate);
    this.register('react-app', createReactAppTemplate);
    this.register('microservice', createMicroserviceTemplate);
  }

  getDescription(name) {
    const descriptions = {
      'web-api': 'REST API with Express and database',
      'cli-tool': 'Command-line tool with Commander.js',
      'react-app': 'React application with routing and state',
      'microservice': 'Microservice with messaging and monitoring'
    };
    return descriptions[name] || 'Custom template';
  }
}

Event System and Progress Tracking

The system uses an event-driven architecture for progress tracking and extensibility.

Event Types

// src/modules/workpackage/events/types.js
export const EventTypes = {
  // Lifecycle events
  EXECUTION_START: 'execution:start',
  EXECUTION_END: 'execution:end',
  EXECUTION_ERROR: 'execution:error',
  
  // Workpackage events
  WORKPACKAGE_START: 'workpackage:start',
  WORKPACKAGE_END: 'workpackage:end',
  WORKPACKAGE_SKIP: 'workpackage:skip',
  
  // Task events
  TASK_START: 'task:start',
  TASK_PROGRESS: 'task:progress',
  TASK_END: 'task:end',
  TASK_RETRY: 'task:retry',
  
  // Test events
  TEST_START: 'test:start',
  TEST_END: 'test:end',
  
  // UI events
  UI_UPDATE: 'ui:update',
  UI_PROMPT: 'ui:prompt'
};

Event Emitter Integration

// src/modules/workpackage/events/emitter.js
import { EventEmitter } from 'events';

export class WorkpackageEventEmitter extends EventEmitter {
  constructor() {
    super();
    this.metrics = {
      tasksTotal: 0,
      tasksCompleted: 0,
      tasksFailed: 0,
      startTime: null,
      endTime: null
    };
  }

  trackExecution() {
    this.on(EventTypes.EXECUTION_START, () => {
      this.metrics.startTime = Date.now();
    });

    this.on(EventTypes.TASK_END, (data) => {
      if (data.success) {
        this.metrics.tasksCompleted++;
      } else {
        this.metrics.tasksFailed++;
      }
    });

    this.on(EventTypes.EXECUTION_END, () => {
      this.metrics.endTime = Date.now();
    });
  }

  getMetrics() {
    return {
      ...this.metrics,
      duration: this.metrics.endTime - this.metrics.startTime,
      successRate: this.metrics.tasksCompleted / this.metrics.tasksTotal
    };
  }
}

Custom Event Handlers

// src/modules/workpackage/events/handlers/slack-notifier.js
export class SlackNotifier {
  constructor(webhookUrl) {
    this.webhookUrl = webhookUrl;
  }

  attach(eventEmitter) {
    eventEmitter.on(EventTypes.EXECUTION_START, async (data) => {
      await this.sendNotification({
        text: `🚀 Workpackage execution started: ${data.workpackageCount} packages`,
        color: 'good'
      });
    });

    eventEmitter.on(EventTypes.TASK_END, async (data) => {
      if (!data.success) {
        await this.sendNotification({
          text: `❌ Task failed: ${data.task}`,
          color: 'danger',
          fields: [
            { title: 'Error', value: data.error }
          ]
        });
      }
    });

    eventEmitter.on(EventTypes.EXECUTION_END, async (data) => {
      const metrics = data.metrics;
      await this.sendNotification({
        text: `✅ Execution complete`,
        color: metrics.tasksFailed > 0 ? 'warning' : 'good',
        fields: [
          { title: 'Total Tasks', value: metrics.tasksTotal },
          { title: 'Succeeded', value: metrics.tasksCompleted },
          { title: 'Failed', value: metrics.tasksFailed },
          { title: 'Duration', value: `${metrics.duration / 1000}s` }
        ]
      });
    });
  }

  async sendNotification(payload) {
    // Send to Slack webhook
    await fetch(this.webhookUrl, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(payload)
    });
  }
}

Debugging and Logging

Debug Mode

Enable comprehensive debugging:

// src/modules/workpackage/debug/debugger.js
export class WorkpackageDebugger {
  constructor(options = {}) {
    this.enabled = options.debug || process.env.DEBUG === 'true';
    this.logLevel = options.logLevel || 'info';
    this.outputDir = options.outputDir || '.sigao/debug';
  }

  async initialize() {
    if (!this.enabled) return;

    // Create debug directory
    await fs.mkdir(this.outputDir, { recursive: true });

    // Set up debug streams
    this.streams = {
      claude: await this.createStream('claude-prompts.log'),
      subprocess: await this.createStream('subprocess.log'),
      context: await this.createStream('context-changes.log'),
      events: await this.createStream('events.log')
    };
  }

  logClaudePrompt(prompt, response) {
    if (!this.enabled) return;

    this.streams.claude.write(JSON.stringify({
      timestamp: new Date().toISOString(),
      prompt: prompt,
      response: response,
      metadata: {
        promptLength: prompt.length,
        responseLength: response.length,
        model: 'claude'
      }
    }) + '\n');
  }

  logSubprocess(command, result) {
    if (!this.enabled) return;

    this.streams.subprocess.write(JSON.stringify({
      timestamp: new Date().toISOString(),
      command,
      exitCode: result.exitCode,
      stdout: result.stdout,
      stderr: result.stderr,
      duration: result.duration
    }) + '\n');
  }

  logContextChange(type, oldValue, newValue) {
    if (!this.enabled) return;

    this.streams.context.write(JSON.stringify({
      timestamp: new Date().toISOString(),
      type,
      oldValue,
      newValue,
      diff: this.computeDiff(oldValue, newValue)
    }) + '\n');
  }

  async createStream(filename) {
    const path = `${this.outputDir}/${filename}`;
    return fs.createWriteStream(path, { flags: 'a' });
  }
}

Contributing Guidelines

Code Style

Follow the project's ES module conventions:

import { BaseClass } from './base.js';
export class MyClass extends BaseClass {
  constructor(options = {}) {
    super(options);
    this.logger = this.createLogger('MyClass');
  }
}

const BaseClass = require('./base');
module.exports = class MyClass extends BaseClass {
  constructor(options) {
    super(options);
  }
};

Testing Requirements

All contributions must include tests:

// test/modules/workpackage/unit/my-component.test.js
import { describe, it, expect, beforeEach, jest } from '@jest/globals';
import { MyComponent } from '../../../../src/modules/workpackage/my-component.js';

describe('MyComponent', () => {
  let component;

  beforeEach(() => {
    component = new MyComponent();
  });

  describe('initialization', () => {
    it('should initialize with default options', () => {
      expect(component.options).toEqual({
        debug: false,
        timeout: 5000
      });
    });
  });

  describe('execute', () => {
    it('should handle success case', async () => {
      const result = await component.execute('test');
      expect(result.success).toBe(true);
    });

    it('should handle error case', async () => {
      component.simulateError = true;
      await expect(component.execute('test'))
        .rejects.toThrow('Simulated error');
    });
  });
});

Documentation Standards

Add JSDoc comments to all public APIs:

/**
 * Manages workpackage execution lifecycle
 * @class
 * @extends {EventEmitter}
 * @example
 * const executor = new WorkpackageExecutor({
 *   dryRun: false,
 *   parallel: true
 * });
 * 
 * const result = await executor.execute(workpackages);
 */
export class WorkpackageExecutor extends EventEmitter {
  /**
   * Creates a new WorkpackageExecutor instance
   * @param {Object} options - Configuration options
   * @param {boolean} [options.dryRun=false] - Preview mode without execution
   * @param {boolean} [options.parallel=false] - Execute independent tasks in parallel
   * @param {number} [options.maxConcurrent=3] - Maximum concurrent tasks
   * @param {Function} [options.onProgress] - Progress callback
   * @throws {Error} If options are invalid
   */
  constructor(options = {}) {
    super();
    this.validateOptions(options);
    this.options = { ...this.defaultOptions, ...options };
  }

  /**
   * Executes an array of workpackages
   * @param {Array<IWorkPackage>} workpackages - Workpackages to execute
   * @returns {Promise<ExecutionResult>} Execution result with metrics
   * @throws {ValidationError} If workpackages are invalid
   * @throws {ExecutionError} If execution fails
   * @public
   */
  async execute(workpackages) {
    // Implementation
  }

  /**
   * @private
   */
  validateOptions(options) {
    // Internal validation
  }
}

Pull Request Process

Fork and Clone

git clone https://github.com/yourusername/sigao-cli.git
cd sigao-cli
git checkout -b feature/your-feature

Implement Changes
- Follow code style guidelines
- Add tests for new functionality
- Update documentation

Run Tests

npm test
npm run lint
npm run test:coverage

Commit with Conventional Commits

git commit -m "feat: add custom validator support"
git commit -m "fix: handle timeout in subprocess execution"
git commit -m "docs: update developer guide"

Submit PR
- Fill out PR template
- Link related issues
- Ensure CI passes

Development Workflow

# Install dependencies
npm install

# Run in development mode
npm run dev

# Run specific tests
npm test -- --testPathPatterns=validator

# Generate coverage report
npm run test:coverage

# Build documentation
npm run docs:build

# Validate workpackages
npm run validate:workpackages

Quick Reference

Key Classes and Their Roles

Class	Purpose	Location
`WorkpackageModule`	CLI entry point	`src/modules/workpackage/index.js`
`WorkpackageOrchestrator`	Main execution controller	`src/modules/workpackage/orchestrator/WorkpackageOrchestrator.js`
`ContextManager`	State management	`src/modules/workpackage/context/ContextManager.js`
`ClaudeInterface`	AI integration	`src/modules/workpackage/claude/ClaudeInterface.js`
`WorkpackageUI`	User interface	`src/modules/workpackage/ui/WorkpackageUI.js`
`TaskExecutor`	Task runner	`src/modules/workpackage/orchestrator/TaskExecutor.js`
`WorkpackageValidator`	Schema validation	`src/modules/workpackage/validator.js`

Common Extension Patterns

// Custom validator
WorkpackageValidator.addCustomValidator('myRule', validator);

// Custom UI component
WorkpackageUI.registerComponent('custom', CustomComponent);

// Event handler
eventEmitter.on('task:end', customHandler);

// Context provider
contextManager.addProvider('custom', customProvider);

// Template registration
templateRegistry.register('my-template', templateFunction);

Workpackage API Reference

This document provides a comprehensive API reference for the Sigao CLI Workpackage Automation system.

Getting Started with Workpackages

This guide will walk you through setting up and running your first workpackage in under 10 minutes!