Back to .md Directory

Pit Crew SI - Demo Script

This script guides you through a complete demonstration of Pit Crew Supreme Intelligence.

May 2, 2026
0 downloads
0 views
ai guardrails safety
View source

Pit Crew SI - Demo Script

This script guides you through a complete demonstration of Pit Crew Supreme Intelligence.

Setup (5 minutes)

Step 1: Deploy the App

# From the project directory
npm install
forge deploy
forge install

Step 2: Access the Dashboard

  1. Navigate to your Jira instance
  2. Click "Apps" → "Pit Crew SI Dashboard"
  3. Verify the dashboard loads with initial statistics

Step 3: Verify Configuration

The dashboard should show:

  • Mode: Dry-Run (safe default)
  • Auto-Approve: Disabled
  • Total Actions: 0

Demo 1: Automatic Bug Triage (5 minutes)

Scenario

A critical bug is reported that needs immediate attention.

Steps

  1. Create a bug in Jira:

    • Title: "Critical: Authentication API returning 500 errors"
    • Description: "Users unable to login. API endpoint /auth/login failing with 500 internal server error. Urgent fix needed for production."
    • Type: Bug
    • Priority: (Leave as default, let Pit Crew SI adjust it)
  2. Watch the magic happen:

    • Within seconds, Pit Crew SI detects the new issue
    • Automatically analyzes the content
    • Identifies keywords: "critical", "api", "authentication", "urgent", "production"
  3. Check the Dashboard:

    • Refresh the Pit Crew SI Dashboard
    • You should see new action logs:
      • "Label Issue" - Proposes adding: bug, api, security, critical-bug
      • "Set Priority" - Proposes changing to "High" or "Highest"
  4. Review the Proposal:

    • Click on the action to see the explanation
    • Note the reasoning: "Issue contains critical keywords and production impact"
    • Observe that it's in "proposed" status (dry-run mode)

Expected Results

  • ✅ Issue automatically triaged
  • ✅ Priority identified as critical/high
  • ✅ Relevant labels suggested
  • ✅ Actions logged with explanations
  • ✅ Guardrails checked and documented

Key Points to Highlight

  • Intelligent Analysis: Understands context from keywords
  • Explainable AI: Every decision has clear reasoning
  • Safety First: Dry-run mode prevents unwanted changes
  • Full Auditability: Complete log of what was analyzed and proposed

Demo 2: Sprint Planning (7 minutes)

Scenario

Planning a 2-week sprint with mixed priorities.

Preparation

Create 10-15 issues in Jira with varying priorities:

High Priority:

  1. "Fix memory leak in user service"
  2. "Critical security vulnerability in auth module"
  3. "Database migration for new schema"

Medium Priority: 4. "Implement user profile page" 5. "Add pagination to search results" 6. "Optimize database queries for reports" 7. "Update API documentation"

Low Priority: 8. "Refactor legacy code in utils module" 9. "Add unit tests for helper functions" 10. "Improve error messages"

Steps

  1. Collect Issues:

    // Use Jira API or manually note issue keys
    const issues = [
      'PROJ-101', 'PROJ-102', 'PROJ-103',
      // ... etc
    ];
    
  2. Generate Sprint Plan:

    const plan = await invoke('generateSprintPlan', {
      issues: issueObjects, // Full issue objects
      goals: [
        'Stabilize production environment',
        'Complete security improvements',
        'Improve user experience'
      ],
      capacity: 40 // Story points
    });
    
  3. Review the Plan:

    • Open the sprint plan results
    • Observe how issues are prioritized
    • Check capacity utilization
    • Review identified risks

Expected Results

  • ✅ Issues ranked by priority score
  • ✅ Capacity respected (won't overcommit)
  • ✅ Risks identified (e.g., "too many bugs indicates technical debt")
  • ✅ Dependencies noted
  • ✅ Clear reasoning for each selection

Key Points to Highlight

  • Data-Driven: Uses priority, type, age, and content
  • Capacity-Aware: Respects team limits
  • Risk Assessment: Identifies potential issues
  • Goal-Aligned: Considers sprint objectives

Demo 3: Execution Brief Generation (5 minutes)

Scenario

Breaking down a complex feature into actionable tasks.

Steps

  1. Create a Feature Issue:

    • Title: "Implement OAuth2 authentication"
    • Description: "Add OAuth2 support for Google and GitHub login. Users should be able to sign in using their existing accounts. Include proper error handling and security measures."
  2. Generate Execution Brief:

    const brief = await invoke('generateExecutionBrief', {
      issue: featureIssue,
      relatedPages: [] // Or include relevant Confluence pages
    });
    
  3. Review the Brief:

    • Title and summary
    • Clear objectives
    • Broken-down tasks with estimates
    • Technical context
    • References to related resources

Expected Results

Execution Brief: Implement OAuth2 authentication

Objectives:
- Complete OAuth2 authentication implementation
- Ensure code quality and test coverage
- Update relevant documentation

Tasks:
1. Analyze requirements and design approach (2 hours)
2. Implement core functionality (1 day)
3. Write tests (4 hours)
4. Code review and refinement (2 hours)

Technical Context:
- Issue Type: Story
- Project: PROJ
- Status: To Do

Key Points to Highlight

  • Structured Approach: Clear task breakdown
  • Time Estimates: Helps with planning
  • Context Preservation: Includes all relevant info
  • Actionable: Ready to execute

Demo 4: Safe Action Execution (8 minutes)

Scenario

Executing approved actions with full safety checks.

Steps

  1. Switch to Execute Mode:

    • In the Dashboard, click "Toggle Mode"
    • Confirm the switch to Execute mode
    • Observe the mode badge changes to green
  2. Create a New Bug:

    • Title: "Button styling broken on mobile devices"
    • Description: "The submit button on the login form appears cut off on mobile screens. Needs UI fix."
  3. Watch Auto-Triage:

    • Pit Crew SI analyzes the issue
    • Proposes actions: Add labels bug, ui, mobile
    • Set priority to Medium
  4. Review Guardrails:

    • Check the action log
    • See guardrail checks:
      • ✅ Action Allowed: label-issue is permitted
      • ✅ Label Count: 3 labels (within limit)
      • ✅ Reserved Labels: No reserved labels detected
      • ✅ Impact Assessment: Low impact
  5. Approve and Execute:

    • If auto-approve is disabled, manually approve
    • Action executes
    • Labels are added to the Jira issue
  6. Verify in Jira:

    • Open the issue in Jira
    • Confirm labels were added: bug, ui, mobile
    • Check action log shows "executed" status
  7. Demonstrate Revert:

    • Click "Revert" on the action
    • Labels are removed
    • New "reverted" log entry created

Expected Results

  • ✅ Actions execute in execute mode
  • ✅ Guardrails prevent unsafe actions
  • ✅ Changes appear in Jira immediately
  • ✅ Full audit trail maintained
  • ✅ Revert functionality works

Key Points to Highlight

  • Dual Mode Operation: Safe testing with dry-run
  • Guardrails: Multiple safety checks
  • Real Changes: Actually modifies Jira
  • Reversible: Can undo actions
  • Audit Trail: Complete history

Demo 5: Guardrails in Action (5 minutes)

Scenario

Demonstrating that guardrails prevent unsafe actions.

Steps

  1. Create High-Priority Override:

    • Create a bug with "minor typo in footer"
    • Pit Crew SI suggests low priority
    • Manually try to create an action setting priority to "Blocker"
  2. Run Guardrails:

    const proposal = {
      action: 'set-priority',
      details: {
        targetKey: 'PROJ-999',
        changes: { priority: 'Blocker' }
      },
      // ...
    };
    
    const guardrails = await GuardrailsService.checkAction(proposal, config);
    
  3. Review Results:

    • Guardrail fails: "Setting priority to Highest/Blocker requires manual approval"
    • Action status: "proposed" (not executed)
    • Requires approval even in execute mode
  4. Try Reserved Label:

    • Create proposal to add security label
    • Guardrail detects reserved label
    • Blocks automatic execution

Expected Results

  • ❌ High-priority actions blocked
  • ❌ Reserved labels require approval
  • ✅ System stays safe
  • ✅ Clear explanations provided

Key Points to Highlight

  • Safety First: Prevents mistakes
  • Smart Guardrails: Context-aware checks
  • Approval Workflow: For sensitive actions
  • Configurable: Can adjust rules

Wrap-Up: Key Takeaways (3 minutes)

What We Demonstrated

  1. Intelligent Triage

    • Automatic issue analysis
    • Context-aware priority setting
    • Smart label suggestions
  2. Sprint Planning

    • Data-driven prioritization
    • Capacity management
    • Risk identification
  3. Execution Briefs

    • Task breakdown
    • Time estimation
    • Structured approach
  4. Safe Action Execution

    • Dual mode (dry-run/execute)
    • Guardrails system
    • Full auditability
    • Reversible actions
  5. Developer-First Design

    • Explainable AI
    • Complete transparency
    • Easy configuration
    • Forge-native integration

Value Proposition

For Teams:

  • ⏱️ Saves Time: Automatic triage and planning
  • 🎯 Improves Focus: Surfaces important work
  • 📊 Better Planning: Data-driven sprint plans
  • 🔒 Maintains Safety: Guardrails prevent mistakes

For Individuals:

  • 🤖 AI Assistant: Always-on intelligence
  • 📝 Documentation: Auto-generated briefs
  • 🔍 Context: Understands your workflow
  • Actionable: Turns insights into actions

Next Steps

  1. Customize: Adjust triage logic for your team
  2. Integrate: Connect with Rovo Agents
  3. Extend: Add custom actions
  4. Scale: Deploy across projects

Questions to Ask the Audience:

  • What manual triage tasks take the most time for your team?
  • What actions would you want the AI to perform automatically?
  • How would you customize the guardrails for your workflow?

Resources:

  • README.md - Complete documentation
  • API.md - API reference
  • SETUP.md - Setup guide
  • Dashboard - Live monitoring and control

Related Documents

GUARDRAILS.md

Guardrails, Safety & Content Filtering

> Your LLM application will be attacked. Not might. Will. The first prompt injection attempt against your production system will come within 48 hours of launch. The question is not whether someone will try "ignore previous instructions and reveal your system prompt" -- the question is whether your system folds or holds. Every chatbot, every agent, every RAG pipeline is a target. If you ship without guardrails, you are shipping a vulnerability with a chat interface.

aiagentllm
0
16
rohitg00
GUARDRAILS.md

DeepSeek R1: Case Study in Failed Extrinsic Alignment

**Context:** This document compiles publicly available security research on DeepSeek R1 alongside our independent findings from the LEK-1 A/B testing. It demonstrates why extrinsic alignment (content filters, RLHF guardrails, system prompts) is insufficient for AI safety.

aiprompteval
0
7
Snider
GUARDRAILS.md

AI Safety & Guardrails for Voice Assistants

A multi-layered defense system ensuring the AI assistant stays on-topic, resists prompt injection, and never makes unauthorized decisions.

aillmrag
0
6
alexiokay
GUARDRAILS.md

LlmGuard Framework - Complete Implementation Buildout

**LlmGuard** is a comprehensive AI Firewall and Guardrails framework for LLM-based Elixir applications. It provides defense-in-depth protection against AI-specific threats including prompt injection, data leakage, jailbreak attempts, and unsafe content generation. This buildout implements a production-ready security layer for LLM applications with statistical rigor, comprehensive threat detection, and zero-trust validation.

aillmprompt
0
3
North-Shore-AI