Carapace Technical Design Specification (SPEC)

1. Purpose

Carapace is a triage orchestration engine for large contribution queues. It detects similarity across pull requests and issues, selects canonical candidates, and routes noisy submissions out of maintainer focus views.

This specification defines a modular, extensible architecture that:

• Works as a live service and offline POC.
• Uses fast local-first embedding and similarity approaches.
• Supports low-pass filtering (labels, stale age, actor behavior, etc.).
• Can swap source systems (GitHub now, Jira or others later) without rewriting core logic.

2. Scope

In Scope

• Similarity detection for PRs and issues.
• Cluster formation and canonical candidate selection.
• Low-pass noise filtering before expensive scoring.
• Pluggable embeddings (local and API-backed).
• Extensible connector and hook system.
• Rule-driven routing actions (label, comment, quarantine queue).
• Offline replay mode and live webhook mode.

Out of Scope (Initial)

• Full autonomous merge/close decisions by default.
• Full code review replacement (reuse existing review signals).
• UI-heavy platform in v1 (GitHub-native views first).

3. System Overview

3.1 High-Level Components

1. Ingestion Layer

• Receives events from connectors (GitHub webhooks, batch importers, replay logs).

2. Normalization Layer

• Maps provider payloads into a canonical Carapace domain schema.

3. Low-Pass Filter Layer

• Early noise suppression to reduce compute and maintainer cognitive load.

4. Fingerprint + Embedding Layer

• Builds text, structural, and lineage fingerprints.
• Produces embeddings via pluggable provider abstraction.

5. Similarity + Clustering Layer

• Candidate retrieval + pair scoring + cluster assignment.

6. Canonical Selection Layer

• Scores candidates in each cluster and chooses canonical PR.

7. Action + Policy Layer

• Applies labels/comments/routes through connector sinks.

8. Storage Layer

• Relational store for entities/events/decisions.
• Vector and signature index for retrieval.

9. Hook/SDK Extension Layer

• Lifecycle hooks and plugin APIs for custom logic.

3.2 Deployment Modes

• Live mode: webhook-driven, near real-time updates.
• Offline mode: snapshot/event replay; identical core pipeline.

4. Architecture Requirements

Functional Requirements

• FR-1: Ingest PR/issue/update events from GitHub.
• FR-2: Support connector abstraction for alternate systems (Jira, GitLab, Bitbucket).
• FR-3: Apply configurable low-pass rules pre-fingerprint.
• FR-4: Generate hybrid fingerprints (text, structural, lineage, quality signals).
• FR-5: Compute similarity with scalable candidate retrieval.
• FR-6: Cluster related submissions incrementally.
• FR-7: Rank and mark canonical entry per cluster.
• FR-8: Emit routing decisions (labels/comments/queues) with confidence and reason traces.
• FR-9: Expose SDK hooks for custom features and decision overrides.
• FR-10: Operate in offline POC mode with deterministic replay.
• FR-11: Load repository-level configuration from .carapace.yaml at repository root.

Non-Functional Requirements

• NFR-1: Handle 10k+ open entities per repo/org.
• NFR-2: P95 online event processing under 3s excluding external API delays.
• NFR-3: Support horizontal worker scaling.
• NFR-4: Full auditability of decisions and feature contributions.
• NFR-5: Idempotent event processing and exactly-once effective actioning.
• NFR-6: Python services must use Pydantic models for boundary validation and typed contracts.

5. Domain Model

Core Entities

• SourceEntity: PR, issue, ticket, task.
• Fingerprint: normalized feature bundle for an entity.
• EmbeddingVector: dense semantic vector(s) by provider/model.
• SimilarityEdge: pairwise score with feature breakdown.
• Cluster: group of related entities.
• CanonicalDecision: winning entity + ranked alternatives.
• RoutingDecision: actions to apply (labels/comments/quarantine).
• PolicyRule: low-pass/decision rule with version.

Key Tables (Relational)

• entities
• entity_events
• fingerprints
• embeddings
• similarity_edges
• clusters
• canonical_decisions
• routing_decisions
• policy_rules
• connector_state (cursor/checkpoint state per provider)

6. Connector and SDK Design

6.1 Connector Interfaces

Define provider-specific connectors behind stable contracts:

SourceConnector
  - subscribe_events()
  - list_open_entities()
  - get_entity(id)
  - get_diff_or_change_set(id)
  - get_reviews_and_checks(id)

SinkConnector
  - apply_labels(id, labels[])
  - post_comment(id, body)
  - set_status(id, state, context)
  - route_to_queue(id, queue_key)

6.2 Hook Lifecycle

Hooks are invoked with immutable context and mutable decision envelope.

before_normalize
after_normalize
before_low_pass
after_low_pass
before_fingerprint
after_fingerprint
before_similarity
after_similarity
before_canonical
after_canonical
before_action
after_action
on_error

6.3 SDK Principles

• Provider-neutral schema.
• Versioned hook contracts.
• Deterministic replay support.
• Safe extension boundaries (timeouts + fallback behavior per hook).
• Language-agnostic API transport (HTTP/JSON now, gRPC optional).

7. Low-Pass Filter Design

7.1 Goals

• Drop or down-prioritize predictable noise early.
• Reduce pairwise comparison cost.
• Preserve recoverability and transparency.

7.2 Rule Types

1. Hard skip

• Ignore stale, invalid, wontfix, archived targets, bot-only formatting PRs.

2. Soft suppress

• De-prioritize low-signal entities into quarantine queues.

3. Priority boost

• Elevate labels like security, regression, release-blocker.

7.3 Rule Inputs

• Labels allowlist/denylist.
• Age windows (updated_at, stale duration).
• Actor type/trust tier (maintainer, external, automation).
• CI status stability.
• File/path filters (vendor/docs-only/config-only).
• Historical behavior (repeat low-signal patterns).

7.4 Output

• filter_state: pass, suppress, skip.
• filter_reason_codes: array of deterministic reason ids.
• priority_weight: numeric multiplier for downstream ranking.

7.5 Example Policy Snippet

low_pass:
  hard_skip_labels: ["invalid", "wontfix", "duplicate", "stale"]
  soft_suppress_labels: ["question", "discussion"]
  suppress_if:
    - condition: "is_docs_only && actor_is_new && ci_state == 'none'"
      reason: "LOW_SIGNAL_DOCS_ONLY"
  boost_if:
    - condition: "has_label('security') || has_label('regression')"
      weight: 1.5

7.6 Configuration Discovery and Precedence

• Primary repo config path: .carapace.yaml at repository root.
• Optional org/global defaults may be provided by service config.
• Precedence order: entity/runtime override > repo .carapace.yaml > org defaults > system defaults.
• Missing repo config must not fail processing; system defaults apply.
• Config schema validation must run at load time and produce actionable errors.

8. Fingerprinting and Feature Engineering

8.1 Text Features

• Title/body normalized tokens.
• Linked issue ids.
• Extracted intent phrases from templates.
• External reviewer summaries (CodeRabbit/Greptile) when present.

8.2 Structural Features

• File path set + module buckets.
• Hunk signatures (path + context + normalized token hashes).
• Churn metrics: files changed, additions, deletions.

8.3 Lineage Features

• Commit SHAs.
• Patch-id set (when available from cloned refs).
• Base branch/head branch ancestry.

8.4 Quality Signals

• CI status history.
• Review approvals/comments.
• External reviewer score/risk/test-gap.

9. Embedding Strategy

9.1 Requirements

• Local-first for speed/cost/privacy.
• High retrieval quality for intent-level similarity.
• API fallback for quality/capacity bursts.

9.2 Provider Abstraction

EmbeddingProvider
  - embed_texts(texts[], mode) -> vectors[]
  - model_id()
  - dimensions()
  - max_batch_size()
  - health_check()

9.3 Recommended Local Models

Primary (balanced accuracy/speed):

• BAAI/bge-m3 served via TEI/Infinity.

Fast path:

• nomic-embed-text-v1.5 or equivalent compact model.

High-accuracy local option:

• jina-embeddings-v3 class model where hardware allows.

9.4 API Option

• OpenAI-compatible embedding endpoint contract:
  • POST /v1/embeddings
  • request includes model, input
  • response normalized into provider abstraction.

9.5 Multi-Vector Strategy (Optional)

• Store two vectors per entity:
  • semantic_text_vector
  • review_summary_vector
• Use weighted late fusion in pair scoring.

9.6 Operational Tuning

• Batch size auto-tuning by latency target.
• Quantization support (int8/fp16) for local serving.
• Model cache warm-up on startup.

10. Similarity Engine

10.1 Candidate Retrieval (Stage 1)

• Inverted indices on module buckets, linked issue ids, label classes.
• MinHash LSH on diff shingles.
• SimHash on normalized text.

Returns top-K candidates per entity for Stage 2 scoring.

10.2 Pair Scoring (Stage 2)

Score uses weighted ensemble:

• Lineage score.
• Structure score.
• Semantic score.
• Shape penalty.

Reference formula:

S = 0.45*Lineage + 0.40*Structure + 0.15*Semantic - 0.10*SizePenalty

10.3 Edge Gating

• Strong edge if lineage overlap exceeds threshold.
• Medium edge if structure overlap + semantic alignment passes joint threshold.
• No edge otherwise.

10.4 Clustering

• Incremental union-find with strong/weak tiers.
• Weak edges require shared strong neighbor to avoid chain bridging.

11. Canonical Selection Engine

11.1 Scoring Features

• Cluster centrality (mean similarity).
• Coverage of cluster touched surface.
• CI health.
• External reviewer score.
• Approvals/review traction.
• Size penalty.
• Filter priority weight from low-pass layer.

11.2 Canonical Formula (Initial)

CanonScore =
  5.0*Coverage +
  4.0*Centrality +
  3.0*CI +
  2.0*ReviewerScore +
  1.5*Approvals +
  1.0*PriorityWeight -
  1.0*SizePenalty

11.3 Decision States

• canonical
• duplicate_of:<id>
• related_non_duplicate
• needs_human_tie_break

12. Actioning and Routing

Label Taxonomy (Baseline)

• triage/canonical
• triage/duplicate
• triage/related
• triage/quarantine
• triage/noise-suppressed
• triage/ready-human

Action Policies

• Default safe mode: label + comment only.
• Optional strict mode: hide/quarantine routing.
• Auto-close disabled by default; enable only with explicit policy.

13. APIs

Internal Service APIs

• POST /events/ingest
• POST /entities/backfill
• GET /config/effective/{repo}
• GET /clusters/{id}
• GET /entities/{id}/similar
• POST /decisions/recompute
• GET /health

SDK/Hook APIs

• register_hook(name, callback, timeout_ms)
• register_feature_extractor(name, extractor)
• register_connector(source|sink, impl)

14. Storage and Indexing

• SQLite is the default relational backend for v1 (single-file local and hosted deployment baseline).
• Storage access must go through backend interfaces so PostgreSQL can replace SQLite with minimal application-layer changes.
• PostgreSQL is the first planned production replacement backend for scale-out and multi-writer deployments.
• Vector index (pgvector, Qdrant, or FAISS-backed sidecar).
• Signature stores for MinHash/SimHash keys.
• Blob storage for raw event snapshots in replay mode.

15. Reliability, Security, and Compliance

• Idempotency keys per event delivery.
• Replay-safe action deduplication.
• Signed webhook verification.
• Principle-of-least-privilege connector scopes.
• PII-safe logs and retention controls.
• Full decision traceability for maintainer audits.

16. Performance Targets

• 10k open entities processed in < 30 min batch refresh.
• Incremental event update P95 < 3s.
• Candidate retrieval P95 < 150ms for top-K query.
• Pair scoring throughput >= 2k pairs/sec per worker (baseline target).

17. Observability

• Metrics:
  • ingest lag
  • filter drop/suppress rates
  • candidate retrieval recall proxy
  • cluster churn
  • canonical flip rate
  • action failure rate
• Structured logs with correlation ids.
• Decision explanation payloads persisted.

18. Rollout Plan

1. Phase A: Offline POC

• Snapshot ingest, clustering, canonical report output.

2. Phase B: Live Read-Only

• Webhooks, decisions visible, no action writes.

3. Phase C: Live Assistive

• Labels/comments + quarantine routing.

4. Phase D: Optimization

• Learned pair scorer, active-learning thresholds, connector expansion.

19. Semantic Commit Strategy

• Use semantic commit prefixes during implementation:
  • feat:, fix:, perf:, refactor:, docs:, test:, chore:
• Keep commits scoped to single subsystem changes.
• Final cleanup pass:
  • squash noisy interim commits only at release boundary if needed.

20. Open Technical Questions

• Which local embedding model meets target latency on available hardware?
• Should canonical formulas remain deterministic or adopt learned ranker in v2?
• Which vector backend is preferred for first production deployment?
• How aggressive should low-pass suppression be by default for new repos?

21. Implementation Standards

• Primary implementation language is Python.
• Use Pydantic BaseModel for all external payloads, connector DTOs, event envelopes, config schemas, and decision outputs.
• Enable strict validation defaults to catch schema drift early.
• Keep business logic separate from transport models for testability.
• Use typed model fixtures in tests to avoid unvalidated dict-based test inputs.