Business Intelligence Dashboard - Comprehensive GitHub Copilot Instructions

ALWAYS follow these instructions first and fallback to additional search and context gathering only if the information in these instructions is incomplete or found to be in error.

Project Overview

The Business Intelligence Dashboard is a comprehensive SaaS Analytics Platform built with Next.js 14, TypeScript, PostgreSQL, and Prisma. It provides real-time data visualization, multi-tenant architecture, and enterprise-grade monitoring capabilities for organizations of all sizes.

Core Capabilities

• Real-time Data Visualization: Interactive charts, graphs, and KPI displays using Chart.js, D3.js, and Recharts
• Multi-tenant Architecture: Secure data isolation for multiple organizations with tenant-aware queries
• Advanced Analytics: Statistical analysis, predictive modeling, and trend analysis
• Custom Dashboards: Drag-and-drop dashboard builder with customizable widgets
• Enterprise Security: Role-based access control, SSO integration, and comprehensive data encryption
• API-First Design: RESTful APIs with comprehensive OpenAPI/Swagger documentation
• Performance Monitoring: Integrated New Relic APM with real-time alerting

Technology Stack & Architecture

Frontend Architecture

• Framework: Next.js 14 with App Router and TypeScript 5+
• UI Library: React 18 with modern hooks pattern
• Styling: Tailwind CSS 3+ with Headless UI components
• State Management: Zustand for lightweight, performant state management
• Data Visualization: Chart.js, D3.js, and Recharts for interactive data displays
• Authentication: NextAuth.js with multi-provider support (OAuth2, SAML)
• Forms: React Hook Form with Zod validation schemas

Backend Architecture

• Runtime: Node.js 20+ with Express.js framework via Next.js API routes
• Database: PostgreSQL 15+ with Prisma ORM for type-safe database access
• Caching: Redis 7+ for session storage, query caching, and real-time features
• API Design: RESTful APIs following OpenAPI 3.0 specifications
• Authentication: JWT tokens with refresh token rotation and secure cookie storage
• Security: CORS, CSRF protection, rate limiting, input validation, and SQL injection prevention
• File Storage: AWS S3 / Azure Blob Storage for document and image management

Infrastructure & DevOps

• Monitoring: New Relic APM for application performance monitoring and alerting
• Logging: Structured logging with Winston and ELK stack integration
• Containerization: Docker with optimized multi-stage builds
• Orchestration: Kubernetes for production deployments with auto-scaling
• CI/CD: GitHub Actions for automated testing, security scanning, and deployment
• Cloud Platforms: Multi-cloud support (AWS, Azure, GCP) with Infrastructure as Code
• CDN: Cloudflare/CloudFront for global content delivery optimization

Development Tools

• Package Manager: npm (primary) with yarn compatibility
• Code Quality: ESLint with strict TypeScript rules, Prettier for formatting
• Git Hooks: Husky for pre-commit hooks with lint-staged
• Testing: Jest, React Testing Library, Playwright for comprehensive test coverage
• Documentation: TypeDoc, Storybook for component library documentation
• API Testing: Postman collections with automated API testing

Critical Setup & Build Instructions

Prerequisites Validation

• Node.js: Version 20+ required (currently validated with 20.19.5)
• npm: Version 9+ required (currently validated with 10.8.2)
• PostgreSQL: Version 15+ required (currently validated with 16.10)
• Redis: Version 7+ required for session management and caching
• System Memory: 8GB RAM minimum, 16GB recommended for optimal performance

Environment Setup

# 1. Copy environment template (REQUIRED)
cp .env.example .env.local

# 2. Install dependencies (~25 seconds)
npm install

# 3. Start PostgreSQL and Redis services
sudo systemctl start postgresql redis

# 4. Create database and user
sudo -u postgres createdb bi_dashboard_dev
sudo -u postgres psql -c "ALTER USER postgres PASSWORD 'password';"

# 5. Update .env.local with correct DATABASE_URL and Redis configuration
# DATABASE_URL=postgresql://postgres:password@localhost:5432/bi_dashboard_dev
# REDIS_URL=redis://localhost:6379

Database Setup Commands

# Set environment variables for Prisma
export DATABASE_URL=postgresql://postgres:password@localhost:5432/bi_dashboard_dev

# Generate Prisma client (~1 second)
npm run db:generate

# Run database migrations (~2 seconds)
npm run db:migrate

# Seed database with comprehensive sample data (~3 seconds)
npm run db:seed

Login Credentials After Seeding:

• Admin: admin@example.com / admin123
• Demo: demo@example.com / demo123
• Viewer: viewer@example.com / viewer123

Build & Development Commands

Development Server

# Start development server with all required services
export DATABASE_URL=postgresql://postgres:password@localhost:5432/bi_dashboard_dev
npm run dev
# Server starts on http://localhost:3000
# Ready in ~1.3 seconds with hot reloading enabled

Build Process

# Production build (~25 seconds) - NEVER CANCEL
# Set timeout to 60+ minutes for safety on slower systems
export DATABASE_URL=postgresql://postgres:password@localhost:5432/bi_dashboard_dev
npm run build

⚠️ CRITICAL: NEVER CANCEL BUILD COMMANDS. Build takes ~25 seconds but may take longer on slower systems. Always set timeout to 60+ minutes.

Testing Commands

Unit Tests

# Run unit tests (~2 seconds) - NEVER CANCEL
export DATABASE_URL=postgresql://postgres:password@localhost:5432/bi_dashboard_dev
npm test
# Set timeout to 30+ minutes for safety

Integration Tests

# Run integration tests with database
npm run test:integration

End-to-End Tests

# Install Playwright browsers first (required once)
npx playwright install

# Run e2e tests (requires dev server to be stopped first)
npm run test:e2e
# Note: Will start its own dev server automatically
# Alternative: npm run test:e2e:headed for headed mode

Test Coverage

# Run tests with comprehensive coverage report
npm run test:coverage
# Coverage reports generated in coverage/ directory

Code Quality Commands

Linting

# Run linting (~2 seconds)
npm run lint

# Fix linting issues automatically
npm run lint:fix

Type Checking

# Run TypeScript type checking (~4 seconds)
npm run type-check
# Note: May show test file type errors - this is expected and documented

Code Formatting

# Check formatting
npm run format:check
# Note: May fail due to missing @tailwindcss/forms dependency

# Fix formatting automatically
npm run format

Multi-Tenant Security Architecture

CRITICAL: This application implements enterprise-grade multi-tenant architecture. Always ensure:

Tenant Isolation Requirements

1. Database Isolation: All database queries MUST include tenant ID filtering
2. Authentication: Verify user authentication on all protected routes
3. Authorization: Check user permissions before any data access
4. Data Validation: Validate all inputs to prevent injection attacks
5. API Security: Implement tenant-aware middleware for all API endpoints

Security Implementation Patterns

Tenant-Aware Database Queries

// ALWAYS include tenant isolation in queries
const dashboard = await prisma.dashboard.findUnique({
  where: { 
    id: dashboardId, 
    tenantId: user.tenantId  // REQUIRED: tenant isolation
  },
  include: {
    widgets: {
      where: { tenantId: user.tenantId }, // Nested tenant filtering
      include: {
        dataSource: {
          where: { tenantId: user.tenantId }
        }
      }
    }
  }
});

API Route Protection

// Implement tenant-aware middleware
export async function GET(request: Request, { params }: { params: { id: string } }) {
  const session = await getServerSession(authOptions);
  if (!session?.user?.tenantId) {
    return NextResponse.json({ error: 'Unauthorized' }, { status: 401 });
  }

  // All data access must include tenant filtering
  const data = await prisma.someModel.findMany({
    where: { tenantId: session.user.tenantId }
  });

  return NextResponse.json({ data });
}

Development Workflow & Best Practices

Branch Strategy & Git Workflow

• Feature Branches: Use descriptive feature branch names (feature/dashboard-widgets)
• Conventional Commits: Follow conventional commit format for clear history
• Code Reviews: All changes require peer review before merging
• Branch Protection: Main branch requires status checks and reviews
• Automatic Updates: Renovate bot manages dependency updates

Code Quality Standards

TypeScript Guidelines

• Strict Configuration: Use strict: true with comprehensive type checking
• Interface Preference: Use interfaces over types for object shapes
• Generic Types: Implement reusable generic types for components
• Error Handling: Use custom error classes with proper type discrimination
• Null Safety: Implement proper null/undefined checking throughout

React/Next.js Patterns

// Use functional components with proper TypeScript typing
interface DashboardProps {
  dashboardId: string;
  tenantId: string;
}

export default function Dashboard({ dashboardId, tenantId }: DashboardProps) {
  // Implement proper error boundaries and loading states
  return (
    <Suspense fallback={<DashboardSkeleton />}>
      <ErrorBoundary fallback={<DashboardError />}>
        <DashboardContent dashboardId={dashboardId} tenantId={tenantId} />
      </ErrorBoundary>
    </Suspense>
  );
}

Database & API Patterns

// Use Prisma for type-safe database operations
export async function createDashboard(data: CreateDashboardInput, tenantId: string) {
  return await prisma.dashboard.create({
    data: {
      ...data,
      tenantId, // Always include tenant isolation
      createdAt: new Date(),
      updatedAt: new Date()
    },
    include: {
      widgets: true,
      user: {
        select: { id: true, name: true, email: true }
      }
    }
  });
}

Comprehensive Testing Strategy

Test Coverage Requirements

• Unit Tests: 90% coverage for business logic and utilities
• Integration Tests: 80% coverage for API endpoints and database operations
• E2E Tests: Cover all critical user journeys and workflows
• Performance Tests: Validate response times and load handling
• Security Tests: Test tenant isolation and access controls

Testing Tools & Configuration

• Jest: Unit and integration test runner with comprehensive matchers
• React Testing Library: Component testing with user-centric approach
• Playwright: End-to-end testing with multi-browser support
• MSW (Mock Service Worker): API mocking for reliable test isolation
• Supertest: HTTP integration testing for API endpoints

Test Organization & Patterns

tests/
├── unit/                 # Unit tests for components and utilities
│   ├── components/       # React component tests
│   ├── lib/             # Utility function tests
│   └── api/             # API route unit tests
├── integration/          # Database and API integration tests
│   ├── api/             # Full API workflow tests
│   └── database/        # Database operation tests
├── e2e/                 # End-to-end user journey tests
│   ├── auth/            # Authentication flow tests
│   ├── dashboard/       # Dashboard functionality tests
│   └── admin/           # Admin panel tests
├── performance/          # Load and performance tests
└── fixtures/            # Test data and mock factories

Writing Effective Tests

// Follow AAA pattern (Arrange, Act, Assert)
describe('Dashboard API', () => {
  describe('GET /api/dashboards', () => {
    it('should return tenant-specific dashboards for authenticated users', async () => {
      // Arrange
      const tenant = await createTestTenant();
      const user = await createTestUser({ tenantId: tenant.id });
      const dashboard = await createTestDashboard({ tenantId: tenant.id });
      
      // Act
      const response = await request(app)
        .get('/api/dashboards')
        .set('Authorization', `Bearer ${user.token}`);
      
      // Assert
      expect(response.status).toBe(200);
      expect(response.body.data).toHaveLength(1);
      expect(response.body.data[0].id).toBe(dashboard.id);
      expect(response.body.data[0].tenantId).toBe(tenant.id);
    });
  });
});

Performance Optimization Guidelines

Frontend Performance

• Code Splitting: Implement route-based and component-based code splitting
• Bundle Optimization: Use tree shaking and proper import strategies
• Memoization: Implement React.memo and useMemo for expensive calculations
• Virtual Scrolling: Handle large datasets with virtual scrolling libraries
• Image Optimization: Use Next.js Image component with proper sizing
• Lazy Loading: Implement lazy loading for components and resources

Backend Performance

• Database Optimization: Use proper indexing and query optimization
• Caching Strategy: Implement Redis caching with proper invalidation
• Connection Pooling: Configure proper database connection pooling
• API Response Caching: Cache API responses with appropriate TTL
• Rate Limiting: Implement rate limiting to prevent abuse and ensure fairness

Real-time Features

// WebSocket implementation for real-time updates
export class DashboardWebSocketService {
  private io: Server;

  constructor(server: HttpServer) {
    this.io = new Server(server);
    this.setupTenantAwareConnections();
  }

  private setupTenantAwareConnections() {
    this.io.on('connection', async (socket) => {
      const token = socket.handshake.auth.token;
      const user = await validateSocketToken(token);
      
      if (!user) {
        socket.disconnect();
        return;
      }

      // Join tenant-specific room for data isolation
      socket.join(`tenant:${user.tenantId}`);
      
      socket.on('subscribe:dashboard', (dashboardId) => {
        socket.join(`dashboard:${dashboardId}:${user.tenantId}`);
      });
    });
  }
}

Deployment & Operations

Environment Configuration

# Production environment variables
NODE_ENV=production
DATABASE_URL=postgresql://user:pass@prod-db:5432/bi_dashboard
REDIS_URL=redis://prod-redis:6379
NEXTAUTH_SECRET=your-super-secure-secret-key
NEXTAUTH_URL=https://yourdomain.com

# Monitoring and observability
NEW_RELIC_LICENSE_KEY=your-new-relic-license-key
NEW_RELIC_APP_NAME=BI-Dashboard-Production

# Security configuration
CORS_ORIGINS=https://yourdomain.com,https://admin.yourdomain.com
RATE_LIMIT_WINDOW_MS=900000
RATE_LIMIT_MAX_REQUESTS=100

# Feature flags
ENABLE_ANALYTICS_AI=true
ENABLE_WHITE_LABEL=false
ENABLE_ADVANCED_SECURITY=true

Docker Deployment

# Multi-stage production Dockerfile
FROM node:20-alpine AS base
RUN apk add --no-cache libc6-compat
WORKDIR /app
COPY package*.json ./

FROM base AS deps
RUN npm ci --only=production && npm cache clean --force

FROM base AS build
RUN npm ci
COPY . .
RUN npm run build

FROM base AS runner
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
COPY --from=deps /app/node_modules ./node_modules
COPY --from=build /app/.next ./.next
COPY --from=build /app/public ./public
COPY --from=build /app/package.json ./package.json

USER nextjs
EXPOSE 3000
ENV PORT 3000
CMD ["npm", "start"]

Monitoring & Observability

// New Relic configuration
exports.config = {
  app_name: ['BI Dashboard Production'],
  license_key: process.env.NEW_RELIC_LICENSE_KEY,
  logging: {
    level: 'info'
  },
  allow_all_headers: true,
  attributes: {
    exclude: [
      'request.headers.cookie',
      'request.headers.authorization',
      'response.headers.cookie'
    ]
  }
};

Critical Timing & Timeout Information

⚠️ NEVER CANCEL ANY OF THESE COMMANDS:

Command	Typical Time	Recommended Timeout	Notes
npm install	25 seconds	300 seconds	Downloads ~1463 packages
npm run build	25 seconds	3600 seconds	NEVER CANCEL - May take longer
npm test	2 seconds	1800 seconds	NEVER CANCEL - Unit tests
npm run test:integration	10 seconds	1800 seconds	NEVER CANCEL - Database tests
npm run test:e2e	Variable	1800 seconds	NEVER CANCEL - E2E tests
npm run db:migrate	2 seconds	120 seconds	Database operations
npm run db:seed	3 seconds	120 seconds	Sample data creation
npm run lint	2 seconds	300 seconds	Code linting
npm run type-check	4 seconds	300 seconds	TypeScript validation

Validation Scenarios

ALWAYS test these scenarios after making changes:

1. Complete Application Flow

# 1. Start development server
export DATABASE_URL=postgresql://postgres:password@localhost:5432/bi_dashboard_dev
npm run dev

# 2. Navigate to http://localhost:3000
# 3. Should redirect to /auth/signin
# 4. Login with admin@example.com / admin123
# 5. Should redirect to /dashboard
# 6. Verify navigation works (Dashboards, Data Sources, Analytics)
# 7. Test dashboard creation and viewing
# 8. Verify multi-tenant data isolation

2. Build & Production Validation

# 1. Run full production build
npm run build

# 2. Start production server
npm start

# 3. Test application functionality in production mode
# 4. Verify all routes work correctly
# 5. Check performance metrics and loading times

3. Database & Security Testing

# Test database connectivity and migrations
npm run db:status
npm run db:migrate

# Test multi-tenant security
# - Login with different tenant users
# - Verify data isolation between tenants
# - Test unauthorized access attempts

Known Issues & Workarounds

1. Prettier/Tailwind Configuration

• npm run format:check may fail due to missing @tailwindcss/forms dependency
• Workaround: This doesn't affect application functionality
• Individual file formatting with prettier works correctly

2. Type Checking Test Files

• npm run type-check may show errors in test files for Jest globals
• Workaround: Tests run correctly despite TypeScript errors
• This is a configuration issue, not a functional problem

3. E2E Test Server Conflicts

• E2E tests fail if dev server is already running on port 3000
• Solution: Stop dev server before running e2e tests, or configure reuseExistingServer: true

4. Database Connection Issues

• PostgreSQL may require password configuration in some environments
• Solution: Set postgres user password as documented in setup commands

Common Development Tasks

Adding New Features

# 1. Create feature branch
git checkout -b feature/new-feature-name

# 2. Update database schema if needed
# Edit prisma/schema.prisma
npm run db:migrate

# 3. Generate TypeScript types
npm run db:generate

# 4. Implement feature with proper testing
# Add tests in tests/ directory
npm test

# 5. Update documentation
# Update relevant .md files in docs/

Creating API Endpoints

// src/app/api/dashboards/route.ts
import { NextRequest, NextResponse } from 'next/server';
import { getServerSession } from 'next-auth';
import { authOptions } from '@/lib/auth';
import { prisma } from '@/lib/database';
import { createDashboardSchema } from '@/lib/validations';

export async function POST(request: NextRequest) {
  try {
    const session = await getServerSession(authOptions);
    if (!session?.user) {
      return NextResponse.json({ error: 'Unauthorized' }, { status: 401 });
    }

    const body = await request.json();
    const validatedData = createDashboardSchema.parse(body);

    const dashboard = await prisma.dashboard.create({
      data: {
        ...validatedData,
        tenantId: session.user.tenantId,
        userId: session.user.id
      }
    });

    return NextResponse.json({ data: dashboard }, { status: 201 });
  } catch (error) {
    console.error('Dashboard creation error:', error);
    return NextResponse.json(
      { error: 'Internal server error' },
      { status: 500 }
    );
  }
}

CI/CD Pipeline Requirements

Pre-commit Validation

# Required checks before committing
npm run lint          # Code linting
npm run type-check     # TypeScript validation  
npm test              # Unit tests
npm run test:integration  # Integration tests
npm run build         # Build verification

GitHub Actions Workflow

name: CI/CD Pipeline
on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main, develop]

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: postgres
        options: --health-cmd pg_isready --health-interval 10s
      redis:
        image: redis:7
        options: --health-cmd "redis-cli ping" --health-interval 10s

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      
      - run: npm ci
      - run: npm run lint
      - run: npm run type-check
      - run: npm run test:ci
      - run: npm run build

Emergency Procedures

Reset Development Environment

# Complete environment reset
# 1. Stop all servers
pkill -f "next dev"

# 2. Reset database
npm run db:migrate:reset

# 3. Clean install
rm -rf node_modules package-lock.json .next
npm install

# 4. Rebuild and restart
npm run build
npm run dev

Debug Performance Issues

# Monitor application performance
npm run analyze        # Bundle analysis
npm run lighthouse     # Performance auditing

# Database performance
npm run db:studio      # Visual database exploration

# Application monitoring
# Check New Relic dashboard for performance metrics

Project Structure (Detailed)

business-intelligence-dashboard/
├── .github/                    # GitHub Actions and configuration
│   ├── workflows/             # CI/CD pipeline definitions
│   └── copilot-instructions.md # GitHub Copilot configuration
├── .next/                     # Next.js build output (auto-generated)
├── docs/                      # Project documentation
│   ├── api.md                # API documentation
│   ├── deployment.md         # Deployment guides
│   ├── testing.md            # Testing guidelines
│   └── troubleshooting.md    # Common issues and solutions
├── prisma/                    # Database configuration
│   ├── migrations/           # Database migration files
│   ├── schema.prisma         # Database schema definition
│   └── seed.ts              # Database seeding script
├── src/                       # Source code
│   ├── app/                  # Next.js App Router pages
│   │   ├── api/             # API routes
│   │   ├── auth/            # Authentication pages
│   │   ├── dashboard/       # Dashboard pages
│   │   └── globals.css      # Global styles
│   ├── components/           # Reusable UI components
│   │   ├── ui/              # Basic UI components
│   │   ├── charts/          # Chart components
│   │   └── dashboard/       # Dashboard-specific components
│   ├── lib/                  # Utility functions and configurations
│   │   ├── auth.ts          # Authentication configuration
│   │   ├── database.ts      # Database connection
│   │   ├── redis.ts         # Redis configuration
│   │   └── validations.ts   # Zod validation schemas
│   └── types/                # TypeScript type definitions
│       ├── auth.ts          # Authentication types
│       ├── dashboard.ts     # Dashboard types
│       └── api.ts           # API response types
├── tests/                     # Test files
│   ├── unit/                # Unit tests
│   ├── integration/         # Integration tests
│   ├── e2e/                 # End-to-end tests
│   └── fixtures/            # Test data and mocks
├── .env.example               # Environment variables template
├── .env.local                # Local environment variables (git-ignored)
├── docker-compose.yml         # Development services
├── Dockerfile                 # Production container build
├── jest.config.js            # Jest testing configuration
├── next.config.js            # Next.js configuration
├── package.json              # Dependencies and scripts
├── playwright.config.ts      # Playwright E2E configuration
├── prisma.yml                # Database configuration
└── tailwind.config.js        # Tailwind CSS configuration

Success Validation Checklist

After any changes, verify:

• Application builds successfully (npm run build)
• All tests pass (npm test && npm run test:integration)
• Development server starts without errors (npm run dev)
• Login functionality works with all user roles
• Dashboard displays correctly with sample data
• Navigation between sections works smoothly
• Database operations complete successfully
• Multi-tenant data isolation is maintained
• API endpoints return proper responses
• Performance metrics meet SLA requirements
• Security scanning shows no critical issues
• No console errors in browser developer tools

Project-Specific Context & Priorities

Current Status

1. Established Foundation: Core architecture implemented with Next.js 14, TypeScript, and PostgreSQL
2. Multi-tenant Security: Comprehensive tenant isolation implemented throughout the application
3. Testing Infrastructure: Complete testing setup with Jest, RTL, and Playwright
4. Monitoring: New Relic APM integration for performance monitoring
5. Documentation: Comprehensive documentation for development and deployment

Immediate Development Priorities

1. Performance Optimization: Improve dashboard loading times and query performance
2. Enhanced Analytics: Implement advanced analytics features and AI-powered insights
3. Security Hardening: Complete security audit and implement additional security measures
4. Scalability: Optimize for high-traffic scenarios and large datasets
5. User Experience: Enhance UI/UX based on user feedback and analytics

Long-term Strategic Goals

1. AI Integration: Implement machine learning models for predictive analytics
2. White-label Solutions: Develop customizable branding for reseller partners
3. Advanced Connectors: Create connectors for popular business applications
4. Real-time Collaboration: Enable team collaboration features within dashboards
5. Enterprise Features: Implement advanced security, audit trails, and compliance features

Remember: This is an enterprise-grade business intelligence platform handling sensitive business data. Always prioritize security, performance, and reliability in all implementations. When in doubt, favor security and data integrity over convenience.