Files
smoothschedule/PLAN_MULTI_EMAIL_TICKETING.md
poduck ee6cf2b802 fix(tickets): Remove invalid source_email_address_id in PlatformEmailReceiver
PlatformEmailReceiver was setting source_email_address_id to a
PlatformEmailAddress ID, but the Ticket model expects a TicketEmailAddress
foreign key. This caused an integrity error that was rolling back the
entire transaction (due to ATOMIC_REQUESTS=True), preventing tickets
from being created.
2025-12-01 19:32:45 -05:00

20 KiB

Implementation Plan: Multi-Email Ticketing System

Executive Summary

Add support for multiple email addresses per business in the ticketing system, with color-coded visual indicators and per-email IMAP/SMTP configuration.

Current System Analysis

Existing Components

  1. Django Backend (tickets app)

    • Ticket model: Core ticket entity
    • TicketComment model: Ticket responses
    • TicketEmailSettings model: Singleton platform-wide email config
    • IncomingTicketEmail model: Email audit log
    • TicketEmailReceiver class: IMAP email fetching
    • TicketEmailService class: SMTP email sending
  2. Frontend

    • Tickets.tsx: Main ticket listing page
    • TicketModal.tsx: Ticket detail modal
    • useTickets hook: Fetch tickets
    • useTicketEmailSettings hook: Manage email settings (singleton)
    • Settings.tsx: Business settings page
  3. Current Email Flow

    • Single email account configured platform-wide
    • Emails matched to tickets by ID in subject/address
    • Comments created from email replies
    • New tickets created from unmatched emails

Requirements (from user clarification)

  1. Per-Business Email Addresses

    • Each business provides their own email account(s) and credentials
    • Multiple email addresses per business
    • Each email has independent IMAP/SMTP settings
  2. Email Address Properties

    • Display name (e.g., "Support", "Billing")
    • Email address
    • IMAP settings (host, port, username, password, SSL)
    • SMTP settings (host, port, username, password, TLS/SSL)
    • Color for visual identification (hex color code)
    • Active/inactive status
  3. Ticket Routing

    • Incoming emails matched to business by email address configuration
    • Reply emails matched to existing tickets
    • New emails create tickets for that business
    • System attempts to match sender email to customer/staff in business
  4. UI Requirements

    • Colored left border on ticket rows indicating source email
    • Business settings page to manage email addresses
    • Test connection buttons for IMAP/SMTP
    • Email address selector when creating tickets manually

Implementation Plan

Phase 1: Django Backend Models

1.1 Create TicketEmailAddress Model

File: /home/poduck/Desktop/smoothschedule2/smoothschedule/tickets/models.py

class TicketEmailAddress(models.Model):
    """
    Per-business email address configuration for ticket management.
    Each business can have multiple email addresses with their own settings.
    """
    tenant = models.ForeignKey(
        Tenant,
        on_delete=models.CASCADE,
        related_name='ticket_email_addresses',
        help_text="Business this email address belongs to"
    )

    # Display information
    display_name = models.CharField(
        max_length=100,
        help_text="Display name (e.g., 'Support', 'Billing', 'Sales')"
    )
    email_address = models.EmailField(
        help_text="Email address for sending/receiving tickets"
    )
    color = models.CharField(
        max_length=7,
        default='#3b82f6',
        help_text="Hex color code for visual identification (e.g., #3b82f6)"
    )

    # IMAP settings (inbound)
    imap_host = models.CharField(max_length=255)
    imap_port = models.IntegerField(default=993)
    imap_use_ssl = models.BooleanField(default=True)
    imap_username = models.CharField(max_length=255)
    imap_password = models.CharField(max_length=255)  # Encrypted in production
    imap_folder = models.CharField(max_length=100, default='INBOX')

    # SMTP settings (outbound)
    smtp_host = models.CharField(max_length=255)
    smtp_port = models.IntegerField(default=587)
    smtp_use_tls = models.BooleanField(default=True)
    smtp_use_ssl = models.BooleanField(default=False)
    smtp_username = models.CharField(max_length=255)
    smtp_password = models.CharField(max_length=255)  # Encrypted in production

    # Status and tracking
    is_active = models.BooleanField(
        default=True,
        help_text="Whether this email address is actively checked"
    )
    is_default = models.BooleanField(
        default=False,
        help_text="Default email for new tickets in this business"
    )
    last_check_at = models.DateTimeField(null=True, blank=True)
    last_error = models.TextField(blank=True, default='')
    emails_processed_count = models.IntegerField(default=0)

    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

    class Meta:
        ordering = ['-is_default', 'display_name']
        unique_together = [['tenant', 'email_address']]
        indexes = [
            models.Index(fields=['tenant', 'is_active']),
            models.Index(fields=['email_address']),
        ]

    def __str__(self):
        return f"{self.display_name} <{self.email_address}> ({self.tenant.name})"

    def save(self, *args, **kwargs):
        # Ensure only one default per tenant
        if self.is_default:
            TicketEmailAddress.objects.filter(
                tenant=self.tenant,
                is_default=True
            ).exclude(pk=self.pk).update(is_default=False)
        super().save(*args, **kwargs)

1.2 Update Ticket Model

Add field to track which email address received/sent the ticket:

class Ticket(models.Model):
    # ... existing fields ...

    source_email_address = models.ForeignKey(
        'TicketEmailAddress',
        on_delete=models.SET_NULL,
        null=True,
        blank=True,
        related_name='tickets',
        help_text="Email address this ticket was received from or sent to"
    )

1.3 Update IncomingTicketEmail Model

Add field to track which email address received the email:

class IncomingTicketEmail(models.Model):
    # ... existing fields ...

    email_address = models.ForeignKey(
        'TicketEmailAddress',
        on_delete=models.SET_NULL,
        null=True,
        blank=True,
        related_name='incoming_emails',
        help_text="Email address configuration that received this email"
    )

Phase 2: Django Backend Logic

2.1 Update Email Receiver

File: /home/poduck/Desktop/smoothschedule2/smoothschedule/tickets/email_receiver.py

  • Modify TicketEmailReceiver to iterate through all active TicketEmailAddress objects
  • Connect to each email address's IMAP server
  • Process emails for each address
  • Associate processed tickets with the source email address

2.2 Update Email Sender

File: /home/poduck/Desktop/smoothschedule2/smoothschedule/tickets/email_notifications.py

  • Modify TicketEmailService to use the ticket's source_email_address for sending
  • Fall back to business's default email address if none specified

Phase 3: Django Backend API

3.1 Create Serializers

File: /home/poduck/Desktop/smoothschedule2/smoothschedule/tickets/serializers.py

class TicketEmailAddressSerializer(serializers.ModelSerializer):
    class Meta:
        model = TicketEmailAddress
        fields = [
            'id', 'tenant', 'display_name', 'email_address', 'color',
            'imap_host', 'imap_port', 'imap_use_ssl', 'imap_username',
            'imap_password', 'imap_folder',
            'smtp_host', 'smtp_port', 'smtp_use_tls', 'smtp_use_ssl',
            'smtp_username', 'smtp_password',
            'is_active', 'is_default', 'last_check_at', 'last_error',
            'emails_processed_count', 'created_at', 'updated_at'
        ]
        read_only_fields = ['tenant', 'last_check_at', 'last_error',
                            'emails_processed_count', 'created_at', 'updated_at']
        extra_kwargs = {
            'imap_password': {'write_only': True},
            'smtp_password': {'write_only': True},
        }

class TicketEmailAddressListSerializer(serializers.ModelSerializer):
    """Lightweight serializer without passwords"""
    class Meta:
        model = TicketEmailAddress
        fields = [
            'id', 'display_name', 'email_address', 'color',
            'is_active', 'is_default', 'last_check_at',
            'emails_processed_count'
        ]

Update TicketSerializer to include email address:

class TicketSerializer(serializers.ModelSerializer):
    # ... existing fields ...
    source_email_address = TicketEmailAddressListSerializer(read_only=True)

3.2 Create ViewSet

File: /home/poduck/Desktop/smoothschedule2/smoothschedule/tickets/views.py

class TicketEmailAddressViewSet(viewsets.ModelViewSet):
    """
    ViewSet for managing ticket email addresses.
    Only business owners and managers can manage email addresses.
    """
    serializer_class = TicketEmailAddressSerializer
    permission_classes = [IsTenantUser]

    def get_queryset(self):
        user = self.request.user
        # Business users see their own email addresses
        if user.role in ['owner', 'manager', 'staff']:
            return TicketEmailAddress.objects.filter(
                tenant=user.tenant
            )
        # Platform users see all
        elif user.role in ['superuser', 'platform_manager']:
            return TicketEmailAddress.objects.all()
        return TicketEmailAddress.objects.none()

    def get_serializer_class(self):
        if self.action == 'list':
            return TicketEmailAddressListSerializer
        return TicketEmailAddressSerializer

    def perform_create(self, serializer):
        # Automatically set tenant from current user
        serializer.save(tenant=self.request.user.tenant)

    @action(detail=True, methods=['post'])
    def test_imap(self, request, pk=None):
        """Test IMAP connection for this email address"""
        email_address = self.get_object()
        # Test IMAP connection logic
        return Response({'status': 'success'})

    @action(detail=True, methods=['post'])
    def test_smtp(self, request, pk=None):
        """Test SMTP connection for this email address"""
        email_address = self.get_object()
        # Test SMTP connection logic
        return Response({'status': 'success'})

    @action(detail=True, methods=['post'])
    def fetch_now(self, request, pk=None):
        """Manually trigger email fetch for this address"""
        email_address = self.get_object()
        # Trigger email fetch
        return Response({'status': 'fetching'})

3.3 Add URL Routes

File: /home/poduck/Desktop/smoothschedule2/smoothschedule/tickets/urls.py

router.register(r'email-addresses', views.TicketEmailAddressViewSet, basename='ticketemailaddress')

Phase 4: Frontend - React Hooks

4.1 Create API Client Functions

File: /home/poduck/Desktop/smoothschedule2/frontend/src/api/ticketEmailAddresses.ts (new file)

export interface TicketEmailAddress {
  id: number;
  display_name: string;
  email_address: string;
  color: string;
  imap_host: string;
  imap_port: number;
  imap_use_ssl: boolean;
  imap_username: string;
  imap_password?: string;
  imap_folder: string;
  smtp_host: string;
  smtp_port: number;
  smtp_use_tls: boolean;
  smtp_use_ssl: boolean;
  smtp_username: string;
  smtp_password?: string;
  is_active: boolean;
  is_default: boolean;
  last_check_at?: string;
  last_error?: string;
  emails_processed_count: number;
  created_at: string;
  updated_at: string;
}

export type TicketEmailAddressCreate = Omit<TicketEmailAddress, 'id' | 'last_check_at' | 'last_error' | 'emails_processed_count' | 'created_at' | 'updated_at'>;

export const getTicketEmailAddresses = async (): Promise<TicketEmailAddress[]> => {
  const response = await apiClient.get('/tickets/email-addresses/');
  return response.data;
};

export const createTicketEmailAddress = async (data: TicketEmailAddressCreate): Promise<TicketEmailAddress> => {
  const response = await apiClient.post('/tickets/email-addresses/', data);
  return response.data;
};

export const updateTicketEmailAddress = async (id: number, data: Partial<TicketEmailAddressCreate>): Promise<TicketEmailAddress> => {
  const response = await apiClient.patch(`/tickets/email-addresses/${id}/`, data);
  return response.data;
};

export const deleteTicketEmailAddress = async (id: number): Promise<void> => {
  await apiClient.delete(`/tickets/email-addresses/${id}/`);
};

export const testImapConnection = async (id: number): Promise<{ status: string; message?: string }> => {
  const response = await apiClient.post(`/tickets/email-addresses/${id}/test_imap/`);
  return response.data;
};

export const testSmtpConnection = async (id: number): Promise<{ status: string; message?: string }> => {
  const response = await apiClient.post(`/tickets/email-addresses/${id}/test_smtp/`);
  return response.data;
};

export const fetchEmailsNow = async (id: number): Promise<{ status: string }> => {
  const response = await apiClient.post(`/tickets/email-addresses/${id}/fetch_now/`);
  return response.data;
};

4.2 Create React Query Hooks

File: /home/poduck/Desktop/smoothschedule2/frontend/src/hooks/useTicketEmailAddresses.ts (new file)

import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
import {
  getTicketEmailAddresses,
  createTicketEmailAddress,
  updateTicketEmailAddress,
  deleteTicketEmailAddress,
  testImapConnection,
  testSmtpConnection,
  fetchEmailsNow,
  TicketEmailAddress,
  TicketEmailAddressCreate,
} from '../api/ticketEmailAddresses';

const QUERY_KEY = 'ticketEmailAddresses';

export const useTicketEmailAddresses = () => {
  return useQuery<TicketEmailAddress[]>({
    queryKey: [QUERY_KEY],
    queryFn: getTicketEmailAddresses,
  });
};

export const useCreateTicketEmailAddress = () => {
  const queryClient = useQueryClient();
  return useMutation({
    mutationFn: (data: TicketEmailAddressCreate) => createTicketEmailAddress(data),
    onSuccess: () => {
      queryClient.invalidateQueries({ queryKey: [QUERY_KEY] });
    },
  });
};

export const useUpdateTicketEmailAddress = () => {
  const queryClient = useQueryClient();
  return useMutation({
    mutationFn: ({ id, data }: { id: number; data: Partial<TicketEmailAddressCreate> }) =>
      updateTicketEmailAddress(id, data),
    onSuccess: () => {
      queryClient.invalidateQueries({ queryKey: [QUERY_KEY] });
    },
  });
};

export const useDeleteTicketEmailAddress = () => {
  const queryClient = useQueryClient();
  return useMutation({
    mutationFn: (id: number) => deleteTicketEmailAddress(id),
    onSuccess: () => {
      queryClient.invalidateQueries({ queryKey: [QUERY_KEY] });
    },
  });
};

export const useTestImapConnection = () => {
  return useMutation({
    mutationFn: (id: number) => testImapConnection(id),
  });
};

export const useTestSmtpConnection = () => {
  return useMutation({
    mutationFn: (id: number) => testSmtpConnection(id),
  });
};

export const useFetchEmailsNow = () => {
  return useMutation({
    mutationFn: (id: number) => fetchEmailsNow(id),
  });
};

Phase 5: Frontend - React Components

5.1 Email Address Management Component

File: /home/poduck/Desktop/smoothschedule2/frontend/src/components/TicketEmailAddressManager.tsx (new file)

Features:

  • List all email addresses for the business
  • Add new email address
  • Edit existing email address
  • Delete email address
  • Test IMAP/SMTP connections
  • Set default email address
  • Color picker for visual identification
  • Enable/disable email addresses

5.2 Update Ticket List UI

File: /home/poduck/Desktop/smoothschedule2/frontend/src/pages/Tickets.tsx

Modify ticket rows to include colored left border:

<div
  className="ticket-row"
  style={{
    borderLeft: ticket.source_email_address
      ? `4px solid ${ticket.source_email_address.color}`
      : '4px solid transparent'
  }}
>
  {/* Ticket content */}
</div>

5.3 Update Types

File: /home/poduck/Desktop/smoothschedule2/frontend/src/types.ts

export interface TicketEmailAddress {
  id: number;
  display_name: string;
  email_address: string;
  color: string;
  is_active: boolean;
  is_default: boolean;
  last_check_at?: string;
  emails_processed_count: number;
}

export interface Ticket {
  // ... existing fields ...
  source_email_address?: TicketEmailAddress;
}

5.4 Add to Business Settings

File: /home/poduck/Desktop/smoothschedule2/frontend/src/pages/Settings.tsx

Add new tab for "Email Addresses" that renders TicketEmailAddressManager component.

Phase 6: Database Migration

6.1 Create Migration

cd /home/poduck/Desktop/smoothschedule2/smoothschedule
docker compose -f docker-compose.local.yml exec django python manage.py makemigrations tickets

6.2 Run Migration

docker compose -f docker-compose.local.yml exec django python manage.py migrate tickets

6.3 Data Migration (if needed)

If there's existing TicketEmailSettings data, create a data migration to convert it to TicketEmailAddress records for each tenant.

Phase 7: Testing

7.1 Backend Tests

  • Test email address CRUD operations
  • Test email receiver with multiple addresses
  • Test email sender using correct source address
  • Test tenant isolation

7.2 Frontend Tests

  • Test email address list rendering
  • Test add/edit/delete operations
  • Test connection testing UI
  • Test ticket list color borders

Phase 8: Documentation

8.1 User Documentation

  • How to add email addresses
  • How to configure IMAP/SMTP settings
  • How to test connections
  • Color coding explanation

8.2 Developer Documentation

  • API endpoints documentation
  • Model relationships
  • Email processing flow
  • Celery task schedule (if applicable)

Migration Strategy

  • Keep TicketEmailSettings for platform-level configuration
  • New TicketEmailAddress for per-business configuration
  • Businesses can opt-in to multi-email system
  • Existing single-email businesses continue working

Option 2: Full Migration

  • Deprecate TicketEmailSettings
  • Migrate all existing data to TicketEmailAddress
  • All businesses use new system

Recommendation: Option 1 for backward compatibility

Risks & Considerations

  1. Security

    • Email passwords stored in database (consider encryption)
    • SMTP/IMAP credentials exposure risk
    • Recommend OAuth2 for Gmail/Outlook in future
  2. Performance

    • Multiple IMAP connections may increase load
    • Consider Celery task queue for email fetching
    • Implement rate limiting
  3. Email Deliverability

    • Each business responsible for their own SPF/DKIM records
    • No centralized email reputation management
  4. UI/UX

    • Color picker needs to be user-friendly
    • Color accessibility (contrast ratio)
    • Mobile responsiveness

Future Enhancements

  1. OAuth2 Support

    • Google Workspace integration
    • Microsoft 365 integration
  2. Email Templates Per Address

    • Different signatures per email address
    • Custom auto-responses
  3. Analytics

    • Email volume by address
    • Response time by address
  4. Auto-Assignment

    • Route tickets to specific staff based on email address

Implementation Timeline

  • Phase 1-2 (Backend Models & Logic): 2-3 days
  • Phase 3 (Backend API): 1-2 days
  • Phase 4-5 (Frontend): 3-4 days
  • Phase 6-7 (Migration & Testing): 1-2 days
  • Phase 8 (Documentation): 1 day

Total Estimated Time: 8-12 days

Approval Required

Before proceeding with implementation, please confirm:

  1. Per-business email addresses (not platform-wide)
  2. Businesses provide their own IMAP/SMTP credentials
  3. Colored left border for visual identification
  4. Email address management in business settings (not platform dashboard)
  5. ⚠️ Security approach for storing email passwords
  6. ⚠️ Migration strategy (keep legacy vs full migration)

Questions for Product Owner

  1. Should we encrypt email passwords in the database?
  2. Do we need email address approval workflow (platform admin approval)?
  3. Should there be a limit on number of email addresses per business?
  4. Do we need email forwarding (forward to another address)?
  5. Should unmatched emails (not tied to a ticket) create new tickets or be ignored?