fix(tickets): Remove invalid source_email_address_id in PlatformEmailReceiver

PlatformEmailReceiver was setting source_email_address_id to a PlatformEmailAddress ID, but the Ticket model expects a TicketEmailAddress foreign key. This caused an integrity error that was rolling back the entire transaction (due to ATOMIC_REQUESTS=True), preventing tickets from being created.
2025-12-01 19:32:45 -05:00
parent c82c60a562
commit ee6cf2b802
2 changed files with 654 additions and 1 deletions
--- a/PLAN_MULTI_EMAIL_TICKETING.md
+++ b/PLAN_MULTI_EMAIL_TICKETING.md
@@ -0,0 +1,653 @@
+# Implementation Plan: Multi-Email Ticketing System
+
+## Executive Summary
+
+Add support for multiple email addresses per business in the ticketing system, with color-coded visual indicators and per-email IMAP/SMTP configuration.
+
+## Current System Analysis
+
+### Existing Components
+
+1. **Django Backend (`tickets` app)**
+   - `Ticket` model: Core ticket entity
+   - `TicketComment` model: Ticket responses
+   - `TicketEmailSettings` model: **Singleton** platform-wide email config
+   - `IncomingTicketEmail` model: Email audit log
+   - `TicketEmailReceiver` class: IMAP email fetching
+   - `TicketEmailService` class: SMTP email sending
+
+2. **Frontend**
+   - `Tickets.tsx`: Main ticket listing page
+   - `TicketModal.tsx`: Ticket detail modal
+   - `useTickets` hook: Fetch tickets
+   - `useTicketEmailSettings` hook: Manage email settings (singleton)
+   - `Settings.tsx`: Business settings page
+
+3. **Current Email Flow**
+   - Single email account configured platform-wide
+   - Emails matched to tickets by ID in subject/address
+   - Comments created from email replies
+   - New tickets created from unmatched emails
+
+## Requirements (from user clarification)
+
+1. **Per-Business Email Addresses**
+   - Each business provides their own email account(s) and credentials
+   - Multiple email addresses per business
+   - Each email has independent IMAP/SMTP settings
+
+2. **Email Address Properties**
+   - Display name (e.g., "Support", "Billing")
+   - Email address
+   - IMAP settings (host, port, username, password, SSL)
+   - SMTP settings (host, port, username, password, TLS/SSL)
+   - Color for visual identification (hex color code)
+   - Active/inactive status
+
+3. **Ticket Routing**
+   - Incoming emails matched to business by email address configuration
+   - Reply emails matched to existing tickets
+   - New emails create tickets for that business
+   - System attempts to match sender email to customer/staff in business
+
+4. **UI Requirements**
+   - Colored left border on ticket rows indicating source email
+   - Business settings page to manage email addresses
+   - Test connection buttons for IMAP/SMTP
+   - Email address selector when creating tickets manually
+
+## Implementation Plan
+
+### Phase 1: Django Backend Models
+
+#### 1.1 Create `TicketEmailAddress` Model
+
+**File:** `/home/poduck/Desktop/smoothschedule2/smoothschedule/tickets/models.py`
+
+```python
+class TicketEmailAddress(models.Model):
+    """
+    Per-business email address configuration for ticket management.
+    Each business can have multiple email addresses with their own settings.
+    """
+    tenant = models.ForeignKey(
+        Tenant,
+        on_delete=models.CASCADE,
+        related_name='ticket_email_addresses',
+        help_text="Business this email address belongs to"
+    )
+
+    # Display information
+    display_name = models.CharField(
+        max_length=100,
+        help_text="Display name (e.g., 'Support', 'Billing', 'Sales')"
+    )
+    email_address = models.EmailField(
+        help_text="Email address for sending/receiving tickets"
+    )
+    color = models.CharField(
+        max_length=7,
+        default='#3b82f6',
+        help_text="Hex color code for visual identification (e.g., #3b82f6)"
+    )
+
+    # IMAP settings (inbound)
+    imap_host = models.CharField(max_length=255)
+    imap_port = models.IntegerField(default=993)
+    imap_use_ssl = models.BooleanField(default=True)
+    imap_username = models.CharField(max_length=255)
+    imap_password = models.CharField(max_length=255)  # Encrypted in production
+    imap_folder = models.CharField(max_length=100, default='INBOX')
+
+    # SMTP settings (outbound)
+    smtp_host = models.CharField(max_length=255)
+    smtp_port = models.IntegerField(default=587)
+    smtp_use_tls = models.BooleanField(default=True)
+    smtp_use_ssl = models.BooleanField(default=False)
+    smtp_username = models.CharField(max_length=255)
+    smtp_password = models.CharField(max_length=255)  # Encrypted in production
+
+    # Status and tracking
+    is_active = models.BooleanField(
+        default=True,
+        help_text="Whether this email address is actively checked"
+    )
+    is_default = models.BooleanField(
+        default=False,
+        help_text="Default email for new tickets in this business"
+    )
+    last_check_at = models.DateTimeField(null=True, blank=True)
+    last_error = models.TextField(blank=True, default='')
+    emails_processed_count = models.IntegerField(default=0)
+
+    created_at = models.DateTimeField(auto_now_add=True)
+    updated_at = models.DateTimeField(auto_now=True)
+
+    class Meta:
+        ordering = ['-is_default', 'display_name']
+        unique_together = [['tenant', 'email_address']]
+        indexes = [
+            models.Index(fields=['tenant', 'is_active']),
+            models.Index(fields=['email_address']),
+        ]
+
+    def __str__(self):
+        return f"{self.display_name} <{self.email_address}> ({self.tenant.name})"
+
+    def save(self, *args, **kwargs):
+        # Ensure only one default per tenant
+        if self.is_default:
+            TicketEmailAddress.objects.filter(
+                tenant=self.tenant,
+                is_default=True
+            ).exclude(pk=self.pk).update(is_default=False)
+        super().save(*args, **kwargs)
+```
+
+#### 1.2 Update `Ticket` Model
+
+Add field to track which email address received/sent the ticket:
+
+```python
+class Ticket(models.Model):
+    # ... existing fields ...
+
+    source_email_address = models.ForeignKey(
+        'TicketEmailAddress',
+        on_delete=models.SET_NULL,
+        null=True,
+        blank=True,
+        related_name='tickets',
+        help_text="Email address this ticket was received from or sent to"
+    )
+```
+
+#### 1.3 Update `IncomingTicketEmail` Model
+
+Add field to track which email address received the email:
+
+```python
+class IncomingTicketEmail(models.Model):
+    # ... existing fields ...
+
+    email_address = models.ForeignKey(
+        'TicketEmailAddress',
+        on_delete=models.SET_NULL,
+        null=True,
+        blank=True,
+        related_name='incoming_emails',
+        help_text="Email address configuration that received this email"
+    )
+```
+
+### Phase 2: Django Backend Logic
+
+#### 2.1 Update Email Receiver
+
+**File:** `/home/poduck/Desktop/smoothschedule2/smoothschedule/tickets/email_receiver.py`
+
+- Modify `TicketEmailReceiver` to iterate through all active `TicketEmailAddress` objects
+- Connect to each email address's IMAP server
+- Process emails for each address
+- Associate processed tickets with the source email address
+
+#### 2.2 Update Email Sender
+
+**File:** `/home/poduck/Desktop/smoothschedule2/smoothschedule/tickets/email_notifications.py`
+
+- Modify `TicketEmailService` to use the ticket's `source_email_address` for sending
+- Fall back to business's default email address if none specified
+
+### Phase 3: Django Backend API
+
+#### 3.1 Create Serializers
+
+**File:** `/home/poduck/Desktop/smoothschedule2/smoothschedule/tickets/serializers.py`
+
+```python
+class TicketEmailAddressSerializer(serializers.ModelSerializer):
+    class Meta:
+        model = TicketEmailAddress
+        fields = [
+            'id', 'tenant', 'display_name', 'email_address', 'color',
+            'imap_host', 'imap_port', 'imap_use_ssl', 'imap_username',
+            'imap_password', 'imap_folder',
+            'smtp_host', 'smtp_port', 'smtp_use_tls', 'smtp_use_ssl',
+            'smtp_username', 'smtp_password',
+            'is_active', 'is_default', 'last_check_at', 'last_error',
+            'emails_processed_count', 'created_at', 'updated_at'
+        ]
+        read_only_fields = ['tenant', 'last_check_at', 'last_error',
+                            'emails_processed_count', 'created_at', 'updated_at']
+        extra_kwargs = {
+            'imap_password': {'write_only': True},
+            'smtp_password': {'write_only': True},
+        }
+
+class TicketEmailAddressListSerializer(serializers.ModelSerializer):
+    """Lightweight serializer without passwords"""
+    class Meta:
+        model = TicketEmailAddress
+        fields = [
+            'id', 'display_name', 'email_address', 'color',
+            'is_active', 'is_default', 'last_check_at',
+            'emails_processed_count'
+        ]
+```
+
+Update `TicketSerializer` to include email address:
+
+```python
+class TicketSerializer(serializers.ModelSerializer):
+    # ... existing fields ...
+    source_email_address = TicketEmailAddressListSerializer(read_only=True)
+```
+
+#### 3.2 Create ViewSet
+
+**File:** `/home/poduck/Desktop/smoothschedule2/smoothschedule/tickets/views.py`
+
+```python
+class TicketEmailAddressViewSet(viewsets.ModelViewSet):
+    """
+    ViewSet for managing ticket email addresses.
+    Only business owners and managers can manage email addresses.
+    """
+    serializer_class = TicketEmailAddressSerializer
+    permission_classes = [IsTenantUser]
+
+    def get_queryset(self):
+        user = self.request.user
+        # Business users see their own email addresses
+        if user.role in ['owner', 'manager', 'staff']:
+            return TicketEmailAddress.objects.filter(
+                tenant=user.tenant
+            )
+        # Platform users see all
+        elif user.role in ['superuser', 'platform_manager']:
+            return TicketEmailAddress.objects.all()
+        return TicketEmailAddress.objects.none()
+
+    def get_serializer_class(self):
+        if self.action == 'list':
+            return TicketEmailAddressListSerializer
+        return TicketEmailAddressSerializer
+
+    def perform_create(self, serializer):
+        # Automatically set tenant from current user
+        serializer.save(tenant=self.request.user.tenant)
+
+    @action(detail=True, methods=['post'])
+    def test_imap(self, request, pk=None):
+        """Test IMAP connection for this email address"""
+        email_address = self.get_object()
+        # Test IMAP connection logic
+        return Response({'status': 'success'})
+
+    @action(detail=True, methods=['post'])
+    def test_smtp(self, request, pk=None):
+        """Test SMTP connection for this email address"""
+        email_address = self.get_object()
+        # Test SMTP connection logic
+        return Response({'status': 'success'})
+
+    @action(detail=True, methods=['post'])
+    def fetch_now(self, request, pk=None):
+        """Manually trigger email fetch for this address"""
+        email_address = self.get_object()
+        # Trigger email fetch
+        return Response({'status': 'fetching'})
+```
+
+#### 3.3 Add URL Routes
+
+**File:** `/home/poduck/Desktop/smoothschedule2/smoothschedule/tickets/urls.py`
+
+```python
+router.register(r'email-addresses', views.TicketEmailAddressViewSet, basename='ticketemailaddress')
+```
+
+### Phase 4: Frontend - React Hooks
+
+#### 4.1 Create API Client Functions
+
+**File:** `/home/poduck/Desktop/smoothschedule2/frontend/src/api/ticketEmailAddresses.ts` (new file)
+
+```typescript
+export interface TicketEmailAddress {
+  id: number;
+  display_name: string;
+  email_address: string;
+  color: string;
+  imap_host: string;
+  imap_port: number;
+  imap_use_ssl: boolean;
+  imap_username: string;
+  imap_password?: string;
+  imap_folder: string;
+  smtp_host: string;
+  smtp_port: number;
+  smtp_use_tls: boolean;
+  smtp_use_ssl: boolean;
+  smtp_username: string;
+  smtp_password?: string;
+  is_active: boolean;
+  is_default: boolean;
+  last_check_at?: string;
+  last_error?: string;
+  emails_processed_count: number;
+  created_at: string;
+  updated_at: string;
+}
+
+export type TicketEmailAddressCreate = Omit<TicketEmailAddress, 'id' | 'last_check_at' | 'last_error' | 'emails_processed_count' | 'created_at' | 'updated_at'>;
+
+export const getTicketEmailAddresses = async (): Promise<TicketEmailAddress[]> => {
+  const response = await apiClient.get('/tickets/email-addresses/');
+  return response.data;
+};
+
+export const createTicketEmailAddress = async (data: TicketEmailAddressCreate): Promise<TicketEmailAddress> => {
+  const response = await apiClient.post('/tickets/email-addresses/', data);
+  return response.data;
+};
+
+export const updateTicketEmailAddress = async (id: number, data: Partial<TicketEmailAddressCreate>): Promise<TicketEmailAddress> => {
+  const response = await apiClient.patch(`/tickets/email-addresses/${id}/`, data);
+  return response.data;
+};
+
+export const deleteTicketEmailAddress = async (id: number): Promise<void> => {
+  await apiClient.delete(`/tickets/email-addresses/${id}/`);
+};
+
+export const testImapConnection = async (id: number): Promise<{ status: string; message?: string }> => {
+  const response = await apiClient.post(`/tickets/email-addresses/${id}/test_imap/`);
+  return response.data;
+};
+
+export const testSmtpConnection = async (id: number): Promise<{ status: string; message?: string }> => {
+  const response = await apiClient.post(`/tickets/email-addresses/${id}/test_smtp/`);
+  return response.data;
+};
+
+export const fetchEmailsNow = async (id: number): Promise<{ status: string }> => {
+  const response = await apiClient.post(`/tickets/email-addresses/${id}/fetch_now/`);
+  return response.data;
+};
+```
+
+#### 4.2 Create React Query Hooks
+
+**File:** `/home/poduck/Desktop/smoothschedule2/frontend/src/hooks/useTicketEmailAddresses.ts` (new file)
+
+```typescript
+import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
+import {
+  getTicketEmailAddresses,
+  createTicketEmailAddress,
+  updateTicketEmailAddress,
+  deleteTicketEmailAddress,
+  testImapConnection,
+  testSmtpConnection,
+  fetchEmailsNow,
+  TicketEmailAddress,
+  TicketEmailAddressCreate,
+} from '../api/ticketEmailAddresses';
+
+const QUERY_KEY = 'ticketEmailAddresses';
+
+export const useTicketEmailAddresses = () => {
+  return useQuery<TicketEmailAddress[]>({
+    queryKey: [QUERY_KEY],
+    queryFn: getTicketEmailAddresses,
+  });
+};
+
+export const useCreateTicketEmailAddress = () => {
+  const queryClient = useQueryClient();
+  return useMutation({
+    mutationFn: (data: TicketEmailAddressCreate) => createTicketEmailAddress(data),
+    onSuccess: () => {
+      queryClient.invalidateQueries({ queryKey: [QUERY_KEY] });
+    },
+  });
+};
+
+export const useUpdateTicketEmailAddress = () => {
+  const queryClient = useQueryClient();
+  return useMutation({
+    mutationFn: ({ id, data }: { id: number; data: Partial<TicketEmailAddressCreate> }) =>
+      updateTicketEmailAddress(id, data),
+    onSuccess: () => {
+      queryClient.invalidateQueries({ queryKey: [QUERY_KEY] });
+    },
+  });
+};
+
+export const useDeleteTicketEmailAddress = () => {
+  const queryClient = useQueryClient();
+  return useMutation({
+    mutationFn: (id: number) => deleteTicketEmailAddress(id),
+    onSuccess: () => {
+      queryClient.invalidateQueries({ queryKey: [QUERY_KEY] });
+    },
+  });
+};
+
+export const useTestImapConnection = () => {
+  return useMutation({
+    mutationFn: (id: number) => testImapConnection(id),
+  });
+};
+
+export const useTestSmtpConnection = () => {
+  return useMutation({
+    mutationFn: (id: number) => testSmtpConnection(id),
+  });
+};
+
+export const useFetchEmailsNow = () => {
+  return useMutation({
+    mutationFn: (id: number) => fetchEmailsNow(id),
+  });
+};
+```
+
+### Phase 5: Frontend - React Components
+
+#### 5.1 Email Address Management Component
+
+**File:** `/home/poduck/Desktop/smoothschedule2/frontend/src/components/TicketEmailAddressManager.tsx` (new file)
+
+Features:
+- List all email addresses for the business
+- Add new email address
+- Edit existing email address
+- Delete email address
+- Test IMAP/SMTP connections
+- Set default email address
+- Color picker for visual identification
+- Enable/disable email addresses
+
+#### 5.2 Update Ticket List UI
+
+**File:** `/home/poduck/Desktop/smoothschedule2/frontend/src/pages/Tickets.tsx`
+
+Modify ticket rows to include colored left border:
+
+```tsx
+<div
+  className="ticket-row"
+  style={{
+    borderLeft: ticket.source_email_address
+      ? `4px solid ${ticket.source_email_address.color}`
+      : '4px solid transparent'
+  }}
+>
+  {/* Ticket content */}
+</div>
+```
+
+#### 5.3 Update Types
+
+**File:** `/home/poduck/Desktop/smoothschedule2/frontend/src/types.ts`
+
+```typescript
+export interface TicketEmailAddress {
+  id: number;
+  display_name: string;
+  email_address: string;
+  color: string;
+  is_active: boolean;
+  is_default: boolean;
+  last_check_at?: string;
+  emails_processed_count: number;
+}
+
+export interface Ticket {
+  // ... existing fields ...
+  source_email_address?: TicketEmailAddress;
+}
+```
+
+#### 5.4 Add to Business Settings
+
+**File:** `/home/poduck/Desktop/smoothschedule2/frontend/src/pages/Settings.tsx`
+
+Add new tab for "Email Addresses" that renders `TicketEmailAddressManager` component.
+
+### Phase 6: Database Migration
+
+#### 6.1 Create Migration
+
+```bash
+cd /home/poduck/Desktop/smoothschedule2/smoothschedule
+docker compose -f docker-compose.local.yml exec django python manage.py makemigrations tickets
+```
+
+#### 6.2 Run Migration
+
+```bash
+docker compose -f docker-compose.local.yml exec django python manage.py migrate tickets
+```
+
+#### 6.3 Data Migration (if needed)
+
+If there's existing `TicketEmailSettings` data, create a data migration to convert it to `TicketEmailAddress` records for each tenant.
+
+### Phase 7: Testing
+
+#### 7.1 Backend Tests
+
+- Test email address CRUD operations
+- Test email receiver with multiple addresses
+- Test email sender using correct source address
+- Test tenant isolation
+
+#### 7.2 Frontend Tests
+
+- Test email address list rendering
+- Test add/edit/delete operations
+- Test connection testing UI
+- Test ticket list color borders
+
+### Phase 8: Documentation
+
+#### 8.1 User Documentation
+
+- How to add email addresses
+- How to configure IMAP/SMTP settings
+- How to test connections
+- Color coding explanation
+
+#### 8.2 Developer Documentation
+
+- API endpoints documentation
+- Model relationships
+- Email processing flow
+- Celery task schedule (if applicable)
+
+## Migration Strategy
+
+### Option 1: Keep Legacy System (Recommended)
+
+- Keep `TicketEmailSettings` for platform-level configuration
+- New `TicketEmailAddress` for per-business configuration
+- Businesses can opt-in to multi-email system
+- Existing single-email businesses continue working
+
+### Option 2: Full Migration
+
+- Deprecate `TicketEmailSettings`
+- Migrate all existing data to `TicketEmailAddress`
+- All businesses use new system
+
+**Recommendation:** Option 1 for backward compatibility
+
+## Risks & Considerations
+
+1. **Security**
+   - Email passwords stored in database (consider encryption)
+   - SMTP/IMAP credentials exposure risk
+   - Recommend OAuth2 for Gmail/Outlook in future
+
+2. **Performance**
+   - Multiple IMAP connections may increase load
+   - Consider Celery task queue for email fetching
+   - Implement rate limiting
+
+3. **Email Deliverability**
+   - Each business responsible for their own SPF/DKIM records
+   - No centralized email reputation management
+
+4. **UI/UX**
+   - Color picker needs to be user-friendly
+   - Color accessibility (contrast ratio)
+   - Mobile responsiveness
+
+## Future Enhancements
+
+1. **OAuth2 Support**
+   - Google Workspace integration
+   - Microsoft 365 integration
+
+2. **Email Templates Per Address**
+   - Different signatures per email address
+   - Custom auto-responses
+
+3. **Analytics**
+   - Email volume by address
+   - Response time by address
+
+4. **Auto-Assignment**
+   - Route tickets to specific staff based on email address
+
+## Implementation Timeline
+
+- **Phase 1-2 (Backend Models & Logic):** 2-3 days
+- **Phase 3 (Backend API):** 1-2 days
+- **Phase 4-5 (Frontend):** 3-4 days
+- **Phase 6-7 (Migration & Testing):** 1-2 days
+- **Phase 8 (Documentation):** 1 day
+
+**Total Estimated Time:** 8-12 days
+
+## Approval Required
+
+Before proceeding with implementation, please confirm:
+
+1. ✅ Per-business email addresses (not platform-wide)
+2. ✅ Businesses provide their own IMAP/SMTP credentials
+3. ✅ Colored left border for visual identification
+4. ✅ Email address management in business settings (not platform dashboard)
+5. ⚠️ Security approach for storing email passwords
+6. ⚠️ Migration strategy (keep legacy vs full migration)
+
+## Questions for Product Owner
+
+1. Should we encrypt email passwords in the database?
+2. Do we need email address approval workflow (platform admin approval)?
+3. Should there be a limit on number of email addresses per business?
+4. Do we need email forwarding (forward to another address)?
+5. Should unmatched emails (not tied to a ticket) create new tickets or be ignored?
--- a/smoothschedule/tickets/email_receiver.py
+++ b/smoothschedule/tickets/email_receiver.py
@@ -901,7 +901,7 @@ class PlatformEmailReceiver:
                    is_sandbox=False,
                    external_email=email_data['from_address'] if not user else None,
                    external_name=email_data['from_name'] if not user else '',
-                    source_email_address_id=self.email_address.id,
+                    # Note: source_email_address is for TicketEmailAddress, not PlatformEmailAddress
                )

                TicketComment.objects.create(