Skip to content
RightCapital Knowledge Base

DITS Roadmap

DITS Roadmap

Section titled “DITS Roadmap”

DITS (Data Issue Tracking System) is a strategic initiative to transform how Support and Engineering teams handle data issues. The goal is to reduce investigation time from 48 hours to 2 hours through intelligent automation and structured logging.

Background

Section titled “Background”

Current Pain Points

Section titled “Current Pain Points”
  1. Poor investigation experience - Requires checking multiple systems and relies heavily on tribal knowledge
  2. Long investigation cycles - Valuable engineering resources spent on repetitive, low business impact work
  3. Inconsistent log formats - File logs have varying success/error formats and inconsistent field names
  4. No automated diagnosis - Manual investigation required for root cause identification
  5. Technical debt - dbSoftTransaction, legacy API calls affecting data accuracy
  6. Lack of tracking - Unable to track complete issue lifecycle

Why DITS

Section titled “Why DITS”

DITS will provide:

  • ✅ Unified log structure and convenient query methods
  • ✅ AI-driven automatic diagnosis and recommendations
  • ✅ Seamless Intercom integration for one-click investigation
  • ✅ Complete issue tracking and analysis capabilities

Vision

Section titled “Vision”

When Support team receives a data issue in Intercom, they will no longer need manual preliminary troubleshooting. Instead, they can trigger DITS directly in the Intercom interface, and the system will use AI and structured logs to immediately return preliminary investigation results and solution recommendations.

For Engineering tickets, the team will benefit from clear, reasonable data and log storage for rapid root cause identification.

OKRs

Section titled “OKRs”

Objective 0: Situation Awareness

Section titled “Objective 0: Situation Awareness”

Key Results:

  • KR1: Categorize current data issue types, output issue trends

Objective 1: Improve Current Data Issue Investigation Experience

Section titled “Objective 1: Improve Current Data Issue Investigation Experience”

Key Results:

  • KR1: Remove dbSoftTransaction to display correct integration mapping status
  • KR2: Structure Filelog (normalize format, add traceId support)

Objective 2: Complete Core Infrastructure Construction

Section titled “Objective 2: Complete Core Infrastructure Construction”

Key Results:

  • KR1: Migrate API-Based integrations to Saloon
  • KR2: Provide structured Filelog data support in Snowflake

Objective 3: Leverage AI to Enhance Data Issue Investigation

Section titled “Objective 3: Leverage AI to Enhance Data Issue Investigation”

Key Results:

  • KR1: Support one-click preliminary investigation results in Intercom

Implementation Timeline

Section titled “Implementation Timeline”
Q4 2025          Q1 2026          Q3 2026        Q4 2026
|----------------|----------------|----------------|
     Phase 1           Phase 2           Phase 3
     Foundation          Architecture        AI Diagnosis
                       Migration

Phase 1: Foundation Optimization (Q4 2025)

Section titled “Phase 1: Foundation Optimization (Q4 2025)”

Goal: Clean technical debt, establish accurate data foundation

1.1 Data Accuracy Improvements

Section titled “1.1 Data Accuracy Improvements”
  • Remove integration accounts restore functionality
  • Remove dbSoftTransaction, support integration success/failed status display

1.2 Log Structuring

Section titled “1.2 Log Structuring”
  • Normalize Filelog - Key task (DEV-18509)
  • Add traceId support to Nightly Sync

1.3 Insurance Log Investigation Research

Section titled “1.3 Insurance Log Investigation Research”
  • Coordinate with Megan on current Support handling of insurance-related issues

Phase 2: Core Architecture Migration (Q1 2026)

Section titled “Phase 2: Core Architecture Migration (Q1 2026)”

Goal: Establish modern, scalable technical architecture

2.1 Core Architecture Migration

Section titled “2.1 Core Architecture Migration”
  • Saloon planning (Tingsong)
  • Filelog migration to Snowflake

2.2 Convenience Tool Development

Section titled “2.2 Convenience Tool Development”
  • One-click email generation script (depends on Normalize Filelog)
  • Support integration root mapping sync retry (depends on removing dbSoftTransaction)

Phase 3: AI Smart Diagnosis (Q4 2026)

Section titled “Phase 3: AI Smart Diagnosis (Q4 2026)”

Goal: Achieve AI-driven automated diagnosis, reach 2-hour investigation target

3.1 Intercom Integration

Section titled “3.1 Intercom Integration”
  • Design Intercom plugin or shortcut method for DITS access

Technical Context

Section titled “Technical Context”

dbSoftTransaction

Section titled “dbSoftTransaction”

The dbSoftTransaction pattern currently affects integration data accuracy by allowing “deleted” records to still be returned in queries. Removing this is critical for:

  1. Correct integration mapping status display
  2. Accurate investigation of data issues
  3. Clean data foundation for AI analysis

Related code locations:

  • retail-api/app/Integrations/Support/LegacyApiBased/Models/ImportsAccountTrait.php
  • retail-api/app/Integrations/Support/LegacyApiBased/Models/ImportsInsuranceTrait.php
  • retail-api/app/Integrations/Support/LegacyApiBased/Integrators/DataIntegrationIntegrator.php

Filelog Normalization

Section titled “Filelog Normalization”

Current file logs have inconsistent formats across vendors. Normalization involves:

  1. Unified success/error response structure
  2. Consistent field naming conventions
  3. Request/response correlation via traceId
Section titled “Related”

Sources

Section titled “Sources”