Skip to content

Integration 架构概览

架构更新(2026-08-20):本文的 API/file 分类和部分历史代码结构仍保留 legacy 视角。12 个 file-based vendor 已由 file-based-integration-lambda → integrations/elt → Snowflake 处理,Retail API 消费 Snowflake target tables,不再直接解析它们的文件。请以 Integration ArchitectureFile-based Integrations 为当前链路准则。 本文介绍 retail-api 的 integration module。它连接 46+ 个外部金融平台,用于同步 accounts、holdings 和 transaction data。本文的 app/Integrations parser 视图主要描述 API-based 和 legacy file-based 代码;ELT vendor 的数据处理边界见上方当前架构文档。

职责

  • 连接外部金融平台(brokerages、custodians、CRMs、aggregators)
  • 同步 account data、holdings、transactions 和 insurance policies
  • 将外部实体映射到内部 Households、Accounts 和 Persons

代码位置

  • /retail-api/app/Integrations/ - 46+ vendor implementations
  • /retail-api/app/Http/Controllers/Advisors/Integrations/ - REST API endpoints
  • /packages/libs/integrations-core/ - Core abstractions (Integrator, EntityProvider, Exceptions)
  • /packages/libs/integrations-file-based/ - Shared file-based integration logic
维度 API-based File-based
数据源 实时调用外部 API Vendor 文件;已迁移 vendor 经 Lambda/ELT,legacy vendor 由 parser 读取
Authentication OAuth 2.0 / JWT / API Key / SOAP 无(仅文件校验)
Sync Trigger Scheduled + Webhook + Manual ELT:SQS/SNS + scheduled sync;legacy:scheduled/manual
Complexity 高(状态机、错误恢复) 中等(解析、mapping)
Status API-based integrations 12 个已完成 ELT cutover,其余 file-based 按 legacy/迁移中处理
示例 Yodlee、Schwab API、Orion、Addepar Fidelity、Pershing、LPL、Schwab (file)

说明:部分 vendor 同时提供 API 和 file-based 版本(例如 Schwab、LPL、Betterment)。

下面的 Controller → Integrator → Parser 图只描述仍由 retail-api 负责解析和写库的 legacy file-based integration,以及 API-based integration 在 retail-api 内的代码组织。已完成 ELT cutover 的 vendor 不经过该 Parser → S3 分支;它们的当前链路见本文顶部链接。

核心数据流(API-based 与 legacy file-based)

Section titled “核心数据流(API-based 与 legacy file-based)”
┌─────────────────────────────────────────────────────────────────────────────┐
│ User Action │
│ (Link Account / Upload File / Sync) │
└─────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────┐
│ Controller Layer │
│ EntityController | IntegrationMappingController | SyncController │
│ #[IntegrationScenario(Scenario::*)] │
└─────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────┐
│ Job Queue (Async) │
│ Dispatch sync jobs for background processing │
└─────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────┐
│ Integrator Layer │
│ Integration::getIntegratorFqcn() → App\Integrations\[Vendor]\Integrator │
│ │
│ API-based: listEntities(), sync(), syncAll() │
│ File-based: parse(), map(), import() │
└─────────────────────────────────────────────────────────────────────────────┘
┌───────────────┴───────────────┐
▼ ▼
┌───────────────────────────────┐ ┌───────────────────────────────┐
│ Connector (API-based) │ │ Parser (File-based) │
│ HttpConnector | OAuthConnector│ │ CSV/Excel parsing & mapping │
│ SoapConnector │ │ │
└───────────────────────────────┘ └───────────────────────────────┘
│ │
▼ ▼
┌───────────────────────────────┐ ┌───────────────────────────────┐
│ External API │ │ S3 Storage │
│ (Vendor REST/SOAP/OAuth) │ │ (Uploaded files) │
└───────────────────────────────┘ └───────────────────────────────┘
│ │
└───────────────┬───────────────┘
┌─────────────────────────────────────────────────────────────────────────────┐
│ Vendor Models Layer │
│ Parse response → Vendor-specific Account/Holding models │
└─────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────┐
│ Internal Models Layer │
│ Map to: Household | Account | Insurance | Person | TargetCategoryMix │
│ Create/Update IntegrationMapping │
└─────────────────────────────────────────────────────────────────────────────┘
app/Integrations/
├── [Vendor]/
│ ├── Api.php # API client wrapper (optional)
│ ├── Connector.php # Transport: HTTP/OAuth/SOAP
│ ├── Integrator.php # Core sync logic
│ ├── Importer.php # Household import logic
│ ├── Models/ # Vendor-specific data models
│ │ ├── Account.php
│ │ ├── Holding.php
│ │ └── ...
│ ├── Exceptions/ # Vendor-specific exceptions
│ └── OAuth2/ # OAuth config (if OAuth-based)
└── Support/
├── Scenario.php # 10 integration scenarios
├── Developer.php # Developer ownership map
├── LegacyApiBased/
│ ├── Connectors/ # Base connector classes
│ │ ├── HttpConnectorWithIntegration.php
│ │ ├── OauthConnectorWithIntegration.php
│ │ └── SoapConnectorWithIntegration.php
│ ├── Integrators/
│ └── Exceptions/
└── ApiBased/ # Modern API-based infrastructure
├── Connectors/
└── Integrators/
Package Location Purpose
integrations-core /packages/libs/integrations-core/ Base classes: Integrator, EntityProvider, CursorPage, Exceptions
integrations-file-based /packages/libs/integrations-file-based/ Shared file parsing and mapping logic
core-models /packages/libs/core-models/ IntegrationType enum, base models

Advisor 已配置的一条 integration 实例。

// Key fields
$integration->type; // IntegrationType enum (46 types)
$integration->reference; // Vendor-specific identifier
$integration->credentials; // Encrypted JSON credentials
$integration->failed_since; // When sync started failing
$integration->failed_biz_days; // Consecutive failure days
// Key methods
$integration->getIntegratorFqcn(); // Get Integrator class
$integration->getConnectorFqcn(); // Get Connector class
$integration->getSynchronizer(); // Get Sync handler

Links an external entity (vendor account) to an internal object (Household, Account, etc.).

// Key fields
$mapping->integration_id; // Parent Integration
$mapping->parent_id; // Hierarchical parent (for nested mappings)
$mapping->mappable_type; // HOUSEHOLD | ACCOUNT | INSURANCE | PERSON | CONTACT | TARGET_CATEGORY_MIX
$mapping->mappable_id; // ID of mapped internal object
$mapping->reference; // Vendor-specific entity ID
$mapping->sync_status; // Current sync status
$mapping->last_completed_at; // Last successful sync
// Key methods
$mapping->sync(); // Synchronize this mapping

Core class implementing sync logic for each vendor.

abstract class Integrator
{
abstract public function syncAll(): void;
abstract public static function sync(IntegrationMapping $mapping): array|null;
abstract protected static function getVendor(): IntegrationType;
}

Transport layer handling API communication.

类型 Class 用例
HTTP HttpConnectorWithIntegration REST APIs with API Key/Token
OAuth OauthConnectorWithIntegration OAuth 2.0 flow (Schwab, Addepar, Orion)
SOAP SoapConnectorWithIntegration Legacy SOAP services

Interface for listing available entities from external service.

interface EntityProvider
{
public function listEntities(array $conditions, ?string $cursor = null): CursorPage;
}

定义在 Support/Scenario.php 中,用于记录操作上下文:

场景 说明
CREATE_INTEGRATION 初始 integration 配置
LINK_ENTITY 将 vendor entity 关联到 household
LIST_ENTITIES 查询可用 accounts
SYNC_HOUSEHOLD 同步单个 household
SYNC_INTEGRATION 同步该 integration 的所有 accounts
NIGHTLY_SYNC 定时后台同步
IMPORT_HOUSEHOLD 批量导入 household
IMPORT_CLIENTS 导入 client data
LIST_TAGS 获取 CRM tags/categories
SEND_PDF 发送文档
GET /advisors/{advisor}/integrations/{integration}/entities
List available entities from vendor (with search, cursor pagination)
GET /advisors/{advisor}/integrations/{integration}/mappings
List linked accounts with household info
POST /advisors/{advisor}/integrations/{integration}/mappings
Link vendor entity to household
DELETE /advisors/{advisor}/integrations/{integration}/mappings/{mapping}
Unlink entity (soft delete)
POST /advisors/{advisor}/integrations/{integration}/mappings/{mapping}/sync
Sync single household mapping
POST /advisors/{advisor}/integrations/{integration}/sync
Sync entire integration
GET /advisors/{advisor}/integrations/{integration}/tags
List household tags from CRM integration

API-based(当前代码中的 vendor 列表)

Section titled “API-based(当前代码中的 vendor 列表)”
Vendor Auth Type Developer
Addepar OAuth 2.0 Winston Li
Advyzon OAuth Kewei Yan
Albridge API Tingsong Xu
Allianz API OAuth Yan Hu
Asset Book OAuth Qianwei Hao
Asset Mark API Qianwei Hao
Black Diamond API Qianwei Hao
Blueleaf HTTP Tingsong Xu
Bridge FT API Yan Hu
Capitect OAuth Yan Hu
Circle Black API Tingsong Xu
Commonwealth HTTP Tingsong Xu
DST API Yan Hu
FinFolio API Yan Hu
Investigo API Kewei Yan
LPL (API) API Qianwei Hao
Max My Interest API Winston Li
Morningstar Advisor API Qianwei Hao
Morningstar Office API Winston Li
Nationwide API Winston Li
Orion OAuth Kewei Yan
Panoramix OAuth Tingsong Xu
Redtail API Qianwei Hao
Riskalyze API Tingsong Xu
Schwab API OAuth 2.0 Yan Hu
Smart Office API Winston Li
Tamarac SOAP Tingsong Xu
Wealth Access API Yan Hu
Wealthbox API Key Kewei Yan
Vendor File Type Developer
Allianz File Yan Hu
Altruist File Yan Hu
Apex File Tingsong Xu
Betterment File Kewei Yan
Fidelity CSV Qianwei Hao
First Clearing File Qianwei Hao
Flourish File Tingsong Xu
Folio Investing File Qianwei Hao
Interactive Brokers File Winston Li
Jackson File Winston Li
LPL (File) File Qianwei Hao
My529 File Kewei Yan
Pacific Life File Winston Li
Pershing Excel Kewei Yan
Raymond James File Winston Li
RBC File Yan Hu
Schwab (File) CSV Yan Hu
SEI File Winston Li
Trust America File Kewei Yan