Integration 架构概览
架构更新(2026-08-20):本文的 API/file 分类和部分历史代码结构仍保留 legacy 视角。12 个 file-based vendor 已由
file-based-integration-lambda → integrations/elt → Snowflake处理,Retail API 消费 Snowflake target tables,不再直接解析它们的文件。请以 Integration Architecture 和 File-based Integrations 为当前链路准则。 本文介绍 retail-api 的 integration module。它连接 46+ 个外部金融平台,用于同步 accounts、holdings 和 transaction data。本文的app/Integrationsparser 视图主要描述 API-based 和 legacy file-based 代码;ELT vendor 的数据处理边界见上方当前架构文档。
职责:
- 连接外部金融平台(brokerages、custodians、CRMs、aggregators)
- 同步 account data、holdings、transactions 和 insurance policies
- 将外部实体映射到内部 Households、Accounts 和 Persons
代码位置:
/retail-api/app/Integrations/- 46+ vendor implementations/retail-api/app/Http/Controllers/Advisors/Integrations/- REST API endpoints/packages/libs/integrations-core/- Core abstractions (Integrator, EntityProvider, Exceptions)/packages/libs/integrations-file-based/- Shared file-based integration logic
两种 Integration 类型
Section titled “两种 Integration 类型”| 维度 | API-based | File-based |
|---|---|---|
| 数据源 | 实时调用外部 API | Vendor 文件;已迁移 vendor 经 Lambda/ELT,legacy vendor 由 parser 读取 |
| Authentication | OAuth 2.0 / JWT / API Key / SOAP | 无(仅文件校验) |
| Sync Trigger | Scheduled + Webhook + Manual | ELT:SQS/SNS + scheduled sync;legacy:scheduled/manual |
| Complexity | 高(状态机、错误恢复) | 中等(解析、mapping) |
| Status | API-based integrations | 12 个已完成 ELT cutover,其余 file-based 按 legacy/迁移中处理 |
| 示例 | Yodlee、Schwab API、Orion、Addepar | Fidelity、Pershing、LPL、Schwab (file) |
说明:部分 vendor 同时提供 API 和 file-based 版本(例如 Schwab、LPL、Betterment)。
Legacy file-based 实现细节
Section titled “Legacy file-based 实现细节”下面的 Controller → Integrator → Parser 图只描述仍由 retail-api 负责解析和写库的 legacy file-based integration,以及 API-based integration 在 retail-api 内的代码组织。已完成 ELT cutover 的 vendor 不经过该 Parser → S3 分支;它们的当前链路见本文顶部链接。
核心数据流(API-based 与 legacy file-based)
Section titled “核心数据流(API-based 与 legacy file-based)”┌─────────────────────────────────────────────────────────────────────────────┐│ User Action ││ (Link Account / Upload File / Sync) │└─────────────────────────────────────────────────────────────────────────────┘ │ ▼┌─────────────────────────────────────────────────────────────────────────────┐│ Controller Layer ││ EntityController | IntegrationMappingController | SyncController ││ #[IntegrationScenario(Scenario::*)] │└─────────────────────────────────────────────────────────────────────────────┘ │ ▼┌─────────────────────────────────────────────────────────────────────────────┐│ Job Queue (Async) ││ Dispatch sync jobs for background processing │└─────────────────────────────────────────────────────────────────────────────┘ │ ▼┌─────────────────────────────────────────────────────────────────────────────┐│ Integrator Layer ││ Integration::getIntegratorFqcn() → App\Integrations\[Vendor]\Integrator ││ ││ API-based: listEntities(), sync(), syncAll() ││ File-based: parse(), map(), import() │└─────────────────────────────────────────────────────────────────────────────┘ │ ┌───────────────┴───────────────┐ ▼ ▼┌───────────────────────────────┐ ┌───────────────────────────────┐│ Connector (API-based) │ │ Parser (File-based) ││ HttpConnector | OAuthConnector│ │ CSV/Excel parsing & mapping ││ SoapConnector │ │ │└───────────────────────────────┘ └───────────────────────────────┘ │ │ ▼ ▼┌───────────────────────────────┐ ┌───────────────────────────────┐│ External API │ │ S3 Storage ││ (Vendor REST/SOAP/OAuth) │ │ (Uploaded files) │└───────────────────────────────┘ └───────────────────────────────┘ │ │ └───────────────┬───────────────┘ ▼┌─────────────────────────────────────────────────────────────────────────────┐│ Vendor Models Layer ││ Parse response → Vendor-specific Account/Holding models │└─────────────────────────────────────────────────────────────────────────────┘ │ ▼┌─────────────────────────────────────────────────────────────────────────────┐│ Internal Models Layer ││ Map to: Household | Account | Insurance | Person | TargetCategoryMix ││ Create/Update IntegrationMapping │└─────────────────────────────────────────────────────────────────────────────┘Vendor 实现模式
Section titled “Vendor 实现模式”app/Integrations/├── [Vendor]/│ ├── Api.php # API client wrapper (optional)│ ├── Connector.php # Transport: HTTP/OAuth/SOAP│ ├── Integrator.php # Core sync logic│ ├── Importer.php # Household import logic│ ├── Models/ # Vendor-specific data models│ │ ├── Account.php│ │ ├── Holding.php│ │ └── ...│ ├── Exceptions/ # Vendor-specific exceptions│ └── OAuth2/ # OAuth config (if OAuth-based)└── Support/ ├── Scenario.php # 10 integration scenarios ├── Developer.php # Developer ownership map ├── LegacyApiBased/ │ ├── Connectors/ # Base connector classes │ │ ├── HttpConnectorWithIntegration.php │ │ ├── OauthConnectorWithIntegration.php │ │ └── SoapConnectorWithIntegration.php │ ├── Integrators/ │ └── Exceptions/ └── ApiBased/ # Modern API-based infrastructure ├── Connectors/ └── Integrators/共享 Packages
Section titled “共享 Packages”| Package | Location | Purpose |
|---|---|---|
integrations-core |
/packages/libs/integrations-core/ |
Base classes: Integrator, EntityProvider, CursorPage, Exceptions |
integrations-file-based |
/packages/libs/integrations-file-based/ |
Shared file parsing and mapping logic |
core-models |
/packages/libs/core-models/ |
IntegrationType enum, base models |
Integration
Section titled “Integration”Advisor 已配置的一条 integration 实例。
// Key fields$integration->type; // IntegrationType enum (46 types)$integration->reference; // Vendor-specific identifier$integration->credentials; // Encrypted JSON credentials$integration->failed_since; // When sync started failing$integration->failed_biz_days; // Consecutive failure days
// Key methods$integration->getIntegratorFqcn(); // Get Integrator class$integration->getConnectorFqcn(); // Get Connector class$integration->getSynchronizer(); // Get Sync handlerIntegrationMapping
Section titled “IntegrationMapping”Links an external entity (vendor account) to an internal object (Household, Account, etc.).
// Key fields$mapping->integration_id; // Parent Integration$mapping->parent_id; // Hierarchical parent (for nested mappings)$mapping->mappable_type; // HOUSEHOLD | ACCOUNT | INSURANCE | PERSON | CONTACT | TARGET_CATEGORY_MIX$mapping->mappable_id; // ID of mapped internal object$mapping->reference; // Vendor-specific entity ID$mapping->sync_status; // Current sync status$mapping->last_completed_at; // Last successful sync
// Key methods$mapping->sync(); // Synchronize this mappingIntegrator
Section titled “Integrator”Core class implementing sync logic for each vendor.
abstract class Integrator{ abstract public function syncAll(): void; abstract public static function sync(IntegrationMapping $mapping): array|null; abstract protected static function getVendor(): IntegrationType;}Connector
Section titled “Connector”Transport layer handling API communication.
| 类型 | Class | 用例 |
|---|---|---|
| HTTP | HttpConnectorWithIntegration |
REST APIs with API Key/Token |
| OAuth | OauthConnectorWithIntegration |
OAuth 2.0 flow (Schwab, Addepar, Orion) |
| SOAP | SoapConnectorWithIntegration |
Legacy SOAP services |
EntityProvider
Section titled “EntityProvider”Interface for listing available entities from external service.
interface EntityProvider{ public function listEntities(array $conditions, ?string $cursor = null): CursorPage;}Integration 场景
Section titled “Integration 场景”定义在 Support/Scenario.php 中,用于记录操作上下文:
| 场景 | 说明 |
|---|---|
CREATE_INTEGRATION |
初始 integration 配置 |
LINK_ENTITY |
将 vendor entity 关联到 household |
LIST_ENTITIES |
查询可用 accounts |
SYNC_HOUSEHOLD |
同步单个 household |
SYNC_INTEGRATION |
同步该 integration 的所有 accounts |
NIGHTLY_SYNC |
定时后台同步 |
IMPORT_HOUSEHOLD |
批量导入 household |
IMPORT_CLIENTS |
导入 client data |
LIST_TAGS |
获取 CRM tags/categories |
SEND_PDF |
发送文档 |
API 端点
Section titled “API 端点”GET /advisors/{advisor}/integrations/{integration}/entities List available entities from vendor (with search, cursor pagination)
GET /advisors/{advisor}/integrations/{integration}/mappings List linked accounts with household info
POST /advisors/{advisor}/integrations/{integration}/mappings Link vendor entity to household
DELETE /advisors/{advisor}/integrations/{integration}/mappings/{mapping} Unlink entity (soft delete)
POST /advisors/{advisor}/integrations/{integration}/mappings/{mapping}/sync Sync single household mapping
POST /advisors/{advisor}/integrations/{integration}/sync Sync entire integration
GET /advisors/{advisor}/integrations/{integration}/tags List household tags from CRM integration全部 46 种 Integration 类型
Section titled “全部 46 种 Integration 类型”API-based(当前代码中的 vendor 列表)
Section titled “API-based(当前代码中的 vendor 列表)”| Vendor | Auth Type | Developer |
|---|---|---|
| Addepar | OAuth 2.0 | Winston Li |
| Advyzon | OAuth | Kewei Yan |
| Albridge | API | Tingsong Xu |
| Allianz API | OAuth | Yan Hu |
| Asset Book | OAuth | Qianwei Hao |
| Asset Mark | API | Qianwei Hao |
| Black Diamond | API | Qianwei Hao |
| Blueleaf | HTTP | Tingsong Xu |
| Bridge FT | API | Yan Hu |
| Capitect | OAuth | Yan Hu |
| Circle Black | API | Tingsong Xu |
| Commonwealth | HTTP | Tingsong Xu |
| DST | API | Yan Hu |
| FinFolio | API | Yan Hu |
| Investigo | API | Kewei Yan |
| LPL (API) | API | Qianwei Hao |
| Max My Interest | API | Winston Li |
| Morningstar Advisor | API | Qianwei Hao |
| Morningstar Office | API | Winston Li |
| Nationwide | API | Winston Li |
| Orion | OAuth | Kewei Yan |
| Panoramix | OAuth | Tingsong Xu |
| Redtail | API | Qianwei Hao |
| Riskalyze | API | Tingsong Xu |
| Schwab API | OAuth 2.0 | Yan Hu |
| Smart Office | API | Winston Li |
| Tamarac | SOAP | Tingsong Xu |
| Wealth Access | API | Yan Hu |
| Wealthbox | API Key | Kewei Yan |
File-based(19,历史清单)
Section titled “File-based(19,历史清单)”| Vendor | File Type | Developer |
|---|---|---|
| Allianz | File | Yan Hu |
| Altruist | File | Yan Hu |
| Apex | File | Tingsong Xu |
| Betterment | File | Kewei Yan |
| Fidelity | CSV | Qianwei Hao |
| First Clearing | File | Qianwei Hao |
| Flourish | File | Tingsong Xu |
| Folio Investing | File | Qianwei Hao |
| Interactive Brokers | File | Winston Li |
| Jackson | File | Winston Li |
| LPL (File) | File | Qianwei Hao |
| My529 | File | Kewei Yan |
| Pacific Life | File | Winston Li |
| Pershing | Excel | Kewei Yan |
| Raymond James | File | Winston Li |
| RBC | File | Yan Hu |
| Schwab (File) | CSV | Yan Hu |
| SEI | File | Winston Li |
| Trust America | File | Kewei Yan |
- API-based Patterns - Connector design, authentication patterns
- API-based Sync Lifecycle - Job flow, error handling
- API-based Vendor Specifics - Yodlee, Schwab details
- File-based Patterns - Parser, mapper design
- File-based Vendor Specifics - Format details
- Technical Debt - Known issues and improvement opportunities