imports

Code
Download import schema file(s) as CSV with pre-signed S3 URLs.

This endpoint generates CSV schema template files and stores them in S3, then returns pre-signed download URLs.

S3 Folder Structure:
- Schema templates: {instance_name}/uploads/migration-data/schemas/
- Pending uploads:  {instance_name}/uploads/migration-data/pending/
- Validated data:   {instance_name}/uploads/migration-data/ (root)

Logic:
- If schema_type is None: Returns all fixed schemas
- If schema_type is provided and it's fixed: Returns that specific fixed schema
- If schema_type is dynamic (policy_transactions) and no policy_type_id: Returns error
- If policy_type_id is provided and schema_type is None: Returns all schemas
- If policy_type_id is provided with schema_type: Returns that specific dynamic schema

CSV Template Format:
- Row 1: Field names (column headers)
- Row 2: Required/Not Required status for each field
- Row 3: Field descriptions
- Row 4: Valid values for categorical fields (pipe-separated, e.g. "Payment | Invoice | Adjustment")

IMPORTANT - Before Uploading Your Data:
The downloaded CSV template contains up to 4 rows of metadata for guidance only.
You MUST:
1. Review the field requirements (rows 2-4)
2. DELETE rows 2, 3, and 4 (metadata rows)
3. Keep only Row 1 (headers) and add your data rows below it
4. Your final CSV should have: Header row + Data rows only

Parameters
----------
schema_type : str, optional
    Type of schema to generate ('insureds', 'agencies', 'policy_transactions', etc.)
policy_type_id : str, optional
    Policy type UUID (required for dynamic schemas like policy_transactions)

Returns
-------
success : bool
messages : list of str
data : dict
    Dictionary containing:
    - schemas: List of schema dictionaries with download URLs
      - schema_type: Schema identifier
      - schema_name: Human-readable name
      - download_url: Pre-signed S3 URL (valid for 1 hour)
      - file_name: Name of the CSV file

Examples
--------
Get all fixed schemas:
    POST /api/v2/imports/download_import_schema
    {}

Get specific fixed schema:
    POST /api/v2/imports/download_import_schema
    {"schema_type": "insureds"}

Get dynamic schema:
    POST /api/v2/imports/download_import_schema
    {"schema_type": "policy_transactions", "policy_type_id": "uuid-here"}

Get all schemas for a policy type:
    POST /api/v2/imports/download_import_schema
    {"policy_type_id": "uuid-here"}

Code
Retrieve detailed status information for migration jobs.

Returns status, timestamps, and optionally logs for migration jobs.

Parameters
----------
job_id : str, optional
    Job identifier UUID returned from run_migration. If omitted, returns recent jobs.
include_logs : bool, optional
    If True, include logs in response (default: False).
log_levels : list, optional
    List of log levels to include.
    Valid values: 'DEBUG', 'INFO', 'WARNING', 'ERROR'.
    Example: ['ERROR', 'WARNING'] to include error and warning logs.
log_limit : int, optional
    Maximum number of log entries to return (default: 100).

Code
List and download migration analysis reports.

Returns pre-signed S3 URLs for available migration reports including:
- Analysis reports (.md) - generated by the analyze migration step
- Premium comparison reports (.csv) - generated by the policy import step

Reports are stored under: {instance_name}/uploads/migration_analyses/
Newer runs store reports in a job-specific subdirectory beneath that root.

Parameters
----------
job_id : str, optional
    Job identifier UUID from run_migration. If provided, returns reports
    for that specific migration run. Newer runs are resolved via a
    job-specific report prefix; older flat reports fall back to time-window
    matching. If omitted, returns all available reports.

Returns
-------
success : bool
messages : list of str
data : dict
    Dictionary containing:
    - reports: List of report dictionaries with:
      - file_name: Basename of the report file
      - report_path: Path relative to the migration_analyses root
      - file_type: 'analysis' or 'premium_comparison'
      - size_bytes: File size in bytes
      - last_modified: ISO timestamp of last modification
      - download_url: Pre-signed S3 URL (valid for 1 hour)
    - job_id: The job_id filter applied (if provided)
    - job_status: The job's status (if job_id provided)
    - reports_path: S3 prefix searched for matching reports

Examples
--------
Get all migration reports:
    POST /api/v2/imports/get_migration_reports
    {}

Get reports for a specific run:
    POST /api/v2/imports/get_migration_reports
    {"job_id": "e7e18ed0-5cec-47bc-8865-a2d215d61974"}

Code
Return detailed column metadata for an import schema, including valid values
for categorical fields.

Works for both fixed schemas (account_history, payments, agencies, etc.)
and dynamic schemas (policy_transactions, historical_terms).

Parameters
----------
schema_type : str
    Schema identifier (e.g. 'account_history', 'payments', 'policy_transactions')
policy_type_id : str, optional
    Policy type UUID (required for dynamic schemas like policy_transactions)

Returns
-------
success : bool
messages : list of str
data : dict
    Dictionary containing:
    - schema_type: Schema identifier
    - columns: dict mapping column name to metadata:
      - source: 'schema'
      - required: bool
      - description: str
      - valid_values: list of str (only present for categorical fields)
      - condition: str (only present for conditionally required fields)
    - sample_rows: list (empty for fixed schemas)

Examples
--------
Get column metadata for a fixed schema:
    POST /api/v2/imports/get_schema_columns
    {"schema_type": "account_history"}

Get column metadata for a dynamic schema:
    POST /api/v2/imports/get_schema_columns
    {"schema_type": "policy_transactions", "policy_type_id": "uuid-here"}

Code
Generate pre-signed S3 POST URL for uploading CSV files to the pending validation folder.

The file will be uploaded to a pending folder and must be validated using
validate_uploaded_files() before being moved to the final migration-data folder.

S3 Folder Structure:
- Pending uploads:  {instance_name}/uploads/migration-data/pending/
- Validated data:   {instance_name}/uploads/migration-data/
- Schema templates: {instance_name}/uploads/migration-data/schemas/

Parameters
----------
schema_type : str, required
    Schema identifier (e.g., 'insureds', 'agencies', 'policy_transactions')
    Must be a valid fixed or dynamic schema type.
policy_type_id : str, optional
    Policy type UUID (required for dynamic schemas like 'policy_transactions')
    The first 8 characters will be used in the filename.

Returns
-------
success : bool
messages : list of str
data : dict
    Dictionary containing:
    - schema_type: The schema identifier
    - policy_type_id: The policy type ID (only for dynamic schemas)
    - upload_url: S3 URL to POST the file to
    - upload_fields: Dictionary of form fields required for the POST request
    - expires_in: URL expiration time in seconds (600 = 10 minutes)

Examples
--------
Get upload URL for fixed schema (insureds):
    POST /api/v2/imports/get_upload_url
    {"schema_type": "insureds"}

Get upload URL for dynamic schema (policy_transactions):
    POST /api/v2/imports/get_upload_url
    {"schema_type": "policy_transactions", "policy_type_id": "a320737f-2aef-42cf-957e-5c5da0e46aa1"}

Response:
    {
        "success": true,
        "data": {
            "schema_type": "policy_transactions",
            "policy_type_id": "a320737f-2aef-42cf-957e-5c5da0e46aa1",
            "upload_url": "https://s3.amazonaws.com/bucket-name",
            "upload_fields": {
                "key": "instance/uploads/migration-data/pending/policy_transactions_a320737f.csv",
                "Content-Type": "text/csv",
                "x-amz-server-side-encryption": "AES256",
                "policy": "...",
                "x-amz-algorithm": "...",
                "x-amz-credential": "...",
                "x-amz-date": "...",
                "x-amz-signature": "..."
            },
            "expires_in": 600
        }
    }

Note: S3 folders are virtual - the pending folder will be created automatically
when the first file is uploaded. No explicit folder creation needed.

Code
Import an insured disclosure

Parameters
----------
override: boolean
    A flag to determine whether we should override existing disclosures
disclosures: list
    The list of parameters containing disclosure data to be imported.
    [
        {
            "property_id": "a320737f-2aef-42cf-957e-5c5da0e46aa1",
            "revision_id": "f2d68d87-08b5-40a1-b165-67650a59b5eb",
            "loss_date": "2025-01-21",
            "description": "Damage to flooring",
            "loss_amount": 1000.00,
            "loss_cause": "Water Damage"
        },
        ...
    ]

Code
Import loss report data for a property

Parameters
----------
property_id: str
    The property id. Required.
revision_id: str
    The revision id. Required.
loss_reports: list
    The list of loss report records to be imported.
loss_history_report_number: str
    The loss history report number.
loss_history_retrieved_date: str
    The loss history report retrieval date.
loss_history_vendor_status: str
    The loss history report vendor status.
cache_key: str
    The loss history report cache key.
created_by: str
    The contact id of the user who requested the loss history report.

Code
Import a note

Parameters
----------
note : dict
    The note to import. Supports legacy alert fields for backward compatibility:
    - alert_status: bool - Whether the note has an alert
    - alert_date: datetime - When to fire the alert
    - alert_email: str - Email to send alert to
    - alert_sent: bool - Whether alert has been sent
    - alert_now: bool - Fire alert immediately
    - alert_displayed: bool - Whether alert has been displayed
    - event_alert: bool - Whether this is an event-based alert
    - event_fired: bool - Whether the event has fired
    - event_preset: str - Event type preset
force_create : bool
    Delete and recreate the note if it already exists
create_only : bool
    Only create the note, do not update if it already exists

import_policy_number

Code
List all available schema types for data imports.

Returns both fixed (hardcoded) and dynamic (database-generated) schemas
with their metadata, field counts, and parameter requirements.

Fixed Schemas: insureds, agencies, producers, mortgagees, additional_insured,
addresses, dates_to_remember, payments, account_history, notes, credit_reports,
x_business_locations_restrictions, x_policy_types_restrictions

Dynamic Schemas: policy_transactions (requires policy_type_id parameter)

Returns
-------
success : bool
messages : list of str
data : dict
    Dictionary containing:
    - schemas: List of schema dictionaries with:
      - schema_type: Schema identifier
      - name: Human-readable schema name
      - description: Schema purpose and usage
      - field_count: Number of fields (for fixed schemas)
      - query_type: Query type (for dynamic schemas)
      - required_parameters: List of required parameters (e.g., ['policy_type_id'])

Examples
--------
List all available schemas:
    POST /api/v2/imports/list_available_schemas
    {}

Code
Purge previously imported migration data.

This endpoint is restricted to lower environments and, when api.v2
permissions apply to the caller, additionally requires the
``britecore/imports/migration_purge`` permission at ``rw``. It launches a
tracked migration purge job so API users can explicitly delete prior
migrated data before running a new migration instead of deleting data
during import.

Parameters
----------
policy_type_mappings : dict, optional
    Map of execution names to policy type UUIDs, as passed to run_migration. The purge
    only removes contacts listed in the migration contact files and policies whose every
    revision is listed in the mapped policy transaction / historical terms files; records
    from any other source are preserved. Required when targets include "policy".
targets : list or str, optional
    Purge targets to run. Supported values are "contacts" and "policy".
    Defaults to both targets.

Code
Run data migration via job runner.

This API simplifies migration by pre-defining the entire configuration.
User only needs to provide policy type ID mappings.

Starting a migration writes configuration to S3 and launches a job that
imports data into the site, so the caller must hold
``britecore/imports/migration_jobs`` at ``rw`` (BC-25688). The status and
report endpoints require the same path at ``r``.

Parameters
----------
policy_type_mappings : dict
    Map of execution names (user-defined labels) to policy type UUIDs.
    File names are generated from policy_type_id: policy_transactions_{uuid[:8]}.csv
    Execution names starting with ``Historical_`` are routed only to the
    historical terms phase; every other name runs the policy and analyze
    phases. Use the prefix for mappings that feed historical_terms files.
disable_rules : dict, optional
    Map of execution names to semicolon-separated rule IDs to disable.
filters : dict, optional
    Runtime filters to control which migration scripts execute.
dry_run : bool, default False
    If True, returns the config and command without executing.

Code
Validate ALL uploaded files from pending folder and move valid ones to migration-data root folder.

This API:
1. Lists all CSV files in pending folder
2. Validates each file against its schema (extracted from filename)
3. For dynamic schemas (policy_transactions): Extracts policy_type_id from filename and generates schema
4. If valid: Copies to root migration-data folder (overwrites existing) and deletes from pending
5. If invalid: Deletes from pending and returns validation errors to user
6. If deep_validation=True: Cross-validates contact references between files

S3 Folder Structure:
- Schema templates: {instance_name}/uploads/migration-data/schemas/ (not touched by this API)
- Pending uploads:  {instance_name}/uploads/migration-data/pending/ (source for validation)
- Validated data:   {instance_name}/uploads/migration-data/ (destination for valid files)

Parameters
----------
deep_validation : bool, optional
    If True, performs additional referential integrity validation:
    - Validates producerId in policy_transactions exists in producers.csv
    - Validates agencyId in policy_transactions exists in agencies.csv
    Referential issues are returned as warnings (non-blocking).
    Default is False for faster validation.

Filename Requirements:
- Fixed schemas: {schema_type}.csv (e.g., insureds.csv, agencies.csv)
- Dynamic schemas: {schema_type}_{uuid_prefix}.csv (e.g., policy_transactions_a320737f.csv)
  The UUID prefix must match an existing policy type in the database.

CRITICAL - CSV File Format:
Uploaded CSV files must contain ONLY:
- Row 1: Column headers (field names)
- Row 2+: Your actual data rows

If you downloaded schema templates using download_import_schema(), those files contain
3 metadata rows. You MUST delete rows 2-3 (Required/Not Required status and descriptions)
before uploading. The validator expects only header + data rows.

Validation Rules:
- CSV must have header row + at least 1 data row (2 rows minimum)
- All expected schema columns must be present in header
- Required fields must have data in at least one row
- Extra columns generate warnings but don't fail validation
- Files with unknown schema types are rejected
- Dynamic schema files with invalid UUID prefixes are rejected
- ALL files are deleted from pending folder after processing (pass or fail)

Returns
-------
success : bool
messages : list of str
data : dict
    Dictionary containing:
    - status: Overall validation status ('all_passed', 'partial', 'all_failed')
    - files: List of validation results for each file
      - schema_type: The schema identifier
      - filename: Original filename
      - validation_errors: List of validation issues (if failed)
      - referential_warnings: List of cross-file reference warnings (if deep_validation=True)
      - row_count: Number of data rows (if validated)
    - summary: Validation summary
      - total_files: Total files found in pending
      - validated: Number of successfully validated files
      - failed: Number of failed validations
      - referential_warnings: Total count of referential warnings (if deep_validation=True)

API Success vs Validation Status:
- success=true: API executed successfully (files were processed)
- success=false: System error (S3 connection failed, permissions error, etc.)
- Check data.status for data-level validation results

Examples
--------
Validate all pending files:
    POST /api/v2/imports/validate_uploaded_files
    {}

Response (all files passed):
    {
        "success": true,
        "data": {
            "status": "all_passed",
            "files": [
                {
                    "schema_type": "insureds",
                    "filename": "insureds.csv",
                    "row_count": 150
                }
            ],
            "summary": {
                "total_files": 1,
                "validated": 1,
                "failed": 0
            }
        }
    }

Response (partial failures):
    {
        "success": true,
        "messages": ["1 of 2 files validated successfully"],
        "data": {
            "status": "partial",
            "files": [
                {
                    "schema_type": "insureds",
                    "filename": "insureds.csv",
                    "row_count": 150
                },
                {
                    "schema_type": "agencies",
                    "filename": "agencies.csv",
                    "validation_errors": ["Missing required columns: name, status"]
                }
            ],
            "summary": {
                "total_files": 2,
                "validated": 1,
                "failed": 1
            }
        }
    }

Note: ALL files are deleted from pending folder after processing, regardless of validation result.
Failed files must be corrected and re-uploaded.