Proven architecture patterns in production environments.

• Fraud signals scattered across claims, policy, and third-party data sources
• Batch-only scoring meant fraud was often flagged after payout
• Investigation teams relied on manual cross-referencing across systems

Karsient consolidated claims, policy, and external risk-signal data onto a governed Lakehouse, then built a real-time scoring pipeline that evaluates every claim at intake rather than after the fact.

• Streaming ingestion from claims and policy systems via CDC
• Feature pipeline on Delta Lake feeding a real-time scoring endpoint
• Unity Catalog governing access across claims, underwriting, and fraud teams

• Store-level demand forecasts relied on spreadsheets and static rules
• POS, promotional, and weather data lived in separate systems with no shared model
• Forecast refreshes took days, too slow to react to demand shifts

Karsient built a unified demand-signal pipeline that blends POS transactions, promotional calendars, and external weather data into a single feature set, replacing spreadsheet-driven planning with a governed, automated model.

• Batch and near-real-time ingestion from POS and inventory systems
• Bronze/Silver/Gold medallion layers for raw, cleaned, and modelled demand data
• Forecasting models orchestrated on a daily schedule via Databricks Workflows

• Regulatory figures were assembled manually across spreadsheets each cycle
• No consistent lineage from source transaction to reported figure
• Reconciliation between source systems and reports was fully manual

Karsient designed a governed reporting warehouse with full source-to-report lineage, replacing manual spreadsheet assembly with automated, auditable pipelines aligned to the bank's regulatory calendar.

• Governed ingestion from core banking and general ledger systems
• A lineage-tracked transformation layer mapping every reported figure to its source
• Automated reconciliation checks between source systems and reporting outputs

• Each market ran its own reporting stack with inconsistent KPI definitions
• Leadership depended on regional teams to manually assemble a consolidated view
• Ad hoc drill-down and trend analysis were effectively impossible

Karsient consolidated on-prem and cloud source systems into a single governed Lakehouse, standardising KPI logic centrally so every market reports against the same definitions while retaining market-specific compliance controls.

• Automated ingestion from ERP, CRM, and manual data sources per market
• A central Lakehouse handling transformation, aggregation, and KPI calculation
• Market-level data isolation for security and regulatory compliance

• On-prem SQL Server ETL pipelines could not scale to growing data volumes
• Processing windows regularly breached agreed SLAs
• High infrastructure overhead to keep the legacy estate running

Karsient re-engineered the SQL-based ETL logic into Databricks-native PySpark and Scala workloads, replacing the legacy processing engine while preserving business logic and output parity.

• A hub-and-spoke ingestion pattern feeding a central Lakehouse
• ELT pipelines built on Databricks replacing SQL Server stored-procedure ETL
• Parallelised, distributed processing replacing single-node batch jobs