Data Integration Engineer (Databricks / Python / SQL)

About Gramian Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs. About the Role Our client is a Big4 Consultancy group that works with leading financial institutions on AI-driven transformation, automation, advanced analytics, and financial crime prevention. Their are building a production-grade AI-assisted KYC / Customer Due Diligence platform for an international insurance environment. We are looking for an experienced Data / Integration Engineer to own the data flowing into and out of the platform. The role focuses on Databricks, Python / PySpark, SQL, data modelling, data contracts, APIs, legacy-system integration, and data quality, replacing mocked sources with reliable production integrations. CONTRACT: B2B or B2C contract COMMITMENT: Full-time, 3-4 months contract with possibility of extension or getting on new projects LOCATIONS: Remote, EU-based candidates START: ASAP PROCESS: Initial qualification followed by client interviews Responsibilities - Inventory and map all internal and external data sources. - Define data contracts and canonical data structures for client, screening, evidence, audit, and reporting data. - Build connectors to external screening providers, public registers, and local data sources. - Connect Databricks with SQL Server, legacy databases, and core enterprise systems. - Support API, database-link, batch, file, and SFTP-based integrations. - Design and maintain data models supporting evidence, audit trails, and reporting. - Implement data quality checks, contract tests, and retention rules. - Validate incoming and outgoing data flows across production services. - Support integration troubleshooting and production-readiness activities. - Contribute to a business-critical platform where data accuracy and reliability are essential. Requirements - Strong hands-on experience with Databricks, including Delta Lake and Unity Catalog. - Strong experience with Python / PySpark. - Advanced SQL skills. - Experience integrating with SQL Server and legacy databases. - Strong experience with REST APIs, DB links, batch processing, files, and SFTP integrations. - Experience with data modelling and data-contract design. - Experience implementing data-quality validation and integration testing. - Fluent English and ability to work remotely within the EU.