Note: The job is a remote job and is open to candidates in USA. v4c.ai is seeking a Senior Platform Engineer to design, secure, and optimize their enterprise data infrastructure. The role involves owning the foundational architecture of the Databricks Lakehouse to ensure efficient building within a secure and high-performance ecosystem.
Responsibilities
- Infrastructure Management: Architecture, deployment, and daily support of enterprise Databricks Workspaces
- Data Governance: Full implementation of Unity Catalog for data lineage and access control
- Guardrails & Governance: Designing global cluster policies to balance user agility with cost
- Performance Tuning: Optimizing complex Spark configurations, storage layouts, and query speeds
- Innovation Delivery: Productionizing cutting-edge capabilities like Databricks Genie and Lakehouse Apps
Skills
- Advanced Databricks administration (Workspaces, Unity Catalog, Delta Lake)
- Strong experience with AWS, Azure, or GCP via Terraform (IaC)
- Deep knowledge of Apache Spark internals and Spark UI tuning
- Implementing Role-Based Access Control (RBAC) and data masking
- Building automated deployment tracks using Git, GitHub Actions, or GitLab CI
- 8+ years in Platform, DevOps, or Data Infrastructure Engineering
- Ability to bridge the gap between Data Science teams and IT security
- Strong root-cause analysis skills for complex, distributed computing issues
Company Overview