Full Time - Databricks Architect with GenAI Exp. - Swanktek

12+ years in data engineering or architecture, with a strong focus on Databricks (at least 4-5 years) and AI/ML

enablement.

Deep hands-on experience with Apache Spark, Databricks (Azure/AWS), and Delta Lake.

Proficiency in AI/ML pipeline integration using Databricks MLflow or custom model deployment strategies.

Strong knowledge of Apache Airflow, Databricks Jobs, and cloud-native orchestration patterns.

Experience with structured streaming, Kafka, and real-time analytics frameworks.

Proven ability to design and implement cloud-native data architectures.

Solid understanding of data modeling, Lakehouse design principles, and lineage/tracking with Unity Catalog.

Excellent communication and stakeholder engagement skills.

Preferred Qualifications

Certification in Databricks Data Engineering Professional is highly desirable.

Experience transitioning from in house data platforms to Databricks or cloud-native environments.

Hands-on experience with Delta Lake, Unity Catalog, and performance tuning in Databricks.

Expertise in Apache Airflow DAG design, dynamic workflows, and production troubleshooting.

Experience with CI/CD pipelines, Infrastructure-as-Code (Terraform, ARM templates), and DevOps practices.

Exposure to AI/ML model integration within real-time or batch data pipelines.

Exposure to MLOps, MLflow, Feature Store, and model monitoring in production environments.

Experience with LLM/GenAI enablement, vectorized data, embedding storage, and integration with Databricks

is an added advantage.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.