Big Data Architect - AI
• LOCATION: REMOTE UNITED STATES
- DEPARTMENT: ENGINEERING
- WORK STATUS: FULL-TIME
Overview
Are you looking for a hybrid or remote work opportunity? Are you interested in a workplace that allows for flexibility in your day? Are you ready for a workplace that provides benefits that suit your needs? About our AI In the past two years, billions of documents have already benefited from the insights of our AI platform - and we are just getting started on our journey to use AI to improve each user experience, product, matter, and investigation at our flagship software. We are focused on helping our users discover the truth more quickly, and act on data with confidence.
- We are focused on algorithm excellence, to provide the most robust and trusted experience possible.
- We are creating a world class toolset to solve complex challenges quickly and iteratively.
- AI will be leveraged everywhere, in all stages of the discovery process to better manage cases and to optimize product operations.
As a team, we believe in exploration, experimentation, and bringing your curiosity to work every day. We know that you cannot innovate without experimentation - and a little failure happens on the path to invention. We use the latest and greatest to ensure we are the best. We strive to experiment, ship, and learn every day. About the Big Data Architect Role The Big Data Architect will work closely with product teams in the within the AI group to build outstanding data lake or data mesh. You will work with real-time and batch data at petabyte scale. You will manage data governance and data access patterns for ensuring our customers' data is protected. You will oversee data catalog and data observability for ensuring data quality. You will be hands-on and will work side-by-side teams in the organization. Your Role in Action
- Work with our data scientists, product managers, and engineering teams to develop a big data architecture that supports our data privacy restrictions while supporting our data science needs
- Oversee data governance policy and data access procedures
- Manage data catalog and data observability
- Build cost modeling for data architecture
- Hands-on work to automate and build tooling for teams to use. Be willing to jump into a project to build things out.
- Contribute to our technical investments roadmap and help prioritize tech debt and architecture investments
- Mentor talent within the AI group to promote career development
Your Skills
- Experience with data lake and warehouse technologies like Hudi, Delta, Snowflake, Synapse, Redshift, S3, and ADLS
- Experience creating batch and stream processing data sets applying technologies like Apache Spark, Apache Flink, Kafka, DBT, AirFlow, Prefect and other ELT tools
- Experience with SQL and relational databases
- Experience creating data governance policies
- Experience in data catalogs and data observability patterns
- Fluent in programming languages suitable to implement big data and machine learning solutions. Ex: Python, Scala
- Experience in performance tuning and optimization
- Experience with product / tool / vendor evaluation and selection
- Experience building cost projections
- Experience in unstructured data sets and designing APIs, service-oriented architectures
- Experience with AWS, Google Cloud, or Azure data infrastructure and tooling
Apply tot his job Apply To this Job Apply tot his job Apply To this Job