Engenheiro de Dados Databricks/DBT/Streaming/ML/AI

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Engenheiro de Dados Databricks/DBT/Streaming/ML/AI based in Brazil.

We are looking for a Senior Data Engineer to join a strategic initiative focused on enabling data capabilities for Artificial Intelligence solutions. This role combines data architecture, engineering excellence, and hands-on implementation to build scalable foundations for AI and machine learning applications. The professional will help define technical standards, create reliable data pipelines, and enable the consumption of structured and unstructured data by advanced AI systems. You will work with modern data platforms, cloud environments, and large-scale processing technologies while collaborating with multidisciplinary teams. The position requires strong technical ownership, curiosity, and the ability to validate new approaches through experimentation and practical solutions. This is an opportunity to contribute to innovative data architectures that support the next generation of AI-driven products and services.

Accountabilities:: The role is responsible for designing, evolving, and enabling scalable data solutions that support AI initiatives, acting as a technical reference for data architecture and engineering practices. Key responsibilities include:

Define architectural directions, standards, and best practices for data platforms focused on AI and machine learning use cases.

Design and implement mechanisms for ingesting, processing, and making unstructured data available, including documents, PDFs, audio, images, and videos.

Prepare and structure data for consumption by AI/ML and generative AI models, including processes such as feature engineering, chunking, and embeddings.

Build, optimize, and maintain scalable ETL/ELT pipelines and data workflows.

Automate, improve, and recommend enhancements to existing data processes and architectures.

Conduct proof-of-concepts and technical experiments to validate solutions before large-scale adoption.

Integrate data platforms with external systems, APIs, SaaS applications, and streaming solutions.

Support multidisciplinary teams by providing technical guidance and enabling data-driven innovation.

Apply best practices for data governance, reliability, scalability, and performance optimization.

Requirements:

We are looking for a professional with strong experience in data engineering, modern data platforms, and AI-enabled data solutions. The ideal candidate combines architectural vision with hands-on technical execution and collaboration skills.

Solid experience with Databricks and Apache Spark for large-scale distributed data processing.

Advanced knowledge of Python and SQL applied to data engineering (Scala experience is a plus).

Experience processing unstructured data such as documents, PDFs, audio, images, or videos for AI consumption.

Experience defining data architectures and technical strategies beyond pure pipeline execution.

Knowledge of data preparation for AI/ML and generative AI solutions, including embeddings, chunking, and model consumption workflows.

Experience building and maintaining scalable ETL/ELT pipelines.

Experience with DBT for data modeling and transformation processes.

Knowledge of Delta Lake, Lakehouse architectures, data versioning, and governance.

Experience integrating data with external systems through APIs, SaaS platforms, or streaming technologies.

Experience with cloud computing environments, preferably GCP.

Experience with Git, CI/CD practices, and agile development environments.

Ability to work independently, propose solutions, test hypotheses, and continuously improve processes.

Nice-to-have skills:

Experience with MLflow and Databricks ML.

Hands-on experience with RAG pipelines, vectorization, embeddings, and vector databases.

Knowledge of generative AI integrations and machine learning APIs.

Experience with Kubernetes for workload orchestration.

Experience with Terraform and Infrastructure as Code practices.

Familiarity with event-driven architectures and streaming technologies such as Kafka or Pub/Sub.

Benefits:

Fully remote work model, providing flexibility to work from anywhere in Brazil.

Health insurance coverage through Porto Seguro, with options for dependents.

Dental insurance coverage through Porto Seguro.

Profit Sharing Program (PLR).

Childcare assistance.

Meal and food allowance through Alelo.

Home office assistance.

Partnerships with educational institutions for professional development.

Support for technology certifications, including cloud certifications in GCP, Azure, and AWS.

Livelo points program.

TotalPass access with discounted fitness options.

Mindself platform focused on mindfulness and well-being.

Opportunity to work on a strategic AI and data transformation project until December 2026, with potential extension.

How Jobgether works: We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best!  Why Apply Through Jobgether?    Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.     #LI-CL1