Senior Data Engineer

Senior Data Engineer

Remote · LATAM · Full-Time Contract LATAM (Remote) Full-Time Contract ASAP English Required

ABOUT THE ROLE We are looking for a Senior Data Engineer with deep expertise in AWS, batch processing, and real-time streaming architectures. This is a hands-on role — you will design and own scalable data pipelines serving both high-throughput batch workloads (GBs to TBs) and low-latency streaming systems. You will collaborate with analysts, data scientists, and business teams, and help mentor junior engineers to raise the bar across the data

platform.

Batch Processing Design & implement batch pipelines for large-scale processing (GBs to

TBs) Build ETL/ELT workflows using PySpark, SQL, Glue, and EMR Optimize jobs for performance, partitioning, and cost Manage data ingestion from DBs, files, and APIs into the data lake Ensure data quality, consistency, and reliability

Real-Time / Streaming Design & implement real-time pipelines using Apache Flink (Scala

preferred) Process high-throughput streams with low-latency requirements Implement stateful processing, windowing, and event-time handling Handle late/out-of-order events; exactly-once / at-least-once semantics Configure checkpointing, fault tolerance, and recovery

Integrate with Kafka or Kinesis

CORE RESPONSIBILITIES Build and maintain data lake / lakehouse architectures on AWS

(S3-based) Integrate multiple data sources — APIs, SaaS platforms, databases Ensure data governance, security, and access control (IAM) Implement monitoring, logging, and alerting for all pipelines Collaborate with analysts, data scientists, and business teams Mentor junior engineers and enforce best practices

TECH STACK Python SQL Scala PySpark Apache Flink Apache Kafka AWS Glue Amazon EMR Amazon S3 AWS Lambda Amazon Athena Amazon Kinesis Delta Lake / Iceberg

REQUIRED QUALIFICATIONS 5+ years of experience in Data Engineering

Strong skills in Python, SQL, and Scala Hands-on AWS experience: S3, Glue, Athena, Lambda, EMR,

Kinesis, IAM Strong batch processing experience with Spark / PySpark at scale Proven real-time streaming experience with Apache Flink in

production Solid understanding of batch vs streaming trade-offs and event-driven

architectures Experience building data lakes or data warehouses in production Experience integrating external systems via APIs Distributed systems fundamentals: fault tolerance, partitioning,

consistency

WHAT MAKES A STRONG CANDIDATE Has built both batch AND real-time pipelines in production — not just one or the other Knows when to apply each paradigm: batch for cost-efficient large volumes, streaming for low-latency real-time use cases Experience handling millions of records in batch and high event throughput in streaming systems Strong debugging and performance optimization skills across distributed data systems Hands-on mindset — you build things and own them end-to-end, not just design them

NICE TO HAVE AWS certifications (Data Analytics Specialty, Solutions Architect) Experience with Delta Lake, Apache Iceberg, or open table formats Familiarity with dbt for transformation workflows Experience with orchestration tools (Apache Airflow, AWS Step

Functions) Background in IoT, telemetry, or event-heavy domains OpenTelemetry or advanced observability experience

THIS ROLE IS NOT FOR YOU IF You have only worked with batch OR only with streaming — this role requires both You need a fully defined architecture and runbook before making engineering decisions You are not comfortable owning pipelines end-to-end in production without daily oversight You optimize for theoretical elegance before proving a real cost or reliability problem exists You are not comfortable communicating directly in English with international teams

WHY COMMITTED STAFF Committed Staff connects the top 1% of LATAM talent with high-growth U.S. companies. We don't just place candidates — we build committed teams that scale. You'll get direct access to a real data engineering challenge with meaningful scope, full-time remote work aligned with U.S. time zones, competitive LATAM-market compensation, and a growing network of senior tech professionals across Latin America. Originally posted on Himalayas