Job Description
We are looking for a Data Engineer to join a fast-paced IT consulting environment in Atlanta, Georgia. In this role, you will design and optimize modern data solutions that support analytics, machine learning, and business decision-making across a variety of client initiatives. The ideal candidate brings strong technical depth in cloud-based data engineering along with a practical, solution-oriented mindset. This position is well suited for someone who enjoys turning complex data challenges into scalable and reliable platforms.
Responsibilities:
• Build and maintain robust data pipelines that move, transform, and prepare information for reporting, analytics, and operational use.
• Design end-to-end data solutions within Azure-based environments, using the right services and frameworks to support performance, reliability, and scale.
• Develop engineering workflows with Python and PySpark to process both structured datasets and more complex unstructured sources.
• Create and support data architecture components in Microsoft Fabric, Databricks, and related platforms to enable efficient data access and delivery.
• Implement pipeline orchestration and workflow automation through Databricks tools to streamline recurring data operations.
• Support machine learning initiatives by preparing high-quality datasets and building pipelines that feed model development and deployment processes.
• Apply sound data management practices to ensure consistency, usability, and governance across data assets.
• Work within Azure DevOps-driven delivery environments to contribute to version control, deployment processes, and collaborative engineering practices.
• Partner with stakeholders to understand business objectives, translate requirements into technical solutions, and deliver strong client-focused outcomes.• At least 4 years of experience in data engineering or a closely related field.
• Strong understanding of core data management concepts, including data quality, organization, and lifecycle considerations.
• Proven experience developing ETL or ELT pipelines in cloud-based environments.
• Hands-on expertise with Microsoft Azure, including Azure Databricks and related data platform services.
• Proficiency in Python and PySpark for large-scale data processing and transformation.
• Experience working with Microsoft Fabric, Databricks workflows, and modern table or storage formats such as Iceberg.
• Ability to work effectively with both structured and unstructured data sources.
• Strong analytical thinking, problem-solving ability, and a balanced approach to customer service and business priorities.
