Data Engineer (Python, Pyspark, Spark SQL, Azure Databricks, Delta Tables, Azure Data Lake, Kafka) Job at Infinity Solutions, Remote

Y1BZMzRRWEVHMUtnNHNpYmZoV3QrL1haTHc9PQ==
  • Infinity Solutions
  • Remote

Job Description

About the Role and Job:

Role: Data Engineer (Python, Pyspark, Spark SQL, Azure Databricks, Delta Tables, Azure Data Lake, Kafka)

Position Title: Sr. Software Engineer

Location: Canada (Remote)

Hire Type: Fulltime

Job Description

  • 7+ years of hands-on experience in software development, with a strong emphasis on Python programming .
  • Deep expertise in PySpark for building and optimizing scalable data pipelines.
  • Proficient in designing and implementing real-time data streaming solutions using Spark SQL, DataFrames , and Datasets .
  • Solid experience with Azure Databricks , including working with Delta tables , workflows , and scripting in Python .
  • Familiar with Azure cloud services , especially Azure Data Lake , Azure Data Factory , and EventHub or Kafka for streaming data ingestion.
  • Skilled in writing complex SQL queries and building views involving joins across multiple tables.
  • Experience across the Software Development Lifecycle (SDLC) , with a strong grasp of Agile Scrum methodologies.
  • Proficient in tools like Jira (project management), Confluence (documentation), and GitHub (version control and CI/CD pipelines).

Soft Skills & Collaboration:

  • Strong collaboration mindset with enthusiasm for mobbing sessions and cross-team coordination .
  • Excellent problem-solving and code review skills.
  • Strong communication skills with the ability to understand business requirements and translate them into technical solutions.
  • Proactive and responsive during business hours for any ad-hoc calls or meetings .
  • Demonstrated ability to prioritize effectively , think creatively , and adapt quickly to new technologies and feedback .

Job Tags

Full time, Remote work,

Similar Jobs

Jobleads-US

Lighting Designer Job at Jobleads-US

 ...team is on a mission to become the worlds leading integrated design practice. Our architects, engineers, interior designers, consultants...  ...place with Stantec.Your OpportunityOur Buildings Specialty Lighting practice is expanding and looking for a Lighting Designer to... 

EW Grobbel

Warehouse Associate (Detroit) - 2nd Shift Job at EW Grobbel

 ...warehouse stocked with essential materials. Job Description: Training on 1st shift approximately 90 days Pull, stack, and...  ...shipping & receiving records as directed Effectively operate Forklift, reach truck, and electric pallet jack Key Requirements:... 

Lincoln University of Missouri

Administrative Assistant - Paula J Carter Center Job at Lincoln University of Missouri

 ...Purpose: The Administrative Assistant will provide a wide range of administrative and some programmatic support to the Paula J Carter Center on Minority Health and Aging. This is a full-time, fully benefit eligible position. Essential Job Functions: Calendar... 

Elevance Health

Investigator II - Carelon Payment Integrity SIU Job at Elevance Health

 ...Investigator II - Carelon Payment Integrity SIU This role requires associates to be in-office 1 - 2 days per week, fostering collaboration...  ...education and experience, which would provide an equivalent background. Preferred Qualifications, Skills, and Experiences:... 

Apple Inc.

Physical Design Engineer Job at Apple Inc.

Cupertino, California, United States HardwareAdd to Favorites Physical Design EngineerDescription APPLE INC has the following available in Cupertino, California and various unanticipated locations throughout the USA. Responsible for all aspects of physical design verifications...