Data Engineer (Python, Pyspark, Spark SQL, Azure Databricks, Delta Tables, Azure Data Lake, Kafka) Job at Infinity Solutions, Remote

Y1BZMzRRWEVHMUtnNHNpYmZoV3QrL1haTHc9PQ==
  • Infinity Solutions
  • Remote

Job Description

About the Role and Job:

Role: Data Engineer (Python, Pyspark, Spark SQL, Azure Databricks, Delta Tables, Azure Data Lake, Kafka)

Position Title: Sr. Software Engineer

Location: Canada (Remote)

Hire Type: Fulltime

Job Description

  • 7+ years of hands-on experience in software development, with a strong emphasis on Python programming .
  • Deep expertise in PySpark for building and optimizing scalable data pipelines.
  • Proficient in designing and implementing real-time data streaming solutions using Spark SQL, DataFrames , and Datasets .
  • Solid experience with Azure Databricks , including working with Delta tables , workflows , and scripting in Python .
  • Familiar with Azure cloud services , especially Azure Data Lake , Azure Data Factory , and EventHub or Kafka for streaming data ingestion.
  • Skilled in writing complex SQL queries and building views involving joins across multiple tables.
  • Experience across the Software Development Lifecycle (SDLC) , with a strong grasp of Agile Scrum methodologies.
  • Proficient in tools like Jira (project management), Confluence (documentation), and GitHub (version control and CI/CD pipelines).

Soft Skills & Collaboration:

  • Strong collaboration mindset with enthusiasm for mobbing sessions and cross-team coordination .
  • Excellent problem-solving and code review skills.
  • Strong communication skills with the ability to understand business requirements and translate them into technical solutions.
  • Proactive and responsive during business hours for any ad-hoc calls or meetings .
  • Demonstrated ability to prioritize effectively , think creatively , and adapt quickly to new technologies and feedback .

Job Tags

Full time, Remote work,

Similar Jobs

HBW Construction

Project Coordinator Job at HBW Construction

 ...three (3) years of administrative experience required, preferably in a construction environment. Proficiency with MS Office Suite, Dropbox, and data entry keyboarding skills with a high degree of accuracy. Working knowledge of Procore Project Management Software a... 

STERIS

Warehouse Shipping & Receiver Job at STERIS

The Shipping and Receiving Clerk will perform all required functions needed to prepare, ship, and/or receive Customer product while ensuring...  ...documentation. May perform the duties of Material Handler Warehouse I and II. Possesses t Receiving Clerk, Shipping, Receiving,... 

Community Dental Partners

Registered Dental Assistant Job at Community Dental Partners

 ...Join Our Team as a Registered Dental Assistant at Family Smiles Burleson- Dental Experience Required! Location: Family SmilesBurleson Schedule: Monday-Saturday (Full-Time) Become a Part of Our Smile-Making Team! At Family SmilesBurleson, we don't just care... 

Alameda Health System

Clinical Nurse Specialist Job at Alameda Health System

 ...direction, evaluates and diagnoses the nursing problems of patients. Directs appropriate...  ...nursing care. Provides a full range of clinical and consultative services. Acts as consultant...  ...American Nurses Association Clinical Specialist Certification. Preferred... 

Focus Staff

Travel Physical Therapist - Inpatient Rehab (Trauma/Stroke/ICU) - $2,200 per week Job at Focus Staff

 ...Focus Staff is seeking a travel Physical Therapist for a travel job in Lafayette, Louisiana. Job Description & Requirements ~ Specialty: Physical Therapist ~ Discipline: Therapy ~ Start Date: 09/01/2025~ Duration: 13 weeks ~40 hours per week ~ Shift: 8...