Job Description
Assignment: RQ08584 - Software Developer - ETL - Senior
Job Title: ETL & Azure Data Engineer
Requisition (SS): RQ08584
Start Date: 2025-04-01
Client: Labour and Transportation Cluster
End Date: 2026-03-31
Office Location: 87 Sir William Hearst Ave., Toronto, Ontario M3M 0B4
Organization: Labour and Transportation Cluster
Extension # of Days: 250.00
Ministry: Ministry of Transportation
# Business Days: 252.00
Note: Candidate is required to come in office 3 days a week
Must haves:
- Extensive experience in gathering requirements and business process knowledge in order to design correct and high-quality data transformation.
Description
Deliverables section:
- Scope of Services and Deliverables - The Services and Deliverables to be provided by the Vendor will include the following:
- Participate in the review of project, work request and maintenance release artefacts - Use Case, System Requirement Specification, etc.
- Deliver detailed system design documents and any other supporting documentation conforming to Ministry standards.
- Design logical and physical data models using Sybase Power Designer.
- Produce transformation mapping document.
- Implement data pipelines using Azure Data Factory (ADF), stored procedure, and Informatica PowerCenter performing extraction, transformation, and loading activities.
- Implement solutions using Logic App and Function App.
- Generate structured JSON file using ADF/ SQL.
- Develop Azure CI/CD pipeline to automate ADF release.
- Develop complex Oracle PL/SQL program to fulfill project requirements.
- Implement complex data conversion, e.g., binary to character, EBCDIC to UTF.
- Implement complex data transformation(s) for derived and calculated values.
- Integrate data sets from diverse source systems.
- Implement ADF for initial data load and incremental load.
- Promote ADF and Informatica ETL/ELT through all Ministry environments including Development, Integration Testing, QA, UAT, Production.
- Resolve and troubleshoot ADF pipeline, stored procedure, and Informatica workflow technical problems.
- Monitor and analyze incident, provide timely resolution.
- Collaborate with IT Professionals throughout the Software Development Life Cycle.
- Document information from diverse business area stakeholders and subject matter experts.
- Optimize performance of ADF pipeline, Azure SQL and Synapse databases, and Informatica workflow.
- Monitor application functionality and performance on daily basis.
- Provide effective knowledge transfer to Ministry staff at all stages of this assignment.
Evaluation Table
ETL Development-Data Integration - 40%
- Extensive experience in gathering requirements and business process knowledge in order to design correct and high-quality data transformation.
- Extensive experience in design, development, and implementation with Azure Data Factory, Databricks, including:
- Self-Hosted Integration Runtime
- Data movement, ETL pipelines
- Performance tuning on large data volumes
- Complex data transformations
- ADF Debugger to validate data transformation.
- Azure Roles
- Experience with Informatica, including:
- Workflow Manager
- Repository Manager
- Designer
- Workflow Monitor
- Experience with developing solutions using Logic App and Function App.
- Experience in rapid application development (RAD) methodologies.
- Experience in maintaining and improving existing ETL processes.
- Experience in investigating data to identify potential issues within ETL pipelines.
- Experience with Business Intelligence tools IBM Cognos and Microsoft PowerBI.
Azure Technologies - 40%
- Experience with CI/CD (DevOps) pipelines and concepts, including:
- Azure Resource Management
- GitHub Repo
- Source code version control
- Branching, pull requests, build, release, multiple parallel development repositories merge.
- Experience with Azure DevOps, including:
- Azure Boards
- Azure Test Plans
- Azure Pipelines
- Azure Repos
- Azure Artefacts
Oracle PL/SQL Development - 10%
- Extensive experience in designing and developing SQL Server Stored Procedure, Oracle PL/SQL programs.
Database Technologies - 10%
- Experience with Oracle, Microsoft SQL Server database and tools.
- Experience with relational and hierarchical database technologies.
- Extensive knowledge of Azure SQL DB and Synapse platforms, including:
- Performance tuning on large data volume
Job Tags
Work at office, 3 days per week,