Western Australia Police Force

Senior Data Engineer

(Technology Used: SQL Server, Azure SQL, SSMS and Transact sql, Azure portal tools, Azure Synapse Analytics, Serverless SQL pools, Datalake, Gen 2 storage, Azure Lakehouse, PySpark notebooks, Logic Apps, Power BI, Oracle SQL developer and PL/SQL, Azure shell, Azure CLI, Azure DevOps Boards and CI/CD pipelines)

Scope: BAU - typical everyday work of an Azure Data Engineer; Project – developing data lakehouse (data lake based DWH)

Work done:

• Engagement with upstream and downstream stakeholders

• ETL - Azure Synapse Analytics – pipelines, Gen2, PySpark notebooks

• Database exploration and Dataset preparation for downstream: Power BI Reporting Analysts (including support for Power

• BI and Data Scientists

• DWH – exploration and development of STAR structures and a Lakehouse (Gen 2 DWH utilizing delta format)

• Logic Apps integration with Synapse pipelines

• DevOps git integrated and Azure Boards

• Re-engineering/refactoring queries Oracle to SQL Server

• Documenting new and some existing processes/products