Senior Data Engineer

  • Role located in Canberra – interstate candidates will be considered
  • Anticipated start date 18/7/2022
  • Contract to 30/6/2023 with extension options
  • A BASELINE clearance is required

Role Description

A Senior Data Engineer is required to drive the design and implementation of solutions for key data-centric projects and ongoing business conducted on a modern Data and Analytics Platform. The agency’s work heavily relies on data, and data engineering is a critical part. The platform has been built with top tier tools and includes Azure Data Lake, Azure Data Factory, Databricks, Apache Spark, Aristotle, RStudio, Power BI and SAS. You will have strong technical experience who are seeking to transition towards a management/consultant style career track. This role will engage with Senior Executives to advise on strategy, business planning and shaping the approach to data engineering, while also remaining closely engaged with the technical work. The ability to communicate well with business areas is an essential and critical requirement of this role.

Working closely with Subject Matter Experts, Data Scientists and Data Analysts, you will be required to:

    • Contribute your expertise to development of data engineering solutions for ingestion and ETL of data assets critical for projects and ongoing business;
    • Establish a streamlined process for the formation of joined data and manipulation to deliver business ready data for use by the many teams;
    • Develop and lead data engineering solutions to join and transform data assets critical to delivery of projects;
    • Review and refine existing engineering solutions on the data and analytics platform;
    • Provide advice and guidance on risks and issues to non-technical managers and stakeholders;
    • Provide ad hoc advice on data governance issues to data governance team members;
    • Assist with data modelling on existing and business intelligence data assets to inform data architecture;
    • Develop, implement and review data-related infrastructure, processes, and procedures;
    • Work closely with our application, infrastructure and networking partners on development of infrastructure to support data and analytics projects, and integration between cloud and on-premise hosted services;
    • Work under limited direction and be accountable for undertaking planning, analysis, design, development, and delivery activities within tight timeframes;
    • Lead a small, technically-focussed team of Data Engineers (to be established).

Skills and Experience

    • Data Engineering
      • Qualifications in Computer Science, IT or a related discipline
      • Data engineer experience
      • Demonstrated experience in developing, configuring, validating, and managing Microsoft cloud platforms, Data Factory, SQL Server, Databricks
      • Understanding of Machine Learning concepts
      • Demonstrated experience in DevOps and/or DataOps
      • Documented, hands-on ETL development using tools like SQL Server Integration Services or Azure Data Factory.
    • Data modelling and database management
      • Data modelling experience
      • Strong knowledge of database structure systems, data analytics, data mining and data modelling techniques
      • Documented experience in data analysis and management, with excellent analytical problem-solving abilities
      • Demonstrated ability to combine multiple large and complex datasets to deliver high-quality datasets.
    • Coding and systems administration
      • Demonstrated experience in coding languages (such as SQL, Python, R, C Sharp/C++, Powershell)
      • Systems administration experience
      • Database / Data Lake management experience from design, development, delivery, validation and ongoing maintenance.
    • Reporting and visualisation
      • Experience using and developing engineering solutions for data visualisation tools such as RStudio, Power BI, SAS, Qlik, Tableau
      • An applied understanding of UI design concepts.
    • Communication and Collaboration
      • Well-developed team collaboration and communication skills
      • Demonstrated ability to work under limited direction and accountability for completion of work
      • Demonstrated ability to provide advice and develop solutions within multi-disciplinary work environments
      • Demonstrated ability to communicate complex technical concepts to non-technical staff.
    • Desirable
      • ¬†Experience with the deployment and maintenance of machine learning models
      • Experience developing datasets API delivery for a variety of scenarios
      • Experience with machine learning technologies
      • Apache Spark (Scala or Python) experience
      • Spark SQL coding experience
      • Automation experience
      • End-to-end Machine Learning pipeline development
      • Proven experience working on Data Science projects.

If you'd like to find out more about this role we'll call you back

    Upload CV (optional)
    Attachments must be doc, docx or odt format and a maximum of 5MB

    By submitting this form you agree to our Privacy Policy

    Or you can apply for this role here

      Are you an Australian citizen or do you have the right to work in Australia? *

      Do you hold a current security clearance? *

      Upload CV*
      Attachments must be doc, docx or odt format and a maximum of 5MB

      Upload Responses to Essential and Desirable Criteria*
      Attachments must be doc, docx or odt format and a maximum of 5MB

      By submitting this form you agree to our Privacy Policy