We are looking for a Data Engineer consultant (Databricks, PySpark and SQL) to reinforce the Pilotage team, in charge of integrating and representing data from the Intervention Management business line within the Electricity Network IS division.
The consultant will be working in a dynamic, fast-paced environment, in collaboration with business teams, user representatives and project IT teams.
He/she will be part of a team of Data Engineers and Data Visualization specialists in a Reporting and Business Intelligence-oriented context.
Main tasks
As part of the job, the Data Engineer consultant will carry out the following activities:
– Design, develop and maintain robust and scalable data pipelines on Databricks (SQL, PySpark)
– Create and enhance applications with “Databricks apps” using the Streamlit or Dash Frameworks,
– Ensure data quality and integrity through validation and cleansing processes,
– Participate in the definition of data architectures,
– Participate in the continuous improvement of best practices,
– Document processes and solutions developed.
Technical skills
– Practical experience with the Databricks platform (notebooks, jobs, clusters, workspace),
– Mastery of SQL language,
– Solid programming skills in a Data environment with Pyspark,
– Appreciated knowledge of Python Streamlit or Dash,
– Experience in relational database management and data integration,
– Knowledge of data modeling (Business Intelligence),
– Mastery of agile methodology,
– Experience with version management tools (Gitlab).
Behavioral skills:
– Ability to work independently while collaborating with the rest of the team,
– Ability to stand back and take an interest in business issues,
– Real interpersonal skills, ability to listen,
– Good communication skills, ability to work with non-technical teams.
Qualifications:
– 2 to 4 years’ experience as a Data Consultant Engineer in Data Bricks,
– 2 to 4 years’ experience as a Data Consultant Engineer in Python (Pyspark),
– Appreciated experience in creating applications using the Streamlit or Dash python frameworks,
– Databricks certification appreciated.
Technical environment
– Databricks
– SQL
– Python
– PySpark
– Python Streamlit, Dash
– Power BI Service,
– AWS
– Jira / Confluence,
– Gitlab