System Engineer / Solution Architekt Big Data (m/w)

Details:

System engineer/solution architect with deep understanding of big data solutions (Hadoop, Cloudera), and in particular building data lakes in an big enterprise environment. As part of the engineering team his/her task would be to support Engineers and Developers integrating different kind of data sources in a large, distributed data lake, and cover especially the data specifications part of it.

Roles and Responsibilities:

* Work closely with solution designers to understand approach, requirements and solution design of current project

* Collect, prepare and process data source specifications from various data source owners

* Cover all significant aspects of data sources like type, volume, velocity, frequency, content description, etc.

* Coordinate with systems engineers and other stakeholders outside the project to align requirements and design

* Support project management and preparing routines

* Provide clear qualitative and quantitative information and knowledge to the team

* Take part in and lead workshops

* Document and visualize source descriptions and other artifacts

* Drive the source management and on-boarding processes for the project

* Support the general solution architecture and design efforts of the project

Skills

Must have:

* Several years of proven experience as architect/engineer in complex Big Data projects (Hadoop, Kafka, Spark, Hbase, HIVE, Impala, etc.) or other mass data applications at Petabyte scale

* At least 3 years of experience with IT Projects in large enterprises

* At least 3 years of experience with (big) data integration projects

* At least 1 year of experience with performance critical data processing at large scale using files, data streams and databases

* Very good documentation and visualization skills

* Excellent oral and written communication skills in English (German nice to have)

* Team player, experience with agile development approach and tools (SCRUM, KANBAN, JIRA)

* Must be able to effectively and professionally communicate within the team

Beneficial:

* At least 3 years of experience with large Hadoop clusters

* Experience with batch and streamed data flows in and out Cloudera (Spark, Kafka)

* Experience with testing IT Systems and Data Integration Systems

* Experience with working in internationally distributed cross-functional teams

* Experience with the Cloudera framework