Senior Data Engineer
Location Redwood City
Consultant Angela Boultinghouse
Date posted September 6, 2016
We’re innovating, solving big challenges and having fun while doing it. As a senior member of our Data Engineering team you will be responsible for realizing deeper data integrations with our customers. You will also leverage your deep database knowledge and experience with large distributed systems to improve both the performance and scalability of the OpenGov platform.
We're looking for folks with can-do attitudes who go above and beyond. At OpenGov our engineering culture is centered around hiring smart talent and building world-class software - we stay humble while driving forward with our audacious mission.
-Build a scalable data pipeline which can handle millions of rows of data in a few seconds
-Measure, tune and optimize existing pipeline components to improve performance
-Investigate and adopt open source technologies for distributed computing and storage
-Write an extensible transformation engine to enable customers to wrangle data in a domain specific manner
Ideal candidate has:
-3+ years of Data Engineering experience with concentration on performance & scalability
-Experience working with large, distributed systems
-Experience performing large scale data ingestion
-Profecient with 1 or more of the following: Apache Spark, Kafka, Hive, Redshift, Cassandra, Mapreduce, HDFS, Pig, or Yarn
-Experience working with files containing 10-100 million rows of data
-Deep database knowledge and practical experience working in databases
-Familiar with OOP languages such as Ruby, Java, Scala, etc.
-BS/MS in Computer Science is preferred