As Data Engineering experts we see many job descriptions for data related jobs. Those might include data science, data warehouse architect or data warehouse engineer, data developer, and of course, data analyst.
These jobs, or more often careers, can span the gamut of data related arts. I like to think of this industry of data, and the needs and hopes every organization that aims to make use of their data, as an art form. You may have heard the term “it is more of an art than a science”. This is is often said about data management roles because much of the value-add portion of data wrangling is artful in nature. It is uncovering the truthful portions of the data. Or, using an analogy of a good sushi chef, it is like removing portions of the delicate fish that interrupt an amazing taste. Knowing which pieces to remove from the fish is often more important than adding more too it. Sushimi anyone?
Here’s an interesting look at a general Data Warehouse Engineer job description. Do you have a need like this in your organization? If so, please reach to us to see if we can help you with your data-driven organization initiatives. Feel free to use this on your next hiring req as needed.
Job Description
Our company is seeking a qualified Data Warehouse Engineer to work with our game analytics team in building and maintaining back end data solutions, utilizing AWS and GCP technologies, to support analytics initiatives, to improve player experience and game performance as well as increase player retention and monetization.
Responsibilities are:
- Work with other data engineers and game analysts to build new and extend existing data structures to support game analytics
- Develop, maintain, and support ETL processes for loading data from multiple data sources into a Redshift data warehouse
- Maintain large, multi-terabyte data warehouse which includes performance tuning and data retention/purge processes
- Research and troubleshoot data quality issues, providing fixes and proposing both short- and long-term solutions
Requirements are:
- 5+ years of experience in a data warehousing, data engineering, or data architect role
- 5+ years of experience in developing, managing, and maintaining large, multi-terabyte data warehouse
- environments including the pipelines of data and ETL processes feeding into it
- Strong experience with AWS Redshift
- Experience working with other AWS data technologies such as S3, Redshift Spectrum, Athena, Data Pipeline,
- EMR, RDS, and Kinesis
- Expert level SQL skills
- Data modeling experience for both transactional and data warehousing environments including familiarity with
- Kimball dimensional and 3NF modeling standards
- Experience working with a variety of data sources such as MySQL, Oracle, SQL Server, PostgreSQL, S3, HDFS, and MongoDB
- Strong interpersonal skills and problem-solving ability
Desired Skills are:
- Experience with AWS CloudFormation for creating and managing a variety of AWS resources
- Experience with Python and Linux shell scripting
- Experience using source control systems (Git, Perforce, SVN, etc.)
- Familiarity or experience working with big data solutions such as Apache Spark, PrestoDB, Kafka, etc.
- Prior experience working in a game development studio or gaming company developing online titles
- Passion or personal interest in video gaming
- BS degree in Computer Science, Math, Engineering, or equivalent
Summary
So we can see that from a typical data warehouse engineering role title, today’s modern role and requirements span beyond a single set of skills such as SQL or a standard ETL tool. It now stretches a broad set of skills such, touching many heterogeneous technologies, cloud and on-premise infrastructure skills, traditional programming languages, streaming technology, and even traditional DevOps components. At AICG you can find top-level consultants, resources, and training to help you accelerate similar implementation activities.
