At CloudFactory we believe that talent is equally distributed around the world, but opportunity is not. We also believe that the future of work is distributed and on-demand. We are leveling the playing field with technology that effectively shrinks the distance between companies and the world’s massive untapped talent pool. CloudFactory helps companies grow by connecting them to a scalable WorkStream, staffed by a global workforce, that is tightly integrated with their teams, processes and systems. CloudFactory helps companies scale by deeply integrating a global workforce into their teams, processes and systems. By harnessing the massive, untapped talent pool that exists in every corner of our planet, we will be able to maximize the potential of people, companies and ideas regardless of where they originate.
As a member of CloudFactory’s engineering team, you will be tasked with building and growing a world-class distributed workforce management platform that will connect one million people in developing countries to basic computer work.
You will contribute by helping to create a self service BI where CloudFactory can self serve amp; scale different needs within the organisation as and when required.
Within the first 3 months of the successful Lead Analytics Engineer coming on board, we are aiming to have a consistent approach in place by which we can ingest, load and model data for self serve. Within 6 months our intention is that 50% of requests for insight can be self served by the individual lines of business.
Roles and Responsibilities
Technical
Working in a multi-disciplinary data / engineering squad, you will:
- Provide clean data sets that support reporting, machine learning, operational analytics as well as financial and regulatory use cases to end users.
- You would drive the adoption of self-service BI, help to understand and implement the steps and process change to achieve this, working with internal and external stakeholders.
- Help define and manage best practices for our Data Warehouse, including design of source data, logical data modelling, implementation, metadata and testing standards.
- Use standard toolset (Prefect, Fivetran, DBT, Atlan, Snowflake, git) to get the necessary data, model for further analysis and reporting, ensuring right automated testing and documentation is in place. If needed, build the dashboards using BI tool.
- Focus on optimisation of our Data Warehouse, spotting opportunities to reduce complexity and cost.
- Help to improve our data estate that is performant, accessible, secure, scalable, maintainable and extensible.
- Implement good practices to ensure that Git is used in all circumstances, clean documentation, reviewing others’ work and having your work reviewed and approved by colleagues.
- Train/mentor members of the data team and help to drive a culture of best practice.
Governance amp; Data Protection
- Ensure that all work follows best security practices and fully adheres to GDPR, PECR and other data regulations.
- Ensure that all work follows the correct SDLC/approval and sign-off process before it is pushed into Production.
- Ensure that all work is documented and if needed - has a runbook in order to allow the business to continually support it.
- Follow existing processes and work to improve / identify gaps in these processes.
- Where required, ensure that a PIA (Privacy Impact Assessment) is completed.
- Work with others in the team to keep the Data Dictionary and Ubiquitous Language complete and up to date.
Requirements
Technical Skills
- Expert experience and knowledge of the SQL query language.
- Expert in Snowflake, including - but not limited to - SnowPipe, streams, security, optimisation, scale-up and scale-out.
- Good Understanding of star schemas and data warehouse concepts.
- Intermediate experience with AWS Data Lake (SNS, SQS, S3, Glue, Athena etc)
- Intermediate experience with FiveTran, DBT, Prefect and Atlan or similar tools.
- Intermediate experience and knowledge of a coding language such as Python.
- ETL and ELT experience - both batch and microservices-led.
- Experience with automated test frameworks, especially DBT.
- Nice to have experience with Infrastructure as code (IaC) concept and use of Terraform to implement.
Tech Stack
- Snowflake
- Fivetran
- DBT
- Prefect
- Atlan
- Github
- Docker
- Terraform
- AWS services (ECS, S3)
Benefits
About CloudFactory:
Join us, and change the world for the better. If you are skilled and humble, with a commitment to lifelong learning, and you’re curious about