Share
Key job responsibilities
* Design and implement automated model performance evaluation pipelines
* Develop standardized metrics and KPIs for measuring model accuracy
* Create interactive dashboards and visualizations for model performance monitoring
* Collaborate with scientists and software developers to define and implement model quality gates
* Build scalable data processing solutions for large-scale model evaluation
* Partner with data engineers to ensure data quality and reliability
* Contribute to the development of our performance evaluation platforms
* Participate in code reviews and software development lifecycle processesA day in the life
As a BIE, your morning might involve collaborating with scientists to investigate forecast model performance deviations, while meeting with business stakeholders to understand their metrics requirements. The afternoon could be spent building QuickSight visualizations for evaluation dashboards and optimizing SQL queries for newly defined business metrics. Throughout the day, you'll partner with data engineers on implementing quality checks, write Python scripts to automate metric processing, and participate in design discussions about new evaluation features. The role offers a dynamic mix of technical work and cross-functional collaboration with science, engineering, and business teams.
- Experience with data visualization using Tableau, Quicksight, or similar tools
- Experience with SQL
- Experience with scripting language (e.g., Python, Java, or R)
- Experience in the data/BI space
- Experience in analyzing and interpreting data with Redshift, Oracle, NoSQL etc.
- Experience with one or more industry analytics visualization tools (e.g. Excel, Tableau, QuickSight, MicroStrategy, PowerBI) and statistical methods (e.g. t-test, Chi-squared)
- Knowledge of data modeling and data pipeline design
- Experience building and maintaining basic data artifacts (e.g., ETL, data models, queries)
- Experience with statistical analysis, co-relation analysis
- Knowledge of how to improve code quality and optimizes BI processes (e.g. speed, cost, reliability)
- Experience gathering business requirements, using industry standard business intelligence tool(s) to extract data, formulate metrics and build reports
- Master's degree, or Advanced technical degree
These jobs might be a good fit