• *Project Overview:**
Join a growing community of professionals advancing the next wave of AI. As an AI Trainer, you’ll play a hands-on role by analyzing and providing feedback on data to improve LLM performance, helping ensure that the next generation of AI technology is accurate and trustworthy.
We are seeking a skilled Power Platform Governance Architect to work as a project consultant in our AI Labor Marketplace. This is not a full-time employment position — you will be engaged as an expert project consultant on a contract basis.
• *Location:**
U.S.-based experts only
• *Engagement:**
Part-time, project-based expert evaluation work
• *Work Type:**
Remote
• *Project Summary:**
Design and produce structured evaluation datasets to train and benchmark an AI Advisor Agent operating on Microsoft Power Platform tenant-state data. The consultant will define evaluation schemas, scoring logic, and governance-aligned validation standards, and create approximately 400 structured JSON-based evaluation test cases across Metrics, Insights, Action Plan Step, and Action Plan Generation components.
• *Consultant Engagement Terms:**
This is a project-based consultant role. Consultants will be paid on a per-project basis; hourly rates are estimates based on anticipated completion time. Consultants control their own schedule, provide their own tools, and may simultaneously provide services to other vendors/employers (subject to those vendors’ allowances).
• *Responsibilities:**
• *Contributors will:**
• Design structured tenant-state schemas for evaluation purposes
• Define evaluation task structure, including expected outputs and scoring criteria
• Develop governance-realistic synthetic Power Platform scenarios
• Author structured JSON inputs and expected outputs
• Create scoring rubrics and validation logic for agent evaluation
• Ensure logical correctness, schema integrity, and compliance alignment
• Produce ~400 gold-standard evaluation test cases over 12 weeks
• Document assumptions and edge-case handling standards
• *Expected Outcomes:**
• A defined evaluation framework and scoring methodology
• A structured tenant-state schema for agent evaluation
• ~400 validated evaluation test cases
• Documented scoring and validation standards aligned to governance best practices
• *Qualifications:**
• 6–10+ years of experience as a Power Platform Administrator or Architect
• Deep experience with tenant governance, DLP, environments, connectors, and ALM
• Hands-on experience with CoE Toolkit or tenant telemetry analysis
• Strong proficiency authoring and validating structured JSON
• Prior experience designing QA frameworks, evaluation rubrics, or scoring guidelines
• Ability to reason at enterprise governance depth
Apply Now
Apply Now