AI-Enabled QA Engineer

This role is open to candidates based in LATAM, Africa, and Eastern Europe. Please note that as this role supports U.S.-based clients, candidates must be available to work during U.S. business hours aligned with the client’s time zone.

The AI-Enabled QA Engineer will test the tools supporting acquisitions, financial analysis, approval workflows, and marketing operations, ensuring they are accurate and production-ready.

This role will write automated checks, test edge cases, validate AI-generated outputs against source data, and help define what “production-ready” means for the organization. The AI-Enabled QA Engineer will work closely with the engineer, product manager, M&A and marketing leads, and the Head of IT.

LOCATION

Fully Remote | Must overlap US Eastern business hours through 5:00 PM ET

KEY RESPONSIBILITIES

QA & TEST AUTOMATION

- Build test automation using Playwright, Cypress, or Selenium for interfaces and pytest or equivalent for underlying systems.

- Design test cases from specifications using equivalence classes, boundaries, negative cases, and cases not covered by the original specification.

- Run tests through CI pipelines so checks execute automatically.

- Track and triage defects using Jira, Linear, or Azure DevOps.

- Write actionable bug reports that engineers can use without requiring additional meetings.

DATA VALIDATION & API TESTING

- Use SQL to independently verify data through joins, aggregation, and record-level investigation.

- Validate data at scale through row counts, duplicate detection, orphaned records, referential integrity, and reconciliation of totals.

- Test APIs using Postman or code.

- Validate status codes, payload structures, authentication failures, and behavior on repeated identical requests.

AI OUTPUT & DOCUMENT TESTING

- Evaluate AI-generated output using a fixed set of examples.

- Measure precision and recall using historical cases.

- Run regression tests following prompt changes.

- Check AI-generated output for fabricated facts.

- Verify that fields extracted from financial statements can be traced back to their source documents.

- Test document extraction against scenarios such as scanned documents, rotated pages, and missing signatures.

- Verify target-analysis summaries against source financials so every number in an AI-generated summary is traceable to a document.

WORKFLOW & NOTIFICATION TESTING

- Test the diligence data room against cases including missing documents, duplicate uploads, and rooms that appear complete but are not.

- Test notifications to ensure they trigger when a room stalls without generating unnecessary alerts.

- Test the approval step on the priority tracker to ensure nothing is submitted without review, including attempts to bypass the approval process.

- Write test cases for AI-generated outputs that every change must continue to pass.

BRAND ASSET QUALITY ASSURANCE

- Check generated brand assets across eight to ten new brands.

- Verify that each generated asset contains the correct brand name, logo, and colours.

- Ensure assets do not contain branding belonging to another brand.

PRODUCTION READINESS

- Develop the production-ready checklist with the Product Manager and Head of IT.

- Apply the production-ready checklist to the first tool.

QUALIFICATIONS

EXPERIENCE

- 3+ years of QA experience across both manual and automated testing.

- Experience testing data-heavy applications rather than only user interfaces.

- Experience with test automation in code.

- Experience evaluating AI-generated outputs methodically.

- Experience with data validation and API testing.

- Experience testing document-heavy or extraction-based systems is a plus.

- Experience checking creative or brand output for consistency is a plus.

- Healthcare data experience is a plus.

- Finance or accounting testing experience is a plus.

- Test automation experience within a Microsoft environment is a plus.

- Experience creating a definition of done that an organization adopted is a plus.

SKILLS

- Hands-on ability to use Playwright, Cypress, or Selenium for interface testing.

- Ability to use pytest or equivalent for automated testing.

- SQL skills for joins, aggregation, and independent data verification.

- Ability to test APIs using Postman or code.

- Strong test-case design skills covering equivalence classes, boundaries, negative cases, and overlooked scenarios.

- Ability to validate row counts, duplicates, orphaned records, referential integrity, and data reconciliation.

- Ability to evaluate AI output using golden sets, precision and recall, regression testing, and fabricated-fact checks.

- Ability to test document extraction and trace extracted information back to source documents.

- Ability to run automated tests through CI pipelines.

- Ability to track and triage defects using Jira, Linear, or Azure DevOps.

- Ability to write clear, actionable bug reports.

- Understanding that AI-generated outputs can fail confidently, plausibly, and inconsistently.

- Ability to document processes and testing standards.

- Performance testing, synthetic test-data generation, accessibility checks, and Power Platform testing are additional skills that are beneficial but not required.

WHAT SUCCESS LOOKS LIKE

Within the first 90 days:

- The diligence data room is tested against realistic edge cases.

- Target-analysis summaries are verified against source financials.

- The notification system is tested for both required notifications and unnecessary alerts.

- The approval step on the priority tracker prevents unreviewed submissions.

- A fixed set of test cases is established for AI-generated output.

- Generated assets across eight to ten brands are checked for correct brand names, logos, and colours.

- A production-ready checklist is written with the Prod