Job intelligence
Senior Data Scientist (AI Evaluation & Improvement)
Evolent
Executive opportunity brief
The decision in one screen
A compact evidence-first read of compensation, flexibility, mobility and what still needs verification. No universal quality score is used.
Fast read
Opportunity DNA
Worth a closer look because location flexibility looks unusually broad; confirm whether a degree is actually required, preferred, or replaceable with equivalent experience.
Current comparable position is around the 22th percentile.
No material geography, office-attendance or timezone restriction is visible in the available wording.
The listing does not make the minimum education or experience threshold clear enough yet.
Sponsorship and relocation support are not clear from the current listing.
Part of the application route can be assessed, but the full external process may contain additional steps.
At least one recent enabled source can still renew this vacancy.
This fingerprint summarizes what is clear, constrained or still unanswered in the current listing. It is a reading aid, not an overall job score.
Decision evidence check
5 of 5 core facts stated
This checks whether the listing states the facts that commonly change an employment decision. It measures disclosure completeness, not whether the opportunity is good or bad.
Facts already stated
- Compensation
- Work arrangement
- Advertised location
- Application closing time
- Employment type
Core evidence complete
All five core facts are stated. Other details can still matter, so continue with the qualification, mobility, employer and application checks below.
Advertised location is not proof of work authorization, sponsorship, or legal eligibility. Unknown information is not a negative score; verify those questions separately.
Decision levers
What could make or break the fit?
Reasons to look closer
- Location flexibility looks unusually broad: The available wording supports high Remote Freedom without a material geography, office-attendance or timezone restriction showing up.
Trade-offs to weigh
No major trade-off is stated clearly in the information available here. Employment terms outside the listing may still matter.
Questions worth asking before you invest more time
- Confirm whether a degree is actually required, preferred, or replaceable with equivalent experience.
- Confirm the actual minimum experience requirement rather than assuming preferred background is mandatory.
- Ask whether visa sponsorship is available before investing significant time in the application.
These questions come from details the listing leaves open. An unanswered point is not treated as a disadvantage.
Opportunity neighborhood
Keep your options open
These live roles sit close enough to this opportunity to make useful comparison points. Look across several employers before deciding what pay, flexibility or access trade-offs you are willing to accept.
Senior Data Engineer, Ads Data (AI Native)
Life360 · Canada, USA
Why it is nearby: Same seniority band · Similar work arrangement · Same employment type
About 21% higher advertised midpoint
Sr Data Analyst
Alignment Health · Remote — Worldwide
Why it is nearby: Same seniority band · Similar work arrangement · Same employment type
About 7% higher advertised midpoint
Senior Data Scientist (Applied ML)
NerdWallet · USA
Why it is nearby: Same seniority band · Similar work arrangement · Same employment type
About 46% higher advertised midpoint
Applied AI Data Scientist
AE Studio · USA
Why it is nearby: Same seniority band · Similar work arrangement · Same employment type
About 58% higher advertised midpoint
This neighborhood is drawn from current inventory in the same broad occupation. Similarity is based on explicit listing facts; missing details do not count against a job.
Role market context
How much current choice is visible around this kind of work?
This compares the vacancy with current High Paid Job inventory in the broad Data & AI roles family. It describes visible choice and disclosure, not future demand or career quality.
200 most recent current roles sampled for the disclosure measures below.
Distinct employers in the bounded current sample, not an estimate of all employers hiring in the market.
Current sampled roles stating at least four of the five Core Decision Facts.
Requires a numeric amount, explicit ISO currency and supported pay period.
Listings with explicit enough remote, hybrid or on-site evidence to classify.
Missing deadline evidence remains unknown and is not interpreted as an open-ended application window.
Currencies with explicit pay evidence in the sample: USD (28) · EUR (11) · GBP (11) · CAD (3). Raw amounts across different currencies are not compared.
This describes current visible High Paid Job inventory in the same broad occupation family. Percentages use a bounded recent sample and are not a forecast, a complete labor-market census or an employer-quality measure.
Constraint opportunity lab
What changes the size of the current opportunity pool?
Each line applies one stricter evidence-based constraint to the same bounded current role-family sample. It helps you see the trade-off in visible choice without deciding which trade-off is right for you.
Counts only affirmative or possible sponsorship evidence, or relocation explicitly offered. Unknown is not treated as no.
A much stricter subset: only listings with explicit worldwide-remote evidence remain.
Requires amount, explicit ISO currency and a supported pay period. Missing pay evidence remains unknown.
Shows the subset with enough stated evidence for a more informed first comparison.
Counts roles explicitly classified as worldwide remote, region-restricted remote or remote with restrictions still unclear.
Only same-currency opportunities with usable annualized pay enter this comparison. Other currencies are not converted or treated as lower.
Uses only roles with a stated employment type.
Includes roles whose seniority is genuinely unspecified rather than assuming they belong to another level.
These are counterfactual filters over a bounded sample of current HPJ inventory in the same broad occupation family. They show how visible choice changes when one evidence-based constraint is applied; they do not tell you which constraint you should choose or predict hiring outcomes. The calculation uses the 200 most recent current roles from 1286 visible roles in this broad family.
Salary transparency context
Is pay disclosure typical for this kind of role?
This listing states usable structured compensation. About 27% of the bounded current role sample does the same.
Compensation counts only when amount, ISO currency and supported pay period are all known.
Pay disclosure is limited in the current role sample.
This employer’s current visible listings disclose pay materially more often than the current same-role sample.
This compares salary-disclosure behavior in current visible listings. It does not compare raw salary amounts across currencies, infer undisclosed pay as zero, or rate employer quality.
Private offer & negotiation lab
Turn an offer into evidence, not guesswork
Enter an annual base offer and, if you want, your own target. HPJ compares only within one explicit currency and never chooses a target for you.
Negotiation evidence available: advertised pay range, current same-currency market context, strong decision evidence.
Enter an annual base offer to compare it with explicit same-currency evidence. HPJ will not suggest a target for you.
Review the full package · 10 terms
Mark what is confirmed, what needs clarification and what you want to prioritize. These choices and notes stay in this browser.
Usable advertised compensation is available.
Variable compensation is mentioned; verify amount, formula, target and eligibility.
Equity or long-term incentive language is mentioned; verify grant type, vesting and valuation basis.
Retirement or pension benefits are mentioned; verify employer contribution and vesting.
Health-benefit language is mentioned; verify coverage, dependants and employee cost.
Paid-leave language is mentioned; verify the exact entitlement and carry-over rules.
A work-arrangement signal is available; verify schedule details before relying on it.
Relocation support not established
Sponsorship not established
A review cycle is mentioned; verify timing and whether compensation can change at that review.
Questions worth taking into the offer conversation
- Is there a bonus, commission or other variable-pay component, and how is it calculated?
- Does the package include equity or another long-term incentive, and what are the vesting terms?
- What retirement or pension contribution does the employer provide?
- How much paid leave is provided and what are the carry-over or blackout rules?
- If relocation is relevant, what expenses, timing and repayment conditions are covered?
- What exact sponsorship or work-authorization conditions apply to this role?
- When is the first performance and compensation review after joining?
Occupation intelligence
What broader career references connect to this role?
HPJ uses broad comparison families for current-market analysis. External taxonomy results are reference candidates for that broad family, not a declaration that this vacancy has been authoritatively classified into a specific occupation.
European occupation references
This service uses the ESCO classification of the European Commission.
U.S. occupation references
- Data Scientists 15-2051.00
- Computer and Information Research Scientists 15-1221.00
- Social Science Research Assistants 19-4061.00
- Atmospheric and Space Scientists 19-2021.00
- Bioinformatics Scientists 19-1029.01
This site incorporates information from O*NET Web Services by the U.S. Department of Labor, Employment and Training Administration (USDOL/ETA). O*NET® is a trademark of USDOL/ETA.
Career reference research
Skills worth investigating
Official occupational context, paired with exact phrases visible in this vacancy. It is not a skills test or an applicant-fit score.
ESCO · European occupation
data scientist
Usually essential in this ESCO occupation
- interact professionally in research and professional environments
- promote open innovation in research
- synthesise information
- write scientific publications
- draft scientific or academic papers and technical documentation
- apply for research funding
- data sciencePhrase in listing
Context-dependent ESCO skills
- make data-driven decisions
- computational biology
- computer simulation
- manage data
- data quality assessment
O*NET · U.S. occupation
Data Scientists
O*NET occupation skills
- Mathematics
- Critical Thinking
- Reading Comprehension
- Complex Problem Solving
- Programming
- Active Learning
- Speaking
- Active Listening
Build my private skill preparation brief Your evidence · Practice · Interview questions
Plan how you might demonstrate or investigate occupational skills. These are broad-family reference examples, not confirmed vacancy requirements. Nothing you enter is sent to High Paid Job.
Private to this browser. Up to 20 reference skills; not a qualification assessment or hiring prediction.
Useful next question: Which of these occupational reference skills does the hiring manager actually expect for this specific role, and what evidence would demonstrate them?
These skills belong to one reference occupation selected for HPJ’s broad career family, not a verified classification of this vacancy. Text overlap only means a phrase appears in the available listing text; it does not establish an employer requirement, your proficiency, or a qualification gap. A missing mention is unknown, not a deficit.
Adjacent current roles to investigate
Advisory Solutions Architect, MSP (New Product Intro)
Rubrik Job Board · Anywhere in the World
same seniority band · same employment type · similar work-arrangement evidenceLead Product Manager, Engagement
Lattice · Anywhere in the World
same seniority band · same employment type · similar work-arrangement evidenceLead Product Manager, Engagement
Lattice · Anywhere in the World
same seniority band · same employment type · similar work-arrangement evidenceAdjacent-family roles are exploration prompts from current HPJ inventory. They are not inferred qualification matches and they do not imply a promotion path.
Career move map
What sits one step away?
This listing is currently read as Senior / lead. Nearby roles one visible seniority step away can help you judge whether to widen the search, stretch upward or compare the responsibilities more carefully.
Broader-entry direction
F-35 Data Analyst Mid - 31190 (Arlington, VA, Virginia, United States)
HII · Remote — Worldwide
Similar work arrangement · Same employment type · Recently posted Advertised midpoint is about 25% lowerBusiness Intelligence Analyst / Data Engineer
GEICO · Remote — Worldwide
Similar work arrangement · Same employment type · Recently posted Advertised midpoint is about 6% higherStretch direction
Staff Data Product Manager (Machine Learning)
NerdWallet · USA
Similar work arrangement · Same employment type · Recently posted Advertised midpoint is about 46% higherSr. Data Scientist, Forward-Deployed
6sense · USA
Similar work arrangement · Same employment type · Recently posted Advertised midpoint is about 51% higherThese are live roles one visible seniority step away in the same broad occupation family. They can reveal nearby options, but they are not a guaranteed promotion path and do not account for your personal experience.
Skill bridge
What does this role connect to?
Large Language Models is one of the signals this listing shares with skills appearing often across current Data & AI roles opportunities.
Skills named in this listing
Shared with the current career sample
- Large Language Models Seen in 43 sampled listings
- Machine Learning Seen in 51 sampled listings
- Python Seen in 90 sampled listings
- SQL Seen in 75 sampled listings
Combinations also appearing together
- Python + SQL appeared together in 55 sampled listings
- Machine Learning + Python appeared together in 42 sampled listings
- Large Language Models + Python appeared together in 38 sampled listings
Application edge
- If these skills match your background, prepare one concrete example that shows how you used Large Language Models, Machine Learning, and Python rather than relying on a keyword-only claim.
- The listing mentions extra application steps. Review those requirements before starting so you can prepare the requested material in one pass.
If the named skills are genuinely transferable in your own experience, the occupation map also points to adjacent current families worth researching: Software engineering roles and Product roles. This is exploration context, not a qualification match.
Skill signals come from explicit text mentions. A mention is not automatically a requirement, and absence from the current top-skill list does not make a skill rare or unimportant. The current career comparison draws on 120 sampled listings; 119 contain a recognized skill signal.
Application Decision Lab
Turn the listing into a preparation plan
Separate what the listing actually states from what you still need to prepare or verify. Your selections and short notes stay in this browser and are never submitted to High Paid Job.
Large Language Models
This skill or technology is explicitly mentioned in the available listing text. A mention is not automatically a mandatory requirement; prepare a concrete example only if it is relevant to your experience.
Machine Learning
This skill or technology is explicitly mentioned in the available listing text. A mention is not automatically a mandatory requirement; prepare a concrete example only if it is relevant to your experience.
Python
This skill or technology is explicitly mentioned in the available listing text. A mention is not automatically a mandatory requirement; prepare a concrete example only if it is relevant to your experience.
SQL
This skill or technology is explicitly mentioned in the available listing text. A mention is not automatically a mandatory requirement; prepare a concrete example only if it is relevant to your experience.
Prepare for a stated application step
A background or pre-employment screening step is explicitly mentioned.
Prepare for a stated application step
A drug or substance screening step is explicitly mentioned.
Questions worth resolving before you rely on the opportunity
- Work authorization / sponsorship: If sponsorship or work authorization matters to you, what exact immigration or employer-sponsorship conditions apply to this role?
Decision analysis
What the listing really tells you
For Senior Data Scientist (AI Evaluation & Improvement) at Evolent, the most useful clues right now are compensation has enough depth for comparison, work-location flexibility has usable detail, and the visible application effort can be estimated. Looking at them together makes the trade-offs easier to see.
Compensation has enough depth for comparison
The annualized midpoint sits near the 22th percentile within 96 comparable USD salary observations. Treat that as current advertised-pay context rather than a forecast of take-home or total compensation.
Work-location flexibility has usable detail
The work-location pattern is “Highly location-flexible”, with Remote Freedom at 96/100.
The visible application effort can be estimated
Provider-mediated application path · 2 visible process steps. The external application may contain additional steps that are not visible from the listing itself.
The listing can be traced to a current source
The opportunity currently has 1 renewable source path across 1 source classes. Multiple independent paths can strengthen continuity, but they are not a substitute for checking the application destination.
How this analysis is assembled
Built from the pay, location, qualification, mobility, application and source details available for this vacancy. Anything the listing leaves unresolved stays unresolved.
Analysis evidence coverage
How much analysis can the current evidence support?
Uses the explainable High Pay Score within the job’s current occupation/seniority/currency comparison sample.
Measures observable location flexibility and explicit restrictions; it does not determine whether remote work is personally desirable.
The configured source supplied structured salary fields, a usable currency and a focused comparison value.
The listing was posted about 1 days ago.
At least one enabled source recently observed the vacancy. The source chain is traceable, but this is not independent employer verification.
There is no universal opportunity score because pay, flexibility and access matter differently to different people. Compare Jobs lets you weight the evidence around your own priorities.
Decision checklist
What is supported, uncertain or worth checking?
This checklist separates supported strengths, visible cautions and missing evidence. It is not an overall recommendation or guarantee that the job is suitable for you.
Supported strengths
- The available work-location evidence indicates strong location flexibility.
Visible cautions
- The description appears to mention an advanced degree requirement or preference.
Still unknown
- Visa sponsorship is not established by the available listing data.
Pay vs. barriers
Pay and barrier context available
The current pay position can be viewed alongside the stated barriers, but the available detail does not support a stronger conclusion.
Comparable disclosed-pay evidence places this job at about 22 / 100 within its current occupation/seniority sample.
- Education requirement is not established.
- Required prior-experience level is not established.
This section weighs pay context against clearly stated barriers without collapsing unlike requirements into one universal score. Degree, experience, licensing, clearance and location limits remain separate, and missing qualification details stay unknown.
Beyond the headline
Not enough evidence for a hidden-value finding
Potential hidden value appears only when several distinct non-pay advantages are actually supported. Missing requirements or benefits are never counted as positives.
The observable Remote Freedom evidence is strong and contains few explicit restriction signals.
Still check: The current application path is provider-mediated rather than directly classified as an employer-authorized or official route.
Hidden Value is an experimental evidence summary, not an investment-style score or guarantee of employability. It only uses explicit current listing evidence and does not infer that an unmentioned qualification is unnecessary.
Explainable opportunity highlight
Why this opportunity stands out
These are supported signals from the current listing and comparison sample—not a guarantee that this is the best job for you.
The source provides structured salary information with a comparatively focused range, improving the usefulness of pay comparisons.
The observable work-location signals indicate unusually strong location flexibility.
The listing was posted about 1 days ago.
Compensation intelligence
How competitive is this pay?
Below many comparable disclosed salaries
The annualized midpoint is below the 25th percentile boundary of the current comparable sample.
The highlighted band is the middle 50% of the current comparable sample after the outlier guard. The dark marker is this job’s annualized salary midpoint.
Why this analysis?
- This score reflects the current advertised-pay distribution for comparable active jobs in the same currency. It measures pay position, not overall job quality.
- Salary confidence: High. The configured source supplied structured salary fields, a usable currency and a focused comparison value.
- Range quality: Focused salary range.
Employer compensation communication
Pay transparency at Evolent
Good current pay transparency
This score uses the employer’s current visible listings and combines how often pay is disclosed with how useful those ranges are. Bonus, equity and benefit mentions are shown separately and do not raise the score.
This pattern comes from the employer’s current visible vacancies. It describes hiring and pay transparency in those listings, not employee satisfaction or a permanent judgement of the employer.
Current employer intelligence
Evolent — current opportunity snapshot
Employer Evidence Profile
How often this employer’s current visible sampled vacancies state decision-critical information. These percentages describe evidence coverage, not employer quality.
Current sampled listings with advertised compensation evidence.
Current sampled listings whose work arrangement is clearer than unknown.
Current sampled listings with a supported country signal.
Current sampled listings with an explicit closing time.
Current sampled listings with a usable posting date.
Current sampled listings that explicitly indicate confirmed, possible or unavailable sponsorship evidence.
Current sampled listings with an explicit relocation signal.
Low coverage means the current visible listings state the information less often. It does not prove that the employer lacks the policy, benefit or support.
Common current occupation families: Data & AI (1) · Healthcare (1) · Software engineering (1)
Visible seniority mix: Senior / lead (2) · Leadership (1)
Current employment-type mix: Full time (3)
Other current roles from this employer
Radiation Oncology Field Medical Director
Remote — Worldwide
Leadership · Worldwide remoteCore Senior .NET Software Engineer
Remote — Worldwide
Senior / lead · Worldwide remoteThese are other current vacancies visible from the same employer, not endorsements or evidence of future hiring plans.
There are not yet five current disclosed salaries in one currency, so an employer pay median would be too thin to be useful.
Questions worth asking this employer
- If work authorization matters to you, what sponsorship policy applies to this specific role and location?
- If a move is required, is relocation assistance available for this position?
These questions are prompted by missing or thin evidence in the employer’s current visible vacancies. Missing evidence is not treated as a negative judgement.
This employer snapshot uses the employer’s current visible listings. Remote and sponsorship rates use only vacancies with explicit evidence, while missing information stays unknown. Salary medians are shown separately by currency and need at least five disclosed salaries. Zero confirmed sponsorship means none was explicitly found in this sample, not that sponsorship is unavailable.
This snapshot describes the employer’s current visible vacancies—things such as pay disclosure and work arrangement. It is not an employee-satisfaction or hiring-quality rating, and different currencies stay separate.
Work-location intelligence
Remote Freedom
Highly location-flexible
Why this score?
- The source describes the role as open worldwide or globally.
Flexibility signals detected
- Worldwide or global remote eligibility is explicitly indicated.
Remote Freedom measures observable location flexibility only. Unknown work-location information remains unscored. Timezone wording is penalized only when it appears restrictive; general collaboration across time zones is not treated as a restriction. A score does not guarantee legal work eligibility, payroll availability or visa sponsorship.
International applicant intelligence
Visa & relocation signals
Sponsorship status is shown only when the listing provides clear enough wording. Employer history or headquarters location is not used to guess an answer.
The available listing details do not contain a clear enough sponsorship statement.
The available listing details do not contain a clear enough relocation statement.
Tradeoff check
What’s the catch?
These are visible conditions worth checking before treating the headline compensation as the whole story:
- The description appears to mention an advanced degree requirement or preference.
Current alternatives
Better-paying alternatives
These current roles show a materially higher annualized salary midpoint in the same broad occupation and currency. Similar seniority, country, work arrangement, employment type and recency are preferred where possible; higher pay alone does not make an alternative a better fit.
Applied AI Data Scientist
AE Studio · USA
Region-restricted remoteWhy comparable: Same seniority band · Similar work-location arrangement · Same employment type
Lead Data Scientist
Life360 · Canada, USA
Region-restricted remoteWhy comparable: Same seniority band · Similar work-location arrangement · Same employment type
Senior Data Scientist (Applied ML)
NerdWallet · USA
Region-restricted remoteWhy comparable: Same seniority band · Similar work-location arrangement · Same employment type
Alternatives are drawn from the current inventory rather than presented as application recommendations. A pay difference needs to be meaningful—about 3% or more at the midpoint—and different currencies are not compared.
Application-path intelligence
What can we see about the application process?
Provider-mediated application path
The current application path comes through a publisher or aggregator. The destination may still lead to an employer-authorized hiring system, but that is not clear enough to assume.
Current destination: himalayas.app · HTTPS
Visible application steps or requirements
- A background or pre-employment screening step is explicitly mentioned.
- A drug or substance screening step is explicitly mentioned.
Application effort is estimated from visible listing and source details. Optional steps are kept separate from mandatory ones, and the external form may contain additional requirements that are not visible here.
Application safety context
Application safety check
Basic application-safety context available
The current application destination has a basic transport-safety signal such as HTTPS. This does not verify the employer, vacancy, recruiter or application process.
Evidence checked: Source provenance · Application destination · Listing wording
Observable transport-safety signals
- The available application destination uses HTTPS.
Before you share money or sensitive information
- Confirm the vacancy on the employer’s official careers site when practical, especially if a recruiter contacts you outside the listed application path.
- Check that recruiter email domains and application destinations are consistent with the employer or an identifiable recruiting provider.
- Do not pay application, processing, training or equipment fees merely to be considered for a job.
- Do not send cryptocurrency, gift cards, online-banking credentials or PINs as part of an application.
- Treat unusually fast offers or interviews conducted only through informal messaging channels as reasons to verify the employer independently.
This is a limited safety screen, not employer verification and not a fraud determination. Never send money, cryptocurrency, passwords or sensitive banking credentials merely because a listing appears on a job platform.
Original listing details
The section below comes from the listing source. The decision sections above use the job’s structured pay, location, qualification, mobility and application details to give you additional context.
Your Future Evolves Here
Evolent partners with health plans and providers to achieve better outcomes for people with most complex and costly health conditions. Working across specialties and primary care, we seek to connect the pieces of fragmented health care system and ensure people get the same level of care and compassion we would want for our loved ones.
Evolent employees enjoy work/life balance, the flexibility to suit their work to their lives, and autonomy they need to get things done. We believe that people do their best work when they're supported to live their best lives, and when they feel welcome to bring their whole selves to work. That's one reason why diversity and inclusion are core to our business.
Join Evolent for the mission. Stay for the culture.
What You’ll Be Doing:
Core Responsibilities
- Own the diagnostic loop for LLM-based clinical services: take failure modes surfaced by clinical reviewers and product managers, form root-cause hypotheses (prompt design, context assembly, retrieval, guideline encoding, model behavior, upstream data), and design experiments that isolate the cause.
- Test candidate fixes — prompt and configuration variants, context changes, model alternatives — and verify improvements with structured evaluations, not anecdotes; confirm fixes don't regress other behavior.
- Own the evaluation roadmap and quality bar for the team's AI services: which metrics gate deployment, how golden sets and regression suites grow, and what "good enough to ship" means in evidence.
- Use and extend the team's evaluation platform: build golden sets and regression suites, define metrics (accuracy, guideline adherence, grounding/faithfulness), run and interpret eval batteries. Fluency in *using* modern eval tooling matters; the platform exists — extending it thoughtfully is in scope, rebuilding it is not.
- Partner with clinical reviewers (medical directors) to turn review findings into labeled evidence and executable evaluation criteria; partner with product managers to prioritize which failure modes matter most.
- Mentor others in evaluation methods and grow the evals function as it scales, including readiness to take direct reports as the team expands.
- Support the annual clinical-guidelines update cycle with regression evaluation as guidelines, prompts, and models change; ramp with the team's senior data scientists in Q4 2026.
- Document the diagnostic playbook: failure taxonomies, experiment templates, variant history — a method others can run, not a private intuition. This is a practicing role — the diagnostic loop is the job, at every level of seniority.
- Handle clinical data (including PHI) according to organizational security, privacy, and compliance requirements.
Minimum Requirements
- Bachelor's degree in Data Science, Computer Science, Statistics, or a related quantitative field — or equivalent experience.
- 5+ years of data science or applied machine learning experience, including shipping and maintaining models or AI systems in production.
- 2+ years of recent, hands-on experience evaluating and improving LLM-based systems: structured error analysis, prompt/configuration iteration, experiment design, metrics interpretation — as a practitioner, not only as a reviewer of others' work.
- Demonstrated experience with LLM evaluation methods and tools — golden/regression sets, LLM-as-judge with validation, tracing and observability tooling.
- Strong Python and solid data-analysis skills (SQL a plus); comfort computing and reasoning about metrics such as sensitivity, specificity, and PPV.
- Hypothesis-driven working style: the instinct to isolate variables and prove a fix, rather than tweak and hope.
- Strong written communication — findings and go/no-go evidence must be legible to engineers, clinicians, and leadership.
Preferred Requirements
- Healthcare experience: utilization management, prior authorization, clinical documentation, or clinical/claims data; comfort reading clinical guideline and medical-policy content.
- Experience mentoring data scientists or leading small technical workstreams; interest in growing into people leadership as a function scales.
- Experience working with clinical reviewers or other domain experts to convert expert judgment into labeled data and evaluation criteria.
- Experience with LLM observability/telemetry stacks (OpenTelemetry-based tracing, Logfire, Langfuse, or similar).
- Statistics or experimentation background (A/B testing, statistical significance, sample-size reasoning).
- Master's degree in a quantitative field.
#EvolentAi
To ensure a secure hiring process we have implemented several identity verification steps, including submission of a government issued photo ID. We conduct identity verification during interviews, and final interviews may require onsite attendance. All candidates must complete a comprehensive background check, in-person I-9 verification, and may be subject to drug screening prior to employment. The use of artificial intelligence tools during interviews is prohibited and monitored. Misrepresentation will result in immediate disqualification from consideration.
Technical Requirements:
We require that all employees have the following technical capability at their home: High speed internet over 10 Mbps and, specifically for all call center employees, the ability to plug in directly to the home internet router.
Evolent is an equal opportunity employer and considers all qualified applicants equally without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran status, or disability status.If you need reasonable accommodation to access the information provided on this website, please contact for further assistance.
The expected base salary/wage range for this position is $120,000 - 145,000. As part of our total compensation package, Evolent is proud to offer comprehensive benefits (including health insurance benefits) to qualifying employees. All compensation determinations are based on the skills and experience required for the position and commensurate with experience of selected individuals, which may vary above and below the stated amounts.Originally posted on Himalayas
Listing provenance
Where did this job come from?
See where this vacancy came from and whether more than one current source can still support it. Source names help with traceability and attribution; they are not endorsements or guarantees.
At least one enabled source recently observed the vacancy. The source chain is traceable, but this is not independent employer verification.
- Continuity
- One recent renewable source
- Source-class independence
- One renewable source class
- Last source observation
- Observed in the past 3 days
This records when High Paid Job last observed the vacancy in source evidence. It confirms source presence, not that the employer edited the vacancy at that time.
- 1 enabled source recently observed this vacancy.
Source confidence is about traceability and continuity. It is separate from application safety and should not be read as an employer reputation rating.
The vacancy is supplied by an external job publisher. Source information helps show where the listing came from; it is not a guarantee about the employer or vacancy.
Listing source: Himalayas. Destination: himalayas.app. The application opens in a new tab so you can keep this report open.
