OpenAI's LifeVal Judges AI on Real-Life Errands, Not Just Coding

OpenAI has introduced LifeVal, an internal evaluation designed to measure how well its models handle personal tasks such as finding a school after a move, planning meals for a relative with diabetes, or coordinating volunteers before a flood. The company says the benchmark has informed model development for about a year.

LifeVal targets a gap in conventional AI testing. Coding, mathematics, and browser benchmarks usually have clear completion criteria, while personal errands depend on preferences, budgets, schedules, location, and other user-specific constraints. A model can complete the required clicks and still produce an unsuitable result.

From time diaries to test cases

OpenAI began with time-use datasets, including the US Bureau of Labor Statistics’ American Time Use Survey and the Multinational Time Use Study. Many common activities, such as sleeping and eating, offered little scope for useful delegation, so the company gathered requests directly from users.

OpenAI says it surveyed roughly 4,000 people, then asked a smaller group of about 100 participants to record tasks they wanted an AI assistant to perform. Participants also supplied the personal context required for each request. A useful school search, for example, may depend on an address, tuition limit, commute, operating hours, and a child’s needs.

Grading the outcome

LifeVal covers tasks that may require a model to interpret chat history, navigate websites, combine information from several sources, and recommend an outcome that fits the user’s circumstances. Its scenarios include:

• Apartment searches and relocation logistics
• Childcare and school selection
• Government services and online forms
• Household finances and budgeting
• Grocery shopping and meal planning
• Care for sick or aging relatives
• Community planning and volunteer coordination

Scenario-specific rubrics replace a single correct answer. A school-search rubric can evaluate tuition, distance, schedule, eligibility, and requested services, while a grocery task can assess dietary restrictions, price, quantity, and whether the selected products are available. This approach measures whether the result satisfies the request rather than whether the agent merely completed a sequence of actions.

An internal scoreboard

LifeVal remains an OpenAI-controlled evaluation rather than a reproducible public benchmark. Outside researchers cannot inspect the full task set, hidden test split, rubric weights, grader calibration, browsing environment, or retry policy. Repeated use during model development also makes the separation between tuning tasks and held-out tests important.

Model percentages require additional context to support a fair comparison. Relevant settings include the exact model snapshot, reasoning effort, tool access, token budget, latency, number of attempts, and scoring method. Selecting the best result across several effort levels produces an upper-bound score that may carry higher production costs than a fixed deployment configuration.

Claims that a model called “GPT-6 Astra” achieved specific LifeVal scores are unsupported by the cited OpenAI material. The linked benchmark information does not establish that model or the percentages attributed to it. Verified comparisons will require results tied to documented model versions and reproducible test settings.

From workplace output to home logistics

LifeVal complements GDPval, OpenAI’s evaluation of economically valuable work across occupations and industries. GDPval examines outputs that resemble professional assignments, while LifeVal focuses on household decisions, errands, care work, and personal administration.

Computer-use benchmarks such as OSWorld and WebArena primarily test whether an agent can operate software and complete browser workflows. LifeVal extends the evaluation to the suitability of the final outcome. That distinction matters for agents that can submit forms, place orders, or act on stored personal information.

Turn the benchmark into product tests

Teams building assistants can apply LifeVal’s evaluation pattern without access to the private benchmark:

1. Collect representative requests with consent. Preserve the constraints that determine a useful answer, while minimizing the personal data stored for testing.
2. Write scenario-specific rubrics. Score hard requirements, preference matching, source quality, task completion, and unsupported assumptions separately.
3. Fix the runtime configuration. Record the model version, system prompt, tools, reasoning setting, retries, token use, latency, and browsing environment.
4. Test stale and conflicting context. User profiles and chat histories can contain outdated preferences, missing details, or contradictory instructions.
5. Require confirmation for consequential actions. Purchases, financial decisions, government submissions, and health-related plans need clear review steps and recovery paths.

Production evaluations should also track tool failures, user corrections, abandoned tasks, unsafe actions, latency, and cost. Those measures show whether improvements on an internal benchmark transfer to a specific product, workflow, and user population.