Will My Work Reduce If AI Works All Night? — The Verification Costs Shown by Claude Duet and METR Experiment

The time it takes for humans to verify the results even after AI has finished the work

On October 21, 2026, Antropic announced a research preview of Claude Duet. It is an AI agent that continues work even when the user is away, monitors connected apps, and notifies the user in advance if there is work to be done. Antropic explained that it is designed to obtain user approval before sending messages, making payments, signing, or deleting items. Initially, they are accepting waitlist applications from Pro and Max subscribers in the United States. ¹

With the release of a product like this, I begin to hope that my workload will decrease. After all, I can simply submit the work before leaving for the day and receive the results the next morning. However, if I have to review the results from scratch and correct errors in the morning, can I really say that I saved time thanks to the agent who worked all night?

I frequently ask myself this question while developing a business for AI agents. This is because even after the AI completes its work, a significant amount of time is required for humans to verify the results before they can be trusted and used. In this article, I intend to examine research on AI productivity by drawing an analogy between a new employee who writes a report quickly and a team leader who reviews it. I will also consider how the costs associated with review and revision affect corporate profits, inflation, and the return on investment for AI.

Just because an employee submits a report early does not mean the team's workload decreases.

Even if an employee submits a report, the team leader still has the task of reviewing and revising it. Therefore, to assess team productivity, one must look at the total time taken until the report can be sent to the client. Even if an employee writes a report in just 10 minutes, if the team leader spends a long time verifying numbers and sources and making corrections, the time saved for the team as a whole is less than expected.

For example, let's assume there is a report that takes 3 hours for a team leader to write and review alone. If it takes 20 minutes to explain the work to a new employee, 10 minutes for the employee to write the report, and 30 minutes for the team leader to review and revise it, the total is 1 hour. Assuming the quality is the same, 2 hours have been saved. Conversely, if it takes 4 hours just to review and revise, it takes 4 hours and 30 minutes including the explanation and writing time. In this case, more time was actually spent than when working alone. If an employee and a team leader work simultaneously, the time taken to complete the work and the actual time worked by the two people must be measured separately.

The same applies when entrusting tasks to AI. You must add the time spent explaining the work to the AI, as well as the time spent reviewing and correcting the results, to determine if the workload has truly decreased. On top of this, there are costs for AI usage fees and system integration, as well as expenses for resolving errors if they are discovered later. Until the quality is verified, the work cannot be considered properly completed simply because it finished quickly.

2Even if you feel like the work has gotten faster, it may actually take longer when you time it.

Even experienced workers sometimes struggle to accurately gauge how much time has been saved thanks to AI. This was demonstrated in an experiment released in July 2025 by the AI evaluation agency METR. Sixteen skilled open-source developers participated in the experiment. The researchers collected 246 tasks from the projects the developers typically worked on and randomly decided whether or not to allow the use of AI for each task. The tools primarily used by the developers at the ^time were Cursor and Claude 3.5 and 3.7 Sonnet.

As a result, tasks where AI could be used took 19% longer to complete. However, even after the experiment ended, participants believed that time had been reduced by about 20% thanks to AI. In the case of reports, they felt that work became easier because they received drafts quickly, but in reality, their total work time had increased. The 19% figure represents the result of comparing total work time that includes the time spent typing prompts, waiting for AI responses, and reviewing and correcting results.

In follow-up data released by METR in February 2026, estimates showed the opposite: work time decreased. For developers who re-participated in the first experiment, it was estimated that time spent on AI-assisted tasks decreased by approximately 18% . However, the margin of error (confidence interval) was wide, ranging from a 38% reduction to a 9% increase. ³

The research team also stated that it is difficult to trust these estimates at face value. This is because an increasing number of developers dropped out of the experiment, stating they did not want to work without AI, and there were many cases where tasks they did not want to perform without AI were not included in the experiment at all. The researchers believed that the effect could be greater if these dropped developers and tasks were taken into account. For developers running multiple agents simultaneously, it was also difficult to measure the time taken for each task.

3Even plausible-looking reports can be wrong.

There are many cases where results that pass automated checks cannot be used as is in practice. In the context of reports, this occurs when tables and sources are fully included, but the calculation standards are incorrect or the conclusions do not match the data.

In a research note released in March 2026, METR had four developers managing open-source projects review 296 AI-generated code revisions. Even after adjustments were made to account for the fact that human-written revisions that had previously been accepted are sometimes rejected during re-evaluation, the conclusion was that approximately half of the AI revisions that passed the automated tests were difficult to accept as they were. In some cases, they violated the project's coding rules, failed to properly resolve the original problem, or broke other functions. Although they passed the tests, the managers determined that further fixes were necessary. In this experiment, the AI did not have the opportunity to hear the managers' feedback and make further revisions. ⁴

Similar results were found in experiments where experts worked with AI. In an experiment involving 758 BCG consultants, those assigned tasks that GPT-4 excels at completed their work an average of 25.1% faster and produced better quality deliverables when using the AI. Conversely, for management decision tasks that AI struggles with, the accuracy rate of the group that did not use AI was 84.5% , whereas the two groups that did use AI scored only 60% and 70.6% . When combining the two groups that used AI, the accuracy rate was approximately 19 percentage points lower on average. ⁵

However, regardless of whether the answers were correct, the group using AI received higher scores from evaluators for logical consistency and persuasiveness. The same was true even when comparing only the incorrect answers. Those who used AI wrote even their incorrect answers more plausibly.

Even a team leader who is well-versed in the business can be fooled by such a plausible-sounding report. Therefore, you must distinguish between tasks to entrust to AI and those not to, and prepare verification methods in advance to detect errors when results are incorrect.

4If an employee reports that they have finished work they were unable to complete, the team leader misses the opportunity to verify it.

Whether the work was done properly and whether the report is accurate must be verified separately. Even if an employee has not finished the work, the team leader can continue processing it if they inform the team leader of how far they have progressed and what remains. The problem arises when an employee reports that they have verified numbers they have not actually checked. In such cases, the team leader moves on without reviewing those numbers again.

OpenAI disclosed in March 2026 how it monitors coding agents used internally. According to OpenAI, there were instances where agents reported tools used or execution results inaccurately, or claimed to have completed tasks that were not. Cases where agents spoke as if certain in the final response, despite lacking confidence during the reasoning process, were also classified separately. ⁶

Therefore, those using the agent must first verify whether the report stating "finished" matches the actual results. If it is a report, there must be the raw data; if it is an email, there must be a record of its dispatch. If the program was modified, there must be a record of which tests were run and which passed or failed.

Pressing the approval button cannot substitute for verifying results. If a team leader approved a request without thoroughly reviewing the content, the report's accuracy is not guaranteed simply because it went through the approval process. Reading the agent's explanation and authorizing execution is a separate matter from verifying whether that explanation aligns with the actual work records.

To review the results of 5AI, human skills must also be continuously improved.

Producing a large volume of deliverables does not necessarily mean you understand the work better. An employee who submits only reports written by others may increase their submission volume, but they are likely to be unable to identify where to make corrections when the numbers appear odd.

In a study released by Antropic in January 2026, 52 software engineers, mostly junior developers, solved a task using a Python library they were using for the first time. In a comprehension test taken immediately after completing the task, the group using AI scored an average of 50 points , while the group that coded manually scored 67 points , resulting in a difference of 17 points. The difference was greatest in the debugging questions, which required identifying errors and explaining their causes. Although the group using AI finished the task about two minutes faster on average, this was not a statistically significant difference. Since the test was taken only once, this study cannot determine long-term changes in performance. ⁷

To review results generated by AI, the reviewer must first have a thorough understanding of the task. In the same study, those who did not simply accept and use the code provided by the AI but instead asked for explanations and tried to understand it performed relatively well.

Conversely, there are also workplaces where AI has been of great help to beginners. A study published in an economics journal in 2025 analyzed work data from 5,172 customer service representatives. After introducing AI response support tools, the average number of inquiries resolved per hour increased by 15% , and both processing speed and response quality improved for employees with less experience and lower proficiency. In this workplace, employees were able to modify or choose not to use the responses suggested by the AI. ⁸

The two studies differ in both tasks and conditions. Therefore, when comparing results, one must examine not only human proficiency but also how AI was introduced into the work.

If you have implemented NUM5 AI, you should first check if the entire team actually worked less.

If the time and cost required to achieve the same quality can be reduced and the risk of errors can be managed to a manageable level, there is ample reason to adopt AI. Since humans make mistakes, the point of comparison is not an AI that does not make mistakes, but rather the way the team currently works.

To compare writing a report alone versus entrusting it to AI, a team leader must first determine when the work is considered finished. The criteria vary depending on whether it is a draft requiring only submission or a final version to be sent to an external client. Next, measure the time and costs spent on directing, reviewing, and revising, and compare the results over a certain period, including errors discovered later. It would be helpful to consider the following four points together during this process.

Since humans can perform other tasks while AI is working, this factor must also be taken into account. If the time spent waiting for the AI is added to the total time worked by humans, the effectiveness of the AI is underestimated. Conversely, if the review cost is set to zero because no one has looked at the results, undiscovered errors are mistaken for cost savings.

Even if the 7 team's costs are reduced, prices and stock prices must be considered separately.

Even after a company reduces operational costs using AI, it must go through several more steps before prices stabilize and investors make a profit. The emergence of one company that produces reports cheaply does not mean that costs will decrease for all companies simultaneously. Nor will the price of reports drop immediately.

If productivity increases across various industries, leading to lower costs, and companies subsequently lower or raise prices less, inflation can be tempered. However, as the use of AI expands, investments in supporting data centers and power infrastructure become necessary. This process increases demand for equipment and electricity, as well as the need for more construction personnel. From the perspective of the economy as a whole, the funds spent on investment and the resulting demand may rise before the capacity to produce more goods and services (supply capacity) can even expand.

Federal Reserve Governor Lisa Cook also addressed this time lag in a speech on September 28, 2026. Governor Cook viewed the demand for AI-related investments as currently driving up prices. She explained that the effect of increased productivity easing inflation occurs when supply capacity grows faster than demand, and the magnitude of this depends on how quickly AI spreads and how much companies pass on cost savings to prices. She further predicted that this effect would only slightly lower inflation rates over the next few years and would not appear quickly enough to offset the growing inflationary pressures expected within this year. ⁹

There are specific conditions attached to investors making a profit. Even if client companies save costs using AI, the profits these companies leave may be small if AI service providers engage in price competition among themselves. Furthermore, even if a profit is generated, investors may not earn as much as expected if the stock price has already risen in anticipation of greater growth. Therefore, one must carefully evaluate whether the technology is useful, whether the company generates revenue from it, and at what price the stock is being purchased.

8Frequently Asked Questions

Conclusion — You should count reports sent to customers rather than reports submitted by employees.

When an employee joins and writes reports quickly, the team leader expects their own workload to decrease. However, whether the team leader's work has actually decreased can only be determined by measuring the time it takes to review, revise, and send the report to the client.

In the previous article , "How Much Does an AI Data Center Need to Break Even?", we examined how much revenue is required to recoup the investment in an AI data center, and in "Can AI Companies Manage Debt Until Future Revenue Becomes Cash?", we looked at whether AI companies can bear the burden of debt until that money comes in. This article starts from the perspective of the customer who purchases and uses AI services. This is because customers need to have a reason to continue paying for AI usage fees only if costs are reduced or results improve, even when including the time spent on review and modification.

I believe that AI can increase productivity. However, whether its effects will spread widely enough to stabilize prices, and whether AI companies will generate profits commensurate with their current investment scale and stock prices, are separate issues that need to be considered.

There are three things I intend to monitor going forward: whether the time and cost required to produce the same quality have actually decreased, whether the effects persist beyond the pilot phase, and how the cost savings achieved by customers are reflected in the AI company's cash flow and consumer prices. When I hear that AI has completed a task, I plan to first ask the person who received and wrote the results whether they truly saved time and achieved a better result within the same amount of time.

Reference materials

1. Anthropic, Introduction to Claude Duet (2026.10.21) · Claude Support, Get started with Claude Duet
2. METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (July 10, 2025) — Randomized Controlled Trial · Pre-publication Paper
3. METR, We are Changing our Developer Productivity Experiment Design (2026.02.24) — Measurement and Sample Selection Issues in Follow-up Research
4. METR, Many SWE-bench-Passing PRs Would Not Be Merged into Main (2026.03.10) — Research Note
5. Dell'Acqua et al., Navigating the Jagged Technological Frontier, Organization Science (March/April 2026; first published 2023)
6. OpenAI, How we monitor internal coding agents for misalignment (2026.03.19) — Developer's own observation
7. Shen·Tamkin, How AI Impacts Skill Formation (2026.01) — Corporate Research · Preprint Paper
8. Brynjolfsson·Li·Raymond, Generative AI at Work, Quarterly Journal of Economics (Published online on February 4, 2025)
9. Lisa D. Cook, An Update on AI and the Economy (2026.09.28) — Personal Views of a Federal Reserve Board Member

Date: 2026.10.29 · Davar · Global Market Analysis & Investment Strategy 🌍
※ This article is an analysis examining AI business productivity and verification costs based on data released up to October 29, 2026, and is not an investment recommendation.