The bar exam broke. The pass rate jumped from 36% to 65%.
Summary
California's State Bar didn't tell its own leadership that AI-generated questions were being added to the February 2025 bar exam. A content-validation process meant to catch bad questions missed problems in a fifth of them. When technical failures compounded the damage, the State Bar raised the exam's pass rate from about 36% to about 65% -- and spent $5.1 million administering an exam meant to save money, plus up to $5.2 million more in lost revenue for the 4,200 test-takers affected.
The problem: the State Bar's Admissions Fund was heading toward insolvency, so the February 2025 exam went remote to save money. The approvals: the State Bar hired Kaplan for questions and Meazure Learning to run it. Where it goes wrong: a topic gap got filled with AI-written questions hidden from leadership, the platform crashed, and the pass rate went from about 36 to about 65 percent. How it's caught: the California State Auditor, July 2026.
The problem ·The State Bar of California's Admissions Fund was heading toward insolvency. Two in-person exams a year cost an average of $7.7 million from 2022 to 2024. A May 2024 estimate said a hybrid remote-or-in-person model would cost about $4.4 million for both 2025 sessions, against $8.4 million in person. The February 2025 exam went hybrid to save money.
The approvals ·Because the National Conference of Bar Examiners does not allow its questions to be given remotely, the State Bar contracted Kaplan Exam Services to write new multiple-choice questions and Meazure Learning to administer the exam. When Kaplan's contract left legal topics uncovered, the admissions office had ACS Ventures, its psychometric contractor, write the missing ones. Those contracts are where the money entered.
Where it goes wrong ·Three controls were missing. Kaplan's contract never listed every topic, so ACS Ventures filled the gap with AI-generated questions, and the admissions office did not tell executive leadership. One expert reviewed all 200 questions instead of one per subject; 40 were later flagged, 19 counted anyway, 29 pulled. Meazure Learning was hired without written business requirements and without confirming it had fixed the freezes and crashes from a November 2024 pretest. In the State Bar's survey, 53 percent reported crashes and 57 percent freezes.
How it's caught ·The California Supreme Court ordered score imputation and a lower raw passing score; the pass rate went from about 36 to about 65 percent. The State Auditor's July 2026 report put the February exam alone at $5.1 million, above the $4.4 million estimated for both 2025 sessions, plus up to $5.2 million in lost revenue from refunds and fee waivers. The State Bar agreed with every recommendation and disputes certain findings; its lawsuit against Meazure Learning was still open in June 2026.
An exam changed to save money cost more than the two-exam budget it replaced, and the remedy was moving the pass line for the 4,200 people who sat it. Next checkpoint: the Supreme Court's decision on the NextGen exam for July 2028.
How AI ended up in the exam
The State Bar of California switched to a hybrid remote-or-in-person format for its February 2025 bar exam, partly to reduce costs and shore up its Admissions Fund⧉. Because the national exam-question bank does not allow remote administration, the State Bar hired Kaplan Exam Services to write new multiple-choice questions -- but Kaplan's contract never specified all the legal topics the exam needed to cover. To fill the gap, the State Bar turned to ACS Ventures, a firm it already used for psychometric consulting, which wrote the missing questions using artificial intelligence. The State Bar's own admissions office knew this -- and did not tell its executive leadership.
View data as table
| Original pass rate | 36% |
|---|---|
| Adjusted pass rate | 65% |
The validation process that missed a fifth of the questions
A content-validation process was supposed to catch bad questions before they counted toward anyone's score. Instead of following the national exam's practice of assigning topic-specific experts, the State Bar had a single subject-matter expert review all 200 questions⧉. After the exam, the psychometric contractor identified 40 of those 200 questions -- one in five -- as having scoring problems. The State Bar included 19 of them anyway, and ultimately pulled 29 questions from scoring altogether.
Crashes, freezes, and a lawsuit
The State Bar also hired ProctorU Inc. (dba Meazure Learning) to administer the exam without first spelling out its own technical requirements, and without confirming Meazure Learning had fixed freezes and crashes flagged during a November 2024 test run. On exam day, media reports the audit cites describe the platform crashing repeatedly -- one test-taker said he had to restart more than 30 times -- along with screen lag, error messages, and broken copy-paste tools. The State Bar's own post-exam survey⧉ separately found 53% of respondents reported computer crashes and 57% reported freezes. In May 2025, the State Bar sued Meazure Learning, alleging the vendor misrepresented its capacity and breached its contract; the case was still open as of June 2026.
View data as table
| Estimated cost, hybrid model (Feb. + July 2025 combined) | 4,400,000 |
|---|---|
| Actual cost, February 2025 exam alone | 5,100,000 |
| 3-year average annual cost, both exams (2022-2024) | 7,700,000 |
The exam meant to save money
The switch to remote testing was meant to save money. The State Bar had estimated⧉ a hybrid model would cost $4.4 million to run both its 2025 exam sessions combined, versus $8.4 million in-person-only. Instead, the February exam alone cost $5.1 million -- not counting the ongoing lawsuit. On top of that, the audit separately tracked up to $5.2 million in lost revenue: $2 million in direct refunds, plus the value of fee waivers the State Bar approved for test-takers to use on later exams.
View data as table
| Refunds provided to test-takers | 2,000,000 |
|---|---|
| Value of used fee waivers | 2,100,000 |
| Value of unused fee waivers (expire after July 2026 exam) | 1,200,000 |
The takeaway
- One exam cost more than the estimate for two. The February 2025 exam alone cost $5.1 million -- more than the entire $4.4 million the State Bar had budgeted for both its 2025 exam sessions combined.
- The AI use was known internally and hidden upward. The admissions office knew AI-generated questions were going into the exam and didn't tell State Bar leadership.
- The State Bar agreed with every recommendation -- but not every finding. Its own written response says it 'disagree[s] with certain findings,' noting the audit never interviewed Meazure Learning's staff -- a boundary the auditor calls deliberate, since the vendor's conduct is now the subject of separate litigation.
The audit states two different lost-revenue figures in different sections: 'at least $4 million' in its Summary, and a more detailed 'up to $5.2 million (rounded)' in Figure 17, broken into refunds and fee-waiver values. This piece reports the Figure 17 breakdown as the more complete accounting and discloses both figures rather than picking one silently. The $4.9 million the State Bar's Board approved as a fee-waiver pool is a separate authorized ceiling, not the same as the $3.3 million in fee-waiver value Figure 17 tracks as used or unused -- this piece does not treat those as interchangeable. The audit notes it did not evaluate the merits of the State Bar's ongoing lawsuit against Meazure Learning, and neither does this piece.
Sources(3) ▾
- California State Auditor, The February 2025 Bar Exam: The State Bar's Poor Implementation of Changes to the Exam Negatively Affected Test Takers (Report 2025-051) (2026-07-09) — The California State Auditor's own investigation, required by a state law amendment after the February 2025 bar exam's technical and administrative failures. Covers question development, exam administration, the use of AI to draft questions, and program costs. Fetched directly and converted with pdftotext -layout. auditor.ca.gov · original document
- State Bar of California, State Bar Announces February 2025 Bar Exam Results (2025-05-05) — The State Bar's own official announcement of the February 2025 General Bar Examination results (55.9% GBX pass rate, highest spring rate since 1965), attributing the outcome to score-imputation and raw-passing-score remedies ordered by the California Supreme Court on May 2, 2025 after the exam's technical failures. Corroborates the pass-rate-adjustment claim from the audited entity's own primary record. Note: reports the overall published pass rate (55.9%), which is a different measure than the audit's pre-/post-remedy 36%->65% comparison (Fig. 15). calbar.ca.gov · original document
- Supreme Court of California (California Courts Newsroom), California Supreme Court Issues Statement on February Bar Exam (2025-03-04) — Statement from the Supreme Court of California (Judicial Branch, independent of both the State Auditor and the State Bar) acknowledging 'troubling reports of technical failures, delays, and other irregularities' during the February 2025 bar exam and committing to determine appropriate remedies. Independently corroborates the technical-failure trigger that led the Court's subsequent May 2, 2025 order to lower the raw passing score (to 534) and impute scores -- the mechanism behind the pass-rate increase the audit reports. newsroom.courts.ca.gov · original document
Comments
Always open. Logged-in readers can annotate paragraphs in place.
In February 2025, California's bar exam failed so badly that the State Bar raised the pass rate from about 36% to about 65%⧉ to compensate. A state audit released this month found the exam used undisclosed AI-generated questions, a flawed review process, and a testing platform that crashed repeatedly -- and cost far more than the money-saving switch was supposed to save.