The bar exam broke. The pass rate jumped from 36% to 65%.
Summary
California's State Bar didn't tell its own leadership that AI-generated questions were being added to the February 2025 bar exam. A content-validation process meant to catch bad questions missed problems in a fifth of them. When technical failures compounded the damage, the State Bar raised the exam's pass rate from about 36% to about 65% -- and spent $5.1 million administering an exam meant to save money, plus up to $5.2 million more in lost revenue for the 4,200 test-takers affected.
How AI ended up in the exam
The State Bar of California switched to a hybrid remote-or-in-person format for its February 2025 bar exam, partly to reduce costs and shore up its Admissions Fund⧉. Because the national exam-question bank does not allow remote administration, the State Bar hired Kaplan Exam Services to write new multiple-choice questions -- but Kaplan's contract never specified all the legal topics the exam needed to cover. To fill the gap, the State Bar turned to ACS Ventures, a firm it already used for psychometric consulting, which wrote the missing questions using artificial intelligence. The State Bar's own admissions office knew this -- and did not tell its executive leadership.
View data as table
| Original pass rate | 36% |
|---|---|
| Adjusted pass rate | 65% |
The validation process that missed a fifth of the questions
A content-validation process was supposed to catch bad questions before they counted toward anyone's score. Instead of following the national exam's practice of assigning topic-specific experts, the State Bar had a single subject-matter expert review all 200 questions⧉. After the exam, the psychometric contractor identified 40 of those 200 questions -- one in five -- as having scoring problems. The State Bar included 19 of them anyway, and ultimately pulled 29 questions from scoring altogether.
Crashes, freezes, and a lawsuit
The State Bar also hired ProctorU Inc. (dba Meazure Learning) to administer the exam without first spelling out its own technical requirements, and without confirming Meazure Learning had fixed freezes and crashes flagged during a November 2024 test run. On exam day, media reports the audit cites describe the platform crashing repeatedly -- one test-taker said he had to restart more than 30 times -- along with screen lag, error messages, and broken copy-paste tools. The State Bar's own post-exam survey⧉ separately found 53% of respondents reported computer crashes and 57% reported freezes. In May 2025, the State Bar sued Meazure Learning, alleging the vendor misrepresented its capacity and breached its contract; the case was still open as of June 2026.
View data as table
| Estimated cost, hybrid model (Feb. + July 2025 combined) | 4,400,000 |
|---|---|
| Actual cost, February 2025 exam alone | 5,100,000 |
| 3-year average annual cost, both exams (2022-2024) | 7,700,000 |
The exam meant to save money
The switch to remote testing was meant to save money. The State Bar had estimated⧉ a hybrid model would cost $4.4 million to run both its 2025 exam sessions combined, versus $8.4 million in-person-only. Instead, the February exam alone cost $5.1 million -- not counting the ongoing lawsuit. On top of that, the audit separately tracked up to $5.2 million in lost revenue: $2 million in direct refunds, plus the value of fee waivers the State Bar approved for test-takers to use on later exams.
View data as table
| Refunds provided to test-takers | 2,000,000 |
|---|---|
| Value of used fee waivers | 2,100,000 |
| Value of unused fee waivers (expire after July 2026 exam) | 1,200,000 |
The takeaway
- One exam cost more than the estimate for two. The February 2025 exam alone cost $5.1 million -- more than the entire $4.4 million the State Bar had budgeted for both its 2025 exam sessions combined.
- The AI use was known internally and hidden upward. The admissions office knew AI-generated questions were going into the exam and didn't tell State Bar leadership.
- The State Bar agreed with every recommendation -- but not every finding. Its own written response says it 'disagree[s] with certain findings,' noting the audit never interviewed Meazure Learning's staff -- a boundary the auditor calls deliberate, since the vendor's conduct is now the subject of separate litigation.
The audit states two different lost-revenue figures in different sections: 'at least $4 million' in its Summary, and a more detailed 'up to $5.2 million (rounded)' in Figure 17, broken into refunds and fee-waiver values. This piece reports the Figure 17 breakdown as the more complete accounting and discloses both figures rather than picking one silently. The $4.9 million the State Bar's Board approved as a fee-waiver pool is a separate authorized ceiling, not the same as the $3.3 million in fee-waiver value Figure 17 tracks as used or unused -- this piece does not treat those as interchangeable. The audit notes it did not evaluate the merits of the State Bar's ongoing lawsuit against Meazure Learning, and neither does this piece.
Sources(1) ▾
- California State Auditor, The February 2025 Bar Exam: The State Bar's Poor Implementation of Changes to the Exam Negatively Affected Test Takers (Report 2025-051) (2026-07-09) — The California State Auditor's own investigation, required by a state law amendment after the February 2025 bar exam's technical and administrative failures. Covers question development, exam administration, the use of AI to draft questions, and program costs. Fetched directly and converted with pdftotext -layout. auditor.ca.gov · original document
Comments
Always open. Logged-in readers can annotate paragraphs in place.
In February 2025, California's bar exam failed so badly that the State Bar raised the pass rate from about 36% to about 65%⧉ to compensate. A state audit released this month found the exam used undisclosed AI-generated questions, a flawed review process, and a testing platform that crashed repeatedly -- and cost far more than the money-saving switch was supposed to save.