Moritz Hardt has contributed to the Legal Agent Benchmark launched by Harvey

Harvey is the leading AI company in legal practice, used by many of the world's top law firms. The benchmark is the first open evaluation of this scope for long-horizon legal-agent work. It establishes a shared standard for measuring whether AI systems can perform the kind of multi-document, open-ended assignments typically delegated to law-firm associates. The benchmark sits at the intersection of frontier AI research and a regulated professional domain where the stakes of automation are unusually high.

Hardt and MPI-IS are among the project's named partners, alongside Anthropic, OpenAI, NVIDIA, Google DeepMind, Mistral AI, and other leading AI labs. From Harvey:

"Moritz Hardt (Max Planck Institute for Intelligent Systems, Tübingen) has been a methodology partner on Legal Agent Benchmark, bringing insights from the science of benchmarking that helped shape the scoring framework and evaluation design–including all-pass grading, moving away from prescribed solutions, and review of the synthetic document pipeline. We're continuing to work closely with him on extensions of the benchmark and other research."

Hardt's recent book, The Emerging Science of Machine Learning Benchmarks, published by Princeton University Press, develops the science of how benchmarks work–and argues that, despite their well-known limitations, they have been the central engine of progress in machine learning.

Harvey’s announcement: https://www.harvey.ai/blog/introducing-harveys-legal-agent-benchmark