Qiskit HumanEval: Evaluation Benchmark for Quantum Code Generation Published

Excited to share our new research paper introducing the Qiskit HumanEval dataset! 🚀

This work addresses a critical need: evaluating Large Language Models’ capability to generate quantum computing code. Our dataset comprises more than 100 quantum computing tasks, each with accompanying prompts, solutions, test cases, and difficulty ratings.

We systematically tested LLMs on their ability to produce executable quantum code, demonstrating the feasibility of using generative AI tools in quantum code development and establishing important benchmarks for the field.

This research opens new possibilities for AI-assisted quantum software development and provides a standardized way to measure progress in this exciting intersection of quantum computing and artificial intelligence.

Read the paper: https://arxiv.org/abs/2406.14712v1

#quantumcomputing #ibmquantum #qiskit #llms

Originally shared on LinkedIn on July 3, 2024 - 70 reactions, 7 comments as of 11/12/2025