CARE-Bench: A Benchmark of Diverse Client Simulations Guided by Expert Principles for Evaluating LLMs in Psychological Counseling

CARE-Bench has been developed to meet the growing need for effective psychological counseling solutions through the use of Large Language Models (LLMs). As mental health services face unprecedented demand, traditional evaluation methods have proven inadequate due to their reliance on unprofessional client simulations and static assessment formats. CARE-Bench offers a dynamic and interactive approach, utilizing diverse client profiles derived from actual counseling scenarios, ensuring a multidimensional performance evaluation based on established psychological scales. This innovative benchmark not only assesses the counseling competence of various LLMs but also highlights their current limitations when engaging with diverse client types. Collaborating with psychologists, the creators of CARE-Bench aim to refine the evaluation process, ultimately enhancing the effectiveness of LLMs in providing psychological support.