HarmonyNote

Problem & Motivation

Mental health is a global crisis. In 2023, the World Health Organization reported that 1 in 8 people worldwide experienced mental illness. In the U.S., the numbers are equally concerning; as of 2021, over 57 million adults—approximately 1 in 5—faced mental health challenges. Depression and anxiety alone cost the global economy an estimated $1 trillion annually, primarily due to reduced productivity. 

Despite the severity of the issue, barriers such as shortages in human resources, financial constraints, social stigma, racial disparities, and inadequate education and support resources continue to leave millions without timely access to care. 

Our Solution

HarmonyNote aims to break down these barriers by providing a safe journaling space enhanced with AI-powered just-in-time emotional support, a 24/7 AI Wellness Coach, and periodic mental wellness insights. These effective tools empower individuals to monitor their mental health and access timely support, helping them address negative emotions before they escalate into more serious challenges. 

The HarmonyNote platform leverages a dynamic framework that seamlessly integrates multiple generative AI technologies, including Langchain Agents for orchestrated task management, strategic API calls for real-time data processing, advanced LLM models for natural language understanding, and sophisticated ensemble models that combine multiple approaches for enhanced accuracy and reliability.

HarmonyNote’s mission is to elevate, cherish, and enrich lives in a way that fosters harmony.

Data Source

We gathered all our data from open-sourced, labeled datasets published by HuggingFace, encompassing diverse emotion labels derived from Reddit comments, surveys, and Twitter messages. After data cleaning and preprocessing, we compiled approximately 172,000 data points. Notable datasets include GoEmotions by Google Research, ISEAR, SetFit Emotions, Dair-AI Emotions, SemEval Workshop, Berkeley Hate Speech, and six additional public datasets.

Data Science Approach

HarmonyNote integrates two key components: (1) Emotion Classification (2)  Langchain Agent, and (3) AI Wellness Coach. These components leverage state-of-the-art machine learning techniques to deliver personalized mental wellness support.

Emotion Classification The emotion classification pipeline processes user journal entries to classify emotions into one of seven categories: Neutral, Happy, Sad, Angry, Fear, Surprise, or Disgust.

• The pipeline uses an ensemble of fine-tuned models, including RoBERTa-Large, BERT-Large-Uncased, and DistilRoBERTa. The ensemble voting mechanism enhances classification accuracy.
• Data preprocessing steps include:
  • Unicode normalization, HTML parsing, and regex-based cleaning.
  • Stopword removal, tokenization, and lemmatization using NLP libraries.
  • Emotion-labeled datasets (e.g., GoEmotions, ISEAR, and Reddit/Twitter datasets) are consolidated for training and evaluation. The final dataset comprises over 171,000 entries, ensuring a balanced distribution of labels.
• Baseline evaluations were conducted using Naive Bayes, with ensemble models achieving a significant accuracy improvement (92%).

Langchain Agent for Recommendation

• Each detected emotion activates a dedicated Langchain Agent, which performs specialized reasoning and analysis based on the specific emotional context, triggering the appropriate API calls to an external music platform for music recommendation to the users.

AI Wellness Coach

• The AI Wellness Coach combines GPT-4.o-mini and custom prompt engineering to provide real-time, empathetic support based on journal inputs.
• The chatbot generates personalized responses tailored to user emotional states by extracting user’s hobbies and interests from the encrypted user profile data. Prompts are designed to include empathetic language and actionable advice.
• The system integrates user safety considerations by embedding context-specific suggestions (e.g., professional support or self-care activities).

Evaluation

Our model evaluation followed a comprehensive methodology that incorporated multiple performance dimensions. The model ensemble combining RoBERTa-Large, BERT-Large-Uncased, and DistilRoBERTa demonstrated superior performance with 92% overall accuracy, outperforming baseline approaches like Naive Bayes across standard metrics including precision, recall, and F1 score. 

The AI Wellness Coach's effectiveness was assessed through human evaluations examining response quality, coherence, and empathetic tone, supplemented by BLEU and ROUGE metrics comparing chatbot outputs against predefined templates. Valuable insights came from MVP testing with users aged 13-55, who reported enhanced self-awareness and emotional understanding through their interactions with the journaling features and Wellness Coach. 

Technical evaluation was further strengthened by calculating semantic similarity scores between predicted and actual responses using sentence embeddings, while systematic error analysis revealed specific areas for improving prompts and model responses.

Key Learnings & Impact

The development of an end-to-end emotion detection pipeline yielded several critical insights into AI-driven mental wellness applications. The foundation of a successful large language model lies in high-quality training data, which demanded sophisticated solutions for collecting, cleaning, and preprocessing text from diverse sources. 

Key technical challenges included addressing dataset imbalances, standardizing emotional terminology across labels, and optimizing for short-text analysis. The fine-tuning process for models like RoBERTa and GPT-4.0-mini revealed the delicate balance between computational efficiency and accuracy. Through strategic hyperparameter optimization, we enhanced model performance while maintaining practical deployment requirements.Perhaps most crucially, the project highlighted the paramount importance of user trust and ethical considerations in mental wellness applications. This led to the implementation of robust privacy measures, including encryption protocols and the deliberate avoidance of conversational tracking in both the journaling and AI Wellness Coach components.

HarmonyNote stands poised to create a transformative long-term impact by addressing mental health challenges at multiple levels of society. At its core, the AI powered journaling solution helps individuals achieve greater emotional stability and clearer decision-making, leading to stronger relationships and improved well-being. By empowering individuals, HarmonyNote creates a ripple effect that extends to broader society, fostering more resilient communities with reduced rates of addiction and crime. This positive feedback loop strengthens organizations and builds healthier communities, fundamentally shifting how society approaches and values mental well-being.

Acknowledgements

We extend our deepest gratitude to our instructors, Puya Vahabi and Korin Reid, whose invaluable guidance and thoughtful feedback shaped our project's development. Special thanks to our domain expert, Meredith Frank, whose professional expertise in mental health provided crucial insights that strengthened our application's framework. We are also grateful to the educators, teachers, and colleagues who generously shared their knowledge and perspectives throughout this journey, enriching our understanding and contributing to the project's success.