Conceptual Metaphors
One way to gauge creative expression is one's ability to construct conceptual metaphors by creating meaningful and unique associations between two distinct concepts using a process called conceptual blends. Are LLMs creative? Can they construct conceptual metaphors? Are metaphors constructed by AI meaningful and decipherable by humans? What kind of conceptual mappings can LLMs make based on their training data. This project was an attempt to answer these questions. And to do that, I created a prototype based on visual card game known as Dixit.
Overview
Metaphors reveal how a multimodal AI model connects concepts to visuals, helping to understand interconnections within training data and interpret user intent.
I developed a game-style research artifact inspired by the card game Dixit to explore how metaphors enhance a multimodal AI model's ability to connect concepts with visuals. In this interactive experience, the AI selects a card and provides a metaphorical clue, prompting players to guess the card. Correct guesses earn points for both the user and the AI, while incorrect guesses result in a point loss for the AI.
After each guess, users rate the accuracy and quality of the metaphor through a brief questionnaire. This feedback is essential for collecting research data, helping us analyze how well AI-generated visual metaphors align with user expectations. This project not only engages users but also provides insights into interpreting user intent and understanding interconnections within training data.
Live Version (Try it!)
Sitong Wang, Savvas Petridis, Taeahn Kwon, Xiaojuan Ma, and Lydia B Chilton. 2023. PopBlends: Strategies for Conceptual Blending with Large Language Models.
The game-style research artifact effectively showed how metaphors can help mediate user interaction with a multimodal AI model. By engaging users with AI-generated visual metaphors, valuable insights were gained into user understanding and expectations, supported by feedback from ratings.
Suggested Research Directions
Benchmarking Metaphor Quality: Develop a framework to evaluate and compare the quality of metaphors generated by different AI models.
User Intent Analysis: Analyze how user guesses and feedback relate to their intent to better understand cognitive processes in interpreting metaphors.
Contextual Understanding: Investigate how different contexts (themes or cultures) influence metaphor interpretation to improve AI responsiveness.
Longitudinal Studies: Conduct studies over time to observe how user interactions with metaphors evolve.
Broader Applications: Explore using this artifact in fields like education or therapy to enhance communication and understanding.
These directions aim to strengthen AI's ability to interpret and respond to user intent through metaphorical language.