LLM-Enhanced Fantasy Adventure Video Game

For my computer science capstone project, I worked with two other Tulane seniors to develop a fantasy adventure video game that incorporates Large Language Models to enhance the dialogue of non-playable characters. Our final product was developed in the Unity platform using C# and included the following features:

  • 12 unique characters models
  • 37 total animations
  • 3 turn-based combat encounters
  • 16 particle effects
  • 3 original world maps (designed by us)
  • 7 AI-Driven NPCs
  • 21 unique attacks

The game follows three fantasy characters—a druid, witch, and thief—who are free to roam around an open map under the player's control:

Game Characters

They encounter NPCs throughout their journey, who prompt the player to ask whatever question they want. That information is then given to an LLM alongside other pieces of context, such as relevant information about the lore of the game and the NPC character's backstory. The NPC then gives an appropriate response that's also designed to guide the players toward their next objective:

NPC Dialogue

At 3 points throughout the game, the player encounters various enemies, which they automatically engage in Pokemon-style turn-based combat encounters. In these encounters, the player chooses one of four spells to cast from each of their three characters, making strategic decisions to manage their character's health & stamina to defeat the enemy:

Combat Encounter

After every attack, the LLM then generates a unique piece of flavor text to narrate the battle. We were quite impressed with the quality of text generated here:

Battle Narration

We also integrated many free visual effects from the Unity Asset store, which were all visually stunning:

Visual Effects

Finally, we designed a poster and presented this project at the 2024 Engineering Expo event, where we presented our game to professors and students in the Tulane community. We also developed ethical guidelines on how to use AI to enhance human-written narratives rather than replace human writers:

Expo Poster

Our final product focused on creating a great overall gameplay experience to showcase how LLMs can enhance video games. In simpler programming languages, we also developed prototypes of more advanced ways to integrate LLMs into video games. These include a RAG (Retrieval Augmented Generation) system to search through a larger database of lore for the video game, and an option for a player to write out their own ideas for spells which various NLP/LLMs algorithms analyze to determine the spell's damage/effects.

Huge thank you to my team members, Luke Gulson and Ben Russell, as well as our advisors for this project, Professor Aaron Maus and Alireza Shirvani.