Dreamville Mart

Introduction

This project is a multiplayer game developed on the Unreal Engine 5 platform.

This game takes the benefit of AI (ChatGPT 3.5 and Dall-E 2.0) to achieve generative gameplay features. 

There will be two players in the game; they are the "guests" of our Dreamville mart. Each of them will input three items at the beginning of the game, and that item will then be generated in the store. The goal of each player is to find the other player's item by solving the riddles generated by Chat-GPT. 

PROJECT Website

https://projects.etc.cmu.edu/cheat-code/ 

demo video

About this Project

1: It is a semester project developed by the ETC CheatCode Team. This project is developed by a group of five people.

• Producer 

  • • Minkyung Shin

• Programmer 

  • Xuan(James) Zhai 

• Artists 

  • Chloe Cho

  • Felicia Li

  • Liyi Tang

2: My role in this project is the programmer. I developed the game architecture and all the gameplay features in Unreal using blueprints and C++. I developed the multiplayer functionality using a Listen Server Mode with State Synchronization. I developed the connection between the game and ChatGPT 3.5 and Dall-E 2.0 in real-time to achieve generative AI gameplay features.

Multiple Feature

I use the Unreal Listen Server mode to achieve multiple functionality. One player will be the server and create a session, and the other player will be the client and join the session. The game uses the LAN Connection but will be switched to the Steam Online Subsystem later. 

Unreal & Gen AI interactions

I use the Http-GPT Unreal Plugin to achieve the Game&GenAI interaction in real-time. After the players enter an input, It will use ChatGPT to categorize the input and generate riddles for the input, and it will use Dall-E to generate the images of those inputs.

To transfer the image texture between players, the game will convert the texture into a byte stream and replicate the bytes from the server to the client. 

reference: https://www.unrealengine.com/marketplace/en-US/product/httpgpt-chatgpt-integrated-in-the-engine 

Unreal & Gen AI interactions

For each player’s input item, our game will use ChatGPT to categorize it and put it into the right section in our market. (Between Fruit, Meat, Bakery, Snack, Dairy, Others)

The game will also ask ChatGPT to generate a riddle for that item. 

Finally, the game uses Dall-E to generate an image of the item.

Unreal & Dall-E Image Generation

We also trained the AI and tested it with different inputs to let the image fit with the dreamy art style of our game.