My journey with my daughter using generative AI (GPT-J, ChatGPT & DALL-E) & creating storybooks

I have been interested in generative AI tools and testing them for a while as I can see the value that they can help spark new ideas as well as help with routine tasks (most of the time). Earlier this summer, my 5-year-old daughter and I started playing with generative AI tools (specifically, DALL-E) together and creating fun pictures. During one of our sessions, she wanted to see a giraffe walking on the moon. That idea led to our first AI-generated storybook together. In this post, I will share how we created our own storybooks following 2 different approaches that I have tried so far. And I will share how open-access AI tools can be used for creating your personal storybooks.

Approach #1: Auto-generated book with a single prompt (GPT-J and DALL-E) My first attempt was to use GPT-J 6B (which is open-access to use) and DALL-E (more specifically, DALL-E 2 by OpenAI). I noticed that GPT-J’s output required a couple of iterations to achieve an optimal result for a kid’s book:

Prompt 1 (bold) and output: I am walking on the moon with a giraffe. It’s the middle of the night, and we are both asleep. My feet hurt and I feel a little confused. “What am I doing here?” I ask myself. “Am I a dream? A memory?” I don’t think so, because I’m wearing a space suit, and I’m quite real. I’m also on the moon, which is one of the most desolate places in the solar system. It’s not even a real place, it’s an orb. An orb that is about the size of a basketball. And then I wake up, and everything is normal again.

Prompt 2 (bold) and output: I am walking on the moon with a giraffe. It’s the middle of the night, and we are both asleep. But not for long. The giraffe wakes up and starts looking around. Then, it says, “I think I know this place.” “Yes,” I say. “It’s the moon.” “Well, maybe not,” the giraffe says. “But it sure feels like it.” And we start to walk.

Prompt 3 (bold) and output: I am walking on the moon with a giraffe. It’s the middle of the night, and we are both asleep. But not for long. The giraffe wakes up and starts looking around. Then, it says, “I think I know this place.” “Yes,” I say. “It’s the moon.” “Well, maybe not,” the giraffe says. “But it sure feels like it.” “Yes,” I say. “It is.” “I mean, you’re right,” the giraffe says. “I can’t see anything, but it sure feels like this is the moon. I mean, it’s got all the right rocks. And the moon is way up high in the sky. You know, this is the kind of place you dream about. So let’s go!” “Yes,” I say. “I know this place. You’re right. Let’s go!” It’s the dream that I’ve been having almost every night since I moved to New York City.

Prompt 2 output seems to have the optimal content for our storybook, and I used it with an ending (a sentence) I created for that story. This process shows that our (human) effort is necessary to evaluate and intervene to complete the written content.

After that, I used each statement as input for DALL-E with slight modifications to indicate the characters and environment to be consistent with the story. My daughter and I selected the most fun images to include in our book. Here is the result:

ChatGPT: the booster shot on creating stories This first experiment of creating a book was to automate a storybook creation mechanism with a single prompt: “I am walking on the moon with a giraffe”. GPT-J was harder to use by default to create the content I intended to do. DALL-E did a fair job with the images. Yet, the whole process is just an example of an on-demand single prompt fairly usable/ fun content creation. After chatGPT is released, the storybook creation has been a lot easier as the input can include the commands (length, type, and scope of the content) towards how to create the storybook. On that variation, the updated prompt and output:

Prompt: Create a 3 sentence kid storybook about “I am walking on the moon with a giraffe” Output: Once upon a time, I was walking on the moon with a giraffe. We were having a great time, jumping and playing in the low gravity. As we looked up at the stars, the giraffe spotted a shooting star and made a wish. We continued our adventure, exploring the wonders of the lunar landscape.

Unlike GPT-J, chatGPT is more capable of understanding the intent and creating a well-fitting story for the prompt. That reduces the time for optimizing the story. After that, as before, each sentence was used as input to DALL-E. And my daughter picked the images for each page. Here is the result:

Approach #2: Step-wise & joint book generation Our next journey was to take a flamingo to the moon this time (She enjoyed taking animals to the moon :) ). This time, we did more of a hands-on storybook creation without giving the pleasure to GPT to create the whole story! On each step (or page), I asked my daughter about the next adventure of our flamingo. And I used her words to create images via DALL-E and she selected the images she thought to be the best. Here is the result:

This process is not much automated as before, yet, I should admit, this was a more fun process than a single prompt storybook as we did before. Why? Because it added more interaction, creativity, and a collective thinking + problem-solving process in the loop where I see a higher value in terms of my kid’s creative skills and building our personal relationship. As we all observe, a disconnect happens as kids engage with digital media by themselves and also mostly because those tools are designed to be engaged individually. Generative models can boost self-creation as well as co-creation (like drawing together or storytelling) in the digital landscape. For me, I value and enjoy co-creation with my kid as most of the other digital products for kids do not emphasize this aspect much.

There are new tools out in the market using similar approaches where a user enters the idea and the tool creates the rest of the story with illustrations similar to my examples before- even also adding a narrator! Yet we need to ask what is the real value here? I believe promoting co-creation will add more value like a DIY project, instead of making the creation process easier with a single prompt. Thank you for reading!