How Do AI-Illustrated Books Work?

The full pipeline, explained without hand-waving — from photo upload to hardcover.

AI-illustrated books are made by combining three kinds of AI. A language model writes the story from a customer's answers. A vision model reads the customer's reference photos and learns what each character looks like. An illustration model paints every page, using the vision reference to keep likenesses consistent. The customer reviews the whole book before anything prints and can re-render any page that's off. Memolio follows this exact pipeline — GPT for the story, a vision stage to learn faces, Seedream 4.5 for the illustrations — and the result is a 24-page hardcover that features the real grandparent on every page.

The Five-Stage Pipeline

1. Intake: photos and memories

   The customer fills a short questionnaire — names, dates, places, a few photos of the grandparent at different ages, optionally photos of a partner and grandchildren, and the memories they want captured. Memolio collects this via WhatsApp or a web form in about 20 minutes.

2. Story generation: the language model writes

   A language model (Memolio uses OpenAI GPT) reads the questionnaire answers and writes a narrative that follows a fixed book structure — childhood, adult life, meeting a partner, children, the grandchild's arrival, dedication. The prompt is tuned so the story stays warm, bilingual, and factual rather than hallucinated.

3. Vision analysis: the AI learns faces

   A vision model reads each uploaded photo and extracts a character description — face shape, hair, build, approximate age. It does this per life stage, so a photo from the grandparent's twenties produces a "young" reference and a recent photo produces an "old" reference. These references are what keeps the grandparent recognisable on every page.

4. Illustration generation: each page gets painted

   An illustration model (Memolio uses Seedream 4.5 via BytePlus) paints each page individually. The prompt for each page includes both a scene description (from the story) and the vision-model reference (from the photos). This reference-bound generation is why the grandparent on page 3 looks like the grandparent on page 15, not two different people.

5. Review and print

   The customer sees the whole book online. Pages that aren't right — wrong outfit, wrong face, wrong scene — can be re-rendered with a click, sending the specific page back through the illustration stage. Once the customer approves the book, a print-on-demand partner produces the hardcover and ships it.

Where Quality Comes From

Most of the quality in an AI-illustrated book isn't from picking the latest model. It's from three engineering choices that are hidden from the customer.

Likeness consistency. A character on page 3 has to look like the character on page 15. The default behaviour of image models is to drift — each page is a fresh generation, and small variations compound. Services that get this right use some combination of a locked reference image, reference-image-to-image prompting, and per-character descriptions that stay constant across all pages.

Age progression. A grandparent book needs the grandparent at different ages — their twenties, fifties, today. Photos of those ages exist, but they look like different people unless the pipeline explicitly bridges them. Good services ask for photos from multiple life stages and feed the right one for each page.

The review step. AI gets a subset of pages wrong on every book. The only reasonable way to ship quality at volume is to let the customer review and re-render. Services that print without review ship broken books; services that review catch the problems before they become printed objects.

Comparing the Approaches

Frequently Asked Questions

How do AI-illustrated books work?

They combine three AIs: a language model that writes the story, a vision model that reads the customer's reference photos, and an illustration model that paints each page using that reference. A review step before print lets the customer fix anything the AI got wrong.

Which AI models are actually used?

It varies by service. Memolio uses OpenAI GPT for the story text and photo analysis, and Seedream 4.5 (via BytePlus, Singapore) for the illustrations. Other services use DALL·E, Midjourney, Stable Diffusion, or Flux. The specific models rotate as better ones ship.

How does the AI get the grandparent to look like the grandparent?

A vision model reads the reference photos and extracts a character description — face shape, hair, build, age. The illustration model uses that description, plus the reference image itself as an input, to paint each page with the same person across every scene. The better the reference photos, the better the likeness.

Are AI-illustrated books any good?

Quality has jumped in the last year as new models have shipped. The best AI-illustrated books are now genuinely good — warm, consistent watercolour illustrations with recognisable likenesses. They're not quite a bespoke hand-painted commission, but they're dramatically cheaper and faster, and the gap is closing with every model release.

What if the AI gets something wrong?

Every reputable AI book service includes a review step before anything prints. You look at every page, flag anything off, and the service re-renders the page. Nothing goes to print until you approve. This is non-negotiable for a product where the subject is a real person.

Is it safe to upload photos to an AI book service?

It depends on the service's data handling. Memolio operates under GDPR: photos are deleted from our image host 90 days after delivery, every international AI processor we use is covered by Standard Contractual Clauses, and customer photos are never used to train AI models.

If the book includes children — your own children, grandchildren, nieces, nephews — always get parental permission before uploading their photos. The images of a child belong to the child's parents or guardians, not the person ordering the book. This is basic consent practice and it matters regardless of which service you use.

Always read the privacy policy before uploading photos to any service.