Creativity Engine

The Creativity Engine is a web app that allows children and young people to kickstart their creative writing experiments by asking for the collective help of some of the foremost children's authors writing in English. The app uses the archives of Seven Stories — the national centre for children's books — to power a text generating Artificial Intelligence capable of helping young writers through even the most solid creative block.

Origins

The idea behind the project came from a small experiment with young writers during the 2020-21 lockdown period. As part of an English Association initiative in which students were given free-reign over one of their publications, Issues in English No. 16 — The Quaranteens, we were invited to speak to a group of eager young creators about the intersection between art, literature, and artificial intelligence. We brought with us a series of examples and playgrounds for them to engage with, including Google's Verse by Verse and a hastily created python implementation of the base model for GPT-2. Using that hasty implementation in particular, we quickly created stories that, at their worst, were nonsensical and, at their best, whimsically absurd. We intended the demonstration to be nothing more than a curio, something that would — hopefully — help the students to value their own creativity and imagination above all else. As it often happens with young eager minds, their engagement and fascination with the capabilities of modern text-generation AIs demonstrated that there was more to our plan than we realised. There was a definitive, useful, task for AI in literary creation: this is exactly what they needed to get over 'writer's block,' a tool that could unleash their creativity when it is too bound by the precepts of reality. The nonsensical stories were an extreme example of that, but even nonsense can be helpful to spark the imagination.

From this conjunction of opportunity and need, the idea for the Creativity Engine was born: an app that would help young writers write the kind of story they would like to write, its name stating its primary purpose: to help creativity rush forwards. The interface would need to be clean, familiar, attractive and, above all, intuitive so as not to distract from the app's primary goal; the model powering the engine would have to be specifically tailored to young writers, trained on an appropriate corpus.

Support

The project was supported from the beginning by NUHRI — Newcastle University's Humanities Research Institute, the English Association (EA) who would provide access to keen beta-testers in the target demographic, and the Vital North Partnership (VNP), the steering group that oversees the partnership between researchers at Newcastle University, Newcastle City Council, and Seven Stories — the national centre for children's books. It also received additional support from The Alan Turing Institute

Seven Stories itself became the principal partner in the project: it is the ideal host for the app, it has a ready audience that would benefit from the app's goals, and it has the perfect archive that could be used to fine-tune the text-generating language model.

How we made the Creativity Engine

The Creativity Engine is made of two separate components: one, the interface used for user interaction -- that is, everything you can see, click, or write on the page. The other, is the generation engine, i.e., the service responsible for taking your story and coming up with its continuation. Althgouh connected, these components are different in the technologies they use, the approach they take, and their development cycle. Let's look at each one in turn.

What we call the generation engine is, itself, a complex product. At its heart is a Large Language Model (LLM) or, to put it more simply, a text generating Artificial Intelligence that will return new text based on a prompt given by the user. You may have heard of GPT-3 or its more famous incarnation, ChatGPT.

The generation engine works in a similar manner, although it uses a different language model as its base. ChatGPT and its base model are owned by a company called OpenAI which, in spite of its name, is a private company with commercial interests. Because of this, when we first started developing the Creativity Engine, GPT-3 (the most advanced language model at the time) was not freely available. We also had some concerns about the base data used to train the base model -- which included a gigantic dataset made of outbound links from Reddit. While this is a valuable dataset for natural language generation, it includes a lot of material that is not suitable for children and young people which increases the risk of the final model returning toxic results.

Instead, we chose EleutherAI's GPT-Neo, an alternative model with a more curated base data, with similar performance to GPT-3 and open source. Choosing the base model was only the start. Although, out of the box, GPT-Neo was capable of producing coherent outputs, it lacked what is sometimes called 'domain knowledge,' that is, the ability to produce content that would be more suitable for its intended use (in our case, write fiction for children and young people). The next step was to fine-tune the model, and we do this by giving it a good amount of data related to the type of result you want to give. So, if you were to fine-tune a model to write newspaper articles, you would have to feed it with thousands (if not millions) of newspaper articles: at the end, the model would be able to write convincing articles by itself; what we did was take the archives of six authors housed at Seven Stories to train the model to write stories like them.

This task sounds a lot easier than it actually is. For the model to be able to understand a dataset like ours, it needs to be prepared in a specific way. Most importantly, it needs to be readable. The Seven Stories archive was mostly made up of photographs of writing material: papers, notebooks, drafts, scraps, both handwritten and printed. While humans can read material like this (although, sometimes, with difficulty, depending on the handwriting), this is very difficult for computers. For a computer, there is a world of difference between the word 'Hello' as it written in a text editor and a photograph of the word 'Hello' -- and all we had were photographs. So our first step was to translate -- or more accurately transcribe -- all that material into digital text.

This difficulty is sometimes circumventd by making use of a specific group of technologies called OCR (Optical Character Recognition), a specific piece of software that transforms a photograph of text into digital text. For many applications, this might be sufficient, but OCRed text is often littered with mistakes, particularly if the original was handwritten. For our purposes, these kinds of misspellings are bad: the final product would create text with similar mistakes. So, after passing the entire dataset through OCR software, we then had to check, correct, and occasionally transcribe in its entirety every single document by hand. This was a long, arduous, but necessary step to create the Creativity Engine.

Once we had our dataset fully prepared, it was time to fine-tune the model. Although, technically, this could be done with any computer, a normal desktop or laptop computer would take days or even weeks to fine-tune the model to an acceptable state, working continuously, 24 hours a day. Fine-tuning consists of the model reading the entire dataset a good number of times and, at each time, altering itself slightly to produce results that are more in line with the dataset. This is all done automatically, but the computing power needed for this kind of 'learning' is tremendous. So, for this part of the process, we made use of an High-Performance Computer (HPC), specifically a computer named Bede (after the 8th Northumbrian scholar and historian), owned and maintained by N8, a consortium of eight northern universities that includes Newcastle University. Instead of weeks, Bede was capable of fine-tuning our language model in a few hours, which allowed us to iterate on the results to improve the performance of the generation engine further.

After a few rounds of fine-tuning, we ended up with four functional language models that we were happy with: two models that would generate poetry and two models that would generate fiction. These models were then tested in workshops with students and teachers from across the country which helped us choose the final one that is available today.

Our first challenge was complete: we had a AI that had been trained on the archive from Seven Stories and could help people write their stories. It was working on our laptops, even if it took a few minutes to come back with an answer. Our second challenge was figuring out how we could make this service available to anyone around the world, without requiring the users to download and configure a complex software package. So we had two more challenges in our horizon: we needed to create a web application that would allow users to interact with the language model in an intuitive way, and we had to package and make the model available in a way that would be quick, cost conscious and resource efficient.

From the very beginning, we had a clear idea of what the interface should look like: we conceived of the Creativity Engine as a way to receive help from some of the best authors in children's literature to write your own story, so it seemed natural that the 'conversation' concept should be translated into a familiar format, a chat app. This would, hopefully, ease the users into being able to immediately interact with the model, without having to first learn how to work with a completely novel piece of software. The objective was that, for the majority of users, there would be no need to give instructions on how to use the Creativity Engine.

The second objective was to make the interface not only functional but also attractive, light-hearted, and, to a certain extent, customisable, so that users would continue to interact with it after the first encounter with the Creativity Engine. You, dear reader, might be best placed to judge whether or not we managed to accomplish that objective but, at the very least, we hope you can seen where we attempted to do that: the bright colours, the clear deleniation between what is written by Humans and what is written by the AI, large responsive buttons, user avatars, and many other features of the user interface.

The first draft of the interface was well received by the project stakeholders, so what you see here now is remarkably similar to what we first draw by hand. We then went on to create a quick prototype that could be shown more widely, specifically to teachers brought together by the English Association, to collect their feedback on the functionality and design of the Creativity Engine. It is thanks to their feedback that you can, for example, change the colours and fonts of the app to better suit your tastes and needs, among many other smaller (and sometimes) less visible improvements.

Although, given the complexity of the functionality we wanted to implement, developing the interface took the biggest chunk of development time, it was, in many ways, the easier part of the project. All the techonologies we employed were familiar and, to some extent, routine. The final step in developing the interface was a series of workshops with students across the country in which we collected feedback on the looks, functionality, and perfomance of the Creativity Engine.

So, we had a model capable of helping users to write their stories, and we had an interface that would make that interaction intuitive. The final piece of the puzzle was to get these two separate elements talk to each other. If fine-tuning the model was (relatively) quick and easy, and creating the interface was long but easy, connecting the two not only took a long time, it was also the hardest aspect of the project, constantly marred by unforseen problems and budgetary concerns.

As we explained before, Large Language Models are, by definition, large and computationally demanding. Generating a small piece of of text in a normal laptop could take the better part of five minutes, far too long for any user. To get around this problem, we needed a special computer in the cloud that could do that job faster and more efficiently. The problem is that that kind of service is usually expensive -- and the faster it is, the more expensive it will become. And when we say 'expensive,' we mean really expensive -- potentially hundreds if not thousands of pounds a month, if the generation engine (i.e., the model and the computer that runs the model combined) was to be left turned on continuously for 24 hours a day. We don't have that kind of budget and we wanted to keep the Creativity Engine free for users, so we had to accept a few compromises, the main one being the wake-up time if the generation engine has been idle for a while. You can read more about why this happens below, in the Frequently Asked Questions.

So, the biggest challenge to serving the model to the interface was finding a service that could do it fast enough that was, at the same time, affordable enough. Because of the size and computational needs of the model, the easiest and cheapest options were all unavaiblable to us; and because all cloud computing providers work slightly differently, each new service we tried and tested usually incurred a few further weeks of development time. It was a long and troublesome process and, even when we thought we had finally figured it out, the service we had settled on was suddenly no longer available, which meant we had to restart our search from the beginning. We could write entire academic papers about the challenges of hosting and serving a Large Language Model (and in fact, we did -- if you are up to it, you can watch Tiago and Kathryn discussing some of this process at the 2022 Research Software Engineering conference), so we won't bother you with the details know. The main take away here is that this was a long and hard process, but eventually we were able to make the language model available to anyone.

The final step was getting these two components -- the user interface and the generation engine -- talking with each other. That is, if a user writes 'Once upon a time' we have to send that to the model and return the result to the user. This is achieved through an Application Programming Interface (API), a fancy way of describing a way for two computers to communicate with each other. To put it in incredibly simple terms, and to continute the chat app analogy, when you write 'Once upon a time' on the interface of the Creativity Engine, the interface sends a text to the generation engine; after a few seconds, the generation engine texts the interface back with 'Ok, here's the continuation of the story', which the interface shows the user.

Obviously, there is a lot more to this process, but at its core, that's how the two components of the Creativity Engine communicate. If you want to know more technical details, you can have a look at the technologies we used to develop it.

In total, the Creativity Engine took about a year and half to be developed. What you see now is the final result, although smaller improvements and corrections can and are being made all the time. We hope you enjoy the Creativity Engine, and we hope that it makes you more curious about Seven Stories, its archive, creative writing, and the technology we used to create it. Above all, we hope you have fun and write some cracking stories!

Technologies

If you are curious about the specific technologies we used to develop the Creativity Engine, here is a list of all of the ones that made it to the final app:

• OCR: Azure Computer Vision
• Model fine-tuning and handling: huggingface Transformers and PyTorch
• Design mockup and prototyping: Figma
• App Interface: Flask, JQuery, Shepherd, plain JavaScript.
• Cloud Infrastructure: AWS Async Inference
• Inference API: huggingface Inference API

Acknowledgements

We would like to extend our thanks, first and foremost, to the authors -- John Agard, Valerie Bloom, Toni Mitton, Grace Nichols, and Michael Rosen --, or their representatives, for their enthusiasm for the project and their permission to use their names and archive in making the Creativity Engine. We'd also like to thank Dr Rebecca Fisher and Elizabeth Draper, from the English Association for their help and support throughout the development of the project, Hollie Barnes-Lomax, Jenny Drain, Vannessa Havill, and Pamela Dale, their colleges Cambridge Regional College, Reaseheath College, Joseph Chamberlain College, and Calderdale College, and their students for welcoming us and allowing us to run feedback workshops. Finally, we would like to think Alice Kim, who helped to develop a small but important part of the interface while on a student placement with the RSE team, our colleagues at the RSE team who helped us throughout the project, our colleages at the Vital North Partnership, and the Newcastle University Humanities Research Center (NUHRI) who supported the project, Clare Vint and Chris O'Keeffe, as well as the many who tried, commented on, and shaped the Creativity Engine at many public demonstrations and talks.

Credits

• Original idea: Tiago Sousa Garcia
• Idea development: Tiago Sousa Garcia, Jennifer Richards, Laura Mckenzie, Mark Turner
• Manual transcription and corrections: Tiago Sousa Garcia, Kathryn Garside, Ibrahim Ali, Rowan Stewart, Isabella Mathieson, Juliana Arcila
• Model fine-tuning: Kathryn Garside
• Interface development and MLOps: Tiago Sousa Garcia
• Marketing, Coordination, PR, Launch program: Rachel Pattinson
• Launch program support: Jacques Verkade, Georgia Conlon
• Seven Stories representative and author liason: Kristopher Mckie
• Principal Investigator: Jennifer Richards

The Creativity Engine is made up of two parts: the user interface and the generation engine.

The user interface is everything that you can see and click on the screen: all the information about the authors, about the project, and the system that allows you to write your story, read the suggestions made by the AI, edit or save your story. This is always available, all day, everyday (in principle -- sometimes things are temporarily unavailable because of technical problems).

The generation engine, however, is a separate part of the Creativity Engine. The generation engine is responsible for taking the bits of your story that you submit and generate a suggestion for its continuation. It does this by using a particular form of Artificial Intelligence that takes your story and determines what is the most likely way to continue it. Unlike the user interface, the generation engine is very demanding in terms of computer power: if you were to run it directly on your computer, it would likely take the better part of 5 minutes to come up with something.

Because it is so demanding, it uses a lot of energy and is very expensive to run. So, if the generation engine hasn't been used by anyone for a while, we simply turn it off until it is needed again. When you see the message 'Our authors have been asleep for some time - we're waking them up! Try again in a few minutes.' it means that the no one (in the world!) has used the generation engine for about 15 minutes, so it has been turned off.

As soon as you've seen that message, an instruction has been sent to turn the generation engine back on. This, unfortunately, takes some time: about 5 minutes, more or less. This is a long time to wait, we know, but it is the only way that does not unnecessarily waste energy and money: every time you have to wait for the generation engine to be available, think about all the energy that we saved by turning it off: it would be like leaving all the heating on at full blast in your home when no one is in: sure, when you get back, the house is nice and warm, but you've wasted a large amount of energy to save a few minutes of discomfort when getting home.

To help you get a sense of whether you will have to wait or not, we created a little traffic light system on the top right of the screen to inform you whether the generation is available or not at a glance: if the first (red) light is lit, it means that the generation is not currently available. It means the generation engine has been turned off, or that the Creativity Engine is currently going through some technical difficulties. If the second (orange) light is lit, it means that the generation engine is currently in the process of being turned on or turned off (most likely turned on): you can submit your bit of story during this time, and the system will reply as soon as it is ready. If the third (green) light is lit, it means everything is ready to help you write your stories.

The good news is that the generation engine is unlikely to become unavailable once it has been turned on and while you are using the Creativity Engine: as long as you (or someone else anywhere in the world) interacts with it within 15 minutes, it will remain available. However, remember that the Creativity Engine is meant to help you write, not write the whole thing for you: so getting away from it for a bit to develop the story on your own might be a good idea.

If you've seen the 'Our authors have been asleep for some time - we're waking them up! Try again in a few minutes' message, there's nothing else for you to do: just sit back and relax and, as soon as the light turns green on the top right of the screen, you are good to go.

If the light on the top right of the scree is orange, it means the generation engine is in the process of being turned on. At this point, you can submit the start (or continuation) of your story, and the system will come back to you as soon as it is ready.

If the light in the traffic light system is red (the first light is lit), all you need to do to wake up the authors is to submit something to the Creativity Engine: this first message is not recorded, so it can be literally anything like 'wake up please!'.

If you have submitted something but the light is still red, don't despair: it takes some time between sending the instruction to start the generation engine and the process to start. You can send another wake up message at this time if you want to make sure, but this will not, unfortunately, make it turn on any faster. While you wait for the generation engine to start, you can always learn more about the authors or how we created the Creativity Engine. Keep an eye out on the traffic light system, it will let you know as soon as the generation engine is available.