What is generative AI?

What is generative AI?

Generative artificial intelligence (AI) is a type of deep learning model that can produce text, images, computer code, and audiovisual content in response to prompts.

Generative AI models are trained on vast quantities of raw data — generally, the same kinds of data they are built to produce. From that data, they learn to form responses, when given arbitrary inputs, that are statistically likely to be relevant for those inputs. For example, some generative AI models are trained on large amounts of text, in order to be able to respond to written prompts in a seemingly organic and original manner.

In simpler terms, generative AI can react to requests much like human artists or authors, but more quickly. Whether the content these models generate can be considered "new" or "original" is up for debate, but in many cases they can match or exceed certain human creative abilities.

Popular generative AI models include ChatGPT for text generation and DALL-E for image generation. Many organizations have built their own models as well.

How does generative AI work?

Machine learning, deep learning, and neural networks

Generative AI is a type of machine learning, which means it relies on mathematical analysis to find relevant concepts, images, or patterns. It then uses this analysis to produce content that is statistically likely to be similar or related to the prompt it received.

Specifically, generative AI relies on a type of machine learning called deep learning. Deep learning models are powerful enough to learn from unlabeled data. They use a type of computing architecture called a neural network. Such architectures are composed of multiple nodes that pass data to each other, similarly to how a human brain passes data via neurons. Neural networks are able to perform highly sophisticated and refined tasks.

Transformers and self-attention

Generative AI models that can interpret language need to understand more than individual words. They must be able to interpret entire sentences, paragraphs, and documents. Early machine learning models struggled with understanding entire sentences and would "forget" the beginning of a sentence by the time they reached the end, resulting in misinterpretation.

Modern generative AI models use a specific kind of neural network called transformers. These use a capability called self-attention to detect how elements in a sequence are connected. Transformers enable generative AI models to do things like process and contextualize large blocks of text instead of just individual words and phrases.

Training data

In order to work well, generative AI models have to be fed a large amount of data: more data, in most cases, than a human could process over the course of a lifetime. For example, the large language model ChatGPT was trained on millions of documents. An image generator might be trained on millions of images, and a code generator on billions of lines of code.

This training data is stored in a vector database. In such a database, points of data are stored as vectors — or, a set of coordinates within a multi-dimensional field. Just as latitude and longitude enable someone looking at a map to find nearby locations, storing data as vectors enables machine learning models to find "nearby" points of data. This allows models to make associations and understand the context of a word, an image, a sound, or any other type of content.

Once the generative AI model reaches a certain level of fine tuning, it does not need quite as much data in order to produce a result. For example, voice-generating AI models may be trained on thousands of hours of vocal recordings. But once they are fine-tuned, some models only need a few seconds of a sample recording to realistically imitate someone's voice.

What are the benefits and drawbacks of generative AI?

Generative AI models are growing in popularity, since they offer a number of potential benefits. These benefits include, but are not limited to:

• Content ideation: The use of generative AI can help content creators come up with a creative direction more quickly.
• Better chatbots: Generative AI models can be integrated into chatbots to better answer customer questions, engage prospects, and so on.
• Enhanced research: Generative AI models can rapidly process vast amounts of data, including medical data or scientific studies, to aid in research.
• Improved search results: Search engines and virtual assistants can incorporate generative AI capabilities to more quickly provide relevant information in response to queries.
• Entertainment: Many people use publicly available generative AI tools solely for fun.
• Other benefits: AI is a rapidly growing field, and further benefits from generative AI are likely still yet to come.

However, generative AI does come with its share of drawbacks, including:

• Hallucination and other inaccuracies: Generative AI models are typically very good at identifying patterns, but sometimes they identify patterns that do not actually exist. This can result in the models providing false information, a phenomenon known as "hallucination." Additionally, generative AI models are only as accurate as the data they are fed, and fact-checking generative AI outputs can prove difficult without access to the source data.
• Data leaks: Models can take the data they are fed in prompts and reveal it in unexpected contexts. Several large enterprises have accidentally leaked confidential information or source code in this way.
• Accidental plagiarism or misuse of intellectual property: Because generative AI models are based on preexisting content, they may reproduce content they were fed without permission from that content's original author or copyright holder.
• Malicious response manipulation: Attackers can feed data to a generative AI model that causes it to produce dangerous or unsafe information for other users.
• Biases: Any biases in the information fed to a model as it is trained are likely to be retained or even exacerbated unless the model is fine-tuned to correct for them. And even then, ensuring results are free from bias without reviewing the entire training set is near-impossible.

What is a large language model (LLM)?

"Large language model" (LLM) is the technical term for generative AI models that process language and can generate text, including both human-spoken languages and programming languages. Popular LLMs include ChatGPT (from OpenAI), Llama (from Meta), Bard (from Google), Copilot (from GitHub), and Bing Chat (from Microsoft).

What is an AI image generator?

AI image generators work similarly to LLMs, but for images instead of text. DALL-E and Midjourney are two examples of popular generative AI-based image generators.

Does Cloudflare support generative AI development?

Cloudflare empowers developers and businesses to build their own generative AI models. Cloudflare offers Vectorize so that developers can generate and store embeddings on the Cloudflare global network from their own data, and Cloudflare Workers AI for running generative AI tasks on a global network of GPUs. Learn more about how Cloudflare is enabling the next generation of generative AI.