AI terms

Artificial Intelligence - The simulation of human intelligence by machines.

Machine Learning is a subset of AI that focuses on developing algorithms that allow computers to learn from data and make decisions based on that data utilizing statistical predictions.

Deep Learning is a subset of machine learning. Focus on artificial neural networks with multiple layers. Excels at handling vast amounts of unlabeled data. Not all Machine Learning is part of deep learning.

Foundation models are specific types of deep learning models. They are typically built using a specific kind of neural network architecture, called a transformer, which is designed to generate sequences of related data elements (for example, a sentence).

Generative AI refers to a set of AI algorithms that can generate new outputs — such as text, images, code,  or audio — based on the training data, unlike traditional AI systems that are designed to recognize patterns and make predictions. Sometimes the AI that powers these solutions is referred to as decoders.

A large language model (LLM) is a type of foundation model that has been trained on large quantities of unlabeled text using self-supervised learning and can perform a variety of natural language processing (NLP) tasks (even when that language is a programming language). Output may range from books, articles, social media posts, online conversations, and even code. The architecture of an LLM consists of layers of neural networks that learn to generate language in a way that is similar to how humans use language.

Hallucination is a well-known phenomenon in large language models (LLMs) in which the system provides an answer that is factually incorrect, irrelevant, or nonsensical because of limitations in its training data and architecture; more concerning is the hallucinated answer often sounds plausible.

Natural language processing (NLP) is the technology that gives computers the ability to understand text and spoken words in much the same way human beings can. NLP combines computational linguistics — rule-based modeling of human language — with statistical, machine learning, and deep learning models. These technologies enable computers to process human language in the form of text or voice data and to ‘understand’ its full meaning, complete with the speaker or writer’s intent and sentiment.

Prompt – input and query that users or programs use to interface with foundation models so they can respond with useful/desirable results. A prompt can be a simple NLP question, or it can be a large body of text. The structure of the prompt is very important in eliciting proper responses from foundation models.

Prompt Engineering – Prompt engineering is the process of crafting prompt text to best effect a given model and parameters.

Decoder-only model – Models designed explicitly for generative AI use cases; represents the architectures used in GPT-3 and other popular Large Language Models.

Encoder-only model – Models with best cost performance trade-off for non-generative use cases but require task-specific labeled data for fine-tuning.

Encoder-decoder model  - Models that support both generative and non-generative use cases. These have the best cost-performance trade-off for generative use cases when the input is large but the generated output is small.