With AI tools becoming more common across industries to assist human tasks, the promising potential of LLMs has become obvious, exhibiting their capacity to automate a wide array of tasks, ranging from simple fact-checking to complex content generation. However, leveraging these AI models effectively poses challenges, particularly in crafting compelling prompts that draw out optimal responses. The art of creating such prompts has given rise to a new field known as Prompt Engineering.

For those using AI tools for quite some time now, overloading queries in a prompt could actually generate the worst response, which is why crafting compelling prompts has emerged as a pivotal skill. In this article, we will explore the most common prompting techniques or methods you can utilize in prompt engineering. These prompting methods are critical for optimizing the models’ capabilities and realizing their potential in real-world applications.

Zero-Shot Prompting

Zero-shot prompting is the most basic form of interaction with an AI. It is a method used to generate responses without providing any template to work with, allowing LLMs to provide a response when the user does not provide any additional context. The whole idea behind this prompting method is to allow a model to rely entirely on its previous training to generate output.

Zero-shot prompting is the most common method of AI prompting used by most people as it is the natural way for them to interact with it. However, this prompting method is prone to hallucinations as you are simply just asking a question, with no extra context, information or examples provided. Poor or inaccurate results are commonplace in this type of prompting, which can cause frustrations on someone inexperienced in using AI.

Few-Shot Prompting

Few-shot prompting is a method where the model is given a number of examples. These examples can condition the model to generate desired outputs, be it text, code, or images. Similar to how retrieval-augmented generation or RAG works, data and information are included in the message or prompts.

While most LLMs are exceptional at basic tasks with zero-shot prompts, they usually struggle with more complex tasks. Few-shot prompting acts as in-context learning, guiding the LLMs to improve performance by providing demonstrations within the prompt.

The efficiency of few-shot prompting varies according to the number of examples provided—typically between two to five examples. However, this method has limitations in handling specific reasoning tasks. This means that more examples provided does not automatically equate optimal responses. Feeding too many examples can become overly complex for some models.

Chain–of–Thought (CoT) Prompting

Chain-of-Thought or CoT prompting is an advanced prompting method that requires breaking down a complex task into simpler and logically connected sub-tasks. At its core, CoT prompting is about guiding the LLM to think step by step. This is accomplished by giving the model a few-shot examples that explains the reasoning process. Then it follows a similar chain of thought when responding to the prompt.

Like the previously mentioned prompting methods, CoT prompting has its limitations. For starters, it only produces performance advantages when used with models of around 100 billion parameters. Smaller models tend to respond with illogical chains of thought, resulting in poorer accuracy compared to regular prompting. Additionally, the performance boosts from CoT prompting are often proportionate to the model’s size.

So how does chain-of-thought differ from few-shot prompting? Well, the latter is when you provide a few examples so the LLM can understand what it should do, while the former is about showing the step-by-step thinking from start to finish, which helps with “reasoning” and getting more detailed answers. Basically, CoT prompting is all about showing the work, not just the answer.

Tree of Thoughts (ToT) Prompting

The Tree of Thoughts (ToT) prompting method utilizes a tree-like structure (unlike CoT prompting that takes a more linear path) of each step of the reasoning process that allows the model to evaluate each reasoning step and decide whether or not that step in the reasoning is valid and will lead to an optimal answer. Once the model sees the reasoning path will not lead to a desirable answer, its prompting strategy requires it to abandon that path and move forward with another branch, until it reaches the sensible result.

Tree of Thoughts is similar to AutoGen by Microsoft having multiple experts with different system messages interacting in a group and chaining their responses. Think of prompting the LLM to debate itself, point out flaws in its reasoning, provide a response, and then critique that response, all in a single prompt. It gives very good responses because rather than just predicting the next token, it takes its output and thinks through it.

While it is a more advanced prompting technique, it is also the least likely to give the structure of the output you want, so it's not appropriate for every use case.

Takeaways

Proper prompting is key to unlocking the full potential of AI models. While zero-shot prompting allows models to rely on their training, few-shot prompting guides models by providing relevant examples. For complex reasoning tasks, chain-of-thought prompting breaks down the task into logical steps. Tree-of-thoughts prompting has models critique their own reasoning paths to reach optimal answers.

Though prompt engineering continues to evolve, techniques like these allow us to tap into the vast capabilities of LLMs. With further research, we may develop more generalized prompting techniques to optimally guide these models, bringing us closer to artificial general intelligence. However, prompt crafting is a learned skill. Experience and experimentation with different techniques can help you get the most out of LLMs for a given task.

In the end, compelling prompts that provide proper context and examples remain the key to deliver the remarkable potential of large language models.

References:

• https://machinelearningmastery.com/what-are-zero-shot-prompting-and-few-shot-prompting/
• https://www.linkedin.com/pulse/prompt-engineering-101-zero-shot-dustin-hughes
• https://www.allabtai.com/prompt-engineering-tips-zero-one-and-few-shot-prompting/
• https://promptsninja.com/few-one-zero-prompting/
• https://www.linkedin.com/pulse/prompt-engineering-education-content-few-shot-niall-mcnulty.
• https://www.kdnuggets.com/2023/07/power-chain-thought-prompting-large-language-models.html
• https://www.vellum.ai/blog/chain-of-thought-prompting-cot-everything-you-need-to-know
• https://www.searchenginejournal.com/tree-of-thoughts-prompting-for-better-generative-ai-results/
• https://dev.to/zokizuan/tree-of-thoughts-a-new-way-to-prompt-ai-2dle
• https://iq.opengenus.org/different-prompting-techniques/#introduction