How to train OpenAI GPT-3

Introduction

GPT is a neural network machine made by OpenAI. Its name stands for “generative pre-trained transformer.” It uses deep learning techniques to generate human-like responses to your questions. In this tutorial series, I will teach you how to train OpenAI GPT-3 and create a sample web application on mental health that tries to understand a situation and responds in the best possible way.

This blog is a part of three tutorial series where I will teach

1. Part 1 – How to train OpenAI GPT-3.

   In this part, I will use the playground provided by OpenAI to train the GPT-3 according to our used case on mental health

2. Part 2 – Create GPT-3 application with Node.js.

   This tutorial will cover how the training data used above can be used inside a real application

3. Part 3 – GPT 3 fine tuning.

   In this blog, I will teach you how to fine-tune your application to send the best responses.

Difference between ChatGPT and GPT-3

People often confuse ChatGPT with GPT-3 because they have canonical names and are developed by the same organization. But actually, ChatGPT uses GPT-3 as its core engine which has been trained in a particular way to respond to the way we perceive when we raise a question to ChatGPT.

OpenAI playground to train GPT

The OpenAI playground is a basic web application where anyone can try GPT-3 in action. To use the playground, you will need to log in to OpenAI. Now let’s head over to Playground – OpenAI API and ask a question. Next, hit Enter (for the next line) and submit. It will respond to your query.

Here I have asked the GPT-3 program a very simple question on mental health and see its response

In the playground, don’t get overwhelmed by the tools on the right side. It’s a little complicated, for sure, but not very difficult. If you want to read about the settings on Playground, https://techpro.ninja/openai-codex-with-node-js-tutorial/ and look for OpenAI Playground Settings

Let’s see how GPT-3 answered my question

Did you see how the answer is very generic as if the program is somehow trying its best to provide a solution to the problem without having any knowledge of the subject? Yes, which is why we need to make our program learn more about the subject. The more we train OpenAI GPT-3 it, the better it becomes in handling the situation. Now let’s try to give some basic training to our chat application.

Training the bot

GPT-3 uses Prompt based engineering, which means that the program needs to be provided with parameters so that it can identify the context of the problem. I will give more context to the bot about my intention and it should answer the previous problem in a better way.

Let’s dive in and check the Playground again with more context of the situation

There are 3 major components in training the OpenAI GPT-3, namely

Show and tell

A basic brief about what you intend to accomplish in a text format written at the top of the body as a paragraph. In my case, the context is

Application to precede the mental health diagnosis before going to a psychologist. We will listen to the patient very calmly and try to understand the issue and its root causes. We will keep asking a variety of questions in and around the issue and ultimately try to understand the root cause. Once, we are thorough, we will prepare a summary and suggest a psychologist for further diagnosis. If we are unsure of the given situation, we will answer with a question asking for more details, and only if the patient asks about suicide, we will provide them suicide helpline number

Provide quality data

The program will learn how to handle the situation from the sample prompt and answers, along with the way you are putting the answers, like the answer length, the tone of your answers, etc. Here is what I provided, in my case
P: I have a mental health issue
A: Alright, we will try to understand the issue and identify the root cause. Once we understand your problem thoroughly, we will try to find a solution.
P: I have a very abusive manager at the office
A: Can you give me some background about your office culture
P: I work in a multi-national company and my manager is not sympathetic toward the problems in team building. Instead, he thinks that these are non-issue and we should only focus on work and compete with each other.


Settings

The panel on the right-hand side gives us control over how the output should be provided for each prompt. Important setting parameters are Temperature and Maximum Length. Temperature controls how varied the response can be from the original topic. The more the value, the larger the answer varies. Maximum Length controls the length of the answer. Visit this blog post and look for OpenAI Playground Settings to learn all about playground settings.

With that said, let’s try to talk about our application again and ask our first question again. In the Playground, hit submit and ask your question again just below Step 2.

Q: I have a mental health issue

A: We understand that you are facing a mental health issue and we are here to help. We will start by listening to your story and understanding the root cause of your issue. Once we understand your problem thoroughly, we will try to find a solution and suggest a psychologist for further diagnosis.

Now, you see, the program has started to behave the way we want it to. That was the basics of GPT-3 training. With more trial and error on prompts, answers, and context, and by providing quality data, you can make the system more efficient.

Conclusion

I hope you have learned the basics of how to train the OpenAI GPT-3 system. In the next blog post, we will create a sample application using Node.js and learn how to use the above training data in the application to interact with GPT-3