Explained: GPT

Explore GPTs: from definition and applications to ethical risks. Discover how they transform content, customer service, healthcare, and more.

What is a GPT?

Generative pre-trained transformers (GPTs) are a type of large language model (LLM) used in generative artificial intelligence (GenAI). GPTs can also be defined as artificial neural networks (ANNs) used in natural language processing (NLP).

GPTs are built on a deep learning architecture called a transformer, pre-trained on large data sets of unlabelled text and can generate human-like content.

What are OpenAI’s GPT-3, 4, and other GPT variants?

OpenAI’s GPT-1, launched in 2018, was the first-ever GPT model. Since then, the Microsoft-backed startup valued at $86 billion has launched four more foundational GPT models aimed at generating content. 

GPT-1 had 117 Mn parameters (internal variables that the model uses to make predictions or decisions) and was trained on 4.5 GB of data at a training cost (amount of computing resources needed to train an LLM) of 1 petaFLOP/s-day (1020 operations a day). 

GPT-4, in contrast, has 1.7 Tn parameters, a million times more than GPT-1, and OpenAI has not publicly stated the corpus of data it was trained on. While the decacorn also did not specify the training cost for GPT-4, it has been estimated to be around 21,000 petaFLOP/s-day (around 2.1*1025 operations a day).

With its capabilities, including learning from human preferences (LFHP) and multimodality (ability to generate output across multiple media forms), GPT-4 is one of the most advanced GPT models to date.

Other GPT models include Google’s PaLM, Gemini, and Meta’s LLaMA. For comparison, Google’s PaLM has 540 Bn parameters, landing it somewhere between GPT-3.5 and GPT-4 in terms of sheer computing power.

What are the applications of GPTs?

GPTs are designed to enable a wide range of downstream applications. This involves fine-tuning the GPT models to adapt to certain applications, such as chatbots. As such, this flexibility gives current GPTs the capability to adapt to applications such as:

• Content Creation: One of the most popular and important applications of GPTs has been to generate content. GPTs generate text, code, music, images, videos, and more. They can also translate languages with increasing accuracy and fluency, as GPT-4 is being touted by OpenAI as its most accurate model to date.
  
• Customer Service & Communication: GPT-powered chatbots can handle customer inquiries and resolve issues more efficiently, freeing up human agents for more complex tasks. GPTs can also generate content for social media platforms, manage online communities, and respond to customer comments. Several Indian startups, including Swiggy and Zomato, have automated their chat support via GPT-based tools.
  
• Education & Learning: GPTs can generate personalized learning plans for individuals based on their needs and learning pace. Further, GPTs can also act as interactive tutors, which Indian and global edtech companies are doing. For instance, Indian edtech startups like BYJU’S have included GPT-based features in their products and interfaces.
  
• Healthcare: GPTs are being explored for their potential to analyze medical data and assist doctors in diagnosis and treatment planning. They are also being used to generate new drug candidates and predict their potential effectiveness, accelerating the drug discovery process. Indian health tech startups are employing GPT-based solutions for use cases such as customer interaction and medical data analysis, among others.

What are some of the Concerns Regarding GPTs?

• Bias and Discrimination: GPTs are trained on large text datasets, which can reflect societal biases and prejudices. This can lead to discriminatory or offensive outputs.
  
• Factual Accuracy and Misinformation: GPTs can generate text that sounds convincing but could be factually incorrect. This can be especially dangerous in news reporting, healthcare, and scientific research, where inaccurate information can have serious consequences.
  
• Misuse: Bad actors use GPTs to generate deepfakes, create fake news, or launch phishing attacks. The recent wave of deepfakes of popular figures across India and globally has become a cause of concern for governments worldwide.
  
• Economic Impact: As GPTs become more sophisticated, they could automate jobs currently done by humans, leading to unemployment and economic disruption. Several companies are already replacing humans with GPT-based AI models in functions such as online customer support. In December 2023, Paytm sacked hundreds of employees, citing AI-based automation across operations and marketing.
  
• Lack of Transparency and Explainability: GPT models’ methods of arriving at their outputs are poorly understood, making it hard to assess reliability and address potential biases. This lack of transparency can hinder trust and accountability in their use.

Which startups are currently working with GPTs?

Since the launch of GPT-3.5 in March 2022 and the subsequent launch of ChatGPT in November 2022, startups have started developing AI-based use cases. Startups across segments such as edtech, fintech, ecommerce, enterprise tech, and consumer services have developed AI-based use cases and features for their products.

However, many startups develop in-house GPT models for highly specific use cases across content generation and no-code platforms. Some examples include Kombai and GlazeGPT.