Intelligence artificial today it is on everyone's lips. By now we know how to use it: just access the internet and use platforms such as ChatGPT, Dall-E and Synthesia. In this regard, we are preparing an interesting article about 10 websites that use artificial intelligence to do very different things. But having said that, let's take the example of the most widely used AI. How was it born? how we arrived at the version that we can all use today free?
Today many of us use ChatGPT but not everyone knows what's behind it and how we got there. Here is a timeline from the 80s to today
Let's start from the 80s: Recurrent Neural Networks
ChatGPT is a version of GPT-3, a very large language model developed by OpenAI. Language models are a type of neural network who has been trained on very many texts. Neural networks are software inspired by the way neurons in human brains communicate with each other. Since any text is composed of sequences of letters and words of varying lengths, language models require a type of neural network capable of making sense of this type of data. The recurrent neural networks (Recurrent Neural Networks) invented in the 80s, can handle sequences of words, but are slow to train and can forget previously learned words in a sequence. LSTMs could handle text strings of several hundred words, but their linguistic capabilities were limited. What are? Acronym for "Long short-term memory"Or"long-term memory” is an artificial neural network used in the fields of artificial intelligence
Read also: Google confirms the compatibility of content generated by Chatbot and AI
2017: the year of artificial intelligence breakthrough with the Transformers
The breakthrough that has led to the current generation of large language models came when a team of researchers at Google invented the Transformers, a type of neural network that can keep track of where each word or phrase appears in a sequence. You will understand for yourself that the biggest problem of LSTM is overcome. But how did they do it? The concept of applied linguistics enters technology. The meaning of a word often depends on the meaning of other words that precede or follow it. Keeping track of these contextual information, Transformers can handle longer strings of text and capture the meaning of words more accurately. For example, "hot dog" has a very different meaning in the sentences "I prefer hamburgers to hot dogs"and "Hot dogs are best eaten with mustard“. In essence, contextual information, which is what human beings grasp and machines don't, has made it possible to make a difference.
2018-2019: the years of GPT development
OpenAI's first two major language models arrived within months of each other. The company wanted to develop multipurpose and general purpose AI and believes that large language models are a key step towards this goal. In this way the software is able to spot patterns in the data yourself, without being told what they are watching or reading. Many previous successes in machine learning have relied on supervised learning and annotated data, but manual data labeling is slow work that limits the size of datasets available for training. It was the GPT-2 to create the biggest stir. In fact, OpenAI said it was so concerned people might be using GPT-2”to generate deceptive, distorted or abusive language” which would not release the full model. But that's not all.
If GPT-2 was impressive, the sequel to OpenAI, the GPT-3, has literally made the revolution. Its ability to generate human-like texts represented a great leap forward. GPT-3 can answer questions, summarize documents, generate stories in different styles, translate between English, French, Spanish and Japanese, and much more. However, it still cannot replace a human being because it lacks the founding characteristics of humanity. We talked about it in depth this article.
Source | MIT
