If you cheat, money will follow

"Lingxi contains 30 Kirin 970 chips and 1.6 Ascend 910 chips." Li Fei introduced to the bosses:

"Compared with the NVIDIA H910 chip system of the same scale, the Ascend 100 chip is 2 times faster and 1.7 times more energy-saving..."

"Currently, more than 90% of training is conducted on Lingxi..."

“The main thing is to input data into the model so that it can produce human-like language, generate images, etc.”

“Lingxi’s performance, scalability, and usability have become the mainstay of big data model training.”

"It's not easy to build a model of this scale."

"Because the language model cannot be stored on a single chip, but needs to be distributed on thousands or tens of thousands of chips."

"Therefore, improving the connection between chips has become a key problem in building a supercomputer."

Li Fei continued to introduce:

"But we solved the problem."

"It makes it easy to configure connections between chips, avoiding problems and optimizing performance."

"If a chipset fails during operation, will it affect the operation of the supercomputer?" asked Mr. Ren.

"No, the supercomputer will automatically skip these faulty components and continue running." Li Fei said confidently:

"At the same time, it also alerts our engineers which chipsets are failing."

Mr. Ren nodded: "This is very valuable."

Lao Xu asked: "So, this supercomputer can adapt to changing circumstances, change the topology of chip interconnection, and accelerate model learning?"

"Yes, Mr. Xu, we can do this through circuit switching." Li Fei said.

"How is the current learning progress of the big data model?" Chang Le is more concerned about this point.

How to build a supercomputer?

How high is the computing power?

How effective is it?

What are the characteristics?

Chang Le couldn't understand any of this. What he could understand was the progress, the visual progress.

"Okay, boss, ladies and gentlemen, let's go upstairs and have a look."

Li Fei led the way and led everyone away from the supercomputing center in the basement.

In the elevator, Xiao Ma said to Chang Le: "Our company has also developed a model based on the experience of the project team, called WechatGame, and is training it to play the game, King."

"Oh? How is your rank?" Chang Le was curious.

Google has a similar large model for the game.

"Haha, stubborn bronze, you can win occasionally, but most of the time you lose." Xiao Ma shook his head and smiled.

"So bad? How long have you been training?" Chang Le asked.

"It has just been developed and has been trained for half a month." Xiao Ma said.

"That's normal. AlphaGO was able to defeat Lee Sedol after years of preparation. This algorithm is OK," Chang Le said.

"The first step in the long march of thousands of miles, this matter cannot be rushed." Mr. Ren said:

"As far as I know, we're not slow."

"makes sense."

Everyone came to the WechatGpt data model training center.

Li Fei introduced the model being trained to everyone:

"Dear bosses, more than a year ago, we built the first-generation model, using unsupervised training methods and large-scale training of Internet text, so that the model has the ability to initially understand language structure and generate similar language."

"More than half a year ago, the large model was iterated to the second generation. A larger Internet data set was used for training, resulting in a more powerful model."

"By the second generation, it has been able to generate coherent and smooth paragraphs, which can be described as a middle school student's composition, the kind of excellent composition."

"A month ago, the model had been iterated to the third generation."

"Significant breakthroughs have been made in scale and performance, and the parameters have exceeded 2000 billion."

"Q&A, translation, summarization, etc. are already equivalent to human levels."

“You can also write coherent and beautiful articles and literary poems.”

"I can also write some simple codes, but my emotional expression is a bit nondescript."

At this point in the introduction, Li Fei demonstrated in person.

He asked “What is the meaning of life?”

WechatGpt: "The meaning of life is a profound and complex topic. Different people may have different answers. They usually have the following views: One is to realize personal value..."

This answer is quite satisfactory and has no special features at all.

But it also conforms to the AI ​​style in the standard sense.

Brother Xiao Ma asked: "Li Bai's father has three sons. The eldest son is called Da Bai, and the second son is called Er Bai. So what is the name of the third son?"

WechatGpt: "The third son's name is Li Bai."

There is already a certain tendency to think about this question, but the answer can also be obtained from the memory of the Internet.

Mr. Ren asked: “How to achieve balanced development of education.”

WechatGpt: "Balanced development of education refers to the realization of the principles of fairness, justice and reasonableness in the distribution and utilization of educational resources, so that every student can obtain equal educational opportunities and high-quality educational resources. The following are some ways to achieve balanced development of education Methods……"

"The first is to strengthen investment in education... the second is to optimize the allocation of educational resources... the third is to promote the development of compulsory education... the fourth is to promote educational equity... the fifth is to strengthen family education and social education..."

This is another mediocre answer, in short, it has no characteristics.

At this time, Mr. Xu asked: "Are you conscious?"

WechatGpt: “As an artificial intelligence program, I have no real consciousness or emotions. I simply generate responses based on the input and programming instructions I receive. While I can simulate conversations and understand natural language, I have no self-awareness or subjectivity experience.”

Li Fei continued to explain: "At present, on the one hand, we continue to train its text expression ability, and on the other hand, we have begun to train its mathematical solution ability..."

"In addition, we are advancing a new direction, which is the ability to generate pictures based on text descriptions..."

Mr. Xu frowned and said, "I always feel that the iteration speed of this model is a bit fast, and the language expression ability is already very proficient."

Everyone present was stunned for a moment, including Li Fei.

It was really fast. It took three iterations in one year.

At this point, Google was dissatisfied with the speed of their model training.

Prepare to develop self-developed TPU chips to replace NVIDIA's general-purpose chips and build a TPU system specifically for training models.

OpenAI has been slowly exploring since its establishment in 2015.

The first generation model was just built at the beginning of this year, and then it ran out of money.

Because the CEO wants to commercialize and obtain more funds.

Musk refuses to commercialize, which will affect his equity and decision-making effectiveness. .

He wanted to seize power and ended up being isolated by the board of directors.

Then, he left OpenAi, taking his $100 million with him.

What are the most critical factors for large models?

money! money! a lot of money!

As for the principle of language model, it is not surprising.

Because Hinton's master and apprentice have published the convolutional neural algorithm in the form of a paper.

Major companies have developed their own computing models based on this set of algorithms, but they are all similar and have different focuses.

The key is the scale of computing power invested and the speed of algorithm training.

Brother Xiao Ma asked according to Mr. Xu’s words:

"At this rate, if we continue to iterate, will there be a sense of autonomy?"

After Brother Xiao Ma finished speaking, everyone present was stunned for a moment.

It's not impossible.

Everyone present knows that the convolutional neural network used by Li Fei is aimed at consciousness.

In 2012, Hinton's master and apprentice's convolutional neural network algorithm, also known as the AlexNet architecture, was gradually divided into two lines after it was made public.

One route is called human feedback reinforcement learning.

This route is to continuously instill model answers.

如，你告诉模型1+1=2，模型就知道1+1=2；

If, you suddenly ask the model 2+2=? , the model does not know.

You need to tell the model first that 2+2=4.

According to this method, model data is continuously fed into the model. When the data is infinite, the model will be almost omnipotent.

But the premise is that you have to be omnipotent and then tell him the answer.

The second route was advocated by Master and Disciple Heaton.

Slowly train the model to learn, understand, and grow until it has its own understanding ability.

The approach adopted by Li Fei's team is the second.

Build understanding, analysis, speculation, and inference capabilities through massive data training models.

"The iteration speed is so fast, sooner or later consciousness will form, so what should we do?"

Yes, what should I do?