Research Report| "Generative AI": application, research, and supervision work together to open up the innovation path of generative artificial intelligence

Source: Qiming Venture Partners

Original title: "Qiming Headlines | Qiming Venture Partners and Unfinished Research jointly released the report "Generative AI" | The State of Generative AI 2023"

At the 2023 World Artificial Intelligence Conference (WAIC)** Qiming Venture Partners Forum "Generative AI and Large-scale Models: Change and Innovation", Qiming Venture Partners joined hands with Unfinished Research to jointly release a blockbuster report "Generative AI 》| State of Generative AI 2023. **

If 2022 is called the year of generative artificial intelligence, breakthroughs have been made in the application of diffusion models, ChatGPT was born, and a series of groundbreaking research papers were published. In 2023, the large model will be pushed to a peak, with the release of GPT-4 as the The sign, generative artificial intelligence, has entered the stage of innovative application towards the direction of general artificial intelligence.

**The most important feature of this stage is that application, research, and supervision are all working together to open up the innovative path of generative artificial intelligence. **

INNOVATIVE APPLICATIONS

People quickly saw the emergence of a new business ecology from generative artificial intelligence, saw layer after layer of technology, such as calculations, models, and applications; saw the generated content, such as text, pictures, videos, codes, 3D structures, multi-modality; also see open data, vertical data, synthetic data, vector data, for large and small models.

Generative artificial intelligence seems to be more enthusiastically accepted in China: the government encourages the development of general artificial intelligence; no big company can ignore it; many small and medium-sized enterprises engaged in knowledge work have already used it first. ** In the face of this revolutionary technology, all businesses are involved. They have different rhythms and different degrees of involvement, and they have become defenders, innovators, and adopters under the wave of new technologies. Their profit margins are permanently changed. **

Computing power is currently the scarcest resource, and it is also in the most profitable position. **Computing power is the largest part of the cost structure of the large model, and the performance of the GPU actually determines the pace of this emerging industry. ** With the advancement of computing power and models, more start-ups are pouring in. They have grabbed the dividend of time, but they are also facing competition and possible giant crushing. It can be said that this is the blue ocean of start-ups, and there are also hidden reefs under the channel.

**Competition fosters innovation. **Different from the rapid emergence of startups in the direction of productivity tools in 2022, in 2023, a greater proportion of new companies will focus on the innovation of underlying technologies; large-scale startups have also begun to differentiate, and in the ascendant general large-scale startups At the same time, many vertical large-scale companies in specific directions such as medical care, e-commerce, scientific research, industry, autonomous driving, and robotics began to emerge. **

Frontier Research

2022 and 2023 are the two years when generative artificial intelligence technology will make breakthroughs. We sorted out the papers and found that a prominent feature in the field of generative artificial intelligence is the close integration of research and innovation processes, many of which are implemented within enterprises. , roll out use cases and products quickly. **This integration of research and entrepreneurship, start-ups and venture capital have played an important role, and the research investment and talents of US technology giants and major artificial intelligence companies, including the research of some underlying technologies, have surpassed universities over the years and other research institutions.

**The frontiers of artificial intelligence are pushing into the future. **Although from GPT-4's technical reports to Microsoft's research papers, it has shown that it has close to human word processing ability, mathematical reasoning ability, and knowledge in many professional fields. "We think it can reasonably be considered an early (albeit still incomplete) version of an artificial general intelligence (AGI) system." too much. Such as confidence calibration, long-term memory, continuous learning, personalization, planning and concept leapfrogging, transparency, cognitive fallacies and irrationality, etc. **

**The most important research direction in the past six months is to decipher and understand the mysterious and exciting "emergence" of intelligence in large models. **Large models not only need to surpass the ability to predict the next word, but also need a richer and more complex deep mechanism of "slow thinking" to supervise the mechanism of "fast thinking" to predict the next word.

**The best cutting-edge research must be to study and solve the problems encountered in the application of large-scale technology. **Research on how to reduce hallucinations, adjust the large model to output real content more accurately, and train stronger reasoning ability; how to train the model more intensively, lower the threshold, launch new products, and let more people from all walks of life and consumers How to interact with the real physical world like a human being; how to become an assistant to human beings in complex work, design and help carry out scientific experiments; how to influence employment and make policy responses; how to make artificial intelligence safe and believable.

Regulation| Security| Policy| Talent

The government's regulatory response to generative artificial intelligence is quite timely, and different characteristics have emerged in different countries. **While rapidly introducing regulatory measures for generative artificial intelligence and seeking opinions, China is also encouraging the development of general artificial intelligence. Beijing, Shanghai, and Shenzhen are the most ambitious first echelons, and they have all proposed more ambitious artificial intelligence Research, innovation and industrial goals. **EU continues to lead in regulation and legislation, as it did with GDPR 5 years ago. The United States is more concerned about the leading position of artificial intelligence technology, and is forming a regulatory framework based on the principle of risk management.

** In the long run, talent will have more impact on the future of artificial intelligence than computing power. **The number of papers published by Chinese researchers has surpassed that of the United States, but the United States still has a clear advantage at the top of the pyramid, whether it is research or entrepreneurship. Globally, the focus of artificial intelligence research and innovation is shifting from universities to enterprises. The top three institutions in the United States with the most top scholars are Google, Microsoft and Meta, which together recruit 30% of the top scholars in the United States. China is still dominated by universities, and only Alibaba ranks among the top 10.

The Ministry of Science and Technology has proposed that artificial intelligence enterprises should accept scientific and technological ethics review; the subject of the review should set up a scientific and technological ethics (review) committee. American artificial intelligence companies started to set up a responsible and credible artificial intelligence department earlier. From last year to this year, some adjustments have been made, reflecting that when generative artificial intelligence is undergoing changes, enterprises are seeking to use better technologies and solutions. Deploy new technologies safely and responsibly.

Ten Prospects

Large Language Model

1. In 2024, China will have a multilingual general-purpose model comparable to GPT-4;

2. Long Context will lead the next breakthrough in LLM technology;

3. Before a more promising large language model emerges, in order to achieve better results in the vertical field, the following three methods will coexist:

i) Without changing the data distribution, use more general data for general large-scale model pre-training, and do not specifically introduce industry data,

ii) Use industry-specific data to fine-tune (Fine-Tuning) the general large model,

iii) Use data sets with a higher proportion of industry data for vertical model pre-training.

Multimodal Model

4 The current Vinsen diagram model of CLIP + Diffusion is a transitional state, and an integrated model structure will appear in the next 2 years;

5. The next-generation Text-to-Image model will have stronger controllability. It will combine the capabilities of the underlying model and the front-end control method, and the design of the model will focus on the combination with the control method;

6. Before 2025, video and 3D modes will usher in milestone models, greatly improving the generation effect;

7. The embodied intelligence (Embodied AI) represented by PALM-E has shown great potential in the direction of robot perception, understanding and decision-making, but there are great challenges in current training and reliability;

8. In the short term, Transformer is becoming the mainstream network structure of multiple modalities, but a general method for compressing the entire digital world has not yet appeared. Transformer is not the end of artificial intelligence technology.

Business opportunity

9. Within 3 years, the core driving force of subversive AI applications will come from the innovation of the underlying model, the two cannot be decoupled, and the role of the model will be greater than that of product design;

10. The current generative AI market is in the early stage of technology dominance, and there are opportunities for platform companies with a market value of hundreds of billions of dollars.

Read the original text