Free to use, Google invested $300 million in Anthropic to officially launch Claude 2, directly benchmarking GPT-4

Image source: Generated by Unbounded AI

With its investment in OpenAI, Microsoft has killed all sides in this AI revolution. Its rival Google, not to be outdone, tried to use the same method to fight, betting heavily on OpenAI's rival generative AI startup Anthropic. Anthropic is to Google what OpenAI is to Microsoft.

Anthropic itself has deep roots with OpenAI. Due to differences in the development direction of OpenAI, worried that after Microsoft's first $1 billion investment in OpenAI, it would make it on a more commercial path and deviate from its original intention. At that time, the team led by OpenAI R&D Vice President Dario Amodei chose Left and founded Anthropic in 2021.

Anthropic has raised $700 million in funding by the end of 2022, with $500 million from Alameda Research. Google's cloud arm then invested $300 million for a 10 percent stake, a deal that required Anthropic to buy computing resources from Google Cloud. In May 2023, Anthropic raised $450 million in a funding round led by Spark Capital. To date, Anthropic has raised nearly $1.5 billion in funding.

In March of this year, Anthropic released Claude, an AI chat robot product similar to ChatGPT. Claude, with 52 billion parameters, was initially available in closed beta via a Slack integration, and users can now access Claude via Quora's Poe app, as well as six other chatbots. However, Claude's heat is obviously far less than ChatGPT and the latest GPT-4.

On July 11, Anthropic launched a new version of its AI chatbot: Claude 2, which aims to compete directly with GPT-4.

Anthropic said its latest model, Claude 2, has improved performance in coding, math and reasoning, and can generate longer responses.

Claude 2 has a context window of 100k tokens, the largest among all commercial models. Claude 2 is capable of processing approximately 75,000 words of text at a time. Users can enter large data sets and ask for a summary in the form of a memo, letter, or story. In comparison, ChatGPT can process 3,000 words of text at a time, while GPT 4 can process 25,000 words of text. 10% of the Claude 2 training data set is non-English data, which is not a problem when processing Chinese requests.

Claude 2 vs. GPT-4

Nvidia senior AI scientist Jim Fan said Claude 2 is cheaper than GPT-4 and much more powerful in inference and encoding than older versions.

But on the standard test, it has not quite reached GPT-4, but compared to Claude v1.3, it has caught up quickly.

• GRE Speaking Test: 165 vs 169 (GPT-4 wins) *GRE Writing Test: 5 vs 4 (Claude 2 wins)
• GRE Math Test: 154 vs 163 (GPT-4 wins)
• USMLE: ~67 vs ~85 (GPT-4 wins)
• Bar (USB): 76.5 vs 75.7 (Claude 2 wins)

In inference benchmarks:

• Human encoding: 71.2% vs 67% (Claude wins). It is worth mentioning that the score of GPT-3.5 is only 48%. Claude's coding ability has been significantly improved.
• GSM-8K Elementary Math: 88% vs 92% (GPT-4 wins). Compared to the previous version, Claude improved from 85.2% to 88%.
• Claude 2 (100K context) is 4-5x cheaper than GPT-4-32K! Assuming similar token lengths, token cost is $11/million token vs $60/million token, and completion cost is $32/million vs $120/million token.

The knowledge update deadline for Claude 2 is early 2023, while GPT-4 is September 2021. So the content of Claude 2 is much fresher. Beyond these benchmarks, Claude 2's improvements in coding, math, and reasoning are impressive.

With its support for excessive text, some users think that Claude2 has killed all "chatPDF" tools - you can now upload PDF files and let Claude2 analyze them for you. Claude2 currently supports summarizing and analyzing up to five documents within 10 MB at the same time. Support pdf, text, csv and other formats.

A user tried to upload Tesla's latest Q1 earnings report and asked, "What are the main takeaways from Tesla's Q1 earnings update? Can you make any price predictions?"

Claude 2 is able to answer this question perfectly and has sources.

Other users uploaded Google's latest earnings records and asked them to summarize the main points. The result is much better than ChatGPT with code interpreter, which has 8k token limit.

Claude 2 is currently free for UK and US users (those interested can try it scientifically, it is a good alternative in the case of GPT-4 requiring a paid subscription.