The UK Treasury has set aside a budget of £900 million to invest in the development of a supercomputer that would be powerful enough to chew through more than one billion billion simple calculations a second. A new exascale computer would fit the bill, for utilization by newly established advanced AI research bodies. It is speculated that one key goal is to establish a "BritGPT" system. The British government has been keeping tabs on recent breakthroughs in large language models, the most notable example being OpenAI's ChatGPT. Ambitions to match such efforts were revealed in a statement, with the emphasis: "to advance UK sovereign capability in foundation models, including large language models."

The current roster of United Kingdom-based supercomputers looks to be unfit for the task of training complex AI models. In light of being outpaced by drives in other countries to ramp up supercomputer budgets, the UK Government outlined its own future investments: "Because AI needs computing horsepower, I today commit around £900 million of funding, for an exascale supercomputer," said the chancellor, Jeremy Hunt. The government has declared that quantum technologies will receive an investment of £2.5 billion over the next decade. Proponents of the technology have declared that it will supercharge machine learning.

We've created GPT-4, the latest milestone in OpenAI's effort in scaling up deep learning. GPT-4 is a large multimodal model (accepting image and text inputs, emitting text outputs) that, while less capable than humans in many real-world scenarios, exhibits human-level performance on various professional and academic benchmarks. For example, it passes a simulated bar exam with a score around the top 10% of test takers; in contrast, GPT-3.5's score was around the bottom 10%. We've spent 6 months iteratively aligning GPT-4 using lessons from our adversarial testing program as well as ChatGPT, resulting in our best-ever results (though far from perfect) on factuality, steerability, and refusing to go outside of guardrails.

Over the past two years, we rebuilt our entire deep learning stack and, together with Azure, co-designed a supercomputer from the ground up for our workload. A year ago, we trained GPT-3.5 as a first "test run" of the system. We found and fixed some bugs and improved our theoretical foundations. As a result, our GPT-4 training run was (for us at least!) unprecedentedly stable, becoming our first large model whose training performance we were able to accurately predict ahead of time. As we continue to focus on reliable scaling, we aim to hone our methodology to help us predict and prepare for future capabilities increasingly far in advance—something we view as critical for safety.