Posts by Tags

Language Models

Recycling is Looping: From Protein Folding to LLMs

Published:

Deep learning runs on a handful of simple ideas that scale with data and compute. And because these methods share a common language of networks, ideas that work for board games or proteins can often be carried over to text. Reasoning models are one example. They borrow from AlphaZero’s recipe, pairing reinforcement learning with more computation at inference time to explore possible solutions.

Model Architecture

Recycling is Looping: From Protein Folding to LLMs

Published:

Deep learning runs on a handful of simple ideas that scale with data and compute. And because these methods share a common language of networks, ideas that work for board games or proteins can often be carried over to text. Reasoning models are one example. They borrow from AlphaZero’s recipe, pairing reinforcement learning with more computation at inference time to explore possible solutions.

OpenAI

How Confident Are You, ChatGPT?

Published:

TL;DR: The key to OpenAI’s IMO gold likely wasn’t a universal verifier. Calibration and uncertainty estimation are central, and will shape future products.

Protein Folding

Recycling is Looping: From Protein Folding to LLMs

Published:

Deep learning runs on a handful of simple ideas that scale with data and compute. And because these methods share a common language of networks, ideas that work for board games or proteins can often be carried over to text. Reasoning models are one example. They borrow from AlphaZero’s recipe, pairing reinforcement learning with more computation at inference time to explore possible solutions.

RL

How Confident Are You, ChatGPT?

Published:

TL;DR: The key to OpenAI’s IMO gold likely wasn’t a universal verifier. Calibration and uncertainty estimation are central, and will shape future products.

Reasoning Models

How Confident Are You, ChatGPT?

Published:

TL;DR: The key to OpenAI’s IMO gold likely wasn’t a universal verifier. Calibration and uncertainty estimation are central, and will shape future products.