skinny.

Latest / Misar.Blog Podcast

DPO vs RLHF: Data Requirements and When to Use Each

DPO vs RLHF compared: pipeline complexity, preference data requirements (5k pairs vs 50k+ comparisons), compute costs, and a practical decision matrix for… From the article "DPO vs RLHF: Data Requirements and When to Use Each" by Synor, published on Misar.Blog.

In this episode:

0:00 Use DPO Unless You Have a Specific Reason Not

0:29 Classic RLHF Requires Substantially More Data and Resources

1:06 RLHF and Online Methods Still Win for Frontier Alignment

1:42 DPO Delivers 90 % of the Value at 10 % of the Complexity for Style

2:23 The Data Floor for RLHF Is High Because a Poorly-Trained RM Can…

The skinny

The skinny isn't ready yet — notes appear once the transcript is processed.