Hugging Face· Platform

PRX Part 4: Our Data Strategy

Welcome back! This is Part 4 of the PRX series. Parts 1 to 3 covered model architectures , training design , and a 24-hour speedrun . This time we're pulling back the curtain on the part that quietly underpins all of it: the data. Of all the things that shaped PRX's quality, the data pipeline was one of the least glamorous parts to build but nevertheless an important piece to get right. Here's what we did, what we'd do differently, and a few things we only learned the slow way. In one sentence: we assemble training data from a mix of public and internal datasets, re-caption the images with a VLM, and turn the result into the streamable corpus we trained PRX on. At a high level, the data pipeline looks like this: In the following we will dive into it in detail. 1. Guiding principles A diverse dataset for pre-training The goal was to assemble a large, diverse dataset for pre-training. …

Announced

06 Jul 2026 15:30 UTC