How does Ultra-FineWeb-L1 compare to FineWeb?

Ultra-FineWeb-L1 and FineWeb are both cleaned Common Crawl corpora. In OpenBMB's own ablation, a MiniCPM5-1B run pretrained on Ultra-FineWeb-L1 reaches a 9.668% macro-average across downstream evaluations against 9.033% for FineWeb, and 9.180% micro-average against 8.484%. Ultra-FineWeb-L1 also draws on 2025 crawl snapshots, so its pages are newer.

Can Ultra-FineWeb-L1 be used commercially?

Ultra-FineWeb-L1 carries an Apache 2.0 license, but OpenBMB adds two conditions. Users must still respect the rights, licenses and terms of the original web sources the text was crawled from. OpenBMB also prohibits reposting, mirroring or commercially repackaging the project artifacts without prior written permission, so check the dataset card before redistributing.

Is there a higher-quality subset of Ultra-FineWeb-L1?

Yes. OpenBMB also lists Ultra-FineWeb-from-L1, a quality-selected subset drawn from the L1 layer. In the same MiniCPM5-1B ablation it reaches a 10.379% macro-average and 9.798% micro-average, the best of the four corpora OpenBMB compares. The L1 release is the broad filtered pool that these tighter subsets are selected from.

How does Ultra-FineWeb-L1 fit with Ultra-FineWeb-L3?

OpenBMB sorts UltraData into tiers from L0 to L4. Ultra-FineWeb-L1 is the filtered layer: raw crawl text after extraction, language filtering, deduplication and cleaning. Ultra-FineWeb-L3, released on 2026-05-28, is the synthetic refinement layer built further up the stack. L1 is the foundation the later tiers are selected and rewritten from.

OpenBMB · 2026-08-20 · major

Ultra-FineWeb-L1 — a 1.3T-token open web corpus for pretraining

Ultra-FineWeb-L1 is a 1.3 trillion token English web corpus from OpenBMB, cleaned from six 2025 Common Crawl snapshots and free under Apache 2.0. OpenBMB reports it beats FineWeb by 0.635 points on its macro-average of downstream evals.

Hugging Face dataset card for OpenBMB's Ultra-FineWeb-L1 pretraining corpus

A 1.3T-token English web corpus from OpenBMB, cleaned from six 2025 Common Crawl snapshots and free under Apache 2.0.

Quick facts

Maker	OpenBMB
Size	1.3T+ tokens, ~1.14B documents
Language	English
Source data	Six Common Crawl snapshots, CC-MAIN-2025-30 to CC-MAIN-2025-51
License	Apache-2.0
Format	Parquet
Layer	L1 (filtered) in the UltraData L0-L4 framework

What is it?

Ultra-FineWeb-L1 adds 1.3 trillion English tokens, roughly 1.14 billion documents, drawn from six Common Crawl snapshots taken during 2025 and ending at CC-MAIN-2025-51. The data ships as Parquet files under an Apache 2.0 license. It sits at the L1 filtered layer of OpenBMB's UltraData framework, which runs from raw crawl data up to synthetic refinement.

How does it work?

Six cleaning steps produce the corpus. trafilatura 2.0 pulls the main text out of each page, fastText drops non-English documents, heuristic rules cut low-quality text, and sensitive fields such as emails, IP addresses and phone numbers are replaced. MinHash deduplication then removes near-copies inside each crawl dump, and a final pass repairs leftover HTML and broken character encodings. Every record keeps a uid, the cleaned text and JSON metadata with the source URL.

Why does it matter?

Pretraining data is the piece of the stack that fewest labs publish, so a fresh 1.3 trillion token corpus gives small teams something they normally cannot buy. OpenBMB reports that the same MiniCPM5-1B training run scores 0.635 points higher on its macro-average of downstream evaluations when it reads Ultra-FineWeb-L1 instead of FineWeb. The 2025 snapshots also move the data's knowledge cutoff forward compared with older public corpora.

Who is it for?

teams pretraining open base models

Frequently asked questions

How does Ultra-FineWeb-L1 compare to FineWeb?: Ultra-FineWeb-L1 and FineWeb are both cleaned Common Crawl corpora. In OpenBMB's own ablation, a MiniCPM5-1B run pretrained on Ultra-FineWeb-L1 reaches a 9.668% macro-average across downstream evaluations against 9.033% for FineWeb, and 9.180% micro-average against 8.484%. Ultra-FineWeb-L1 also draws on 2025 crawl snapshots, so its pages are newer.
Can Ultra-FineWeb-L1 be used commercially?: Ultra-FineWeb-L1 carries an Apache 2.0 license, but OpenBMB adds two conditions. Users must still respect the rights, licenses and terms of the original web sources the text was crawled from. OpenBMB also prohibits reposting, mirroring or commercially repackaging the project artifacts without prior written permission, so check the dataset card before redistributing.
Is there a higher-quality subset of Ultra-FineWeb-L1?: Yes. OpenBMB also lists Ultra-FineWeb-from-L1, a quality-selected subset drawn from the L1 layer. In the same MiniCPM5-1B ablation it reaches a 10.379% macro-average and 9.798% micro-average, the best of the four corpora OpenBMB compares. The L1 release is the broad filtered pool that these tighter subsets are selected from.
How does Ultra-FineWeb-L1 fit with Ultra-FineWeb-L3?: OpenBMB sorts UltraData into tiers from L0 to L4. Ultra-FineWeb-L1 is the filtered layer: raw crawl text after extraction, language filtering, deduplication and cleaning. Ultra-FineWeb-L3, released on 2026-05-28, is the synthetic refinement layer built further up the stack. L1 is the foundation the later tiers are selected and rewritten from.

Try it

load_dataset("openbmb/Ultra-FineWeb-L1")