Earth & Climate • NASA
Climate science: scaling petabyte-scale climate analytics without sacrificing scientific fidelity
Modern climate science relies on petabytes of simulation and observational data, but analyzing these datasets at full resolution is often prohibitively expensive. NSDF enables scientists to work at the fidelity required for their analysis—reducing storage, data movement, and computation by orders of magnitude while preserving the scientific accuracy needed for climate research.
The Scientific Challenge
Climate datasets such as NASA’s 2.8 PB ECCO ocean simulation contain decades of high-resolution observations essential for understanding Earth’s climate. Processing these datasets at full resolution requires enormous storage, bandwidth, and computational resources, making large-scale analyses expensive and inaccessible to many researchers. Traditional compression reduces cost but often sacrifices scientific fidelity.
How NSDF Helps
NSDF organizes climate data into a hierarchical multiresolution representation, allowing scientists to analyze lower-resolution data first and retrieve higher-resolution information only when needed. Machine learning reconstructs fine details from reduced datasets, while adaptive fidelity enables users to balance computational cost against scientific accuracy for each analysis.
Scientific Impact
Using the 2.8 PB NASA ECCO climate dataset, NSDF reduced data usage by 99%, lowering storage, transfer, and computational costs from more than $100,000 to approximately $24, while maintaining an RMS error of only 1.46°C. This enables researchers to perform large-scale climate analyses much more efficiently without losing the essential scientific information needed for decision making
Key Outcomes
- 99% reduction in data volume
- $100K → $24 computational and storage cost
- 2.8 PB NASA ECCO dataset analyzed efficiently
- Adaptive fidelity balances scientific accuracy with computational efficiency
- Machine learning reconstruction preserves critical climate features while minimizing data movement