2023 Publications and Conferences
Studying Latency and Throughput Constraints for Geo-Distributed Data in the National Science Data Fabric
Conference: HPDC ‘23: Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing
Date: August 2023
Authors: Jakob Luettgau, Heberth Martinez, Glenn Tarcea, Giorgio Scorzelli, Valerio Pascucci, Michela Taufer.
BibTeX
@inproceedings{10.1145/3588195.3595948,
author = {Luettgau, Jakob and Martinez, Heberth and Tarcea, Glenn and Scorzelli, Giorgio and Pascucci, Valerio and Taufer, Michela},
title = {Studying Latency and Throughput Constraints for Geo-Distributed Data in the National Science Data Fabric},
year = {2023},
isbn = {9798400701559},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3588195.3595948},
doi = {10.1145/3588195.3595948},
abstract = {The National Science Data Fabric (NSDF) is our solution to the problem of addressing the data-sharing needs of the growing data science community. NSDF is designed to make sharing data across geographically distributed sites easier for users who lack technical expertise and infrastructure. By developing an easy-to-install software stack, we promote the FAIR data-sharing principles in NSDF while leveraging existing high-speed data transfer infrastructures such as Globus and XRootD. This work shows how we leverage latency and throughput information between geo-distributed NSDF sites with NSDF entry points to optimize the automatic coordination of data placement and transfer across the data fabric, which can further improve the efficiency of data sharing.},
booktitle = {Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing},
pages = {325–326}, numpages = {2},
keywords = {xrootd, high-performance computing, perfsonar, data democratization, cloud computing},
location = {Orlando, FL, USA},
series = {HPDC '23}
}
Development of Large-Scale Scientific Cyberinfrastructure and the Growing Opportunity to Democratize Access to Platforms and Data
Conference: Proceedings of the 25TH International Conference On Human-Computer Interaction (HCII)
Date: July 2023
Authors: Jakob Luettgau, Giorgio Scorzelli, Valerio Pascucci, and Michela Taufer
BibTeX
@inproceedings{luettgau2023development,
title={Development of Large-Scale Scientific Cyberinfrastructure and the Growing Opportunity to Democratize Access to Platforms and Data},
author={Luettgau, Jakob and Scorzelli, Giorgio and Pascucci, Valerio and Taufer, Michela},
booktitle={International Conference on Human-Computer Interaction},
pages={378--389},
year={2023},
organization={Springer}
}
Enabling Scalability in the Cloud for Scientific Workflows: An Earth Science Use Case
Conference: 2023 IEEE 16th International Conference on Cloud Computing (CLOUD)
Date: June 2023
Authors: Paula Olaya, Jakob Luettgau, Camila Roa, Ricardo Llamas, Rodrigo Vargas, Sophia Wen, I-Hsin Chung, Seetharami Seelam, Yoonho Park, Jay Lofstead, and Michela Taufer.
BibTeX
@inproceedings{olaya2023enabling,
title={Enabling Scalability in the Cloud for Scientific Workflows: An Earth Science Use Case},
author={Olaya, Paula and Luettgau, Jakob and Roa, Camila and Llamas, Ricardo and Vargas, Rodrigo and Wen, Sophia and Chung, I-Hsin and Seelam, Seetharami and Park, Yoonho and Lofstead, Jay and others},
booktitle={2023 IEEE 16th International Conference on Cloud Computing (CLOUD)},
pages={383--393},
year={2023},
organization={IEEE}
}
National Science Data Fabric
Conference: Campus Research Computing Consortium’s (CaRCC) Emerging Centers
Authors: Christine Kirkpatrick
Tailoring the National Science Data Fabric to Open Science and FAIR Aims
Conference: ISC
Authors: Christine Kirkpatrick
FAIR Digital Objects in Distributed Research Environments
Conference: Throughput Computing 2023 OSG All Hands Meeting
Authors: Natalie Meyers
Bridging the HPC/Data Divide
Conference: ISC
Authors: Christine Kirkpatrick
Why FAIR is Worthwhile for Big Science & Easy Steps to Get Started
Conference: National Virtual Biosecurity Bioenergy Crops Center (NVBCC) Computing and Cross-Cutting Workshops
Authors: Christine Kirkpatrick
A National Science Data Fabric to Democratize Data Access and Reusability
Conference: HPC ISC Conference
Authors: Michela Taufer, Jay Lofstead, Christine Kirkpatrick, Jakob Luettgau, and Valerio Pascucci
Unleashing the Power within Data Democratization: Needs, Challenges, and Opportunities.
Conference: International Conference for High Performance Computing, Networking, Storage, and Analysis (SC)
Authors: Valerio Pascucci, Michela Taufer, Ian Foster, Ilya Baldin, Franck Wuerthwein
A National Science Data Fabric to Democratize Data Access and Reusability
Conference: International Conference for High Performance Computing, Networking, Storage, and Analysis (SC)
Authors: Valerio Pascucci, Michela Taufer, Christine Kirkpatrick, and Jakob Luettgau
PC and Cloud Converged Computing: Merging Infrastructures and Communities
Conference: International Conference for High Performance Computing, Networking, Storage, and Analysis (SC)
Authors: Daniel Milroy, Michela Taufer, Seetharami Seelam, Bill Magro, Heidi Poxon, Todd Gamblin
The National Science Data Fabric: Democratizing Data Access for Science and Society
Authors: Valerio Pascucci
Other Publications 2023
Name | Conference/ Workshop | Authors |
---|---|---|
The state of NSDF | NSDF - AHM | Valerio Pascucci |
National Research Platform - A tatus Update | NSDF - AHM | Würthwein F |
Data Intensive Searches for Dark Matter with LUX-ZEPLIN (LZ) | NSDF - AHM | Monzani, M |
Data Locality for Large scale AI / ML training. | NSDF - AHM | Chen, S |
Testing Compressions with OpenVisus. | NSDF - AHM | Panta, A |
Web3 Cloud Storage - Decentralized Cloud Storage Use Cases | NSDF - AHM | Malki, S. |
Efficient Data Access & Migration Across Clouds. | NSDF - AHM | Fan, B |
Filecoin for researchers | NSDF - AHM | Langstroth, K |
Rolling Deck to Repository (R2R) - Challenges managing large data from the US Academic Research Fleet. | NSDF - AHM | K. Stocks; S. O’Hara; D. Clark; R. Hudak; E. Miller; S. Smith; L. Stolp; R. Uribe; V. Ferrini; S. Carbotte. (2023). |
Needs for Pacific Regional Cyberinfrastructure. | NSDF - AHM | Cleveland, S. |
A Data Fabric For Social Good?. | NSDF - AHM | Gupta, A |
Distributed Data Access in the National Security Complex. | NSDF - AHM | Bremer, P. |
Linking scientific instruments and computation: Patterns, technologies, and experiences. | NSDF - AHM | Chard, K |
Open Science Data Federation - OSDF | NSDF - AHM | Andrijauskas, F |
Clowder: Open Source, Customizable, Data and Work flow Management. | NSDF - AHM | Marini, L |
NSDF User Communities working group mission: gather user data stories. | NSDF - AHM | Gyulassy, A |
NSDF Protect and Access Management. | NSDF - AHM | Rodero, I |
Metadata Management to Aid Data Discovery. | NSDF - AHM | Lofstead, J |
OpenVisus evolution under the NSDF project. Backend evolution (modvisus), multiple deployments (Docker, Kubernetes, DockerSwarm), Material Science and CHESS experience. Preliminary results on WebAssembly: Bokeh dashboards and Jupyter notebooks running inside the browser. | NSDF - AHM | Scorzelli, G |
NSDF-Plugin: Integrating and Benchmarking Geographically Distributed Storage. | NSDF - AHM | Heberth Martinez; Jakob Luettgau; Giorgio Scorzelli; Glenn Tarcea; Valerio Pascucci; Michela Taufer. |
NSDF-Catalog: Lightweight Indexing Service for Democratizing Data Delivery. | NSDF - AHM | Jakob Luettgau, Giorgio Scorzelli, Glenn Tarcea, Christine Kirkpatrick, Valerio Pascucci, Michela Taufer. |
Research Cybersecurity: What’s Next, What Now. | NSDF - AHM | Corn, M. |
Portability of Applications to Heterogeneous Systems and Exascale. | NSDF - AHM | Petruzza, S |
Tailoring the National Data Science Fabric to Open Science & FAIR Aims. | NSDF - AHM | Kirkpatrick, C |
Kingfisher: Storage Management for Data Federations. | NSDF - AHM | Bockelman. B |
Leveraging Structured Data on the Web to Address FAIR Principles. | NSDF - AHM | Fils. D. |
Building a Large-Scale Community Data Portal with Commodity Hardware. | NSDF - AHM | Klacansky. P |
An overview of the “NSDF-Visualization-Portal.”: how NSDF allows the sharing of interactive visualizations through the Web. | NSDF - AHM | Koppe. O |
Enabling Scalability in the Cloud for Scientific Workflows: An Earth Science Use Case. | NSDF - AHM | Paula Olaya; Jakob Luettgau; Ricardo Llamas; Rodrigo Vargas; Jay Lofstead; Sophia Wen; I-Hsin Chung; Seetharami Seelam; Michela Taufer. |
Materials Commons Updates and Thinking Big By Thinking Small. | NSDF - AHM | Glenn Tarcea; Brian Puchala; Tracy Berman; John Allison |
NSDF Development at CHESS. | NSDF - AHM | Bougie, D |
Everything, Everywhere All at Once, All the Time - Challenges for Astronomy. | NSDF - AHM | Dodds, C |
Ochestration of materials science workflows for heterogenous resources at large scale | NSDF - AHM | Zhou, Naweiluo, Scorzelli, Giorgio, Luettgau, Jakob, Kancharla, Rahul R., Kane, Joshua J., Wheeler, Robert, Croom, Brendan P., Newell, Pania, Pascucci, Valerio, and Taufer, Michela. |
Other Products 2023
Type | Name | Url |
---|---|---|
Other | National Science Data Fabric Catalog Grows toward AI-Integrated Scientific Innovation | Url |
Other | Conference poster. NSDF@CHESS Democratizing the Cornel Light Source! | Url |
2022 Publications and Conferences
NSDF-Catalog: Lightweight Indexing Service for Democratizing Data Delivery
Conference: 2022 IEEE/ACM 15th International Conference on Utility and Cloud Computing (UCC)
Date: December 2022
Authors: Jakob Luettgau, Giorgio Scorzelli, Valerio Pascucci, Glenn Tarcea, Christine R. Kirkpatrick, Michela Taufer
BibTeX
@article{luettgau2022nsdf,
title={NSDF-Catalog: Lightweight Indexing Service for Democratizing Data Delivering},
author={Luettgau, Jakob and Kirkpatrick, Christine R and Scorzelli, Giorgio and Pascucci, Valerio and Tarcea, Glenn and Taufer, Michela},
year={2022}
}
Bridging the HPC/Data Divide BoF
Conference: Supercomputing 2022
Date: November 2022
Authors: Christine Kirkpatrick
Open Storage Network
Conference: Supercomputing 2022
Date: November 2022
Authors: Christine Kirkpatrick, John Goodhue, Melissa Cragin
Augmenting Singularity to Generate Fine-grained Workflows, Record Trails, and Data Provenance
Conference: 2022 IEEE 18th International Conference on e-Science (e-Science)
Date: October 2022
Authors: Dominic Kennedy, Paula Olaya, Jay Lofstead, Rodrigo Vargas, Michela Taufer
BibTeX
@INPROCEEDINGS{9973642,
author={Kennedy, Dominic and Olaya, Paula and Lofstead, Jay and Vargas, Rodrigo and Taufer, Michela},
booktitle={2022 IEEE 18th International Conference on e-Science (e-Science)},
title={Augmenting Singularity to Generate Fine-grained Workflows, Record Trails, and Data Provenance},
year={2022},
volume={},
number={},
pages={403-404},
doi={10.1109/eScience55777.2022.00059}}
The Materials Commons Data Repository
Conference: 2022 IEEE 18th International Conference on e-Science (e-Science)
Date: October 2022
Authors: Glenn Tarcea, Brian Puchala, Tracy Berman, Giorgio Scorzelli, Valerio Pascucci, Michela Taufer, John Allison
BibTeX
@inproceedings{tarcea2022materials,
title={The Materials Commons Data Repository},
author={Tarcea, Glenn and Puchala, Brian and Berman, Tracy and Scorzelli, Giorgio and Pascucci, Valerio and Taufer, Michela and Allison, John},
booktitle={2022 IEEE 18th International Conference on e-Science (e-Science)},
pages={405--406},
year={2022},
organization={IEEE}
}
Converged Computing: Bringing Together HPC and Cloud Communities
Conference: IEEE/ACM International Conference for High-Performance Computing, Networking, Storage, and Analysis
Authors: Daniel Milroy, Marquita Ellis, Sameer Shende, Michela Taufer, Ward Harold, and Yan Fisher
Toward a Lightweight Indexing Service for the National Science Data Fabric
Conference: Proceedings of the 18th IEEE International Conference on e-Science (eScience)
Authors: Jakob Luettgau, Giorgio Scorzelli, Nauweiluo Zhou, Glenn Tarcea, Jay Lofstead, Valerio Pascucci, and Michela Taufer
A Software Framework for Scientific Workflow Orchestration at Large Scale
Conference: Proceedings of the 18th IEEE International Conference on e-Science (eScience)
Authors: Nauweiluo Zhou, Jakob Luettgau, Rahul Reddy Kancharla, Joshua Kane, Brendan Croom, Robert Wheeler, Pania Newell, Giorgio Scorzelli, Valerio Pascucci, and Michela Taufer.
The National Science Data Fabric
Conference: PRISMS Center Annual Workshop
Authors: Valerio Pascucci
The National Science Data Fabric
Conference: Data-Joint Annual Workshop
Authors: Valerio Pascucci