Data analysis at the speed of light source

At the upgraded Advanced Photon Source (APS), a powerful X-ray light source in the U.S. Department of Energy’s (DOE) Argonne National Laboratory, new analysis capabilities are changing how experiments unfold. Instead of waiting until an experiment ends, researchers can now use near real-time feedback from the X-ray beamlines to guide their next steps.

APS experiments are now tightly integrated with supercomputers at the Argonne Leadership Computing Facility (ALCF). Building on years of collaboration between APS and ALCF, Argonne researchers created an automated pipeline that streams experimental data from the beamlines to ALCF systems for analysis as it is being collected. The APS and ALCF are DOE Office of Science user facilities.

Linking light and compute The connection between APS and ALCF is powered by the APS Data Management System and Globus. The APS Data Management System provides a uniform way to connect to data from the approximately 100 unique instruments at the APS. It also keeps track of information about data and experiments at the facility. Globus, a research automation and data management platform developed at Argonne and the University of Chicago, handles the movement of data between the APS and the ALCF’s Polaris supercomputer, automatically running analyses and returning results to the beamline while experiments are still underway.