New chip technology enables real-time insights from scientific data

By shrinking and processing massive data streams right at the detector, the new technology gives researchers immediate feedback and helps them make discoveries faster — without overwhelming storage or computing resources.
Every second, scientific experiments produce a flood of data — so much that transmitting and analyzing it can slow down even the most advanced research. To help scientists better manage this data deluge, researchers at the U.S. Department of Energy’s (DOE) Argonne National Laboratory have developed a new computer chip that rapidly compresses and processes the huge amounts of data generated by advanced X-ray detectors, like those at the Advanced Photon Source (APS), a DOE Office of Science user facility at Argonne. By compressing data right at the source, like shrinking a movie or song to make it easier to send, this technology makes experiments faster, more efficient and more insightful than ever.

When X-rays or electrons hit a sample, detectors capture the resulting signals — much like a digital camera captures light to produce photos. These signals are converted into electrical pulses and then digitized into numbers that computers can process. But with modern detectors, the amount of data generated is enormous. Every frame, even those with little useful information, is sent out for storage and analysis. This can overwhelm computer systems and slow down research, making it harder for scientists to find what matters most.