Device developed in Brazil is key to LHC upgrade

OpenIPMC ensures reliable data collection from the Large Hadron Collider’s detectors.

By José Tadeu Arantes  |  Agência FAPESP – A device designed and manufactured in Brazil will be the first fully open management controller for the electronics of the detectors at the Large Hadron Collider (LHC), the world’s largest particle accelerator. OpenIPMC is the result of a partnership between the São Paulo Research and Analysis Center (SPRACE) and Lynx Tecnologia Eletrônica. It will be used in 1,000 boards in the Compact Muon Solenoid (CMS), one of the collider’s two main detectors, as well as in ATLAS, another multipurpose detector. The LHC is an underground ring with a circumference of 27 kilometers, where proton beams collide at extremely high energies. The goal is to investigate the fundamental structure of matter and the universe. It was built by the European Organization for Nuclear Research (CERN, in its official French acronym) on the border between Switzerland and France.

SPRACE is a research center that works in the areas of basic science, high-performance computing, and digital innovation. Created in 2003 with the support of FAPESP, the center enables researchers of high-energy physics in the state of São Paulo to participate in the CMS collaboration.

An IPMC (Intelligent Platform Management Controller) is an intelligent management device that supervises electronic boards in high-performance systems. It monitors temperature, voltage, electrical current intensity, and error messages. It also controls operating states (turn on, turn off, restart) and integrates different components. It is essential for ensuring that thousands of boards function stably in critical environments. At the LHC, the IPMC is a key component for coordinating collision sensing units with processing units, which translate data into useful information for researchers.

Until recently, the problem was that IPMCs were being provided by large private companies and were subject to non-disclosure agreements (NDAs). Every adjustment, large or small, required the intervention of the supplier company, creating time and cost bottlenecks and exasperating researchers. The solution was to develop an open IPMC that allows researchers to freely configure and adapt the system. This triggered the creation of OpenIPMC.

“OpenIPMC broke down an important barrier: hardware, firmware, and software are open. It’ll be a key component of a system that processes 100 terabits of experimental data per second online, minimizes failures, and ensures continuous operation, allowing LHC collisions to be observed with the experiment always at peak performance,” says Sérgio Novaes, a physicist, full professor at São Paulo State University (UNESP), and coordinator of SPRACE.

The device is an essential part of the ATCA (Advanced Telecommunications Computing Architecture), responsible for the tracker functions that record the trajectory of particles immediately after collisions in the CMS – whose upgrade will cost USD 120 million. Initially created for telecommunications, ATCA is now an international standard for electronic architecture used in areas requiring high reliability and high bandwidth, such as particle physics, quantum computing, and aerospace engineering. ATCA’s functionality depends critically on the performance of the IPMC.

CMS tracker: the billion-dollar upgrade of the LHC will enable the detection of even rarer phenomena, but will also increase the challenge of analyzing up to 200 simultaneous particle collisions in each event (photo: CERN)

“The CMS tracker is an extremely sophisticated detector that determines the trajectory of particles produced in collisions, providing crucial information about charge, momentum, and decay vertices. The system is divided into two parts: the front end – sensor modules installed in the experimental cavern, next to the detector – and the back end – the electronics for power supply, reading, synchronization, preprocessing, triggering [the system responsible for deciding, in real time, which collisions should be recorded and which can be discarded], and control. The back end uses about 750 boards in the tracker and 250 boards in other subdetectors, totaling 1,000 boards that need to be managed. That’s where the IPMC comes in. It ensures the reliable functioning of this system. In short, it’s the ‘manager’ that ensures the reliability of the detector,” Novaes explains.

Founded by professors from the Engineering School of the University of São Paulo (POLI-USP), Lynx Tecnologia Eletrônica has already delivered the first batch of 100 boards to CERN for testing. The company is now preparing to manufacture 1,000 units for the CMS under the supervision of Professor Luiz Antonio Coelho from POLI-USP. “Manufacturing the 1,100 OpenIPMC units for CERN represents not only a technical milestone, but also demonstrates the ability of Brazilian industry to meet the most rigorous international standards,” Coelho celebrates.

First batch of OpenIPMC boards: open version of IPMC, developed in Brazil by SPRACE in partnership with Lynx. Unlike proprietary solutions, it allows researchers to freely access, modify, and adapt hardware, firmware, and software, accelerating scientific and technological innovation (photo: CERN)

Brazil is the birthplace of the OpenIPMC project, which began in 2019 under the leadership of Italian physicist Luigi Calligaris. Calligaris received support from FAPESP in the form of research grants focused on the development of instrumentation in the Young Investigators program.

Although OpenIPMC was designed for the CMS, its success caught the attention of the team at ATLAS, the other large multipurpose detector at the LHC. “It’s rare for the same component to be shared between the two experiments, because they have independent architectures. The impact of what we did was so great that ATLAS also decided to adopt our solution,” notes Novaes.

In addition to the LHC, OpenIPMC is already being implemented in the QSolid quantum computing ecosystem as part of Germany’s national strategy and will be installed in TRISTAN, an extension of the KATRIN experiment dedicated to searching for sterile neutrinos. The solution is also attracting interest from sectors such as defense, aerospace, and nuclear fusion.

The CMS operates in a highly complex environment. Each bunch (particle packet) contains approximately 10¹⁵ protons. Two bunches traveling in opposite directions collide every 25 nanoseconds. In the High Luminosity LHC regime, the billion-dollar upgrade currently underway at the LHC, there will be up to 200 simultaneous collision vertices per event (pileup). The tracker must identify the origin of each particle amid this tangle. “It’s a huge scientific and technological challenge, and OpenIPMC plays a crucial role in ensuring that the system of 1,000 plates operates flawlessly,” emphasizes Calligaris.

For SPRACE researchers, OpenIPMC elevates Brazilian technology to new heights. “We’re helping transform Brazil from a consumer to a supplier of critical technology at the frontier of scientific knowledge,” Novaes summarizes.