Brazilian software can be used to send data from telescope carrying the world’s largest digital camera

Kytos software developed by SPRACE, a research center supported by FAPESP, is being tested at the production network of the consortium responsible for transmitting data from the Large Synoptic Survey Telescope.

Agência FAPESP – Kytos software developed in Brazil has begun to be used in the production network of AmLight, the consortium that will be responsible for the international transmission of data from the Large Synoptic Survey Telescope (LSST), which is under construction in Chile and will photograph the entire visible sky.

Kytos is an open-source platform for the orchestration of software-defined networks (SDN) developed by São Paulo Research and Analysis Center (SPRACE), which is supported by FAPESP.

According to SPRACE, production of the software involved close collaboration with engineers and developers in three other organizations: Rede ANSP, a FAPESP program; the National Higher Education and Research Network (RNP), Brazil’s academic internet; and AmLight.

The software has been tested and validated on several platforms in Brazil and abroad. During demonstrations performed at the International Conference on High Performance Computing, Networking, Storage and Analysis (SC 2018), held last November in Dallas, Texas (USA), Kytos orchestrated an intercontinental ring encompassing AmLight’s Atlantic and Pacific links and sustained a data transfer bandwidth of 350 Gbps.

After the demo, the Kytos team was invited to implement and test the software on a test platform run by the Energy Sciences Network (ESNet), a high-speed computer network serving US Department of Energy (DoE) scientists and collaborators in more than 40 institutions worldwide.

AmLight is also using Kytos in experimentation environments, and the aim is now to prepare it to meet the demands of the LSST, which will be the telescope with the world’s largest digital camera.

Scheduled to see first light in 2022, the LSST is an international project that was proposed in 2001. Construction began officially in 2014 on a site over 2,600 m above sea level atop Cerro Pachón in northern Chile.

According to SPRACE, the LSST’s design objective of photographing the entire observable sky entails major challenges in data transmission, storage and analysis. Some 20 terabytes will be generated every night for transmission to different analysis centers around the world.

One of these is the National Center for Supercomputing Applications (NCSA), located in Urbana, Illinois (USA), where all data will be sent in real time. It has been operating for ten years and is expected to process hundreds of petabytes (PB), generating a database with more than 15 PB of information on the Universe.

The AmLight consortium, which involves ANSP, RNP and Florida International University (FIU), will be responsible for transmitting data from Chile to the United States. To transfer large amounts of data at high speed between the two countries, AmLight will use a complex network with several 100 Gbps links. The consortium’s infrastructure will also include SDNs to reduce operating costs and support complex dynamic services.

According to the team of developers who produced Kytos, the platform is designed for use in large-scale scientific experiments such as the Compact Muon Solenoid (CMS), a particle capturing detector built on the Large Hadron Collider (LHC) at CERN, and the LSST.

“It’s a great pleasure to see an initiative developed by SPRACE, which was initially designed for high-energy physics, crossing this frontier into other areas of e-science,” said Sérgio Novaes, principal investigator at SPRACE.

According to Novaes, the inclusion of Kytos in AmLight’s production network demonstrates the platform’s significant potential and is a major step toward its inclusion in the LSST project.

“The development of Kytos has ceased to be performed solely by a group of researchers at SPRACE and has become truly cross-border. Because it is an open-source platform, collaborators in several groups are concentrating efforts to deliver a robust and reliable solution. Engagement by AmLight’s engineers has been vital in that it has enabled the project to be used in a production environment, in the links between Brazil and the US. This shows we’re on the right track to achieve the maturity required for a project of this size,” said Beraldo Leal, principal software engineer at Kytos.

SPRACE

SPRACE develops fundamental and applied research in areas such as high-energy physics, high-performance computing and digital innovation. It was established in 2003 with support from FAPESP.

SPRACE enables high-energy physicists from São Paulo State to participate in the CMS collaboration at CERN’s Large Hadron Collider (LHC). As part of its contribution to CMS, it operates BR-SP-SPRACE, a Tier 2 computer cluster of the Worldwide LHC Computing Grid (WLCG) that processes, analyzes and stores part of the data generated by the experiment. Its engineering team is working on the development of scientific instrumentation that will be implemented in the second stage of the CMS tracking detector upgrade.

SPRACE also partners with the private sector to develop digital innovation projects with emphasis on high-performance computing, machine learning, and SDNs. Its team developed Kytos, an open-source SDN platform used in AmLight’s international links. The LSST is funded by the US National Science Foundation (NSF) and DoE, as well as private sources captured by the LSST Corporation.

More information: sprace.org.br.