Subjects: Computer Science >> Other Disciplines of Computer Science submitted time 2022-06-28
Abstract：
Data processing of the Square Kilometre Array (SKA) is carried out in pipeline mode, and the execution efficiency of pipeline is an important factor to be considered in SKA data processing. Continuum imaging is one of the main observation modes of SKA and is a prerequisite for many other scientific works. In this paper, we take the imaging pipeline of SKA low-frequency precursor Murchison Widefield Array (MWA), as an example and optimize the parallel processing pipeline on the China SKA Regional Centre prototype (CSRC-P). Previous optimization schemes have focused on a few performance hotspots and lacked systematic optimization of the overall pipeline, resulting in a relatively poor overall speedup ratio. In this paper, we propose a global optimization scheme that combines C++ multi-threading, Python multi-processing, and Shell multi-tasking parallelism for pipelines using multiple programming languages and image datasets that can be processed independently, and verify the accuracy of the optimization results. Experiments show taht the optimized pipeline achieves an overall speedup of 2.7 and 2.4 times on the x86 and ARM nodes of CSRC-P, respectively, and the ARM compute nodes shows good adaptability to SKA applications. The optimization strategies and methods in this paper are also applicable to other SKA applications and will be useful for the scientific operation and future operation of the SKA precursor telescope.
Peer Review Status:
Awaiting Review
Subjects: Computer Science >> Other Disciplines of Computer Science submitted time 2022-06-28
Abstract：
The connection between astronomy and high performance computing is becoming stronger with the devel- opment of cutting-edge observing facilities such as the Square Kilometre Array (SKA) and the proposed innovative platform for big data and high performance computing. Astronomical computation is character- ized by huge data volume and massive parallelism, especially for pulsar search which is one of the leading scientific directions of the SKA. In this paper, we present an approach to accelerate the pulsar search pipeline based on OpenMP and multiprocessing techniques, propose a method to solve the load imbalance problem,and successfully has the pipeline installed on both x86 and ARM compute nodes on China SKA regional center prototype (CSRC-P). The performance evaluation from the tests on the Murchison Widefield Array (MWA) VCS observations shows that our optimization method works well on both x86 and ARM nodes, improving the relative speedup by a factor of 10.4–12.2 and 24.5–27.6, respectively, compared to the original single- thread approach. The ARM platform was found to be 1.1–1.3 times faster than the x86 platform in the tested cases, showing its great potential for SKA data processing. Recently, this optimized pulsar search pipeline deployed on CSRC-P will be especially used for low-frequency pulsar survey of the Southern-sky MWA Rapid Two-metre (SMART) program,for various scientific goals including pulsar timing arrays for gravitational wave detections.
Peer Review Status:Awaiting Review
Hits
Hits
Downloads
Comment