Computational requirements

Computational Requirements for the Pre-processing pipeline

 

The computational requirements of the imaging mode can be substantial and depend both on observing parameters and image characteristics.

In the following, we present practical estimates for the Processing over Observing time ratio (P/O ratio) separately for the pre-processing and the imaging steps. Note that when considering the computational requirements for the observing proposals, users should account for BOTH of these factors. 

Pre-processing Time

Each of the software elements in the pre-processing pipelines has a varied and complex dependence on both the observation parameters and the cluster performance, and hence a scaling relation is difficult to determine.

To have realistic estimates of pipeline processing times, typical LBA and HBA observations with durations longer than 2 hours and adopting the full Dutch array were selected from the LOFAR pipeline archive and were statistically analyzed. The results are summarized in the following table:

Type 

 Nr CHAN

Nr Demixed Sources

Nr SB  P/O 
LBA 
 64 0 244 0.25

LBA 

 64

2

244 0.51
LBA 
 64 0 80 
0.2 [CEP2]
LBA 
 64 1 80 0.3 [CEP2]
LBA 
 64 2
80 
1.0 [CEP2]
LBA 
 256 2 244  0.72

 

 

 

   

HBA

 64

0

244 0.81 
HBA 
 64 2 244  3.0
HBA   64 0

122

0.9 [CEP2]
HBA 
 64 1 122 1.0 [CEP2]
HBA 
 64 0 366 1.4 [CEP2]

HBA

 64

1

392 2.0
HBA 
 64 0
380
1.5 [CEP2]

HBA

 64

0

480 1.4 [CEP2]
HBA  256 2 244 4.0 

Table 4: Pre-processing performance for >2h observations with different observation parameters and settings for demix for HBA and LBA. Although the case of 3 demixed sources has not been characterized, a large increase of the P/O ratio for both LBA and HBA is expected. Note that for setups with no CEP4 statistics, we reported the P/O values for the old CEP2 cluster: thus these values must be considered upper limits for CEP4.

These guidelines have been implemented in NorthStar, such that pipeline durations are automatically computed for the user.  

Note that:

 - The case of 3 demixed sources is expected to drastically increase in terms of P/O ratio for both LBA and HBA and of claimed computing resources. To safeguard the overall operations of the LOFAR system, the Radio Observatory does not support 3 demixed sources on the CEP4 cluster.

The processing of data with resolution of 256 channels and demixed source(s) is granted based on a solid scientific justification.

 

Current Offline Calibration Status and Performance

 

In the last couple of years the calibration and imaging software has been expanded considerably allowing users to reach noise levels few factors from the thermal noise.  An advanced, direction-independent calibration pipeline (pre-FACTOR) has been developed and documentation is availbale in the LOFAR Imaging Cookbook and here).  

Users can request processing time on CEP3 to perform offline calibration and imaging using the most recent tested version of pre-factor or FACTOR pipelines if they do not have the requisite resources available themselves. If you are interested in this offline option, please make this clear in your proposal by answering the relevant question in the technical section of the proposal "Off-line data processing on RO facilities (CEP3) requirement". Alternatively, proposers may describe how they plan offline processing to achieve the required image quality on their own compute resources.

Based on users experience one CEP3 node a typical observation of 243 sub bands grouped in blocks of 10 sub bands will need a P/O ~ 80 to be fully processed. Consequently a typical 8-hour observation will require 640 hours to be processed on one node, which is within the amount of time of a default CEP3 reservation block. 

The noise level of the images obtained by using the pre-FACTOR calibration pipeline can reach 4 times the thermal noise (calculated using the noise calculator tool). These values are based on a limited set of cases and on a fraction of the total frequency band. We advise the user to take this number into account as an indication of the best possible result achievable with this pipeline. More detailed information could be found here.    

 

 

Installing the LOFAR Software Stack 

 

The Lofar LTA software stack is the collection of software that is needed to run the Lofar imaging pipeline. That includes all needed libraries with a specific version. An overview of the LOFAR Software Stack, together with a discussion of various aspects of the software stack, are discussed at this Wiki page. Currenly a docker image including the latest LOFAR software can be found in the LOFAR Imaging Cookbook.   


Design: Kuenst.    Development: Dripl.    © 2020 ASTRON