Inaccurate flagging of LBA data

Low band (LBA) data recorded and (pre-)processed between 19-10-2012 and 25-01-2016 were not properly flagged by the Radio Observatory pipelines because of an error in the definition of the flagging strategy used for LBA data (LBAdefault) by AOflagger via NDPPP. The level of accuracy of the flagging step differs in different data, depending on both pointing and processing specifications such as, e.g., the target elevation and frequency/time averaging factors. A solution to this problem is described below. The error has been corrected at the parset level so all data recorded and processed after 25-01-2016 are not affected by this problem.

An error occurred in the code (specifically a “write flags to file” action being incorrectly placed) of the LBA flagging strategy which resulted in a crash of the AO-flagger and a non-optimal RFI detection / flagging results.
This implies that some RFI could be left in the visibilities of your observations, so we ask that you inspect your data carefully during the data reduction.

NOTE: The visibilities were properly recorded and can be fixed in various ways, according to the degree of processing applied by ASTRON in the pre-processing phase before the visibilities were sent to the LTA.

Below we describe the steps to perform to fix the issue, depending on the stage of pre-processing applied to your visibilities.
  1. Flagged data that did not undergo any frequency compression: to solve the issue the data need to be flagged again. A new version of the LBAdefault strategy can be used, but the developer’s advise is to use the standard AOFlagger strategy (see below).
  2.  Flagged data that have been averaged in frequency: in order to mitigate the issue the data need to be flagged again using a new version of the LBAdefault strategy. The developer’s advise is to use the HBAdefault strategy, instead of the LBAdefault strategy. The effectiveness of this step will depend on the level of frequency and time averaging.
USAGE

To select the flagging strategies, users should use the following syntax in the NDPPP parset:
aoflagger.type=aoflagger
aoflagger.strategy=           # to select the default AOFlagger strategy
or
aoflagger.type=aoflagger
aoflagger.strategy=LBAdefault # to select the LBA (HBA) strategies

To inform future users of this error, a pop-up warning will be implemented in the LTA when users retrieve data observed during the aforementioned period.

In case you experience issues when applying these instructions or you need help in any of these steps, please do not hesitate to contact SDC Operations through the ASTRON helpdesk which is hosted at https://support.astron.nl/rohelpdesk
Design: Kuenst.    Development: Dripl.    © 2020 ASTRON