Anomaly Detection

Given a dataset with radar data we wish to detect anomalous structures in the ionosphere. The ionosphere is a region of the upper atmosphere where there are large concentrations of free ions and electrons. While the ions give the ionosphere its name, it is the free electrons that affect the radio waves and radio communications. Anomalous structures in the ionosphere could lead to deterioration of radio signals, which could effect long distance communication systems.

This radar data was collected by a system in Goose Bay, Labrador.  This system consists of a phased array of 16 high-frequency antennas with a total transmitted power on the order of 6.4 kilowatts. The targets were free electrons in the ionosphere. "Good" radar returns are those showing evidence of some type of structure in the ionosphere. "Bad" returns are those that do not; their signals pass through the ionosphere. We are going to work under the assumption that we don't have this information ahead of time for new data.

Received signals were processed using an auto-correlation function whose arguments are the time of a pulse and the pulse number. There were 17 pulse numbers for the Goose Bay system. Instances in this database are by 2 attributes per pulse number, corresponding to the complex values returned by the function resulting from the complex electromagnetic signal. Below is an overview of that data;

In order to detect anomalous data we can simply connect the data to the Anomaly Detection block;

This gives us an additional field, scoring the likelihood of each record being anomalous (higher is more anomalous), these values will usually have a distribution that looks like that below;

We have 34 fields in the data, much of which is binary (just 0 and 1) which makes plotting the data difficult, so we use the Reduce Dimensions block to make it easier to visualise;

This shows that there is an area of higher density, and that points outside this area are more anomalous,

And the outliers seem quite likely to be 'bad' radar returns. The anomaly detection block can be used for a variety of different data, and can be used before visualising data in order to detect outliers/data entry errors.

0 Votes