Summary of information related to WP11

he purpose of this Work Package is to develop the software modules that are required for the data processing and analysis tasks of the EISCAT_3D radar system. The development of new data algorithms and software is needed because of the complexity of a distributed phased-array incoherent radar system compared to earlier systems.

This Work Package involves the following activities:

  • Parallelisation of the basic inverse problem-solving tools for signal processing and data analysis for use in the EISCAT_3D radar system.
  • Production of software for the signal processing and beam-forming systems, which is done with tight connections to WP7 (Digital Signal Processing).
  • Development of multi-purpose codes to allow the EISCAT_3D system to be used in an optimised fashion.
  • Development of data analysis software to to be applicable to multi-beam measurements and imaging applications, and to allow for some new experimental methods that the new system will allow.
  • Integration of EISCAT_3D hardware and software.

The software development will interact with the system control work (WP12), and also needs a strong link with the science requirements (WP3) and the data handling and distribution activities (WP13), since the software performance will set constraints on the time and spatial resolutions and measurement accuracies which can be achieved, as well as critically determining the size and content of the EISCAT_3D low-level data.
For several of the Tasks in this Work Package access to a parallel computer is required. This device needs to be located physically near the Digital Signal Processing system developed in WP7 since the data rate speeds are too high to use a remotely located parallel system. For this reason, the project plan includes funding to buy or lease a parallel computing device for the duration of the work.

Deliverables

Deliverable 11.1: Software package for EISCAT_3D data analysis

December 6, 2013
Work Package 11 the software modules that are required for the data
processing and analysis tasks of the EISCAT_3D radar system are developed.

Deliverable 11.1 consists of two software products, the Lag Profile Inversion (LPI) software and R Linear Inverse Problem Solver (RLIPS) software. Both products are packages in the R software programming language. The documentation of the software is also part of the Deliverable.

Both software included in this Deliverable are licensed under freeBSD license.

Deliverable 11.2: Comprehensive report, describing and justifying the choice of the selected hardware and software

December 6, 2013
The purpose of Work Package 11 is to develop the software modules that are required for the data processing and analysis tasks of the EISCAT_3D radar system. The development of new data algorithms and software is needed because of the complexity of a distributed phased-array incoherent radar system compared to earlier systems.

The Work Package consisted of five tasks

Task 11.1
Parallelisation of the basic inverse problem-solving tools for signal processing and data analysis for use in the EISCAT_3D radar system.
Task 11.2
Production of software for the signal processing and beam-forming systems, which is done with tight connections to WP7 (Digital Signal Processing).
Task11.3
Development of multi-purpose codes to allow the EISCAT_3D system to be used in an optimised fashion.
Task 11.4
Development of data analysis software to to be applicable to multibeam measurements and imaging applications, and to allow for some new experimental methods that the new system will allow.
Task 11.5
Integration of EISCAT_3D hardware and software
In the report we give a summary of the work done. The actual theory behind the software and the documentation of their implementation is given as Appendices, a list of which is given at the end of this document.

Milestones

Milestone 11.1: Initial versions of signal processing and beam-forming

June 19, 2012
Task 11.2 of the EISCAT_3D Preparatory Phase project concerns the development of signal processing and beam-forming software. It consists of the implementation of the software architecture defined in the handbook of measurement principles (produced in Work Package 6), comprising time-delay filtering, and summation calculations for beam-forming.

This Milestone was reached when the initial version of this software was ready, which was in September 2012.

Milestone 11.2: Completion of FLIPS development

April 9, 2013
Task 11.1 of the EISCAT_3D Preparatory Phase deals with parallelisation of the basic inverse problem-solving tools for signal processing and data analysis, to be used with the EISCAT_3D radar system.

Inversion techniques will be at the core of EISCAT_3D computations in signal processing, beam-forming, interferometry and data analysis. The Fortran Linear Inverse Problem Solver (FLIPS) is an open-source software tool, developed by the Finnish Centre of Excellence in Inverse Problems, funded by the Academy of Finland. For the use in EISCAT_3D, it has to be adapted to run on an HPC cluster, making optimum use of multiple processors in a single CPU. It also requires an efficient interface to a standard programming language.

Now the R package RLIPS is ready and usable, satisfying the requirements. The comprehensive documentation will be included in the final report of Work Package 11 (Deliverable 11.2).

This Milestone was reached in October 2012.

Milestone 11.3: Completion of multi-purpose codes and analysis development

December 6, 2013
This Milestone was reached in September 2013.

Milestone 11.4: A consistent set of optimised hardware and software identified

December 6, 2013
A consistent set of optimised hardware and software was identified in June 2014. The details are described in Deliverable 11.1 and Deliverable 11.2, and it constitutes Milestone 11.4.

Attachments

Documents related to WP11 can be found in our externa library.

Related documents