Designing the SKA's precursor with digital simulation

The challenge of developing instruments to cope with the flood of data from the next generation of radio telescopes is pushing the boundaries of technology. Simulation can help.

When it is complete, the MeerKAT array in South Africa’s Karoo region will include 64 dish antennas, making it the largest and most sensitive radio telescope in the Southern Hemisphere. As a forerunner to the Square Kilometre Array (SKA), slated for completion in 2024, MeerKAT will be used by scientists worldwide on a range of research projects, including observing neutral hydrogen gas and molecular hydrogen levels in the early universe; the physics of enigmatic neutron stars; and the nature and behaviour of pulsars, dark matter and the cosmic web.

Figure 1. Artist’s impression of the MeerKAT array in the Karoo region of South Africa.

Each of these research initiatives depends on the ability to process the massive amounts of data collected by the antennas 24 hours per day. To meet this need, the Digital Back-End (DBE) team at SKA South Africa is developing a digital signal processing (DSP) system capable of handling 5 terabits of data per second. This system is designed with a Simulink library that enables our team and fellow researchers to rapidly design and deploy radioastronomy instrumentation on modular, re-usable hardware using model-based design.

Instrument design challenges

Early radioastronomy relied on large, single-dish antennas up to 300 metres in diameter that focused on relatively small areas of the sky. Modern antenna arrays, which are constructed using multiple small dishes, offer a wider field of view and greater flexibility. Moving the dishes further apart increases the telescope’s resolution and enables it to detect smaller objects. Moving the dishes closer together decreases the resolution but expands the field of view.

The challenge is to correlate data received from all the antennas in the array. To determine the position of an object in the sky, the DSP system must calculate the phase difference between radio signals arriving at each pair of antennas in the array. Because data from all the antennas must be cross-correlated (that is, each antenna’s signal must be correlated with every other antenna’s signal), the processing task is an N² problem: when we double the number of antennas, we have to quadruple the processing power of the DSP.

Compounding the challenge is the continuous nature of the data stream. Optical astronomy data can be collected at night, stored and then processed the following day. Radioastronomy data, by contrast, is collected 24 hours per day and must be processed as it is received.

Moving to re-usable libraries and hardware components

Specialised hardware based on ASICs or FPGAs is needed to process the raw data received from the antenna array in real time. In the past, developing this hardware required a team of experienced hardware engineers and years of effort. Today, scientists with little or no hardware design experience can develop their own instrumentation in months using Simulink, a library of drag-and-drop components, and standardised reconfigurable open architecture computing hardware (ROACH) boards.

The Simulink library and ROACH boards form the basis of an open-source design environment for radioastronomy instrumentation maintained by the Collaboration for Astronomy Signal Processing and Electronics Research (CASPER). SKA South Africa has been a key contributor to the CASPER library during the development of the DSP systems for MeerKAT.

The CASPER library consists of components commonly used in radioastronomy, including mixers, oscillators, downconverters, filters, matrix transforms, accumulators, adders and wideband fast Fourier transform (FFT) blocks (Figure 2).

Figure 2. A 4-input wideband FFT Simulink library block.

The blocks are parameterised and configured automatically by a MATLAB script based on settings specified by the user (Figure 3). For example, to create a five-input parallel adder block, the script automatically inserts the required number of basic adders and delays, connects them and packages them as a self-contained block for use in a larger design.

Figure 3. CASPER Library adder tree configuration interface. Users can specify the number of inputs, latency and other implementation settings.

Scientists assemble signal processing systems from CASPER library blocks and simulate their design in Simulink, using recorded radio signal data as input. After verifying their design via simulation, they use HDL Coder to generate HDL or Verilog code from the Simulink model for deployment on the ROACH board’s Virtex FPGA. In addition to the FPGA, the ROACH board includes analog-to-digital converters (ADCs), a 10-gigabit ethernet interface and other hardware components frequently required in radioastronomy instruments (Figure 4).

Figure 4. A ROACH-II board.

Initially developed with blocks from the Xilinx System Generator for DSP blockset, the CASPER library made it easy for scientists to generate HDL code for their designs and target Xilinx FPGAs. At SKA South Africa, a project is underway to migrate the library to native Simulink blocks, which will enable the use of HDL Coder to generate the HDL code and partition designs so that functional blocks can be targeted not only to FPGAs but also to ASICs or ARM processors. HDL Coder also opens the possibility of expanding CASPER beyond radioastronomy and into other disciplines that require similar real-time signal processing capabilities.

From KAT-7 to MeerKAT

The seven-dish KAT-7 array is an engineering prototype for the MeerKAT array. KAT-7 is the world’s first radio telescope array with fibreglass dishes. It is located in the Karoo region of South Africa, a sparsely populated area with low levels of radio interference from human activity, where MeerKAT and parts of the Square Kilometre Array will also be built (Figure 5).

Figure 5. Aerial view of the KAT-7 array.

We developed the KAT-7 DSP system in Simulink using the CASPER library and deployed it to 16 ROACH boards. While its primary purpose was to serve as a proof-of-concept for MeerKAT, KAT-7 is a valuable telescope in its own right and has already produced images of Centaurus A, a galaxy 14 million light years away.

We are currently using Simulink and the CASPER library to design the signal processing systems for the MeerKAT array, which is slated for completion in 2016. With 64 dishes, MeerKAT has almost 10 times as many antennas as KAT-7. This means that it will require roughly 100 times the signal processing to handle the data, which will be coming in at more than 5 terabits per second.

A key advantage of using Simulink and Model-Based Design for this effort is that designs will be easily re-targeted to the next generation of ROACH boards when they become available. This re-targeting enables us to take advantage of the more powerful FPGAs that we expect to be available in the years to come. We anticipate using about 200 to 300 next-generation ROACH boards for MeerKAT.

The SKA and beyond

The Square Kilometre Array will have between 3000 and 4000 antennas, each about 15 metres in diameter. Combined, the dishes will have a square kilometre of surface area, making SKA the largest and most powerful radioastronomy telescope in the world - about 100 times more powerful than its closest competitor. SKA dishes will produce data at a rate equivalent to roughly 10 times the world’s current internet traffic, and the system will be so sensitive that it will be able to detect signals equivalent in strength to airport radar, on a planet 50 light-years away. Although construction will not begin until 2017, early signal processing design work has already begun and the team has submitted a proposal to design the SKA’s central processing system using Simulink and the CASPER library.

Our work with Simulink and the CASPER library is not only accelerating the development of radioastronomy instruments for MeerKAT and SKA; it is also helping researchers across the globe. Prior to CASPER, it was commonplace for the development of new instruments to take five years or more; with CASPER we have seen groups develop eight instruments in less than two years.

A principal benefit of using Simulink and CASPER is that beginners can become productive designers very quickly. We run workshops on the use of Simulink and CASPER. Attendees are typically hardware design novices, but after a week, they know enough to design their own instruments by dragging and dropping blocks in Simulink and deploying them to ROACH hardware with a single click. In fact, each year some of the trainees develop a new, working instrument during the week-long training course. That kind of productivity is driving interest and enthusiasm in our field, and it is accelerating advances in radioastronomy not only in South Africa, but worldwide.