2011/06/07 Duan 2011/09/07 Duan update final testing resutls of TCP server
We currenlty use 10Gbit Ethernet to output data from ROACH to computer,
which is hard and expensive to scale to 16 Units. First a few things on
software todo list is to switch our 10Gbit ethernet to normal 1Gbit
We had tried to use a traditinal way to output from 1Gbit internet
around 1 year ago. At that time, we directly grab data from Computer to
FPGA throught PPC. This is realative slow, we only get a few Mbps with
large amount of overlapping. This time, Tom and I are trying to write
code running on PPC to work as network server.
The following design is the first step on FPGA side, which can simulate
exact same output package as in real channelizing applicaiton with
programmable buffer we want to play. it is stand alone design, so we
don't need ADC/DAC boards/synthesizer/1pps etc to run.
• For DAQ of 16 readout units, we want to switch from
10GE to 1 Gbit Ethernet(normal internet cable, we can use commercial
multiple port switches).
• To study how we can handle data in high data rate with 16 readout unit.
Method and Results
1Gbit Ethernet FPGA test design:
• In this design, FPGA is running at 100Mhz with its
own clock (no ADC board/synthesizer needed). The output is stored on
the BRAM(which is inside FPGA fabric). We will try to grab data from
that BRAM and send out via 1Gbit Ethernet. Two BRAM used each with size
2^12 (this can change to large value if needed) positions and each
positions store 32 bits data. Since our channelizing design will be
able to handle 192 carriers and cover 491.52 MHz bandwidth. Each
resonators need total 64 bits to store (32bit for real part, 32 bits
for imag part). We will store the 32bit into different BRAM at same
For simulation purpose, the data that play inside the FPGA
is programmed by matlab: “generate_FIR_test_data_1Gethernet.m”.
Currently it is linear straight line, so we can easily check if there
is any data shortage/overlap/missing.
The designed output rate for
each ROACH unit is 256*64bit*100Hz = 1.64 Mbps which corresponding to
100Hz for each resonator. The design is able to adjust the output speed
from 10000Hz to 1 Hz. (so it is possible to use just one ROACH to test
the 16 Readout unit configuration)
Includes: FPGA bit file; FPGA design file; Python code corresponding to
it; matlab code to plot data; matlab code to generate test buffer.
delta(t): dt_20110803_packtest.pdf delta(header counter): dh_20110803_packtest.pdf
Test Results (7th Sep, 2011)
We run 10 boards at same time to take 100Hz data through tcp server and
client. TCP server on Power PC (on ROACH) run continuely for 24 hours.
We open and close clinet on Matlab (Python codes) every 1 hours, and
attached here is a typical response of the time stampe difference (time
stamp difference should equals to 0.01): tcp_1hours_9boards_data.pdf