Do you understand Bitstreams generated for your FPGA design?
This post would be interesting to those who already have experience with FPGAs.
If you have ever designed a hardware using hdl programming language like Vhdl or Verilog on tools like Vivado or ModelSim then you would be familier with the FPGA design flow. Be it a simple logical AND gate design or a complex computer vision applicaiton, the design flow still remains the same. The design flow involves pre-synthesis simulation, synthesis, post-synthesis simulation, optimize-place-routing, and finally bitstream genreation. But, how many of us can understand the bitstreams generated for our design? not many and most of the designers ignore it and don’t care how it looks. Indeed, It is intentionally made to look complex for security purpose, a successful hack will open the door to modify the harware design on the FPGA fabric. In this post I will explain how you can understand the bitstream structure. However, the immediate question would be why is it important to understand a bitstream structure. Well, you will get the answer at the end of this post, you will know what all you can do by knowing the bitstream structure.
The bitstream file is a binary file that contains the configuration information of our design, it follows a Header-Payload format, 32bits each. A bitstream generation is the last stage in the FPGA design process after which it is ready to be programmed on FPGA. You can generate bitstreams in Vivado tool via simple mouse clicks or sometimes even using Tcl script.Bitstreams are complex in structure and almost impossible to be decoded by a layman, However, it can be converted into .rbt or .hex file for better readability. Vivado has an option to generate bitstreams in .rbt format but an online tool ‘Tomeko’ converts into a .hex file, for better readability. The size of bitstreams vary from kilo-bytes to mega-bytes and even more sometimes depending on the complexity of the design and area utilized on the FPGA floorplan.
Figure 1 is the the bitstream structure from Xilinx UG570 and figure 2 is the real bitstream converted into .hex format. On comparing both, it can be seen that the figure 1 starts with “FFFFFFFF” but figure 2 with some random numbers. All the numbers until the first encounter of “FFFFFFFF” at line 8 is a header. A header contains a file name, date of creating and timing information, but since it is not downloaded to FPGA, it can be ignored. From this point both figures go in parallel, it has a set of dummy words “FFFFFFFF” to align the parser. It is followed by buswidth auto detection header “000000BB” and “11220044” payload, where 0x11 is 8bit, 0x22 is 16bit and 0x44 is 32bit bus width. The second byte is 0x00 instead of 0x33 because 0x33 equates to 27 bits which is not a standard buswidth. Finally a synchronization key “AA995566” which indicate the beginning of the bitstream. None of the packet will be processed until the synchronization key is discovered.
Figure 2 shows the real bitstream example, From line 1 until the middle of line 8 is all header information, which will not be downloaded to FPGA and it is not mentioned in the figure 1 because file name and time of creating keeps changing for every bitstream. At line 12 the buswidth auto detector header can be seen and its immediate next word is its payload. On line 13 the synchronization key is found and from this point the packet processing starts. Once the synchronization key is detected it is not necessary that both the figure go in parallel.
Another important thing to know in bitstream structure is that in the header-payload pair, the size of the payload is not always 32bit. It varies according to the word count in header format. Figure 3 shows the header format and figure 4 shows the header-payload pair in bitstreams. Most of the time Type1 header is used however Type2 header is just the extention to Type1. The word count in the Type1 indicates the total payload word count. A count 0 means 0 word and a count 1 is 1 word and so on. Figure 4 shows the examples of header-payload example.
Look at the highlighted sections in both the images, As we can see that on the left side 0x20000000 refers to a dummy header, the count bits are 00 hence no payload, that is the reason we have a series of 0x20000000 headers but no payload. However, on the right hand side the header 0x30008001 has 1 word count and hence there is a 32bit payload 0x00000000 immediately after it.
Figure 5 shows the FDRI register, writing to this will write the data to the Frame Address Register(FAR).
The command register(CMD) is used to instruct the configuration control logic to strobe global signals and perform other configuration functions. The command present in the CMD register is executed each time the FAR register is loaded with a new value. It can be seen in the figure 6, The highlighted word is FAR and next to it is FAR payload. After the payload we can see the CMD register.
All the above explaination is just an addons to your knowledge. You can skip it as it is practically not benificial. But what we have to focus on is is Frame Address Register (FAR).
What we are interested in is to track the FARs in the bitstream, a bitstream can have many FARs, In the bitstreams that I generated there were minimum 7 and maximum FAR values. Now the bigger challenge is to identify the correct FAR amont the 7,9 or 11 FARs. Well, One key point to know is the FARs in bitstreams follow a recurssive pattern, meaning out of 7,9 or 11 payloads most of them are recursive. Only one payload is unique as shown in the figure 7. Remember, the bitstreams do not have FARs in hexadecimal format, these are extracted from 4 different bitstream using a python script and then another python script to convert these binary extracts in hexadecimal form.
Having a look at the FARs in figure 7, It can be related it with above paragraph which talks about the recursiveness of the FARs. In figure 3, the first set of FARs are associated with the ‘X0Y4_left_partial’ bitstream, which is the name of my bitstream file. It can be seen that the 0x10040d, 0x10060d, 0x7FC0000 are all repeated hence they cannot be FAR of the configuration frame, only 0x100000 and 0x1100000 are unique. In order to decide the correct FAR among these two, one has to look at the FAR address in the bitstreams of the e immediate next clock region, because FARs are incremented either in ascending order. From the figure 7 it can be seen that there is no FAR closely related to 0x1100000, however, 0x103400 is close to 0x100000. Similary, in the 3rd and 4th FAR set, 0x140000 and 0x143400 are quite close to each others and they are in synchronous with the first 2 FAR sets. if you notice there is a pattern being followed among the pairs, 0x100000 and 0x103400, 0x140000 and 0x143400. This is how we determine the FARs.
Why so much emphasis on the understanding bitstreams and FARs in particular would be your next question. The answer is, by understanding the bitstream structure and knowing the FAR addresses it is possible to trace the exact location of our design on the FPGA, This will also give control to the designer to place and route his design on the required location of the FPGA. This can also reduce the time for routing and placement process, reduce the latency.
In the upcoming two posts I will explain, the internal structure of the FPGA fabric, its composition, its resources etc.. Also, in a separate post I will explain just by knowing the FAR addresses how one can have control on secure bitstream configuration process to certain regions only.
