Description
This project demonstrates the usage of the FMC Pcam Adapter as an interface from one to four different Pcam cameras and the ZedBoard platform. The Video Stream from each different camera is getting in through the MIPI/FMC connectors and out through the carrier VGA port. For errors and feed-back messages, an UART interface is present.
Additional information on how this demo can be used can be found in appropriate READMEs, linked below.
Inventory
ZedBoard with an external 12V Power Supply
2 Micro-USB cables. One for programming and the other one for the uart interface
Vivado and Vitis installations compatible with the latest release of this demo (2020.1)
See Installing Vivado, Vitis, and Digilent Board Files for installation instructions.
* Serial Terminal application to receive messages printed by the demo
See Installing and Using a Terminal Emulator for more information.
1 VGA cable
From 1 to 4 Pcam-5C cameras
VGA monitor
Download and Usage Instructions
The following releases of this demo can be used with instructions found in the corresponding READMEs in order to run the demo.
Releases are only compatible with the version of the Xilinx tools specified in the release version number. In addition, releases are only compatible with the specified variant of the board. For example, the v2020.1-1 release for ZedBoard can only be used with Vivado 2020.1.
The latest release version for this demo is highlighted in green.
Note: Releases for FPGA demos from before 2020.1 used a different git structure, and used a different release tag naming scheme.
| Board Variant | Release Tag | Release Downloads | Setup Instructions |
|---|---|---|---|
| ZedBoard | FMC-Pcam-Adapter/2023.1-1 | Zedboard-FMC-Pcam-Adapter-hw.xpr.zip Zedboard-FMC-Pcam-Adapter-sw.ide.zip | See Using the Latest Release, below |
| ZedBoard | FMC-Pcam-Adapter/2022.1-1 | Zedboard-FMC-Pcam-Adapter-hw.xpr.zip Zedboard-FMC-Pcam-Adapter-sw.ide.zip | See Using the Latest Release, below |
| ZedBoard | v2019.1-1 | Release ZIP downloads | Github README |
| ZedBoard | v2018.2-1 | Release ZIP downloads | Github README |
Note for Advanced Users: GitHub sources for this demo can be found in the FMC-Pcam-Adapter/master branch of the Zedboard repository. Further documentation on the structure of this repository can be found on this wiki's Digilent FPGA Demo Git Repositories page.
Instructions on the use of the latest release can be found in this dropdown:
- Using the Latest Release
Note: This workflow is common across many Digilent FPGA demos. Screenshots may not match the demo you are working with.
Important: These steps are only to be used with releases for Xilinx tools versions 2020.1 and newer. Older releases may require other flows, as noted in the table of releases.
First, download the '*.xpr.zip' and '*.ide.zip' files from the demo release, linked above. The XPR archive contains the Vivado project used to build the hardware platform for this demo. The project can be opened, modified, and used to update the hardware platform later if so desired, but this is optional. The IDE archive contains a set of projects to be imported into a Vitis workspace.
Note: Unlike with Vivado XPR archives, do NOT extract the Vitis project archive ('*.ide.xip'). Vitis imports sources from the archive file directly.
- Import Vitis Projects from a Release
Select the dropdown corresponding to your operating system, below.
- Windows
Open Vitis through the start menu or desktop shortcut created during the installation process.
- Linux
- Open a terminal and run the following commands. The install path is /opt/Xilinx by default.
source <install_path>/Vitis/2020.1/settings64.shvitis
Note: Regardless of OS, if Vivado is open, Vitis can also be launched through the Tools → Launch Vitis toolbar option.
Upon launching Vitis, a dialog will appear where a workspace must be chosen. The workspace is the directory where all of the projects and files for the application being developed will live. If a folder that does not currently exist is chosen, it will be created. Choose a workspace and click Launch to finish launching Vitis.
With Vitis open, click the Import Project button to import projects from a Vitis project exported zip file, then navigate to and select the IDE zip file you downloaded.
Make sure each project in the archive is checked, then click Finish to import them into your workspace.
- Apply Fix for Linker Script Import Bug
Due to a bug in some versions of Xilinx's Vitis IDE, two more steps are needed for the project to import correctly. If your projects build without errors, ignore this section.
After importing the project, open the .sprj file. In the System Project Settings, select a platform by clicking on the
…icon, as shown in the image. Select the platform presented and click Ok.Right click on the .prj file and select Properties. In the opened window, go into C/C++ Build and select Settings, as shown in the image. After opening the Settings tab, select Linker Script and change the existing path to
../src/lscript.ld. Click Apply and Close.The project is now imported correctly.
- Set up the ZedBoard
- Plug one microUSB programming cable into the ZedBoard's JTAG port and the other microUSB cable into the UART port, the 12V Power Supply into the Barrel Jack next to the JTAG port and plug the FMC Pcam Adapter into the FMC connector on the board.
- Launch a Vitis Application
Make sure your board is set to boot from JTAG before it's powered on. JTAG programming can override other boot modes on some devices, but it's easier to tell when a project is programmed into the board if there isn't already one in there.
First, many applications require that a serial console is connected to the board, so that standard output (from print statements) can be viewed. For this purpose, a serial terminal should be used. Use a serial terminal application to connect to the board's serial port. Unless otherwise stated, Zynq designs use a baud rate of 115200 and Microblaze designs with an AXI UART Lite IP use a baud rate of 9600. Flow control should be set to NONE.
Note: While Vitis has a built in serial terminal included in its Debug view, it sends characters to a board on a line-by-line basis. Some software examples require the use of character-by-character reception of data. Tera Term or PuTTY are recommended if you are not sure what will work.
In the Explorer pane at the left side of the screen, right click on the application or system project that is to be run, and select Run as → 1 Launch on Hardware (Single Application Debug). The FPGA will be programmed with the bitstream, the ELF file created by the software build is loaded into system memory, and the application project will begin to run. You will need to click back over to the Vitis Serial Terminal from the Console tab.
Note: Once the project has been run at least once, you can use the green run button (
) in the toolbar at the top of the screen to program the board instead.
At this point, the demo is now running on your board. Refer to the Description section of this document for more information on what it does.
Additional steps beyond here present how you can use the other archive provided in the release, containing the hardware project, to rebuild the Vivado project, and use a newly exported XSA file to update the platform in Vitis.
In order to modify and switch out the hardware platform for a baremetal demo, you should first open the Vivado project from the release. Extract the previously downloaded '*.xpr.zip' file.
- Open a Block Design Project in Vivado
Launch Vivado
Select the dropdown corresponding to your operating system, below.
- Windows
Open Vivado through the start menu or desktop shortcut created during the installation process.
- Linux
Open a terminal, and change directory (cd) to a folder where log files for your Vivado session can be placed, then run the following commands:
source <install_path>/Vivado/<version>/settings64.shvivado
In Vivado's welcome screen, use the Open Project button to navigate to and open the XPR file contained in the folder the release was extracted into.
The project's block diagram, which contains the design, with all of the existing components and their connections, can be opened by either double-clicking on the “*.bd” file in the sources pane (which also includes other source files, such as constraints), or by clicking the Open Block Design button in the Flow Navigator pane.
Making changes to the design is out of the scope of this particular document. More information on how to use IP Integrator to create or modify a project can be found through Getting Started with Vivado and Vitis for Baremetal Software Projects. The remainder of this document will discuss how to generate a bitstream, export a new hardware platform, and load it into Vitis.
Before the Vivado project can be built, the block design must be validated. This step runs an automatic check of the block design to see if there are any potential issues with it. Click the Validate Design button () in the Diagram pane's toolbar (or press the F6 key).
If the design has issues, a dialog will pop up that lists them. It should be noted that most Warnings can be ignored, as can some Critical Warnings. These issues can also be viewed in the Messages tab of the pane at the bottom of the window.
If there are no issues, a dialog will pop up that will tell you so. Click OK to continue.
Note: Some Zynq boards may produce critical warnings at this stage relating to PCW_UIPARAM_DDR_DQS_TO_CLK_DELAY parameters. These warnings are ignorable and will not affect the functionality of the project. See the Hardware Errata section of your board's reference manual for more information.
- Build a Vivado Project
At this point, the Vivado Project is ready to be built, by running it through Synthesis and Implementation, and finally generating a bitstream. Click the Generate Bitstream button in the Program and Debug section of the Flow Navigator pane at the left side of the window.
A dialog will pop up with several options for how Synthesis and Implementation should be run. Most should be left as defaults. Of particular importance is the Number of jobs dropdown, which is used to specify how much of the resources of your computer should be dedicated to the build. A larger number of jobs will dedicate more resources, which will allow the build to be completed faster. It is recommended to choose the highest available number.
Note: Critical warnings about how IPs included within another IP were packaged with a different board value can be safely ignored. The same is true for warnings related to negative CK-to-DQS delays seen on some Zynq boards.
Depending on the complexity of the design, the board used, and the strength of your computer, the process of building the project can take between 5 and 60 minutes.
When complete, a dialog will pop up that presents several options for what to do next:
Open Implemented Design can be used to view the actual hardware design that has been implemented and will be placed onto the chip.
View Reports can be used to view additional information about the design, including how much of the resources of the FPGA will be used by the design.
Open Hardware Manager can be used to go directly to Vivado's Hardware Manager, which can be used to program a hardware design onto a board. This is typically used for designs that do not involve a software component.
Generate Memory Configuration File can be used to create a file for programming an FPGA-only design into flash memory.
If none of these options are desired, click Cancel to continue.
- Export a Hardware Platform
Once the project has been built, the design must be exported from Vivado so that Vitis has access to information about the hardware that a software application is being developed for. This includes the set of IP connected to the processor, their drivers, their addresses, and more. Exporting hardware after the bitstream has been generated allows you to program your board directly from within Vitis.
To export the hardware design, click Export → Export Hardware in the File menu.
The wizard that pops up guides you through the options available for hardware export. The first screen allows you to select a Fixed or Expandable platform. In this case, choose a Fixed platform and click Next to continue.
This screen is not present in Vivado 2022.1, proceed to the next
The Output screen allows you to select whether only the hardware specification (Pre-synthesis) should be exported, or whether the bitstream should be included. Since the bitstream has already been generated, it should be included in the platform so that Vitis can automatically figure out where it is when programming a board. Select Include bitstream and click Next to continue.
The Files screen gives you the option to choose a name for the Xilinx Shell Architecture (XSA) file, and provide a path to a folder that the file will be placed within. Give your XSA file a name, and choose a memorable location to place it in. This file will later be imported into Vitis, so take a note of where it is placed and what it is called.
Important: Do not use spaces in the file name or export path. Underscores or camelCase are recommended instead.
Click Next to continue.
The final screen of the wizard summarizes the options you selected. Click Finish.
- Update a Hardware Platform in Vitis
If a hardware design is changed after having created a Vitis application project, several steps must be taken in order to update the Vitis workspace with a newly exported XSA file. The XSA file contains all of the information relevant to Vitis about the hardware platform, and changing a platform project's specification based on this file will automatically load in any changes. This includes adding new drivers for new IP that have been installed and changing the files that define the addresses and other details of any installed IP that may have been renamed or had their addresses changed.
These steps assume that you have already regenerated the bitstream and reexported hardware in the same way that would be done prior to creating a new Vitis workspace.
Within Vitis' Assistant pane, find the platform project that you wish to update with the new hardware. This project will typically have a name that ends with “_wrapper”, and is marked with the text “[Platform]”.
Right click on this project and select Update Hardware Specification.
In the dialog that pops up, click Browse, and navigate to the location of the XSA file that you want the platform to target. Click Open to select this file.
Double check that the Hardware Specification File path matches that of the XSA file you want to use, then click OK to start the automatic process of updating the platform.
When complete, a dialog will pop up to state that the platform project has been updated. Click OK to acknowledge this.
At this point, changes to the hardware specification have been loaded into the hardware platform. The bitstream will have been updated, if it was loaded into the XSA file. The set of drivers and the xparameters file will have changed to match what is in the modified design. Changes to the software application may be required before the application can be built and programmed onto the board, however, detailing what may need to be done is outside of the scope of this guide.
) in the toolbar at the top of the screen to program the board instead.Functionality
1. Connecting the Pcams into the FMC Pcam Adapter
First, you have to make sure that all the Pcam connectors in which you want to insert the FFC cable are open. Pull the black plastic tab on the edge of the connector out, away from the opening of the connector. This opens the connector, as it can be seen in the first image bellow.
Insert the FFC with the contacts facing up, toward the Pcam 5C PCB. The blue side (without contacts exposed) should be facing down on the Pcam side. Repeat the same procedure with the Pcam connector on the FMC Pcam Adapter, as you can see in the image number 2.
After this, please ensure the FFC is fully inserted, and press both sides of the black plastic tab back towards the rest of the connector to latch the FFC in. Bellow you can see two cables which are fully inserted.
Repeat the same procedure with the camera C and D. But, this time the cables should go trough the two slots which are present on the Adapter. If the cable has become disconnected from the Pcam 5C or the FMC Pcam Adapter, please repeat the steps presented above.
Finally, after you connected all the cameras, you have to connect the FMC connector to the ZedBoard. To ensure that it stays firmly in place you should use one or two bolts as you can see in the image bellow.
2. Setting Up the ZedBoard
Plug one end of the VGA cable into the VGA port of the ZedBoard and the other end into your VGA monitor. Attach a USB micro cable between the J7 programming port and your computer. Also, attach a second USB micro cable between J14 UART port and your computer. After this, attach an external 12V power supply. Finally, make sure that VADJ is set to 3V3 and all the MIOs pins are connected to SIG and GND. This sets the primary boot mode to JTAG.
3. Setting Up the Serial Terminal
To see the UART communication channel, open a terminal program on your computer set to 115200 baud rate, 8 data bits, no parity bit and 1 stop bit. On startup, the ZedBoard will show feed-back and error messages on the serial console.
If a camera is unconnected you will receive an warning message. The program will skip this warning and will start to initialize the rest of the cameras.
As has been said before, if a camera is unconnected the project will continue the initialization of the rest of the cameras, and the VGA monitor will output a gray image, signaling the lack of video data in the DDR memory.
Implementation details: Notes on VDMA synchronization
Read this if you don't want to spend a month reverse engineering the project and dreaming of IP datasheets. The synchronization configuration we arrived at isn't particularly conventional, but it's the next best thing to deploying a fully custom synchronization generator.
Logic overview
This demo receives four 1920×1080 30fps streams from the four pcams, passes them through four identical processing pipelines (MIPI D-PHY receiver → MIPI CSI 2 receiver → Bayer to RGB → Gamma Correction → Video scaler) and writes them to memory using the VDMA IP (Stream to Memory-Mapped / S2MM). Each of the cameras gets mapped to one of four distinct regions inside the output framebuffers. These are then read from memory to be displayed, also using the VDMA IP (Memory-Mapped to Stream / MM2S). The output is a 1920×1080 60fps feed that is sent through VGA.
The framerate mismatch between the inputs and output and the asynchronous nature of the five pipelines (4 inputs, 1 output) means that all VDMA memory operations must be properly synchronized so that we never read from partially written frames. To do this, we need to use multiple framebuffers (in this configuration, 4).
Genlock and why we can't really use it
Genlock synchronization seems to be mainly designed for systems with one active master (which may be selected from a bunch of candidate masters) and one or multiple slaves. The VDMA channels must be set as master/slave from the hardware design and cannot be reassigned at runtime. The master blindly advances its frame pointer. Each slave is configured to always position itself X frames behind the master, whenever the slave needs to move to a new frame.
Dynamic genlock synchronization is a slightly smarter variant in which the dynamic-master receives a frame pointer as “feedback” from one of its slaves, so it may avoid overlapping that one slave. While this may initially seem like a suitable solution, the fact that the master only tries to avoid overlapping one of its slaves and cannot be configured to avoid multiple of them makes using this mode unfeasible.
The demo must work with any camera or cameras missing ⇒ no single VDMA input channel may be set up as a master (or dynamic-master) in the block design, with the others being its slaves (or dynamic-slaves), because if that VDMA's pcam is missing, there will be no frame progression for the others to synchronize to. Any scheme we choose must be able to select any of the inputs as a sort of a master. The software chooses the first working camera, but this may be configured in any way, as long as the chosen master has a working video stream.
Similarly, the output channel may not be set as a master because it has a higher framerate than the inputs and it will start “running circles around them”. Even if it would be dynamic-master and it would avoid one of its slaves, it may still run over slaves which are lagging one frame behind.
Working synchronization scheme
We found two similar working configurations, but which (very subjectively) feel like we're abusing barely documented behaviour, brokenheartedly embracing Hyrum's law. Both configurations have all VDMA channels set up as slaves and/or dynamic-slaves and then “trick” one of the inputs to behave like a blind master through its control register, at runtime, making all others follow that “appointed master”.
All channels receive a concatenated bundle of all S2MM frame pointers, from which we select the appointed master through each channel's PointNum, at runtime. In this way, any of the input channels may become the master to all of the other channels.
The other input channels try to always be on the same frame as the appointed master, but they will often lag one frame behind.
The output/MM2S channel is set up as a slave with delay 2, so it will never overlap with any input slave. It may also lag behind by one frame.
Thus, the total number of framebuffers must be at least 4:
master + non-lagging input slaves
lagging input slaves, if any
non-lagging output slave
lagging output slave, if lagging
Note: the output doesn't lag nearly as often as the inputs, especially because it runs at a higher framerate and gets to jump to the latest framebuffer index more often. If memory is very limited, using just 3 framebuffers may suffice, with frame tearing only occuring once every few seconds, rather than constantly.
The two working configurations only differ in the way we design the appointed master:
All input channels are slaves. The appointed master's frame progression is done by setting it as a slave to itself, with a delay of
NUM_FRAMESTORES - 1. Going that many frames “back” relative to itself is equivalent to advancing one frame. $x - (N-1) \equiv x + 1 \mod N$All input channels are slaves/dynamic-slaves. The appointed master's frame progression is done by setting
EnableSync = 0andEnableCircularBuf = 1. In this situation, it ignores the genlock mechanism and falls back to looping through the framebuffers in a circular manner, thus acting like a blind master.
We arbitrarily chose the second variant with non-dynamic slaves.
Additional Resources
All materials related to the use of the ZedBoard can be found on its Resource Center.
For a walkthrough of the process of creating a simple baremetal software project in Vivado and Vitis, see Getting Started with Vivado and Vitis for Baremetal Software Projects. Information on important parts of the GUIs, and indirect discussion of the steps required to modify, rebuild, and run this demo in hardware can also be found here.
For technical support, please visit the FPGA section of the Digilent Forum.
