README.md 10.7 KB
Newer Older
Simpson, Thomas's avatar
Simpson, Thomas committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
<p>
<div align="center">
  <h1 align="center">ZCU104 FMC Quad-Camera + ML Example</h1>
</div>
</p>

# Introduction
This document describes steps that can be used to build the ``ZCU104 FMC Quad-Camera + ML`` example.  The base Vitis Acceleration platform for the ZCU104 FMC Quad-Camera (zcu104_mc4) is used as a starting point.  ML capabilities are then added to the platform using Vitis-AI v1.1.

# Prerequisites

+ Linux host machine
+ 2019.2 Xilinx tools
+ Docker
+ HDMI IP License
+ MIPI CSI Rx IP License

Simpson, Thomas's avatar
Simpson, Thomas committed
18
# Overview
Simpson, Thomas's avatar
Simpson, Thomas committed
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
This document assumes that the 2019.2 Xilinx tools are set up on the host machine.  The structure of this document is organized as follows:

- <a href="#part-0-project-set-up">Part 0: Project set up</a>
- <a href="#part-1-building-the-zcu104-vitis-platform-with-fmc-quad-cam-and-ml-support">Part 1: Building the ZCU104 Vitis platform with FMC Quad-Cam and ML support</a>
- <a href="#part-2-adding-the-dpu-to-the-platform">Part 2: Adding the DPU to the platform</a>
- <a href="#part-3-compiling-the-demo-software">Part 3: Compiling the demo software</a>
- <a href="#part-4-running-on-the-zcu104-target-hardware">Part 4: Running on the ZCU104 target hardware</a>

# Part 0: Project set up

- Create a project directory of your choosing (i.e. ~/zcu104_mc4_ml) and create an environment variable that points to that location.

  ```bash
  mkdir ~/zcu104_mc4_ml
  export PROJ_DIR=~/zcu104_mc4_ml
  ```

- Clone this repository
  ```bash
  cd $PROJ_DIR
  git clone https://xterra2.avnet.com/xilinx/ZCU104/zcu104-mc4-ml-example
  ```

# Part 1: Building the ZCU104 Vitis platform with FMC Quad-Cam and ML support
This section describes how to build the Vitis embedded platform with FMC Quad-Camera and ML support.  

- From your Linux host machine clone the Avnet platform repository
  ```bash
  mkdir -p $PROJ_DIR/platform
  cd $PROJ_DIR/platform
  git clone https://xterra2.avnet.com/xilinx/vitis_embedded_platform_source/zcu104/zcu104_mc4
  ```

- Navigate to the zcu104_mc4 directory in the cloned repository
  ```bash
  cd zcu104_mc4
  ```

- The ``zcu104_mc4`` platform located in the Avnet platform repository needs to be modified to host the root file system on the SD card instead of in RAM.  Execute the following command to update the petalinux project:

  ```bash
  bash -x $PROJ_DIR/zcu104-mc4-ml-example/scripts/update_plnx.sh
  ```

- If you would like to use the Vitis-AI Runtime and Vitis-AI Library then you will need to add additional packages.  If you only plan to use the legacy DNNDK APIs then you do not need to execute this step.
  ```bash
  bash -x $PROJ_DIR/zcu104-mc4-ml-example/scripts/update_for_vart.sh
  ```

- Build the platform (this will take some time)
  ```bash
  make all
  ```

- Generate the root file system for cross-compilation
  
  ```bash
  make peta_sysroot
  ```

- Create an environment variable that points to the platform location
  ```bash
  export PFM_DIR=$PROJ_DIR/platform/zcu104_mc4/platform_repo/zcu104_mc4/export/zcu104_mc4
  ```

# Part 2: Adding the DPU to the platform
 This section describes how to use the DPU-TRD from the Vitis-AI v1.1 release to add the DPU IP as an accelerator to our project.  

- Clone the Vitis-AI v1.1 repository
  ```bash
  git clone --branch v1.1 --single-branch https://github.com/Xilinx/Vitis-AI ~/Vitis-AI-v1.1
  ```

- Copy the DPU-TRD directory from the Vitis-AI v1.1 repository to the project directory
  ```bash
  cp -rp ~/Vitis-AI-v1.1/DPU-TRD $PROJ_DIR/.
  ```

- Modify the DPU configuration to enable URAM and the project configuration for a single B4096 DPU kernel

  ```bash
  cd $PROJ_DIR/DPU-TRD/prj/Vitis
  bash -x $PROJ_DIR/zcu104-mc4-ml-example/scripts/update_dpu_config.sh
  ```

- Source the XRT setup script (**your path may vary**)
  ```bash
  source /opt/xilinx/xrt/setup.sh
  ```

Simpson, Thomas's avatar
Simpson, Thomas committed
109
- Build with updated DPU configuration to insert the DPU IP as an accelerator
Simpson, Thomas's avatar
Simpson, Thomas committed
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
  ```bash
  cd $PROJ_DIR/DPU-TRD/prj/Vitis
  export SDX_PLATFORM=$PFM_DIR/zcu104_mc4.xpfm
  make KERNEL=DPU DEVICE=zcu104
  ```

# Part 3: Compiling the demo software
This section describes how to compile the demo software on the ZCU104. The example GStreamer face detection plug-in is taken from the ZCU104_VCU_ML demo available for download from Xilinx.  

## Compile the Application

- Set up the Linux host environment for cross-compilation
  ```bash
  source $PFM_DIR/../../../sysroot/environment-setup-aarch64-xilinx-linux
  export SYSROOT=$PFM_DIR/../../../sysroot/sysroots/aarch64-xilinx-linux
  ```  

  **Note:** When sourcing the environment setup script you may receive a warning indicating that you need to ``unset LD_LIBRARY_PATH``.  If that is the case then please execute ``unset LD_LIBRARY_PATH`` and then source the environment setup script again.

- Download the demo package from the Xilinx download site - <a href="https://www.xilinx.com/member/forms/download/design-license-zcu104-vcu-8channel.html?filename=zcu104_vcu_ml_2019_2_demo.zip">link</a>.

- Extract the demo archive to the project directory
  ```bash
  unzip zcu104_vcu_ml_2019_2_demo.zip -d $PROJ_DIR/zcu104_mc4_ml_2019_2_demo
  ```

- Several make files will need to be modified to point to your cross-compilation root file system location
  ```txt
  zcu104_mc4_ml_2019_2_demo/workspaces/gst/allocator/Debug/makefile
  zcu104_mc4_ml_2019_2_demo/workspaces/gst/allocator/Debug/subdir.mk
  zcu104_mc4_ml_2019_2_demo/workspaces/gst/base/Debug/makefile
  zcu104_mc4_ml_2019_2_demo/workspaces/gst/base/Debug/subdir.mk
  zcu104_mc4_ml_2019_2_demo/workspaces/gstsdxfacedetect/Debug/makefile
  zcu104_mc4_ml_2019_2_demo/workspaces/gstsdxfacedetect/Debug/src/subdir.mk
  zcu104_mc4_ml_2019_2_demo/workspaces/xrtutils/Debug/makefile
  zcu104_mc4_ml_2019_2_demo/workspaces/xrtutils/Debug/subdir.mk  
  ```

  <br>
  In the files listed above you will see several hard coded paths that need to be updated.  

  <br>
  
  Modified make files can be created by the ``update_make.sh`` script to modify files to use the ``$SYSROOT`` environment variable
  
  ```bash
  cd $PROJ_DIR/zcu104_mc4_ml_2019_2_demo/workspaces
  bash -x $PROJ_DIR/zcu104-mc4-ml-example/scripts/update_make.sh
  ```

- Compile the demo application
  * The ``$PROJ_DIR/zcu104-mc4-ml-example/scripts/compile_all.sh`` script can be used to compile the demo application  
    ```bash
    bash -x $PROJ_DIR/zcu104-mc4-ml-example/scripts/compile_all.sh
    ```

  * If you want to manually compile each library component then you will need to do so in the following order
    1. zcu104_mc4_ml_2019_2_demo/workspaces/xrtutils
    2. zcu104_mc4_ml_2019_2_demo/workspaces/gst/allocator
    3. zcu104_mc4_ml_2019_2_demo/workspaces/gst/base
    4. zcu104_mc4_ml_2019_2_demo/workspaces/gstsdxfacedetect
    <br>

## Compile the Face Detection Model
- Download the face detection model from the Vitis AI v1.1 Model Zoo
  ```bash
  mkdir -p $PROJ_DIR/models
  cd $PROJ_DIR/models
  wget https://www.xilinx.com/bin/public/openDownload?filename=cf_densebox_wider_360_640_1.1.zip -O cf_densebox_wider_360_640_1.1.zip
  unzip cf_densebox_wider_360_640_1.1.zip 
  ```

- Compile the model for execution on the DPU
  * If you don't have docker set up on your host machine follow the instructions <a href="https://github.com/Xilinx/Vitis-AI/blob/master/doc/install_docker/README.md">here</a>  
  
  * Start the docker container
    ```bash
    cd $PROJ_DIR
Simpson, Thomas's avatar
Simpson, Thomas committed
188
    ln -sf ~/Vitis-AI-v1.1/docker
Simpson, Thomas's avatar
Simpson, Thomas committed
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
    ~/Vitis-AI-v1.1/docker_run.sh xilinx/vitis-ai-cpu:1.1.56
    ```
  
  * Load the Caffe conda environment within the docker container
    ```bash
    conda activate vitis-ai-caffe
    ```

  * Create a JSON file which specifies the DPU architecture options
    ```bash
    dlet -f DPU-TRD/prj/Vitis/binary_container_1/sd_card/zcu104_mc4.hwh
    mv *.dcf models/zcu104_mc4.dcf
    cd models
    echo '{"target":"dpuv2", "dcf":"zcu104_mc4.dcf", "cpu_arch":"arm64"}' > zcu104_mc4.json 
    ```
Simpson, Thomas's avatar
Simpson, Thomas committed
204
205

    **Note:** The ``dlet`` command is documented in the <a href="https://www.xilinx.com/support/documentation/sw_manuals/ai_inference/v1_6/ug1327-dnndk-user-guide.pdf#page=30">DNNDK user guide</a>.
Simpson, Thomas's avatar
Simpson, Thomas committed
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
  
  * Compile the face detection model
    ```bash
    vai_c_caffe --prototxt cf_densebox_wider_360_640_1.11G/quantized/deploy.prototxt \
                --caffemodel cf_densebox_wider_360_640_1.11G/quantized/deploy.caffemodel \
                --arch zcu104_mc4.json \
                --output_dir vai_c_output_face/ \
                --net_name densebox \
                --options "{'save_kernel':''}"
    
    ```

  * Exit the docker container by pressing ``ctrl-d`` or typing ``exit``

  * Convert the DPU executable to a shared library
    ```bash
    cd $PROJ_DIR/models
    aarch64-xilinx-linux-g++ -nostdlib -fPIC -shared vai_c_output_face/dpu_densebox.elf -o libdpumodeldensebox.so
    ```

# Part 4: Running on the ZCU104 target hardware
This section describes how to compile the demo software on the ZCU104

- Copy files to an ``sd_card`` directory
  * You can do this by using the ``copy_sd_files.sh`` script 
    ```bash
    cd $PROJ_DIR/zcu104_mc4_ml_2019_2_demo
    bash -x $PROJ_DIR/zcu104-mc4-ml-example/scripts/copy_sd_files.sh
    ```

    The files will be located in an ``sd_card`` directory at ``$PROJ_DIR/zcu104_mc4_ml_2019_2_demo``
  
  * If you want to copy the files manually then see the ``copy_sd_files.sh`` script for reference

 - Copy files to the SD card

    + Format the SD card with 2 partitions using the instructions - <a href="https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842385/How+to+format+SD+card+for+SD+boot">here</a>

    + Copy the contents of the ``$PROJ_DIR/zcu104_mc4_ml_2019_2_demo/sd_card/boot`` directory to the BOOT partition of the SD card.  The following command assumes the BOOT partition is mounted at ``/media/BOOT`` on your Linux host, **modify accordingly**
      
      ```bash
      cp -rp $PROJ_DIR/zcu104_mc4_ml_2019_2_demo/sd_card/boot/* /media/BOOT/.
      ```

    + Extract the root file system to the ROOT partition of the SD card.  The following command assumes the root partition is mounted at ``/media/ROOT`` on your Linux host, **modify accordingly**

      ```bash
      sudo tar -xvzf $PROJ_DIR/zcu104_mc4_ml_2019_2_demo/sd_card/root/rootfs.tar.gz -C /media/ROOT
      ```

    + Unmount and eject the SD card from your Linux host

 - Insert the SD card into the ZCU104 and boot the board
  
   **Note:** This demo uses the HDMI Tx port on the board for displaying results - make sure your hardware is connected properly

- After the board boots for the first time navigate to the ``/media/card`` directory and source the ``scripts/setup.sh`` script

  ```bash
  ZCU104: cd /media/card
  ZCU104: source scripts/setup.sh
  ```

- Change the display resolution on the HDMI monitor to 720p
  ```bash
  ZCU104: source scripts/hdmi_display_720p30.sh
  ```

- Run the quad-camera face detection example
  ```txt
  ZCU104: ./scripts/face_detect.sh
  ```
Simpson, Thomas's avatar
Simpson, Thomas committed
278