README.md 13.6 KB
Newer Older
Simpson, Thomas's avatar
Simpson, Thomas committed
1
2
<p>
<div align="center">
Simpson, Thomas's avatar
Simpson, Thomas committed
3
  <h1 align="center">ZCU104 VCU + ML Platform & Demo Build Example</h1>
Simpson, Thomas's avatar
Simpson, Thomas committed
4
5
6
</div>
</p>

7
# Introduction
Simpson, Thomas's avatar
Simpson, Thomas committed
8
This document describes steps that can be used to build the ``ZCU104 VCU 8-channel video decode + ML`` demo provided on the Xilinx download page - https://www.xilinx.com/support/download/index.html/content/xilinx/en/downloadNav/embedded-platforms/2019-2.html
9

Simpson, Thomas's avatar
Simpson, Thomas committed
10
11
12
13
# Prerequisites

+ Linux host machine
+ 2019.2 Xilinx tools
14
+ Docker
Simpson, Thomas's avatar
Simpson, Thomas committed
15
16
17
+ HDMI IP License (if building or modifying the platform)

# Introduction
Simpson, Thomas's avatar
Simpson, Thomas committed
18
This document assumes that the 2019.2 Xilinx tools are set up on the host machine.  The structure of this document is organized as follows:
Simpson, Thomas's avatar
Simpson, Thomas committed
19

Simpson, Thomas's avatar
Simpson, Thomas committed
20
- <a href="#part-0-project-set-up">Part 0: Project set up</a>
Simpson, Thomas's avatar
Simpson, Thomas committed
21
- <a href="#part-1-building-the-zcu104-vitis-platform-with-vcu-and-ml-support">Part 1: Building the ZCU104 Vitis platform with VCU and ML support</a>
Simpson, Thomas's avatar
Simpson, Thomas committed
22
- <a href="#part-2-adding-the-dpu-to-the-platform">Part 2: Adding the DPU to the platform</a>
23
24
- <a href="#part-3-compiling-the-demo-software">Part 3: Compiling the demo software</a>
- <a href="#part-4-running-on-the-zcu104-target-hardware">Part 4: Running on the ZCU104 target hardware</a>
Simpson, Thomas's avatar
Simpson, Thomas committed
25

Simpson, Thomas's avatar
Simpson, Thomas committed
26
# Part 0: Project set up
Simpson, Thomas's avatar
Simpson, Thomas committed
27

Simpson, Thomas's avatar
Simpson, Thomas committed
28
29
30
31
32
33
34
35
36
37
38
39
- Create a project directory of your choosing (i.e. ~/zcu104_vcu_ml) and create an environment variable that points to that location.

  ```bash
  mkdir ~/zcu104_vcu_ml
  export PROJ_DIR=~/zcu104_vcu_ml
  ```

- Clone this repository
  ```bash
  cd $PROJ_DIR
  git clone https://xterra2.avnet.com/xilinx/ZCU104/zcu104-vcu-ml-build-example
  ```
Simpson, Thomas's avatar
Simpson, Thomas committed
40

41
42
# Part 1: Building the ZCU104 Vitis platform with VCU and ML support
This section describes how to build the Vitis embedded platform with VCU and ML support.  
Simpson, Thomas's avatar
Simpson, Thomas committed
43
44
45
46
47

- From your Linux host machine clone the Xilinx platform repository
  ```bash
  cd $PROJ_DIR
  git clone https://github.com/Xilinx/Vitis_Embedded_Platform_Source.git
Simpson, Thomas's avatar
Simpson, Thomas committed
48
49
  cd Vitis_Embedded_Platform_Source
  git checkout 2019.2
Simpson, Thomas's avatar
Simpson, Thomas committed
50
51
52
53
  ```

- Navigate to the zcu104_vcu_ml directory in the cloned repository
  ```bash
Simpson, Thomas's avatar
Simpson, Thomas committed
54
  cd $PROJ_DIR/Vitis_Embedded_Platform_Source/Xilinx_Official_Platforms/zcu104_vcu_ml
Simpson, Thomas's avatar
Simpson, Thomas committed
55
56
  ```

57
- If you are using Ubuntu 18.04 and have less than 16 cores on your machine then you will need to modify the Vivado build script to reduce the number of parallel jobs.  The path to the script is ``vivado/zcu104_vcu_ml_xsa.tcl``.  Searching for the keyword ``launch_runs`` will show that the default script is setup to run 16 jobs.  On my virtual machine this causes the build to fail since my machine only has 6 cores.  
Simpson, Thomas's avatar
Simpson, Thomas committed
58
59
60
61
62
63
64
65

  ```bash
  cat -n vivado/zcu104_vcu_ml_xsa.tcl | grep launch_runs
    2734  launch_runs impl_1 -to_step write_bitstream -jobs 16
  ```

  You may need to reduce the number of jobs to a suitable number for your machine

Simpson, Thomas's avatar
Simpson, Thomas committed
66
- The ``zcu104_vcu_ml`` platform located in the Xilinx platform repository needs to be modified to host the root file system on the SD card instead of in RAM.  Execute the following command to update petalinux project:
Simpson, Thomas's avatar
Simpson, Thomas committed
67
68
69
70
  ```bash
  bash -x $PROJ_DIR/zcu104-vcu-ml-build-example/scripts/update_plnx.sh
  ```

71
72
73
74
75
- If you would like to use the Vitis-AI Runtime and Vitis-AI Library then you will need to add additional packages.  If you only plan to use the legacy DNNDK APIs then you do not need to execute this step.
  ```bash
  bash -x $PROJ_DIR/zcu104-vcu-ml-build-example/scripts/update_for_vart.sh
  ```

Simpson, Thomas's avatar
Simpson, Thomas committed
76
77
78
79
80
81
82
83
84
85
86
- Build the platform (this will take some time)
  ```bash
  make all
  ```

- Generate the root file system for cross-compilation
  
  ```bash
  make peta_sysroot
  ```

87
- Create an environment variable that points to the platform location
Simpson, Thomas's avatar
Simpson, Thomas committed
88
89
90
91
  ```bash
  export PFM_DIR=$PROJ_DIR/Vitis_Embedded_Platform_Source/Xilinx_Official_Platforms/zcu104_vcu_ml/platform_repo/zcu104_vcu_ml/export/zcu104_vcu_ml
  ```

92
# Part 2: Adding the DPU to the platform
Simpson, Thomas's avatar
Simpson, Thomas committed
93
The pre-built zcu104_vcu_ml <a href="https://www.xilinx.com/member/forms/download/design-license-zcu104-vcu-8channel.html?filename=zcu104_vcu_ml_2019_2_platform.zip">platform</a> and <a href="https://www.xilinx.com/member/forms/download/design-license-zcu104-vcu-8channel.html?filename=zcu104_vcu_ml_2019_2_demo.zip">demo package</a> are set up for Vitis-AI v1.0.  This section will use the DPU-TRD from the Vitis-AI v1.1 release to update to the latest version of the DPU IP.  This will force us to recompile the face detection and traffic detection models from the Xilinx Model Zoo.  The instructions for recompiling the models is provided in <a href="#part-3-compiling-the-demo-software">Part 3</a>.
Simpson, Thomas's avatar
Simpson, Thomas committed
94

95
- Clone the Vitis-AI v1.1 repository
Simpson, Thomas's avatar
Simpson, Thomas committed
96
  ```bash
97
  git clone --branch v1.1 --single-branch https://github.com/Xilinx/Vitis-AI ~/Vitis-AI-v1.1
Simpson, Thomas's avatar
Simpson, Thomas committed
98
99
  ```

100
- Copy the DPU-TRD directory from the Vitis-AI v1.1 repository to the project directory
Simpson, Thomas's avatar
Simpson, Thomas committed
101
  ```bash
Simpson, Thomas's avatar
Simpson, Thomas committed
102
  cp -rp ~/Vitis-AI-v1.1/DPU-TRD $PROJ_DIR/.
Simpson, Thomas's avatar
Simpson, Thomas committed
103
104
  ```

105
- Modify the DPU configuration to enable URAM and the project configuration for 1 DPU kernel. 
Simpson, Thomas's avatar
Simpson, Thomas committed
106
  ```bash
107
  cd $PROJ_DIR/DPU-TRD/prj/Vitis
Simpson, Thomas's avatar
Simpson, Thomas committed
108
  bash -x $PROJ_DIR/zcu104-vcu-ml-build-example/scripts/update_dpu_config.sh
Simpson, Thomas's avatar
Simpson, Thomas committed
109
  ```
110

Simpson, Thomas's avatar
Simpson, Thomas committed
111
112
113
114
- Source the XRT setup script (your path may vary)
  ```bash
  source /opt/xilinx/xrt/setup.sh
  ```
115

Simpson, Thomas's avatar
Simpson, Thomas committed
116
117
- Build with default DPU configuration
  ```bash
118
119
  cd $PROJ_DIR/DPU-TRD/prj/Vitis
  export SDX_PLATFORM=$PFM_DIR/zcu104_vcu_ml.xpfm
Simpson, Thomas's avatar
Simpson, Thomas committed
120
121
122
  make KERNEL=DPU DEVICE=zcu104
  ```

123
124
125
126
# Part 3: Compiling the demo software
This section describes how to compile the demo software on the ZCU104

## Compile the Application
Simpson, Thomas's avatar
Simpson, Thomas committed
127
128

- Set up the Linux host environment for cross-compilation
129
130
131
132
  ```bash
  source $PFM_DIR/../../../sysroot/environment-setup-aarch64-xilinx-linux
  export SYSROOT=$PFM_DIR/../../../sysroot/sysroots/aarch64-xilinx-linux
  ```  
Simpson, Thomas's avatar
Simpson, Thomas committed
133

Simpson, Thomas's avatar
Simpson, Thomas committed
134
135
  **Note:** When sourcing the environment setup script you may receive a warning indicating that you need to ``unset LD_LIBRARY_PATH``.  If that is the case then please execute ``unset LD_LIBRARY_PATH`` and then source the environment setup script again.

136
137
138
139
140
141
142
- Download the demo package from the Xilinx download site - <a href="https://www.xilinx.com/member/forms/download/design-license-zcu104-vcu-8channel.html?filename=zcu104_vcu_ml_2019_2_demo.zip">link</a>.

- Extract the demo archive to the project directory
  ```bash
  unzip zcu104_vcu_ml_2019_2_demo.zip -d $PROJ_DIR/zcu104_vcu_ml_2019_2_demo
  ```

Simpson, Thomas's avatar
Simpson, Thomas committed
143
- Several make files will need to be modified to point to your cross-compilation root file system location
144
145
146
147
148
149
150
151
152
153
154
155
156
157
  ```txt
  zcu104_vcu_ml_2019_2_demo/workspaces/gst/allocator/Debug/makefile
  zcu104_vcu_ml_2019_2_demo/workspaces/gst/allocator/Debug/subdir.mk
  zcu104_vcu_ml_2019_2_demo/workspaces/gst/base/Debug/makefile
  zcu104_vcu_ml_2019_2_demo/workspaces/gst/base/Debug/subdir.mk
  zcu104_vcu_ml_2019_2_demo/workspaces/gstsdxfacedetect/Debug/makefile
  zcu104_vcu_ml_2019_2_demo/workspaces/gstsdxfacedetect/Debug/src/subdir.mk
  zcu104_vcu_ml_2019_2_demo/workspaces/gstsdxtrafficdetect/Debug/makefile
  zcu104_vcu_ml_2019_2_demo/workspaces/gstsdxtrafficdetect/Debug/subdir.mk  
  zcu104_vcu_ml_2019_2_demo/workspaces/rtsp/makefile
  zcu104_vcu_ml_2019_2_demo/workspaces/rtsp/subdir.mk
  zcu104_vcu_ml_2019_2_demo/workspaces/xrtutils/Debug/makefile
  zcu104_vcu_ml_2019_2_demo/workspaces/xrtutils/Debug/subdir.mk  
  ```
Simpson, Thomas's avatar
Simpson, Thomas committed
158

Simpson, Thomas's avatar
Simpson, Thomas committed
159
  <br>
Simpson, Thomas's avatar
Simpson, Thomas committed
160
161
  In the files listed above you will see several hard coded paths that need to be updated.  

Simpson, Thomas's avatar
Simpson, Thomas committed
162
  <br>
Simpson, Thomas's avatar
Simpson, Thomas committed
163
  
Simpson, Thomas's avatar
Simpson, Thomas committed
164
  Modified make files can be created by the ``update_make.sh`` script to modify files to use the ``$SYSROOT`` environment variable
Simpson, Thomas's avatar
Simpson, Thomas committed
165
  
Simpson, Thomas's avatar
Simpson, Thomas committed
166
  ```bash
Simpson, Thomas's avatar
Simpson, Thomas committed
167
  cd $PROJ_DIR/zcu104_vcu_ml_2019_2_demo/workspaces
Simpson, Thomas's avatar
Simpson, Thomas committed
168
169
  bash -x $PROJ_DIR/zcu104-vcu-ml-build-example/scripts/update_make.sh
  ```
Simpson, Thomas's avatar
Simpson, Thomas committed
170

Simpson, Thomas's avatar
Simpson, Thomas committed
171
172
173
174
175
176
177
- Compile the demo application
  * The ``$PROJ_DIR/zcu104-vcu-ml-build-example/scripts/compile_all.sh`` script can be used to compile the demo application  
    ```bash
    bash -x $PROJ_DIR/zcu104-vcu-ml-build-example/scripts/compile_all.sh
    ```

  * If you want to manually compile each library component then you will need to do so in the following order
178
179
180
181
182
183
    1) zcu104_vcu_ml_2019_2_demo/workspaces/xrtutils
    2) zcu104_vcu_ml_2019_2_demo/workspaces/gst/allocator
    3) zcu104_vcu_ml_2019_2_demo/workspaces/gst/base
    4) zcu104_vcu_ml_2019_2_demo/workspaces/gstsdxfacedetect
    5) zcu104_vcu_ml_2019_2_demo/workspaces/gstsdxtrafficdetect
    6) zcu104_vcu_ml_2019_2_demo/workspaces/rtsp
Simpson, Thomas's avatar
Simpson, Thomas committed
184
    <br>
Simpson, Thomas's avatar
Simpson, Thomas committed
185

186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
## Compile the Face and Traffic Models
- Download the face detection and traffic detection models from the Vitis AI Model Zoo
  ```bash
  mkdir -p $PROJ_DIR/models
  cd $PROJ_DIR/models
  wget https://www.xilinx.com/bin/public/openDownload?filename=cf_densebox_wider_320_320_1.1.zip -O cf_densebox_wider_320_320_1.1.zip
  wget https://www.xilinx.com/bin/public/openDownload?filename=cf_ssdtraffic_360_480_0.9_1.1.zip -O cf_ssdtraffic_360_480_0.9_1.1.zip
  unzip cf_densebox_wider_320_320_1.1.zip 
  unzip cf_ssdtraffic_360_480_0.9_1.1.zip 
  ```
- Modify the ``cf_ssdtraffic_360_480_0.9_11.6G/quantized/deploy.prototxt`` file to specify a new input layer
  * Replace
    ```txt
    name:"test"
    input:"data"
    input_shape{
    dim:1
    dim:3
    dim:360
    dim:480
    }
    ```
  * With
    ```txt
    layer {
      name: "data"
      type: "Input"
      top: "data"
      transform_param {
        mean_value: 104
        mean_value: 117
        mean_value: 123
        force_color: true
        resize_param {
          prob: 1
          resize_mode: WARP
          height: 360
          width: 480
          interp_mode: LINEAR
        }
      }
      input_param {
        shape {
          dim: 1
          dim: 3
          dim: 360
          dim: 480
        }
      }
    }
    ```

- Remove the following unsupported output layers
  + ``mbox_conf_reshape``
  + ``mbox_conf_softmax``
  + ``mbox_conf_flatten``
  + ``detection_out``

- Compile the models for execution on the DPU
  * If you don't have docker set up on your host machine follow the instructions <a href="https://github.com/Xilinx/Vitis-AI/blob/master/doc/install_docker/README.md">here</a>  
  
  * Start the docker container
    ```bash
    cd $PROJ_DIR
Simpson, Thomas's avatar
Simpson, Thomas committed
250
    ln -sf ~/Vitis-AI-v1.1/docker
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
    ~/Vitis-AI-v1.1/docker_run.sh xilinx/vitis-ai-cpu:latest
    ```
  
  * Load the Caffe conda environment within the docker container
    ```bash
    conda activate vitis-ai-caffe
    ```

  * Create a JSON file which specifies the DPU architecture options
    ```bash
    dlet -f DPU-TRD/prj/Vitis/binary_container_1/sd_card/zcu104_vcu_ml.hwh
    mv *.dcf models/zcu104_vcu_ml.dcf
    cd models
    echo '{"target":"dpuv2", "dcf":"zcu104_vcu_ml.dcf", "cpu_arch":"arm64"}' > zcu104_vcu_ml.json 
    ```
  
  * Compile the face detection model
    ```bash
    vai_c_caffe --prototxt cf_densebox_wider_320_320_0.49G/quantized/deploy.prototxt \
                --caffemodel cf_densebox_wider_320_320_0.49G/quantized/deploy.caffemodel \
                --arch zcu104_vcu_ml.json \
                --output_dir vai_c_output_face/ \
                --net_name densebox \
                --options "{'save_kernel':''}"
    
    ```
  
  * Compile the traffic detection model
    ```bash
    vai_c_caffe --prototxt cf_ssdtraffic_360_480_0.9_11.6G/quantized/deploy.prototxt \
                --caffemodel cf_ssdtraffic_360_480_0.9_11.6G/quantized/deploy.caffemodel \
                --arch zcu104_vcu_ml.json \
                --output_dir vai_c_output_traffic/ \
                --net_name ssd \
                --options "{'save_kernel':''}"
    ```  

  * Exit the docker container by pressing ``ctrl-d``

  * Convert the DPU executable files to shared libraries
    ```bash
    cd $PROJ_DIR/models
    aarch64-xilinx-linux-g++ -nostdlib -fPIC -shared vai_c_output_face/dpu_densebox.elf -o libdpumodeldensebox.so
    aarch64-xilinx-linux-g++ -nostdlib -fPIC -shared vai_c_output_traffic/dpu_ssd.elf -o libdpumodelssd.so
    ```

# Part 4: Running on the ZCU104 target hardware
This section describes how to compile the demo software on the ZCU104


Simpson, Thomas's avatar
Simpson, Thomas committed
301
302
303
304
305
306
307
308
- Copy files to an ``sd_card`` directory
  * You can do this by using the ``copy_sd_files.sh`` script 
    ```bash
    cd $PROJ_DIR/zcu104_vcu_ml_2019_2_demo
    bash -x $PROJ_DIR/zcu104-vcu-ml-build-example/scripts/copy_sd_files.sh
    ```

    The files will be located in an sd_card directory at ``$PROJ_DIR/zcu104_vcu_ml_2019_2_demo``
309
  
Simpson, Thomas's avatar
Simpson, Thomas committed
310
311
  * If you want to copy the files manually then see the ``copy_sd_files.sh`` script for reference

312
    **Note:** There are several scripts in the pre-built sdcard directory included in the ``zcu104_vcu_ml_2019_2_demo.zip`` archive that are copied to the ``sd_card`` directory.
Simpson, Thomas's avatar
Simpson, Thomas committed
313

314
- Copy files to the SD card
Simpson, Thomas's avatar
Simpson, Thomas committed
315
316
317
318
319
320
321
322
323
324
325
326
327

    + Format the SD card with 2 partitions using the instructions - <a href="https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842385/How+to+format+SD+card+for+SD+boot">here</a>
    + Copy the contents of the ``$PROJ_DIR/zcu104_vcu_ml_2019_2_demo/sd_card`` directory to the BOOT partition of the SD card.  The following command assumes the BOOT partition is mounted at ``/media/BOOT`` on your Linux host, modify accordingly
      
      ```bash
      cp -rp $PROJ_DIR/zcu104_vcu_ml_2019_2_demo/sd_card/* /media/BOOT/.
      ```

    + Extract the root file system to the ROOT partition of the SD card.  The following command assumes the root partition is mounted at ``/media/ROOT`` on your Linux host, modify accordingly

      ```bash
      sudo tar -xvzf $PROJ_DIR/zcu104_vcu_ml_2019_2_demo/sd_card/rootfs.tar.gz -C /media/ROOT
      ```
Simpson, Thomas's avatar
Simpson, Thomas committed
328

Simpson, Thomas's avatar
Simpson, Thomas committed
329
330
    + Unmount and eject the SD card from your Linux host

331
 - Insert the SD card into the ZCU104 and boot the board
Simpson, Thomas's avatar
Simpson, Thomas committed
332
  
333
   **Note:** This demo uses the HDMI Tx port on the board for displaying results - make sure your hardware is connected properly
Simpson, Thomas's avatar
Simpson, Thomas committed
334

335
- Once the ZCU104 boots, navigate to the ``/media/card`` directory and execute the ``setup.sh`` script
Simpson, Thomas's avatar
Simpson, Thomas committed
336
  ```txt
Simpson, Thomas's avatar
Simpson, Thomas committed
337
  ZCU104: cd /media/card
Simpson, Thomas's avatar
Simpson, Thomas committed
338
  ZCU104: ./setup.sh
Simpson, Thomas's avatar
Simpson, Thomas committed
339
340
  ```

341
- Run an 8-channel face & traffic detection example
Simpson, Thomas's avatar
Simpson, Thomas committed
342
  ```txt
343
  ZCU104: ./8_ch_traffic_face.sh
Simpson, Thomas's avatar
Simpson, Thomas committed
344
  ```
345