Xilinx OpenCV User Guide Ug1233
ug1233-xilinx-opencv-user-guide
ug1233-xilinx-opencv-user-guide
User Manual:
Open the PDF directly: View PDF .
Page Count: 189
Download | ![]() |
Open PDF In Browser | View PDF |
Xilinx OpenCV User Guide UG1233 (v2018.2) July 2, 2018 Revision History Revision History The following table shows the revision history for this document. Section Revision Summary 07/02/2018 Version 2018.2 Houghlines Updated the function description and its respective tables xfOpenCV Library Functions Added a note to the xfOpenCV Library Functions table 06/06/2018 Version 2018.2 Houghlines Added a new function Semi Global Method for Stereo Disparity Estimation Added a new function 04/04/2018 Version 2018.1 General Updates Minor editorial updates for 2018.1 release InitUndistortRectifyMapInverse Added a new function cornersImgToList() Added a new function UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 2 Send Feedback Table of Contents Revision History...............................................................................................................2 Chapter 1: Overview......................................................................................................4 Basic Features.............................................................................................................................. 4 xfOpenCV Kernel on the reVISION Platform............................................................................5 Chapter 2: Getting Started........................................................................................ 7 Prerequisites................................................................................................................................ 7 xfOpenCV Library Contents........................................................................................................7 Using the xfOpenCV Library.......................................................................................................8 Changing the Hardware Kernel Configuration......................................................................21 Using the xfOpenCV Library Functions on Hardware...........................................................21 Chapter 3: xfOpenCV Library API Reference................................................. 25 xf::Mat Image Container Class................................................................................................ 25 xfOpenCV Library Functions.................................................................................................... 33 Chapter 4: Design Examples Using xfOpenCV Library........................... 172 Iterative Pyramidal Dense Optical Flow............................................................................... 172 Corner Tracking Using Sparse Optical Flow.........................................................................177 Color Detection........................................................................................................................182 Difference of Gaussian Filter................................................................................................. 183 Stereo Vision Pipeline............................................................................................................. 185 Appendix A: Additional Resources and Legal Notices........................... 187 Xilinx Resources.......................................................................................................................187 Documentation Navigator and Design Hubs...................................................................... 187 References................................................................................................................................188 Please Read: Important Legal Notices................................................................................. 189 UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 3 Send Feedback Chapter 1: Overview Chapter 1 Overview This document describes the FPGA device optimized OpenCV library, called the Xilinx® xfOpenCV library and is intended for application developers using Zynq®-7000 SoC and Zynq UltraScale+ MPSoC devices. xfOpenCV library has been designed to work in the SDx™ development environment, and provides a software interface for computer vision functions accelerated on an FPGA device. xfOpenCV library functions are mostly similar in functionality to their OpenCV equivalent. Any deviations, if present, are documented. Note: For more information on the xfOpenCV library prerequisites, see the Prerequisites. To familiarize yourself with the steps required to use the xfOpenCV library functions, see the Using the xfOpenCV Library. Basic Features All xfOpenCV library functions follow a common format. The following properties hold true for all the functions. • All the functions are designed as templates and all arguments that are images, must be provided as xf::Mat. • All functions are defined in the xf namespace. • Some of the major template arguments are: ○ Maximum size of the image to be processed ○ Datatype defining the properties of each pixel ○ Number of pixels to be processed per clock cycle ○ Other compile-time arguments relevent to the functionality. The xfOpenCV library contains enumerated datatypes which enables you to configure xf::Mat. For more details on xf::Mat, see the xf::Mat Image Container Class. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 4 Send Feedback Chapter 1: Overview xfOpenCV Kernel on the reVISION Platform The xfOpenCV library is designed to be used with the SDx™ development environment. xfOpenCV kernels are evaluated on the reVISION™ platform. The following steps describe the general flow of an example design, where both the input and the output are image files. 1. Read the image using cv::imread(). 2. Copy the data to xf::Mat. 3. Call the processing function(s) in xfOpenCV. 4. Copy the data from xf::Mat to cv::Mat. 5. Write the output to image using cv::imwrite(). The entire code is written as the host code for the pipeline , from which all the calls to xfOpenCV functions are moved to hardware. Functions from OpenCV are used to read and write images in the memory. The image containers for xfOpenCV library functions are xf::Mat objects. For more information, see the xf::Mat Image Container Class. The reVISION platform supports both live and file input-output (I/O) modes. For more details, see the reVISION Getting Started Guide. • File I/O mode enables the controller to transfer images from SD Card to the hardware kernel. The following steps describe the file I/O mode. 1. Processing system (PS) reads the image frame from the SD Card and stores it in the DRAM. 2. The xfOpenCV kernel reads the image from the DRAM, processes it and stores the output back in the DRAM memory. 3. The PS reads the output image frame from the DRAM and writes it back to the SD Card. • Live I/O mode enables streaming frames into the platform, processing frames with the xfOpenCV kernel, and streaming out the frames through the appropriate interface. The following steps describe the live I/O mode. 1. Video capture IPs receive a frame and store it in the DRAM. 2. The xfOpenCV kernel fetches the image from the DRAM, processes the image, and stores the output in the DRAM. 3. Display IPs read the output frame from the DRAM and transmits the frame through the appropriate display interface. Following figure shows the reVISION platform with the xfOpenCV kernel block: UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 5 Send Feedback Chapter 1: Overview Figure 1: xfOpenCV Kernel on the reVISION Platform Processing System (PS) ARM Core Central Interconnect HPM/GP Ports HDMI TX and RX IPs DDR Controller HP Ports Data Movers AXIS AXI Interconnects AXIMM xfOpenCV Kernel Interface xfOpenCV Kernel Programmable Logic (PL) Note: For more information on the PS-PL interfaces and PL-DDR interfaces, see the Zynq UltraScale+ Device Technical Reference Manual (UG1085). UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 6 Send Feedback Chapter 2 Getting Started This chapter provides the information you need to bring up your design using the xfOpenCV library functions. Prerequisites This section lists the prerequisites for using the xfOpenCV library functions on ZCU102 based platforms. The methodology holds true for ZC702 and ZC706 reVISION platforms as well. • Download and install the SDx development environment according to the directions provided in SDSoC Environments Release Notes, Installation, and Licensing Guide (UG1294). Before launching the SDx development environment on Linux, set the $SYSROOT environment variable to point to the Linux root file system, delivered with the reVISION platform. For example: export SYSROOT =/zcu102_[es2_]rv_ss/sw/aarch64-linux-gnu/ sysroot • Download the Zynq® UltraScale+™ MPSoC Embedded Vision Platform zip file and extract its contents. Create the SDx development environment workspace in the zcu102_[es2_]rv_ss folder of the extracted design file hierarchy. For more details, see the reVISION Getting Started Guide. • Set up the ZCU102 evaluation board. For more details, see the reVISION Getting Started Guide. • Download the xfOpenCV library. This library is made available through github. Run the following git clone command to clone the xfOpenCV repository to your local disk: git clone https://github.com/Xilinx/xfopencv.git xfOpenCV Library Contents The following table lists the contents of the xfOpenCV library. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 7 Send Feedback Chapter 2: Getting Started Table 1: xfOpenCV Library Contents Folder Details include Contains the header files required by the library. include/common Contains the common library infrastructure headers, such as types specific to the library. include/core Contains the core library functionality headers, such as the math functions. include/features Contains the feature extraction kernel function definitions. For example, Harris. include/imgproc Contains all the kernel function definitions, except the ones available in the features folder. examples Contains the sample test bench code to facilitate running unit tests. The examples/ folder contains the folde rs with algorithm names. Each algorithm folder contains host files, .json file, and data folder. For more details on how to use the xfOpenCV library, see xfOpenCV Kernel on the reVISION Platform. Using the xfOpenCV Library This section describes using the xfOpenCV library in the SDx development environment. Note: The instructions in this section assume that you have downloaded and installed all the required packages. For more information, see the Prerequisites. The xfOpenCV library is structured as shown in the following table. The include folder constitutes all the necessary components to build a Computer Vision or Image Processing pipeline using the library. The folders common and core contain the infrastructure that the library functions need for basic functions, Mat class, and macros. The library functions are categorized into two folders, features and imgproc based on the operation they perform. The names of the folders are self-explanatory. To work with the library functions, you need to include the path to the include folder in the SDx project. You can include relevant header files for the library functions you will be working with after you source the include folder’s path to the compiler. For example, if you would like to work with Harris Corner Detector and Bilateral Filter, you must use the following lines in the host code: #include “features/xf_harris.hpp” //for Harris Corner Detector #include “imgproc/xf_bilateral_filter.hpp” //for Bilateral Filter UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 8 Send Feedback Chapter 2: Getting Started After the headers are included, you can work with the library functions as described in the Chapter 3: xfOpenCV Library API Reference using the examples in the examples folder as reference. The following table gives the name of the header file, including the folder name, which contains the library function. Table 2: xfOpenCV Library Contents File Path in the include folder Function Name xf::accumulate imgproc/xf_accumulate_image.hpp xf::accumulateSquare imgproc/xf_accumulate_squared.hpp xf::accumulateWeighted imgproc/xf_accumulate_weighted.hpp xf::absdiff, xf::add, xf::subtract, xf::bitwise_and, xf::bitwise_or, xf::bitwise_not, xf::bitwise_xor core/xf_arithm.hpp xf::bilateralFilter imgproc/xf_histogram.hpp xf::boxFilter imgproc/xf_box_filter.hpp xf::Canny imgproc/xf_canny.hpp xf::merge imgproc/xf_channel_combine.hpp xf::extractChannel imgproc/xf_channel_extract.hpp xf::convertTo imgproc/xf_convert_bitdepth.hpp xf::filter2D imgproc/xf_custom_convolution.hpp xf::nv122iyuv, xf::nv122rgba, xf::nv122yuv4, xf::nv212iyuv, xf::nv212rgba, xf::nv212yuv4, xf::rgba2yuv4, xf::rgba2iyuv, xf::rgba2nv12, xf::rgba2nv21, xf::uyvy2iyuv, xf::uyvy2nv12, xf::uyvy2rgba, xf::yuyv2iyuv, xf::yuyv2nv12, xf::yuyv2rgba imgproc/xf_cvt_color.hpp xf::dilate imgproc/xf_dilation.hpp xf::erode imgproc/xf_erosion.hpp xf::fast features/xf_fast.hpp xf::GaussianBlur imgproc/xf_gaussian_filter.hpp xf::cornerHarris features/xf_harris.hpp xf::calcHist imgproc/xf_histogram.hpp xf::equalizeHist imgproc/xf_hist_equalize.hpp xf::HOGDescriptor imgproc/xf_hog_descriptor.hpp xf::Houghlines imgproc/xf_houghlines.hpp xf::integralImage imgproc/xf_integral_image.hpp xf::densePyrOpticalFlow imgproc/xf_pyr_dense_optical_flow.hpp xf::DenseNonPyrOpticalFlow imgproc/xf_dense_npyr_optical_flow.hpp xf::LUT imgproc/xf_lut.hpp xf::magnitude core/xf_magnitude.hpp xf::MeanShift imgproc/xf_mean_shift.hpp xf::meanStdDev core/xf_mean_stddev.hpp xf::medianBlur imgproc/xf_median_blur.hpp xf::minMaxLoc core/xf_min_max_loc.hpp UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 9 Send Feedback Chapter 2: Getting Started Table 2: xfOpenCV Library Contents (cont'd) File Path in the include folder Function Name xf::OtsuThreshold imgproc/xf_otsuthreshold.hpp xf::phase core/xf_phase.hpp xf::pyrDown imgproc/xf_pyr_down.hpp xf::pyrUp imgproc/xf_pyr_up.hpp xf::remap imgproc/xf_remap.hpp xf::resize imgproc/xf_resize.hpp xf::Scharr imgproc/xf_scharr.hpp xf::SemiGlobalBM imgproc/xf_sgbm.hpp xf::Sobel imgproc/xf_sobel.hpp xf::StereoPipeline imgproc/xf_stereo_pipeline.hpp xf::StereoBM imgproc/xf_stereoBM.hpp xf::SVM imgproc/xf_svm.hpp xf::Threshold imgproc/xf_threshold.hpp xf::warpAffine imgproc/xf_warpaffine.hpp xf::warpPerspective imgproc/xf_warpperspective.hpp xf::warpTransform imgproc/xf_warp_transform.hpp The different ways to use the xfOpenCV library examples are listed below: • Downloading and Using xfOpenCV Libraries from SDx GUI • Building a Project Using the Example Makefiles on Linux • Using reVISION Samples on the reVISION Platform • Using the xfOpenCV Library on a non-reVISION Platform Downloading and Using xfOpenCV Libraries from SDx GUI You can download xfOpenCV directly from SDx GUI. To build a project using the example makefiles on the Linux platform: 1. From SDx IDE, click Xilinx and select SDx Libraries. 2. Click Download next to the Xilinx xfOpenCV Library. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 10 Send Feedback Chapter 2: Getting Started Figure 2: SDx Libraries The library is downloaded into /Xilinx/SDx/2018.2/xfopencv. After the library is downloaded, the entire set of examples in the library are available in the list of templates while creating a new project. Note: The library can be added to any project from the IDE menu options. 3. To add a library to a project, from SDx IDE, click Xilinx and select SDx Libraries. 4. Select Xilinx xfOpenCV Library and click Add to project. The dropdown menu consists of options of which project the libraries need to be included to. All the headers as part of the include/ folder in xfOpenCV library would be copied into the local project directory as /libs/xfopencv/include. All the settings required for the libraries to be run are also set when this action is completed. Building a Project Using the Example Makefiles on Linux Use the following steps to build a project using the example makefiles on the Linux platform: 1. Open a terminal. 2. Set the environment variable SYSROOT to the /sw/ aarch64-linux-gnu/sysroot folder. 3. Change the platform variable to point to the downloaded platform folder in makefile. Ensure that the folder name of the downloaded platform is unchanged. 4. Change the directory to the location where you want to build the example. cd UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 11 Send Feedback Chapter 2: Getting Started 5. Set the environment variables to run SDx development environment. • For c shell: source /settings.csh • For bash shell: source /settings.sh 6. Type the make command in the terminal. The sd_card folder is created and can be found in the folder. Using reVISION Samples on the reVISION Platform Use the following steps to run a unit test for bilateral filter on zcu102_es2_reVISION: 1. Launch the SDx development environment using the desktop icon or the Start menu. The Workspace Launcher dialog appears. 2. Click Browse to enter a workspace folder used to store your projects (you can use workspace folders to organize your work), then click OK to dismiss the Workspace Launcher dialog. Note: Before launching the SDx IDE on Linux, ensure that you use the same shell that you have used to set the $SYSROOT environment variable. This is usually the file path to the Linux root file system. The SDx development environment window opens with the Welcome tab visible when you create a new workspace. The Welcome tab can be closed by clicking the X icon or minimized if you do not wish to use it. 3. Select File → New → Xilinx SDx Project from the SDx development environment menu bar. The New Project dialog box opens. 4. Specify the name of the project. For example Bilateral. 5. Click Next. The the Choose Hardware Platform page appears. 6. From the Choose Hardware Platform page, click the Add Custom Platform button. 7. Browse to the directory where you extracted the reVISION platform files. Ensure that you select the zcu102_es2_reVISION folder. 8. From the Choose Hardware Platform page, select zcu102_es2_reVISION (custom). 9. Click Next. The Templates page appears, containing source code examples for the selected platform. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 12 Send Feedback Chapter 2: Getting Started 10. From the list of application templates, select bilateral - File I/O and click Finish. 11. Click the Active build configurations drop-down from the SDx Project Settings window, to select the active configuration or create a build configuration. The standard build configurations are Debug and Release. To get the best runtime performance, switch to use the Release build configuration as it uses a higher compiler optimization setting than the Debug build configuration. Figure 3: SDx Project Settings - Active Build Configuration 12. Set the Data motion network clock frequency (MHz) to the required frequency, on the SDx Project Settings page. 13. Right-click the project and select Build Project or press Ctrl+B keys to build the project, in the Project Explorer view. 14. Copy the contents of the newly created sd_card folder to the SD card. The sd_card folder contains all the files required to run designs on the ZCU102 board. 15. Insert the SD card in the ZCU102 board card slot and switch it ON. Note: A serial port emulator (Teraterm/ minicom) is required to interface the user commands to the board. 16. Upon successful boot, run the following command in the Teraterm terminal (serial port emulator.) #cd /media/card #remount 17. Run the .elf file for the respective functions. For more information, see the Using the xfOpenCV Library Functions on Hardware. Using the xfOpenCV Library on a non-reVISION Platform This section describes using the xfOpenCV library on a non-reVISION platform, in the SDx development environment. The examples in xfOpenCV require OpenCV libraries for successful compilation. In the case of a non-reVISION platform, you are responsible for providing the required OpenCV libraries, either as part of the platform or otherwise. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 13 Send Feedback Chapter 2: Getting Started Note: The instructions in this section assume that you have downloaded and installed all the required packages. For more information, see the Prerequisites. Use the following steps to import the xfOpenCV library into a SDx project and execute it on a custom platform: 1. Launch the SDx development environment using the desktop icon or the Start menu. The Workspace Launcher dialog appears. 2. Click Browse to enter a workspace folder used to store your projects (you can use workspace folders to organize your work), then click OK to dismiss the Workspace Launcher dialog. Note: Before launching the SDx IDE on Linux, ensure that you use the same shell that you have used to set the $SYSROOT environment variable. This is usually the file path to the Linux root file system. The SDx development environment window opens with the Welcome tab visible when you create a new workspace. The Welcome tab can be closed by clicking the X icon or minimized if you do not wish to use it. 3. Select File → New → Xilinx SDx Project from the SDx development environment menu bar. The New Project dialog box opens. 4. Specify the name of the project. For example Test. 5. Click Next. The the Choose Hardware Platform page appears. 6. From the Choose Hardware Platform page, select a suitable platform. For example, zcu102. 7. Click Next. The Choose Software Platform and Target CPU page appears. 8. From the Choose Software Platform and Target CPU page, select an appropriate software platform and the target CPU. For example, select A9 from the CPU dropdown list for ZC702 and ZC706 reVISION platforms. 9. Click Next. The Templates page appears, containing source code examples for the selected platform. 10. From the list of application templates, select Empty Application and click Finish. The New Project dialog box closes. A new project with the specified configuration is created. The SDx Project Settings view appears. Notice the progress bar in the lower right border of the view, Wait for a few moments for the C/C++ Indexer to finish. 11. The standard build configurations are Debug and Release. To get the best run-time performance, switch to use the Release build configuration as it uses a higher compiler optimization setting than the Debug build configuration. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 14 Send Feedback Chapter 2: Getting Started Figure 4: SDx Project Settings - Active Build Configuration 12. Set the Data motion network clock frequency (MHz) to the required frequency, on the SDx Project Settings page. 13. Select the Generate bitstream and Generate SD card image check boxes. 14. Right-click on the newly created project in the Project Explorer view. 15. From the context menu that appears, select C/C++ Build Settings. The Properties for dialog box appears. 16. Click the Tool Settings tab. 17. Expand the SDS++ Compiler → Directories tree. icon to add the “ \include and 18. Click the “ \include folder locations to the Include Paths list. Note: The OpenCV library is not provided by Xilinx for custom platforms. You are required to provide the library. Use the reVISION platform in order to use the OpenCV library provided by Xilinx. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 15 Send Feedback Chapter 2: Getting Started Figure 5: SDS++ Compiler Settings 19. Click Apply. 20. Expand the SDS++ Linker → Libraries tree. icon and add the following libraries to the Libraries(-l) list. These libraries are 21. Click the required by OpenCV. • opencv_core • opencv_imgproc UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 16 Send Feedback Chapter 2: Getting Started • opencv_imgcodecs • opencv_features2d • opencv_calib3d • opencv_flann • lzma • tiff • png16 • z • jpeg • dl • rt • webp icon and add /lib folder location to the Libraries 22. Click the search path (-L) list. Note: The OpenCV library is not provided by Xilinx for custom platforms. You are required to provide the library. Use the reVISION platform in order to use the OpenCV library provided by Xilinx. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 17 Send Feedback Chapter 2: Getting Started Figure 6: SDS++ Linker Settings 23. Click Apply to save the configuration. 24. Click OK to close the Properties for dialog box. 25. Expand the newly created project tree in the Project Explorer view. 26. Right-click the src folder and select Import. The Import dialog box appears. 27. Select File System and click Next. 28. Click Browse to navigate to the /examples folder location. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 18 Send Feedback Chapter 2: Getting Started 29. Select the folder that corresponds to the library that you desire to import. For example, accumulate. Figure 7: Import Library Example Source Files 30. Right-click the library function in the Project Explorer view and select Toggle HW/SW to move the function to the hardware. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 19 Send Feedback Chapter 2: Getting Started Figure 8: Moving a Library Function to the Hardware 31. Right-click the project and select Build Project or press Ctrl+B keys to build the project, in the Project Explorer view. The build process may take anytime between few minutes to several hours, depending on the power of the host machine and the complexity of the design. By far, the most time is spent processing the routines that have been tagged for realization in hardware. 32. Copy the contents of the newly created .\ \ \Release \sd_card folder to the SD card. The sd_card folder contains all the files required to run designs on a board. 33. Insert the SD card in the board card slot and switch it ON. Note: A serial port emulator (Teraterm/ minicom) is required to interface the user commands to the board. 34. Upon successful boot, navigate to the ./mnt folder and run the following command at the prompt: #cd /mnt Note: It is assumed that the OpenCV libraries are a port of the root filesystem. If not, add the location of OpenCV libraries to LD_LIBRARY_PATH using the $ export LD_LIBRARY_PATH= /lib command. 35. Run the .elf executable file. For more information, see the Using the xfOpenCV Library Functions on Hardware. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 20 Send Feedback Chapter 2: Getting Started Changing the Hardware Kernel Configuration Use the following steps to change the hardware kernel configuration: 1. Update the /xfOpenCV/examples/ / xf_config_params.h file. 2. Update the makefile along with the xf_config_params.h file: a. Find the line with the function name in the makefile. For bilateral filter, the line in the makefile will be xf::BilateralFilter<3,1,0,1080,1920,1>. b. Update the template parameters in the makefile to reflect changes made in the xf_config_params.h file. For more details, see the Chapter 3: xfOpenCV Library API Reference. Using the xfOpenCV Library Functions on Hardware The following table lists the xfOpenCV library functions and the command to run the respective examples on hardware. It is assumed that your design is completely built and the board has booted up correctly. Table 3: Using the xfOpenCV Library Function on Hardware Example Function Name Usage on Hardware accumulate xf::accumulate ./ .elf accumulatesquared xf::accumulateSquare ./ .elf accumulateweighted xf::accumulateWeighted ./ .elf arithm xf::absdiff, xf::add, xf::subtract, xf::bitwise_and, xf::bitwise_or, xf::bitwise_not, xf::bitwise_xor ./ .elf Bilateralfilter xf::bilateralFilter ./ .elf Boxfilter xf::boxFilter ./ .elf Canny xf::Canny ./ .elf channelcombine xf::merge ./ .elf UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 21 Send Feedback Chapter 2: Getting Started Table 3: Using the xfOpenCV Library Function on Hardware (cont'd) Example Function Name Usage on Hardware Channelextract xf::extractChannel ./ .elf Colordetect xf::RGB2HSV, xf::colorthresholding, xf:: erode, and xf:: dilate ./ .elf Convertbitdepth xf::convertTo ./ .elf Cornertracker xf::cornerTracker ./exe Customconv xf::filter2D ./ .elf cvtcolor IYUV2NV12 xf::iyuv2nv12 ./ .elf cvtcolor IYUV2RGBA xf::iyuv2rgba ./ .elf cvtcolor IYUV2YUV4 xf::iyuv2yuv4 ./ .elf cvtcolor NV122IYUV xf::nv122iyuv ./ .elf cvtcolor NV122RGBA xf::nv122rgba ./ .elf cvtcolor NV122YUV4 xf::nv122yuv4 ./ .elf cvtcolor NV212IYUV xf::nv212iyuv ./ .elf cvtcolor NV212RGBA xf::nv212rgba ./ .elf cvtcolor NV212YUV4 xf::nv212yuv4 ./ .elf cvtcolor RGBA2YUV4 xf::rgba2yuv4 ./ .elf cvtcolor RGBA2IYUV xf::rgba2iyuv ./ .elf cvtcolor RGBA2NV12 xf::rgba2nv12 ./ .elf cvtcolor RGBA2NV21 xf::rgba2nv21 ./ .elf cvtcolor UYVY2IYUV xf::uyvy2iyuv ./ .elf cvtcolor UYVY2NV12 xf::uyvy2nv12 ./ .elf cvtcolor UYVY2RGBA xf::uyvy2rgba ./ .elf cvtcolor YUYV2IYUV xf::yuyv2iyuv ./ .elf UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 22 Send Feedback Chapter 2: Getting Started Table 3: Using the xfOpenCV Library Function on Hardware (cont'd) Example Function Name Usage on Hardware cvtcolor YUYV2NV12 xf::yuyv2nv12 ./ .elf cvtcolor YUYV2RGBA xf::yuyv2rgba ./ .elf Difference of Gaussian xf:: GaussianBlur, xf:: duplicateMat, xf:: delayMat, and xf::subtract ./ .elf Dilation xf::dilate ./ .elf Erosion xf::erode ./ .elf Fast xf::fast ./ .elf Gaussianfilter xf::GaussianBlur ./ .elf Harris xf::cornerHarris ./ .elf Histogram xf::calcHist ./ .elf Histequialize xf::equalizeHist ./ .elf Hog xf::HOGDescriptor ./ .elf Houghlines xf::HoughLines ./ .elf Integralimg xf::integralImage ./ .elf Lkdensepyrof xf::densePyrOpticalFlow ./ .elf Lknpyroflow xf::DenseNonPyrLKOpticalFlow ./ .elf Lut xf::LUT ./ .elf Magnitude xf::magnitude ./ .elf meanshifttracking xf::MeanShift ./ .elf meanstddev xf::meanStdDev ./ .elf medianblur xf::medianBlur ./ .elf Minmaxloc xf::minMaxLoc ./ .elf otsuthreshold xf::OtsuThreshold ./ .elf Phase xf::phase ./ .elf Pyrdown xf::pyrDown ./ .elf UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 23 Send Feedback Chapter 2: Getting Started Table 3: Using the xfOpenCV Library Function on Hardware (cont'd) Example Function Name Usage on Hardware Pyrup xf::pyrUp ./ .elf remap xf::remap ./ .elf Resize xf::resize ./ .elf scharrfilter xf::Scharr ./ .elf SemiGlobalBM xf::SemiGlobalBM ./ .elf sobelfilter xf::Sobel ./ .elf stereopipeline xf::StereoPipeline ./ .elf stereolbm xf::StereoBM ./ .elf Svm xf::SVM ./ .elf threshold xf::Threshold ./ .elf warpaffine xf::warpAffine ./ .elf warpperspective xf::warpPerspective ./ .elf warptransform xf::warpTransform ./ .elf UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 24 Send Feedback Chapter 3 xfOpenCV Library API Reference To facilitate local memory allocation on FPGA devices, the xfOpenCV library functions are provided in templates with compile-time parameters. Data is explicitly copied from cv::Mat to xf::Mat and is stored in physically contiguous memory to achieve the best possible performance. After processing, the output in xf::Mat is copied back to cv::Mat to write it into the memory. xf::Mat Image Container Class xf::Mat is a template class that serves as a container for storing image data and its attributes. Note: The xf::Mat image container class is similar to the cv::Mat class of the OpenCV library. Class Definition template class Mat { public: Mat(); // default constructor Mat(int _rows, int _cols); Mat(int _rows, int _cols, void *_data); Mat(int _size, int _rows, int _cols); ~Mat(); void init(int _rows, int _cols); void copyTo(XF_PTSNAME(T,NPC)* fromData); unsigned char * copyFrom(); Mat(const Mat& src); Mat& operator=(const Mat& src); template void convertTo(Mat &dst, int otype, double alpha=1, double beta=0); int rows, cols, size; // actual image size #ifndef __SYNTHESIS__ XF_TNAME(T,NPC)*data; #else XF_TNAME(T,NPC) data[ROWS*(COLS>> (XF_BITSHIFT(NPC)))]; #endif }; Parameter Descriptions The following table lists the xf::Mat class parameters and their descriptions: UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 25 Send Feedback Chapter 3: xfOpenCV Library API Reference Table 4: xf::Mat Class Parameter Descriptions Parameter Description rows The number of rows in the image or height of the image. cols The number of columns in the image or width of the image. size The number of words stored in the data member. The value is calculated using rows*cols/ (number of pixels packed per word). *data The pointer to the words that store the pixels of the image. Member Functions Description The following table lists the member functions and their descriptions: Table 5: xf::Mat Member Function Descriptions Member Functions Description Mat() This default constructor initializes the Mat object sizes, using the template parameters ROWS and COLS. Mat(int _rows, int _cols) This constructor initializes the Mat object using arguments _rows and _cols. Mat(const xf::Mat &_src) This constructor helps clone a Mat object to another. New memory will be allocated for the newly created constructor. Mat(int _rows, int _cols, void *_data) This constructor initializes the Mat object using arguments _rows, _cols, and _data. The *data member of the Mat object points to the memory allocated for _data argument, when this constructor is used. No new memory is allocated for the *data member. convertTo(Mat &dst, int otype, double alpha=1, double beta=0) copyTo(* fromData) Copies the data from Data pointer into physically contiguous memory allocated inside the constructor. copyFrom() Returns the pointer to the first location of the *data member. ~Mat() This is a default destructor of the Mat object. Template Parameter Descriptions Template parameters of the xf::Mat class are used to set the depth of the pixel, number of channels in the image, number of pixels packed per word, maximum number of rows and columns of the image. The following table lists the template parameters and their descriptions: Table 6: xf::Mat Template Parameter Descriptions Parameters Description TYPE Type of the pixel data. For example, XF_8UC1 stands for 8-bit unsigned and one channel pixel. More types can be found in include/common/xf_params.h. HEIGHT Maximum height of an image. WIDTH Maximum width of an image. NPC The number of pixels to be packed per word. For instance, XF_NPPC1 for 1 pixel per word; and XF_NPPC8 for 8 pixels per word. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 26 Send Feedback Chapter 3: xfOpenCV Library API Reference Pixel-Level Parallelism The amount of parallelism to be implemented in a function from xfOpenCV is kept as a configurable parameter. In most functions, there are two options for processing data. • Single-pixel processing • Processing eight pixels in parallel The following table describes the options available for specifying the level of parallelism required in a particular function: Table 7: Options Available for Specifying the Level of Parallelism Option Description XF_NPPC1 Process 1 pixel per clock cycle XF_NPPC2 Process 2 pixels per clock cycle XF_NPPC8 Process 8 pixels per clock cycle Macros to Work With Parallelism There are two macros that are defined to work with parallelism. • The XF_NPIXPERCYCLE(flags) macro resolves to the number of pixels processed per cycle. ○ ○ ○ XF_NPIXPERCYCLE(XF_NPPC1) resolves to 1 XF_NPIXPERCYCLE(XF_NPPC2) resolves to 2 XF_NPIXPERCYCLE(XF_NPPC8) resolves to 8 • The XF_BITSHIFT(flags) macro resolves to the number of times to shift the image size to right to arrive at the final data transfer size for parallel processing. ○ ○ ○ XF_BITSHIFT(XF_NPPC1) resolves to 0 XF_BITSHIFT(XF_NPPC2) resolves to 1 XF_BITSHIFT(XF_NPPC8) resolves to 3 Pixel Types Parameter types will differ, depending on the combination of the depth of pixels and the number of channels in the image. The generic nomenclature of the parameter is listed below. XF_ C For example, for an 8-bit pixel - unsigned - 1 channel the data type is XF_8UC1. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 27 Send Feedback Chapter 3: xfOpenCV Library API Reference The following table lists the available data types for the xf::Mat class: Table 8: xf::Mat Class - Available Data Types Number of bits per Pixel Option Unsigned/ Signed/ Float Type Number of Channels XF_8UC1 8 Unsigned 1 XF_16UC1 16 Unsigned 1 XF_16SC1 16 Signed 1 XF_32UC1 32 Unsigned 1 XF_32FC1 32 Float 1 XF_32SC1 32 Signed 1 XF_8UC2 8 Unsigned 2 XF_8UC4 8 Unsigned 4 XF_2UC1 2 Unsigned 1 Manipulating Data Type Based on the number of pixels to process per clock cycle and the type parameter, there are different possible data types. The xfOpenCV library uses these datatypes for internal processing and inside the xf::Mat class. The following are a few supported types: • XF_TNAME(TYPE,NPPC) resolves to the data type of the data member of the xf::Mat object. For instance, XF_TNAME(XF_8UC1,XF_NPPC8) resolves to ap_uint<64>. • Word width = pixel depth * number of channels * number of pixels to process per cycle (NPPC). • XF_DTUNAME(TYPE,NPPC) resolves to the data type of the pixel. For instance, XF_DTUNAME(XF_32FC1,XF_NPPC1) resolves to float. • XF_PTSNAME(TYPE,NPPC) resolves to the ‘C’ data type of the pixel. For instance, XF_PTSNAME (XF_16UC1,XF_NPPC2) resolves to unsigned short. Note: ap_uint<>, ap_int<>, ap_fixed<>, and ap_ufixed<> types belong to the high-level synthesis (HLS) library. For more information, see the Vivado Design Suite User Guide: High-Level Synthesis (UG902). Sample Illustration The following code illustrates the configurations that are required to build the gaussian filter on an image, using SDSoC tool for Zynq® UltraScale™ platform. Note: In case of a real-time application, where the video is streamed in, it is recommended that the location of frame buffer is xf::Mat and is processed using the library function. The resultant location pointer is passed to display IPs. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 28 Send Feedback Chapter 3: xfOpenCV Library API Reference xf_config_params.h #define #define #define #define #define FILTER_SIZE_3 1 FILTER_SIZE_5 0 FILTER_SIZE_7 0 RO 0 NO 1 #if NO #define NPC1 XF_NPPC1 #endif #if RO #define NPC1 XF_NPPC8 #endif xf_gaussian_filter_tb.cpp int main(int argc, char **argv) { cv::Mat in_img, out_img, ocv_ref; cv::Mat in_gray, in_gray1, diff; in_img = cv::imread(argv[1], 1); // reading in the color image extractChannel(in_img, in_gray, 1); xf::Mat imgInput(in_img.rows,in_img.cols); xf::Mat imgOutput(in_img.rows,in_img.cols); imgInput.copyTo(in_gray.data); gaussian_filter_accel(imgInput,imgOutput,sigma); // Write output image xf::imwrite("hls_out.jpg",imgOutput); } xf_gaussian_filter_accel.cpp #include "xf_gaussian_filter_config.h" void gaussian_filter_accel(xf::Mat &imgInput,xf::Mat &imgOutput,float sigma) { xf::GaussianBlur (imgInput, imgOutput, sigma); } xf_gaussian_filter.hpp #pragma SDS data data_mover("_src.data":AXIDMA_SIMPLE) #pragma SDS data data_mover("_dst.data":AXIDMA_SIMPLE) #pragma SDS data access_pattern("_src.data":SEQUENTIAL) #pragma SDS data copy("_src.data"[0:"_src.size"]) #pragma SDS data access_pattern("_dst.data":SEQUENTIAL) #pragma SDS data copy("_dst.data"[0:"_dst.size"]) template UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 29 Send Feedback Chapter 3: xfOpenCV Library API Reference void GaussianBlur(xf::Mat & _src, xf::Mat & _dst, float sigma) { //function body } The design fetches data from external memory (with the help of SDSoC data movers) and is transferred to the function in 8-bit or 64-bit packets, based on the configured mode. Assuming 8-bits per pixel, 8 pixels can be packed into 64-bits. Therefore, 8 pixels are available to be processed in parallel. Enable the FILTER_SIZE_3 and the NO macros in the xf_config_params.h file. The macro is used to set the filter size to 3x3 and #define NO 1 macro enables 1 pixel parallelism. Specify the SDSoC tool specific pragmas, in the xf_gaussian_filter.hpp file. #pragma #pragma #pragma #pragma #pragma #pragma SDS SDS SDS SDS SDS SDS data data data data data data data_mover("_src.data":AXIDMA_SIMPLE) data_mover("_dst.data":AXIDMA_SIMPLE) access_pattern("_src.data":SEQUENTIAL) copy("_src.data"[0:"_src.size"]) access_pattern("_dst.data":SEQUENTIAL) copy("_dst.data"[0:"_dst.size"]) Note: For more information on the pragmas used for hardware accelerator functions in SDSoC, see SDSoC Environment User Guide (UG1027). Additional Utility Functions for Software xf::imread The function xf::imread loads an image from the specified file path, copies into xf::Mat and returns it. If the image cannot be read (because of missing file, improper permissions, unsupported or invalid format), the function exits with a non-zero return code and an error statement. Note: In an HLS standalone mode like Cosim, use cv::imread followed by copyTo function, instead of xf::imread. API Syntax template xf::Mat imread (char *filename, int type) Parameter Descriptions The table below describes the template and the function parameters. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 30 Send Feedback Chapter 3: xfOpenCV Library API Reference Table 9: xf::imread Function Parameter Descriptions Parameter Description PTYPE Input pixel type. Value should be in accordance with the ‘type’ argument’s value. ROWS Maximum height of the image to be read COLS Maximum width of the image to be read NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 and XF_NPPC8 for 1 pixel and 8 pixel operations respectively. filename Name of the file to be loaded type Flag that depicts the type of image. The values are: • '0' for gray scale • '1' for color image xf::imwrite The function xf::imwrite saves the image to the specified file from the given xf::Mat. The image format is chosen based on the file name extension. This function internally uses cv::imwrite for the processing. Therefore, all the limitations of cv::imwrite are also applicable to xf::imwrite. API Syntax template void imwrite(const char *img_name, xf::Mat &img) Parameter Descriptions The table below describes the template and the function parameters. Table 10: xf::imwrite Function Parameter Descriptions Parameter Description PTYPE Input pixel type. Supported types are: XF_8UC1, XF_16UC1, XF_8UC3, XF_16UC3, XF_8UC4, and XF_16UC4 ROWS Maximum height of the image to be read COLS Maximum width of the image to be read NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 and XF_NPPC8 for 1 pixel and 8 pixel operations respectively. img_name Name of the file with the extension img xf::Mat array to be saved xf::absDiff The function xf::absDiff computes the absolute difference between each individual pixels of an xf::Mat and a cv::Mat, and returns the difference values in a cv::Mat. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 31 Send Feedback Chapter 3: xfOpenCV Library API Reference API Syntax template void absDiff(cv::Mat &cv_img, xf::Mat & xf_img, cv::Mat &diff_img ) Parameter Descriptions The table below describes the template and the function parameters. Table 11: xf::absDiff Function Parameter Descriptions Parameter Description PTYPE Input pixel type ROWS Maximum height of the image to be read COLS Maximum width of the image to be read NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1, XF_NPPC4, and XF_NPPC8 for 1-pixel, 4-pixel, and 8-pixel parallel operations respectively. cv_img cv::Mat array to be compared xf_img xf::Mat array to be compared diff_img Output difference image(cv::Mat) xf::convertTo The xf::convertTo function performs bit depth conversion on each individual pixel of the given input image. This method converts the source pixel values to the target data type with appropriate casting. dst(x,y)= cast (α(src(x,y)+β)) Note: The output and input Mat cannot be the same. That is, the converted image cannot be stored in the Mat of the input image. API Syntax template void convertTo(xf::Mat &dst, int ctype, double alpha=1, double beta=0) Parameter Descriptions The table below describes the template and function parameters. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 32 Send Feedback Chapter 3: xfOpenCV Library API Reference Table 12: xf::convertTo Function Parameter Descriptions Parameter Description DST_T Output pixel type. Possible values are XF_8UC1, XF_16UC1, XF_16SC1, and XF_32SC1. ROWS Maximum height of image to be read COLS Maximum width of image to be read NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1, XF_NPPC4, and XF_NPPC8 for 1-pixel, 4-pixel, and 8-pixel parallel operations respectively. dst Converted xf Mat ctype Conversion type : Possible values are listed here. //Down-convert: • XF_CONVERT_16U_TO_8U • XF_CONVERT_16S_TO_8U • XF_CONVERT_32S_TO_8U • XF_CONVERT_32S_TO_16U • XF_CONVERT_32S_TO_16S //Up-convert: • XF_CONVERT_8U_TO_16U • XF_CONVERT_8U_TO_16S • XF_CONVERT_8U_TO_32S • XF_CONVERT_16U_TO_32S • XF_CONVERT_16S_TO_32S alpha Optional scale factor beta Optional delta added to the scaled values xfOpenCV Library Functions The xfOpenCV library is a set of select OpenCV functions optimized for Zynq®-7000 SoC and Zynq UltraScale+ MPSoC devices. The following table lists the xfOpenCV library functions. Table 13: xfOpenCV Library Functions Computations Absolute Difference Input Processing Bit Depth Conversion Filters Bilateral Filter Other Canny Edge Detection Accumulate Channel Combine Box Filter FAST Corner Detection Accumulate Squared Channel Extract Custom Convolution Harris Corner Detection Accumulate Weighted Color Conversion Dilate Histogram Computation UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 33 Send Feedback Chapter 3: xfOpenCV Library API Reference Table 13: xfOpenCV Library Functions (cont'd) Computations Input Processing Filters Other Atan2 Histogram Equalization Erode Dense Pyramidal LK Optical Flow Bitwise AND, Bitwise NOT, Bitwise OR, Bitwise XOR Look Up Table Gaussian Filter Dense Non-Pyramidal LK Optical Flow Gradient Magnitude Remap Sobel Filter MinMax Location Gradient Phase Resolution Conversion (Resize) Median Blur Filter Thresholding Scharr Filter WarpAffine Integral Image Inverse (Reciprocal) WarpPerspective Pixel-Wise Addition SVM Pixel-Wise Multiplication Otsu Threshold Pixel-Wise Subtraction Mean Shift Tracking Square Root HOG Mean and Standard Deviation Stereo Local Block Matching WarpTransform Pyramid Up Pyramid Down Delay Duplicate Color Thresholding RGB2HSV InitUndistortRectifyMapInver se Houghlines Semi Global Method for Stereo Disparity Estimation Notes: 1. The maximum resolution supported for all the functions is 4K, except Houghlines, HOG (RB mode), and Canny Edge Detection. The following table lists the functions that are not supported on Zynq-7000 SoC devices, when configured to use 128-bit interfaces in 8 pixel per cycle mode. Table 14: Unsupported Functions Using 128-bit Interfaces in 8 Pixel Per Cycle Mode on Zynq-7000 SoC Computations Accumulate Input Processing Bit Depth Conversion Accumulate Squared UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide Filters Box Filter: signed 16-bit pixel type, and unsigned 16-bit pixel type Custom Convolution: signed 16-bit output pixel type www.xilinx.com [placeholder text] 34 Send Feedback Chapter 3: xfOpenCV Library API Reference Table 14: Unsupported Functions Using 128-bit Interfaces in 8 Pixel Per Cycle Mode on Zynq-7000 SoC (cont'd) Computations Input Processing Filters Accumulate Weighted Sobel Filter Gradient Magnitude Scharr Filter Gradient Phase Pixel-Wise Addition: signed 16-bit pixel type, and unsigned 16-bit pixel type Pixel-Wise Multiplication: signed 16-bit pixel type, and unsigned 16-bit pixel type Pixel-Wise Subtraction: signed 16-bit pixel type, and unsigned 16-bit pixel type Note: Resolution Conversion (Resize) in 8 pixel per cycle mode, Dense Pyramidal LK Optical Flow, and Dense Non-Pyramidal LK Optical Flow functions are not supported on the Zynq-7000 SoC ZC702 devices, due to the higher resource utilization. Absolute Difference The absdiff function finds the pixel wise absolute difference between two input images and returns an output image. The input and the output images must be the XF_8UC1 type. I out (x, y) = | I in1 (x, y) - I in2(x, y)| Where, • Iout(x, y) is the intensity of output image at (x,y) position. • Iin1(x, y) is the intensity of first input image at (x,y) position. • Iin2(x, y) is the intensity of second input image at (x,y) position. API Syntax template void absdiff( xf::Mat src1, xf::Mat src2, xf::Mat dst ) Parameter Descriptions The following table describes the template and the function parameters. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 35 Send Feedback Chapter 3: xfOpenCV Library API Reference Table 15: absdiff Function Parameter Descriptions Parameter Description SRC_T Input pixel type. Only 8-bit, unsigned, 1 channel is supported (XF_8UC1) ROWS Maximum height of input and output image (must be a multiple of 8) COLS Maximum width of input and output image (must be a multiple of 8) NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 and XF_NPPC8 for 1 pixel and 8 pixel operations respectively. src1 Input image src2 Input image dst Output image Resource Utilization The following table summarizes the resource utilization in different configurations, generated using Vivado® HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image. Table 16: absdiff Function Resource Utilization Summary Operating Mode Utilization Estimate Operating Frequency (MHz) BRAM_18K DSP_48Es FF LUT CLB 1 pixel 300 0 0 62 67 17 8 pixel 150 0 0 67 234 39 Performance Estimate The following table summarizes the performance in different configurations, as generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1, to process a grayscale HD (1080x1920) image. Table 17: absdiff Function Performance Estimate Summary Latency Estimate Operating Mode Max Latency (ms) 1 pixel operation (300 MHz) 6.9 8 pixel operation (150 MHz) 1.69 Deviation from OpenCV There is no deviation from OpenCV, except that the absdiff function supports 8-bit pixels. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 36 Send Feedback Chapter 3: xfOpenCV Library API Reference Accumulate The accumulate function adds an image (src1) to the accumulator image (src2), and generates the accumulated result image (dst). dst(x, y) = src1(x, y) + src2⎛⎝x, y⎞⎠ API Syntax template void accumulate ( xf::Mat src1, xf::Mat src2, xf::Mat dst ) Parameter Descriptions The following table describes the template and the function parameters. Table 18: accumulate Function Parameter Descriptions Parameter Description SRC_T Input pixel type. Only 8-bit, unsigned, 1 channel is supported (XF_8UC1) DST_T Output pixel type. Only 16-bit, unsigned, 1 channel is supported (XF_16UC1) ROWS Maximum height of input and output image (must be a multiple of 8) COLS Maximum width of input and output image (must be a multiple of 8) NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 and XF_NPPC8 for 1 pixel and 8 pixel operations respectively. src1 Input image src2 Input image dst Output image Resource Utilization The following table summarizes the resource utilization in different configurations, generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1, to process a grayscale HD (1080x1920) image. Table 19: accumulate Function Resource Utilization Summary Operating Mode Utilization Estimate Operating Frequency (MHz) BRAM_18K DSP_48E FF LUT CLB 1 pixel 300 0 0 62 55 12 8 pixel 150 0 0 389 285 61 UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 37 Send Feedback Chapter 3: xfOpenCV Library API Reference Performance Estimate The following table summarizes the performance in different configurations, as generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1, to process a grayscale HD (1080x1920) image. Table 20: accumulate Function Performance Estimate Summary Latency Estimate Operating Mode Max Latency (ms) 1 pixel operation (300 MHz) 6.9 8 pixel operation (150 MHz) 1.7 Deviation from OpenCV In OpenCV the accumulated image is stored in the second input image. The src2 image acts as both input and output, as shown below: src2(x, y) = src1(x, y) + src2⎛⎝x, y⎞⎠ Whereas, in the xfOpenCV implementation, the accumulated image is stored separately, as shown below: dst(x, y) = src1(x, y) + src2⎛⎝x, y⎞⎠ Accumulate Squared The accumulateSquare function adds the square of an image (src1) to the accumulator image (src2) and generates the accumulated result (dst). dst(x, y) = src1(x, y) 2 + src2⎛⎝x, y⎞⎠ The accumulated result is a separate argument in the function, instead of having src2 as the accumulated result. In this implementation, having a bi-directional accumulator is not possible as the function makes use of streams. API Syntax template void accumulateSquare ( xf::Mat src1, xf::Mat src2, xf::Mat dst) UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 38 Send Feedback Chapter 3: xfOpenCV Library API Reference Parameter Descriptions The following table describes the template and the function parameters. Table 21: accumulateSquare Function Parameter Descriptions Parameter Description SRC_T Input pixel type. Only 8-bit, unsigned, 1 channel is supported (XF_8UC1) DST_T Output pixel type. Only 16-bit, unsigned, 1 channel is supported (XF_16UC1) ROWS Maximum height of input and output image (must be a multiple of 8) COLS Maximum width of input and output image (must be a multiple of 8) NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 and XF_NPPC8 for 1 pixel and 8 pixel operations respectively. src1 Input image src2 Input image dst Output image Resource Utilization The following table summarizes the resource utilization in different configurations, generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image. Table 22: accumulateSquare Function Resource Utilization Summary Operating Mode Utilization Estimate Operating Frequency (MHz) BRAM_18K DSP_48E FF LUT CLB 1 pixel 300 0 1 71 52 14 8 pixel 150 0 8 401 247 48 Performance Estimate The following table summarizes the performance in different configurations, as generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1, to process a grayscale HD (1080x1920) image. Table 23: accumulateSquare Function Performance Estimate Summary Latency Estimate Operating Mode Max Latency (ms) 1 pixel operation (300 MHz) 6.9 8 pixel operation (150 MHz) 1.6 UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 39 Send Feedback Chapter 3: xfOpenCV Library API Reference Deviation from OpenCV In OpenCV the accumulated squared image is stored in the second input image. The src2 image acts as input as well as output. src2(x, y) = src1(x, y) 2 + src2⎛⎝x, y⎞⎠ Whereas, in the xfOpenCV implementation, the accumulated squared image is stored separately. ⎞ ⎛ 2 dst(x, y) = src1(x, y) + src2⎝x, y⎠ Accumulate Weighted The accumulateWeighted function computes the weighted sum of the input image (src1) and the accumulator image (src2) and generates the result in dst. dst(x, y) = alpha*src1(x, y) + ⎛⎝1 - alpha⎞⎠*src2⎛⎝x, y⎞⎠ The accumulated result is a separate argument in the function, instead of having src2 as the accumulated result. In this implementation, having a bi-directional accumulator is not possible, as the function uses streams. API Syntax template void accumulateWeighted ( xf::Mat src1, xf::Mat src2, xf::Mat dst, float alpha ) Parameter Descriptions The following table describes the template and the function parameters. Table 24: accumulateWeighted Function Parameter Descriptions Parameter Description SRC_T Input pixel type. Only 8-bit, unsigned, 1 channel is supported (XF_8UC1) DST_T Output pixel type. Only 16-bit, unsigned, 1 channel is supported (XF_16UC1) ROWS Maximum height of input and output image (must be a multiple of 8) COLS Maximum width of input and output image (must be a multiple of 8) NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 and XF_NPPC8 for 1 pixel and 8 pixel operations respectively. src1 Input image src2 Input image UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 40 Send Feedback Chapter 3: xfOpenCV Library API Reference Table 24: accumulateWeighted Function Parameter Descriptions (cont'd) Parameter Description dst Output image alpha Weight applied to input image Resource Utilization The following table summarizes the resource utilization in different configurations, generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image. Table 25: accumulateWeighted Function Resource Utilization Summary Operating Mode Utilization Estimate Operating Frequency (MHz) BRAM_18K DSP_48Es FF LUT CLB 1 pixel 300 0 5 295 255 52 8 pixel 150 0 19 556 476 88 Performance Estimate The following table summarizes the performance in different configurations, as generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1, to process a grayscale HD (1080x1920) image. Table 26: accumulateWeighted Function Performance Estimate Summary Latency Estimate Operating Mode Max Latency (ms) 1 pixel operation (300 MHz) 6.9 8 pixel operation (150 MHz) 1.7 Deviation from OpenCV The resultant image in OpenCV is stored in the second input image. The src2 image acts as input as well as output, as shown below: src2(x, y) = alpha*src1(x, y) + ⎛⎝1 - alpha⎞⎠*src2⎛⎝x, y⎞⎠ Whereas, in xfOpenCV implementation, the accumulated weighted image is stored separately. dst(x, y) = alpha*src1(x, y) + ⎛⎝1 - alpha⎞⎠*src2⎛⎝x, y⎞⎠ UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 41 Send Feedback Chapter 3: xfOpenCV Library API Reference Bilateral Filter In general, any smoothing filter smoothens the image which will affect the edges of the image. To preserve the edges while smoothing, a bilateral filter can be used. In an analogous way as the Gaussian filter, the bilateral filter also considers the neighboring pixels with weights assigned to each of them. These weights have two components, the first of which is the same weighing used by the Gaussian filter. The second component takes into account the difference in the intensity between the neighboring pixels and the evaluated one. The bilateral filter applied on an image is: BF[I] p = 1 ∑ G σ s⎛⎝ ‖ p - q ‖ ⎞⎠G σ r⎛⎝ ‖ I p - I q ‖ ⎞⎠I q W p qϵS Where Wp = ∑ qϵS G σ s⎛⎝ ‖ p - q ‖ ⎞⎠G σ r⎛⎝ ‖ I p - I q ‖ ⎞⎠ and G σ is a gaussian filter with variance σ . ⎛ ⎞ - ⎝x 2 + y 2⎠ The gaussian filter is given by: G σ = e 2σ 2 API Syntax template void bilateralFilter ( xf::Mat src, xf::Mat dst, float sigma_space, float sigma_color ) Parameter Descriptions The following table describes the template and the function parameters. Table 27: bilateralFilter Function Parameter Descriptions Parameter FILTER_SIZE Description Filter size. Filter size of 3 (XF_FILTER_3X3), 5 (XF_FILTER_5X5) and 7 (XF_FILTER_7X7) are supported BORDER_TYPE Border type supported is XF_BORDER_CONSTANT TYPE Input and output pixel type. Only 8-bit, unsigned, 1 channel is supported (XF_8UC1) ROWS Maximum height of input and output image (must be a multiple of 8) COLS Maximum width of input and output image (must be a multiple of 8) UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 42 Send Feedback Chapter 3: xfOpenCV Library API Reference Table 27: bilateralFilter Function Parameter Descriptions (cont'd) Parameter Description NPC Number of pixels to be processed per cycle; this function supports only XF_NPPC1 or 1 pixel per cycle operations. src Input image dst Output image sigma_space Standard deviation of filter in spatial domain sigma_color Standard deviation of filter used in color space Resource Utilization The following table summarizes the resource utilization of the kernel in different configurations, generated using Vivado HLS 2018.2 version tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to progress a grayscale HD (1080x1920) image. Table 28: bilateralFilter Resource Utilization Summary Operating Mode 1 pixel Utilization Estimate Operating Frequency Filter Size (MHz) BRAM_18K DSP_48Es FF LUT 3x3 300 6 22 4934 4293 5x5 300 12 30 5481 4943 7x7 300 37 48 7084 6195 Performance Estimate The following table summarizes a performance estimate of the kernel in different configurations, as generated using Vivado HLS 2018.2 tool for Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image. Table 29: bilateralFilter Function Performance Estimate Summary Latency Estimate Operating Mode Filter Size 168 MHz Max (ms) 1 pixel 3x3 7.18 5x5 7.20 7x7 7.22 Deviation from OpenCV Unlike OpenCV, xfOpenCV only supports filter sizes of 3, 5 and 7. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 43 Send Feedback Chapter 3: xfOpenCV Library API Reference Bit Depth Conversion The convertTo function converts the input image bit depth to the required bit depth in the output image. API Syntax template void convertTo(xf::Mat &_src_mat, xf::Mat &_dst_mat, ap_uint<4> _convert_type, int _shift) Parameter Descriptions The following table describes the template and the function parameters. Table 30: convertTo Function Parameter Descriptions Parameter SRC_T Description Input pixel type. 8-bit, unsigned, 1 channel (XF_8UC1), 16-bit, unsigned, 1 channel (XF_16UC1), 16-bit, signed, 1 channel (XF_16SC1), 32-bit, unsigned, 1 channel (XF_32UC1) 32-bit, signed, 1 channel (XF_32SC1) are supported. DST_T Output pixel yype. 8-bit, unsigned, 1 channel (XF_8UC1), 16-bit, unsigned, 1 channel (XF_16UC1), 16-bit, signed, 1 channel (XF_16SC1), 32-bit, unsigned, 1 channel (XF_32UC1) 32-bit, signed, 1 channel (XF_32SC1) are supported. ROWS Height of input and output images COLS Width of input and output images NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 and XF_NPPC8 for 1 pixel and 8 pixel operations respectively. _src_mat Input image _dst_mat Output image _convert_type This parameter specifies the type of conversion required. (See XF_convert_bit_depth_e enumerated type in file xf_params.h for possible values.) _shift Optional scale factor Possible Conversions The following table summarizes supported conversions. The rows are possible input image bit depths and the columns are corresponding possible output image bit depths (U=unsigned, S=signed). UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 44 Send Feedback Chapter 3: xfOpenCV Library API Reference Table 31: convertTo Function Supported Conversions INPUT/ OUTPUT U8 U16 S16 U32 S32 U8 NA yes yes NA yes U16 yes NA NA NA yes S16 yes NA NA NA yes U32 NA NA NA NA NA S32 yes yes yes NA NA Resource Utilization The following table summarizes the resource utilization of the convertTo function, generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image. Table 32: convertTo Function Resource Utilization Summary For XF_CONVERT_8U_TO_16S Conversion Operating Mode Utilization Estimate Operating Frequency BRAM_18K (MHz) DSP_48Es FF LUT CLB 1 pixel 300 0 8 581 523 119 8 pixel 150 0 8 963 1446 290 Table 33: convertTo Function Resource Utilization Summary For XF_CONVERT_16U_TO_8U Conversion Operating Mode Utilization Estimate Operating Frequency BRAM_18K (MHz) DSP_48Es FF LUT CLB 1 pixel 300 0 8 591 541 124 8 pixel 150 0 8 915 1500 308 Performance Estimate The following table summarizes the performance in different configurations, as generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1, to process a grayscale HD (1080x1920) image. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 45 Send Feedback Chapter 3: xfOpenCV Library API Reference Table 34: convertTo Function Performance Estimate Summary Latency Estimate Operating Mode Max Latency 1 pixel operation (300 MHz) 6.91 ms 8 pixel operation (150 MHz) 1.69 ms Bitwise AND The bitwise_and function performs the bitwise AND operation for each pixel between two input images, and returns an output image. I out⎛⎝x, y⎞⎠ = I in1⎛⎝x, y⎠⎞ & I in2⎛⎝x, y⎞⎠ Where, • I out⎛⎝x, y⎞⎠ is the intensity of output image at (x, y) position • I in1⎛⎝x, y⎞⎠ is the intensity of first input image at (x, y) position • I in2⎛⎝x, y⎞⎠ is the intensity of second input image at (x, y) position API Syntax template void bitwise_and ( xf::Mat src1, xf::Mat src2, xf::Mat dst ) Parameter Descriptions The following table describes the template and the function parameters. Table 35: bitwise_and Function Parameter Descriptions Parameter Description SRC_T Input and output pixel type. Only 8-bit, unsigned, 1 channel is supported (XF_8UC1) ROWS Maximum height of input and output image (must be a multiple of 8) COLS Maximum width of input and output image (must be a multiple of 8) NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 and XF_NPPC8 for 1 pixel and 8 pixel operations respectively. src1 Input image src2 Input image UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 46 Send Feedback Chapter 3: xfOpenCV Library API Reference Table 35: bitwise_and Function Parameter Descriptions (cont'd) Parameter Description dst Output image Resource Utilization The following table summarizes the resource utilization in different configurations, generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image. Table 36: bitwise_and Function Resource Utilization Summary Operating Mode Utilization Estimate Operating Frequency (MHz) BRAM_18K DSP_48Es FF LUT CLB 1 pixel 300 0 0 62 44 10 8 pixel 150 0 0 59 72 13 Performance Estimate The following table summarizes the performance in different configurations, as generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1, to process a grayscale HD (1080x1920) image. Table 37: bitwise_and Function Performance Estimate Summary Latency Estimate Operating Mode Max Latency (ms) 1 pixel operation (300 MHz) 6.9 8 pixel operation (150 MHz) 1.7 Bitwise NOT The bitwise_not function performs the pixel wise bitwise NOT operation for the pixels in the input image, and returns an output image. I out⎛⎝x, y⎞⎠ = ~ I in⎛⎝x, y⎞⎠ Where, • I out⎛⎝x, y⎞⎠ • I in⎛⎝x, y⎞⎠ is the intensity of output image at (x, y) position is the intensity of input image at (x, y) position UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 47 Send Feedback Chapter 3: xfOpenCV Library API Reference API Syntax template void bitwise_not ( xf::Mat src, xf::Mat dst ) Parameter Descriptions The following table describes the template and the function parameters. Table 38: bitwise_not Function Parameter Descriptions Parameter Description SRC_T Input and output pixel type. Only 8-bit, unsigned, 1 channel is supported (XF_8UC1) ROWS Maximum height of input and output image (must be a multiple of 8) COLS Maximum width of input and output image (must be a multiple of 8) NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 and XF_NPPC8 for 1 pixel and 8 pixel operations respectively. src Input image dst Output image Resource Utilization The following table summarizes the resource utilization in different configurations, generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image. Table 39: bitwise_not Function Resource Utilization Summary Operating Mode Utilization Estimate Operating Frequency (MHz) BRAM_18K DSP_48Es FF LUT CLB 1 pixel 300 0 0 97 78 20 8 pixel 150 0 0 88 97 21 Performance Estimate The following table summarizes the performance in different configurations, as generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1, to process a grayscale HD (1080x1920) image. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 48 Send Feedback Chapter 3: xfOpenCV Library API Reference Table 40: bitwise_not Function Performance Estimate Summary Latency Estimate Operating Mode Max Latency (ms) 1 pixel operation (300 MHz) 6.9 8 pixel operation (150 MHz) 1.7 Bitwise OR The bitwise_or function performs the pixel wise bitwise OR operation between two input images, and returns an output image. I out (x, y) = I in1 (x, y) | I in2⎛⎝x, y⎞⎠ Where, • I out(x, y) is the intensity of output image at (x, y) position • I in1(x, y) is the intensity of first input image at (x, y) position • I in2(x, y) is the intensity of second input image at (x, y) position API Syntax template void bitwise_or ( xf::Mat src1, xf::Mat src2, xf::Mat dst ) Parameter Descriptions The following table describes the template and the function parameters. Table 41: bitwise_or Function Parameter Descriptions Parameter Description SRC_T Input and output pixel type. Only 8-bit, unsigned, 1 channel is supported (XF_8UC1) ROWS Maximum height of input and output image (must be a multiple of 8) COLS Maximum width of input and output image (must be a multiple of 8) NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 and XF_NPPC8 for 1 pixel and 8 pixel operations respectively. src1 Input image src2 Input image dst Output image UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 49 Send Feedback Chapter 3: xfOpenCV Library API Reference Resource Utilization The following table summarizes the resource utilization in different configurations, generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image. Table 42: bitwise_or Function Resource Utilization Summary Operating Mode Utilization Estimate Operating Frequency (MHz) BRAM_18K DSP_48Es FF LUT CLB 1 pixel 300 0 0 62 44 10 8 pixel 150 0 0 59 72 13 Performance Estimate The following table summarizes the performance in different configurations, as generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1, to process a grayscale HD (1080x1920) image. Table 43: bitwise_or Function Performance Estimate Summary Latency Estimate Operating Mode Max Latency (ms) 1 pixel operation (300 MHz) 6.9 8 pixel operation (150 MHz) 1.7 Bitwise XOR The bitwise_xor function performs the pixel wise bitwise XOR operation between two input images, and returns an output image, as shown below: I out⎛⎝x, y⎞⎠ = I in1⎛⎝x, y⎞⎠ ⊕ I in2⎛⎝x, y⎞⎠ Where, • I out⎛⎝x, y⎞⎠ is the intensity of output image at (x, y) position • I in1⎛⎝x, y⎞⎠ is the intensity of first input image at (x, y) position • I in2⎛⎝x, y⎞⎠ is the intensity of second input image at (x, y) position UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 50 Send Feedback Chapter 3: xfOpenCV Library API Reference API Syntax template void bitwise_xor( xf::Mat src1, xf::Mat src2, xf::Mat dst ) Parameter Descriptions The following table describes the template and the function parameters. Table 44: bitwise_xor Function Parameter Descriptions Parameter Description SRC_T Input and output pixel type. Only 8-bit, unsigned, 1 channel is supported (XF_8UC1) ROWS Maximum height of input and output image (must be a multiple of 8) COLS Maximum width of input and output image (must be a multiple of 8) NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 and XF_NPPC8 for 1 pixel and 8 pixel operations respectively. src1 Input image src2 Input image dst Output image Resource Utilization The following table summarizes the resource utilization in different configurations, generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image: Table 45: bitwise_xor Function Resource Utilization Summary Operating Mode Utilization Estimate Operating Frequency (MHz) BRAM_18K DSP_48Es FF LUT CLB 1 pixel 300 0 0 62 44 10 8 pixel 150 0 0 59 72 13 Performance Estimate The following table summarizes the performance in different configurations, as generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1, to process a grayscale HD (1080x1920) image: UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 51 Send Feedback Chapter 3: xfOpenCV Library API Reference Table 46: bitwise_xor Function Performance Estimate Summary Latency Estimate Operating Mode Max Latency (ms) 1 pixel operation (300 MHz) 6.9 8 pixel operation (150 MHz) 1.7 Box Filter The boxFilter function performs box filtering on the input image. Box filter acts as a low-pass filter and performs blurring over the image. The boxFilter function or the box blur is a spatial domain linear filter in which each pixel in the resulting image has a value equal to the average value of the neighboring pixels in the image. ⎡1 . . . 1⎤ 1 ⎢1 . . . 1⎥ K box = (ksize*ksize) ⎣ 1 . . . 1⎦ API Syntax template void boxFilter(xf::Mat & _src_mat,xf::Mat & _dst_mat) Parameter Descriptions The following table describes the template and the function parameters. Table 47: boxFilter Function Parameter Descriptions Parameter Description FILTER_SIZE Filter size. Filter size of 3(XF_FILTER_3X3), 5(XF_FILTER_5X5) and 7(XF_FILTER_7X7) are supported BORDER_TYPE Border Type supported is XF_BORDER_CONSTANT SRC_T Input and output pixel type. 8-bit, unsigned, 16-bit unsigned and 16-bit signed, 1 channel is supported (XF_8UC1) ROWS Maximum height of input and output image (must be a multiple of 8) COLS Maximum width of input and output image (must be a multiple of 8) NPC Number of pixels to be processed per cycle; possible options are XF_NPPC1 and XF_NPPC8 for 1 pixel and 8 pixel operations respectively. _src_mat Input image _dst_mat Output image UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 52 Send Feedback Chapter 3: xfOpenCV Library API Reference Resource Utilization The following table summarizes the resource utilization of the kernel in different configurations, generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1 FPGA, to process a grayscale HD (1080x1920) image. Table 48: boxFilter Function Resource Utilization Summary Operating Mode 1 pixel 8 pixel Filter Size Utilization Estimate Operating Frequency (MHz) BRAM_18K DSP_48Es FF LUT CLB 3x3 300 3 1 545 519 104 5x5 300 5 1 876 870 189 7x7 300 7 1 1539 1506 300 3x3 150 6 8 1002 1368 264 5x5 150 10 8 1576 3183 611 7x7 150 14 8 2414 5018 942 Performance Estimate The following table summarizes the performance of the kernel in different configurations, as generated using Vivado HLS 2018.2 tool for the Xilinx Xczu9eg-ffvb1156-1-i-es1, to process a grayscale HD (1080x1920) image: Table 49: boxFilter Function Performance Estimate Summary Operating Mode 1 pixel 8 pixel Latency Estimate Operating Frequency Filter Size (MHz) Max (ms) 300 3x3 7.2 300 5x5 7.21 300 7x7 7.22 150 3x3 1.7 150 5x5 1.7 150 7x7 1.7 Canny Edge Detection The Canny edge detector finds the edges in an image or video frame. It is one of the most popular algorithms for edge detection. Canny algorithm aims to satisfy three main criteria: 1. Low error rate: A good detection of only existent edges. UG1233 (v2018.2) July 2, 2018 Xilinx OpenCV User Guide www.xilinx.com [placeholder text] 53 Send Feedback Chapter 3: xfOpenCV Library API Reference 2. Good localization: The distance between edge pixels detected and real edge pixels have to be minimized. 3. Minimal response: Only one detector response per edge. In this algorithm, the noise in the image is reduced first by applying a Gaussian mask. The Gaussian mask used here is the average mask of size 3x3. Thereafter, gradients along x and y directions are computed using the Sobel gradient function. The gradients are used to compute the magnitude and phase of the pixels. The phase is quantized and the pixels are binned accordingly. Non-maximal suppression is applied on the pixels to remove the weaker edges. Edge tracing is applied on the remaining pixels to draw the edges on the image. In this algorithm, the canny up to non-maximal suppression is in one kernel and the edge linking module is in another kernel. After non-maxima suppression, the output is represented as 2-bit per pixel, Where: • 00 - represents the background • 01 - represents the weaker edge • 11 - represents the strong edge The output is packed as 8-bit (four 2-bit pixels) in 1 pixel per cycle operation and packed as 16bit (eight 2-bit pixels) in 8 pixel per cycle operation. For the edge linking module, the input is 64bit, such 32 pixels of 2-bit are packed into a 64-bit. The edge tracing is applied on the pixels and returns the edges in the image. API Syntax The API syntax for Canny is: template void Canny(xf::Mat & _src_mat,xf::Mat & _dst_mat,unsigned char _lowthreshold,unsigned char _highthreshold) The API syntax for EdgeTracing is: template voidEdgeTracing(xf::Mat