VisionSDK_UserGuide_2D_SurroundView_Demo Vision SDK User Guide 2D Surround View Demo

User Manual:

Open the PDF directly: View PDF .
Page Count: 27

Download
Open PDF In Browser	View PDF

Vision SDK
SurroundView Demo Set Up Guide
Information in this document is subject to change without notice. Texas Instruments may have pending
patent applications, trademarks, copyrights, or other intellectual property rights covering matter in this
document. The furnishing of this document is given for usage with Texas Instruments products only and
does not give you any license to the intellectual property that might be contained within this document.
Texas Instruments makes no implied or expressed warranties in this document and is not responsible for
the products based from this document

Page 1 of 27

TABLE OF CONTENTS
1

Introduction ................................................................................................. 3
1.1

Purpose & Scope .................................................................................... 3

1.2

Overview............................................................................................... 4

1.2.1 TDA3xx Platform .................................................................................... 5
2

Demo Set up ................................................................................................. 6
2.1

Cameras to be used in TDA2xx platform ................................................... 6

2.2

Cameras to be used in TDA3xx platform ................................................... 6

2.3

Mounting of cameras .............................................................................. 6

2.4

Sample Non-calibrated output.................................................................. 8

2.5

A Brand-new Calibration ......................................................................... 9

2.5.1 Obtaining Surround View Images for the given set up ................................. 9
2.5.2 Contact TI FAE / Support ...................................................................... 12
2.5.3 Using new calibration tables .................................................................. 12
2.6
3

Re-Calibrating Existing Demo Set-up ...................................................... 13

Calibration Tool .......................................................................................... 14
3.1

Purpose for using this tool ..................................................................... 14

3.2

Prerequisites........................................................................................ 14

3.3

Input files required by the tool ............................................................... 14

3.3.1 Obtain four surround view calibration frames ........................................... 14
3.3.2 Obtain bird-eye reference image ............................................................ 14
3.4
4

Tool Instruction.................................................................................... 14

Fish-eye distortion center tool ................................................................... 21
4.1

Purpose for using this tool ..................................................................... 21

4.2

Prerequisites........................................................................................ 21

4.3

Input files required by the tool ............................................................... 22

4.4

Tool Instruction.................................................................................... 22

5

Update Vision SDK with New Initial Perspective Matrix and Fish-eye
Distortion Centers ...................................................................................... 24

6

Revision History ......................................................................................... 26

Page 2 of 27

1

Introduction
This document describes Surround View Demo (SRV) environment requirements and
steps to configure / caliber this demo. SRV is supported as part of demo application
in VisionSDK. SRV on TDA2xx platform is support on VisionSDK versioned
v02.02.00.00 onwards & SRV on TDA3xx platform is supported on VisionSDK
versioned v02.07.00.00 onwards.

1.1

Purpose & Scope
This document details the environment and calibration procedures required for
Surround view + Front camera analytics demo.
This document assumes that the audiences of this document are familiar with
VisionSDK build procedure, board configurations, required hardware (such as EVM,
power supply) etc…
Terms & Abbreviations
SDK

Software development kit.

SRV

Surround View

Page 3 of 27

1.2

Overview
Vision SDK (release v02.02.00.00 onwards) has a demo option for Surround View +
Front camera analytics on TDA2xx platform. This demo or use case uses five
cameras connected via LVDS to TDA2xx based EVM, stich the output video streams
to provide surround view. Output of this demo can be seen on the HDMI display. For
more details on LVDS and HDMI connections, please refer user guide.
Four cameras are mounted on four sides of the car. Video feeds from these cameras
are used to generate the Surround View output.
Fifth camera is placed on front of the car. Analytics (Edge detection) is performed on
the video feed from this camera.
Display layout consists of following:
1. Original feeds from four surround view cameras
2. Generated surround view image
3. Original feed of the front analytics camera
4. Edge detector output
5. Graphics comprising of demo banners, loading of processors
Sample display output is as follows:

Figure 1: Sample Output Display

Page 4 of 27

1.2.1

TDA3xx Platform
Very similar to TDA2xx platform but the “front camera analytics” is not supported.
The hardware required is different for TDA3xx platform, as compared to TDA2xx.
Please refer the TDA3xx user guide for details.
Below is a snap shot of the SRV on TDA3xx

Page 5 of 27

2

Demo Set up
This section provides specific details of surround view demo set up.

2.1

Cameras to be used in TDA2xx platform
Four surround view cameras to be used need to have Fish Eye lens (Which have
about 180 degrees of view). Fish eye lens to be used is of type DSL219E-670-F2.0 as
shown in below diagram. Cameras to be used are of type OV10635.

Figure 2 - Fish Eye Lens camera
Front analytics camera can be of a non-fish eye lens camera (OV10635-EAAE-AA0A),
as shown below

Figure 3 - Simple (Non fish eye) Lens camera

2.2

Cameras to be used in TDA3xx platform
Please refer the TDA3xx user guide for details

2.3

Mounting of cameras
Dimensions of the car (rectangular cube) are as follows - 9 inch width, 21 inch length
and about 6 inch height.
Four surround view cameras are mounted at centre of each of the four sides / edges
of top of car cube as shown below. Cameras need to be facing ground plane. Hence
cameras need to have an inclination of about 45 degrees.
Mapping of surround view cameras to camera numbering mentioned in user guide is
as follows:
Page 6 of 27

Front Camera – Camera 1
Right Camera - Camera 2
Rear Camera - Camera 3
Left Camera - Camera 4
On TDA2xx platform, front analytics camera is placed on the front side in such a way
that its view is not blocked by the car or other camera.

Front Camera
(Cam 1)

Left
Camera
(Cam 4)

Front Analytics
Camera (Cam 5)

Right Camera
(Cam 2)

Rear Camera
(Cam 3)

Figure 4 : Sample mounting of cameras

Note that the cameras need to be mounted firmly, so that during usage, camera
positions do not get altered.

Page 7 of 27

For general development purposes, a car is not needed. Cameras can be mounted on
any rectangular cubic setup / cubic arrangement as follows. However height, width,
length and angle of mounting need to be adhered to.
21 inches
Front Analytics
Camera (Cam 5)

9 inches

Front Camera
(Cam 1)

6 inches
6 inches

Right Camera
(Cam 2)

Left and rear cameras are mounted on other two sides of the cube (Hidden in above
figure).

2.4

Sample Non-calibrated output
Surround view output generation use certain geometric mapping tables and the
output quality is sensitive to these table values. These table values need to be
calibrated based on the actual given set up. Vision SDK release has certain default
tables, which will give an output which is not best aligned. Below is a sample of noncalibrate output. Note that, along the diagonals of the surround view output, images
from two views do not have smooth transition.

Page 8 of 27

Figure 5 - Sample output without calibration

2.5

A Brand-new Calibration
To obtain best quality surround view output, proper geometric calibration tables for
the given demo set up need to be used.
When the demo is set-up for the very first time, calibration images may need to be
extracted from the EVM to obtain correct geometric calibration table. This involves
following two steps:
Step1: Obtain surround view images for the given set up.
Step2: Share images with TI FAE / Contact to receive calibration tables.
Step3: Using new calibration tables

2.5.1

Obtaining Surround View Images for the given set up
1. Four print outs of the below checkered pattern needs to be taken in an A4 sheet
size paper. You can use either one of the patterns shown below.

Page 9 of 27

2. These checkered print outs need to be placed around the car as shown below.

(a)

(b)

Figure 6 - Positioning of calibration charts. (a) An example of the
reference picture which shows how charts are placed around the jeep. (b)
Further illustration of how charts should be placed. The black box (i.e., the car
box) in the middle of the image is where the graphics of the jeep will appear. The
car box should encompass surround view cameras inside (i.e., the orange
circles). From the car box, green dotted lines are drawn to show the overlapping
regions between adjacent cameras. The red lines are the diagonal lines of each
overlapping region. Each calibration chart (blue squares) should be placed inside
the overlapping region, roughly symmetrical against the red diagonal lines.
Please follow steps 3-5 listed below for additional calibration requirement.

3. Important to keep all these checkered patterns on a non-reflective surface, with
uniform lighting. Note that the demo should be set-up on non-reflective surface.
Glares from the floor will cause problems for both geometric alignment and
photometric alignment.
4. Make sure no wires or cables go through the charts, and people are not standing
too close to the charts.
5. Another thing to note is that each camera must clearly capture all the corners of
the two checkered patterns coming in its view as below.

Page 10 of 27

Figure 7 - Each view capturing two charts fully

6. These images need to be dumped out from DDR memory, while running the
demo, using following steps:
a. Build code of IPU1_0 in Debug mode (Control via Rules.make)
b. Run the demo in the usual manner
c. Insert a break point on IPU1_0 CPU at return statement inside function
SyncLink_makeCompositeBuffer
()
present
in
file
\VISION_SDK_02_02_xx_xx\vision_sdk\src\links_common\sync\syncLink
_tsk.c
d. Upon hitting break point, drag pSysCompBuf to watch window of CCS
e. Check the value of pSysCompBuf ->numFrames. If it’s not 4, then just
give free run until it hits this break point again with value of pSysCompBuf
->numFrames being 4.
f.

Once value of pSysCompBuf ->numFrames is 4, then record four pointers
pSysCompBuf
->bufAddr[0][0],
pSysCompBuf
->bufAddr[0][1],
pSysCompBuf ->bufAddr[0][2] and pSysCompBuf ->bufAddr[0][3]

g. Above four pointers give addresses in DDR where the images from four
surround view cameras are stored in YUV420 Semi-planar format.
h. From these addresses take a memory dump in RAW format using CCS
save data option. Number of bytes to be dumped out = (1280*720*1.5) =
1382400 bytes. Select 8-bit as dumping mode and file dump needs to be
saved in *.bin format.
i.

Just open above dumps in any YUV viewer so as to make sure that the
dumped out content is the image from the four cameras.
OR

Vision SDK ver 2.10 onwards supports Surround View Calibration Use Case for
both VIP Multi capture (TDA2XX) and ISS Multi capture (TDA3XX) which has
option to dump the captured YUV frames to MMC/SD card.
Page 11 of 27

a. Run the Surround View Calibration Use case (option ‘6’)
b. Select the Manual Calibration (option ‘2’)
c. Save the ISP output frames to MMC/SD card (option ‘1’).
7. Either before or after dumping the frames, capture a reference bird-eye view
image with a camera for your current setting. When taking the reference, not
only camera positions should not change, charts positions and jeep position
should all remain the same. You can take the reference image with a camera, try
to give the camera a perfect bird-eye view of the jeep such that all the charts
have equal sizes in the reference. One example is shown in Fig. 6.
8. In case of TIDA00455/OV490 based surround view use-case, if video contains
green artifacts throughout the image, try toggling the pixel polarity.
In case of BIOS setup, run-time menu provides an option to toggle the polarity
while running the use-case.
In case of Linux setup, use the following command - “omapconf write
0x4A002534 0x0”. To revert use “omapconf write 0x4A002534 0x5”. 0x5 is the
recommended setting – it might vary based on different OV490 firmwares/board
versions/etc.
This
command
can
be
run
before
running
vision_sdk_linux_demo.out or while the use-case is running from another
terminal by connecting via telnet.
2.5.2

Contact TI FAE / Support

Share the above four frames as well as the reference bird-eye image with TI FAE
/ Contact to obtain geometric tables for the given demo set up.
More details on how to obtain initial geometric tables using calibration tool is
given in Section 3.

2.5.3

Using new calibration tables

1. In
the
file
\vision_sdk\examples\tda2xx\src\alg_plugins\surroundview\GAlignExt_tda2xx
.c replace the array permatin[] with the newly provided geometric tables and
in
file
\vision_sdk\examples\tda2xx\src\alg_plugins\surroundview\GAlignLUT_tda2x
x.c replace the array GAlignLut []. Rebuild the Vision SDK code with above
change.
2. On
TDA3xx
platform,
update
following
files\vision_sdk\examples\tda2xx\src\alg_plugins\surroundview\GAlignExt_td
a3xx.c replace the array permatin[] with the newly provided geometric tables
and
in
file
\vision_sdk\examples\tda2xx\src\alg_plugins\surroundview\GAlignLUT_tda3x
x.c
3. Run the demo and select option “2: Erase entire calibration tbl from flash”
- Above will erase old tables, if any in QSPI flash.
4. And then in the next run, pick option “Run with GA LUT from flash (If not
available, use default table)”
Page 12 of 27

- Above will use the tables included in the code, since there are no tables in
flash.
After using the correct geometric tables, surround view output should not have
any geometric misalignments along the diagonals (common view of two cameras)
as shown below. Note that the checkered patterns do not have any distortion.

Figure 8 - Output with perfect geometric alignments.

2.6

Re-Calibrating Existing Demo Set-up
After the current set-up has been calibrated, try to keep all the cameras
positions as fixed as possible. If for any reason, one or more camera positions
are changed, and the output surround view becomes misaligned, then it
means the current surround view demo requires a re-calibration. This
normally can be achieved by the on-line calibration software.
To do that, first place the calibration charts on the floor around the demo setup following instructions given in Sec 2.4.1. Next, power-cycle the EVM. The
calibration will complete in a few seconds, and the fully aligned output will
shown on the screen. The correct new geometric alignment tables have been
obtained and saved automatically.
If on-line calibration fails, the output surround view will look misaligned. In
this case, first try to slightly adjust the position of the cameras by hand to
achieve calibration. If it does not work, then following “Step 6” in Sec. 2.4.1

Page 13 of 27

to extract the calibration images from EVM, and send calibration images to
FAE to get the new geometric alignment tables.

3

Calibration Tool

3.1

Purpose for using this tool
This interactive tool produces the initial perspective matrix for Geometric Alignment
algorithm using four input frames, one reference bird-eye image, and user selected
corners.

3.2

Prerequisites
Verify the MATLAB Compiler Runtime (MCR) is installed and ensure you have
installed version 8.4. MCR 8.4 can be obtained for free from the Mathworks website.
If the MCR is not installed, do the following:
(1) Run “MyAppInstaller_mcr.exe” to install or
(2) Run “MyAppInstaller_web.exe” to download it from the web

3.3

Input files required by the tool
This tool requires four surround view calibration frames and a bird-eye reference image.

3.3.1

Obtain four surround view calibration frames
Please follow instructions given in Sec. 2.4.1 step 1 – 6 to obtain the surround view
calibration frames.
Mapping of surround view cameras to camera numbering mentioned in user guide is
as follows:
Front Camera – Camera 1
Right Camera - Camera 2
Rear Camera - Camera 3
Left Camera - Camera 4

3.3.2

Obtain bird-eye reference image
Please follow instructions given in Sec. 2.4.1 step 7 to obtain the bird-eye reference
image. One example is shown in Fig. 6.

3.4

Tool Instruction
1. By running “calibration_tool.exe” in folder ./calibration_tool/, first you will
see the windows shown in Figure 9. It will ask you to select the .txt file
which contains the distortion center. For Jeep No. 1 – 8, the files have

Page 14 of 27

already been generated. For new assembled car, please refer to Sec. 4 to
generate new distortion center file.
2. Enter other parameters needed for calibration, as shown in Figure 10.
Please follow the instructions for entering the parameters. If you have
DSP code and you changed any of the default values for “Focal Length”,
“Output Height & Width”, “Car Box Size Horizontal & Vertical” or “Camera
Image Height, Width or Pitch”, change the value here, too. Otherwise,
leave default values.
In this interface, select the output directory to be the same folder as
where the input frames are located, and load input frames (yuv) and
reference image (jpg).
Reference image must be in portrait orientation. Rotate the image before
loading if it is not.

Figure 9 – Calibration Tool start window.

Figure 10 – Enter parameters.

Page 15 of 27

3. Crop the reference image
The purpose of cropping is to create a reference that more tightly encloses
the calibration charts (i.e., reducing field of view), as well as having the
jeep centered in the frame. This can ensure the best quality for surround
view output.
If you have cropped the reference image before, the coordinates will be
saved in your chosen output directory. The program will show you the
previous selected region of interest and ask whether if you would like to
use that data, as shown in Figure 11. You can hit “Continue” to use the
previous selection or hit “Reselect” to reselect the region of interest.

Figure 11 – Previous selected region of interest.
If you have not selected the region before or you choose to reselect the
region of interest, you will be asked to select the upper-left corner and the
lower-right corner of the desired cropping region using you cursor. This
interface is shown in Figure 12.

Page 16 of 27

Selected
upper-left
corner for
cropping

Selected
lower-right
corner for
cropping
Figure 12 – Crop reference image.
After successfully select the two points, you will get a message at the
bottom of the window as shown in Figure 13. If you are not satisfied with
the selection, you can follow the instruction and hit “Redo” to reselect the
points.

Page 17 of 27

Figure 13 – Successfully select the points to crop reference image. The
region with red borders will be the cropped reference image.
If you did not successfully select the points, you will get a message at the
bottom of the window as shown in Figure 14. Please follow the instructions
to reselect the points.

Page 18 of 27

Figure 14 – Cropping reference image failed, wrong number of points are
selected.
4. Select the corner positions in reference image as shown in Figure 15.
Please follow the instructions on the GUI, you will get a message
indicating whether you have succeeded to select the points as shown
before.

Page 19 of 27

Figure 15 – Select corner positions in reference image.
5. Select corners in each of the four input camera frames.
There are only two charts and a total of 8 corners to select for each input
camera frame. Start with the left chart, click on the upper left corner first
then go clock-wise to click on the rest of the three corners. An example is
shown in Fig. 16. After finishing the left chart, following the same
procedure to select the four corners in the right chart.
Repeat this process for all four input frames.

Page 20 of 27

Figure 16 – Select 8 corners in the input frame

6. Finally, you will get a message saying that you have successfully gone through the
calibration process. You will find two output files (.c and .csv) in your selected output
folder. The .csv file is needed for the SurroundView.exe demo PC code. XXX

Figure 17 – Calibration process finished message.

4

Fish-eye distortion center tool

4.1

Purpose for using this tool
This interactive tool estimates the fish-eye lens distortion centers for the four input
cameras. The results will be saved to an txt file, and subsequently used by the
Calibration Tool to generate initial perspective matrix.
Fish-eye distortion centers only need to be obtained once for a new set of
cameras/lens.

4.2

Prerequisites
Verify the MATLAB Compiler Runtime (MCR) version 8.4 is installed. MCR 8.4 can be
obtained for free from the Mathworks website.
Page 21 of 27

4.3

Input files required by the tool
This tool requires four surround view calibration frames. Please follow instructions
given in Sec. 2.4.1 step 1 – 6 to obtain the surround view calibration frames.
Mapping of surround view cameras to camera numbering mentioned in user guide is
as follows:
Front Camera – Camera 1
Right Camera - Camera 2
Rear Camera - Camera 3
Left Camera - Camera 4

4.4

Tool Instruction
1. By running “fisheye_tool.exe” in folder ./fisheye_tool/, first you will see
the windows shown in Figure 18. It will ask you to select input file, output
folder and output filename. Choose output folder to be the same as where
the camera images are located. Also, specify the height, width and pitch
of the input camera images, if they differ from the default values.

Figure 18 – Fisheye Tool starting window
2. Select the fish-eye view boundaries in each input frame
You will be asked to select points which represent the border of the fisheye image. The border means the boundary between the fish-eye image
and the black area around the four corners of the image. You can see an
example in Figure 19. You may select as many points as you want to
around the dark boundary encompassing the fish-eye view.
At the region where you are more confident about the dark boundaries,
you may click more points. At the region where you are less confident,
you may select fewer points.
You will repeat this process for all 4 input camera frames.

Page 22 of 27

Figure 19 – Select points along the border of fish-eye image, as
highlighted by the red circles
After selecting the points, hit “Enter” on your keyboard.
You can then hit “Next” to continue.
If you are not satisfied with the selections, you can hit “Redo” to redo the
selection as shown in Figure 20.
You will also see all the selected points and the estimated distortion center
of the frame (red cross) together with the frame center as a reference
(blue cross). You can see the distortion center may be different from the
geometric center of the frame. So it is very important to use the tool to
estimate the actual fish-eye distortion centers.

Page 23 of 27

Figure 20 – Selected points and distortion center
3. Finally, you will get a message saying that you have successfully gone
through the process. You will find two output files (.txt and .csv) in your
selected output folder. You can then use the .txt file for your
CalibrationTool to generate the initial perspective matrices as file. The .csv
file is needed for the SurroundView.exe demo PC code.

Figure 21 – DistortionCenterTool finished message

5

Update Vision SDK with New Initial Perspective Matrix and Fisheye Distortion Centers
After using the calibration tool described in Sec 3 and/or fish-eye distortion center
tool in Sec 4, please update the respective files under the vision_sdk folder
“.\vision_sdk\vision_sdk\examples\tda2xx\src\alg_plugins\surroundview\” as below.
1. After using Calibration
InitialPerspectiveParams.c.

tool

(Sec

3),

you

will

get

a

file

called

2. After using the Fish-eye distortion center tool (Sec 4), you will get a .txt file with
the distortion centers for all four cameras.

Page 24 of 27



For TDA2x, Open the file,
vision_sdk\examples\tda2xx\src\alg_plugins\surroundview\GAlignExt_tda
2xx.c. Update the persmatin[] & InCamFrmDistortionCen[] with the newly
generated persmatin & distortion centers values.



For TDA3x, Open the file,
\vision_sdk\examples\tda2xx\src\alg_plugins\surroundview\GAlignExt_tda
3xx.c. Update the persmatin[] & InCamFrmDistortionCen[] with the newly
generated persmatin & distortion centers values.

3. Save all changes, compile vision sdk.

Page 25 of 27

6

Revision History
Version #

Date

Revision History

00.10

28/Feb/2014

First draft

00.20

05/Mar/2014

Addressed
team

00.30

23/May/2014

Add manual on how to use calibration
tool to generate calibration

00.31

29/May/2014

Updated
Sec
2.4.1,
added
requirement for capturing a reference
bird-eye view in addition to dumping
four input frames

00.32

2/June/2014

Updated Sec 3 on the requirement for
input images for using the calibration
tool

00.33

3/June/2014

Updated Sec 4 on selecting input file
for distortion center

00.40

5/June/2014

Add DistortionCenterTool

00.41

10/June/2014

(1) updated manual

review

comments

from

(2) added Sec 5 for updating vision
sdk after obtaining new fish-eye
distortion
center
and
initial
perspective matrix
00.42

25/June/2014

Update reference image for checkered
print outs placement

00.43

17/July/2014

Updated both the calibration tool and
the manual accordingly
In the tool, added “car box sizes” as
input parameters to the tool

00.44

11/Aug/2014

Updated sec 2.4.1 on placing the
calibration charts

0.50

1/Oct/2014

Updated instructions for Sec. 3 and
Sec 4. to reflect that tools output two
files (.txt & .cvs in Sec 4, .c & .csv in
Sec. 3). Tools now require MCR 8.3,
not 8.2 (Mueller)

0.51

4/Feb/2015

Default values of carbox width and
height in calibration tool were
changed from 240 to 160 and from
360 to 320, respectively, to match
changes in VSDK code. No changes
needed in this document. (Mueller)

0.52

27/Apr/2015

Added

parameters

Camera

Image

Page 26 of 27

Height, Width and Pitch to Calibration
and Fisheye Tool GUIs and this
document. (Mueller)
0.53

20/Aug/2015

Corrected names of executables of
calibration tool (calibration_tool.exe)
and fisheye lens distortion correction
tool (fisheye_tool.exe) in Sections 3.4
and 4.4 (Mueller)

0.54

31/Aug/2016

Fixed a bug when cropping reference
image
in
2D
calibration
tool.
Upgraded calibration_tool.exe and
fisheye_tool.exe from MCR8.3 to
MCR8.4 (Mueller)

««« § »»»

Page 27 of 27



---

Source Exif Data:

File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.5
Linearized                      : No
Page Count                      : 27
Language                        : en-US
Tagged PDF                      : Yes
Title                           : VisionSDK_UserGuide_2D_SurroundView_Demo
Author                          : TI
Creator                         : Microsoft® Word 2010
Create Date                     : 2017:07:05 16:38:30+05:30
Modify Date                     : 2017:07:05 16:38:30+05:30
Producer                        : Microsoft® Word 2010

EXIF Metadata provided by EXIF.tools