Mapping Virtual And Physical Reality Guide For

User Manual:

Open the PDF directly: View PDF .
Page Count: 10

Download
Open PDF In Browser	View PDF

Mapping Virtual and Physical Reality
Qi Sun

Li-Yi Wei

Arie Kaufman

Guide by Swanand Sawant
Introduction:
As the name suggests this paper aims at mapping the virtual environment to a real
world environment for a more immersive virtual reality experience. In the current
Virtual reality hardware like HMD(Head Mounted Displays) a user usually walks
at the same place and thus is not completely immersive. This paper implements a
mapping system such that a virtual environment is mapped on to a real world
environment like a small room in a office and the user is able to move around the
room without hitting any walls or interior obstacles while immersed in the virtual
environment through the wireless HMD. Thus it has the potential for realistic
interaction and immersive presence. The problem faced in doing so is that
generally a virtual environment is very different or unrelated to the real world
environment and hence a general method to bridge this gap is crucial for a real
immersive VR navigation. They propose a method to render a proper
representation of the virtual environment inside the HMD but change the camera
projections for navigating within the boundaries of real world environment as well
as avoid any interior obstacles within it.
Figure 1.)Mapping of a) to b) and c) is the actual room setup.

The Method proposed here as two key components: 1) a planar map between
virtual world and real world and 2) a camera projection from the planar map and
scene content. This planar map aims to preserve both angle and distance between
both the worlds for visual and locomotive consistency. The camera rendering
preserves both the virtual world appearance and real world geometry, while
guiding user navigation to avoid physical obstacles and maintain the balance of the
user at the same time and thus in the end aim to derive their camera projection

according to the planar map and scene content to balance between virtual realism,
geometric consistency, and perceptual discomfort.
Figure 2: Left side is virtual world view and the right side is user’s HMD view.

The main contributions of the paper include:
1. An HMD-VR system that allows real walking in a given physical
environment while perceiving a given virtual environment.
2. A custom planar map that minimizes the angular and distal distortions for
walkthroughs.
3. Optimization methods to compute this planar map as two maps:
a. Static forward map that minimizes angular and distal distortions
between the virtual and real world.
b. A dynamic reverse map that guides natural locomotion and resolves
local ambiguities
4. And finally a rendering method to preserve the virtual world appearance
while observing the physical world geometry to balance visual fidelity and
navigational comfort.
Proposed Method
With the given input world’s floor plan for Sv (virtual) and Sr (real) scenes, first
compute a static forward map f from Sv to Sr. This map is surjective and not
bijective in general when Sv > Sr, but should also minimize angular and distal
distortions for VR walkthroughs. It should reach every point in Sr and Sv, while
keeping inside Sr and away from interior obstacles. As the size of the both worlds
is different generally (Sv>Sr), folding is introduced tearing or breaking the virtual
world appart.
At runtime we compute the dynamic reverse map of f to place the user at proper
location in the virtual world (Sv) inside the HMD based on the user’s current
position in the real-world(Sr). This reverese mapping should be consistent with the
forward map f while maintaining motion and perception consistency for the user.

The last step is to render the virtual scene into the HMD which should have enough
quality and speed, and fit the appearance of the virtual scene into the geometry of
the real scene to balance between visual and motion fidelity.
Static Forward Mapping
This forward mapping proposed relies on conformality and isometry to minimize
angular and distal distortion during VR navigation. It does not require global
bijectivity (one to one correspondence) to allow proper folding of large virtual
scenes into small real scenes.
Figure 3. Static sampling example

The goal of this step is to surjectively map each virtual scene pixel
𝑥 = (𝑥, 𝑦) ∈ 𝑆 to a real scene point 𝑢 = (𝑢, 𝑣) ∈ 𝑆 .
Their Method adopts a basis-function form to facilitate analytical computation of
the Jacobians and Hessians:
𝑢(𝑥, 𝑦), 𝑣(𝑥, 𝑦) = 𝑢 = 𝑓 (𝑥) =

𝑐 𝑏 (𝑥) + 𝑇𝑥 ,

(1)

where {𝑏 } are basis functions with weights {𝑐 }, and T is an
affine transformation matrix. We use Gaussians for b, i.e.,
𝑏 (𝑥) = 𝑒

|

_ |

(2)

Where 𝑥 is is the i-th basis center (blue points in Figure below) and x is a sample
point in Sv (green points in Figure below).

Figure : Stratified sampling of example for part of a game scene

The goal of this step is to find proper 𝑐 = {𝑐 } and T so that mapping f as globally
conforming and locally isometric as possible via a collection of objectives and
constraints as follows:
Conformal Objective: 2d Mappings satisfy the Cauchy-Riemann function when it
preserves angles, define conformal objective as :

Distance constraint: This mapping only requires to be locally isometric and hence
requires Jacobian J to satisfy 𝐽 𝐽 = 1 i.e

But to note an important fact here is that different virtual regions need different
amount of distance preservation in VR applications. Distances near the boundaries
should be more strictly preserved as the users can examine the virtual wall closely
than when the user is in the middle of a large room. To include this practical fact
the boundary conditions in Equation (4) are made more flexible

where ∈ [0, 1] and ∈ [1,+∞) are stretching ranges for each virtual scene point x.
When both and equal to 1, the mapping is strictly locally isometric. However, for
better conformality, we can relax the isometry into a range: the lower/higher the
𝛼/β value is, the more shrinking/stretching is allowed.
Exterior Boundary Constraint: To keep all u inside the real space Sr, construct a
polygon convex hull of Sr as a set of straight line functions {𝐵 } and add a series
of linear constraints:

Where 𝐶 is the center of the physical space and the idea is to keep 𝐶 on the same
side of 𝐵 for test point inclusion.
Interior Obstacle Constraint: Preventing users from hitting interior obstacles can be
formulated as the opposite of the point inclusion test in Equation (6).
Relaxed Distance Constraint: To facilitate local bijectivity relax the distance
constraints in Equation (5) to encourage stretching over folding.

The authors imagine the virtual domain as a plastic floor plan sheet that can be
bent or fold but never cut. To encourage bending over folding we increase the
upper limit of Equation (5) as follows:

(7)
Dynamic Inverse Mapping
For VR walkthroughs we need the reverse map of users current position in Sr to
Sv. This revers map needs to deal with the fact that the forward map might not be
bijective and thus there can be multiple solutions. It also must minimize angular
and distal distortions during navigation.
Start by the given user positions u(t) and u(t + 1) as well as orientations U(t) and
U(t + 1) tracked in the real world Sr at time steps t and t + 1, and the corresponding
virtual position x(t) and orientation X(t) at time t, our goal is to compute the
corresponding virtual position x(t+1) and orientation X(t+1). Note that this is a
path dependent process as x(t + 1) and X(t + 1) are computed from x(t), X(t),
u(t+1), and U(t+1). We manually assign x(0) and X(0) for the initial virtual world
position and orientation.
Direction update: To compute x(t + 1), we first compute the moving direction:
(8)
The virtual and real world directions are related by the Jacobians of their mapping:
(9)
Where,

(10)
is the real world direction. Thus, the goal is to find the Jacoabian of the reverse
function of f in Equation (1):

(11)
Even though f might not be globally bijective, the local bijectivity satisfies the
inverse function theorem and allows us to compute the inverse Jacobian via:
(12)
where Jx(u) can be computed from the analytic function f at position x(t).
Position update: We next compute the new virtual position x(t+1) based on the
estimated direction δx(t). We focus on the 2D x-y position, as the z/height value of
x can be directly assigned fromu after an initial correspondence. For computation
purposes, we define ∆x(t) = x(t+1)−x(t), and represent it in a polar coordinate
system, i.e., ∆x(t) = ∆xt(d, 𝜃) = (d cos(θ), d sin(θ)). The goal is to find optimized
(d, θ) to minimize an energy function as follows.
The first energy term measures how close the actual direction is to the estimated
direction δx(t):

(13)
The second term is to keep the virtual distance close to the real distance:
(14)
The last term is to match the mapping function f in Equation (1):
(15)
We find x(t + 1) = x(t) +∆x(t) to minimize
(16)
where 𝜆dir and λdis are relative weights.

For fast convergence, we make the initial guess as:

(17)
Orientation update: For rendering, we also need to compute virtual camera
orientation X(t) from real camera orientation U(t), which is tracked by the HMD.
Represent both orientations by their Euler angles:

(18)
Since planar map f has only x-y positions, compute only 𝑦𝑎𝑤 and simply copy
𝑝𝑖𝑡𝑐ℎ and 𝑟𝑜𝑙𝑙 from 𝑝𝑖𝑡𝑐ℎ and 𝑟𝑜𝑙𝑙 :

Rendering
Our goal is to render the appearance of the virtual world into the environment of
the real world, so that users can perceive the former while navigating in the latter.
The original virtual scene rendering, however, cannot be used for direct navigation
as it would cause motion sickness due to incompatibility with the real scene. We
thus fit the rendering of the virtual world into the geometry of the real world, as
discussed below.
Algorithm :
 We first render the virtual image Iv with virtual scene geometry Gv and
virtual camera Cv.
 We then initialize the real image Ir by mapping/warping Iv into Cr via f to
maintain visibility consistency with Iv.
 Parts of Ir might remain uncovered due to dis-occlusion, for which we
perform another rendering pass via the real scene geometry Gr and camera
Cr.

The figure below displays the effect of this rendering algorithm:

Similar to standard game level design, we surround the scene with an
environment-map box to ensure all pixels in Iv are initially covered. Thus,
all uncovered pixels in forward-warped Ir are caused by dis-occlusion. The
environment map is important to ensure robust dis-occlusion to prevent faraway objects being mistakenly rendered into the background, as exemplified
in Figure below.



---

Source Exif Data:

File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.7
Linearized                      : No
Warning                         : Info object (114 0 obj) not found at 1924388

EXIF Metadata provided by EXIF.tools