AVPro Video User Manual

AVProVideo-UserManual

AVProVideo-UserManual

User Manual:

Open the PDF directly: View PDF PDF.
Page Count: 109 [warning: Documents this large are best viewed by clicking the View PDF Link!]

AVPro Video
for Android, iOS, tvOS, macOS, WebGL
Windows Desktop, Windows Phone and UWP
Unity plugin for fast and flexible video playback
Version 1.9.12
Released 28 January 2019
Contents
1. Introduction
1. Features
2. Trial Version
3. Media Credits
2. System Requirements
1. Platforms not supported
3. Installation
1. Trial Version & Watermark Notes
2. Installing Multiple Platform Packages
4. Usage Notes
1. Platform Notes
2. Video File Locations
3. Streaming Notes
4. Audio Notes
5. Augmented / Virtual Reality Notes
6. Hap Codec Notes
7. Transparency Notes
8. Hardware Decoding
9. Multi-GPU SLI / CrossFire Notes
10. Subtitle Notes
11. DRM / Encryption Notes
12. Video Capture Notes
13. Seeking / Playback Rate Notes
5. Quick Start Examples
1. Quick Start Fastest Start for Unity Experts
2. Quick Start Fullscreen Video Player using Prefabs
3. Quick Start 3D Mesh Video Player Example using Components
6. Usage
1. Getting Started
2. Unsupported Platform Fallback
3. Components
4. Scripting
5. Platform Specific Scripting
6. Third-party Integration
7. Asset Files
1. Demos
2. Prefabs
3. Scripts
8. Scripting Reference
9. Supported Media Formats
10. Support
11. About RenderHeads Ltd
Appendix A - FAQ
Appendix B - Version History
Appendix C - Roadmap
2 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
1. Introduction
AVPro Video is the newest video playback plugin from RenderHeads. We previously
developed the AVPro QuickTime and AVPro Windows Media plugins for Unity. In this next
generation of plugins we aim to create an easy to use, cross-platform video playback system
that uses the native features of each platform.
1.1 Features
Versions for iOS, tvOS, OS X, Android, WebGL, Windows, Windows Phone and
UWP
One API for video playback on all supported platforms
*NEW* Facebook Audio 360 and Android ExoPlayer support
8K video support (on supported hardware)
VR support (mono, stereo, equirectangular and cubemap)
Transparency support (native and packed)
Unity Pro 4.6.x and above supported
Unity Personal 5.x - 2018.1.x and above supported
Free watermarked trial version available ( download here )
Fast native Direct3D, OpenGL and Metal texture updates
Linear and Gamma colour spaces supported
Graceful fallback in editor
Components for IMGUI, uGUI and NGUI
Extensive PlayMaker support
Easy to use, drag and drop components
Desktop support for Hap, Hap Alpha, Hap Q and Hap Q Alpha
Streaming and adaptive video from URL
1.2 Trial Version
We offer an unlimited trial version of AVPro Video for download from our website at
http://renderheads.com/product/avpro-video/ . The trial version has no missing features or
time restrictions but it does apply a watermark to the rendered output. The watermarking
does have a small performance impact which is only really noticeable on very high resolution
videos.
1.3 Media Credits
BigBuckBunny_360p30.mp4 - (c) copyright 2008, Blender Foundation /
www.bigbuckbunny.org
BigBuckBunny_720p30.mp4 - (c) copyright 2008, Blender Foundation /
www.bigbuckbunny.org
SampleSphere.mp4 - (c) copyright Daniel Arnett, https://vimeo.com/97887646
3 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
2. System Requirements
Unity 2018.x, Unity 2017.x, Unity 5.x Personal Edition, or Unity Pro 4.6 and above
iOS 8.1 and above
tvOS 9.0 (Apple TV 4th Gen) and above
macOS 10.7 and above, 64-bit only
Android 4.0.3 (Ice Cream Sandwich, API level 15) and above (ARM7, ARM8 and
x86)
Windows XP (SP 3) and above (32-bit and 64-bit)
Windows 8.0 and above (32-bit and 64-bit)
Windows Phone UWP 8.1 (32-bit and ARM)
Windows Desktop UWP 8.1 (32-bit, 64-bit and ARM)
Universal Windows Platform 10 (32-bit, 64-bit and ARM)
WebGL compatible browser
2.1 VR / AR / MR / XR Headsets Supported
Microsoft Hololens (UWP)
Windows mixed-reality headsets (UWP):
Samsung Odyssey
Asus
HP
Acer
Lenovo Explorer
Dell Visor
HTC Vive (Windows desktop)
HTC Vive Pro (Windows desktop)
HTC Vive Focus (Android)
Oculus Rift (Windows desktop)
Oculus Rift Go (Android)
Oculus Gear VR (Android)
Google Cardboard (Android)
Google Daydream (Android)
Pico Goblin & Neo (Android)
StarVR (Windows desktop)
Lenovo Mirage Solo (Android)
2.2 Platforms not Supported
WebPlayer
Linux desktop
Tizen
Samsung TV
Game Consoles (XBox*, PS4 etc)
* XBox One may be supported using UWP build option. We have not tested this though.
4 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
3. Installation
1. Open up a fresh Unity session (to clear any locked plugin files)
2. Import the unitypackage file into your Unity project. If prompted to upgrade some
scripts click Yes.
3.1 Trial Version & Watermark Notes
3.1.1 Watermark
If you are using a trial version of the plugin then you will see a watermark displayed over the
video. The watermark is in the form of a “RenderHeads” logo that animates around the
screen, or a thick horizontal bar that move around the screen. In Windows if the GPU
decoding path is used when the watermark isn’t displayed - instead every few seconds the
video size will scale down.
The full version of AVPro Video has no watermarks for any platforms. If you use one of the
platform specific packages (eg AVPro Video for iOS, or AVPro Video for Windows) then you
will not see the watermark on the platform you purchased for, but you will see the watermark
on the other platforms. For example if you purchased AVPro Video for iOS then you will still
see the watermark in the Unity editor as this is running on Windows/OS X, but the videos
played back when you deploy to your iOS device will be watermark-free.
3.1.2 Updating from Trial Version
If you are upgrading from the trial version, make sure you delete the old /Assets/Plugins
folder as this contains the trial plugin and could conflict. You may need to close Unity first,
delete the files manually and then restart Unity and re-import the package (because Unity
locks native plugin files once they are loaded).
You can check which version you have installed by adding an MediaPlayer component to
your scene and clicking on the ‘about’ button in the Inspector for that component. The
version number is displayed in this box.
3.2 Installing Multiple Platform Packages
If you are not using the full all-in-one AVPro Video package and instead have opted to
purchase multiple individual platform packages then the installation must be done carefully,
especially when upgrading to a new version.
If you have installed the iOS package then it will also contain plugins for all of the other
platforms but with the watermark enabled. This means that if you then try to install another
5 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
AVPro Video package it may not override the plugins correctly. Here is how to resolve this
using the iOS and Android package as examples:
1. Open a fresh Unity instance (this is important as otherwise Unity may have locked
the plugin files which prevents them from being upgraded)
2. Import the iOS package
3. Import the Android package, but make sure that you have the iOS native plugin file
unticked (so that it is not overwritten)
A similar process can be applied for other package combinations.
List of native plugin files:
Android
Plugins/Android/AVProVideo.jar
Plugins/Android/libs/armeabi-v7a/libAVProLocal.so
Plugins/Android/libs/arm64-v8a/libAVProLocal.so
Plugins/Android/libs/x86/libAVProLocal.so
macOS
Plugins/AVProVideo.bundle
iOS
Plugins/iOS/libAVProVideoiOS.a
tvOS
Plugins/tvOS/libAVProVideotvOS.a
WebGL
Plugins/WebGL/AVProVideo.jslib
Windows
Plugins/WSA/PhoneSDK81/ARM/AVProVideo.dll
Plugins/WSA/PhoneSDK81/x86/AVProVideo.dll
Plugins/WSA/SDK81/ARM/AVProVideo.dll
Plugins/WSA/SDK81/x86/AVProVideo.dll
Plugins/WSA/SDK81/x86_64/AVProVideo.dll
Plugins/WSA/UWP/ARM/AVProVideo.dll
Plugins/WSA/UWP/x86/AVProVideo.dll
Plugins/WSA/UWP/x86_64/AVProVideo.dll
Plugins/x86/AVProVideo.dll
○ Plugins/x86_64/AVProVideo.dll
6 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
4. Usage Notes
4.1 Platform Notes
Most graphics APIs are supported:
D3D9
D3D11
OpenGL
GLES2.0
GLES3.0
Metal
Vulcan
Android
N/A
N/A
N/A
Yes
Yes
N/A
No
iOS / tvOS
N/A
N/A
Yes
Yes
Yes
Yes
No
macOS
N/A
N/A
Yes
N/A
N/A
Yes
No
Windows
Yes
Yes
Yes
N/A
N/A
N/A
No
UWP
N/A
Yes
N/A
N/A
N/A
N/A
No
N/A = not applicable
4.1.1 Android
Supported systems are arm-v7a, arm64-v8a and x86
Under the hood we’re using the Android MediaPlayer API and ExoPlayer 2.8.4 API
This plugin requires a minimum Android API level of 15 when using the MediaPlayer
API, and API level 16 when using ExoPlayer (due to its use of MediaCodec).
If you want to support streaming don’t forget to set the “Internet Access” option in
Player Settings to “require”
For rendering we supports OpenGL ES 2.0 and OpenGL ES 3.0
Multi-threaded rendering is supported
The only 3rd-party libraries used are:
ExoPlayer 2.8.4
https://github.com/google/ExoPlayer
Facebook Audio 360 1.6.0
https://facebook360.fb.com/spatial-workstation/
4.1.2 iOS / tvOS
Supported systems are armv7, arm64
The iOS simulator is supported on x86 and x86_64
Under the hood we’re using the AVFoundation API
If you want to support streaming you need to enable HTTP downloads explicitly. For
iOS this is an option in newer versions of Unity, but for Mac OS X and older versions
of Unity you have to do this explicitly by editing the plist file. There are notes below
on how to do this.
For rendering we support OpenGL ES 2.0, OpenGL ES 3.0 and Metal
Multi-threaded rendering is supported
No 3rd-party libraries are used in the iOS / tvOS binaries
7 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
4.1.3 macOS
Only 64-bit (x86_64) builds are supported (Apple dropped 32-bit support back in
2012 when launching OS X 10.8).
Under the hood we’re using the AVFoundation API
If you want to support streaming you need to enable HTTP downloads explicitly. For
iOS this is an option in newer versions of Unity, but for Mac OS X and older versions
of Unity you have to do this explicitly by editing the plist file. There are notes below
on how to do this.
For rendering on Mac OS X we support OpenGL Legacy and OpenGL Core and
Metal.
Multi-threaded rendering is supported
The only 3rd-party libraries used in the macOS binaries are:
Hap
https://github.com/Vidvox/hap
zlib
http://zlib.net/
4.1.4 Windows Desktop
Under the hood we’re using the Media Foundation and DirectShow API’s. Media
Foundation is used for Windows 8 and above while DirectShow is used as a fallback
for Windows 7 and below.
For rendering we support Direct3D 9, Direct3D 11 and OpenGL Legacy.
Multi-threaded rendering is supported.
Windows N / KN edition notes:
There are some editions of Windows (N and KN) that ship with greatly
reduced built-in media playback capabilities.
It seems like these editions don’t include MFPlat.DLL, but do include some
basic DirectShow components. This means the Media Foundation playback
path will not work.
These editions of Windows require either a 3rd party codec installed (such as
the LAV Filters for DirectShow), or the Microsoft Media Feature Pack:
Media Feature Pack for Windows 7 SP1
https://www.microsoft.com/en-gb/download/details.aspx?id=16546
Media Feature Pack for Windows 8.1
https://www.microsoft.com/en-gb/download/details.aspx?id=40744
Media Feature Pack for Windows 10
https://www.microsoft.com/en-gb/download/details.aspx?id=48231
We found that MJPEG DirectShow codec still works on these editions without
installing the Media Feature Pack
The only 3rd-party libraries used in the Windows Desktop binaries are:
Hap
https://github.com/Vidvox/hap
Google Snappy
8 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
https://github.com/google/snappy
GDCL Mpeg-4
https://github.com/roman380/gdcl.co.uk-mpeg4
GLEW
http://glew.sourceforge.net/
Facebook Audio 360 1.6.0
https://facebook360.fb.com/spatial-workstation/
4.1.5 Windows Store / UWP / Hololens
For best compatibility and performance add
appCallbacks.AddCommandLineArg("-force-d3d11-no-singlethreaded");
To your MainPage.xaml.cs/cpp or MainPage.cs/cpp. You should call this before the
appCallbacks.Initialize() function.
For streaming video don’t forget to enable to “InternetClient” capability option in
Unity’s Player Settings. If you’re streaming video from a local server / LAN then you
need to enable the “PrivateNetworkClientServer” option.
No 3rd-party libraries are used in the WSA / UWP binaries
4.1.6 WebGL
The supported formats and features is dependant on the web browser capabilities
For best compatibility you can always force WebGL 1.0 instead of 2.0 which is the
default. This is done by going to Player Settings > Other Settings > Auto Graphics
API and removing WebGL 2.0. We have tested successfully with the following
browsers
macOS
Safari 9.1
Safari 10 (WebGL 1.0 seemed to work better than WebGL 2.0)
Safari 11.0.1
Windows
Microsoft Edge 38.14393.0.0
Mozilla Firefox 51.0
Google Chrome 56.0 - 62.0
The following browsers are not supported:
Internet Explorer 11 (any version), instead use the Microsoft Edge browser
4.2 Video File Location
Video files can be played in almost any location, however we recommend placing video files
in the /Assets/StreamingAssets/ folder in your Unity project as this is the easiest folder to
get started with. StreamingAssets is a special folder that Unity copies to the build without
processing. Files copied elsewhere will require manual copying to the build location.
9 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
The MediaPlayer component allows you to browse for video files and specify them relative to
a parent folder:
The Video Location field specifies the master location of the video file while the Video Path
field specifies where to locate the file relative to the Location.
For example if your file is stored in “Assets/StreamingAssets/video.mp4” you would set the
Location to “Relative To Streaming Assets Folder” and set the Video Path to “video.mp4”.
Sub-folders are also supported so a video located at
“Assets/StreamingAssets/myfolder/video.mp4” would have it’s Video Path set to
“myfolder/video.mp4”.
You can also specify absolute paths, URLs or paths relative to other locations:
4.2.1 Relative To StreamingAssets Folder
This is the best and most common location for video files. This folder is located at
“Assets/StreamingAssets/” and you must create it if it doesn’t exist. Files copied to this
folder will not be imported or processed by Unity but they will be copied with the build
automatically.
On Android though this folder isn’t ideal for massive files, as they are memory mapped and
so we’ve seen some devices have memory issues. For Android this folder is fine for
small-medium files (up to about 800MB depending on the device types you’re targeting), but
beyond that it’s better to copy or download the file into the persistent data folder and load it
from there.
4.2.2 Absolute Path or URL
Here you can specify a full URL or absolute path to the video file. A URL could be in the
form “http://myserver.com/myvideo.mp4” or “rtsp://myserver.com:8080/mystream.rtsp”
depending on the platform support and streaming service used.
An absolute path would look like:
10 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
C:/MyFolder/AnotherFolder/MyVideo.mp4 (Windows)
/Users/Mike/downloads/MyVideo.mp4 (Mac/Linux)
/Storage/SD/Videos/MyVideo.mp4 (Android external SDCARD)
/Storage/emulated/0/MyFolder/MyVideo.mp4 (Android local file system)
Using absolute paths can be useful for testing but isn’t useful when deploying to other
machines that don’t necessarily have the same file structure.
4.2.3 Relative To Project Folder
The project folder is the folder of your Unity project, so the folder containing the Assets,
Library and Project Settings sub-folders. Specifying files relative to the project folder can be
useful when you don’t want to include the video files in your Unity Assets folder but want to
keep them within the project folder structure. Often making a sub-folder called “Videos” is
useful. One possible problem of using this location is that when making a build your video
files will not be copied automatically to the build destination so they require manual copying.
For builds this folder should be located:
Windows - at the same level as your EXE
Mac - at the same level as the Contents folder in your app bundle
iOS - at the same level as the AppName.app/Data folder
Android - not accessible due to APK packaging unless you build the APK manually.
4.2.4 Relative To Data Folder
The data folder is specified by Unity here:
http://docs.unity3d.com/ScriptReference/Application-dataPath.html
It isn’t that useful to put video files into this folder directly as they would then be processed
by Unity into VideoClip’s or MovieTexture’s and will bloat your project size. If you want to
stop Unity processing the video files simply rename the extension to something Unity doesn’t
understand, so “myvideo.mp4” could be renamed to “myvideo.mp4.bin”. Files within the
data folder (Assets folder in the editor) are not copied automatically to builds so you would
have to manually copy them.
4.2.5 Relative to Persistent Data Folder
The persistent data folder is specified by Unity here:
http://docs.unity3d.com/ScriptReference/Application-persistentDataPath.html
For UWP platforms this would resolve to “ms-appdata:///local/”
11 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
4.3 Streaming Notes
AVPro Video supports several streaming protocol depending on the platform:
HTTP
Progressive
Streaming
Windows
Desktop
Yes
UWP
Yes
Mac OS X
Yes
iOS
Yes
tvOS
Yes
Android
Yes
WebGL
Yes*
* Remember for WebGL streaming you need to have proper CORS set up when accessing
other servers / ports. See our notes below on streaming with WebGL.
** In WebGL you can use the hls.js and dash.js libraries. See Streaming section for
implementation details.
HTTP Progressive Streaming
When encoding MP4 videos for streaming make sure they are encoded with the video
header data at the beginning of the file. You normally do this by selecting “Fast Start” in
QuickTime encoder, or use the “-movflags faststart” in FFMPEG, Other encoders will have a
similar option. To prepare an MP4 for streaming using FFMPEG you can use the following
command:
ffmpeg -i %1 -acodec copy -vcodec copy -movflags faststart %1-streaming.mp4
12 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Vimeo Note:
If you are streaming videos from VIMEO as MP4 then you should note that you can replace
the “.mp4” part in the URL with “.m3u8” to instead make it an HLS stream. This may be
particularly useful if you are developing apps for the Apple’s App Store as you would need to
use HLS streaming to pass certification (as for April 2016).
4.3.1 OS X, iOS and tvOS Streaming
This platform supports streaming of HLS streams which typically end with the m3u or m3u8
extension.
If you have an HTTPS URL it should work fine because Apple trusts the secure connection.
If you can only use HTTP then you app has to have a special flag set to let it use HTTP
connections (this is a security issue for Apple).
This setting is exposed in the Unity Player Settings here for iOS and tvOS:
The setting is also exposed in the scripting API here:
http://docs.unity3d.com/ScriptReference/PlayerSettings.iOS-allowHTTPDownload.html
If for some reason your version of Unity doesn’t expose this then you will have to add it
manually. In the Unity editor you need to edit "Unity.app/Contents/Info.plist" and in your built
application you would need to edit "your.app/Contents/Info.plist". These files need to have
these keys added:
<key>NSAppTransportSecurity</key>
<dict>
<key>NSAllowsArbitraryLoads</key>
<true/>
</dict>
You can find more information about this here:
http://ste.vn/2015/06/10/configuring-app-transport-security-ios-9-osx-10-11/
We’ve also included a post process build script called “PostProcessBuild.cs” in the project
which edits the plist and adds this attribute. Currently it’s only set for iOS but you can edit
the #define at the top to allow Mac OS X too.
4.3.2 Android Streaming
Using the ExoPlayer API is recommended for streaming video as it generally has wider
13 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
support for streaming protocols. Android streaming requires the Internet Access setting (in
Player Settings) to be set to “Require”:
4.3.3 UWP / Windows Phone / Hololens Streaming
Make sure to tick the “InternetClient” capabilities option in Player Settings. If you’re
streaming video from a local server / LAN then you need to enable the
“PrivateNetworkClientServer” option.
4.3.4 WebGL Streaming
If you are trying to access a URL on another server/port/domain then you need to have
CORS (cross-origin resource sharing) configured on that server to allow access. Websites
like https://enable-cors.org/ show you how to configure CORS on different web servers. If
you are hosting on a S3 bucket there are also ways to configure this. You can also test
whether CORS is the issue by installing a browser plugin to toggle CORS, for example this
one for Chrome:
https://chrome.google.com/webstore/detail/allow-control-allow-origi/nlfbmbojpeacfghkpbjhddi
hlkkiljbi
HLS and MPEG-DASH are not natively supported by all browsers. We have added hooks to
include third-party javascript libraries to handle these formats. Under the “Platform Specific
> WebGL” section you can select “External Library”. This will force the MediaPlayer to use
either hls.js or dash.js . You can also select “custom” if you wish to add support for your own
javascript library.
To add support or dash.js:
1. Download the latest dash.js release (we tested with 2.8.0):
https://github.com/Dash-Industry-Forum/dash.js/releases
2. Copy “dash.all.min.js” to the Assets/Plugins/WebGL folder and rename it
“dash.all.min.jspre”
3. In the MediaPlayer component set Platform Specific > WebGL > External Library to
dash.js
4. Build for WebGL
To add support for hls.js:
1. In the MediaPlayer component set Platform Specific > WebGL > External Library to
hls.js
14 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
2. Build for WebGL
3. Download the latest hls.js release (we tested with 0.10.1):
https://github.com/video-dev/hls.js/releases
4. Once your build is made, copy “hls.min.js” to the Build folder
5. Edit the index.html to add <script src="Build/hls.min.js"></script> before the
UnityLoader.js script is loaded. Ideally you would add this to your template.
4.3.5 Test Streams
We found these streams handy for testing (no guarantee that they’re still working):
Streaming MP4
HTTP
http://downloads.renderheads.com/2016/BigBuckBunny_360p30_Streaming.
mp4
HTTPS
https://drive.google.com/uc?export=download&id=0B0JMGMGgxp9WMEdWb
1hyQUhlOWs
HLS
http://qthttp.apple.com.edgesuite.net/1010qwoeiuryfg/sl.m3u8
http://184.72.239.149/vod/mp4:BigBuckBunny_115k.mov/playlist.m3u8
Apple Test streams (from https://developer.apple.com/streaming/examples/ )
Basic complexity
https://devimages.apple.com.edgekey.net/streaming/examples/bipbop
_4x3/bipbop_4x3_variant.m3u8
Medium complexity
https://devimages.apple.com.edgekey.net/streaming/examples/bipbop
_16x9/bipbop_16x9_variant.m3u8
Advanced TS
https://tungsten.aaplimg.com/VOD/bipbop_adv_example_v2/master.m
3u8
Advanced fMP4
https://tungsten.aaplimg.com/VOD/bipbop_adv_fmp4_example/master
.m3u8
MPEG-Dash
○ http://rdmedia.bbc.co.uk/dash/ondemand/bbb/2/client_manifest-high_profile-c
ommon_init.mpd
http://www.bok.net/dash/tears_of_steel/cleartext/stream.mpd
RTSP
rtsp://rtmp.infomaniak.ch/livecast/latele
RTMP
RTMP is not supported on any platform yet (unless 3rd party support is used)
4.4 Audio Notes
4.4.1 Facebook Audio 360
15 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Spatial audio support is currently available using Facebook Audio 360 on Windows desktop
and Android. On Windows only Windows 10 and above is supported and the Media
Foundation video API must be selected. On Android the ExoPlayer video API must be
selected. The video files must be using a MKV file container and audio must be using the
Opus codec encoded with Facebook Audio 360.
The settings are located under the Audio section of the MediaPlayer component. The “Head
Transform” field must be set to the transform that represents the player's head so that
rotation and positional changes affect the audio rendering. Usually this is the main camera.
“Enable Focus” can be enabled when a specific region of audio in the 360 field needs to be
given focus. The rest of the audio has its volume reduced.
Next the Facebook Audio 360 support must be enabled for each platform that needs it via
the “Platform Specific” panel. Currently it is only available on Windows desktop and Android.
The “Channel Mode” must be set to the channel encoding mode used when creating the
video. Currently this can not be determined automatically. The default is “TBE_8_2” which
means 8 channels of hybrid ambisonics and 2 channels of head-locked stereo audio.
More information on encoding etc can be found on the Facebook Audio 360 website at:
https://facebook360.fb.com/spatial-workstation/
Alternative steps for encoding manually
1. Create a WAV file with the audio format they need (Eg 9 channels ambisonics with 2
channels of head-locked audio will require a 11 channel WAV file with the 2
head-locked channels at the end)
2. Use Opus tools, as described here to convert the WAV file to Opus:
https://opus-codec.org/downloads/
https://facebookincubator.github.io/facebook-360-spatial-workstation/Documentation/
SDK/Audio360_SDK_GettingStarted.html#encoding-opus-files
3. Use ffmpeg to mux this opus file into the video container (ensure that the video file
doesn’t have any audio first):
ffmpeg -i audio.opus video.mp4 -c:a copy -c:v copy audio_video.mkv
16 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
4. In AVPro Video specify the required channel map
Converting existing ambisonic videos
It is also possible to convert existing ambisonic videos so they are compatible. For example
if you have an existing MP4 file with 4-channel 1st order ambisonic audio, then it is possible
to convert this into the above format (MKV container with Opus audio) using a tool like
FFMPEG. Simple put the following command in a .BAT file and then drag your MP4 into the
batch file:
ffmpeg -y -i input.mp4 -c:v copy -acodec libopus -mapping_family 255 output.mkv
This should then generate an MKV file that you can play with AVPro Video. All that remains
is to set the channel mapping in the MediaPlayer component to AMBIX_4.
4.4.2 Audio Redirection
Currently only the Windows plugin has support for audio manipulation beyond the standard
volume and stereo panning.
Some VR systems such as the Oculus Rift and HTC Vive have their own audio output device
and one needs to redirect audio to these devices instead of the system default audio device.
Unity does this automatically for its internal audio, but AVPro Video renders to the system
default audio device.
This issue can be solved by either using the AudioOutput component (see next section).
This component redirects the audio to be rendered by Unity and so it goes to the correct
device. AudioOutput requires that the Media Foundation video API be used, so if you need
to use the DirectShow API you can specify the name of the output device manually in the
field “Force Audio Output Device”:
The device name to use can be retrieved from the VR API or hard coded. For Oculus Rift
the name is usually “Rift Audio” or “Headphones (Rift Audio)” and for HTC Vive it is “HTC
VIVE USB Audio”. The Facebook Audio 360 audio output option on Windows also allows
17 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
you to specify a specific audio output device name, but this requires that the your audio is
encoded with the Opus codec.
4.4.3 Audio Spatialisation
Audio needs to rotate as the user moves their head in the virtual world. This can be achieve
by using the AudioOutput component which redirects the audio from the video into Unity (via
Unity’s AudioListener).
This component should be stacked above its required AudioSource component.
AudioOutput requires that the video API is set to Media Foundation and that the “Use Unity
Audio” tickbox is selected.
For proper 3D audio placement you will also need to add a spatialiser plugin to the Unity
Audio Settings screen (in Unity 5.4 and above) and tick the “spatialize” tickbox on the Audio
Source. Multiple instances of AudioOutput component can be created, each one outputting
a different audio channel from a different world position.
If Unity Audio is enabled but the Stereo checkbox is not selected, AVProVideo will try to
detect how many channels the video has, and send to Unity as many channels as
18 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
requested. To get all audio channels, make sure that the default speaker mode in the Unity
Audio Settings supports enough channels, and ensure that the OS audio sound settings are
in the correct speaker mode (otherwise Unity will just default to the maximum number of
supported channels). If the Stereo box is checked, the audio will get resampled to 2
channels.
If you wish to only get specific channels from the video, you can choose to mask the
channels using the AudioOutput component.
4.4.4 Channel Remapping
The AudioOutput component has the ability to route different audio channels from the media
to different physical channels. For best results make sure your video only has mono audio.
The audio will then get duplicated to the other channels.
4.5 Augmented / Virtual Reality Notes
So far we have tested AVPro Video with:
Gear VR
Google Cardboard
Google Daydream
Oculus Rift
HTC Vive
Microsoft Hololens
VR is still very new and you should always check for the latest recommended installation
steps when creating your project. We found a lot of out of date setup instructions on the
net.
AVPro Video supports 4K MP4 playback for creating 360 degree experiences. Stereo 4K
videos in both top-bottom and side-by-side formats are also supported. If you’re using
Windows 10 and have an Nvidia Geforce 1000 series (eg 1070) you can display 8K H.265
videos (requires 64-bit build).
See the FAQ for tips to achieve high resolution video playback for VR.
For software decoders reducing the complexity of the encoded video will give the decoding
engine a much easier time and could result in higher frames rates and lower CPU/GPU
usage. Possible encoding tweaks include:
19 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Use the lowest profile level possible
Don’t use too many reference frames
Don’t use too many b-frames
Disable CABAC
Use the slices option (eg -slices 4)
4.5.1 Stereo VR
AVPro Video supports stereoscopic videos in the top-bottom and left-right formats. You can
set the stereo packing format of your video in the Media Properties panel:
Now when using the InsideSphere shader on a mesh it will automatically map the right part
of the video to each eye. See the “Demo_360SphereVideo” scene for an example of how
this works.
Optionally you can manually set the material properties. The included shader
“InsideSphere.shader” allows you to easily set what format your video is in via a drop-down
in the material:
Select “Stereo Debug Tinting” to colour the left and right eyes different colours so you can be
sure the stereo is working.
20 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
NOTE : Be sure to add the "UpdateStereoMaterial” component script to your scene when
using this material and a stereo video. Often stereo VR requires 2 cameras, each set to a
different layer mask and 2 spheres also set to a different mask. AVPro Video doesn’t require
this and just uses your normal single camera and single sphere.
NOTE: If you’re playing stereo videos on Windows and the stereo isn’t appearing properly,
please read our FAQ #8.
An example of how to pack 2 videos together into a left-right stereo packed layout using
FFMPEG:
ffmpeg -i left.mp4 -vf "[in] pad=2*iw:ih [left]; movie=right.mp4
[right];[left][right] overlay=main_w/2:0 [out]" stereo.mp4
4.5.2 Android OES playback path
For Android there is a special playback option called “Use Fast OES Path”. This option
caters especially for VR where users are trying to get the highest possible frame rate and
resolution out of the device (without it overheating at the same time). The option is
available in the Platform Specific section of the MediaPlayer component:
The OES path is not enabled by default because it requires some special care to be taken
and can be tricky for beginners. When this option is enabled the Android GPU returns
special OES textures (see EGL extension OES_EGL_image_external) that are hardware
specific. Unfortunately Unity isn’t able to use these textures directly, so you can’t just map
them to a material or UI. To use the texture a GLSL shader must be used. Unfortunately
Unity’s GLSL support isn’t as good as its CG shader support so again this makes things
more tricky. The GLSL compiler only happens on the device (not inside Unity) so errors in
the shader can be difficult to debug.
We have included a version of the VR sphere shader that supports stereo videos as an
example. Hopefully in the future we can improve the integration of these shaders so they
aren’t such special cases. This playback path is much faster though, so is definitely worth
exploring. Note that for VR stereo rendering, OES only currently supports multi-pass
21 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
rendering path, and not single-pass or single-pass instanced.
For the sphere demo scene, simply change the shader on the sphere mesh material to be
one of the “VR” “OES” ones and tick the “Use Fast OES Path” on the MediaPlayer
component.
4.5.3 iOS YCbCr playback path
For iOS and tvOS we have added support for YCbCr textures which results in memory
saving compared to standard RGBA32 textures. This option is enabled by default but can
be disabled on the MediaPlayer here:
This option is ideal for memory saving which is especially important when targeting low-end
devices (with 1GB RAM) and when playing back very high resolution video, such as 4K for
VR content. Improved performance may also be experienced using this option. The
DisplayIMGUI and DisplayUGUI components automatically detect this and switch to a
suitable shader. ApplyToMesh/ApplyToMaterial also detect this setting and tries to set up the
shader on the material to the correct settings, however it requires the shader to have the
correct properties. The AVPro Video shaders support this, so if you want to use this on a
mesh then make sure you’re using these shaders.
4.6 Hap Codec Notes
The Hap video codec is natively supported by AVPro Video on certain platforms and has the
following benefits:
Very low CPU usage
GPU decompression
Low memory usage
Supports very high resolutions
Supports alpha channel transparency
The main down side is:
Very large files
AVI and MOV containers can both be used however we recommend the MOV container.
22 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Hap is only supported on Windows and Mac OS X platforms.
4.6.1 Windows Support
Hap, Hap Alpha, HapQ and HapQ Alpha are supported. Hap currently requires the
“DirectShow” video API to be selected:
4.6.2 Mac OS X Support
Hap, Hap Alpha, HapQ and HapQ Alpha are supported.
4.6.3 Encoding
You can download the QuickTime codec for Windows and macOS here:
https://github.com/Vidvox/hap-qt-codec/releases
This codec can be used with QuickTime Pro or any other software that supports QuickTime
codecs such as Adobe After Effects and Adobe Premiere.
Alternatively you can use a recent build of FFMPEG with the following command-lines:
ffmpeg -i input.mov -vcodec hap -format hap output-hap.mov
ffmpeg -i input.mov -vcodec hap -format hap_alpha output-hap.mov
ffmpeg -i input.mov -vcodec hap -format hap_q output-hap.mov
Notes:
You can also add the “-chunks 4” option which will encode each frame into 4 chunks
so the decoding work can be split across multiple threads, resulting in faster
decoding as long as the disk can keep up.
Width and height must be multiple of a 4.
Hap Alpha requires straight not pre-multipled alpha.
Sadly ffmpeg doesn’t yet support the HapQ Alpha format.
We don’t support Hap Q Alpha variant in Windows when using the legacy D3D9
graphics API
23 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
4.7 Transparency Notes
Not many video codecs have native support for transparency / alpha channels. Formats
supported by some platforms of AVPro Video are:
Hap Alpha
Great support on Windows and Mac OS X. Fast and low overhead format,
though file size can get large depending on the content. Currently this is the
format we recommend for transparent video.
Hap Q Alpha
Great support on Windows and Mac OS X. Slightly higher quality and file size
compared to Hap Alpha.
Uncompressed RGBA
Uncompressed YUVA
Uncompressed isn’t ideal for file size or disk bandwidth but can still be used
as a fallback
ProRes 4444
Best support is on Mac OS X. Files are huge.
VP6
Legacy format. We support it only via 3rd party DirectShow plugins for
Windows (eg LAV Filters)
4.7.1 Alpha Packing
Alternatively you can encode your videos in video formats that don’t support an alpha
channel by packing the alpha channel into the same frame. You can double the width for a
left-right packing layout, or double the height for a top-bottom packing layout. This packing
could be created in software such as AfterEffects, or the command-line ffmpeg tool can be
used. The packing format is set in the “Media Properties” panel of the AVPro Video
MediaPlayer component:
Here we show two examples using ffmpeg to convert a source video containing a
24 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
transparency/alpha channel into an alpha packed format:
Left-right alpha packing:
ffmpeg -i %1 -vf "split [a], pad=iw*2:ih [b], [a] alphaextract, [b]
overlay=w" -y %1.mp4
Top-bottom alpha packing:
ffmpeg -i %1 -vf "split [a], pad=iw:ih*2 [b], [a] alphaextract, [b]
overlay=0:h" -y %1-tb.mp4
25 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
4.8 Hardware Decoding
AVPro Video supports hardware decoding on most platforms for optimal performance.
WebGL, Windows 8.1, Windows 10, macOS, tvOS, iOS and Android all default to hardware
decoding when possible. When playing back multiple videos simultaneously one must be
careful not to exceed the number of videos that the hardware can play back smoothly. We
have tried to collect information about the limits of specific hardware.
4.8.1 NVidia
NVidia GPU’s use a technology called “Purevideo” or “NVDec” to off-load decoding from the
CPU. More information can be found here:
https://en.wikipedia.org/wiki/Nvidia_PureVideo#Nvidia_VDPAU_Feature_Sets
Some NVidia Purevideo capabilities:
KEPLER (GK107,
GK104)
MAXWELL 1
(GM107, GM204,
GM200)
MAXWELL 2
(GM206)
PASCAL (GP100)
MPEG-2, MPEG-4,
H.264
MPEG-2, MPEG-4,
H.264, HEVC with
CUDA acceleration
MPEG-2, MPEG-4,
H.264 HEVC/H.265
fully in hardware
MPEG-2, MPEG-4,
H.264 HEVC/H.265
fully in hardware
H.264: ~200 fps at
1080p; 1 stream of
4K@30
H.264: ~540 fps at
1080p 4 streams of
4K@30
H.264: ~540 fps at
1080p 4 streams of
4K@30
?
H.265: Not
supported
H.265: Not
supported
H.265: ~500 fps at
1080p 4 streams of
4K@30
?
(table adapted from presentation “HIGH PERFORMANCE VIDEO ENCODING WITH NVIDIA GPUS”)
GPU
Architect
ure
MPEG-2
VC-1
H.264/
AVCHD
H.265/
HEVC
VP8
VP9
Fermi (GF1xx)
Maximum
Resolution:
4080x4080
Maximum
Resolution:
2048x1024
1024x2048
Maximum
Resolution:
4096x4096
Profile:
Baseline,
Main, High
profile up to
Level 4.1
Unsupported
Unsupported
Unsupported
Kepler
(GK1xx)
Maximum
Resolution:
4080x4080
Maximum
Resolution:
2048x1024
1024x2048
Maximum
Resolution:
4096x4096
Profile: Main,
Unsupported
Unsupported
Unsupported
26 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
High profile up
to Level 4.1
Maxwell Gen
1 (GM10x)
Maximum
Resolution:
4080x4080
Maximum
Resolution:
2048x1024
1024x2048
Maximum
Resolution:
4096x4096
Profile:
Baseline,
Main, High
profile up to
Level 5.1
Unsupported
Unsupported
Unsupported
Maxwell Gen
2 (GM20x)
Maximum
Resolution:
4080x4080
Maximum
Resolution:
2048x1024
1024x2048
Max bitrate:
60Mbps
Maximum
Resolution:
4096x4096
Profile:
Baseline,
Main, High
profile up to
Level 5.1
Unsupported
Maximum
Resolution:
4096x4096
Unsupported
Maxwell Gen
2 (GM206)
Maximum
Resolution:
4080x4080
Maximum
Resolution:
2048x1024
1024x2048
Interlaced
Maximum
Resolution:
4096x4096
Profile:
Baseline,
Main, High
profile up to
Level 5.1
Maximum
Resolution:
4096x2304
Profile: Main
profile up to
Level 5.1
Maximum
Resolution:
4096x4096
Maximum
Resolution:
4096x2304
Profile: Profile
0
Pascal
(GP100)
Maximum
Resolution:
4080x4080
Maximum
Resolution:
2048x1024
1024x2048
Maximum
Resolution:
4096x4096
Profile:
Baseline,
Main, High
profile up to
Level 5.1
Maximum
Resolution:
4096x4096
Profile: Main
profile up to
Level 5.1
Maximum
Resolution:
4096x4096
Maximum
Resolution:
4096x4096
Profile: Profile
0
Pascal
(GP10x)
Maximum
Resolution:
4080x4080
Maximum
Resolution:
2048x1024
1024x2048
Maximum
Resolution:
4096x4096
Profile:
Baseline,
Main, High
profile up to
Level 5.1
Maximum
Resolution:
8192x8192
Profile: Main
profile up to
Level 5.1
Supported2
Maximum
Resolution:
4096x4096
Maximum
Resolution:
8192x8192
Profile: Profile
0
(Table adapted from Nvidia Video Decoder Interface documentation:
https://developer.nvidia.com/nvdec-programming-guide )
4.8.2 AMD
AMD UVD
https://en.wikipedia.org/wiki/Unified_Video_Decoder
UVD Version
Hardware Code Names
UVD 1.0
RV610, RV630, RV670, RV620, RV635
UVD 2.0
RS780, RS880, RV770
27 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
UVD 2.2
RV710, RV730, RV740
UVD 2.3
CEDAR, REDWOOD, JUNIPER, CYPRESS
UVD 3.0
PALM (Wrestler/Ontario), SUMO (Llano), SUMO2 (Llano
UVD 3.1
BARTS, TURKS, CAICOS, CAYMAN
UVD 3.2
ARUBA (Trinity/Richland), TAHITI
UVD 4.0
CAPE VERDE, PITCAIRN, OLAND
UVD 4.2
KAVERI, KABINI, MULLINS, BONAIRE, HAWAII
UVD 5.0
TONGA
UVD 6.0
CARRIZO, FIJ
UVD 6.2
STONEY
UVD 6.3
POLARIS10, POLARIS11, POLARIS12
(table adapted from X.org RadeonFeature wiki:
https://www.x.org/wiki/RadeonFeature/#index8h2 )
UVD
Version
MPEG 2
MPEG 4
MPEG-4
AVC/VC1
HEVC
Max Size
Notes
UVD 1.0
No
No
Yes
No
2K
UVD 2.0
No
No
Yes
No
2K
UVD 2.2
No
No
Yes
No
2K
UVD 2.3
No
No
Yes
No
2K
UVD 3.0
Yes
Yes
Yes
No
2K
UVD 3.1
Yes
Yes
Yes
No
2K
UVD 3.2
Yes
Yes
Yes
No
2K
UVD 4.0
Yes
Yes
Yes
No
2K
UVD 4.2
Yes
Yes
Yes
No
2K
UVD 5.0
Yes
Yes
Yes
No
4K
UVD 6.0
Yes
Yes
Yes
Yes
4K
UVD 6.2
Yes
Yes
Yes
Yes
4K
Supports 10bit
UVD 6.3
Yes
Yes
Yes
Yes
4K
Supports 10bit
28 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
(table adapted from X.org RadeonFeature wiki:
https://www.x.org/wiki/RadeonFeature/#index8h2 )
4.8.3 Intel
Intel’s Quick Sync technology is a dedicated video decoder hardware built into the CPU.
Intel Kaby Lake supports H.265 10-bit decoding!
4.9 Multi-GPU SLI / CrossFire Notes
Multiple GPU’s are often used to accelerate high-end rendering but it brings with it some
subtle challenges for video playback. Here we write some notes about our experience using
Nvidia SLI on Windows (specifically Windows 8.1 with dual Nvidia M6000 cards).
Using Alternate Frame Rendering SLI (AFR SLI) can cause stuttering when using the Media
Foundation hardware decoding because only one GPU is doing the decoding and then it
must pass the frames back to the other GPU which is a slow operation and defeats the
purpose of SLI. One option here is to disable GPU decoding in AVPro Video, which is fine
for lower resolution videos, but often the CPU just isn’t fast enough for very high resolution
content. AFR can also suffer from so called “micro stutters” caused by each GPU handling
the flip presentation independently which isn’t ideal for video playback on large video walls.
Another option is to use the cards in “SLI VR” mode … To be continued..
4.10 Subtitle Notes
AVPro Video supports external subtitles in the SRT format. Subtitles internal to media files
are not yet supported. See the subtitle demo scene to see how to use subtitles.
29 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
4.11 DRM / Encryption Notes
DRM support is currently not a key feature of AVPro Video, but we aim to improve support in
the future. DRM that we know works with the plugin includes:
Android
HLS with AES-128 clear-key (make sure your TS segments are 188 bytes
aligned for maximum Android compatibility)
We have a file offset feature which allows you to access files hidden within a
file at an offset. Not strictly DRM but it can be used as a quick way to hide
video files within other data
In Windows you can easily append your video to a dummy video file
with the following command:
copy /b DummyVideo.mp4 + %1 %~n1-hidden.mp4
Custom HTTP header fields can be specified which can help with server side
validation
macOS & iOS & tvOS
HLS with AES-128 clear-key, direct key and key request using an auth token
in the HTTP header (“Authorization” field). More information about HLS
encryption can be read in the RFC here:
https://tools.ietf.org/html/draft-pantos-http-live-streaming-23
Custom HTTP header fields can be specified which can help with server side
validation
*DRM schemes Fairplay, Widevine, PlayReady etc are not yet supported.
4.11.1 HLS Decryption
On the Apple platforms (macOS, iOS and tvOS) we support HLS streams encoded with AES
encryption. Key retrieval from a server URL usually requires an authentication token, which
can be specified using the method SetKeyServerAuthToken(string), or this can be ignored if
your key retrieval server doesn’t require any authentication (clear-key). The auth token is a
string that is inserted into the “Authorization” HTTP field when retrieving the decryption key
from the server URL specified in the HLS manifest.
30 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
We also added some functionality to override the key URL and the to specify the key data
directly. SetDecryptionKey(byte[]) and SetDecryptionKeyBase64(string) can be used to
specify the key directly. Using this will bypass any server key retrieval.
SetKeyServerURL(string) can be used to override the key retrieval URL. Both of these are
useful for debugging.
These options can also be specified in the MediaPlayer inspector UI (in the Platform Specific
section), and in the PlatformOptions properties, eg mediaPlayer.PlatformOptionsIOS
4.12 Video Capture Notes
To make a non-realtime video capture of your Unity scenes which include videos, requires
the video playback to slow down or speed up to match the video capture rate. AVPro Video
supports this through the “TimeScale Support” option which is found in the Global Settings
panel of the Media Player component. This means you can create high quality renders
running at 1fps to produce a smooth 60fps video, and any videos in your scene will play
back at the correct rate for the recording. Audio is not supported though when using this
option (as is the case in Unity itself).
4.13 Seeking / Playback Rate Notes
Most videos are optimally encoded for the main use case: normal forward playback with
inaccurate seeking.
If you want to start changing the playback rate, play in reverse or have frame accurate
seeking then you may run into issues where the playback becomes extremely slow or the
seeking is not accurate. There are several reasons for this, but it mostly is related to how
the video is encoded and specifically the key-frame distribution. There are also some
platform differences to consider.
4.13.1 Video Codecs and Encoding
Codecs such as H.264 and H.265 generally compress video frames so that they depend on
data included with previously decoded frames. These are called P and B frames and
seeking to one of these is computationally expensive as in order to display them the decoder
must first decode the other frames that they depend on. The other type of frame is called a
key-frame or I-frame and these frames can be decoded immediately because they don’t
depend on any other frames. Compressing using P And B frames is known as temporal
31 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
compression and isn’t ideal for accurate random seeking or playback rate changes.
For the best results you would to encode your video with only key-frames, as then you can
seek accurately and quickly anywhere in the video. This is possible, but increases the file
size dramatically. Using FFMPEG you can encode a video to use key-frames only using the
“-g 1” option. Another option would be to use a codec that only supports key-frames, such
as Hap or ProRes - but again these result in large file sizes and limited GPU decoding
capabilities.
In most codec with temporal compression the key-frames are spaced every 250 frames.
Some platforms can only seek to the key-frames (see table below), while others can do
accurate seeking but this can be very slow if the distances between key-frames is too large.
Try reducing the key-frame distance for faster seeking. You can also reduce the decoder
complexity by encoding with a fastdecode tuning option.
4.13.2 Playback Rate
Generally we recommend these rates:
0.25, 0.5, 1.0, 1.25, 1.5, 1.75, 2.0
Going up to 4.0 might be possible depending on your platform, machine specs and the
codec used. Increasing playback rate usually places more demand on the video decoder
and also on the disk/network source, and these limit how much you can increase the
playback rate by.
Using negative values isn’t generally recommended as it isn’t as well supported, but if you
do have to use a negative rate then also try keeping the numbers small such as:
-0.25, -0.5, -1.0
Audio also may or may not play when changing the playback rate - this depends on the
platform (see table below).
One safe alternative to adjusting rate is to pause the video and fast seek to simulate a
change in playback rate. This approach would work on all platforms.
Video encoding can also help the performance of a change in playback rate. Videos with
more key-frames (or ideally all key-frames) and with less complex encoding (eg no B frames,
CABAC disabled etc) will work better. Alternatively a key-frame-only codec could be used,
such as Hap.
4.13.3 Platform Differences
Seeking:
Platform and
Video API
Fast Inaccurate
Keyframe Seeking
Slow Accurate
Frame Seeking
Adjust
Playback Rate
32 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Android
MediaPlayer API
Yes
API 26 and above
API 6 and above
Android
ExoPlayer API
Yes
Yes
Yes
Windows
Media Foundation
Yes
Yes
Yes
Windows
DirectShow
Yes
Depends on the
codec
Yes but not negative
UWP
Media Foundation
Yes
Yes
Yes
macOS
Yes
Yes
Yes If source
supports it
iOS & tvOS
Yes
Yes
Yes If source
supports it
webGL
Yes
Varies
Yes but not negative
Playback Rate:
Platform and
Video API
Adjust
Playback Rate
Audio Plays
Negative Rates
Android
MediaPlayer API
API 23 and above
Yes
?
Android
ExoPlayer API
Yes
Yes
?
Windows
Media Foundation
Yes
Depends on codec
Yes
Windows
DirectShow
Yes
No
No
UWP
Media Foundation
Yes
Depends on codec
Yes
macOS
Yes
Yes
Depends on media
source
iOS & tvOS
Yes
Yes
Depends on media
source
webGL
Yes
Depends on
browser
No
33 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
5. Quick Start Examples
5.1 Quick Start: Fastest Start for Unity Experts
1. Put video files in the StreamingAssets folder
2. Use the MediaPlayer script to play your video (set Video Path to the file name of your
video file).
3. Use one of the display scripts to display your video (eg DisplayIMGUI, DisplayUGUI,
ApplytoMaterial)
5.2 Quick Start: Fullscreen Video Player using Prefabs
AVPro Video includes a number of example prefabs you can use to easily add video
playback to your project. The following steps will create an application that plays back a
fullscreen video:
1. Create a new Unity project
2. Import the AVProVideo package
3. From the AVPro/Prefabs folder in the Project window, drag the FullscreenVideo
prefab to your Hierarchy window
4. Create a folder called StreamingAssets in your Project window and copy your video
34 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
file (say MP4 file) into that folder
5. Enter the file name (including extension) into the Video Path field in the MediaPlayer
component (eg myvideo.mp4)
6. Build and deploy your application, the video will be displayed fullscreen
The DisplayIMGUI component script is just one of the components for displaying video. It
uses the legacy Unity IMGUI system which always renders on top of everything else. Try
using the DisplayBackground or DisplayUGUI components for more control if you don’t want
your video to be on top.
5.3 Quick Start: 3D Mesh Video Player using Components
AVPro Video includes a number of easy to use script components you can add to your
scene. In this example we show how to use the components to play a video onto a material
which is applied to a 3D model in the scene.
1. Create a new Unity project
2. Import the AVProVideo package
3. Create a new GameObject from the “GameObject > AVPro Video > Media Player”
menu command
4. Click the “Add Component” button and add “AVPro Video > Apply To Mesh”
5. Drag the Media Player script to the “Media” field in the Apply To Mesh script, this tells
the Apply to Mesh script which media player to use
6. Create a sphere via the “GameObject > 3D Object > Sphere” menu command
7. Drag the Mesh Renderer component to the “Mesh” field in the Apply To Mesh script,
this tells the Apply to Mesh script which mesh to use
8. Create a folder called StreamingAssets in your Project window and copy your video
35 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
file (say MP4 file) into that folder
9. Enter the file name (including extension) into the Video Path field in the MediaPlayer
component (eg myvideo.mp4)
10. Build and deploy your application, the video will be displayed on your 3D sphere
36 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
6. Usage
6.1 Getting Started
The easiest way to get started is to look at the included demos and see what script
components have been used. For video playback you need 3 things in your scene:
1. The video file to play:
Create a “StreamingAssets” folder in your Project window
Copy your video file (usually MP4 file, but consult the list of supported formats for
your platform below) to the StreamingAssets folder
2. A MediaPlayer script to load and play the video:
Create a GameObject and add the MediaPlayer script to it
Set the Video Path field to the name of your video file (e.g. myvideo.mp4)
3. A script to display the video:
Decide how and where you want your video file to appear. There are a number of
different display component scripts included for different usage scenarios. If you
want to display the video on top of everything in your scene just add the
DisplayIMGUI script to a GameObject in your scene and set the Media Player field
your MediaPlayer component. Other display components work similarly.
6.2 Unsupported Platform Fallback
AVPro Video is designed to still function even on platforms that aren’t natively supported.
Instead of displaying the actual video though, a dummy 10 second long “AVPro” visual is
shown. All of the video controls should still work. For example if you are running your editor
in Linux the dummy video player will appear in the editor and the real video will appear when
you deploy to supported platforms. If you deploy to an unsupported platform such as
Samsung TV you will also see the dummy video player. The code is easily extendable to
add custom video players for any unsupported platform.
6.3. Components
Included are a number of components to make this asset easy to use. The components are
located in the AVProVideo/Scripts/Components folder or you can add them from the
Components menu:
37 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
6.3.1 Media Player Component
This is the core component for playing media. This component only handles the loading and
playback of media and doesn’t handle how it is displayed. Use the display script
components to control how and where the video is displayed. Fields are:
38 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Video Location
Where to look for the file specified in the Video Path below. This can be an
absolute path/URL, or relative to one of the Unity folders. The
StreamingAssets folder is the easiest to use. Options are:
Absolute or URL
This is an absolute path on your device, or an http URL
Relative to Project Folder
The root folder is the folder above your Assets folder
Relative to Streaming Assets Folder
The root folder is /Assets/StreamingAssets
Relative to Data Folder
The root folder is /Assets
Unity manual has more information:
http://docs.unity3d.com/ScriptReference/Application-dataPath.
html
Relative to Persistent Data Folder
Unity manual has more information:
http://docs.unity3d.com/ScriptReference/Application-persistent
DataPath.html
Video Path
The file path to the video in the StreamingAssets folder (e.g. myvideo.mp4 or
AndroidVideos/myvideo.mp4 if you want to use a subfolder)
Auto Open
Whether to open the file when this component is enabled/starts
Auto Start
Whether to play the video once a video is opened
Loop
Whether to loop the video
Playback Rate
Sets a multiplier that affects video playback speed
Please see section 4.13 notes about this
Volume
0..1 range for audio volume
Muted
Whether the audio is muted
Persistent
Applies DontDestroyOnLoad to the object so that it survives scene/level loads
Debug Gui
Whether to display an overlay with statistics on the video playback - useful for
debugging
Events
This event can be hooked up to scripting functions which will get called when
a non-looping video completes playback. See the Events section below for
more details and a scripting example
Visual
Texture
Set the desired filtering and wrap mode for the final texture that the
frames are written to. Useful if you wish to tile your textures in a specific way.
39 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Transparency
Set the pack mode used for transparency, where one half of the video
contains the colour data, and the other half contains the monochrome
alpha channel data
Stereo
Set the pack mode used for stereo, where one half of the video
contains the left eye data, and the other half contains the right eye
data
Resampler
The resampler helps smoothen out jitters in video playback caused by
unity and decoder vsync drift. The downsides to using it is that it uses
more processing power and GPU memory, and that it is out of sync for
videos with audio as it displays the video frames a couple of frames
later than the decoder.
There are two modes in the resampler:
POINT picks the closest frame in the buffer and displays it
LINEAR picks the two frames directly before and after the
current display time, and displays the interpolated version of
the two based on how far away the current time is from both of
them
Resample buffer size determines how many frames are stored in the
buffer. A larger buffer size uses more GPU memory as there are more
textures, but increases the chance that the desired frames will be
found in the buffer, which will allow for smoother playback. A larger
buffer size will also mean a larger delay between the video displayed
and the actual video on the decoder.
Platform Specific
These allow you to set a different options per platform, including setting a
different file path.
Windows
DirectShow options
Preferred Filters - list the names of filters you want to force, eg
“LAV Video Decoder”, “Lav Splitter Source”, “Microsoft
DTV-DVD Video Decoder”
6.3.2 Display IMGUI Component
40 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
This is the most basic component for displaying the video. It uses the legacy Unity IMGUI
system to display the video to the screen. IMGUI is always rendered on top of everything
else in the scene, so if you require your video to be rendered in 3D space or as part of the
uGUI system it’s better to use the other components. Fields are:
Media Player
The media player to display
Display in Editor
Whether to display the rectangle in the editor - useful for debugging
Scale Mode
How to fit the video to the screen
Color
The color to tint the video, including alpha transparency
Alpha Blend
Whether the video texture controls transparency. Leaving this off for opaque
videos is a minor optimisation
Depth
The IMGUI depth to display at. Use this to change the order of rendering with
other IMGUI scripts
Full Screen
Whether to ignore the X, Y, Width, Height values and just use the whole
screen
X
The normalised (0..1) x position
Y
The normalised (0..1) y position
Width
The normalised (0..1) width
Height
The normalised (0..1) height
6.3.3 Display uGUI Component
41 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
This component is used to display a video using Unity’s uGUI system. Field are:
Media Player
The media player to display
Default Texture (optional)
A texture to display while the video isn’t playing (while it is buffering for
example).
No Default Display
Will not show anything until there are frames available
Color
The color to tint, including alpha transparency
Material
Standard uGUI field, change this to one of the included materials when using
packed alpha or stereo videos
UV Rect
Standard uGUI field
Set Native Size
When the video loads will resize the RectTransform to the pixel dimensions of
the video
Keep Aspect Ratio
Whether to keep the correct aspect ratio or stretch to fill
6.3.4 Apply To Mesh Component
This component takes the texture generated by the Media Player component and assigs it to
the texture slot of the material on a 3D Mesh. This is useful for playing videos on 3D
42 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
meshes. Field are:
Offset
The XY translation to apply to the texture
Scale
The XY scale to apply to the texture
Mesh
The mesh (renderer) to apply the texture to
Media
The media player
Default Texture (optional)
A texture to display while the video isn’t playing (while it is buffering for
example).
Texture Property (optional)
By default this script assigns to the main texture (_MainTex) but if you want to
assign to another slot you can put the name in here. If you’re using the
Standard Shader and want the texture to drive the emissive property, set the
property name to “_EmissionMap”. You may also need to apply a dummy
texture to the emissive texture slot in the material to initialise it, and set the
emissive colour so it isn’t black.
6.3.5 Apply To Material Component
This component takes the texture generated by the Media Player component and assigns it
to a texture slot in a Material. This is useful for playing videos on 3D meshes. Fields are:
Offset
The XY translation to apply to the texture
Scale
The XY scale to apply to the texture
Material
The material to apply the video texture to
Texture Property Name (optional)
By default this script assigns to the main texture (_MainTex) but if you want to
assign to another slot you can put the name in here. If you’re using the
Standard Shader and want the texture to drive the emissive property, set the
property name to “_EmissionMap”. You may also need to apply a dummy
texture to the emissive texture slot in the material to initialise it, and set the
43 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
emissive colour so it isn’t black.
Media
The media player
Default Texture (optional)
A texture to display while the video isn’t playing (while it is buffering for
example).
6.3.6 Cubemap Cube Component
This component generates a cube mesh that is suitable for 3:2 cubemap 360 VR videos.
Fields are:
Material
The material to apply to the cube. This is usually just a standard unlit
material.
Media Player
The media player that will have its video texture displayed on the cube
Expansion_coeff
The value used during enabling to pad the edges to the video to prevent
bilinear bleed artifacts. Default is 1.01.
6.3.7 Audio Output Component
This component currently only supports the Windows platforms (UWP, Windows 8 and
above) and is used to pass audio from the Media Player to Unity. This allows audio
effects,3D placement and 360 VR spatialisers to be used. Fields are:
Media Player
The media player that will have its audio outputted via Unity
6.4 Scripting
44 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
6.4.1 Namespace
All scripts in this plugin use the namespace RenderHeads.Media.AVProVideo so be sure to
add:
using RenderHeads.Media.AVProVideo;
to the top of your source files.
6.4.2 Media Player Scripting
Most scripting is likely to center around the MediaPlayer.cs script. This script handles the
loading, playback and updating of videos. The script exposes a number of interfaces related
to different use cases and can be found in Interfaces.cs
MediaPlayer exposes 3 main interfaces:
Info Interface
The IMediaInfo interface is exposed by the Info property
This interface is used to access information about the media, eg:
MediaPlayer mp;
mp.Info.GetVideoWidth();
Control Interface
The IMediaControl interface is exposed by the Control property
This interface is used to control playback, eg:
MediaPlayer mp;
mp.Control.Pause();
TextureProducer interface
The IMediaProducer interface is exposed by the TextureProducer property
This interface is used to get information about how to display the current
texture and is used by the Display components, eg:
MediaPlayer mp;
Texture videoTexture = mp.TextureProducer.GetTexture();
The MediaPlayer script also has a number of methods for controlling loading of media:
OpenVideoFromFile()
Loads the video specified. Useful if you need manual control over when the
video is loaded
CloseVideo()
Closes the video, freeing memory
45 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
6.4.3 Events
MediaPlayer currently has these events:
MetaDataReady - Called when the width, height, duration etc data is available
ReadyToPlay - Called when the video is loaded and ready to play
Started - Called when the playback starts
FirstFrameReady - Called when the first frame has been rendered. (NB: This event
currently doesn’t get fired for certain browsers in the WebGL build. This includes all
non-mozilla non-webkit browsers)
FinishedPlaying - Called when a non-looping video has finished playing
Closing - Called when the the media is closing
Error - Called when an error occurred, usually during loading
SubtitleChanged - Called when the subtitles change
Stalled - Called when media is stalled (eg. when lost connection to media stream)
Unstalled - Called when media is resumed form a stalled state (eg. when lost
connection is re-established)
Scripting example:
// Add the event listener (can also do this via the editor GUI)
MediaPlayer mp;
mp.Events.AddListener(OnVideoEvent);
// Callback function to handle events
public void OnVideoEvent(MediaPlayer mp, MediaPlayerEvent.EventType et, ErrorCode
errorCode)
{
switch (et)
{
case MediaPlayerEvent.EventType.ReadyToPlay:
mp.Control.Play();
break;
case MediaPlayerEvent.EventType.FirstFrameReady:
Debug.Log("First frame ready");
break;
case MediaPlayerEvent.EventType.FinishedPlaying:
mp.Control.Rewind();
break;
}
Debug.Log("Event: " + et.ToString());
}
6.5 Platform Specific Scripting
6.5.1 Windows Store / UWP / Hololens
See the Demos/Scriptlets/NativeMediaOpen.cs script for details on how to load directly from
the native file system (eg from the Camera Roll)
6.6 Third-Party Integration
46 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
6.6.1 PlayMaker Support
AVPro Video includes over 64 components (actions) for PlayMaker, the popular visual
scripting system created by Hutong Games. This makes AVPro Video much easier to use
for PlayMaker users. The actions can be found in the /Scripts/Support/PlayMaker folder
6.6.2 NGUI Support
AVPro Video includes basic support for NGUI, the popular UI system from Tasharen
Entertainment. The ApplytoTextureWidgetNGUI component is used to set the AVPro Video
texture to an NGUI UITexture widget.
47 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
7. Asset Files
7.1 Demos
Demo_360SphereVideo.unity
Demo contains a video player that plays a 360
degree video using equirectangular(lat-long)
mapping.
The video is applied to a sphere, inside of which
is the main camera.
If the target device has a gyroscope then moving
the device around with rotate the camera to view
the video from different angles. For platforms
without gyroscope the mouse/touch can be used
to look around.
A special shader and script are used to allow a
single camera to render in stereo on a VR
headset. Click on the material to set whether it
should display the video as monoscopic, stereo
top-bottom or stereo left-right.
Demo_360CubeVideo.unity
Same as the sphere demo above, but using a
cubemap 3x2 layout source video.
Demo_BackgroundShader.unity
Basic demo that plays a video using the
background material which allows the video to
appear behind all content.
Demo_FrameExtract.unity
Shows go to read frames out of the video for
saving to disk (jpg/png) or accessing pixel data.
Demo_imGui.unity
Basic demo that plays a video and uses the legacy IMGUI display component
to draw the video to the screen.
Also has an audio clip to show audio-only media playback.
Also has 3 different streaming URLs to demonstrate streaming.
IMGUI is drawn on top of all other visual components.
Demo_Mapping3D.unity
Demo containing a video player and a 3D scene
Some of the 3D models have the video mapped to them via the
ApplyToMaterial script
Demo_Multiple.unity
This demo allows you to programmatically multiple load videos and test
multiple videos playing at once. Display is via the AVPro Video uGUI
component
Demo_uGUI.unity
This demo shows how to display videos within the uGUI system. It uses the
DisplayUGUI component in the canvas hierarchy.
It also uses a custom shader to overlay text with a video texture.
48 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Demo_VideoControl.unity
This demo shows how to query the video state and control playback
7.2 Prefabs
360SphereVideo.prefab
Prefab containing a video player and mapping to a sphere. Useful for
playback of equirectangular 360 degree videos
BackgroundVideo.prefab
Prefab containing a video player and a quad model with a special background
material applied. This material makes the quad get drawn before everything
else so it appears in the background.
FullscreenVideo.prefab
Prefab controls a video player and the IMGUI display component for very
easy basic video playback creation
7.3 Scripts
Components
ApplyToMaterial.cs
Applies the texture produced by the MediaPlayer component to a unity
material texture slot
ApplyToMesh.cs
Applies the texture produced by the MediaPlayer component to a
Untiy mesh (via MeshRenderer) by setting the mainTexture field of all
its materials
CubemapCube.cs
Generates a cube mesh that can be used for displaying a 3:2
cubemap packed video
DisplayBackground.cs
Displays the texture produced by the MediaPlayer component behind
all other content (not compatible with SkyBox)
DisplayIMGUI.cs
Displays the texture produced by the MediaPlayer component using
Unity’s legacy IMGUI system
DisplayUGUI.cs
Displays the texture produced by the MediaPlayer component using
Unity’s new uGUI system
MediaPlayer.cs
The main script for loading and controlling an instance of video
playback
UpdateStereoMaterial.cs
A helper script for VR stereo rendering to update the camera position
variable in a spherical material to help work out which eye to render
AudioOutput.cs
Used to play audio from the media vis Unity’s sound system (currently
Windows only)
ApplyToTextureWidgetNGUI.cs
49 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Applies the texture produced by the MediaPlayer component to an
NGUI Texture widget texture slot
Editor
DisplayUGUIEditor.cs
The editor script that controls how the DisplayUGUI component is
rendered in the Inspector
MediaPlayerEditor.cs
The editor script that controls of the MediaPlayer component is
rendered in the Inspector
Internal
AndroidMediaPlayer.cs
Android specific media player
BaseMediaPlayer.cs
Common base class for all platform media players
Interfaces.cs
Interfaces and events
NullMediaPlayer.cs
The fallback dummy media player for unsupported platforms
OSXMediaPlayer.cs
iOS and OSX specific media player
WebGLMediaPlayer.cs
WebGL specific media player
WindowsMediaPlayer.cs
Windows specific media player
50 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
8. Scripting Reference
AVPro Video is designed to be used mainly with the supplied drag and drop component but
there are always times when a bit of scripting is needed. The asset includes sample scenes
which give some examples of how to use scripting to control video playback, apply video
textures to materials etc which are useful to learn from. The full class reference is available
online here:
http://www.renderheads.com/content/docs/AVProVideoClassReference/
In this document we have included a simplified version of the highlights.
MediaPlayer class
The MediaPlayer class is the main class for video playback and is where video files are
specified and controlled. This class is mainly controlled via the Unity Inspector UI and for
scripting through the interface properties it exposes.
Properties
Events
returns the MediaPlayerEvent class
Info
returns the IMediaInfo interface
Control
returns the IMediaControl interface
TextureProducer
returns the IMediaProducer interface
Methods
All of these methods use the interfaces exposed above and are just handy shortcuts
void OpenVideoFromFile(FileLocation location, string path, bool autoPlay)
Opens the video specified
void CloseVideo()
Closes the current video and frees up allocated memory
void Play()
Starts playback of the video
void Pause()
Pauses the video
void Stop()
Pauses the video
void Rewind(bool pause)
Rewinds the video with an option to pause it as well
Texture2D ExtractFrame(Texture2D target, float timeSeconds, int timeoutMs)
Extracts a frame from the specified time of the current video as a readable
Texture2D. This can then be used to save out the pixel data. The texture
51 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
must be destroyed by the user. The texture can be passed in again via the
“target” parameter to reuse it.
IMediaInfo interface
This interface is used to query properties of the video
Methods
float GetDurationMs();
Returns the duration of the video in milliseconds
int GetVideoWidth();
Returns the width of the video in pixels
int GetVideoHeight();
Returns the height of the video in pixels
float GetVideoFrameRate();
Returns the frame rate of the video in frames per second
float GetVideoDisplayRate();
Returns the actual frame rate achieved by the video decoder
bool HasVideo();
Returns whether the media has visual tracks
bool HasAudio();
Returns whether the media has audio tracks
int GetAudioTrackCount();
Returns the number of audio tracks
string GetPlayerDescription();
Returns a string describing the internal playback mechanism
IMediaControl interface
This interface is used to control loading and playback of the video
Methods
bool OpenVideoFromFile(string path);
Starts loading the file from the specified path or URL. Returns false if any
error was encountered. This function is asynchronous so the video properties
will not be available immediately. This function shouldn’t be used, instead use
the MediaPlayer OpenVideoFromFile function.
void CloseVideo();
Closes the video and any resources allocated
void SetLooping(bool looping);
Sets whether the playback should loop or not. This can be changed while the
video is playing.
bool CanPlay();
Returns whether the video is in a playback state. Sometimes videos can take
52 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
a few frames before they are ready to play.
void Play();
Starts playback of the video
void Pause();
Pause the video
void Stop();
Stops the video (essentially the same as Pause)
bool IsPlaying();
Returns whether the video is currently playing
bool IsPaused();
Returns whether the video is currently paused
bool IsFinished();
Returns whether the video has completed playback
bool IsLooping();
Returns whether the video has been set to loop
bool IsBuffering();
Returns whether a streaming video has stopped and is buffering. A buffering
video will resume after it has downloaded enough data.
void Rewind();
Sets the current time to the beginning of the video
void Seek(float timeMs);
Sets the current time to a specified value in milliseconds
void SeekFast(float timeMs);
Sets the current time to a specified value in milliseconds but sacrifices
accuracy for speed. This is useful if you just want to jump forward/back in a
video but you don’t care about the accuracy.
bool IsSeeking();
Returns whether the video is currently seeking. During seeking no new
frames are produced.
float GetCurrentTimeMs();
Returns the current time (playback position) in milliseconds
void SetPlaybackRate(float rate);
Sets the current playback rate. 1.0f is normal rate. Negative rates aren’t
supported on all platforms.
float GetPlaybackRate()
Returns the current playback rate
void MuteAudio(bool mute)
Sets the audio mute or not
void SetVolume(float volume)
Sets the volume between 0.0 and 1.0
float GetVolume()
Returns the volume level between 0.0 and 1.0
int GetCurrentAudioTrack()
Returns the index of the currently enabled audio track
void SetAudioTrack(int index)
Sets the index to select the audio track to enable exclusively
float GetBufferingProgress()
Returns a value between 0.0 and 1.0 representing network buffering
53 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
ErrorCode GetLastError()
Returns an error code is an error occurred this frame
IMediaProducer interface
Methods
Texture GetTexture();
Returns a Unity Texture object if there is a texture available otherwise null is
returned.
int GetTextureFrameCount();
Returns the number of times the texture has been updated by the plugin.
This can be useful to know when the texture was updated as the value will
increment each time.
long GetTextureTimeStamp();
Returns the presentation time stamp of the current texture in 100-
nanosecond units. This is useful for accurate frame syncing.
bool RequiresVerticalFlip();
Some textures are decoded up-side-down and need to be vertically flipped
when displayed. This method returns whether the texture needs to be flipped
during display.
54 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
9. Supported Media Formats
In general the most common format that is supported are MP4 files with H.264 encoding for
video and AAC encoding for audio. This format is supported across all platforms though not
necessarily all bit-rates and profiles.
Container support:
Windows
Desktop
Mac OS X
Desktop
iOS, tvOS
Android
MP4
Yes
Yes
Yes
Yes
MOV
Yes
Yes
Yes
No
AVI
Yes
No
No
No
MKV
Yes in Windows
10
?
?
Yes
Android
5.0+
Webm
Yes in Windows
10 - 1607
Anniversary and
above
No
No
Yes
ASF/WMV
Yes
No
No
No
MP3
Yes
Yes
Yes
Yes
WAV
Yes
?
?
?
Audio Codec support:
Windows
Desktop
Mac OS X
Desktop
iOS, tvOS
Android
AAC
Yes
Yes
Yes
Yes
MP3
Yes
Yes
Maybe*****
Yes
FLAC
Yes in Windows
10
No
No
Yes
AC3
Yes
Yes
?
?
WMA
Yes
No
No
No
MIDI
Yes
?
?
?
55 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Vorbis
No
?
?
Yes
Opus
Yes in Windows
10 - 1607
Anniversary and
above
No
No
Yes,
Android
5.0+
ALAC (Apple Lossless)
No
Yes
Yes
No
µLAW
Yes
Yes
Yes
No
ADPCM
Yes
Yes
Yes
No
Linear PCM
Yes
Yes
Yes
Yss
Video Codec support:
Windows
Desktop
Mac OS X
Desktop
iOS, tvOS
Android
HEVC / H.265
Yes in Windows
10
Yes in High
Sierra and
above
Yes in iOS
11 and
above
Yes
H.264
Yes****
Yes
Yes
Yes
H.263
Yes
?
?
Yes
MJPEG
Yes
No
No
No
WMV
Yes
No
No
No
VP8
Yes*
No
No
Yes
VP9
Yes*
No
No
Yes
Hap
Yes***
Yes
No
No
Hap Alpha
Yes***
Yes
No
No
Hap Q
Yes***
Yes
No
No
Hap Q Alpha
Yes***
Yes
No
No
ProRes 422
No
Yes
No
No
ProRes 4444
No
Yes
No
No
DV
Yes
Yes in
Yosemite
and above
No
No
Lagarith
Yes, with codec
No
No
No
56 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Uncompressed RGBA
Yes
?
?
?
Uncompressed YUV
Yes
?
?
?
Uncompressed R10K
No
Yes in
Yosemite
and above
No
No
Uncompressed V210
?
Yes in
Yosemite
and above
No
No
Uncompressed 2VUY
?
Yes in
Yosemite
and above
No
No
For Windows, other codecs can be played if the DirectShow mode is used and 3rd party
codecs are installed. We recommend LAV filters for this purpose as this adds support for
almost all codecs.
* Yes, only in Windows 10 and only 4:2:0. Native VP9 support only comes in Yes in
Windows 10 1607 Anniversary Update and above, but it may be available before that
via Intel GPU drivers. If you use DirectShow and 3rd party filter then 4:4:4 can be
supported. Using Media Foundation no audio codecs (Vorbis or Opus) are supported
and will cause the video to fail to load if included.
*** Requires option “Force DirectShow” to be ticked
**** Older versions of Windows (Vista and XP) do not have support for H.264
decoding
***** iOS seems to often have trouble with MP3 audio tracks in a video file, so best to
use AAC instead
Cells with “?” are one’s we’re not sure about. We will do more testing and continue to
update this table. For more details on which codecs and what type of encoding is best, see
the per-platform details below.
9.1 Android
Android supports many media formats. For a complete list check the Android MediaPlayer
documentation here: https://developer.android.com/guide/appendix/media-formats.html and
the ExoPlayer documentation here:
https://google.github.io/ExoPlayer/supported-formats.html
HEVC (H.265) support was officially added in Android 5.0 (Lollipop) but only as a software
decoding implementation on older devices.
We have found that using GearVR on Samsung Galaxy S6 and S7 that H.265 codec works
best, with a resolution of 3840x1920 at 30fps, or 2048x2048 at 60fps.
57 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
A list of media-player related Android chipsets and which formats they support for hardware
decoding: http://kodi.wiki/view/Android_hardware
9.2 iOS, tvOS and OS X
9.2.1 iOS
Many media formats are supported by iOS including H.264. iOS 11 adds support for H.265
(HEVC). The iPhone 7 is the first device with the hardware to support H.265.
The iPhone5C is a low-powered mobile device and has a limitation that the videos must be
less than 2 megapixels in resolution.
iOS seems to often have trouble with MP3 audio tracks in a video file, so best to use AAC
instead.
iOS also seems to allocate a lot of system memory when loading videos. Unfortunately
we’re not able to control this which means you have to be very careful when trying to load
large or multiple videos, especially with the 1GB iOS devices. To combat this we have
added the YCbCr video decoding option (See Platform Specific panel in MediaPlayer
component), which uses much less memory.
We recommend only using a single MediaPlayer instance and reusing it for all video loads.
The memory issue may be helped by encoding your videos with less reference frames but
we haven’t tested this for sure.
It has proven difficult getting the true video decoding capabilities of iOS devices. Apple’s
website has information, but we found it to be slightly inaccurate (for example we can
decode 4K video on iPhone5s, which apparently can only do 1080p). It seems that if your
device has a 64-bit processor then it will be able to decode 4K H.264, but older devices with
32-bit processors will not.
Device specs according to Apple:
Device
MPEG-4
H.264
MJPEG
AAC-LC
Dolby
Audio
iPad 4th
generation
(Nov 2012)
Maximum
Resolution:
640x480 at
30fps, Simple
profile. Up to
1.5Mbps
In M4V, MP4
and MOV
containers
Maximum
Resolution:
1920x1080 at
30fps, High
profile up to
level 4.1
In M4V, MP4
and MOV
containers
Maximum
Resolution:
1280x720 at
30fps,
Up to 35 Mbps
In AVI
container with
PCM audio
160 Kbps
48kHz stereo
In M4V, MP4
and MOV
containers
Unsupported
iPhone 6
(Sept 2014)
Maximum
Resolution:
640x480 at
30fps, Simple
profile. Up to
1.5Mbps
In M4V, MP4
and MOV
containers
Maximum
Resolution:
1920x1080 at
60fps, High
profile up to
level 4.2
In M4V, MP4
and MOV
containers
Maximum
Resolution:
1280x720 at
30fps,
Up to 35 Mbps
In AVI
container with
PCM audio
160 Kbps
48kHz stereo
In M4V, MP4
and MOV
containers
Unsupported
58 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
iPad 5th
generation
(March 2017
iPad 9.7inch
iPad Pro
iPhone 6S
(Sept 2016)
iPhone SE
(March 2016)
iPhone 7
(Sept 2016)
Maximum
Resolution:
640x480 at
30fps, Simple
profile. Up to
1.5Mbps
In M4V, MP4
and MOV
containers
Maximum
Resolution:
3840x2160 at
30fps, High
profile up to
level 4.2
In M4V, MP4
and MOV
containers
Maximum
Resolution:
1280x720 at
30fps,
Up to 35 Mbps
In AVI
container with
PCM or ulaw
stereo audio
160 Kbps
48kHz stereo
In M4V, MP4
and MOV
containers
Up to
1008Kbps,
48kHz, stereo
or
multi-channel
In M4V, MP4
and MOV
containers
iPod 6
Unsupported
Maximum
Resolution:
1920x1080 at
30fps ,Main
Profile level
4.1
In M4V, MP4
and MOV
containers
Unsupported
160 Kbps
48kHz stereo
In M4V, MP4
and MOV
containers
Unsupported
iPad 4 supported formats according to Apple are from here:
https://support.apple.com/kb/sp662?locale=en_US
iPad 5 supported formats according to Apple are from here:
https://support.apple.com/kb/SP751?locale=en_US
iPad 9.7inch supported formats according to Apple are from here:
https://www.apple.com/ipad-9.7/specs/
iPad Pro supported formats according to Apple are from here:
https://www.apple.com/ipad-pro/specs/
iPod 6 supported formats according to Apple are here:
https://support.apple.com/kb/SP720?locale=en_US&viewlocale=en_US
9.2.2 macOS
Many media formats are supported by macOS including H.264, H.265, ProRes 422 and
ProRes 4444.
macOS Yosemite adds support for
DV
Uncompressed R10k
Uncompressed v210
Uncompressed 2vuy
macOS High Sierra adds support for
H.265 / HEVC
59 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
9.3 Windows
A full list of supported formats can be found here:
https://msdn.microsoft.com/en-us/library/windows/desktop/dd757927(v=vs.85).aspx
https://msdn.microsoft.com/en-us/windows/uwp/audio-video-camera/supported-codecs
H.264 decoder supports up to profile L5.1, but Windows 10 supports above L5.1 profile:
https://msdn.microsoft.com/en-us/library/windows/desktop/dd797815(v=vs.85).aspx
H.265 decoder specs are here:
https://msdn.microsoft.com/en-us/library/windows/desktop/mt218785(v=vs.85).aspx
Windows 10 adds native support for the following formats:
H.265 / HEVC
MKV
FLAC
HLS Adaptive Streaming
MPEG-DASH
Windows 10 Fall Update seems to remove native H.265 / HEVC support for some users and
requires them to download the (free) HEVC Video Extension . Before update KB4056892 (4
Jan 2018), users also had to open a H.265 video in the Films & TV app after a restart before
AVProVideo could play H.265 videos. This update seems to fix that however.
9.4 Windows Phone / UWP
Details on media supported by this platform can be found is platform are here:
https://msdn.microsoft.com/library/windows/apps/ff462087(v=vs.105).aspx
https://msdn.microsoft.com/en-us/windows/uwp/audio-video-camera/supported-codecs
9.5 WebGL
Support for WebGL platform is still varied and depends on the platform and browser support.
Some formats such as AVI file container are not supported at all. As with all other platforms,
H.264 video in an MP4 container is the most widely supported format.
Adaptive streaming (such as HLS) is still not supported natively by most browsers, but we
have seen it working in the Microsoft Edge and Safari browsers.
For best compatibility make sure to force WebGL 1.0 by going to Player Settings > Other
Settings > Auto Graphics API and removing WebGL 2.0. Failure to do so can make videos
on Chrome not render.
HLS and MPEG-DASH are not natively supported on all browsers. We have added the
ability to include 3rd party javascript libraries to handle these (dash.js and hls.js). See the
streaming section for how to implement these.
60 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
On newer versions of Safari videos are not allows to auto-play unless given permission by
the user (in the preferences menu). This doesn’t affect videos that have no audio track so
this may be a workaround. More details can be found here:
https://webkit.org/blog/7734/auto-play-policy-changes-for-macos/
Some resources about the supported formats:
https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_media_formats
https://en.wikipedia.org/wiki/HTML5_video#Browser_support
http://www.encoding.com/html5/
10. Support
If you are in need of support or have any comments/suggestions regarding this product
please contact us.
Email: unitysupport@renderheads.com
Website: http://renderheads.com/product/avpro-video/
Unity Forum:
http://forum.unity3d.com/threads/released-avpro-video-complete-video-playback-solution.38
5611/
10.1 Bug Reporting
If you are reporting a bug, please include any relevant files and details so that we may
remedy the problem as fast as possible.
Essential details:
Error message
The exact error message
The console/output log if possible
If it’s an Android build then an “adb logcat” capture
Hardware
Phone / tablet / device type and OS version
Development environment
Unity version
Development OS version
AVPro Video plugin version
Video details:
Resolution
Codec
Frame rate
Better still, include a link to the video file
Better still, send us a full or reduced copy of your Unity project
61 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
11. About RenderHeads Ltd
RenderHeads Ltd is an award-winning creative and technical company that has been
designing and building cutting edge technology solutions since its formation in 2006. We
specialise in creating interactive audio-visual software for installations at auto shows,
museums, shows and expos.
11.1 Services
Unity plugin development and consulting
Bespoke software development:
High-end technology for events and product launches
Interactive installations and educational museums
Video walls
Virtual reality experiences
Augmented reality apps
11.2 Careers at RenderHeads
We’re always looking for creative technical people to join our team. RenderHeads is an
innovative software company that develops interactive experiences lead by high-end
technology and beautiful graphics. We have created installations for Nike, Ford, Nissan, Dell,
Intel, PwC, Shell, Cisco, and others. Our work also appears in the National Maritime
Museum Greenwich, Museum of Science and Industry Manchester, Sheikh Abdullah Al
Salem Cultural Centre Science Museum Kuwait and Guiness Storehouse Ireland.
We create games for museums and brands, bespoke AR, VR and huge video wall
experiences for shows, real-time visualisations for events and audio-visual products for
developers and the broadcast industry. All of the software we develop is unique, and our
clients have high expectations. We work with some of the latest hardware and software
technologies, and each project gives us a new opportunity to explore the edge of possibility.
RenderHeads offer a flexible working opportunity with remote or in-office working. We have
offices in Cape Town, South Africa and Glasgow, Scotland, and team members working
remotely from around the world.
62 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
If you’re looking for a new opportunity to push the limits and create awesomeness, please
read through the requirements below and email us at jobs@renderheads.com . Send us your
CV and links to showreel, past projects or code examples.
General Requirements
You must be able to show a history of making software, either by professional
experience or personal projects
Pragmatic software development approach - we ship software frequently
Either very strong technical skills, or a mix of creative and technical skills
Good communication skills - most of our communication is online, via
Slack/Skype/Hangouts and email/Google Docs
Positions we have available:
Video Software Developer
You would be developing in-house camera and video related software,
including our AVPro series of Unity plugins, related to cross-platform video
playback, streaming, capture and encoding
Required experience:
Strong C++
Multi-threaded programming
Ideal experience:
C# and Unity
AVFoundation / Media Foundation / DirectShow / libAV / ffmpeg /
gstreamer
DeckLink / NDI SDK
Low-level MP4 / H.264 / H.265 / HEVC / HLS / DASH / RTSP / RTP
Shader development
Camera / broadcast experience
360 video workflows
360 audio technologies
Interactive Software Developer
You would be creating software for a wide variety of technically challenging
and creative projects, and implementing cool interactive experiences using
the latest technologies
Features of our typical projects:
Most development is in Unity
Visualisation and effects
Educational games
VR experience development
Integration with cameras, sensors and hardware
UI and animation development
Required experience:
C# and Unity
SVN / Git
Ideal experience:
63 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
iOS / Android development
An interest in UI and UX
An interest in improving workflow / tools
Shadertoy and other graphics rendering tech
Ability to travel (for on-site installations)
Senior Software Developer
You would oversee the progress of the other developers in your team: making
sure everyone is on track, helping to instill good development practices,
helping to grow everyone’s experience and skills
You would tackle some of the more difficult programming problems and make
sure that the final software is of high quality
You would also help to scope and plan the approaches for new projects,
working closely with the other senior members
Required experience:
Many years of software development
Many projects/products released
Optimisation
Unity and C#
Ideal experience:
Graphics programming
An interest in improving workflow / tools (Jenkins, CI, Dev ops)
11.3 Our Unity Plugins
Many of the apps and projects we develop require features that Unity doesn’t yet provide, so
we have created several tools and plugins to extend Unity which are now available on the
Unity Asset Store. They all include a free trial or demo version that you can download
directly from the website here:
http://renderheads.com/product-category/for-developers/
11.3.1 AVPro Video
Powerful cross-platform video playback solution for Unity, featuring support for
Windows, OS X, iOS, Android and tvOS.
11.3.2 AVPro Movie Capture
Video capture to AVI files direct from the GPU and encoded to files using DirectShow
codecs. Features include 8K captures, lat-long (equirectangular) 360 degree captures,
off-line rendering and more. Windows only.
11.3.3 AVPro Live Camera
Exposes high-end webcams, tv cards and video capture boards to Unity via DirectShow.
Windows only.
64 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
11.3.4 AVPro DeckLink
Integrates DeckLink capture card functionality into Unity, allowing users to send and receive
high-definition uncompressed video data to and from these capture cards.
65 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Appendix A - Frequently Asked Questions (FAQ)
1. Why won’t my high-resolution video file play on Windows?
The ability to play high resolution videos depends on the version of Windows
operating system and which video codecs you have installed.
By default AVPro Video will try to use the standard Microsoft codecs that come with
Windows. These have limitations, for example:
Decoder
Windows 7 and
below
Windows 8+
h.264
1080p
2160p (4K)
h.265 (HEVC)
unsupported
2160p (4K)
or 4320p (8K) if your
video driver supports
If you want to use 3rd party codecs you can install them to improve the range of
resolution and format options available. These come in the form of Media
Foundation or DirectShow codecs. The LAV Filters are a great example of
DirectShow codecs that will allow for higher resolution video decoding.
2. Does Time.timeScale affect video playback speed?
Yes we have BETA support for time.timeScale and time.captureFramerate. Simply
enable the option on the Media Player component in the panel labelled “Global
Settings”.
3. Does AVPro Video support playing YouTube videos and live streams?
Yes and no. If you enter a YouTube URL into AVPro Video it will not be able to play it
because this is the URL to the website and not the video. It is possible to gain
access to the underlying MP4 file or HLS stream URL which will work with AVPro
Video. This may be against the terms and conditions of YouTube though. We have
heard that some users have experimented with using the youtube-dl
https://rg3.github.io/youtube-dl/ command-line tool to extract the video URL and then
use that for playback within AVPro Video.
4. How can I get smoother video playback for my Windows VR app?
This is usually caused by the video decoding taking too much CPU time. Some
videos (especially highly compressed ones) require a lot of CPU time to decode each
frame.
66 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Try enabling the hardware decoding option in “Platform Specific” panel. This will
enable your application to use GPU decoding which is much faster. This option is
only supported on Windows 8.1 and above and when D3D11 graphics API is used.
You could also try re-encoding your video using settings that are less demanding for
the decoder. Most H.264 encoders have a ‘fast decode’ preset, or you can manually
change settings such as disabling CABAC, reducing the number of reference frames,
reducing bitrate, disabling B-frames, disabling the loop (deblocking) filter etc.
You could also try switching to another video codec that allows for less CPU intensive
decoding. Codecs such as H.264 / H.265 are generally very heavy on the CPU. Hap
is an extremely fast codec but will result in large files.
5. Is GPU hardware decoding available?
Yes it is on most platforms. Android, iOS, tvOS and macOS mostly use GPU
decoding - but it depends on the codec. For Windows GPU decoding is enabled by
default but is only available for Windows 8.1 and above and when using D3D11
graphics API. You can toggle GPU decoding is enabled via the Platform Specific
panel:
6. Is multi-channel audio supported?
Audio with more than 2 channels should be supported on desktop platforms. On
Windows you will need to set your sound settings in the Control Panel for the number
of channels you want to output. If you leave your Windows sound settings as stereo
then the audio will get mixed to stereo.
7. Why isn’t seeking accurate / responsive?
The way seeking works depends on the platform. Some platforms support frame
accurate seeking but most will only seek to the nearest key-frame in the video. In
general to improve seek accuracy and responsiveness you can re-encode your
videos with more frequent key-frames (smaller GOP). The downside of this is that
67 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
the size of the video file will increase (more key-frames = larger file).
We have two seek functions: Seek() and SeekFast(). SeekFast can be used when
accuracy isn’t important but whether it’s faster or not depends on support from the
platform and can vary based on the video codec used.
Platform / API
Accurate Seeking Supported
Windows Desktop / DirectShow
Yes (but depends on codec used, native
Microsoft H.264 decoder doesn’t
support)
Windows Desktop / Media Foundation
Yes
Windows Store / Phone / UWP
Yes
Android / MediaPlayer
No (but coming in Android “O”)
Android / ExoPlayer
Yes
macOS
Yes
iOS / tvOS
Yes
WebGL
Depends on browser
8. Windows - Why isn’t my stereo top-bottom or left-right video displaying
correctly?
If your stereo video is only decoding one of the views then this is because there is
stereo metadata encoded within the video which causes our Media Foundation
decoder to interpret incorrectly. The only way to currently solve this is to remove the
metadata from the video. One way to do this is using FFMPEG:
ffmpeg -i %1 %1-video.nut
ffmpeg -i %1-video.nut -crf 10 %1-stripped.mp4
We have also had reports that this works:
ffmpeg -i in.mp4 -map_metadata -1 -c:v copy -c:a copy out.mp4
9. Windows - Why do videos take a long time (1-2 seconds) to open when using
DirectShow?
68 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
We have seen this happen when the Nvidia Geforce Experience is installed and the
Share option is enabled (ShadowPlay). Disabling the share option or uninstalling this
software resolves this issue.
10. Windows - Why doesn’t my H.265 8K video play when I make a build?
This is probably because your build is 32-bit. Try making a 64-bit build. See our
other notes above for other 8K requirements.
11. Laptop - Why can’t my powerful laptop decode a 8K video?
Laptops can have multiple GPUs and will often default to the built-in GPU (eg Intel)
instead of the more powerful discrete GPU (eg NVidia / AMD). Try going to the
NVidia Control Panel to make the discrete GPU the default and then reload Unity or
your application.
12. Hap Codec - Why don’t my Hap encoded videos play?
If your platform is Windows desktop then this is usually because the DirectShow
video API has not been selected. See the notes above about how to use the Hap
codec.
13. Scrubbing - Why is seeking/scrubbing unresponsive when using a Slider GUI
component?
This is a common mistake where the OnValueChanged event in the Slider
component is set to trigger a method that calls Seek() on the video. The problem is
that OnValueChanged doesn’t only trigger when the user is interacting with the slider,
but also when the video is playing normally and is updating the slider position. So
the Update method might be changing the slider position based on the video position,
which then causes OnValueChanged to trigger, which causes a seek. You need to
guard against this by checking that the user is interacting with the slider, perhaps by
adding OnPointerDown and OnPointerUp events. OnPointerDown you would seek to
that position and also set a flag. Then in OnValueChanged you would only seek if
that flag was set. In OnPointerUp you would unset the flag.
14. macOS - Errors when using Unity Cloud Build
Make sure that you set the correct XCode version in the Cloud Build settings. Try
XCode 8.3.3 as this is the version we currently use for building the plugin.
69 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
15. macOS - Publishing for Mac App Store and get error “Invalid Signature”?
We’re not sure why this happens but we’ve had reports that simply deleting all of the
.meta files inside the AVProVideo.bundle will resolve this issue.
16. macOS - Why is the video playback very jumpy when I make a build, but fine in
the editor?
We have seen this happen when using the Metal graphics API. Try going to Player
Settings, unticking the option “Auto Graphics API for Mac” and make “OpenGLCore”
at the top of the list.
17. iOS / tvOS - Why doesn’t my video file play properly?
Make sure that your video file doesn’t contain an MP3 audio track. In our experience
iOS doesn’t handle MP3 audio correctly and can often lead to the whole file not
loading/playing correctly. Use AAC for audio instead.
18. iOS - Can I playback a video from the Videos library?
Yes if the video is not content protected and you have it's URL. URL's for items in
the video library take the following form:
ipod-library://item/item.m4v?id=1234567890123456789
You can get the URL of an item by using the MPMediaQuery class from the
MediaPlayer framework.
19. iOS - Can I playback a video from the Photo Library?
Yes. You need to use the UIImagePickerController to select the video. This will
result in the video being processed into your app's temp folder where you can access
it via the URL returned from the controller.
20. iOS / tvOS - Why doesn’t my HLS m3u8 stream play?
First you should check the XCode console for error messages. This can give you a
clue about the problem.
Next you should run your HLS manifest through the Media Stream Validator Tool
( https://developer.apple.com/library/archive/technotes/tn2235/_index.html ) provided
by Apple. iOS and tvOS are VERY strict when it comes to streaming video, so it is
possible that a stream will play on macOS, but be refused on iOS or tvOS.
21. iOS - How can I fix the error message: "Missing Push Notification Entitlement -
Your app includes an API for Apple's Push Notification service, but the
aps-environment entitlement is missing from the app's signature" ?
70 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
If you are not using Push notifications in your app, this Unity Asset does the job of
fixing this error: https://www.assetstore.unity3d.com/en/#!/content/59160 . Just import
it, and build the xcode files again.
22. iOS - How can I fix the error message: "This app attempts to access
privacy-sensitive data without a usage description. The app's Info.plist must
contain an NSCameraUsageDescription key with a string value explaining to
the user how the app uses this data." ?
Open the Info.plist file in xcode, and manually add "NSCameraUsageDescription"
(Privacy - Camera Usage Description), with a value explaining the reason why you
are using the webcam. In our case, this was caused by using the Google Cardboard
code and that needs the webcam to read the Cardboard device QR code, so we
added the following value "Requires camera for reading Cardboard QR code".
23. Android - Why doesn’t my huge video file play from StreamingAssets folder?
Files in the StreamingAssets folder for Android are packed into a JAR file and so
before we can play them we must extract them to the apps persistent data folder.
Huge files can take a long time to extract and copy and sometimes they could even
cause the device to run out of storage space or memory. For really large files we
recommend placing them the videos in other folders on the device and then
referencing the absolute path to that file. This also has the added benefit of not
having a copy huge files and wait ages when deploying builds to the Android device.
24. Android - Why does the adb logcat receive so many logs “getCurrentPosition”
from MediaPlayer-JNI?
Android’s MediaPlayer logs this to verbose channel, simply change your logging filter
to ignore verbose logs.
25. Android - Why doesn’t the video display when using Vuforia?
In our test Vuforia doesn’t play well with AVPro Video when multi-threaded rendering
is enabled. Simple disable this option in the Player Settings screen.
26. Android - Why does my build fail with a message about DEX and zip classes?
This can happen when there is a conflict in the Java code. We include a JAR called
zip_file.jar which contain classes for handling zip files. If you have another plugin in
your project that also contains these classes then Unity will fail the build. Read the
exact Unity error message and if it is related to zip classes then you can try deleting
zip_file.jar file.
Or you may need to upgrade from JDK 1.7.0 to 1.8.0
27. Cardboard/Google VR - Why is the image shaking?
71 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
We’ve had reports that this is caused by enabling the option “track position” in the
Gvr head.
28. Windows 10 - Why doesn’t my H.265 / HEVC video play?
Microsoft removed the HEVC codec that was previously included in Windows during
the Fall Creators Update in April 2018, Windows 10 version 1709.
Currently in Windows 10 if you want to play HEVC content you either need to open a
HEVC video in the default Film & Movie app, or you need to install this extension:
“HEVC Video Extensions from Device Manufacturer”
https://www.microsoft.com/en-us/p/hevc-video-extensions-from-device-manufacturer/
9n4wgh0z6vhq
29. macOS / iOS / tvOS - Why doesn’t my H.265 / HEVC video play?
Apple is very strict with the 4 character (FOURCC) ‘tag’ that is used to identify the
codec of the video, and only accept videos with the HVC1 tag, whereas many video
encoders embed the tag HEV1, which would cause the video not to play on Apple
platforms.
To fix this you either need to be using encoding software that lets you specify the
HVC1 tag, or use a tool such as FFMPEG to change the tag for you using this
command-line:
ffmpeg -i video.mp4 -c:v copy -tag:v hvc1 -c:a copy video-hvc1.mp4
30. Why is my video displaying with the wrong rotation?
Videos recorded from mobile devices contain orientation metadata. Some platforms
parse this metadata during decoding and produce the texture in the correct final
orientation, while others don’t and we have to correct it in C# or in the shader. We do
handle this for most cases, however if you are using the ApplyToMesh or
ApplyToMaterial on macOS, iOS and tvOS then this rotation will not be applied.
There are a few ways to fix this:
1) Re-encode the video with the rotation metadata removed and with the video
encoded in its final orientation. Using FFMPEG you can do this via:
ffmpeg -y -i %1 -metadata:s:v:0 rotate=0 %1-norotation.mp4
2) Detect the orientation of the video and apply a rotation of the texture coordinates
in the shader:
Orientation ori = Helper.GetOrientation(_media.Info.GetTextureTransform());
Matrix4x4 oriMatrix = Helper.GetMatrixForOrientation(ori);
_material.SetMatrix("_texRotation", oriMatrix);
72 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
and then the shader define the matrix variable:
uniform float4x4 _texRotation;
and apply the matrix rotation in the shader shader via something like:
o.uv.xy = mul(_texRotation, v.texcoord.xy).xy;
You may also have to change the texture wrapping mode in MediaPlayer > Visual >
Wrap to Repeat
If you are using WebGL, then we have found that most browsers (Edge and Chrome)
pre-transform the video so that it is in the correct orientation, as there is no
mechanism in HTML 5 to retrieve the video orientation/transform. We have found
that Firefox doesn’t support this though.
31. iOS - Why do I get this message about privacy permission using camera?
Some users have reported getting this message:
“This app attempts to access privacy-sensitive data without a usage description. The
app's Info.plist must contain an NSCameraUsageDescription key with a string value
explaining to the user how the app uses this data.”
This seems to be a bug in Unity 5.4.1 and has been resolved in Unity 5.4.2.
32. Will you support MPEG-DASH or another adaptive streaming system?
MPEG-DASH is currently supported on Windows 10, Windows UWP 10 and Android
(when using ExoPlayer API) only. We hope to improve support for these sorts of
streaming systems eventually. The plugin already supports some basic streaming
protocols and for now we’re focusing on basic features before we look at advanced
features such as these. HLS adaptive streaming is supported on most platforms.
33. Android - Why is my video playback freezing?
We have found that H.264 videos with GOP = 1 (all I-Frames / Key-frames) freeze on
Android. GOP interval of 2 seems to be fine.
34. Windows - Why can’t I stream HLS at 4K in Windows 10?
Windows 10 (before Anniversary update) seems to limit the stream resolution based
on your screen resolution and the monitor DPI scaling setting. To get 4K using HLS
you need a 4K monitor and your DPI scaling set to 100%. This issue now seems to
be resolved in the latest version of Windows 10.
35. Windows - Why does the video control react correctly in Windows 7, but not in
Windows 8 or above?
73 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
If you try to call video control function (eg Seek, Mute, SetVolume etc) just after you
open the video file, this can cause problems. Video loading can be synchronous or
asynchronous, so you need to wait until the video has completed loading before
issuing any control commands. In Windows 7 and below the DirectShow API is used
by default. Currently the open function works synchronously, so control functions will
work immediately after loading. In Windows 8 and above the Media Foundation API
is used by default, and this is asynchronous. The best approach is to use the event
system to wait for the video to be ready (contain meta data), or to poll the
MediaPlayer (Control.HasMetaData()) before issuing control functions.
36. Why does my HLS / RTSP stream created with FFMPEG work so badly in AVPro
Video?
Some users have reported issues with creating a stream with FFMPEG where the
stream will take a long time to connect and will only send through frames
occasionally. We have found that you need to manually add the GOP size
parameters to be a multiple of your frame-rate. So for example if your frame-rate is
25 then you should add “-g 25” in your FFMPEG command-line.
37. Android - How do I authenticate my RTSP stream?
RTSP on Android is currently only supported when using the MediaPlayer API. You
should be able to pass in your username and password in the URL using the
following format:
rtsp://<user>:<password>@<host>:<port>/<url-path>
Another way is to pass it via HTTP header fields. We also support this. In the
MediaPlayer component, go to Platform Specific > Android section and use the
JSON HTTP HEADER field to put something like this JSON:
{
"SERVER_KEY_TO_PARSE_USER_NAME":"username",
"SERVER_KEY_TO_PARSE_PASSWORD":"password"
}
38. Windows - Why is there a slight colour shift between software and hardware
decoding?
We’re not completely sure why this is. It appears that sometimes that hardware
decoder decodes the video slightly darker in places. On the whole the effect is very
small, so it only affects videos that contain data that must be reproduced exactly. We
have found that the following FFMPEG command does improve the colour matching.
ffmpeg -i video.mp4 -y -vf
"scale=in_range=tv:out_range=tv:in_color_matrix=bt601:out_color_matri
x=bt709" out.mp4
74 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
39. Windows Audio - Why does my HLS/DASH stream audio have gaps or play at
the wrong rate when using the AudioOutput component?
This currently can happen when using adaptive streams as we can’t determine the
sample rate, and thus cannot set up a resampler to convert to the sample rate Unity
is using internally. To solve this you need to set Unity’s sample rate in the Audio
Settings to match that of the stream (eg 44100).
40. Audio - Why does my multi-track audio file play all tracks at once?
This can happen when all of the audio tracks are set to default. You need to encode
the video with only one of the tracks set to default. Using FFMPEG you can do this:
ffmpeg -i video_noaudio.mp4 -i audio_esp.m4a -i audio_jap.m4a -i
audio_eng.m4a -map 0 -map 1 -map 2 -map 3 -disposition:a:1 none
-disposition:a:2 none -codec copy output.mp4
41. Which DRM content protection schemes does the plugin support?
See the DRM section in this documentation.
42. How do I flip my video horizontally?
a. In the MediaPlayer component, in the Media Properties section, set the wrap
mode to "Repeat".
b. If you're using the ApplyToMesh component, then you just need to set the
Scale X value to -1
c. If you're using the ApplyToMaterial component, then you just need to set the
Scale X value to -1
d. If you're using the DisplayIMGUI component, then this doesn't have any way
to flip video without editing the script.
e. If you're using the DisplayUGUI component, then just set the UVRect W value
to -1.
43. Windows - Why doesn’t the plugin support Windows XP SP2 and below?
Modern Visual Studio compilers require XP SP3 as a minimum. One of the reasons
for this is builds reference the function GetLogicalProcessorInformation() in
Kernel32.dll which isn’t present in XP SP 2. The lack of this function causes our DLL
to fail to load. More information here:
https://connect.microsoft.com/VisualStudio/feedback/details/811379/
44. How do I use a video in a reflection probe?
If you want a video to appear as part of a reflection in your scene, then you can add a
reflection probe, set to update in real-time and refresh every frame.
Then add geometry to your scene and use the ApplyToMesh component to apply the
video to that geometry. If you are using a equirectangular video then you can apply
75 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
the video to our OctahedronSphere mesh and use a material on it with the
InsideSphere Unlit shader. Then as long as your reflection probe is inside this
sphere, it should collect the video.
If you don't want this geometry to appear in your scene, then you can either move the
geometry and reflection probe to another area of your scene (you will then have to
set the Anchor Override on your MeshRenderer ot make it use that reflection probe),
or create a new Layer name and assign it to the geometry and reflection probe, also
setting the Culling Mask of the reflection probe so it only renders that geometry.
Then in your other cameras you will need to exclude that layer from the camera
culling mask list.
45. How do I use a video as a skybox?
If you want the video to appear as the Skybox, create a new material that uses
Skybox/Panoramic shader, then go to Lighting Settings and assign this material to
the Skybox field. Then add a ApplyToMaterial component and assign it to this
material and to the mediaplayer. The video should now render to the default skybox
in your scene. You can also leave the Lighting Settings default Skybox field as
default, and just add a Skybox component to your camera to override which skybox
material it uses.
46. What’s the difference between your different Unity video playback plugins?
We previously had 3 video playback plugins for Unity:
a. AVPro Video
b. AVPro Windows Media (deprecated August 2016)
c. AVPro QuickTime (deprecated May 2016)
Here is a table giving a rough comparison of the plugin features:
AVPro Video
AVPro Windows Media
AVPro QuickTime
First Released
2016
2011
2011
Windows XP-7
Yes**
Yes
Yes (with QT
installed)
Windows 8-10
Yes
Yes
Yes (with QT
installed)
Windows UWP
Yes
No
Windows Phone
Yes
No
OS X
Yes
No
Yes
Android
Yes
No
No
iOS
Yes
No
No
tvOS
Yes
No
No
76 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
WebGL
Yes
No
64-bit
Yes
Yes
No
2K H.264
Yes
Only with 3rd party
codec
Yes
4K H.264
Yes
Only with 3rd party
codec
Yes but very slow
8K H.265
Yes
Only with 3rd party
codec
No
Streaming URL
Yes
Not really
Yes a bit
Hap Codec
Yes
Yes
Yes
MP4 Container
Yes
Only with 3rd party
codec
Yes
Works with VR
Yes
Yes best with 3rd party
codecs
Not great
ASync Loads
Yes
No
Yes
Transparency
Yes
Yes
Yes
Speed
Fast
Medium, fast with 3rd
party codecs
Medium
** Currently only using DirectShow path, adding Media Foundation path soon.
77 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Appendix B - Release History
Version 2.0.0 - Alpha (INTERNAL)
General
Huge code cleanup and refactoring
New MediaPlayerUI demo
Added assembly definition (asmdef) files to improve compile times
and organisation
Dropped support for all Unity versions before before Unity 5.6.6
Android
iOS / tvOS
macOS
Windows
New Windows RT plugin, uses the latest Microsoft video APIs and
offers a lot of improvements, especially for streaming media
For Windows RT added an option to select the highest bitrate when an
adaptive stream starts
New Windows RT plugin fixes previous issue where stereo videos
containing stereo metadata didn’t display correctly
Added support for AES HLS encryption
Dropped support for Windows Phone and Windows Store 8.1, now we
only support the UWP (10) Windows platforms
Version 1.9.12 - 28 January 2019
Android
Fixed typos in OES shader introduced in 1.9.10
WebGL
Fixed issue where compile errors would happen when building with
Development Mode
Windows
Fixed crash bug when running in 32-bit
General
Fixed script warnings and errors showing in Unity 2018.3.x
Version 1.9.10 - 10 January 2019
Known Issues
Android OES shader is broken, fixed in 1.9.12
Android
Fixed a crash bug with ExoPlayer that would happen randomly when
loading or closing a video
Added HIGH_QUALITY option to OES shader which fixes the issue of
equirectangular videos having distortion at the poles
Added texture transform to the OES shader
General
78 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Fixed bug where UVRect didn't work in DisplayGUI when in
ScaleAndCrop mode
Version 1.9.8 - 13 December 2018
Windows
Fixed a rare crash bug that could happen when closing multiple videos
at once
Version 1.9.7 - 11 December 2018
macOS / iOS / tvOS
Fixed rare memory leak when rendering with Metal where a texture
was not freed after a seek operation on multi-resolution HLS media
Android
Added support for automatic rotation of portrait videos when using
ExoPlayer
Added support for linear color-space to OES rendering path
Windows
Fixed the device name for the Oculus Rift audio device
Fixed Facebook Audio 360 support for playing audio to a specific
named device
General
Fixed a shader issue where stereo top-bottom videos wouldn’t display
the correct eye on Windows and Android
Other minor improvements
Version 1.9.6 - 16 November 2018
macOS
Fixed a Metal rendering issue where the watermark didn’t display in
the editor
Fixed issue with Hap and Metal rendering in the editor
Upgraded to latest Hap version (1.6.9)
iOS / tvOS
Fixed a crash in iOS 9 and lower that could happen when closing a
player
Fixed packed transparent shaders to expand range correctly when
using YpCbCr mode
Android
Fixed bug where using the OES shader would cause video to display
pink/black, this was a regression introduced in 1.9.5
General
Shaders refactored
Version 1.9.5 - 7 November 2018
Known Issues
A regression in our Android OES shader causes videos to appear
black/pink when using OES mode - Fixed in 1.9.6
Custom shaders may not compile due to removal of
Convert420YpCbCr8ToRGB - Fixed in 1.9.6
macOS / iOS / tvOS
79 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Fixed issue where videos would appear too bright when using Linear
colorspace and YpCbCr mode
Improved support for different video colour transforms(Rec601,
Rec709)
Convert420YpCbCr8ToRGB shader function removed and replaced
with ConvertYpCbCrToRGB
Android
Fixed a crash and memory leak bug regression introduced in 1.9.3,
caused videos not to release resources when a player is destroyed
Windows
Fixed issue where using Low Latency option and Unity Audio would
cause the application to lock up
Version 1.9.4 - 25 October 2018
Known Issues
A regression in our Android implementation causes videos not to
release resources when a player is destroyed - Fixed in 1.9.5
iOS / tvOS
Fixed issue where subsequent video loads would sometimes fail to
display frames
Version 1.9.3 - 22 October 2018
Known Issues
A regression in our Android implementation causes videos not to
release resources when a player is destroyed - Fixed in 1.9.5
Android
Fixed Oculus Go texture wrap mode being set to repeat instead of the
default clamp. This was due to a new firmware update that allowed
different texture wrap modes on OES textures
Added support for SeekFast() to ExoPlayer which seeks to the closest
keyframe
Various robustness improvements related to resource freeing and
plugin shutdown for Android apps that contain a Unity player instance
Built with new version of Android Studio (3.2)
macOS / iOS / tvOS
Added support for handling key requests for encrypted HLS streams
(eg using AES). Supports direct key setting and key retrieval using
authentication token via HTTP header. See DRM section for details.
Added new SetPlayWithoutBuffering(bool) method to allow playback
to start immediately, bypassing the API buffering duration logic
WebGL
Added support for mip-map texture generation
General
Fixed regression in 1.9.2 causing some events not to fire, in particular
the subtitle change event
Fixed issue where videos would open twice when calling
OpenVideoFromFile() while AutoLoad was enabled and Start() hadn’t
run yet
80 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Many improvements to PlaylistMediaPlayer, including StartMode and
ProgressMode options, improved UI and per-item transitions
Version 1.9.2 - 2 October 2018
Known Issues
A regression in the the general event handler can cause some events
not to fire. Fixed in 1.9.3
Android
Fixed rare crash bug when using ExoPlayer due to commands being
called in the render thread
Added file offset support when using ExoPlayer and loading from
StreamingAssets folder
iOS
Fixed issue where some HLS streams wouldn’t play on older iOS
devices
Fixed XCode 8/9 compatibility issue
WebGL
Fixed support for WebGL 2.0
General
Exposed options for automatically pausing / playing the media when
application pauses
Exposed a mask to control which events the MediaPlayer should
trigger. This is a minor optimisation to allow event logic to be skipped.
Fixed a minor issue with OnApplicationPaused and
m_WasPlayingOnPause
Fixed a logic bug where a non-auto-play video loaded from a prefab
wouldn’t play after calling Play()
Fixed a minor issue with PlaylistMediaPlayer where
stereo/alpha/loop/path state wasn’t set correctly
Version 1.9.1 - 7 September 2018
macOS and iOS
Fixed bug where video would always auto-play (only iOS < 10.0 and
macOS < 10.12)
Fixed bug where texture properties (wrap, filter etc) weren’t being set
WebGL
Fixed regression introduced in 1.9.0 which prevents videos from
loading
Fixed support for 3rd party decoder libraries
Added support for “blob:” URL scheme
General
Resampler now honours texture properties set in MediaPlayer
Added position bar to PlaylistMediaPlayer editor
Windows Desktop
Fixed focus, volume and mute on videos using FB 360 Audio
Fixed some issues with switching between instances of videos using
system audio and FB 360 Audio
Fixed a bug where audio sample rate and channel count were
81 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
swapped around
Version 1.9.0 - 23 August 2018
Known Issues
A regression in the WebGL player causes videos to not load, fixed in
version 1.9.1
Android
Upgraded Facebook Audio 360 to version 1.6.0 from 1.5.0
Upgraded ExoPlayer to 2.8.4 from 2.8.2
ExoPlayer is now the default video API instead of MediaPlayer
Changed Android API level requirement to 15 from 16 when using
MediaPlayer API (ExoPlayer still requires 16)
Windows Desktop
Upgraded Facebook Audio 360 to version 1.6.0 from 1.5.0
WebGL
Added support for 3rd party decoder libraries such as dash.js and
hls.js, see documentation for details
Added warning for unsupported WebGL 2.0
General
Added new (beta) PlaylistMediaPlayer component which allows
seamless playlist progression and has a number of optional
transitions, and a new demo scene using it
Added a new demo scene and video showing stereo 360 playback
with spatial audio (on Windows and Android only)
Minor
Resampler now has some support for videos with unknown frame rate
Sphere demo no longer has camera affected by VR positional head
tracking
Added demo scriptlet to show how to toggle stereo mode via script
Version 1.8.9 - 9 July 2018
Android
Upgraded ExoPlayer to version 2.8.2 from 2.8.1
Fixed audio glitches on Oculus Go when using Facebook Audio 360
by increasing the default spatial buffer size
Reduced garbage generation by caching frequently used methods
Windows UWP / Hololens
Fixed an issue where non-debug builds would crash on some versions
of Unity
WebGL
Added notes to AVProVideo.jslib on how to integrate dash.js
General
Reduced some garbage generation in various components
Improved shader support for multi-view stereo rendering
Version 1.8.8 - 26 June 2018
Android
Upgraded ExoPlayer to version 2.8.1 from 2.6.0
Upgraded Facebook Audio 360 to version 1.5.0 from 1.4.0
82 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Fixed audio track 1 not being set initially on ExoPlayer
Removed unneeded debugging and logging code on ExoPlayer
Windows Desktop
Upgraded Facebook Audio 360 to version 1.5.0 from 1.4.0
Fixed bug where videos using FB Audio 360 wouldn’t stop at the end
of the video
Fixed a bug where using D3D9 and DirectShow and resizing the
window or switching fullscreen mode would cause a crash
macOS / iOS / tvOS
Fixed an issue in Unity 2018.1 and above that caused the projects not
to compile
WebGL
Fixed issue where Buffering and Stall events weren’t triggered
correctly
General
Fixed garbage generation in DisplayUGUI component
Added support for Stall event on platforms that don’t natively support it
Added documentation notes on seeking and playback rate
Version 1.8.7 - 16 May 2018
macOS
Fixed a bug in the HapQ colour conversion
WebGL
Fixed compile issue in Unity 2017.4.1 by removing unused method
and variable
Windows
Fixed a bug where FinishedPlaying event wouldn’t get retriggered
after the first time when playing a WAV or MP3 audio file
Now when emulating mobile builds in the editor, texture size
limitations are ignored to allow videos to play locally
General
Fixed a bug with the 180 video shader where left-right stereo packed
videos appear behind the viewer
Fixed the SSL certificate validation code in HLSStream so it doesn’t
ignore bad certificates
Version 1.8.6 - 13 May 2018
macOS, iOS, tvOS
Fixed issue where playback wouldn't resume after setting playback
rate to zero
Fixed crash issue when playing back HLS streams on iOS/tvOS
versions older than 10.0
Fixed issue playing some unescaped URLs
Android
Added option to force a streaming format using Exoplayer regardless
of the file extension
Added option to prefer software decoders using ExoPlayer
Fixed bug where video wasn't pausing correctly when seeking and the
83 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
app is suspended
Windows
Fixed rare runtime error when audio is disabled in Unity
General
Fixed stretching issue in 180 degree video display and added a new
mapping layout field in the Visual section for specifying that the video
is using equirectangular 180 degree layout
Fixed null reference exception in ApplyToMaterial script when no
material is assigned
Added buffered progress rendering to the 03_Demo_VideoControls
sample scene timeline
Moved the MediaPlayer debug display option into a separate
component called DebugOverlay
Added new option to UpdateStereoMaterial component so allow
stereo rendering to force just the left or right eye
PlayMaker actions now have a logo in the header
Shaders updated to use a common function for vertex transform
Version 1.8.5 - 5 April 2018
macOS, iOS, tvOS
Fixed issue since iOS 11 where live HLS streams refused to go higher
than the device native resolution
Fixed a potential crash bug on old iOS versions (before 10.0) when
querying audio or video bitrate
General
Fixed minor bug where newly loaded subtitles wouldn’t immediately
update
Version 1.8.4 - 23 March 2018
macOS, iOS, tvOS
Fixed crash bug introduced in 1.8.0 related to seekableTimeRanges in
HLS streams
Android
Added OES shader support for the GUI component
Version 1.8.2 - 19 March 2018
Known Issues
There is a crash bug on macOS, iOS and tvOS when closing down a
HLS stream. This is fixed in 1.8.4
Windows
Improved accuracy of the Stalled state/event which triggers when the
player is unable to decode the next frame, usually due to network
connection issues
Improved accuracy of IsBuffering state/event which occurs when data
is being downloaded over the network
Fixed a bug with HLS live streams where pausing the stream or a
dropped connection would stop streaming indefinitely. The stream
now resumes correctly, or throws an error if the stream cannot be
resumed
84 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Version 1.8.0 - 14 March 2018
Known Issues
There is a crash bug on macOS, iOS and tvOS when closing down a
HLS stream. This is fixed in 1.8.4
Android
Performance improvement by minimising texture copy operations
Upgraded ExoPlayer from 2.5.4 to 2.6.0
Upgraded Facebook Audio 360 from 1.3.0 to 1.4.0
New method GetCurrentDateTimeSecondsSince1970() for getting the
absolute date-time of a video stream
New method GetSeekableTimeRanges() returns an array of seekable
time ranges
Fixed bug where only up to 255 video player instances could be
created
Fixed issue where camera set to skybox clear mode would make OES
videos not appear, by changing shader to write to the depth buffer, just
like the other VR shaders do
OES mode now uses less memory and starts faster
Fixed bug when using Facebook Audio 360 where sometimes the
audio or video wouldn’t play
macOS, iOS and tvOS
New method SeekWithTolerance() offers fine seek control
New method GetCurrentDateTimeSecondsSince1970() for getting the
absolute date-time of a video stream
New method GetSeekableTimeRanges() returns an array of seekable
time ranges
Fixed YCbCr support in shader used by 180 video demo
Windows
Improved adaptive stream resolution switch seamlessness by
eliminating the black frame flash
Added DirectShow chunk-based file loading method to allow for
loading large files into memory without making large managed buffer
allocations. See LoadFromBufferInChunks.cs
Fixed a bug when loading from buffer using DirectShow where
memory was deleted twice
Fixed a D3D9 rendering bug that caused only the skybox to render on
some newer versions of Unity due to renderstate restore issues
Upgraded Facebook Audio 360 from 1.3.0 to 1.4.0
Various improvements when using the AudioOutput component,
including better AV sync and fixed glitching. Also added manual
sample rate setting for streaming videos.
Improved UWP file loading demo script NativeMediaOpen.cs
WebGL
Changed the way textures are updated to hopefully be more
compatible
General
DisplayUGUI component now exposes Raycast Target field so videos
85 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
can allow touch/mouse input to pass through
Added new events for video resolution change, stream stalling,
buffering and seeking
Added SSL/HTTPS support to StreamParser
Various code refactoring and cleanup
Version 1.7.5 - 22 January 2018
macOS
Fixed build issue with Unity 2017.3 related to the post build script
Fixed build issue with Unity 2017.3 related to universal/64-bit plugin
metadata
macOS, iOS and tvOS
Fixed a build issue with Unity 5.6.x related to a missing zlib
dependency
Android
Fixed stereo support in OES shader
Fixed support for IsBuffering state in ExoPlayer
Windows desktop
Fixed a rare bug when using DirectShow and Unity 2017.3 that would
cause the video texture to display as a single channel or black
Fixed a performance bug in the DirectShow player that meant a
paused video would continue to upload frames to the GPU
Version 1.7.4 - 28 December 2017
WebGL
Added support for URL’s that start with “chrome-extension://”
Fixed recent regression causing video to not render when not
rendering anything else
iOS
Plugin rebuilt with XCode 8.3.3 to work around linker missing
framework issue when building Unity apps using XCode 8 and
previous versions of the plugin that were compiled with XCode 9
Fix to the ApplyToMaterial component when using YCbCr mode so the
default texture gets displayed correctly when the MediaPlayer is
closed
General
Added support for Facebook’s “360-capture-sdk” cubemap layout to
the CubemapCube component
Version 1.7.3 - 9 December 2017
Windows
Fixed a bug when using D3D9 that would cause the DirectShow video
to either render black or cropped. This was a regression since version
1.7.0
Fixed DirectShow loading from buffer support for 32-bit builds
WebGL
Added support for error events
Fixed issue where resampler could be enabled when it is not
supported for WebGL
86 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
General
Optimised SRT subtitle file parse to be much faster
Version 1.7.2 - 30 November 2017
Known Issues
There is a bug that causes video to appear black or cropped, only
when using Windows desktop with D3D9. This is fixed in 1.7.3
Android
Fixed two major GLES bugs which sometimes prevented the video
appearing. This mostly affected Galaxy S8 devices, Lenovo tablets
and anyone using the Vuforia plugin
Added support for accurate seeking when using MediaPlayer API -
requires Android 8.0 Oreo
Added support for HTTP/HTTPS redirects when using ExoPlayer API
macOS, iOS and tvOS
Added support for switching between multiple audio and video tracks
Added support for querying audio and video track bitrate
Windows Desktop
Added the ability to set a list of preferred decoders for DirectShow.
This makes it easier to predict which decoder installed on the user's
system will be used. For example they could have many codec packs
installed, but you prefer to use the Microsoft decoders for stability
When using the Use Unity Audio option, the video will no longer fail to
load if the audio is not able to pass through to Unity
Fixed a bug where Facebook Audio 360 channel mode wouldn’t be
passed through to the plugin
Fixed a shader bug causing Main10 videos in DirectShow to display
too dark
General
Added new HLSParser class to help parse HLS m3u8 manifests. This
allows for the discovery of all of the streams available, their
resolutions and bitrates
Improved Facebook Audio channel mode enumeration UI
Version 1.7.0 - 15 November 2017
Known Issues
There is a bug that causes video to appear black or cropped, only
when using Windows desktop with D3D9. This is fixed in 1.7.3
Facebook Audio 360
Added Facebook Audio 360 support for spatial audio in VR. Currently
we have support for Windows desktop (Rift, Vive etc) and Android
(Cardboard, GearVR, Daydream etc). Uses MKV videos with Opus
audio tracks encoded using Facebook Audio 360
Windows Desktop
Added Facebook Audio 360 support
Added “Alpha Channel Hint” option to DirectShow for cases where
alpha channel is needed - currently required by Lagarith codec
Added native NV12 support for legacy D3D9 with DirectShow - this a
87 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
performance optimisation, but also fixes some issues when using
subtitle rendering DirectShow filters (such as DirectVobSub) which
would cause the image to become flipped
Added Hap Q support for legacy D3D9 with DirectShow
Fixed a memory leak when an invalid Hap video (non multiple of 4
resolution) was loaded
Fixed an issue where AudioOutput wouldn’t redirect audio to another
device (eg Rift Audio) for streaming media
Fixed “Low Latency” option so it is actually used
Windows UWP
Fixed “Low Latency” option so it is actually used
Android
Added Facebook Audio 360 support (requires ExoPlayer API)
Added ExoPlayer video API option which can offer improved format
support, including MPEG-DASH and in the future will include more
control over streaming video
iOS / tvOS
The YCbCr optimisation is now enabled by default
WebGL
Fixed issue with Safari on macOS calling Play() with default auto-play
permission disabled as outlined here:
https://webkit.org/blog/7734/auto-play-policy-changes-for-macos/
General
Added new frame resampler option which can be used to get
smoother playback by buffering frames and presenting them at the
right interval. This feature does use more memory and is still in beta
Fixed a bug where the FinishedPlaying event didn’t fire when a
second stream was played on the same MediaPlayer
Fixed a bug where Instantiating ApplyToMaterial would cause a null
exception
Fixed compile error in Unity 2017.2 related to mobile multi-threading
Fixed warnings in Unity 2017.2 related to API changes to
playModeStateChanged
Changed shader line endings to Unix to prevent mixed line ending
warnings
Inspector UI improvements
Documentation updates
Version 1.6.15 - 5 October 2017
Windows
Fixed an issue mainly affecting Windows mixed reality and VR
headsets using “hybrid-gpu” mode where more than one GPU is
available to the system
Fixed the WSA 8.1 x86 DLL that didn’t get recompiled since 1.6.12
Version 1.6.14 - 23 August 2017
macOS / iOS / tvOS
Fixed issue where audio-only media would not update current time
88 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Improved error reporting
iOS
Fixed texture error when using Unity 4.x
Android
Fixed file offset not working for videos in StreamingAssets folder. Until
now file offset only worked for files at other locations in the file system
Windows
Fixed regression introduced in 1.6.12 that would cause DirectShow
videos to not play when using Direct3D9 or OpenGL
AudioOutput component now handles different sample rates and
channel counts set in Unity’s audio settings and automatically detects
number of audio channels in source media
Version 1.6.12 - 6 July 2017
Known Issues
On Windows when using DirectShow and Direct3D9 or OpenGL,
videos do not display and VRAM runs out. This is a regression
introduced in this version and is fixed in 1.6.14
Android
Fixed a rare crash bug that could happen during seeking
Windows Desktop / UWP
Added new low latency option for Media Foundation which provides a
hint to the decoder to do less buffering
Windows Desktop
Added support for 10-bit HDR video decoding (Main10) when using
DirectShow any suitable decoder such as LAV Filters. The decoding
still resolves to an 8-bit texture but we plan to add 10-bit or floating
point texture resolve soon
Fixed a crash bug when using DirectShow and LoadFromBuffer to
load a video from memory
General
ExtractFrame() method now faster and more accurate
Time Scale support (where a non-realtime video capture solution
changes Time.timeScale or time.captureFrameRate) has been
improved, making it faster and more accurate
Added new 180 degree sample video
Fixed a bug in AudioOutputManager when channel count <= 0
Version 1.6.10 - 14 June 2017
iOS
Improved support for iOS 8.0 by fixing an issue with CoreVideo
Android
Fixed issue where loading over 255 videos would cause videos to no
longer load correctly
Windows Desktop / UWP
The AudioOutput component now supports routing audio to a specific
channel, or to multiple channels. It also now supports multiple
instances so you can more easily have audio coming from different
89 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
world positions.
Fixed a memory leak when using AudioOutput component
Windows Desktop
Added new function to load media from memory for DirectShow API.
This is demonstrated in the scriptlet LoadFromBuffer.cs
General
Started adding a new 180 degree video playback demo, it just lacks a
media file which will come in the next release.
Videos with auto-start enabled no longer fire the ReadyToPlay event.
Added new method ForceUpdate() to ApplyToMesh component which
refreshes the material when changing media properties such as stereo
packing.
Version 1.6.8 - 31 May 2017
iOS / tvOS
No longer requires a separate plugin file for Unity 5.3 and below. New
bootstrap C code handles the change in Unity’s plugin API.
Improved Metal compatibility with older versions of Unity
General
The AudioOutput component has some new experimental changes,
including the ability to have multiple AudioOutput instances and to
remap which channels the audio outputs to
Version 1.6.6 - 25 May 2017
General
PlayMaker (the popular visual scripting system for Unity) is now
supported by AVPro Video. We have added over 64 new PlayMaker
actions for seamlessly integration into your PlayMaker workflow
Added audio stereo balance support
Added new stereo packing mode that uses a pair of UV sets for
custom left and right mapping.
Added new Support & Help window to make reporting issues easier
and to include helpful diagnostic information
Added documentation about 3rd-party library usage
Added documentation about iOS video decoders
Windows Desktop
Added multi-threaded decoding for chunked Hap videos
Windows Store / UWP / Hololens
Added new function to load media from native file locations using
IRandomAccessStreamWithContentType. This means you can now
play media from the camera roll. See the example script in
Demos/Scripts/Scriptlets/NativeMediaOpen.cs for details of usage.
Android
Fixed lost GLES context issue that can happens at least when
switching Google Cardboard on/off
Fixed one of the OES shaders that was failing to compile on Android
due to undefined TRANSFORM_TEX macro
Refactored some of the low-level code
90 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
iOS
Added support for headphone connection changes. Unplugging /
connecting headphones will now correctly set state of the video so
that IsPaused() and IsPlaying() return the correct value
Fixed a rare bug in the VCR demo script which caused the
FinishedPlaying event not to fire due to triggering a Seek() at the end
of an HLS stream
iOS / tvOS / macOS
Fixed frame flash issue due to FirstFrameReady event firing too early
due to frame count increment bug
Version 1.6.4 - 5 May 2017
Windows Desktop
Fixed an intermittent bug introduced in version 1.6.2 which caused
videos to sometimes not appear or appear black
Version 1.6.3 - 3 May 2017
Windows Desktop
Fixed mistake from 1.6.2 which included a Visual Studio 2015
dependency (the OpenMP vcomp140.dll)
Chunked Hap decoding is now no longer multi-threaded. We will add
this optimisation back in 1.6.6
Added some notes to documentation about Windows N / KN editions
Known Issues
There is an intermittent bug in Windows desktop which causes videos
to not appear or to appear black sometimes - fixed in 1.6.4
Version 1.6.2 - 25 April 2017
iOS / tvOS
Added support for Y'CbCr Biplanar textures which results in less
memory usage. This feature means older devices with only 1GB of
RAM (iPhone 5+, iPhone6+) will no longer struggle with memory when
playing back 4K video content.
Tweaks which hopefully fix a rare flickering issue for Metal when using
multi-threaded rendering
Fixed shader compile issue for legacy Unity 4.x
91 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
macOS
Fixed bug for high resolution Hap decoding due to apparent bug in
Apple OpenGL extension
Fixed an issue with transparent ProRes video
Android
Fixed bug that would cause video to not display when using the Mask
uGUI component
Fixed OES GLSL shader support for Unity 5.5.2
Fixed shader compile issue for legacy Unity 4.x
Reported bug in Unity 5.6.0f3 support for GLSL (OES) shaders
Windows
Performance improvement for Hap codec when using “chunks”
encoding option. Now decoding happens over multiple threads.
Added support for the “-force-d3d11-no-singlethreaded” command-line
parameter which doesn’t require the use of a new D3D context with a
shared texture in the MF-Hardware path
Media Foundation code refactored
General
Fixed bugs in NGUI script, previously it could crash when changing
videos
Fixed very specific bug related changing texture quality settings while
the MediaPlayer is disabled which would cause texture to become
black. Removed the global setting that disabled the above texture
quality fix as it no longer has a performance benefit
Improved UI of ApplyToMesh component with custom editor
Some shaders now default to black textures instead of white
Added basic playback controls to the debug UI
Added documentation about AMD hardware decoding (UVD)
capabilities
Added support for Unity 2017 beta
Known Issues
Windows desktop - this release includes a Visual Studio 2015
OpenMP dependency by mistake (vcomp140.dll) which means that
the Visual Studio 2015 runtime must be installed. Fixed in 1.6.3
There is an intermittent bug in Windows desktop which causes videos
to not appear or to appear black sometimes. Fixed in 1.6.4
Version 1.6.0 - 29 March 2017
iOS / tvOS
Added support for linear color-space (available in Unity 5.5 and
above)
Performance improvements, especially when reusing MediaPlayer
component to load videos
iOS simulator support added
macOS
Added support for Hap Q Alpha format (HapM)
Performance improvements, especially when reusing MediaPlayer
component to load videos
92 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Android
Fixed OES rendering path support in Unity 5.6
Improved linear colour-space support (available in Unity 5.5 and
above)
Performance improvement due to alternative way to get around
checking GetNativeTexturePtr() and instead checking for when texture
quality level changes
Removed unused AndroidManifest.xml file
Windows
Performance improvement due to alternative way to get around
checking GetNativeTexturePtr() and instead checking for when texture
quality level changes
General
Fixed DisplayIMGUI component depth value not working
Improved IMGUI linear color-space conversion function
Version 1.5.28 - 24 March 2017
Windows
Added support for HapQ+Alpha format
Fixed Unstalled event not firing
macOS / iOS / tvOS
Added support to ExtractFrames demo to handle videos containing
rotation
Minor C# performance improvements
WebGL
Fixed support for Safari browser
Added documentation about supported browsers
General
Added some DRM notes to the documentation
Fixed PlayMaker example script
Version 1.5.26 - 14 March 2017
Android
Added support for passing custom HTTP header fields
Fixed minor bug when calling SetPlaybackRate too soon
Android API 15 (4.0.4 Ice Cream Sandwich) is now supported,
previously API 16 (4.1 Jelly Bean) was required
Added plugin files for the arm64-v8a platform (currently disabled in the
inspector, but simply enable it once Unity has added support for this
architecture)
macOS / iOS / tvOS
Added support for passing custom HTTP header fields
Added support for videos containing orientation data. The orientation
transform is now retrievable and the uGUI display component correctly
displays the video
WebGL
Fixed bug in Internet Explorer and Firefox where video would not
display
93 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Added support for displaying multiple ranges of buffered time
Windows / WSA / Hololens
Fixed FinishedPlaying rarely not firing on Media Foundation
General
DisplayUGUI component no longer generates any garbage when
vertices are dirtied. Also some minor optimisations
ApplyToMesh component no longer generates any per-frame garbage
ApplyToMesh component has improved interface
Improved time display in the editor to be millisecond accurate, and
also fixed time rounding error bug
Changed EnableSubtitles() to return true while it is loading in a
coroutine
Added new example script VideoTrigger that shows how to trigger
videos when entering/exiting collider volumes, including audio fading
Added more streaming video URLs to documentation
Version 1.5.25 - 20 February 2017
Windows Desktop
Patch fix for regression in 1.5.24 which caused DirectShow videos to
display in green
Version 1.5.24 - 19 February 2017
General
Shaders improved to also work on older hardware that only support
shader model 2.0
Windows Phone / WSA / UWP / Hololens
Added support for audio to be played through Unity’s audio system
Fixed a serious bug which could cause video to not appear and native
plugin functions to return incorrect values
Fixed the hardware decoding option not being passed through to the
plugin correctly
Fixed support for older GPUs that only supported D3D11 feature set
9.x
Trial version watermarking optimised for speed and memory
Windows Desktop
Fixed support for older GPUs that only supported D3D11 feature set
9.x
Trial version watermarking optimised for speed and memory
Known Issues
Regression in this version causes Windows DirectShow decoded
videos to display green. Fixed in v1.5.25
Version 1.5.22 - 14 February 2017
General
Improved performance of CubemapCube component. Due to a typo it
would previously regenerate the cube mesh each frame.
Improved the fix for Unity 5.4.2 and above where changing texture
quality during runtime would cause the video texture to turn black.
The expensive fix now doesn’t apply to Unity 5.4.1 and 5.4.0, and can
94 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
be disabled in the global options
Fixed bug in subtitles that would cause them to go blank for a frame
Workflow
Android OES path now displays hint about using correct shaders
Error message display is more detailed
Improved user feedback when selecting stereo packing to warn about
the need for the UpdateStereoMaterial component
Improved UI for UpdateStereoMaterial component to give more
feedback about its setup
Android
Better reports error when a video fails to decode due to insufficient
resources
Added option to not show the “poster” frame, leading to faster start
and less resource usage for loaded videos. A video that is loaded but
not played and not displaying the poster frame will not engage the
decoder hardware and use up resources
Saved some memory in OES mode by no longer allocating
unnecessary render texture
Improved playback rate timer
WebGL
Fixed issue where no video displays when using WebGL 2.0 (default
in Unity 5.5.0 and above)
Cached texture pointer for improved performance
Fixed subtitle loading by changing to use a coroutine
Windows
Added new events for when a stream has stalled and resumed -
useful for detecting connection loss when live streaming
Added new option to generate texture mipmap levels automatically -
this reduces aliasing when the texture is scaled down
Updated Snappy library (used in Hap codec) to latest 1.1.4 which has
20% decompression speedup
Fixed bug in DirectShow where calls to SetMuted() or setting very low
audio volume would be ignored
UWP / Windows Phone / Hololens
Added new option to generate texture mipmap levels automatically -
this reduces aliasing when the texture is scaled down
Breaking Changes
For Android if you previously loaded videos in a paused state to see
the initial “poster” frame now you’ll need to tick the new “Show Poster
Frame” option in the Platform Specific settings. In previous versions
this was enabled by default, but now it is disabled by default as it has
performance implications.
Version 1.5.20 - 25 January 2017
General
Added more notes to the Hap codec documentation
Added more notes to the Android OES documentation
95 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Android
Fixed fast OES rendering path in Unity 5.4.x and 5.5.x
Fixed various stereo rendering issues
Fixed OES GLSL shaders failing to compile due to inline keyword
Fixed OES path displaying debug GUI panel
Fixed OES shader compiling on Google Daydream platform
Fixed Unity 5.6beta GearVR stereo rendering issue
Disabled minor Daydream stereo eye shader optimisation as it was
causing too many problems with older versions of Unity
Windows
Fixed bug where changing playback rate when playback has finished
would result in an error
Fixed the DirectShow audio volume level so it matches that of Media
Foundation
Version 1.5.18 - 18 January 2017
General
Added support to UpdateStereoMaterial for DisplayUI component so
that uGUI can render stereo videos correctly
WSA / UWP / Hololens
Fixed a script compile error introduced in 1.5.16
Version 1.5.16 - 16 January 2017
General
Optimised stereo left-right shader for Google VR support
Improved shaders to use UnityObjectToClipPos() instead of
UNITY_MATRIX_MVP to give better support for Unity 5.6 and above
Fixed bug in stereo left-right shader introduced in 1.5.14
Fixed various bugs in subtitle support
Android
Fixed subtitles not loading
Known Issues
Regression from 1.5.14
WSA and UWP platforms will fail to compile due to new
Thread.Sleep() which isn’t defined on these platforms.
Version 1.5.14 - 9 January 2017
General
Added subtitle support for external SRT files
Added a new demo showing the new subtitle support
Improved 03_Demo_VideoControls demo scene to use two
MediaPlayers for seamless playlisting
Fixed issue where initial playback fields (volume, mute, playback rate,
looping) could be reset to incorrect values in certain cases such as
such AutoOpen option was disabled
Shader improvements such as inlining for performance, sRGB to
Linear shader corrections and optimisations
iOS
Added documentation note about using AAC instead of MP3 for the
96 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
audio tracks
WSA / UWP / Hololens
Fixed bug where video would often not appear, due to audio track
being loaded first and reporting metadata loaded. This appears to be
a bug/change in the new Anniversary UWP Media Foundation version.
Plugin DLL file sizes are smaller
Windows Desktop
Plugin DLL file sizes are smaller
Known Issues
Regression from 1.5.12
Bug in stereo shader causing only left eye to render
Subtitles
New subtitle feature doesn’t load files from StreamingAssets
on Android and WebGL
Bug in script don’t allow subtitles to be loaded via code
Version 1.5.12 - 12 December 2016
WSA / UWP / Hololens
Fixed crash issue when using Unity 5.5
Version 1.5.10 - 9 December 2016
macOS
Fixed issue where OpenVideoFromFile() called multiple times quickly
would cause a crash
Removed PLIST modification from the PostProcess build script
iOS
Fixed issue where OpenVideoFromFile() called multiple times quickly
would cause a crash
Fixed issue with Unity 5.5.x which caused XCode project not to run
due to missing Metal symbol (only affects watermarked trial version)
tvOS
Fixed issue where OpenVideoFromFile() called multiple times quickly
would cause a crash
Fixed issue with Unity 5.5.x which caused XCode project not to run
due to missing Metal symbol (only affects watermarked trial version)
Android
Fixed issue with Unity 5.4.2 and above where changing the
TextureQuality level in the QualitySettings would cause the texture to
go black
Windows
Fixed issue with Unity 5.4.2 and above where changing the
TextureQuality level in the QualitySettings would cause the texture to
go black
General
Added support to uGUI shaders for RectMask2D
ApplyToMesh component now works with other renderer components
(eg Skinned Mesh)
Updated documentation and FAQs
97 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Version 1.5.8 - 23 November 2016
macOS
Fixed issue with OpenGLCore where texture wouldn’t update due to
GL state not being restored correctly when using a uGUI mask
Fixed issue where RelativeToProjectFolder was pointing to inside the
.app package instead of at the same level
Android
Streaming video no longer blocks while loading (but video frame-rate
may now not be available)
Removed duplicate function call, minor performance win
Fixed number of audio tracks reported for streams
Fixed potential case issue with URL detection
Windows
Platform specific settings exposed better in the editor
General
Fixed some shader build errors for older versions of Unity
Added some in-editor warnings for using unsupported file extensions
on various platforms
Fixed broken scripting reference link in ‘About’
Version 1.5.7 - 16 November 2016
Android
Minor optimisation for non-OES rendering path by eliminating blending
during blit
Added frame timestamp support via
TextureProducer.GetTextureTimeStamp()
macOS, iOS and tvOS
Added support for OS X 10.8
Added frame timestamp support via
TextureProducer.GetTextureTimeStamp()
Fixed issue when Seek and Play were called before another
command had completed
Fixed issue of events not firing
Added support for custom URL schemes
Improved support for device simulator
Various small bug fixes
Version 1.5.6 - 8 November 2016
General
Added “Scale and Crop” scaling mode to uGUI component (replaces
old “keep aspect ratio” option)
Added helper function for seeking to a specific frame
Improved MediaPlayer event handling code
Fixed null exception when unloading scene from the event callback
Windows
Fixed major issue using hardware decoding which caused videos to
crash or not load
Improved DirectShow colour reproduction by using more accurate
98 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
linear colour-space and YUV conversion functions
Added ability to get the presentation timestamp of the current video
frame - useful for more accurate frame syncing:
TextureProducer.GetTextureTimeStamp()
Fixed issue with HLS streaming crashing when changing stream
quality and resizing texture
Fixed AudioOutput issue where playing back multi-channel videos
would cause the video not to load
Fixed AudioOutput issue where some videos would play back at the
wrong pitch
Version 1.5.5 - 1 November 2016
Windows
Fixed issue on old Windows versions without D3D11 which caused the
plugin to not load
WebGL
Fixed width and height being zero when MetaDataReady event fires
Version 1.5.4b - 31 October 2016
Fixed broken iOS, tvOS and macOS release from v1.5.4
Fixed “null id” error in in multi-video demo in WebGL builds
Version 1.5.4 - 28 October 2016
General
02_Demo_imGui demo scene updated to show network streaming
progress
Improved how destroyed MediaPlayer components are shut down
Fixed alpha packing in background transparent shader
Fixed some bugs in three of the sample scene scripts that would
cause a crash due to null values or invalid textures left behind when
destroying MediaPlayer
Added documentation about hardware GPU decoding
Android
Fixed bug where IsPlaying would return false during playback
Fixed a shader compiler error for newer Android phones with
Snapdragon chipset when using the OES shaders
Optimised code to generate less garbage
Some internal JNI caching for speed
Added new unlit OES shader
macOS
Added support for Linear colour space
Improved feedback when progressive and streamed movies are
buffering
Added support for macOS 10.12 streaming auto wait behaviour
iOS and tvOS
Fixed a case where some HLS streams wouldn’t play
Improved feedback when progressive and streamed movies are
buffering
Added support for iOS/tvOS 10 streaming auto wait behaviour
99 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Windows
Added new AudioOutput component so that audio can be played
directly through a Unity AudioSource node. This allows Unity audio
effects to be used, spatial 3D movement with falloff and doppler, and
most importantly allows sound to move with the head in VR!
Fixed a visual glitch when using the hardware decoder path, most
noticeable at the end of videos
Streaming can now return multiple buffered ranges using method
GetBufferedTimeRange()
Windows watermark improved for GPU decoding path
Potential fix for DirectShow seeking issue causing hanging on some
systems
VR
Support for 3D audio via the new AudioOutput component (Windows
only, supports Oculus Rift and HTC Vive)
InsideSphere transparency shader now supports transparency
packing
Version 1.5.2 - 11 October 2016
General
uGUI and IMGUI components now automatically displays videos with
stereo or alpha packing correctly without having to assign material
manually
Fixed shader build errors in Unity 5.4.1
Fixed shader build errors in Unity 4.x for D3D9 and OpenGL
Added stereo support to more shaders including uGUI
Workflow
Improved component menu layout by grouping components
Logging improved so some log messages can be clicked on to
highlight the logging MediaPlayer
Windows
Linear colour space now automatically rendered correctly when using
GPU decoder
Fixed memory leak in GPU decoder introduced in v1.5.0
Fixed old memory leak in Notify system
Fixed HLS adaptive resolution changing when using GPU decoder
and improved texture switch to remove 1 frame glitch
WebGL
Cubemap script fixed to not show texture seam
Texture filtering and wrapping modes applied correctly now
Added crossOrigin=”anonymous” to video element
Version 1.5.1 - 30 September 2016
Windows
Fixed crash bug when playing audio files in DirectShow path
Fixed visual flicker bug in Unity editor in DirectShow path
Version 1.5.0 - 29 September 2016
100 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
General
Added new shader for uGUI to support stereo videos
Added global option to disable logging
Added audio track selection to IMGUI demo
Fixed editor bug where “\” character could be used in file path
Updated documentation
Android
Added display of initial “poster” frame for videos set to not auto-play
Fixed bug that prevented files with escape characters from loading
Fixed bug causing audio files to not play
Fixed issue where GetCurrentTimeMs() could report values greater
than Duration
macOS
Fixed no error being returned for loading videos without correct plist
permissions set
Windows
Hardware decoding enabled by default
Hardware (GPU) decoding no longer require command-line parameter
Better video API selection in editor
Added support for multiple audio tracks to DirectShow playback path
via IAMStreamSelect.
Added Media Foundation multiple audio track code back in
Fixed some minor resources leaks
Windows Phone / UWP
Added option for hardware decoding (enabled by default)
WebGL
Added support for multiple audio tracks
Fixed frame rate and count not displaying
Fixed issue where multiple videos wouldn’t unload cleanly
Cleaned up plugin code, removing unneeded code
Version 1.4.8 - 12 September 2016
General
Less garbage generation
Optimised ApplyToMesh component
Optimised rendering coroutine of MediaPlayer component
Added global option to disable debug GUI
Workflow
Improved file path display and editing
Added watermark description to trial version
VR
Cubemap script (for 3:2 cubemap VR)
Fixed vertical flip issue
Improved performance
Added fog and transparency options to VR sphere shader
Android OES optimisation (see below)
Shaders support GLSL better
Android
101 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
New faster rendering path, less memory, faster, no overheating - see
OES playback path notes above
Added ability to load file from within another file by specifying an offset
Which lets you hide video files from prying eyes
macOS
Fixed occasional memory leak when destroying player
Fixed editor warning about absolute path
iOS / tvOS
Fixed occasional memory leak when destroying player
Windows
HLS streams now dynamically adjust texture size
Resulting in support for higher quality streams
Fixes issue where videos would become letterboxed
WebGL
Fixed broken build issue when using the “use pre-built” build option
Version 1.4.4 - 20 August 2016
General
Packed transparency support added
Use top-bottom or left-right packed layout to encode alpha
channel
Set the packing type in the “Media Properties panel”
Supported by all display components automatically
See “Transparency Notes” section of this document for
encoding tips
“Platform Specific” section now highlights platforms with modified
settings
Minor optimisations to the ApplyToMaterial and ApplyToMesh
components (used property ints instead of strings)
Minor optimisation to DisplayIMGUI (disables layout pass)
Fixed CubemapCube.cs to handle vertically flipped video textures
Changed many components to use UpdateLate() instead of Update()
to fix some crash issues
Various minor improvements
macOS
Fixed compatibility issue with OS X 10.10 (Yosemite)
Windows
Huge performance improvements when using DirectShow playback
path due to usage of NV12 format (D3D11 only)
Added support for Hap Q format (D3D11 only)
Added performance warning message when using software decoder
instead of GPU decoder for large videos
Fixed bug in DirectShow audio mute not restoring volume
WebGL
Fixed issue where IMGUI text became flipped
Upgrade Notes
Some shader files have moved (and been renamed) from /Materials
and /Demos/Materials to /Resources/Shaders, so make sure to delete
102 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
duplicates that results from merging
Version 1.4.0 - 10 August 2016
WebGL
WebGL platform support added
macOS
Now with Metal rendering support
The Hap codec is now supported on macOS
iOS / tvOS
Metal rendering path now supports multi-threaded rendering
Various fixes to rendering and memory resources
Android
Ability to adjust the playback rate (Android 6.0 and above only)
Windows
Fixed audio device override not working in builds
Fixed fast seeking function
General
Added support for video files containing multiple audio tracks
Video frame rate is now available via Info.GetVideoFrameRate()
Fixed some issues with events firing at the wrong time
Streaming videos can now query IsBuffering() and
GetBufferingProgress()
Improved errors reporting
Renamed Info.GetVideoPlaybackRate() to GetVideoDisplayRate() to
avoid confusion
New “scriptlets” have been added to the /Demos/Scripts folder which
give mini examples of how to do simple scripting
VR
Fixed bug in stereo section of VR sphere shader which caused eyes
to flip on some platforms
Fixed a bug in Windows where the audio override wasn’t being
passed through for VR headphone device name
Workflow
Added new “SA” button for shortcut to StreamingAssets folder
Improved editor inspector performance
Upgrade notes
Demos scenes have been renamed so you should delete the old .unity
files in the /Demos folder otherwise you will have duplicate files.
The sample video files in StreamingAssets have been moved into a
subfolder within StreamingAssets/AVProVideoDemos/ so make sure
to delete the old ones.
Version 1.3.9 - 15 July 2016
Android
Fixed bug for Unity versions < 5.2.0 that caused videos not to display
Fixed bug for Android versions < 6 that caused video pausing and
other operations to break playback
Removed zip classes from plugin package to fix conflicts with other
103 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
plugins using the same zip classes
General
Better error handling via new GetLastError() function and a new Error
event type
NGUI support added via new component
TimeScale support feature now disabled by default and exposed in
new Global Settings panel
ApplyToMesh/Material script improved with new scale and offset
parameters
Added platform overrides for Windows Phone and Windows UWP
Improved documentation
Workflow
Added new Android warning for loading absolute path without external
permission set in PlayerSettings
VR
Fixed VR audio output device not working
New high quality option for sphere shader to improve quality at the
poles and general optimisations
Fixed issue with left/right eyes becoming swapped sometimes
Minor fixes to cube map script
Version 1.3.8 - 30 June 2016
Bug fixes
Fixed stereo VR script broken in v1.3.6
Fixed issues with UWP and Hololens support
Fixed Windows DirectShow issue preventing Hap videos playing when
another source filter is installed
Fixed Windows DirectShow bug in watermark for Hap videos
Workflow
Improved recent file list to be sorted by when the files were last used
Version 1.3.6 - 27 June 2016
General
Added (BETA) support for Universal Windows Platform 10 and
Windows Phone 8.1
Added (BETA) support for Time.timeScale and Time.captureFramerate
allowing video playback to work properly with video capture systems
that alter Unity time
Added ExtractFrame function to MediaPlayer
Added Extract Frames demo scene
Added SeekFast function which will try to seek to keyframes only so
it’s less accurate but faster. On unsupported platforms it will just do a
normal seek operation
Added functions to change the playback rate. Note that the behaviour
of this varies between platforms and is not available yet on Android
General API improvements, platform overrides system improved
Fixed bug where disabled gameObject would cause rendering to stop
Fixed bug where destroying MediaPlayer while uGUI component is
104 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
using it could cause a crash
Fixed rare bug where uGUI component would become offset or hidden
Fixed rare bug where m_events would be null
VR
Fixed VR sphere shader coordinates on some platforms, especially
cardboard
Added “Media Properties” section to MediaPlayer where you can
define the stereo packing of a video. This then automatically adjust
the sphere shader to render the video correctly per eye
Fixed bug in InsideSphere shader where stereo rendering would flip
the eyes at some angles
Added support for Unity 5.4 “Single-Pass Stereo Rendering” to 360
InsideSphere shader
Added new VR sample with “cubemap 3x2” (facebook layout) support
Workflow
Added button in preview panel to save frame to PNG
Added persistent option to preserve the media player between level
loads
Added multi-object MediaPlayer component editing support
Texture properties (wrap mode, filtering) can be set via Media
Properties panel
Debug GUI now shows preview of video
Android
Fixed bug where no visuals would appear when old devices
Switched from AAR to JAR/SO files to improve Unity 4 compatibility
Added x86 support
Fixed bug where Pausing a stopped video would lock it up
OS X / iOS / tvOS
Large performance boost on iOS and tvOS (removed texture copy)
Streaming video buffering stalls now recover and IsBuffering function
added
Seek accuracy improved
Looping handled better to reduce seam time
Fixed texture memory leak in Metal rendering path
Fixed mute not working with autoStart
Windows
DirectShow video playback smoother
Fixed DirectShow support for showing images or videos with zero
duration
Added platform-override option to force DirectShow player
Added platform-override option to force audio output device. This is
useful for VR hardware that has its own audio device.
Fixed poster frame not displaying
Fixed videos not displaying due to empty RGB32 alpha channel
Fixed D3D9 lost device issue
Fixed D3D9 issue of texture not being released
Fixed DirectShow player time displaying NaN sometimes at the end of
video
105 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Fixed crash bug if you closed Unity with video was playing
Fixed Windows N edition crash
Version 1.3.0 - 20 April 2016
Android
Added multi-threaded rendering support
Fixed various rare bugs
Windows
Exposed GPU video player path (when using command-line argument
“-force-d3d11-no-singlethreaded”), this requires Win8.1 minimum and
D3D11
Windows XP, Windows Vista and Windows 7 support added by adding
DirectShow playback path
Hap1 and Hap5 codec support added for Windows 7 and below (via
DirectShow playback path)
Fixed audio only media not being seekable
iOS
Fixed iOS 8.2 compatibility issue
Workflow
Editor pause button now pauses/resumes playing videos
Added new ‘about’ section with handy links
Improvements to editor UI
UI fixes for standard Unity editor ‘light’ skin
Bugs
Fixed IsFinishedPlaying event not firing on some platforms
Fixed player freezing when component is disabled. It now pauses
OnDisable() and resumes OnEnable().
Fixed crash in VCR demo related to uGUI component when changing
videos
Fixed bug closing application with many videos open
Fixed seeking bug for audio-only media
Documentation
Updated FAQ
Added stereo virtual reality notes
Version 1.2.0 - 1 April 2016
General
Improved support for audio-only media
Improved streaming support
Added HasVideo, HasAudio and HasMetaData functions
Added MetaDataReady event
First frame now displays without starting playback
Added new ApplyToMesh component
Removed troubled #AVPRO defines
Android
Fixed issue where Finished event wouldn’t fire
Minimum API level bumped from 15 to 16
Mac, iOS, tvOS
106 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
improved performance of multiple video playback
Windows
Improved performance (duplicate frame handling)
Added missing IsSeeking function
Fixed IsFinished function always returning false
Fixed URL parsing
Fixed OpenGL texture cleanup
Fixed minor D3D11 leak caused by views not being released
Improved init/deinit
Workflow
Resulting file name is now displayed clearly and is copyable
More file extensions in inspector file browser
Components now have AVPro icon
Added warnings for incorrect Player Settings for streaming videos
Editor platform override tab is restored
Debugging
Improved logging
VR
Improved 360 Sphere demo support for GearVR
InsideSphere shader has color tint option for stereo debugging
Docs
Added video streaming notes
Added VR notes
Improved notes on supported formats
Version 1.1.0 - 11 March 2016
Added Windows support
Added lat-long stereo top-bottom and left-right video support for VR
Added per-platform file path overrides
Absolute and relative path loading support added
Android loading from OBB support added
Workflow improvements
Added browse button
Added recent file list
Added relative folder support
Improved MediaPlayer UI, including preview in editor
Created a scripting class reference
Improved documentation
Fixed Linear colour space support in Windows
Fixed shutdown and memory freeing
Various bugs fixed
Version 1.0.0 - 25 February 2016
Initial release on the asset store
Added new events
Version 0.9.0 - 17 February 2016
Initial beta release
107 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
Appendix C - Roadmap
Version 2.0.0
High quality glitch free multi video player with optional transitions and nice
user interface
Version X.X.X
Split out media definitions from player
Loading from memory/resources
Fallback to using MovieTexture/VideoPlayer on some platforms?
10-bit / HDR support?
Version X
← Your suggestions here, let us know :)
Appendix D - Unity Bugs
Some versions of Unity have new bugs or regressions that prevent AVPro Video from
operating as expected.
In our experience it’s best to avoid using new major releases of Unity (eg version X.Y.0) and
wait for the follow-up releases where bugs introduced in the major update have had time to
be fixed.
Please vote and comment on these issues so they get high priority to be fixed.
Unity 5.6.0f3 - Present
Has an Android bug that prevents GLSL shaders from compiling, which
prevents our OES rendering path to work. This is fixed now:
https://issuetracker.unity3d.com/issues/android-ios-glsl-shader-compilation-fai
lure-on-some-mobile-devices
Latest version where it works it 5.5.4
Single-pass stereo mode on the Android GearVR introduces strange shader
bugs on some platforms. We added two bug reports, but it won’t be fixed as
it’s a mobile GPU driver issue, so hopefully the drivers got fixed:
https://issuetracker.unity3d.com/issues/android-gearvr-left-eye-is-rendered-wit
h-green-distortions-when-using-single-pass-stereo-rendering-method-star-on-
mali-gpu
https://issuetracker.unity3d.com/issues/android-gearvr-both-eyes-are-rendere
d-in-the-same-color-when-using-single-pass-stereo-rendering-method-star-on
-mali-gpu
Unity 5.x - Present
Support for Android OBB (split application binary option) seems slightly
broken in many versions of Unity. It seems to try to load OBB content from
108 / 109
AVPro Video © 2016-2019 RenderHeads Ltd
the APK file, so it’s as though it’s resolving Application.streamingAssetsPath
incorrectly. It appears that the first time you deploy to the Android device it
works but then subsequent builds will fail. Uninstalling the application from
the device and building to a new APK file seems to solve this.
Android OES Texture Support
OES Textures used to work reasonably well, but in some versions of Unity
they no longer work:
https://forum.unity3d.com/threads/android-oes-texture-support.436979/
Android Cardboard in Unity 2017.3.1
Unity temporarily broke Cardboard support for non-Daydream devices around
these versions:
2018.1.0 b3 - Fixed in 2018.1.0 b12
2017.3.1 p1 - Fixed in LTS 2017.4.1 f1
2017.2.1 p3 - Fixed in 2017.2.2 p3
2017.1.3 p1 - Fixed in 2017.1.3 p3
https://forum.unity.com/threads/for-people-targeting-cardboard-on-android-ple
ase-update-your-google-vr-sdk-to-1-12-or-later.520986/
109 / 109
AVPro Video © 2016-2019 RenderHeads Ltd

Navigation menu