Install Guide

User Manual:

Open the PDF directly: View PDF .
Page Count: 13

Download
Open PDF In Browser	View PDF

VIAN
Datamodel, JSON Serialization and XML Export Format

Author:
Gaudenz Halter, 27.06.2018
ERC Advanced Grant FilmColors
Department of Film Studies
University of Zürich

Visualization and MultiMedia Lab
Department of Informatics
University of Zürich

ExternalMediaObject

IMediaObjectAttachable

+name: String
+file_path: String

+ExternalMediaObjects: List

+0..*

Screenshot

ScreenshotGroup
+0...*

+1..*

VisualAnnotationBaseClass
+position: Vector2
+size: Vector2
+type: EnumAnnotationType

MediaDescriptor

VIANProject

+screenshots: List

+name: String
+file_path: String

+screenshotGroups: List
+segmentations: List
+annotationLayers: List
+experiments: List
+mediaDescriptor: MediaDescriptor
+analyzes: List
+mediaObjects: List

AnnotationLayer
+0...*

+annotations: List

+0...*

IClassifyable
1
IAnalysis
Segment

Segmentation
+0..*

+body: String

+Parameters: Dict
+AnalysisData: Dict

+segments: List
+0...*

+0..n

ii
ITimeRange
+t_start
+t_end

IContainerGroup
+children

+0..*

Classification Object
UniqueKeyword

ClassificationEntry

+classificationObject
+vocabularyWord

+Classifyable: IClassifyable
+Keyword: UniqueKeyword

+Vocabularies: List
+TargetContainers: IContainerGroup[0..1]

Experiment
+0...n

1

Vocabulary
+words: List
+category: String

VocabularyWord

+0..n

+ClassificationObjects: Classification Object[0..*]
+MendatoryAnalyzes
+ClassificationResults

1 Introduction
Essentially, VIAN projects are represented as a directory system. VIAN serializes its projects into a json file and
a collection of mostly numeric analysis result which are hold in a sqlite database. (Currently there exist additional
binary files that stored in a projects filesystem, but these will be deprecated shortly.)
1
2
3
4
5
6
7
8

VIAN FileSystem:
Root - |
| - myproject.json
| - data ------- |
| - database.sqlite
| - results // contains visualizations
| - export
// default exported data
| - shots
// default exported shots

Since a lot of the data stored is application and python specific, VIAN supports an XML export of its projects,
its elements are described in this document. Links between different entities are serialized using the ID attribute.
IDs are guaranteed to be unique and declared before referenced in the document.
Additionally the VIAN project json file is described at the end of this document by an example.

2 XML Schema
2.1 Element: ANNOTATION DOCUMENT
1



2
3
4
5

@AUTHOR: The Person that created the File
@DATE: The creation date
@VERSION: The VIAN version this file was produced with

2.2 Element: HEADER
1



2
3
4

## HEADER
@ PROJECT_NAME: The name of the project

2.3 Element: MEDIA DESCRIPTOR
1



2
3
4
5
6

@FPS: a float indicating the fps of this media
@IS_RELATIVE: if MEDIA_URL is relative or not {True, False}
@MEDIA_URL: Absolute or relative path to the media
@YEAR: The creation year of the media

iii

2.4 Element: TIME ORDER and TIME SLOT
1
2
3
4
5
6
7




TIME_VALUE="0"/>
TIME_VALUE="19300"/>
TIME_VALUE="56300"/>
TIME_VALUE="91700"/>

8
9

10

@TIME_SLOT_ID: A string that can be referenced by the serialization of any ITimeRange
implementing object
@TIME_VALUE: The time of this slot in MS

VIAN uses an implicit timeline, that is, each Segment, Visual Annotation or Screenshot object stores it’s media
time directly. In the XML export format, a list of TIME SLOTS is created, and each serialization of an object
media-time information references one or two TIME SLOTS.
Note: Currently TIME SLOTS are unique, thus for each time there can not be more than one TIME SLOT.

2.5 Element: SEGMENTATION and SEGMENT
1
2
3





4
5
6
7

##SEGMENTATION
@ID: Unique ID of this segmentation
@NAME: The name of this segmentation

8
9
10
11
12
13

#SEGMENT
@BODY: The annotation text
@ID: The unique ID of this annotation
@T_END: A reference to a TIME_SLOT where this annotation ends
@T_START: A reference to a TIME_SLOT where this annotation starts

The first kind of annotation supported by VIAN are called ”Segment” and are aggregated in a Tier called ”Segmentation”. A segment has a start and end-point and may have a text body.

2.6 Element: SCREENSHOTS and SCREENSHOT
1
2
3
4
5
6





...


7
8
9

@ID: The unique id of this screenshot
@T_START: A reference to a TIME_SLOT indicating the start of this annotation

The second type of annotations are ”Screenshots”, which are aggregated in ”ScreenshotGroups”. A screenshot
only has a start-time in the XML export.

iv

2.7 Element: ANNOTATION LAYER and VISUAL ANNOTATION
Finally there are ”Visual Annotations”, aggregated in ”Annotation Layers”, which represent vector graphics, that
are placed on the screen.
1
2
3

4





5
6
7
8

## ANNOTATION_LAYER
@ID: The unique id of this annotation layer
@NAME: The name of this annotation layer

9
10
11

12
13
14
15

16
17
18
19

## VISUAL_ANNOTATION
@A_TYPE: The type of this annotation {AnnotationType.Rectangle, AnnotationType.Ellipse,
AnnotationType.Line, AnnotationType.Text, AnnotationType.Image, AnnotationType.
FreeHand}
@COLOR: The Color of this annotation
@ID: The unique id of this annotation
@POS: A vector2 indicating the screen position of this annotation
@RESSOURCE_PATH: If A_TYPE == AnnotationType.Image, this specifies the relative path to
the Image-Source {.png, .jpg format}
@SIZE: A vector2 indicating the screen size of this annotation
@TEXT: If A_TYPE == AnnotationType.Text, this specifies the text of the annotation
@T_END: A reference to a TIME_SLOT, when the annotation ends
@T_START: A reference to a TIME_SLOT, when the annotation starts

Note that the export of visual annotations does currently not support:
• Export of keys on annotations
• Export of FreeHand annotations.

v

2.8 Element: EXPERIMENT
The process of data acquisition (i.e. creating segments, annotations and screenshots) and classification of these
entities using vocabularies is splitted, such that different classifications can be performed on the same segments
and annotations. The second process is contained in the Experiment entity.
Essentially a subject of interest (Classification Object), can be present in several tiers (the targets), and can be
classified by multiple vocabularies. For each classification object a list of unique keywords is generated which
can then be attached to the classification targets as tags. All this information is contained in an experiment.
Example: The classification object ”Background” can be present in the scene segmentation
as well as the screenshots, and would probably be classified with the vocabularies ”Color”,
”Shadows”.
1
2
3
4
5
6
7
8
9
10

11
12
13
14
15





...

...






...


16
17
18
19

20

21

22
23
24
25
26
27
28
29
30






...





...



vi

2.9 Element: CLASSIFICATION OBJECT
1
2
3





4
5
6
7

@ID: The unique id of this classification object
@NAME: The name of this classification object
@TARGET_ID: A reference to IContainerGroup (Segmentation, AnnotationLayer,
ScreenshotGroup) this classification object can be found in

2.10 Element:VOCABULARY and VOCABULARY WORD
1
2
3
4
5
6








7
8
9
10
11

## VOCABULARY
@CATEGORY: The topic this vocabulary should belong to
@ID: The unique id of this Vocabulary
@INFO_URL: A optional url to more information about this vocabulary

12
13
14
15
16

#VOCABULARY_WORD
@ID: The unique id of this VocabularyWord
@INFO_URL: A optional url to more information about this vocabulary
@NAME: The name of this word

2.11 Element: KEYWORDS
1
2
3
4
5
6
7




ID="4400252826"
ID="8650499482"
ID="1149034722"
ID="9690251691"
ID="2806440146"

WORD_ID="5741784264"/>
WORD_ID="2534552219"/>
WORD_ID="9103031091"/>
WORD_ID="3412950183"/>
WORD_ID="8178481154"/>

8
9

10
11

@CLASSIFICATION_OBJECT_ID: The unique ID of the classification object this keyword
target.
@ID: The unique ID of this keyword
@WORD_ID: The unique ID of the VocabularyWord this keyword targets.

2.12 Element: CLASSIFICATION
The classification represents the mapping of tags their targets. (screenshots, segments, visual annotations)
1
2
3
4
5




...


6
7
8

@KEYWORD_ID: The unique ID of the keyword
@TARGET_ID: The unique ID of the IClassifyable

vii

2.13 Element: EXTERNAL MEDIA OBJECT
1
2
3





4
5
6
7

@ID: The unique id of this external media object
@PATH: The relative path to the external media object in the export directory
@TARGET_ID: The unique ID of the container this media objects is attached to

2.14 Element: ANALYSES
1
2
3

4






5
6
7
8
9

## COLORIMETRY_ANALSIS
@ID: The unique id of this analysis object
@NAME: Constant = "Colorimetry"
@PATH: The path to the numpy numeric serialized data

10
11
12
13
14

15

## JOB_ANALYSIS
@ID: The unique id of this analysis object
@NAME: The name of the JobAnalysis Class (can be used in an python::eval() statement)
@PATH: The path to the numpy numeric serialized data or "No Export" if data export has
been turned off
@CLASSIFICATION_OBJECT_REF: The unique id of the classification object this analysis has
been performed of or -1 if None

viii

3 Json Schema
Since the VIAN json file has not been intended to be read by other applications, the XML export may be the
better way to enforce interoperability with other applications. But for the sake of completeness, a documented
JSON is shown in the rest of this document.
1

{

2
3
4

5
6
7

8

"name": "3774_1_1_Blade_Runner_1900_DVD",
"path": "C://****/****/documents/VIAN/3774_1_1_Blade_Runner_1900_DVD/3774
_1_1_Blade_Runner_1900_DVD.eext", // The abs path to the project
"version": "0.6.4",
"corpus_id": -1,
"main_segmentation_index": 0, // THe Index of the Main Segmentation (Which is used for
sorting screenshots)
"notes": "", // Additional information that can be set in the Inspector

9
10
11
12
13
14

/*Project Directories */
"data_dir": "C://****/****/documents/VIAN/3774_1_1_Blade_Runner_1900_DVD/data",
"shots_dir": "C://****/****/documents/VIAN/3774_1_1_Blade_Runner_1900_DVD/shots",
"export_dir": "C://****/****/documents/VIAN/3774_1_1_Blade_Runner_1900_DVD/export",
"results_dir": "C://****/****/documents/VIAN/3774_1_1_Blade_Runner_1900_DVD/results",

15
16
17
18
19
20
21
22
23
24
25
26
27
28

29

30
31
32
33

34
35

36
37
38
39
40
41
42
43
44
45
46
47

/* SEGMENTATIONS*/
"segmentation": [ /* A List of serialized Segmentations */
{
"segments": [ /* A List of serialized Segments */
{
"name": "1", // The Name of the Segment
"end": 107607, // MS of the End
"notes": "", // Additional Notes
"start": 0, // MS of the Start
"media_objects": [ // A List of serialized Media Objects
{
"dtype": 2, // What type the media object is (External, Data,)
"file_path": "C:\\*******/documents/VIAN//3774_1_1_Blade_Runner_1900_DVD//
data/01_DOF_Explanation.png", // An absolute path to the File, this
is regenerated when loading the project
"name": "DOF_Explanation.png", // The name of the file in the data_dir
directory
"unique_id": 1741505363
}
],
"scene_id": 1, // The index of this segment in the Segmentation (starting with
1)
"unique_id": 1166372514,
"annotation_body": "Intertitle explains the already passed plot", // The text
annotation
"locked": false, // If the segment is locked in the timelnie
"duration": null // @obsolete
}
/* ... more Segments ...*/
],
"notes": "", // Additional information that can be set in the Inspector
"unique_id": 1049798528,
"locked": false, // If the segmentation is locked in the Timeline
"name": "Auto Segmentation" // The name of the segmentation
}
/*... more Segmentation ...*/
],

48
49

/* ANNOTATION LAYERS */

ix

50
51
52
53
54
55
56
57
58
59

60
61
62
63

64
65
66
67
68

69

70

71
72
73
74
75

76
77
78
79
80
81
82
83
84
85
86
87

"annotation_layers": [
{
"name": "New Layer",
"locked": false, // If it is locked in the Timeline
"is_visible": true, // If it should be displayed
"unique_id": 2927829419,
"notes": "", // Additional information that can be set in the Inspector
"annotations": [ // A list of Annotation serializations
{
"free_hand_paths": [], // A list of FreeHand Paths if this is a FreeHand
annotation
"name": "New Rectangle",
"notes": "", // Additional information that can be set in the Inspector
"t_end": 82300, // When the visibility should be turned off
"is_automated": false, // if this annotation is driven by another containers
value
"line_w": 5, // the line width of the frame if it has one
"orig_position": [ 178, 6 ], // The position in MovieSpace
"color": [ 230, 83, 162 ], // The color of the frame if it has one
"automate_property": null, // The value this is driven by if it is automated
"automated_source": -1, // The unique id of the object that contains the value
above if it has any
"curr_size": [ 1512.5951940850277, 421.8022181146026 ], // The size of this
annotation in MovieSpace
"t_start": 53900, // The MS start when the annotation should start beeing
visible
"unique_id": 1432985272,
"font_size": 6, // The Font-Size if it has text
"resource_path": "", // The Image path it is an Image Annotation
"text": "", // The Text if it is a Text Annotation
"size": [ 1512.5951940850277, 421.8022181146026 ], // The current size (no
need to be serialized) in PlayerSpace
"media_objects": [], // Attached media objects if there are any
"a_type": 0, // The AnnotationType Enum Value
"font": "Agency FB", // The Font Family
"tracking": "Static", // If this annotation should track something
"keys": [], // A list of Position Tuples if this annotation is animated
"widget": null // The displaying Widget
}
/* ... more Annotations .. */
],
"is_current_layer": false // If this is the current top layer
}
],

88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105

/* NODE SCRIPTS*/
"scripts": [
{
"notes": "", // Additional information that can be set in the Inspector
"connections": [
{
"output_node": 6264180387,
"input_pin_id": 2,
"output_pin_id": 0,
"input_node": 8803892119
}
/*... more Connections ...*/
],
"name": "New Script", // The Name of the Script
"nodes": [
{
"name": "Read Frame Movie",

x

106

107
108
109
110
111
112
113
114
115
116
117
118

"default_values": [ "test.mp4", 3000 ], // The Default Values of the input
pins
"node_pos": [ 107, 94 ], // The position of the node in the Editor
"unique_id": 8803892119,
"notes": "", // Additional information that can be set in the Inspector
"operation": "OperationFrameReader", // The associated OperationClass Name.
"node_size": [ 200, 200 ] // The size of the Node in the editor
}
/*... more Nodes ...*/
],
"unique_id": 1284084303
}
/*... more Scripts ...*/
],

119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142

143
144

/* VOCABULARIES */
"vocabularies": [
{
"name": "Scene Locations",
"category": "default", // The topic category it belongs to
"unique_id": 1095049086,
"words": [
{
"parent": 1095049086,
"name": "House",
"unique_id": 3390127240,
"children": [] // Child words if any
}
//....
]
}
// ....
],
"screenshot_groups": [
{
"unique_id": 1404536708,
"name": "All Shots",
"shots": [ 1651945337, 7522186061, 8671808241 /*...*/ ] // A list of associated
shots as unique id
}
],

145
146
147
148
149
150
151
152
153
154
155

156
157
158

159
160

161
162
163

/* EXPERIMENTS */
"experiments": [
{
"name": "ERC Advanced Grant FilmColors",
"analyses": [], // A list Analysis classes to perform
"unique_id": 4846731777,
"classification_objects": [
{
"parent": 4846731777,
"target_container": [ 1049798528 ], // A list of IClassifyable that contain
this classification object
"name": "Global",
"children": [], // Child Classification objects if any
"classification_vocabularies": [ 7835076715, 3427161829 /*...*/ ], // The
Vocabularies associated with this classification object
"unique_id": 6353598060,
"unique_keywords": [ // These are essentially the tags consisting of VocWord,
ClassificationObject and Vocabulary
{
"voc_obj": 7835076715,
"external_id": 8,

xi

"word_obj": 5741784264,
"unique_id": 4400252826,
"class_obj": 6353598060

164
165
166

}
/*...*/

167
168

]

169

}

170

],
"classification_results": [ // A List of Mappings: (Target.unique_id, UKeyqord.
unique_id)
[ 1166372514, 2016330087 ],
[ 1166372514, 3935737480 ],
[ 1166372514, 3145357142 ],
[ 1166372514, 3663305810 ]
/*...*/
]

171
172

173
174
175
176
177
178
179
180

}
],

181
182
183
184
185

186
187
188
189
190
191

192

193
194

195
196
197

/* SCREENSHOTS */
"screenshots": [
{
"shot_id_global": 1, // The index of the screenshot in all screenshots sorted by
time
"notes": "", // Additional notes from the inspector
"title": "Auto_Scr_0",
"creation_timestamp": "2018-06-27 10:28:14.640474",
"movie_timestamp": 42, // Time in MS
"frame_pos": 1, // Frame IDX of this shot
"shot_id_segm": 1, // The Index of this Shot within the segment of the main
segmentation (Starting with one)
"scene_id": 1, // The idx of the segment in the main segmentation it is associated
with (Starting with one)
"unique_id": 1651945337,
"annotation_item_ids": null // Annotations that should be rendered ontop of this
Screenshot
}
/*...*/
],

198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217

/* ANALYSES */
"analyzes": [
{
"has_finished": true, // If the Colorimetry is completely finished
"unique_id": 8908799312,
"notes": "",
"name": "Colormetry",
"analysis_container_class": "ColormetryAnalysis"
},
{
"parameters": { "resolution": 100 },
"name": "Color-Palette",
"analysis_job_class": "ColorPaletteAnalysis",
"notes": "",
"unique_id": 1105697457,
"container": 3778593545,
"analysis_container_class": "IAnalysisJobAnalysis"
}
],

218
219
220
221

/* MOVIE DESCRIPTOR */
"movie_descriptor": {
"is_relative": false, // If the movie is located in the project directory

xii

"movie_name": "Blade Runner",
"movie_id": "3774_1_1", // A ID related to the ERC FilmColors Project
"notes": "",
"year": 2017, // The Production Year
"source": "DVD", // The Source of the Movie
"movie_path": "//****/Filme/spaete_Filme/3774_Blade_Runner_2049/3774
_1_1_Blade_Runner_2049_DVD.m4v", // Abs or rel path depending on the location of
the movie
"unique_id": 5420540984,
"duration": 9807872 // Duration in MS

222
223
224
225
226
227

228
229

}

230
231
232

}

xiii

Source Exif Data:

File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.5
Linearized                      : No
Page Count                      : 13
Page Mode                       : UseOutlines
Author                          : 
Title                           : 
Subject                         : 
Creator                         : LaTeX with hyperref package
Producer                        : pdfTeX-1.40.17
Create Date                     : 2018:07:12 08:42:33+02:00
Modify Date                     : 2018:07:12 08:42:33+02:00
Trapped                         : False
PTEX Fullbanner                 : This is MiKTeX-pdfTeX 2.9.6211 (1.40.17)

EXIF Metadata provided by EXIF.tools

Install Guide

Navigation menu

Versions of this User Manual:

Views

Navigation