XXX Procedure Huawei Hi AI DDK User Manual

huawei-hiai-ddk-user-manual

User Manual:

Open the PDF directly: View PDF PDF.
Page Count: 16

DownloadXXX Procedure Huawei Hi AI DDK User Manual
Open PDF In BrowserView PDF
Huawei HiAI DDK User Manual

Issue: V100.150.10
Date: 2018-03-09
Huawei Technologies Co., Ltd.

Copyright © Huawei Technologies Co., Ltd. 2018. All rights reserved.
No part of this document may be reproduced or transmitted in any form or by any means without prior written consent
of Huawei Technologies Co., Ltd.

Trademarks and Permissions
and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective holders.

Notice
The information in this document is subject to change without notice. Every effort has been made in the preparation of
this document to ensure accuracy of the contents, but all statements, information, and recommendations in this
document do not constitute a warranty of any kind, express or implied.
The method of applying for HiAI is described as follows:

1.

Send an application email to developer@huawei.com.

2.

The format of the email subject is HUAWEI HiAI+Company name+Product name.

3.

The format of the email body is Cooperation company+Contact person+Contact information+Contact
email address.

4.

We will send you feedback within five workdays after receiving your email.

Official website: http://developer.huawei.com/consumer/cn/devunion/ui/server/HiAI.html

2018-03-19

Huawei confidential. No spreading without permission.

Page 2 of 16

Huawei HiAI DDK User Manual

INTERNAL

Contents
1 Overview......................................................................................................................................... 4
2 Integration Description ................................................................................................................ 4
2.1 Function Introduction ....................................................................................................................................... 4
2.2 Execution Mode ............................................................................................................................................... 4
2.2.1 Offline Model Generation ....................................................................................................................... 4
2.2.2 Offline Model Computation .................................................................................................................... 5
2.3 Supported Operators......................................................................................................................................... 5
2.4 Restrictions ....................................................................................................................................................... 5
2.5 Supported Interfaces ......................................................................................................................................... 5
2.5.1 Obtaining the DDK Version Number ...................................................................................................... 5
2.5.2 Creating a Model Manager ..................................................................................................................... 6
2.5.3 Loading a Model ..................................................................................................................................... 7
2.5.4 Obtaining the Input and Output Shape Information of a Model.............................................................. 9
2.5.5 Running the Model (1) .......................................................................................................................... 10
2.5.6 Running the Model (2) .......................................................................................................................... 12
2.5.7 Unloading a Model ............................................................................................................................... 13
2.5.8 Destroying a Model Manager................................................................................................................ 14

3 Integration .................................................................................................................................... 14
4 Appendix ...................................................................................................................................... 14
4.1 Definition of Error Codes ............................................................................................................................... 14

2018-03-19

Huawei confidential. No spreading without permission.

Page 3 of 16

Huawei HiAI DDK User Manual

INTERNAL

Huawei HiAI DDK User Manual
1 Overview
HiAI is an artificial intelligence (AI) computing platform oriented to mobile terminals. The computing
library of the mobile computing platform, which is formed by the HiAI application programming interfaces
(APIs), is designed for developers to conveniently and efficiently write AI applications running on mobile
devices.
Released in a unified binary file, the HiAI APIs are used to accelerate the computing of neural networks
using the HiAI heterogeneous computing platform. Currently, these APIs can run only on the Kirin system
on chip (SoC) and are integrated to the Android system that uses the Kirin SoC, allowing developers to run
the neural network model in the integrated environment and invoke the HiAI APIs to accelerate
computation. The HiAI device development kit (DDK) is the HiAI resource package available to
third-party developers.

2 Integration Description
2.1 Function Introduction
The HiAI DDK provides the AI model manager for AI application developers. It is a model management
interface that provide functions such as model loading, model computation, and model unloading.

2.2 Execution Mode
The HiAI DDK computing library runs in offline mode. It uses the compiled and optimized offline model
to perform neural network computation. This reduces memory usage while improving computing
performance. You can convert a Caffe or TensorFlow model to an offline model using a dedicated
conversion tool.

2.2.1 Offline Model Generation
The HiAI DDK provides a model conversion tool to convert the Caffe or TensorFlow models to an NPU
model. For details about how to use the conversion tool, see Chapter 5"Model Conversion" in the Huawei
HiAI DDK Integration Manual.

2018-03-19

Huawei confidential. No spreading without permission.

Page 4 of 16

Huawei HiAI DDK User Manual

INTERNAL

2.2.2 Offline Model Computation
When the user APK uses the DDK, the interfaces for loading, running, and unloading the offline model
needs to be invoked to calculate the user-input data on the HiAI heterogeneous acceleration system. For
details, see section 6.2 "Interface Integration" in the Huawei HiAI DDK Integration Manual.

2.3 Supported Operators
For details, see the Huawei HiAI DDK Operator Specification Document.

2.4 Restrictions
The ROM and RAM space on a mobile phone chip is limited. Therefore, the sizes of the model and the
running memory must be restricted as follows:


The model size is less than or equal to 100 MB.



The size of the peak memory is less than or equal to 200 MB.

2.5 Supported Interfaces
2.5.1 Obtaining the DDK Version Number
The HIAI_GetVersion interface is used to obtain the DDK version number. The function maintains a static
version number and invokes the Android system interface __system_property_get to obtain the value of the
system attribute ro.config.HiAIversion, that is, the DDK version number.
Table 2-1 Description of the HiAI_GetVersion interface
Function
Description

Obtaining the DDK version number in the system.

Interface
Prototype

char* HiAI_GetVersion()

Parameter
Description

None

Return Value

If the execution succeeds, the corresponding DDK version number is returned.
The version number is described in the format of
....
: Indicates a product form.
● XX: mobile phone,
● XX: edge computing.
● XX: Cloud
: Indicates the V version of a product form, expressed in three digits
(XXX), for example, HiAI V100 and HiAI V200 for a mobile phone.
: Indicates the incremental C version with new features. The value is
expressed in three digits (XXX).
< point >: Indicates the B version or patch version. The value is expressed in
three digits (XXX). If the last digit is not 0, it indicates a patch version.
For example, the version number of the Kirin 970 system is 100.150.010.010. If
000.000.000.000 is returned, the current HIAI version does not support NPU

2018-03-19

Huawei confidential. No spreading without permission.

Page 5 of 16

Huawei HiAI DDK User Manual

INTERNAL

acceleration.
If the execution fails, an error code is returned.

2.5.2 Creating a Model Manager
2.5.2.1 Synchronization Interface
A model interface can be used only after model management object instances are created. Currently, a
process supports three model management instances. Table 2-2 describes the synchronization interface
prototype.
Table 2-2 Description of the HiAI_ModelManager synchronization interface
Function
Description

Creating a model management instance

Interface
Prototype

HiAI_ModelManager* HiAI_ModelManager_create(void);

Parameter
Description

None

Return Value

Pointer to the model management object instance. This pointer needs to be destroyed
by calling HIAI_ModelManager_destroy.

2018-03-19

Huawei confidential. No spreading without permission.

Page 6 of 16

Huawei HiAI DDK User Manual

INTERNAL

2.5.2.2 Asynchronization Interface
Table 2-3 Description of the HiAI_ModelManager asynchronization interface
Function
Description

Creating a interface class for the model management engine

Interface
Prototype

HiAI_ModelManager* HiAI_ModelManager_create(HiAI_ModelManagerListener*
listener);

Parameter
Description

HiAI_ModelManagerListener: Indicates the structure of an asynchronous callback
function pointer, including the callback function pointer to the model loading
completion, model running completion, model unloading completion, timeout, error
handling, and monitoring of the service status of the peer end through the
onServiceDied interface.
HiAI_ModelManagerListener is defined as follows:
typedef struct HiAI_ModelManagerListener_struct
{
void (*onLoadDone)(void* userdata, int taskStamp);
void (*onRunDone)(void* userdata, int taskStamp);
void (*onUnloadDone)(void* userdata, int taskStamp);
void (*onTimeout)(void* userdata, int taskStamp);
void (*onError)(void* userdata, int taskStamp, int errCode);
void (*onServiceDied)(void* userdata);

void* userdata;
} HiAI_ModelManagerListener;

void (*onLoadDone)(void* userdata, int taskStamp): Indicates the callback
function pointer to successful model loading.
void (*onRunDone)(void* userdata, int taskStamp): Indicates the callback
function pointer to successful model running.
void (*onUnloadDone)(void* userdata, int taskStamp): Indicates the callback
function pointer to successful model unloading.
void (*onTimeout)(void* userdata, int taskStamp): Indicates the pointer to the
timeout callback function.
void (*onError)(void* userdata, int taskStamp, int errCode): Indicates the
callback function pointer to error handling.
void (*onServiceDied)(void* userdata): Indicates the callback function pointer to
monitoring the service status of the peer end through the onServiceDied interface.
void* userdata: Indicates user data.
Return Value

Object interface of the model management engine

2.5.3 Loading a Model
A model can be loaded from the application layer in either of the following method.

2018-03-19

Huawei confidential. No spreading without permission.

Page 7 of 16

Huawei HiAI DDK User Manual

INTERNAL



Load the memory from the assets directory of the app. The app manages the model and implements
model reading.



Load a file from an SD card.

Specifically, perform the following steps:
Step 1 Create the HIAI_ModelBuffer interface.
Step 2 Invoke the model loading interface to load the model.
Step 3 Destroy the HIAI_ModelBuffer interface.
----End

2.5.3.1 Creating the HIAI_ModelBuffer
Table 2-4 Description of creating the HIAI_ModelBuffer interface from a file
Function
Description

Loading the model from a path to create the HIAI_ModelBuffer interface (used to
load the model from an SD card at the application layer)

Interface
Prototype

HiAI_ModelBuffer* HiAI_ModelBuffer_create_from_file(const char* name, const
char* path, HiAI_DevPerf perf)

Parameter
Description

name: Specifies the name of the model to be loaded.
path: Specifies the path of the model to be loaded.
perf: Specifies the NPU frequency, which includes the high, medium, and low
levels.

Return Value

HIAI_ModelBuffer

Table 2-5 Description of creating the HIAI_ModelBuffer interface using the model data address
Function Description

Reading model data to load the model and create the HIAI_ModelBuffer
interface (used to load the model from the assets directory at the application
layer)

Interface Prototype

HiAI_ModelBuffer* HiAI_ModelBuffer_create_from_buffer(const char*
name, void* modelBuf, int size, HiAI_DevPerf perf);

Parameter Description

name: Specifies the name of the model to be loaded.
modelBuf: Specifies the model data address.
size: Specifies the model data length.
perf: Specifies the NPU frequency, which corresponds to the high, medium,
or low level.

Return Value

2018-03-19

HIAI_ModelBuffer

Huawei confidential. No spreading without permission.

Page 8 of 16

Huawei HiAI DDK User Manual

INTERNAL

2.5.3.2 Model Loading
Table 2-6 Description of the HiAI_ModelManager_loadFromModelBuffers interface
Function Description

Loading the model

Interface Prototype

int HiAI_ModelManager_loadFromModelBuffers(HiAI_ModelManager*
manager, HiAI_ModelBuffer* bufferArray[], int nBuffers);

Parameter Description

manager: Specifies the object interface of the model management engine.
bufferArray[]: HiAI_ModelBuffer. Single- and multi-model are both
supported.
nBuffers: Specifies the number of models to be loaded.

Return Value

If the execution succeeds, 0 is returned. If the execution fails, an error code
is returned.

2.5.3.3 Destroying the HIAI_ModelBuffer Interface
Table 2-7 Description of the HiAI_ModelBuffer_destroy interface
Function Description

Destroying the HIAI_ModelBuffer interface

Interface Prototype

void HiAI_ModelBuffer_destroy(HiAI_ModelBuffer* b);

Parameter Description

b: Specifies the HIAI_ModelBuffer to be destroyed.

Return Value

None

2.5.4 Obtaining the Input and Output Shape Information of a Model
After the model is loaded successfully, the input shape and output shape information of the specified model
can be read. After the model is read, the input and output shape information is saved in the memory, and
then the HIAI_ModelTensorInfo memory is released.

2018-03-19

Huawei confidential. No spreading without permission.

Page 9 of 16

Huawei HiAI DDK User Manual

INTERNAL

2.5.4.1 Obtaining the Input and Output Shape Information of a Model
Table 2-8 Description of the HIAI_ModelManager_getModelTensorInfo interface
Function Description

Obtaining the input and output shape information of a model

Interface Prototype

HIAI_ModelTensorInfo*
HIAI_ModelManager_getModelTensorInfo(HIAI_ModelManager*
manager, const char* modelName);

Parameter Description

manager: Specifies a model manager instance.
modelName: Specifies a model name.

Return Value

If the execution succeeds, the pointer to the input and output shape
information of a mode is returned. If the execution fails, null is returned.

2.5.4.2 Releasing the Memory That Stores the Input and Output Shape
Information of a Model
Table 2-9 Description of the HIAI_ModelManager_getModelTensorInfo interface
Function Description

Releasing the memory that stores the input and output shape information of
a model

Interface Prototype

void
HIAI_ModelManager_releaseModelTensorInfo(HIAI_ModelTensorInfo*
modelTensor);

Parameter Description

modelTensor: Specifies the pointer to the input and output shape
information of a model.

Return Value

None

2.5.5 Running the Model (1)
When the model is running, the interfaces for inputting feature data, running the model, and obtaining data
after model running are available.

2018-03-19

Huawei confidential. No spreading without permission.

Page 10 of 16

Huawei HiAI DDK User Manual

INTERNAL

2.5.5.1 Creating the HIAI_TensorBuffer Interface
Table 2-10 Description of the HiAI_TensorBuffer_create interface
Function Description

Creating the HIAI_TensorBuffer interface

Interface Prototype

HiAI_TensorBuffer* HiAI_TensorBuffer_create(int n, int c, int h, int w);

Parameter Description

nchw of model input or output
n: batch of the tensor
c: channel of the tensor
h: height of the tensor
w: width of the tensor

Return Value

HIAI_TensorBuffer

2.5.5.2 Obtaining the Input or Output Data Address of a Model
Table 2-11 Description of the HiAI_TensorBuffer_getRawBuffer interface
Function Description

Obtaining the data address of model input or output

Interface Prototype

void* HiAI_TensorBuffer_getRawBuffer(HiAI_TensorBuffer* b);

Parameter Description

b: Specifies the HIAI_TensorBuffer if the model input or output.

Return Value

Data address of the model input or output

2.5.5.3 Obtaining the Data Length of Model Input or Output
Table 2-12 Description of the HiAI_TensorBuffer_getBufferSize interface
Function Description

Obtaining the data length of model input or output

Interface Prototype

int HiAI_TensorBuffer_getBufferSize(HiAI_TensorBuffer* b);

Parameter Description

b: Specifies the HIAI_TensorBuffer of the model input or output.

Return Value

Data length of model input or output

2.5.5.4 Running a Model
Table 2-13 Description of the HiAI_ModelManager_runModel interface
Function Description

2018-03-19

Running a model

Huawei confidential. No spreading without permission.

Page 11 of 16

Huawei HiAI DDK User Manual

Interface Prototype

INTERNAL

int HiAI_ModelManager_runModel(
HiAI_ModelManager* manager,
HiAI_TensorBuffer* input[],
int nInput,
HiAI_TensorBuffer* output[],
int nOutput,
int ulTimeout,
const char* modelName);

Parameter Description

manager: Specifies the object interface of the model management engine.
input[]: Specifies the model input. Multiple inputs are supported.
nInput: Specifies the number of inputs by a model.
output[]: Specifies the model output. Multiple outputs are supported.
nOutput: Specifies the number of outputs by a model.
ulTimeout: Specifies the timeout, which does not take effect during
synchronous invoking.
modelName: Specifies a model name.

Return Value

If the execution succeeds, 0 is returned. If the execution fails, an error
code is returned.

2.5.5.5 Destroying the HIAI_TensorBuffer Interface
Table 2-14 Description of the HiAI_TensorBuffer_destroy interface
Function Description

Destroying the HIAI_TensorBuffer interface

Interface Prototype

void HiAI_TensorBuffer_destroy(HiAI_TensorBuffer* b);

Parameter Description

b: Specifies the HIAI_TensorBuffer to be destroyed.

Return Value

None

2.5.6 Running the Model (2)
Besides the HIAI_ModelManager_runModel interface described in section 2.6.4zz, the interfaces described
in section 2.6.5zz can also be used for running models. Section 2.6.5zz describes only synchronization
interfaces.

2.5.6.1 Setting the Model Input and Output
Table 2-15 Description of the HiAI_ModelManager_setInputsAndOutputs interface
Function
Description

2018-03-19

Setting the model input and output

Huawei confidential. No spreading without permission.

Page 12 of 16

Huawei HiAI DDK User Manual

Interface
Prototype

INTERNAL

int HiAI_ModelManager_setInputsAndOutputs(
HiAI_ModelManager* manager,
const char* modelname,
HiAI_TensorBuffer* input[],
int nInput,
HiAI_TensorBuffer* output[],
int nOutput);

Parameter
Description

manager: Specifies the object interface of the model management engine.
modelName: Specifies a model name.
input[]: Specifies the model input. Multiple inputs are supported.
nInput: Specifies the number of inputs by a model.
output[]: Specifies the model output. Multiple outputs are supported.
nOutput: Specifies the number of outputs by a model.

Return Value

If the execution succeeds, 0 is returned. If the execution fails, an error code is
returned.

2.5.6.2 Starting Computing
Table 2-16 Description of the HiAI_ModelManager_startCompute interface
Function
Description

Starting computing

Interface
Prototype

int HiAI_ModelManager_startCompute(HiAI_ModelManager* manager, const
char* modelname);

Parameter
Description

manager: Specifies the object interface of the model management engine.

Return Value

If the execution succeeds, 0 is returned. If the execution fails, an error code is
returned.

modelName: Specifies a model name.

2.5.7 Unloading a Model
After all data is processed, this interface is called to unload a model.

2018-03-19

Huawei confidential. No spreading without permission.

Page 13 of 16

Huawei HiAI DDK User Manual

INTERNAL

Table 2-17 Description of the HiAI_ModelManager_unloadModel interface
Function
Description

Unloading a model

Interface
Prototype

int HiAI_ModelManager_unloadModel(HiAI_ModelManager* manager);

Parameter
Description

manager: Specifies the object interface of the model management engine.

Return Value

If the execution succeeds, 0 is returned. If the execution fails, an error code is
returned.

2.5.8 Destroying a Model Manager
After the model is uninstalled, you can destroy the model manager.
Table 2-18 Description of the HiAI_ModelManager_destroy interface
Function
Description

Destroying the model manager.

Interface
Prototype

void HiAI_ModelManager_destroy(HiAI_ModelManager* manager);

Parameter
Description

manager: Specifies the object interface of the model management engine.

Return Value

None

3 Integration
For details, see the Huawei HiAI DDK Integration Manual.

4 Appendix
4.1 Definition of Error Codes
Error Code

Error Type

Prototype

Triggering Condition

1

The length of a model
name is incorrect.

MODEL_NAME_LEN_ERROR

The model length ranges
from 1 to 128.

2

The model file path is
empty.

MODEL_DIR_ERROR

The model file path is
empty.

3

The length of the
decryption key of a
model file is incorrect.

MODEL_SECRET_KEY_ERROR

The length of the
decryption key is not 0 or
64.

2018-03-19

Huawei confidential. No spreading without permission.

Page 14 of 16

Huawei HiAI DDK User Manual

INTERNAL

Error Code

Error Type

Prototype

Triggering Condition

4

The length of the
decryption key of a
model parameter file
is incorrect.

MODEL_PARA_SECRET_KEY_
ERROR

The length of the
decryption key is not 0 or
64.

5

The selected model
framework type is
incorrect.

FRAMEWORK_TYPE_ERROR

The selected framework
type is not TensorFlow,
Caffe, or Kaldi.

6

The selected model
type is incorrect.

MODEL_TYPE_ERROR

Th selected model type is
not online or offline.

7

The IPU frequency is
incorrectly set.

IPU_FREQUENCY_ERROR

The frequency is not low,
normal, or high.

8

The number of loaded
models is incorrect.

MODEL_NUM_ERROR

The number of models is
0 or greater than 20.

9

The model size is
incorrect.

MODEL_SIZE_ERROR

The model size is 0.

10

The configured
timeout is incorrect.

TIMEOUT_ERROR

The configured timeout is
greater than 60000 ms.

11

The shape of the input
data is incorrect.

INPUT_DATA_SHAPE_ERROR

The value of n x c x h x
w is 0.

12

The shape of output
data is incorrect.

OUTPUT_DATA_SHAPE_ERRO
R

The value of n x c x h x
w is 0.

13

The number of input
data segments is
incorrect.

INPUT_DATA_NUM_ERROR

The number of input data
segments is 0 or greater
than 20.

14

The number of output
data segments is
incorrect.

OUTPUT_DATA_NUM_ERROR

The number of output
data segments is 0 or
greater than 20.

15

The number of created
model manager
instances exceeds the
upper limit.

MODEL_MANAGER_TOO_MAN
Y_ERROR

More than three clients
are created for a single
process.

18

The model name is
repeated.

MODEL_NAME_DUPLICATE_E
RROR

Multiple models have the
same name.

19

The HiAIserver
connection fails.

HiAI_SERVER_CONNECT_ERR
OR

The HiAIserver service is
not started.

20

The HiAIserver
connection is tore
down.

HiAI_SERVER_CONNECT_IRPT

The HiAIserver
connection is tore down.

500

The input or output
nchw value does not
match the model n x c
x h x w.

MODEL_TENSOR_SHAPE_NO_
MATCH

The input or output nchw
value does not match the
model n x c x h x w.

2018-03-19

Huawei confidential. No spreading without permission.

Page 15 of 16

Huawei HiAI DDK User Manual

INTERNAL

Error Code

Error Type

Prototype

Triggering Condition

999

The interface lifecycle
expires.

EXPIRATION_FUCNTION

The compilation model
interface is invoked.

1000

An internal error
occurs.

INTERNAL_ERROR

An internal error occurs.

2018-03-19

Huawei confidential. No spreading without permission.

Page 16 of 16
Source Exif Data: 
File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.5
Linearized                      : No
Page Count                      : 16
Language                        : zh-CN
Tagged PDF                      : Yes
Title                           : XXX Procedure
Author                          : wtest222
Creator                         : Microsoft® Word 2013
Create Date                     : 2018:03:19 17:48:42+08:00
Modify Date                     : 2018:03:19 17:48:42+08:00
Producer                        : Microsoft® Word 2013
EXIF Metadata provided by EXIF.tools

Navigation menu