OneMediaHub Version 14.5 Installation And Operation Guide 14 5 0
User Manual:
Open the PDF directly: View PDF .
Page Count: 251
Download | |
Open PDF In Browser | View PDF |
OneMediaHub Version 14.5 Installation and Operation Guide OneMediaHub Version 14.5 Installation and Operation Guide Version 14.5.0 - Revision 175192 [2014-12-12] Copyright © 2012-2014 Funambol, Inc. This document is provided for informational purposes and should be used for reference only. 1 Introduction ........................................................................................................................ 1 1.1 Audience .................................................................................................................. 1 1.2 Prerequisites ............................................................................................................. 1 2 System overview .................................................................................................................. 2 2.1 OneMediaHub architecture .......................................................................................... 2 2.1.1 Roles and responsibilities ................................................................................. 2 2.2 The Synchronization Engine ........................................................................................ 5 2.3 Execution flow of a request ......................................................................................... 5 2.4 OneMediaHub push ................................................................................................... 6 2.4.1 Cloud push using APNS ................................................................................... 7 2.4.2 SMS push ...................................................................................................... 7 2.4.3 Push compatibility table ................................................................................... 8 2.5 OneMediaHub clustering ............................................................................................ 8 2.5.1 Data Synchronization Service cluster .................................................................. 9 2.5.2 PIM Listener Service cluster ........................................................................... 10 2.6 Media synchronization .............................................................................................. 11 2.6.1 File system structure ...................................................................................... 11 2.6.2 -ext subdirectories ....................................................................................... 12 2.6.3 Deleting media files ....................................................................................... 12 2.6.4 Temporary items management ......................................................................... 13 2.6.5 Scaling the file system ................................................................................... 13 2.6.6 Encryption ................................................................................................... 13 2.6.6.1 How encryption works ......................................................................... 13 2.6.6.2 How to configure the server to use encryption .......................................... 13 2.6.7 Storing files on an online file storage provider .................................................... 14 2.6.7.1 How to create an Amazon S3 bucket ...................................................... 14 2.6.8 Transcoding .................................................................................................. 16 2.6.8.1 How to configure Amazon Simple Notification Service .............................. 17 2.6.8.2 How to create a pipeline on Amazon Elastic Transcoder ............................. 19 2.6.9 Deleting transcoding jobs ................................................................................ 20 3 Installation and configuration ............................................................................................... 21 3.1 Installing the OneMediaHub ...................................................................................... 21 3.2 The config directory ............................................................................................. 22 3.2.1 OneMediaHub configuration files ..................................................................... 22 3.3 Quick configuration .................................................................................................. 22 3.4 Database configuration .............................................................................................. 27 3.4.1 MySQL database creation ............................................................................... 27 3.4.2 Further database configuration ......................................................................... 28 3.4.3 Limiting the maximum number of open connections ............................................ 30 3.4.4 Database partitioning ...................................................................................... 30 3.4.5 MySQL events .............................................................................................. 30 3.5 IPv6 Support ........................................................................................................... 31 3.6 Adding new nodes to a OneMediaHub cluster ............................................................... 31 3.6.1 Adding a new Data Synchronization Service node to the cluster ............................. 31 3.6.2 Adding a new PIM Listener Service node to the cluster ........................................ 32 3.7 Data Synchronization Service configuration .................................................................. 32 3.7.1 Funambol.xml .......................................................................................... 32 3.8 PIM Listener Service configuration ............................................................................. 34 3.8.1 PIMListenerConfiguration.xml ........................................................... 34 3.9 Data Synchronization Service cluster configuration ........................................................ 35 3.10 Enabling default users ............................................................................................. 35 3.11 Starting/stopping OneMediaHub services .................................................................... 36 3.11.1 Starting/stopping all services together .............................................................. 37 3.11.2 Starting/stopping the Data Synchronization service ............................................ 37 iii OneMediaHub Version 14.5 Installation and Operation Guide 3.11.3 Starting/stopping the PIM Listener service ........................................................ 37 3.12 Portal configuration ................................................................................................ 37 3.12.1 Configuring the login with or without country code ............................................ 37 3.12.2 ROOT.xml ................................................................................................. 38 3.12.3 portal-ext.properties ....................................................................... 39 3.12.3.1 sp.syncportal.messages.url and sp.syncportal.url .......... 39 3.12.3.2 admin.email.from.address and admin.email.from.name ...... 39 3.12.3.3 sp.mediaserver.url .................................................................. 39 3.12.3.4 sync.slow.min-interval .......................................................... 40 3.12.3.5 sync.blocked-sources .............................................................. 40 3.12.3.6 sync.min-interval .................................................................... 40 3.12.3.7 sync.max-session-allowed ...................................................... 40 3.12.3.8 sync.response-time-threshold ............................................... 40 3.12.3.9 sp.syncportal.device.url.automaticRedirect, sp.syncportal.device.url.androidApp, sp.syncportal.device.url.iPhoneApp, and sp.syncportal.device.url.windowsPhoneApp .................................. 41 3.12.3.10 sp.syncportal.url.download.page ....................................... 41 3.12.3.11 storage.provider, storage.identity, storage.credential and storage.container-name ........................ 41 3.12.3.12 ets.pipeline-id, ets.client-region and ets.s3container-name ...................................................................................... 41 3.12.3.13 audio.enabled ........................................................................... 42 3.12.3.14 passwords.toolkit and passwords.regexptoolkit.pattern .................................................... 42 3.12.3.15 passwords.allow.username ..................................................... 42 3.12.3.16 push.apple.keystore.file, push.apple.keystore.password, push.apple.production and push.apple.connection-pool-size .................................................... 43 3.12.3.17 subscription.enabled ............................................................. 43 3.12.3.18 subscription.warning-before-renewal-in-minutes, subscription.insufficient-funds-delay-in-minutes, subscription.service-error-delay-in-minutes, subscription.deletion-delay-in-minutes, subscription.keep-trying-to-charge-in-minutes, subscription.allow-downgrade-overquota, subscription.allow-immediate-downgrade ...................................... 43 3.12.3.19 subscription.notification-sender-class, subscription.notification-builder-class, subscription.payment-service-class, subscription.managerclass, subscription.currency ............................................................ 44 3.12.3.20 subscription.payment.web ..................................................... 44 3.12.3.21 subscription.paymentverifier.apple.sandbox ................. 44 3.12.3.22 subscription.paymentverifier.web.validationurl ......... 44 3.12.3.23 media.account-pre-population ............................................. 44 3.12.3.24 ui.media.upload-file-size-limit and ui.media.multiple-upload-size-limit ............................................ 44 3.12.3.25 sapi.upload.max-concurrent-uploads ................................. 45 3.12.3.26 sapi.upload.danger-zone.concurrent-uploads ................. 45 3.12.3.27 sapi.upload.danger-zone.device.max-concurrentuploads .................................................................................................... 45 3.12.3.28 user-import-tool.device.countrya2, user-importtool.device.carrierid and user-import-tool.device.modelid ................................................................................................................... 45 iv OneMediaHub Version 14.5 Installation and Operation Guide 3.12.3.29 user-communication.default-channel ................................. 45 3.12.3.30 sapi.login.persistent-login-token-max-age-in-days ................................................................................................................... 46 3.12.3.31 sharing.email-counter.max-messages ................................. 46 3.12.4 Restricting access to administrative Server API calls .......................................... 46 3.12.5 How to configure the OAuth 2.0 client ............................................................ 47 3.12.6 How to enable user subscriptions .................................................................... 49 3.12.7 How to view and edit current subscription plans ................................................ 51 3.12.8 How to enable user validation in Mobile signup ................................................ 51 3.12.9 How to enable user validation in Portal signup .................................................. 51 3.12.10 How to enable gzip compression ................................................................... 52 3.12.11 How to configure the disk quota for media ..................................................... 53 3.12.12 How to configure quota notification .............................................................. 55 3.12.13 How to configure the Portal to store items on the local file system ....................... 55 3.12.13.1 How to configure the Portal to store media on the local file system ............ 55 3.12.14 Serving media files using the Apache web server ............................................. 56 3.12.15 How to configure the Portal to store items on an online file storage provider .......... 57 3.12.15.1 How to configure the Portal to store Media items on an online file storage provider ...................................................................................................... 57 3.12.16 How to configure the max item size allowed by the DS Server ............................ 58 3.12.16.1 How to configure the max picture size allowed by the DS Server .............. 58 3.12.16.2 How to configure the max video size allowed by the DS Server ................ 59 3.12.16.3 How to configure the max file and music size allowed by the DS Server ..... 59 3.12.17 How to configure the Server API base URL .................................................... 59 3.12.18 Help link .................................................................................................. 60 3.12.19 Contact Us link ......................................................................................... 60 3.12.20 How to configure Google Analytics ............................................................... 61 3.12.21 How to enable iOS Push ............................................................................. 61 3.12.22 How to enable/disable music management ...................................................... 61 3.12.23 How to enable media transcoding .................................................................. 61 3.12.24 How to customize the Privacy Policy and Term of Use pages redirecting the OneMediaHub links to external pages. ...................................................................... 62 3.12.25 How to configure user-level communication channel ........................................ 62 3.13 Server URL configuration ........................................................................................ 63 3.13.1 Changing the default server port ..................................................................... 63 3.13.2 Supporting multiple server URL ..................................................................... 63 3.14 Email configuration ................................................................................................ 64 3.14.1 New user messages ...................................................................................... 64 3.14.2 Email counter configuration for messages containing the app download URL .......... 64 3.15 Reminder Emails .................................................................................................... 65 3.16 SMS Sender configuration ....................................................................................... 65 3.17 OTA settings provisioning configuration .................................................................... 66 3.18 Forgot password ..................................................................................................... 67 3.19 Push policy configuration ........................................................................................ 68 3.19.1 The PushSender configuration file .................................................................. 68 3.20 OneMediaHub SNMP Appender configuration ............................................................ 70 3.20.1 Configuration parameters .............................................................................. 70 3.20.2 Configuration file ........................................................................................ 71 3.20.3 Log records filtering ..................................................................................... 74 3.20.4 OneMediaHub SNMP error trap ..................................................................... 74 3.21 Location from IP address and Accept-Language header ........................................... 74 3.21.1 IP address and IP2Location ........................................................................... 75 3.21.2 Accept-Language header ......................................................................... 75 3.22 Collecting client logs .............................................................................................. 75 v OneMediaHub Version 14.5 Installation and Operation Guide 3.22.1 Configuration parameters .............................................................................. 75 3.23 Client log cleanup service ........................................................................................ 76 3.23.1 Configuration file ........................................................................................ 76 3.24 Antivirus service .................................................................................................... 76 3.24.1 Installing McAfee VirusScan ......................................................................... 77 3.24.2 Updating virus definitions of McAfee VirusScan ............................................... 77 3.24.3 Configuring the OneMediaHub server ............................................................. 78 4 OneMediaHub Portal administration ...................................................................................... 79 4.1 Changing the admin user password ............................................................................. 79 4.2 Handling user roles .................................................................................................. 79 4.3 Creating new administrative users ............................................................................... 81 5 Operation tasks .................................................................................................................. 82 5.1 Monitoring OneMediaHub services ............................................................................. 82 5.1.1 Monitoring the Data Synchronization Service ..................................................... 82 5.1.2 Monitoring the PIM Listener Service ................................................................ 87 5.2 Storage cleanup ....................................................................................................... 89 5.3 Remove orphan media files from file system storage provider .......................................... 89 5.4 How to adjust the startup memory of the JVM .............................................................. 90 5.4.1 Data Synchronization Service .......................................................................... 90 5.4.2 PIM Listener Service ..................................................................................... 90 5.5 Synchronizing node clocks ........................................................................................ 91 5.5.1 Installing NTP .............................................................................................. 91 5.5.2 Checking that NTP is synchronized .................................................................. 91 5.5.3 Configuring NTP ........................................................................................... 91 5.6 Configuring OneMediaHub load balancing with Apache HTTP Server (httpd) and mod_cluster ................................................................................................................. 92 5.6.1 httpd requirements ...................................................................................... 92 5.6.2 Installing mod_cluster httpd modules ............................................................. 92 5.6.3 Configuring OneMediaHub instances for running in a cluster ................................. 93 5.6.3.1 Enabling mod_cluster .......................................................................... 93 5.6.4 Simple deployment architectures ...................................................................... 94 5.6.4.1 Case 1 .............................................................................................. 94 5.6.4.2 Case 2 .............................................................................................. 96 5.6.5 How to migrate OneMediaHub from mod_cluster v1.2.0 to mod_cluster v1.2.6 ......... 97 5.6.5.1 Update of the httpd .......................................................................... 98 5.6.5.2 Update of the OneMediaHub installation ................................................. 98 5.6.5.3 How to redirect HTTP traffic to HTTPS .................................................. 99 5.7 How to change the sync URL .................................................................................... 99 5.8 Setting the Portal URL in device configuration pages .................................................... 100 5.9 How to install a patch ............................................................................................. 101 5.9.1 Rollback procedure ...................................................................................... 101 5.10 How to install clients ............................................................................................ 102 5.11 How to install a new Portal User Interface ................................................................ 102 5.12 Using the Device Simulator Tool ............................................................................ 102 5.13 Gathering information about the OneMediaHub environment ........................................ 103 5.14 Performance statistics ............................................................................................ 103 5.14.1 Examples .................................................................................................. 104 5.15 Event tracking and reporting ................................................................................... 107 5.15.1 Antivirus .................................................................................................. 108 5.15.2 Media ...................................................................................................... 108 5.15.3 Push flow ................................................................................................. 108 5.15.4 Sync ........................................................................................................ 109 5.15.5 Sync session .............................................................................................. 109 5.15.6 Transcoding .............................................................................................. 109 vi OneMediaHub Version 14.5 Installation and Operation Guide 5.15.7 Events information by database columns ........................................................ Configuring the Stuck Thread Detection valve ........................................................... How to enable and disable devices .......................................................................... How to disable clients offered in mobile portal .......................................................... Marketing KPI ..................................................................................................... 5.19.1 How to use ............................................................................................... 5.20 Import users tool .................................................................................................. 5.20.1 Pre-conditions ............................................................................................ 5.20.2 User command batch file ............................................................................. 5.20.3 Usage ....................................................................................................... 5.20.4 Commands ................................................................................................ 5.20.5 Configuration ............................................................................................ 5.20.6 Status codes .............................................................................................. 5.21 User reporting tool ................................................................................................ 5.21.1 How to use ............................................................................................... 5.22 How to enable OneMediaHub proxy support ............................................................. Database partitioning ......................................................................................................... 6.1 Creating core and user databases ............................................................................... 6.2 Separating the user database ..................................................................................... 6.3 Separating the reporting database .............................................................................. 6.4 Partitioning the user database ................................................................................... 6.5 MySQL replication ................................................................................................. 6.5.1 What does MySQL replication mean? .............................................................. 6.5.2 MySQL Connector/J ..................................................................................... Logging .......................................................................................................................... 7.1 Changing the log rotation frequency and size .............................................................. 7.1.1 Daily log rotation ........................................................................................ 7.1.2 Hourly log rotation ...................................................................................... 7.2 Changing the logging level ...................................................................................... 7.3 Understanding log files ........................................................................................... 7.3.1 Example ..................................................................................................... 7.4 Customizing DS Service's log settings ....................................................................... 7.4.1 Customizing log files on a per-user basis ......................................................... 7.5 Syslog configuration ............................................................................................... 7.6 SNMP errors ......................................................................................................... Configuring External Services ............................................................................................. 8.1 Introduction ........................................................................................................... 8.2 General requirements .............................................................................................. 8.3 Application keys .................................................................................................... 8.3.1 Google external services authorization (Picasa, YouTube, and Gmail contacts/ calendar import) .................................................................................................. 8.3.2 Flickr ......................................................................................................... 8.3.3 Facebook .................................................................................................... 8.3.3.1 Review and submission ...................................................................... 8.3.3.2 Native login and save authorization tokens support .................................. 8.3.4 Twitter ....................................................................................................... 8.4 Configuring the Portal ............................................................................................. 8.5 Troubleshooting ..................................................................................................... OneMediaHub Server URL configuration .............................................................................. 9.1 Introduction ........................................................................................................... 9.1.1 Conventions ................................................................................................ 9.2 Server available on two different URLs ...................................................................... 9.3 Server with a new URL (same IP address) .................................................................. 9.4 Server with a new IP address (same URL) .................................................................. 5.16 5.17 5.18 5.19 6 7 8 9 vii 110 110 111 112 112 115 116 116 116 118 118 119 119 120 121 121 123 124 124 126 126 128 129 129 130 130 132 132 132 133 134 135 135 136 138 140 140 140 140 140 146 149 152 156 158 161 162 163 163 163 163 163 164 OneMediaHub Version 14.5 Installation and Operation Guide 9.5 Changing external service settings ............................................................................. Appendix A Default Ports used by OneMediaHub ..................................................................... Appendix B Device IDs ....................................................................................................... Appendix C OneMediaHub error messages .............................................................................. Appendix D FUNAMBOL-LOGGING-MIB.txt ........................................................................ Appendix E Database Schema ............................................................................................... E.1 OneMediaHub Core ............................................................................................... E.1.1 Contact_ .................................................................................................... E.1.2 Country ..................................................................................................... E.1.3 fnbl_bucket ................................................................................................ E.1.4 Partitioning Schema ..................................................................................... E.1.5 fnbl_connector ............................................................................................ E.1.6 fnbl_connector_source_type .......................................................................... E.1.7 fnbl_country_language ................................................................................. E.1.8 fnbl_country_timezones ................................................................................ E.1.9 fnbl_deleted_sync_user_role .......................................................................... E.1.10 fnbl_deleted_user ....................................................................................... E.1.11 fnbl_device ............................................................................................... E.1.12 fnbl_device_caps ....................................................................................... E.1.13 fnbl_device_datastore ................................................................................. E.1.14 fnbl_email_account .................................................................................... E.1.15 fnbl_email_enable_account .......................................................................... E.1.16 fnbl_external_service .................................................................................. E.1.17 fnbl_external_service_account ...................................................................... E.1.18 fnbl_family ............................................................................................... E.1.19 fnbl_family_user ........................................................................................ E.1.20 fnbl_file_data_object_user_lock .................................................................... E.1.21 fnbl_id ..................................................................................................... E.1.22 fnbl_import_csv_commands ......................................................................... E.1.23 fnbl_last_deleted_msisdn ............................................................................. E.1.24 fnbl_last_sync ........................................................................................... E.1.25 fnbl_module .............................................................................................. E.1.26 fnbl_module_connector ............................................................................... E.1.27 fnbl_partition ............................................................................................ E.1.28 fnbl_pim_listener_registry ........................................................................... E.1.29 PIM Push Tables Schema ............................................................................ E.1.30 fnbl_principal ............................................................................................ E.1.31 fnbl_public_mailservers .............................................................................. E.1.32 fnbl_push_listener_registry .......................................................................... E.1.33 fnbl_push_sender_notification ...................................................................... E.1.34 fnbl_role .................................................................................................. E.1.35 fnbl_storage .............................................................................................. E.1.36 fnbl_subscription_family ............................................................................. E.1.37 fnbl_subscription_payment .......................................................................... E.1.38 fnbl_subscription_plan ................................................................................ E.1.39 fnbl_sync_history ....................................................................................... E.1.40 fnbl_sync_source ....................................................................................... E.1.41 fnbl_sync_source_type ................................................................................ E.1.42 fnbl_temp_paying_users .............................................................................. E.1.43 fnbl_timezone ........................................................................................... E.1.44 fnbl_user .................................................................................................. E.1.45 fnbl_user_alias .......................................................................................... E.1.46 fnbl_user_picture ....................................................................................... E.1.47 fnbl_user_preference .................................................................................. viii 164 165 166 167 174 178 178 178 178 179 179 179 180 180 180 180 181 181 182 182 183 184 184 184 185 185 185 185 186 186 187 187 188 188 188 189 189 189 190 190 191 191 191 191 192 193 194 194 194 195 195 195 196 196 OneMediaHub Version 14.5 Installation and Operation Guide E.1.48 fnbl_user_properties ................................................................................... E.1.49 fnbl_user_role ........................................................................................... E.1.50 fnbl_user_subscription ................................................................................ E.1.51 fp_carrier ................................................................................................. E.1.52 fp_device ................................................................................................. E.1.53 fp_manufacturer ........................................................................................ E.1.54 fp_model .................................................................................................. E.1.55 User_ ....................................................................................................... E.1.56 Other Liferay tables ................................................................................... E.2 OneMediaHub User ................................................................................................ E.2.1 fnbl_client_mapping ..................................................................................... E.2.2 fnbl_comment ............................................................................................. E.2.3 fnbl_device_config ...................................................................................... E.2.4 fnbl_email_folder ........................................................................................ E.2.5 fnbl_email_inbox ......................................................................................... E.2.6 fnbl_email_sentpop ...................................................................................... E.2.7 fnbl_exported_file_data_object ....................................................................... E.2.8 fnbl_file_data_object .................................................................................... E.2.9 fnbl_file_data_object_label_items ................................................................... E.2.10 fnbl_file_data_object_property ..................................................................... E.2.11 fnbl_file_data_object_set ............................................................................. E.2.12 fnbl_file_data_object_set_item ..................................................................... E.2.13 fnbl_file_data_object_tag ............................................................................. E.2.14 fnbl_file_data_object_thumbnail ................................................................... E.2.15 fnbl_folder ................................................................................................ E.2.16 fnbl_label ................................................................................................. E.2.17 fnbl_last_activity ....................................................................................... E.2.18 fnbl_last_activity_item ................................................................................ E.2.19 fnbl_pending_notification ............................................................................ E.2.20 fnbl_pim_address ....................................................................................... E.2.21 fnbl_pim_calendar ...................................................................................... E.2.22 fnbl_pim_calendar_alarm ............................................................................ E.2.23 fnbl_pim_calendar_attendee ......................................................................... E.2.24 fnbl_pim_calendar_exception ....................................................................... E.2.25 fnbl_pim_contact ....................................................................................... E.2.26 fnbl_pim_contact_item ................................................................................ E.2.27 fnbl_pim_contact_photo .............................................................................. E.2.28 fnbl_pim_note ........................................................................................... E.2.29 fnbl_pim_organization ................................................................................ E.2.30 fnbl_temp_paying_users .............................................................................. E.3 OneMediaHub Reporting ......................................................................................... E.3.1 fnbl_client_download_stats ............................................................................ E.3.2 fnbl_event .................................................................................................. E.3.3 fnbl_marketing_kpi ...................................................................................... Appendix F Examples of sent SNMP traps .............................................................................. Appendix G Xuggle Xuggler FAQs ........................................................................................ Appendix H Capptain integration on Android App .................................................................... Glossary ............................................................................................................................ References ......................................................................................................................... ix 197 197 198 199 199 199 200 201 202 202 202 203 203 203 204 204 204 205 207 208 208 208 209 209 209 210 210 210 211 211 212 214 215 217 217 219 220 220 221 221 222 222 222 223 227 230 233 239 240 Chapter 1. Introduction This document details how to install, configure, operate, and maintain the OneMediaHub; it also describes the system architecture and the role and usage of the different components. Funambol also offers a set of Server APIs that can be used as extensions to OneMediaHub features and to build third party applications into it; one example is the AJAX Portal, available out of the box in the OneMediaHub, and completely built on top of the Server API layer. For developers' specific documentation on Server APIs, please refer to [26]. 1.1 Audience This guide is addressed to system administrators. 1.2 Prerequisites Important OneMediaHub is supported only on GNU/Linux 32/64-bit distributions. The OneMediaHub installation depends on the installation of certain software packages on the target system. The following packages must be installed and can all be downloaded for free from the Internet. Please make sure that the package version is correct; if it is not or if a package is not installed, please download the correct package from the URL provided: • MySQL 5.5 or 5.6 (see [5]) • MySQL Connector/J (see [6]) • Java Platform (JDK) 7 (see [3]) Note If you wish to run the OneMediaHub on a 64-bit architecture, you must use the 64-bit JDK. Warning OneMediaHub does not support OpenJDK. • Xuggle Xuggler (see [29] and Appendix G, Xuggle Xuggler FAQs) 1 Chapter 2. System overview The following sections describe the OneMediaHub architecture and the various OneMediaHub components. To get started with the installation procedure, skip to Chapter 3, Installation and configuration. 2.1 OneMediaHub architecture The system deployment architecture in the OneMediaHub is logically made up of the components illustrated in Figure 2.1, “OneMediaHub system architecture”. For the sake of clarity, each component in the figure is represented by a single box, but as explained later, all systems can be made redundant to increase availability and support a greater load. The overall goal of the system is to offer cloud-based synchronization of Media (Pictures, Videos, and Files) and PIM (Contacts, Calendar, Tasks, and Notes) across mobile phones, tablets, computers, and other devices. 2.1.1 Roles and responsibilities This section describes the role and the main responsibilities of the components illustrated in Figure 2.1, “OneMediaHub system architecture”. Figure 2.1. OneMediaHub system architecture Device Any physical device (phone, tablet, computer, connected device) that can communicate with the OneMediaHub server for Media and/or PIM Sync, either natively or through a downloadable client. Examples are: • smartphones or tablets (e.g. iPhone/iPad, Android, BlackBerry, etc.) running OneMediaHub • mobile phones with a native SyncML client • computers running OneMediaHub for Windows Devices are the main interface through which users access OneMediaHub. Main responsibilities include: • providing the graphical user interface • initiating the communication with the server 2 System overview • hosting the local data (address book, pictures, etc.) • hosting the sync engine (for Media only) • collecting/detecting the changes The communication between the device and the OneMediaHub Server is based on the TCP/IP protocol. HTTP Load Balancer Both for Media and for PIM, device-server communication is built on protocols transported over HTTP. As for common HTTP traffic, a load balancer (see Figure 2.1, “OneMediaHub system architecture”) can therefore be used to balance the incoming load by distributing it amongst different nodes of a server cluster. The main responsibilities of the HTTP load balancer include: • providing the front-end of the OneMediaHub system • distributing the device requests amongst the nodes of the server-side cluster • detecting failures on the cluster's nodes, redirecting traffic to the active nodes if one of the nodes fails Note The HTTP load balancer is not provided as part of the default installation or deployment. Many different solutions, both hardware and software, can be adopted and organizations may have different best practices already in place. A common solution is to use Apache and mod_cluster as described in Section 5.6, “Configuring OneMediaHub load balancing with Apache HTTP Server (httpd) and mod_cluster”. OneMediaHub Server The Server is the core of the OneMediaHub PIM synchronization. As illustrated in Figure 2.1, “OneMediaHub system architecture”, it comprises several components, described in detail in the following sections. Data Synchronization Service The role of the Data Synchronization (DS) Service is to provide the synchronization services and to communicate directly with the devices using the OMA DS protocol, formerly known as SyncML (see Section 2.3, “Execution flow of a request”). The main responsibilities of the Data Synchronization Service are: • hosting the synchronization engine (see Section 2.2, “The Synchronization Engine”) • accepting and serving synchronization requests • handling low level device information • providing an interface towards the back-end services Media Connector The Media Connector is the counterpart of the PIM Connector for Media synchronization. It is deployed together with the DS synchronization engine. It has the following responsibilities: • searching for the Media items that the user has modified on the server • storing the Media items on the Media Storage 3 System overview See Section 2.6, “Media synchronization” for more details. PIM Connector The PIM Connector allows the OneMediaHub server to sync PIM data such as contacts, events, and notes. It consists of two main components: the connector itself and the PIM Listener Service. The PIM Connector is deployed together with the DS synchronization engine. It has the following responsibilities: • searching the PIM items that the user has modified on the server • keeping the client updated with the PIM data stored on the server PIM Listener Service The PIM Listener Service is a separate process from the Data Synchronization Service and Portal process; it has the following responsibilities: • polling the user PIM database regularly to check for updates • triggering an action in the Data Synchronization Service if there are any changes to the user's PIM data to be delivered to the device Portal The OneMediaHub Portal implements the main interface through which users and administrators interact with the OneMediaHub platform over the Internet. The Portal component consists of • a web-based consumer portal, built with AJAX technology, through which users can sign up for the service, set up their devices, download the OneMediaHub apps, access and manage their Media data, PIM data, and profile • a web-based customer service representative (CSR) interface, which allows an operator to access user information and perform maintenance of user accounts Server API Funambol also offers a set of Server APIs that can be used to extend the OneMediaHub and to build third party applications on top of it; one example is the AJAX Portal, available out of the box within the OneMediaHub, and completely built on top of the Server API layer. The OneMediaHub client Apps use the Server APIs to perform Media synchronization without basing on the SyncML protocol, while PIM synchronization remains SyncML-based. For developers' documentation specific to the Server APIs, refer to [26]. Media Storage The Media Storage is the final repository where the Media Connector stores the user's media and files. In the current version, the OneMediaHub can use a local filesystem and an online store provider as well. See Section 2.6, “Media synchronization” for more details. SMS Service This is the service used to send SMS messages to user devices. The OneMediaHub platform uses an external SMS gateway for this, which translates the HTTP-based messages sent by the server into SMS messages, and injects them into the network servicing the target user. 4 System overview SMS messages are used to • send users the download link for OneMediaHub client Apps • configure over-the-air the native clients embedded in the phones • notify the device that a synchronization is needed because something has changed server-side: this is SMS push, as alternative to TCP-based push (see Section 2.4.2, “SMS push”) Note The SMS Service is not provided out of the box. OneMediaHub, by default, supports SubitoSMS. Support for other SMS service providers is configurable. For more information, see Important. SMTP Server This is the server used by the OneMediaHub to send Emails to external recipients. Email can be used for • inviting users to join the service • activating user accounts • supporting users (e.g. forgot password) • any other communication to users Database This is the database server. OneMediaHub supports the MySQL database system. 2.2 The Synchronization Engine The Synchronization Engine is the component that implements the synchronization logic, i.e. • identify the sources and the destinations of the data sets to be synchronized • identify the data that needs to be updated/added/deleted • determine how updates must be applied • detect conflicts • resolve conflicts In other words, the Synchronization Engine is the core of any data synchronization server. The basic framework interfaces and classes are grouped in the package sync4j.framework.engine. 2.3 Execution flow of a request The execution flow of an OMA DS request is illustrated in Figure 2.2, “Execution flow of an OMA DS request”. 5 System overview Figure 2.2. Execution flow of an OMA DS request A synchronization session starts with the client device sending a first SyncML message to the server. The request then follows the flow described below: 1. When a new request comes from the client, the HTTP handler takes care of it. After some processing, for example the transformation of the binary message into a more manageable form or the association of the incoming message to an existing synchronization session, the HTTP handler passes the request to the synchronization server. 2. The message first goes through the input message processing pipeline according to the application needs. 3. The manipulated message comes out of the input pipeline and goes into the server engine for synchronization processing. 4. When needed, the server engine calls the services of the external (and custom) SyncSources in order to access the real data stores. 5. After processing the incoming message, the server engine builds the response message, which goes through the output message processing pipeline for post-processing. 6. The response message is then returned to the HTTP handler, which packs the SyncML message into the HTTP response and sends it back to the device. 2.4 OneMediaHub push This section describes the OneMediaHub push technology. OneMediaHub push is based on the delivery of a so called push notification. A push notification is a small packet of data that OneMediaHub sends to a device in order to trigger a new synchronization. The notification package is technically called PKG#0 and contains information about the server that is requesting a synchronization, which data source must be synchronized, which type of synchronization should be performed. The PKG#0 can be delivered in many different ways. OneMediaHub supports the following delivery mechanisms: 1. Server-to-client Push • TCP/IP Push • Cloud push using Apple Push Notification Service (APNS) 6 System overview • SMS Push 2. Client-to-server Push Note New data is not automatically sent to the device; it is always the device that starts the communication for the exchange of data. Note Out-of-the-box, OneMediaHub detects the most appropriate push mechanism for the device and uses it. 2.4.1 Cloud push using APNS As illustrated in Figure 2.3, “Cloud push using APNS”, the iPhone obtains a token from the APNS server and registers the token on OneMediaHub. OneMediaHub then uses the registered token to send push notifications to a particular device. Figure 2.3. Cloud push using APNS 2.4.2 SMS push As illustrated in Figure 2.4, “SMS push”, in the case of SMS push, PKG#0 is delivered with one or more SMS messages. The basic flow is the same as in the other two techniques: once the device receives an SMS containing a push notification, it starts a new synchronization for the specified data sources. 7 System overview Figure 2.4. SMS push Note This mechanism requires an integration with an SMS service that is able to deliver binary SMS messages (see the section called “SMS Service” for more details). 2.4.3 Push compatibility table Not all devices have the same capabilities in terms of push or even synchronization. Certain devices have a built-in PIM SyncML client, others do not. The table below shows which push technology can be used with particular classes of devices. Device class APNS SMS push PIM BlackBerry devices (with OneMediaHub for BlackBerry) N N iPhone (with OneMediaHub) N N Symbian devices (with OneMediaHub for Symbian) N N Android devices (with OneMediaHub for Android) N N Desktop clients (with OneMediaHub for Windows) N N MEDIA Desktop clients (with OneMediaHub for Windows) N N iPhone (with OneMediaHub) Y N 2.5 OneMediaHub clustering OneMediaHub clustering has been designed with the following principles in mind: • high availability: it must be possible to have redundant architectures for all components so that users will not experience a permanent error if a problem arises in one of the components in the system • high load support: the redundant components must work in a load balanced architecture • low maintenance: it must be easy to modify the configuration of each cluster 8 System overview • automatic recovery: no administrative action must be needed when a cluster node goes down in order for the load to be redistributed amongst the remaining nodes As illustrated in Figure 2.5, “OneMediaHub clustering”, which should be seen as an alternative representation of the elements in Figure 2.1, “OneMediaHub system architecture”, OneMediaHub can be split into three clusters: • Data Synchronization Service cluster • PIM Listener Service cluster Additional clustering techniques for improving high availability, performance and reliability of the Database are described in Chapter 6, Database partitioning. Figure 2.5. OneMediaHub clustering 2.5.1 Data Synchronization Service cluster A Data Synchronization Service cluster is made up of one or more Data Synchronization Service nodes. Each node has the same properties and configuration as the others so that all nodes are identical from a SyncML client perspective. All nodes of the cluster must be installed on a network that allows IP multicast traffic. The multicast group of the Data Synchronization Service cluster has the following properties: Multicast group name ds-server Multicast address 228.10.58.01 Multicast port 47101 Note All Data Synchronization Service nodes in a cluster are dynamically aware of other nodes. This means that each node is dynamically updated with the changes in the cluster when a new node is 9 System overview added or removed. No administrative tasks are required when, for example, a new node is added in order to improve the number of users to support. Load balancing of SyncML and HTTP traffic is achieved by simply using any HTTP load balancing technique commonly used in this space. A common practice is to do this with an Apache load balancer battery, connected to the Data Synchronization Services through the mod_cluster module (see Figure 2.6, “Data Synchronization Service cluster”). Figure 2.6. Data Synchronization Service cluster See Section 5.6, “Configuring OneMediaHub load balancing with Apache HTTP Server (httpd) and mod_cluster” on how to configure Apache with mod_cluster in a clustered environment. Note The Data Synchronization Service stores its main configuration files in the file system under/config. If a cluster is installed, all nodes in this directory must have the same content. This can be achieved using a shared file system or keeping the node in sync with rsync. 2.5.2 PIM Listener Service cluster A PIM Listener Service cluster is made of one or more PIM Listener Service nodes; each node has the same properties and configuration as the others so that all nodes are completely interchangeable. The load of these listeners is measured in terms of how many users each PIM Listener Service monitors for changes; this load is automatically balanced by being distributed amongst all available nodes. This means that every time the cluster changes, all active nodes re-compute the subset of users they have to monitor, automatically redistributing the users. Note This distribution is based on a hashing algorithm which spreads users equally across the nodes of the cluster. Still, there is no guarantee that each node monitors exactly 1/N (where N is the number of nodes in the cluster) of the users at all times. 10 System overview All nodes of the cluster must be installed on a network that allows IP multicast traffic. The multicast group of the PIM Listener Service cluster has the following properties: Multicast group name pimlistener Multicast address 228.10.31.01 Multicast port 43101 Note All PIM Listener Service nodes in a cluster are dynamically aware of other nodes. This means that each node is dynamically updated with the changes in the cluster when a new node is added or removed. No administrative tasks are required when, for example, a new node is added in order to improve the number of users to support. 2.6 Media synchronization The OneMediaHub is able to synchronize media such as normal files, pictures and videos (from now on all of these will be simply referred to as media). Media files are stored on the Media Store - whereas media meta data is stored in the database. In the current version, OneMediaHub can use the local filesystem or an online storage provider as media storage. 2.6.1 File system structure For each user, the media sync sources define a subdirectory where all media files belonging to the user are stored. The entry points for the media types are: Picture /ds-server/db/picture File /ds-server/db/file Video /ds-server/db/video Note In a cluster environment, the above mentioned directories must be shared between all server nodes. To rationalize the file system, user directories are organized in a tree structure, where each user's directory path is structured in eight nested sub-directories, which may be located on different disks to split the disk load. The name of each subdirectory is composed of two characters, chosen from a pseudo-random string of 16 characters, e.g. /ds-server/ db/picture/ab/cd/ef/gh/il/mn/op/qr. 11 System overview The reason behind this complicated structure is that each directory can have 676 subdirectories (26 × 26), so for 8 levels the maximum number of subdirectories is 26^16; the great number of possible combinations makes the picture folder location difficult to guess. The nested tree structure avoids having too many subdirectories in any given directory. The string is computed as a hash of the username (using the MD5 algorithm), in order to make the directory location unpredictable and to have a balanced tree, where user folders are equally distributed. The actual user folder is located in the last subdirectory of the tree; in order to make it less comprehensible, the folder name is computed as the username encoded in Base64. Security considerations Even if it is difficult to guess, the directory tree is not secure, since it is computed starting from the username. This means that, given the user name and knowing the algorithm used, you are always able to derive the directory path. For example, if the hash for the user johndoe is hgkvnviumvngrdpo, the user's pictures are stored in the directory /ds-server/db/ picture/hg/kv/nv/iu/mv/ng/rd/po/am9obmRvZQ==. Security is guaranteed by the file name, a random string of thirteen alphanumeric characters, followed by the real file extension (added to guarantee the correct content type in the HTTP response.) So even if you know the path where the file is stored, you cannot guess the file name and retrieve the picture. The hash function guarantees that the distribution of hashes in the hash space is adequate, and that, for a large number of files, they are evenly distributed inside the hierarchy, thus splitting the load. 2.6.2 -ext subdirectories For each picture (or video) stored in the user directory there is also a subdirectory with the name equals to the file name followed by -ext, where additional files related to the picture (or video) are stored, such as thumbnails, transcoded video or any other useful data. For example, when a new picture with name 1pbo6y7xoyjr1 is saved (how files are named on the local file system is explained in Section 2.6.1, “File system structure”), the folder 1pbo6y7xoyjr1-ext is also created. The -ext subdirectory is created when the file is saved into the user directory, and it is removed when the corresponding file is deleted. Note Files stored in the -ext directories and temporary files are not computed in the user quota. 2.6.3 Deleting media files The OneMediaHub provides a scheduled job (executed every one hour) that manages the media binaries deletion. If a user deletes some media files, first of all the tuples will be set to deleted in the database, then, by execution of the scheduled job, the binaries will be deleted from the file system (or from the media storage). This behavior also impacts the deletion of users via Server API (SAPI): when a user is deleted, all the tuples in the fnbl_file_data_object table owned by that user will be marked as deleted and the owner will be changed into admin, while the original user will be saved in the column deleted_owner. As mentioned above, the binaries will be deleted by the scheduled job. 12 System overview Note All media items marked as soft deleted won't be removed by the scheduled job. 2.6.4 Temporary items management Temporary items are used to allow the resumable upload for media. They should be considered valid for 24 hours; in that time frame the server reserves disk memory quota to finish the upload. In no case will the temporary items be considered by the server as part of the user quota. After the 24 hours, the task that manages the media deletion (see Section 2.6.3, “Deleting media files”) deletes the database tuples older than one day for temporary items and their related binaries, if they exist. It deletes also the items older than one day for which only the metadata are saved. 2.6.5 Scaling the file system Since there is a single mount point for pictures, /ds-server/db/picture, OneMediaHub supports the following systems for scaling the file system: • DAS (Direct Attached Storage) for small deployments. • NAS (Network Attached Storage) for medium deployments. • SAN (Storage Area Network) for very large deployments. 2.6.6 Encryption Media may be encrypted once it is moved to the final storage destination. A media item is encrypted when the whole item is received on the server, so the encryption process is completely managed by the server itself. 2.6.6.1 How encryption works If the server is configured to use encryption (see Section 2.6.6.2, “How to configure the server to use encryption”), OneMediaHub encrypts your data as it writes it to its media storage and decrypts it when you access it using the authenticated download API. This means that if you would directly access stored media, you would see encrypted objects. However, all the APIs are trasparent regarding media encryption, so that if your server has encryption enabled all the API calls will work properly. Two different algorithms can be used to encrypt media: AES128 and AES256. They are block cipher symmetric-key algorithms, so the same key is used for both encrypting and decrypting the data. They differ in the key size (128 and 256 bits respectively). 2.6.6.2 How to configure the server to use encryption To enable encryption, in the configuration file /config/portal/portal-ext.properties these properties have to be modified: # # Set property to true for enabling the encryption, false otherwise. # 13 System overview encryption.enabled=true encryption.keyfactoryclass=com.funambol.framework.tools.encryption.AES128EncryptionKeyFactory encryption.keyfactory.salt=cGFzc3dvcmQ= To enable AES128 use the AES128EncryptionKeyFactory class - for AES256 the class AES256EncryptionKeyFactory has to be used. For both a Base64-encoded salt has to be specified. The salt should be a random value - and it MUST not change once media has been uploaded. For AES256 strength encryption the Unlimited Strength Java(TM) Cryptography Extension Policy Files have to be installed. They may be freely downloaded from the Oracle Java SE website (see [4]). Warning If encryption is disabled, the media items are served by the storage system (e.g. S3) directly, with no additional load on the server. If instead encryption is enabled, all requests pass through the server, so that the increased load needs to be considered in sizing the system and tuning its performance. Encryption causes also additional storage usage and costs more computational power. 2.6.7 Storing files on an online file storage provider Media and files may be stored on an online file storage provider instead of being stored on the server's local file system. Online file storage providers are Internet hosting services specifically designed to host static content, typically large files that are not web pages. OneMediaHub supports only the Amazon S3 file storage provider. There are few basic concepts behind online file storage providers (the names used by Amazon S3 are emphasized): Provider the specific online file storage provider Blobs (objects) are the fundamental entities. They consist of object data and metadata (set of name-value pairs) Container (bucket) blobs are stored in containers. For Amazon S3 there are no limits to the number of objects you can store in a bucket, and each user can have up to 100 buckets. The namespace for bucket names is global - this means that there could be only one bucket name for the provider for all accounts. The bucket name is in fact part of the URL used to download objects. For instance, all objects in a 'mydocs' bucket can be downloaded using http://mydocs.s3.amazonaws.com (you may use a different URL by setting a CNAME entry for your domain in the DNS server settings) Key is the unique identifier for a blob in a container. Together, a container name and a key uniquely identify an object on the online storage provider 2.6.7.1 How to create an Amazon S3 bucket In order to use Amazon S3 as media storage provider for the OneMediaHub, a bucket must be created on S3. 14 System overview Go through the following steps to create a bucket: 1. Use a plug-in for your web browser (e.g. S3 Organizer, available for Firefox) and connect to your Amazon S3 account: Figure 2.7. Connecting to your Amazon S3 account 2. Click on Create Bucket/Directory or right click on the bucket panel and select the Create Directory option: Figure 2.8. Create a directory on Amazon S3 3. Enter the bucket name and then click Ok. The bucket will be created at the specified location, and your bucket will be visible in the buckets list. 15 System overview Note Don't use a dot (.) in the bucket's name, because in this case SSL over HTTP (HTTPS) will not work, as users will end up with S3-related certificate errors caused by bucket names containing a dot. 4. The bucket Access Control List (ACL) should be configured in order to grant access for the owner only, but in any case all the contents will always be private even if the bucket is public. 2.6.8 Transcoding The video transcoding feature allows the user to play from the Portal and the iOS and Android mobile apps any of her videos uploaded to the cloud. It allows to play a wide range of video formats. When a user uploads a video, a transcoding service is running asynchronously, and when the transcoded video is available, its URL (called "playbackurl") will be included in the response of the Retrieve videos API call (see Section 3.5.8, “Retrieve videos” in OneMediaHub Version 14.5 Server API Developer's Guide). The transcoded video is called playback.mp4 and it is saved in the user -ext folder like the thumbnails. The service used for video transcoding is the Amazon Elastic Transcoder (see http:// aws.amazon.com/elastictranscoder.) This service manages all aspects of the transcoding process in a transparent way for OneMediaHub and is designed to be highly scalable. Amazon Elastic Transcoder is built to work with the content stored on the Amazon S3 service and it uses the Amazon Simple Notification Service (Amazon SNS) to notify when a transcoded video is available (OneMediaHub uses HTTP notifications.) Note To enable media transconding on OneMediaHub, see Section 3.12.23, “How to enable media transcoding”. Warning If the transcoding feature is not enabled, the Portal is not able to play the following video formats: 3g2, AVI, MOV, MP2, MP4, MPEG, MPEG4, MPG, WMV. To use Amazon Elastic Transcoder, you need an Amazon Web Services (AWS) account (see http:// docs.aws.amazon.com/elastictranscoder/latest/developerguide/gettingstarted.html.) If you don't have an account yet, you'll be prompted to create one when you sign up. There are few basic concepts behind Amazon Elastic Transcoder: • A transcoding pipeline is a queue that manages the transcoding jobs. It specifies the input Amazon S3 bucket, the output Amazon S3 bucket, and an AWS Identity and Access Management (IAM) role that is used by the transcoder to access the videos. • A transcoding preset is a template that contains the settings that the transcoder should apply during the transcoding process (i.e. the codec or the resolution.) When a job is created, the preset to be used must 16 System overview be specified. In this way, it's possible to specify the details of the transcoded video. The preset used by OneMediaHub is called System preset: Generic 480p 4:3 and allows to generate a 480p transcoded video in mp4 format, with video codec H264 and bitrate 900 kbps (refer to the Amazon Elastic Transcoder Management Console for further details.) • A transcoding job transcodes the video on the Amazon S3 output bucket specified in the pipeline. The steps for using Amazon Elastic Transcoder are the following: 1. Create a bucket on Amazon S3 (see Section 2.6.7.1, “How to create an Amazon S3 bucket”). If OneMediaHub is configured to use S3 as media storage repository, you don't need to create a bucket for this goal, since you will use the one used as media repository 2. Configure Amazon SNS for sending HTTP notifications 3. Create a pipeline on Amazon Elastic Transcoder 2.6.8.1 How to configure Amazon Simple Notification Service For more details, see Amazon documentation at [1]. 1. Access the AWS Console and select SNS. You will be redirected to the SNS Console 2. Click on Create New Topic Figure 2.9. Amazon SNS: Create New Topic 3. Associate to the topic the subscription http:// : /sapi/media/ video?action=set-transcoding-status Important The server must be up and running since it will receive a notification. 17 System overview Figure 2.10. Amazon SNS: Create Subscription The subscription is created with a "SubscriptionID" as "PendingConfirmation" until the subscription is confirmed. 4. Search in the file portal.log for the string "Type":"SubscriptionConfirmation" and retrieve the value for SubscribeURL 5. Open this URL in a browser 6. Check from within the SNS Console that the Subscription ID field will change from PendingConfirmation into a specific string value 7. Select the created subscription and click on Delivery Policy. Configure the policy with: Number of retries: 60 Retries with no delay: 0 Minimum delay: 60 Minimum Delay Retries: 60 Maximum Delay: 60 Maximum delay retries: 0 Maximum receive rate: EMPTY Retry backoff function: Linear (see Figure 2.11, “Amazon SNS: Delivery Policy”.) At this point the subscription is enabled. 18 System overview Figure 2.11. Amazon SNS: Delivery Policy 2.6.8.2 How to create a pipeline on Amazon Elastic Transcoder For more details, see Amazon documentation at [2]. 1. Open the AWS Elastic Transcoder Console at https://console.aws.amazon.com/ elastictranscoder/ 2. Click Create New Pipeline 3. Fill in the form as follows: 1. Pipeline name: the name of the pipeline you want to create 2. Input Bucket: the name of the bucket on S3 3. IAM Role: choose Elastic_Transcoder_Default_Role 19 System overview Note The first time you work with Amazon Elastic Transcoder this role could be not available. Keep the default value in such a case 4. In the Configure Amazon S3 Bucket for Transcoded Files and Playlists section: a. Bucket: the name of the bucket on S3 (it's the same used in Input Bucket) b. Storage class: leave it empty c. Click on Add permissions: choose the permissions for Open/Download, View, and Edit the bucket's content to the AWS user or group that you want to have access to transcoded files and playlists 5. In the Configure Amazon S3 Bucket for Thumbnails section: a. Bucket: the name of the bucket on S3 (it's the same used in Input Bucket) b. Storage class: leave it empty 6. In the Notifications section: turn on all the events notifications selecting Use an existing SNS Topic and selecting the SNS topic you created on SNS 7. Click on the Create Pipeline botton at the end of the form At the end of this procedure, you will have a Pipeline ID and this value should be added to the file portal-ext.properties as value of the property ets.pipeline-id. 2.6.9 Deleting transcoding jobs OneMediaHub provides a scheduled job (executed every 7 days) that manages the deletion of the transcoding jobs stored in the database that are older than 7 days. When the scheduled job is executed, the records in the table fnbl_trascoding_job with creation date older than 7 days are removed. And, if the media item associated to the transcoding job, has transcoding status (field transcoding_status in table fnbl_file_data_object) set to Q (in queue) or P (in progress), this status must be set to E (error), since no notification about the result of the transcoding has been received. The execution frequency of the scheduled job is set in the property intervalInDays contained in the file TranscodingJobDeleteScheduledTask.xml. 20 Chapter 3. Installation and configuration This section explains how to install and configure the OneMediaHub on your system from your distribution medium. Due to the highly modular architecture of the OneMediaHub platform, there are many configuration parameters that the system needs in order to integrate all the different parts. Instead of providing a large and complex single configuration file, OneMediaHub configuration parameters are stored in small XML files organized in a tree structure under the file system. This allows easy look up of changes that need to be made and is an easy way to change the configuration of the servers: in fact all that is really needed is a text editor. Note OneMediaHub makes use of third-party software which may require their own configuration files. These files will also be described in the following sections. In general, the OneMediaHub configuration is composed of: • OneMediaHub configuration files • Apache Tomcat (see [13]) configuration files • JGroups (see [14]) configuration files • log4j (see [15]) configuration files 3.1 Installing the OneMediaHub Download the archive for the OneMediaHub on your server: • onemediahub-x.x.x.tgz and extract it in a directory of choice (for example: /opt) using the following command: tar -xzvf onemediahub-x.x.x.tgz Important Take care to set appropriate file permissions for the files involved, so that the system user in charge to start the OneMediaHub server will be able to access them. For example, if the archive is extracted using the root user, without appropriate file permissions a user funambol will not be able to start all the needed processes or to access libraries. Double check that the environment variable JAVA_HOME is properly set to the Java Development Kit (JDK) home. For example: export JAVA_HOME=/opt/jdk1.7.0 Install the JDBC driver, by copying the jar file under $JAVA_HOME/jre/lib/ext 21 Installation and configuration 3.2 The config directory In OneMediaHub, all the configuration files for all components are stored under /config. 3.2.1 OneMediaHub configuration files OneMediaHub configuration files are simple XML files that configure a specific aspect or component of the OneMediaHub system. The XML schema used by these files is flexible so that it is not necessary to change it even when a new set of configuration parameters is introduced. Together with this flexibility the syntax of the file is simple enough to be easily understood. This is an example of a server JavaBean: 3.3 Quick configuration OneMediaHub provides a way to quickly configure some of the most common settings through a single configuration file. To quickly configure a OneMediaHub installation, go to the directory /bin and edit the file config.properties, customizing the properties listed in the table below to reflect your deployment: Property Description ${api.baseurl} The base URL of the server API. Used by the AJAX UI as well. Default value is 'sapi' ${customer.app.name} The application name used in the markets, such as App Store, Android Market or App World (use only printable characters) ${customer.name.long} The extended name of the customer running the portal (use only printable characters) ${customer.name.short} The short name of the customer (use only printable characters, no spaces) 22 Installation and configuration Property Description ${ds.admin.password} User admin password ${encryption.algorithm} The encryption algorithm's name (may be empty if encryption is disabled) ${encryption.enabled} Use or not encryption for media files ${encryption.salt} The encryption key for the algorithm defined above (may be empty if encryption is disabled) ${ets.client-region} The Region where the AWS account has been created. It should be one of the following: us-east-1, us-west-1, us-west-2, eu-west-1, ap-northeast-1, apsoutheast-1, ap-southeast-2, saeast-1 ${ets.pipeline-id} The pipeline identifier where your transcoding jobs will be added ${ets.s3-container-name} The bucket name to be used on Amazon S3 (if the storage is S3, this bucket name should be the same as set in the property ${storage.container.name}) ${facebook.id} The "Application ID" of your Facebook application ${facebook.secret} The "Secret" of your Facebook application ${flickr.key} The "Key" of your Flickr application ${flickr.secret} The "Secret" of your Flickr application ${google-analytics.account} Google Analytics web property ID ${jdbc.driver} The JDBC driver to use ${jdbc.password} The database user password ${jdbc.url} The URL to use in database connection. Note The JDBC URL must include the parameter specification characterEncoding=UTF-8; for example: mysql://172.16.11.24/ funambol? characterEncoding=UTF-8 ${jdbc.user} The database user ${limit.items} The max number of items that can be contained in a JSON array ${mail.from} The mail address used as "from" in the sent Email ${mail.smtp.auth} Whether the mail server requires authentication or not (true or false) ${mail.smtp.host} Mail server host name ${mail.smtp.password} Mail server user password 23 Installation and configuration Property Description ${mail.smtp.port} Mail server port (e.g. 25) ${mail.smtp.ssl} Specifies if the SMTP server to be used requires SSL. Can be true or false ${mail.smtp.user} Mail server user name ${media.server} The storage provider domain for media and file ${media.transcodingservice.enabled} Used for enabling/disabling the media transcoding ${ota.account.name} Name of the sync profile created by the OTA configuration message (use only printable characters) ${portal.server} Public hostname:port of the server (e.g. my.server.com:80) Note The specified URL must be exactly the same as the one used to access the service. Failure in doing so may cause errors in various phases including signup. If you need to support multiple servers see Section 3.13.2, “Supporting multiple server URL” for additional information. ${quota.roles} The list of the available quota roles ${quota.demo} The storage space quota for Demo users ${quota.premiumplus} The storage space quota for Premium Plus users ${quota.premium} The storage space quota for Premium users ${quota.standard} The storage space quota for Standard users ${quota.ultimate} The storage space quota for Ultimate users ${sms.password} SMS provider user password ${sms.sender} Name of the SMS sender (max 11 characters, use only printable characters) ${sms.user} SMS provider user account ${storage.container.name} The bucket name. If the storage provider is filesystem, there is no need this to be set. If the storage provider is s3, it must be configured with the bucket name to be used on Amazon S3 ${storage.credential} The secret key. To be set when the storage provider is Amazon S3; otherwise to be left empty ${storage.identity} The access key. To be set when the storage provider is Amazon S3; otherwise to be left empty ${storage.provider} The storage provider name. Values allowed: filesystem to use filesystem storage, or s3 to use Amazon S3 24 Installation and configuration Property Description ${subscription.enabled} Used for enabling/disabling the user subscriptions feature ${twitter.key} The "Consumer Key" of your Twitter application ${twitter.secret} The "Consumer Secret" of your Twitter application ${udp.binding.addr} UDP traffic binding address (usually the local IP address of the server) ${ws.server} The server host to be used use by the PIM Listener Service to call the web services exposed by the Data Synchronization Service ${youtube.key} The "API Key" of your YouTube application ${antivirus.enabled} Enable the antivirus service, default is false. For more information, see Section 3.24, “Antivirus service” When done, run the command: ./configure-portal Here is an example of the config.properties file for quick configuration: ${jdbc.user}=syncuser ${jdbc.password}=changeme ${jdbc.url}=jdbc:mysql://localhost/funambol? characterEncoding=UTF-8&connectTimeout=10000&socketTimeout=60000 ${jdbc.driver}=com.mysql.jdbc.Driver ${portal.server}=myserver.com ${ws.server}=localhost:8080 ${ds.admin.password}=adminpass ${udp.binding.addr}=localhost ${sms.user}=sms ${sms.password}=smspwd ${sms.sender}=onemediahub ${mail.smtp.host}=smtp.gmail.com ${mail.smtp.port}=465 ${mail.smtp.auth}=true ${mail.smtp.user}=user@server.com ${mail.smtp.password}=mypwd ${mail.from}=portal@server.com ${mail.smtp.ssl}=true ${customer.name.long}=Funambol ${customer.name.short}=Funambol ${customer.app.name}=OneMediaHub 25 Installation and configuration ${ota.account.name}=onemediahub ${api.baseurl}=sapi ${facebook.secret}=b28b383c51971112384422359b3at56 ${facebook.id}=546540546197143 ${flickr.key}=b128b383c19719112322412359b3at56 ${flickr.secret}=546540546197143e ${youtube.key}=AI39si6KC1971QFGUacL9HMsOKdJJ56h8fgdziY09jeF9ZkzRF501J0sJCCxcaCbazcxhWnfwW-5Gg3XbrsLX_UUYOHVaz4g ${twitter.key}=aa999b9cc19719 ${twitter.secret}=82Z5VBtIZ1971kwCBLYrPbzIMx6NMxSddaLkys3MnUI # # Set it to 'filesystem' for on local file system or to's3' for on Amazon S3 # ${storage.provider}=filesystem # # Set it to the storage provider for media and file content # (for instance media-container.s3.amazonaws.com) # # Using 'filesystem' this property can be empty # ${media.server}= # # The following properties are not needed using 'filesystem' # # # Set it to the bucket name using 's3' # (for instance media-container) # ${storage.container.name}= ${storage.identity}=AHHAISAPOB1 ${storage.credential}=ixXaHH1TPLABIwWEaHyx # The portal Mobile sign-up path ${portal.server.msupath}=m # The available quota roles ${quota.roles}=demo,standard,premium,premiumplus,ultimate,noquota # The storage space quotas ${quota.demo}=150M ${quota.standard}=1G ${quota.premium}=5G ${quota.premiumplus}=10G ${quota.ultimate}=50G # The limit of items managed by JSON array 26 Installation and configuration ${limit.items}=15000 ${encryption.enabled}=false ${encryption.algorithm}=AES128 ${encryption.salt}=cGFzc1971mQ= # Google Analytics UA ID ${google-analytics.account}= ${media.transcoding-service.enabled}=true ${ets.pipeline-id}=1135067402107-71a0b1 ${ets.client-region}=us-west-1 ${ets.s3-container-name}=container-playlist Tip In order to prevent to erroneously run the quick configuration, you can set the environment variable FUNAMBOL_QUICK_CONFIGURATION_NOT_ALLOWED to true. In this way, if you try to run the configure-portal command, a message saying that the quick configuration is not allowed is shown and the script exits. Note If you wish to further customize your OneMediaHub installation, the following sections provide detailed instructions for configuring the various components. 3.4 Database configuration The OneMediaHub Portal requires a database (e.g. "funambol") and a user (e.g. "syncuser") that has select/ insert/update/delete grants on the database tables. In the following sections, you will find instructions on how to create them using MySQL. If you wish to use a partitioned database, refer to Chapter 6, Database partitioning for further details. 3.4.1 MySQL database creation Here we will assume that MySQL has been installed successfully, is up and running, and is listening on port 3306 (default). To enable the execution of scheduled events, the value of the MySQL global system variable event_scheduler must be set to true. Since MySQL server can operate in different Server SQL Modes [10], the System Administrator confirms that the MySQL server is running with the default configuration or a not strict mode. Warning Since MySQL server can operate with ACID compliance or without ACID compliance for commit operations [11], the System Administrator confirms that the MySQL server is running without ACID compliance. So innodb_flush_log_at_trx_commit is either 0 or 2. The default value for innodb_flush_log_at_trx_commit is not supported. 27 Installation and configuration Below is a simple example on how to create the required user and database. You can run the following commands on the database server or on a different client machine; in this case, additional parameters (such as the hostname) could be required - see [5] for more details. 1. Create the database funambol: mysql -u root -e "create database funambol character set 'UTF8';" 2. Create the database user "syncuser": mysql -u root -e "create user syncuser identified by 'changeme';" 3. Grant all privileges on the database "funambol" to the user "syncuser": mysql -u root -e "grant all privileges on funambol.* to 'syncuser'@'localhost' identified by 'changeme';" mysql -u root -e "flush privileges;" 4. Import the initial database data using UTF-8 encoding: mysql -h localhost -D funambol –u root --default_character_set utf8 < /portal/database/ mysql/cared-mysql.sql 5. Limit user "syncuser"'s privileges: mysql -u root -e "revoke create, grant option, alter on funambol.* from syncuser@localhost;" Security consideration The aforementioned commands create tables and required objects using "syncuser" as the user. This is not a good practice in a production environment where the user used by the application should be different than the one used to create the database and the tables. You should use your own database (super)user to create the database, run the cared-mysql.sql script and then give select/insert/update/delete rights to "syncuser" and, in case of future patches containing SQL scripts, execute the scripts themselves. If your MySQL server is running on a different server from the OneMediaHub services, you need to grant remote access to "syncuser" (see MySQL Documentation for more details about user remote access.) 3.4.2 Further database configuration OneMediaHub is configured to use the following values by default: • host name: localhost • database name: funambol If a different configuration is required, update the following file: • /config/com/funambol/ server/db/db.xml This is an example of the configuration file to use MySQL as database server running on 172.16.11.24. 28 Installation and configuration Note The JDBC URL must include the parameter specification characterEncoding=UTF-8. All the OneMediaHub components use the db.xml configuration file located in the /config/com/funambol/server/db directory; you can modify all the configuration parameters by editing this file. The main parameters are: Property Description username The database user password The database user password url The URL to use in database connection driverClassName The JDBC driver to use In addition, the following parameters can be set for advanced tuning: Property Description initialSize The initial number of connections that are created when the pool is started. maxActive The maximum number of active connections that can be allocated from this pool at any one time, or negative for no limit. maxIdle The maximum number of connections that can remain idle in the pool, without extra ones being released, or negative for no limit. 29 Installation and configuration Property Description minIdle The minimum number of connections that can remain idle in the pool, without extra ones being created, or zero to create none. maxWait The maximum number of milliseconds that the pool will wait (when there are no available connections) for a connection to be returned before throwing an exception, or -1 to wait indefinitely. connectionProperties The connection properties that will be sent to our JDBC driver when establishing new connections. The format of the string must be [propertyName=property;]* Note The user and password properties will be passed explicitly, so they do not need to be included here. 3.4.3 Limiting the maximum number of open connections By default, the server can open a maximum number of 100 connections. Your database server will have difficulty with too many open connections, therefore you will need to increase the maximum number of connections allowed, or to change the OneMediaHub database configuration in order to reduce the number of connections used (see Section 3.4.2, “Further database configuration” for more information on the initialSize, maxActive, minIdle and maxIdle parameters). Note OneMediaHub uses a pool of connections, therefore having 100 connections open does not mean that the server is using all of them concurrently, but rather that almost all of them are idle and ready to be used. In order to change the maximum number of connections allowed, refer to [9]. 3.4.4 Database partitioning OneMediaHub supports database partitioning; see Chapter 6, Database partitioning for further details. 3.4.5 MySQL events To enable the execution of scheduled events, the value of the MySQL global system variable event_scheduler must be set to true. The default value INTERVAL_VALUE in the mysql.event table is different according to the specific event, and can be tuned depending on the load of the given database (see [8]). You can find more information about the Event Scheduler and MySQL events in the official database documentation (see [7]). 30 Installation and configuration Event name Description delete_monitor_used_storage Delete media storage information older than 30 days delete_old_client_download_stats Delete download statistics for the clients older than 60 days delete_old_contacts Delete contacts in status D since more than 30 days delete_old_events Delete events in status D since more than 30 days delete_old_fnbl_events Delete reporting events older than 7 days delete_old_folders Delete folders in status D since more than 30 days delete_old_history Delete synchronization activities older than 180 days delete_old_media Delete media items in status D since more than 30 days delete_old_notes Delete notes in status D since more than 30 days delete_old_notifications Delete undelivered push messages older than 7 days disable_pim_push_account Disable PIM push for users without a sync in the last 60 days 3.5 IPv6 Support IPv6 clients are fully supported out-of-the-box without any change if OneMediaHub is installed on IPv4 hosts and if in front of the OneMediaHub server an IPv6 HTTP proxy or load balancer (like the Apache HTTP Server) is used, since in such configuration the proxy translates IPv6 addresses into IPv4 addresses. Otherwise, if OneMediaHub is installed on IPv6 hosts, the scripts /bin/funambol-server /bin/pim-listener must be changed by removing (or commenting out) the following line: JAVA_OPTS="$JAVA_OPTS -Djava.net.preferIPv4Stack=true" 3.6 Adding new nodes to a OneMediaHub cluster This section describes how to add new nodes in a OneMediaHub cluster. The sections cover the scenarios where only one service per box is deployed. If more than one service is deployed on a box, information in the different sections can be combined. Warning In a cluster environment, the directories where all media files belonging to the user are stored must be shared between all server nodes. See Section 2.6.1, “File system structure” for more details. 3.6.1 Adding a new Data Synchronization Service node to the cluster To add a new node to the Data Synchronization Service cluster, simply install OneMediaHub on a new machine as explained in Section 3.1, “Installing the OneMediaHub” and configure the load balancer to make it aware of the new system. 31 Installation and configuration 3.6.2 Adding a new PIM Listener Service node to the cluster To add a new node to the PIM Listener Services cluster just install OneMediaHub on a new machine as explained in Section 3.1, “Installing the OneMediaHub” and start the PIM Listener Service. 3.7 Data Synchronization Service configuration Most of the configuration files are forged at build time and do not need to be changed at all. The following configuration files, found in the / config directory, are of interest to system administrators: File Description Funambol.xml This is the root of all Data Synchronization Service configuration files; see the dedicated section below for further details. com/funambol/server/sms/ SMSProvider.xml Configuration file for the SMS service provider used to send SMSs. Out of the box, OneMediaHub uses SubitoSMS (see the section called “SMS Service”) com/funambol/server/notification/ PIMPushSender.xml Push configuration file for SyncML devices which support PIM push. 3.7.1 Funambol.xml This is the main Data Synchronization Service configuration file. It also serves as a directory of links to other configuration files for specific components. The properties contained in Funambol.xml are divided into two sections and generally do not require any changes. Table 3.1. engineConfiguration Property Description officer Represents the component that controls how users are authenticated and granted access to the system. serverURI Specifies the URI to be used by the client when responding to server messages. Note that this value doesn't affect the URL or port used by the application server that runs the OneMediaHub application. sessionHandler Represents the component (instantiated using the contents of an XML file) that manages the synchronization session. strategy Represents the component that handles the synchronization process. userManager Represents the component that handles all the users. minMaxMsgSize This option specifies the minimum value to be used by the client for MaxMsgSize. This value is the maximum size that all messages sent by the server must have (specified by the SyncML protocol). If a client specifies a value for MaxMsgSize that is 32 Installation and configuration Property Description smaller than minMaxMsgSize, the server refuses the synchronization and will log an error. deviceInventory Represents the component that manages all the device records. dataTransformerManager Represents the component that allows you to customize the encryption applied to incoming and outgoing messages. checkForUpdates Enables the Data Synchronization Service to check the Funambol website daily for updates. [Active by default] Warning Be careful when applying changes to the properties in this section as it may result in the server malfunctioning. Table 3.2. serverInfo Property Description man The manufacturer. [Default: 'funambol'] mod The model. [Default: 'DS Server CarEd'] oem The OEM. [Default: empty] fwV The firmware version. [Default: empty] hwV The hardware version. [Default: empty] swV The server version. devID The device ID. [Default: 'funambol'] devType The device type. [Default: 'server'] utc Does the server support UTC? [Default: 'yes'] supportLargeObjs Does the server support large object? [Default: 'yes'] supportNumberOfChanges Does the server support number of changes? [Default: 'yes'] X-funambol-smartslow The server supports the smart slow sync that allows the OneMediaHub client Apps and the server to optimize traffic during slow syncs (present since v8.5) X-funambol-media-http-upload The server supports improved file data object synchronization: as pictures may be large and the bandwidth small, the server supports HTTP upload of media files (present since v8.7) X-funambol-msu The server supports mobile signup. If the property is not present, the latest OneMediaHub client Apps will not display the signup option (present since v9.0) 33 Installation and configuration Note These options include all the settings that determine what is initially sent to the SyncML device or client, to describe important server characteristics. 3.8 PIM Listener Service configuration Most of the configuration files are forged at build time and do not need to be changed at all. The following configuration files, found in the / config directory, are of interest to system administrators. File Description PIM Listener Service configuration file. The following sections should be updated: com/funambol/pimlistener/ PIMListenerConfiguration.xml For more see the section below. JGroups configuration file used by the PIM Listener Service cluster. jgroups-pimlistener.xml bind_addr="${jgroups.udp.bind_addr. ctp-nofiticationgroup:192.168.1.15}" 3.8.1 PIMListenerConfiguration.xml This file contains the following configuration parameters. Property Description maxThreadPoolSize Specifies the maximum number of threads that can be used. This should be a function of the number of users to monitor. healthThreadPollingTime Sets the interval (in seconds) between executions of the HealthThread, a thread that assesses the PIM Listener Service status at regular intervals. 34 Installation and configuration Property Description registryMonitorPollingTime Sets the interval (in seconds) between RegistryMonitor executions, a thread that regularly checks the database for changes to monitored accounts. taskPeriodTolerance Sets the tolerance accepted on the period between two data checks. If a user is scheduled to be checked every x seconds (period), a warning is triggered if the data check occurs after x + periodTolerance/x. registryTableName Defines the table that contains the PIM Listener push registry entries. By default fnbl_push_listener_registry. pluginDirectory The directory from which the PIM Listener Service loads plug-ins. serverInformation Records the information used to call any webservices exposed by the Data Synchronization Service, as noted below: • url: the OneMediaHub engine administration URL • username: the OneMediaHub engine administrator username • password: the OneMediaHub engine administrator password Contains the information used to create a cluster between two or more PIM Listener Services. clusterConfiguration 3.9 Data Synchronization Service cluster configuration Most of the configuration files are forged at build time and do not need to be changed at all. The following configuration files, found in the / config directory, are of interest to system administrators. File Description JGroups configuration file used by the Data Synchronization Service cluster jgroups-dsserver.xml cluster.bind_addr= "${jgroups.udp.bind_addr.ds-servergroup:192.168.0.15}" 3.10 Enabling default users Before starting for the first time the OneMediaHub services, it is mandatory to enable the default users and choose a password for them. These users are required for using the customer service representative (CSR) interface and running the OneMediaHub services. Any user available by default in OneMediaHub is disabled as a security measure. 35 Installation and configuration To enable the default users, run the command: /bin/enable-default-users To enable a single default user, run the command: /bin/enable-default-users Note The password of the default user 'admin' must be the same value of the property ${ds.admin.password} in the file /bin/config.properties 3.11 Starting/stopping OneMediaHub services This section describes how to start and stop the different OneMediaHub services. Note You can verify whether the server services started successfully by looking at the server logs; see Chapter 7, Logging for more information on logging. The following table details which services can autonomously be started/stopped, the script to use, and the TCP ports involved: OneMediaHub service Script TCP ports used All services together /bin/ funambol (all ports listed here below) Data Synchronization Service /bin/ funambol-server 8005 (shutdown port) /bin/pimlistener 3101 (JMX port) PIM Listener Service 8080 (connector port) 8101 (JMX port) Warning You must not change the timezone settings in the funambol-server script. The Data Synchronization service must start in GMT. At the moment there is no chance to have this service working with a timezone different from GMT. This is because the current implementation expects the Java Virtual Machine running in GMT, since in handling different timezones a timezone to be used as reference is needed, and GMT is the most generic one. When run, these scripts inform the user via a warning message if the default TCP ports are already in use. In this case, the start-up procedure is terminated. 36 Installation and configuration This could mean that: 1. the service is already up and running (therefore ports are already used) 2. TCP ports are being used by another application 3.11.1 Starting/stopping all services together To start all services together, go to the directory /bin and run the command: ./funambol start To stop all the services, use the command: ./funambol stop 3.11.2 Starting/stopping the Data Synchronization service To start the Data Synchronization service, go to the directory /bin and run the command: ./funambol-server start To stop the service, use the command: ./funambol-server stop 3.11.3 Starting/stopping the PIM Listener service To start the PIM Listener service, go to the directory /bin and run the command: ./pim-listener start To stop the service, use the command: ./pim-listener stop 3.12 Portal configuration Portal configuration files are stored under the root directory of your OneMediaHub installation. The table below lists the portal configuration files that a system administrator can be interested in changing. File Description Mail session configuration file. Change the SMTP server settings according to system deployment. See below for details. tools/tomcat/conf/Catalina/ localhost/ROOT.xml config/portal/portal-ext.properties Main portal configuration file. See below for details. 3.12.1 Configuring the login with or without country code The OneMediaHub can be configured to allow the users to login with or without the country code. When they registered with a phone number as username, the server should be configured in this way when all the users are from the same country (and share the same phone country code). 37 Installation and configuration In order to configure the server to allow this type of login, a new property must be set into the file /config/portal/portalext.properties: unique-country-code=49 In the example above, the value of the element unique-country-code (49) is the country code of Germany. Warning This feature should be configured before having users in the database. If there are already users in the database, those will not be able to login with or without country code, but only with the actual number used as username when they registered. 3.12.2 ROOT.xml In order to be able to send Emails, the following mail service must be configured: /tools/tomcat/conf/ Catalina/localhost/ROOT.xml Edit the file and set appropriate values for the SMTP server of choice and the from address that should appear in the Emails. The parameters that need to be customized are highlighted in the following example: Below is a practical customization example: 38 Installation and configuration This SMTP server is the same as the one specified in the Email account configuration. The configuration parameters that must be specified for the SMTP server are: • mail server (i.e., mail.smtp.host) • whether it requires authentication or not (mail.smtp.auth= ["true"/"false"]) • username (mail.smtp.user) and password (mail.smtp.password) that are used for all Email accounts registered in the table fnbl_email_account for authentication. When a portal user registers a private Email account, the username and password combination for that SMTP server will be saved in fnbl_email_account, under the columns: out_login and out_password. After these changes, you will need to restart the server; go to the directory /bin and run the command: ./funambol-server start 3.12.3 portal-ext.properties Change the following properties in the file /config/portal/portal-ext.properties: 3.12.3.1 sp.syncportal.messages.url and sp.syncportal.url Set the properties used to create the link in SMS and invitation mail: # This property is used for the mail body and when # an SMS message is sent sp.syncportal.messages.url= : ... # OneMediaHub Server URL sp.syncportal.url=http:// : 3.12.3.2 admin.email.from.address and admin.email.from.name Set the properties used to send mail: # Set signature (for example, 'The Funambol Team') and email address for # registration, invitation and forgot password email admin.email.from.address= admin.email.from.name= 3.12.3.3 sp.mediaserver.url Set the property used to specify the storage provider URL for media and file content: # This property is used to specify the storage provider for media and 39 Installation and configuration # file content as they can be stored on a remote repository by default # using the configure-portal is the same as portal URL sp.mediaserver.url=http:// : 3.12.3.4 sync.slow.min-interval Set the minimum interval in minutes between two slow syncs for a specific principal: # # # # "-1" means always allowed, "never" means always rejected. If the engine rejects a slow sync, the status returned to the client is 407 - Retry sync.slow.min-interval=-1 3.12.3.5 sync.blocked-sources Set the list of sync sources a client should be prevented to sync against: # # # # # List of sync source names blocked on this server, separeted by "," eg: card,cal,task The status returned to the client is 407 - Retry Note: Currently this does not block SAPI paths (picture, video, file, etc.) sync.blocked-sources=cal 3.12.3.6 sync.min-interval Set the minimum interval in minutes between two syncs (of any type) for a specific principal: # minimum interval in minutes between a sync and another of any kind for # a user, "-1" means always allowed, "never" means that the syncs are # blocked at all sync.min-interval=-1 3.12.3.7 sync.max-session-allowed Set the maximum number of SyncML sessions allowed by the DS Server: # maximum number of SyncML sessions allowed by the DS Server. # "-1" means always allowed sync.max-session-allowed=-1 3.12.3.8 sync.response-time-threshold Set max allowed threshold in milliseconds for sync requests. If the average sync response time is higher than this threshold, all new sync sessions will be rejected until the average response time will be under the threshold. If the value of the property is empty, the threshold is not considered: # Set max allowed threshold in milliseconds for sync requests. # If the value of the property is empty, the threshold is not 40 Installation and configuration # considered. sync.response-time-threshold= 3.12.3.9 sp.syncportal.device.url.automaticRedirect, sp.syncportal.device.url.androidApp, sp.syncportal.device.url.iPhoneApp, and sp.syncportal.device.url.windowsPhoneApp Set the properties to automatically download the proper mobile client: # Automatic mobile download redirect sp.syncportal.device.url.automaticRedirect=true sp.syncportal.device.url.androidApp= sp.syncportal.device.url.iPhoneApp= sp.syncportal.device.url.windowsPhoneApp= If the property sp.syncportal.device.url.automaticRedirect is set to false, a download page is shown to the user and the other properties can be left empty. 3.12.3.10 sp.syncportal.url.download.page Set the property to specify the download page URL. This is the URL that is sent to the user when the senddownload-link Server API is called: sp.syncportal.url.download.page=${sp.public.portal.url}/d 3.12.3.11 storage.provider, storage.identity, storage.credential and storage.container-name # # Set property to 'filesystem' for local file system or to 's3' for # Amazon S3 # storage.provider= # # Set properties with identity and credential specific for the storage. # They are not needed when using 'filesystem' as provider. # storage.identity= storage.credential= # # Set property to the container name on S3; leave it empty if the # storage provider is file system # storage.container-name= 3.12.3.12 ets.pipeline-id, ets.client-region and ets.s3-containername # 41 Installation and configuration # Set the pipeline identifier and the Region's name (it should be one # of the following: us-east-1, us-west-1, us-west-2, eu-west-1, # ap-northeast-1, ap-southeast-1, ap-southeast-2, sa-east-1) where # the pipeline has been created. # Set the S3 bucket where to store the transcoded media. # This property must be set both in the case the storage is S3 # and file system. # In case the storage is S3, this bucket should be the same set # in the property storage.container-name # ets.pipeline-id= ets.client-region= ets.s3-container-name= 3.12.3.13 audio.enabled # # Set the property audio.enabled to true for enabling the music feature. # Otherwise the music items will be managed as file and no metadata # will be stored into db. # audio.enabled=true 3.12.3.14 passwords.toolkit and passwords.regexptoolkit.pattern This property indicates which Password Toolkit will be used to validate user passwords. Default toolkit is RegExpToolkit, which is configurable with a regular expression (see below) to validate the password. A custom toolkit can be used as well. To use a custom password validator, provide a class that extends the Liferay BasicToolkit abstract class and put in the system classpath. # Input a class name that extends # com.liferay.portal.security.pwd.BasicToolkit. This class will be called to # generate and validate passwords. # passwords.toolkit=com.liferay.portal.security.pwd.RegExpToolkit # If you choose to use com.liferay.portal.security.pwd.RegExpToolkit as # your password toolkit, set the regular expression pattern that will be # used to generate and validate passwords. # # Note that \ is replaced with \\ to work in Java. # # default only letters (a-z, A-Z) or numbers (0-9) or dash and they # must be at least 4 characters and at most 16 characters long. # passwords.regexptoolkit.pattern=^[\\w\\d\\-]{4,16}$ 3.12.3.15 passwords.allow.username This property indicates if the username is allowed to be part of the password. If the property is set to false, any possible occurrence of the username within the password will not be accepted by the system. # Allow that the username appears in the password. passwords.allow.username=true 42 Installation and configuration 3.12.3.16 push.apple.keystore.file, push.apple.keystore.password, push.apple.production and push.apple.connection-pool-size These properties configure the apple push notification services. # Set property to the name of the file containing the key used for authenticating the # server with the Apple Push Server. The file has to be placed in the config directory push.apple.keystore.file=push_keystore_file.p12 # Set property to the password used for protecting the key store file push.apple.keystore.password=password # Set property to false if the sandbox Apple Push Server should be used push.apple.production=true # Set property to the number of threads to be reserved for connections to the Apple Push Server push.apple.connection-pool-size=1 3.12.3.17 subscription.enabled Set the property to true for enabling the user subscriptions feature. 3.12.3.18 subscription.warning-before-renewal-inminutes, subscription.insufficient-funds-delay-inminutes, subscription.service-error-delay-in-minutes, subscription.deletion-delay-in-minutes, subscription.keep-tryingto-charge-in-minutes, subscription.allow-downgrade-overquota, subscription.allow-immediate-downgrade Set the properties according to the requirements of the subscription strategy. Set the property subscription.warning-before-renewal-in-minutes for specifying the delay in minutes of the notification warning sent before the renewal of the current user's subscription plan. Set the property subscription.insufficient-funds-delay-in-minutes for specifying the delay in minutes for which payment has failed because of insufficient funds. Set the property subscription.service-error-delay-in-minutes for specifying the delay in minutes for the retry in case the payment service is not reachable. Set the property subscription.deletion-delay-in-minutes for specifying the delay in minutes for the deletion of the subscription plan. Set the property subscription.keep-trying-to-charge-in-minutes with the time for keeping trying to charge. Set the property subscription.allow-downgrade-overquota to enable the downgrade of a subscription without checking the quota of the user. Set the property subscription.allow-immediate-downgrade to enable the immediate downgrade of subscription without waiting the termination of the old (greater) one. 43 Installation and configuration 3.12.3.19 subscription.notification-sender-class, subscription.notification-builder-class, subscription.paymentservice-class, subscription.manager-class, subscription.currency Set the properties for specifying the Java classes that implements Notification Sender, Notification Builder, Payment Service, Subscription Manager, and Currency to be used by the subscription engine. 3.12.3.20 subscription.payment.web Set the URL that will be used to trigger the payments. 3.12.3.21 subscription.paymentverifier.apple.sandbox Set the property to true for using the Apple sandbox (i.e. https:// sandbox.itunes.apple.com/verifyReceipt) to verify payments, otherwise https:// buy.itunes.apple.com/verifyReceipt will be used. 3.12.3.22 subscription.paymentverifier.web.validationurl Set the server URL that will be used to verify payments. 3.12.3.23 media.account-pre-population This property allows the administrator to specify a certain number of files that should appear in the user account when the user first logs in. These files can be of any of the supported types: videos, pictures, audio, or generic files. Configuration This feature can be enabled by setting the property media.account-pre-population in /config/ portal.properties to true, and the files to be added to the user's account have to be placed in the /default-media directory. The files placed here will appear in the user's account. Requirements The videos used for the account pre-population should be already transcoded in a format that can be played in the browsers, e.g. encoded in MPEG4. Note The thumbnails for pictures and videos are created only once and cached for future usage, and when Amazon S3 is used as the backend the files are copied to Amazon S3 once and for each new user they are simply copied inside Amazon S3. When encryption is enabled and Amazon S3 is used as storage provider, each file will be encrypted and uploaded to Amazon S3 when the user signs-up. For this reason the amount of files should be limited in this scenario. 3.12.3.24 ui.media.upload-file-size-limit and ui.media.multipleupload-size-limit Set the maximum limit for one or for multiple media upload operations: # Size limit for one media upload operation 44 Installation and configuration ui.media.upload-file-size-limit=2GB # Size limit for a media upload operation of a set of files ui.media.multiple-upload-size-limit=2GB 3.12.3.25 sapi.upload.max-concurrent-uploads Define the maximum number of concurrent uploads this server instance can handle. Note Default value is -1, which allows all upload requests. 3.12.3.26 sapi.upload.danger-zone.concurrent-uploads Define the threshold at which the server enters the danger zone for concurrent uploads. Note Default value is -1, which allows all upload requests. 3.12.3.27 sapi.upload.danger-zone.device.max-concurrent-uploads Define the maximum number of concurrent uploads a single device is allowed to perform while the server is in danger zone mode. Note Default value is -1, which allows all upload requests. 3.12.3.28 user-import-tool.device.countrya2, user-importtool.device.carrierid and user-import-tool.device.modelid Set the device properties for the default device when provisioning users with the import users tool: # Country A2 code used by the user import tool when inserting a new device to a user user-import-tool.device.countrya2= # Carrier identifier used by the user import tool when inserting a new device to a user user-import-tool.device.carrierid= # Model identifier used by the user import tool when inserting a new device to a user user-import-tool.device.modelid= 3.12.3.29 user-communication.default-channel Set the default communication channel to contact users. Possible values are "email" and "sms". 45 Installation and configuration # Default communication channel for sending user notifications user-communication.default-channel=email 3.12.3.30 sapi.login.persistent-login-token-max-age-in-days Set the maximum age for the persistent login token, returned to the HTTP client as a cookie, when requested. # Defines the persistent login token maximum age, in days sapi.login.persistent-login-token-max-age-in-days=90 3.12.3.31 sharing.email-counter.max-messages Set the max number of sharing Email messages a user can send. # set the max number of sharing email a user can send sharing.email-counter.max-messages=100 3.12.4 Restricting access to administrative Server API calls It is possible to limit the access to administrative Server API calls to specific IP addresses by setting the sapi.admin.allowedips property in the portal-ext.property file. The property must be set to a comma-separated list of IP addresses or IP address masks. Three different types of masks are allowed: • simple IP addresses (e.g. 123.12.34.56,123.12.34.60) • IP addresses containing the * wildcard (e.g. 123.12.34.*) • range of IP addresses (e.g. 123.12.34.1-123.12.34.100) When a request from an administrative account is performed, the IP address from which the request is made is compared with the allowed IP addresses. If the IP address matches one of the allowed IP addresses, the request is accepted; otherwise, an HTTP status code 401 (unauthorized) is returned and the SAPI call is not executed. Warning The default value of the sapi.admin.allowedips property is 127.0.0.1, which means that administrative Server API calls are allowed from localhost only. Removing the property from the file has the same effect as setting it to *.*.*.*, i.e. allowing calls from any IP address. Note If OneMediaHub is running behind a load balancer (for example, ELB from Amazon Web Services), the default remote IP addresses in Tomcat are going to be the ones of the load balancer itself, as it's not trusted by Tomcat out of the box. To identify the latest IP before the load balancer and restrict access to administrative Server API calls, the following valve should be enabled in the file /tools/tomcat/ conf/server.xml: --> 3.12.5 How to configure the OAuth 2.0 client In order to configure the HTTP OAuth 2.0 client change the following properties in the file /config/portal/portalext.properties: # generic configuration of the http oauth client # (timeout and max connection) oauth.http-client.connection-timeout-in-seconds=30 oauth.http-client.max-total-connections=10 # web app and officer configuration # URL of the login page oauth.authorize.code.url= #oauth.authorize.code.url=https://accounts.google.com/o/oauth2/auth # URL to get the oauth access and refresh tokens (aka oauth tokens) oauth.access.token.url= # oauth.access.token.url=https://accounts.google.com/o/oauth2/token # URL to refresh the oauth tokens oauth.refresh.token.url= # oauth.refresh.token.url=https://accounts.google.com/o/oauth2/token # URL to get the user info # (e.g. unique identifier, first name, last name....) oauth.user.info.url= # oauth.user.info.url=https://www.googleapis.com/oauth2/v3/userinfo # unique identifier label in the oAuth server that will be # used also in the OMH server # this makes the feature more flexible because # it could depend on the oAuth server implementation oauth.user.info.id.mapping= # URL that the web app will use to open the OMH main page # this has been added in order to increase the security # of the system in fact the back end can use it to 47 Installation and configuration # validate the URL send by the web app; this is a OMH URL # and it doesn't depend on oauth server oauth.response.redirect.url= # URL that the web app will use to open the error page # this has been added in order to increase the security # of the system in fact the back end can use it to # validate the URL send by the web app; this is a OMH URL # and it doesn't depend on oauth server oauth.response.redirect-error.url= # URL that the web app client will use in case of error # during the authentication flow; this is a OMH URL # and it doesn't depend on oauth server oauth.client.web.logout.redirect.url= # logout URL to invalidate the tokens on the oauth server oauth.client.web.logout.url= #oauth.client.web.logout.url=https://accounts.google.com/logout # property to control the set of resources and operations # that an access token permits # it could depend on the oAuth server implementation oauth.client.web.scope= #oauth.client.web.scope=profile # this section has the keys of the client apps # since the Officer component will impersonate the apps # this info depends on the oauth server side # # these are the 3 main parameters of the web app; oauth.client.web.id= oauth.client.web.secret= oauth.client.web.redirect.url= # windows PC oauth.client.windows.id= oauth.client.windows.secret= oauth.client.windows.redirect.url=urn:ietf:wg:oauth:2.0:oob # mac oauth.client.macos.id= oauth.client.macos.secret= oauth.client.macos.redirect.url=urn:ietf:wg:oauth:2.0:oob # android oauth.client.android.id= oauth.client.android.secret= oauth.client.android.redirect.url=urn:ietf:wg:oauth:2.0:oob # ios oauth.client.ios.id= oauth.client.ios.secret= 48 Installation and configuration oauth.client.ios.redirect.url=urn:ietf:wg:oauth:2.0:oob # blackberry oauth.client.bb.id= oauth.client.bb.secret= oauth.client.bb.redirect.url=urn:ietf:wg:oauth:2.0:oob # windows phone oauth.client.winph.id= oauth.client.winph.secret= oauth.client.winph.redirect.url= # This section configures the custom ssl context for oauth client connections, # this custom context will be used when all properties are configured # # Location of the keystore file to be used by the OAUTH requests oauth.keystore.file= # Keystore password to be used by the OAUTH requests oauth.keystore.password= # Location of the truststore file to be used by the OAUTH requests oauth.truststore.file= # Truststore password to be used by the OAUTH requests oauth.truststore.password= 3.12.6 How to enable user subscriptions If you want to enable the user subscriptions, follow these steps: 1. Set the property subscription.enabled to true in the configuration file /config/portal/portalext.properties 2. Set all the other properties with prefix subscription. according to your subscription strategy (see Section 3.12.3, “portal-ext.properties”) 3. To enable the task for renewing subscriptions, edit the configuration file /config/com/funambol/server/plugin/ SubscriptionRenewalTask.xml setting the enabled property to true 49 Installation and configuration 4. To enable the task for migrating users from a subscription plan to another, edit the configuration file/config/com/ funambol/server/plugin/SubscriptionMigrationTask.xml setting the enabled property to true 5. To enable the task for deleting a user subscription, edit the configuration file /config/com/funambol/server/plugin/ AccountTerminationTask.xml setting the enabled property to true 6. To enable the task for verifying if the payment has been done by a user, edit the configuration file /config/com/ funambol/server/plugin/PaymentVerificationTask.xml setting the enabled property to true 3.12.7 How to view and edit current subscription plans The set of the currently available subscription plans is retrievable and editable in the database, as per table described at Section E.1.38, “fnbl_subscription_plan”. 3.12.8 How to enable user validation in Mobile signup During the signup phase it is possible to validate the user by setting in a proper way the property sp.syncportal.mobilesignup.validation. If it is empty, no user validation will be performed. If it is set to sms, an SMS with the activation link is sent to the phone number specified by the user. If it is set to captcha, the user is validated by checking the captcha code. A CAPTCHA image is displayed by default to the new user during the mobile signup. To remove the CAPTCHA, the sp.syncportal.mobilesignup.validation property should be changed to sms or let empty. The CAPTCHA will not be displayed to mobile clients and during in-browser signup. To edit the CAPTCHA image (number of characters, complexity), the following fragment should be modified in the /tools/ tomcat/webapps/ROOT/web-inf/web.xml file. For more information on valid configurations, refer to [27]. 3.12.9 How to enable user validation in Portal signup During the signup phase it is possible to validate the user by setting in a proper way the property sp.syncportal.signup.validation. 51 Installation and configuration If it is empty, no user validation will be performed. If it is set to sms, an SMS with the activation link is sent to the phone number specified by the user. If it is set to captcha, the user is validated by checking the captcha code. A CAPTCHA image is displayed by default to the new user during the portal signup. To remove the CAPTCHA, the sp.syncportal.signup.validation property should be changed to sms or let empty. The CAPTCHA on the signup page will then be ignored by the Server API and can accordingly be removed from the AJAX signup page. To edit the CAPTCHA image (number of characters, complexity), the following fragment should be modified in the MobileCaptcha nl.captcha.servlet.CaptchaServlet cap.font.arr Courier cap.char.arr 2,3,4,8,9 cap.char.arr.l 4 /tools/ tomcat/webapps/ROOT/web-inf/web.xml file. For more information on valid configurations, refer to [27]. 3.12.10 How to enable gzip compression Gzip compression is used on the AJAX user interface to lower the amount of data sent to browsers that support this type of compression as per RFC 2616 (see [19]). Note All modern browsers and all browsers supported by OneMediaHub handle zip compression. Running an Apache server is mandatory in order to take advantage of gzip compression with OneMediaHub. To configure it, follow these steps: 1. Open the Apache httpd.conf configuration file 2. Check that the module rewrite is loaded: LoadModule rewrite_module modules/mod_rewrite.so 52 Installation and configuration 3. After the DefaultType, add the content type and encoding for jgz and cgz: AddType text/javascript .jgz AddEncoding gzip .jgz AddType text/css .cgz AddEncoding gzip .cgz 4. Add the following rules to the VirtualHost section: RewriteEngine on RewriteCond %{HTTP:Accept-Encoding} gzip RewriteRule (.*)-single\.js$ $1\-single.js.jgz [PT] RewriteCond %{HTTP:Accept-Encoding} gzip RewriteRule (.*)-single\.css$ $1\-single.css.cgz [PT] RewriteCond %{HTTP:Accept-Encoding} gzip RewriteRule (.*)portal-([a-z-]+)\.js$ $1\portal-$2.js.jgz [PT] To test that gzip compression is correctly configured and is working you can: • Add logs to the Apache server: RewriteLog /tmp/rewrite.log RewriteLogLevel 2 Note According to Apache documentation, using a high value for RewriteLogLevel will slow down your Apache server dramatically. Remove it once the testing is over. • Check response headers using a tool for Firefox such as Firebug. If the Content-Encoding is "gzip", the response from Apache is compressed using gzip. Or, you can use wget/curl with a list of URLs. 3.12.11 How to configure the disk quota for media The media sync sources (picture, video, and file) are used to store pictures, videos, and other files on the server's file system (see Section 2.6.1, “File system structure”). The disk quota assigned to each user depends on the user role. Once a user has reached the limit, they cannot upload any more files. By default there are five roles for which a specific quota is assigned. See the table below for the corresponding disk quota assigned to each role. Role Role Description demo Demo user standard Standard user 1G premium Premium user 5G premiumplus Premium Plus user 10G ultimate Ultimate user 50G 53 Quota 150M Installation and configuration Therefore a 'Premium' user can upload up to 5 GB of media. Note Thumbnails and other information generated by the OneMediaHub server and stored on the file system in the -ext subdirectories (see Section 2.6.2, “-ext subdirectories”) are not considered when computing the user quota, even though they use some disk space. New users is assigned the 'standard' role by default. The default role assigned to each new user can be changed by modifying the UserManager configuration. If you want to change the default role, follow these steps: 1. Open the file CaredUserManager.xml found in the directory: Captcha nl.captcha.servlet.CaptchaServlet cap.font.arr Courier cap.char.arr 2,3,4,8,9 cap.char.arr.l 4 /config/com/funambol/ server/admin 2. Locate the defaultRoles property 3. Change the default value standard with one of the following: demo, premium, premiumplus, ultimate. For example: In CaredUserManager.xml there is also a mandatoryRoles property, defining the 'sync_user' role as mandatory. Users who want to sync must have the 'sync_user' role. The 'sync_user' role is therefore configured as mandatory and assigned by default to each new user. This configuration should not be changed. The maximum file system quota assigned to each role can be changed by modifying the file portalext.properties. To change the quota, follow these steps: 1. Open the file portal-ext.properties found in the directory: demo /config/portal/ 2. Locate the quota.roles property. Here are all the allowed quota roles for the users. 3. Locate the role for which you want to change the quota. 4. Change the default value (expressed in bytes) For example, if you want to change the quota assigned to the 'standard' role to 10 MB: .. quota.standard=10M .. 54 Installation and configuration Note The value for the quota may be followed by 'M' if it is expressed in MB or 'G' if it is expressed in GB. For example: 100M stands for '100 Megabytes'; 2G stands for '2 Gigabytes'. 3.12.12 How to configure quota notification Users may receive a notification whenever their occupied storage quota exceeds a configurable percentage. Notifications are disabled by default - to enable them, set the following properties in the file portalext.properties: quota.notification.enabled=true quota.notification.percentage=80 quota.notification.senderclass=com.funambol.portal.quota.SMSNotificationSender The property quota.notification.percentage defines the quota threshold in %. Users will receive an SMS with the contents of the template quota_percentage_reached_notification.txt under /tools/tomcat/webapps/ROOT/template/sms/. 3.12.13 How to configure the Portal to store items on the local file system Note If you want to serve media files and thumbnails stored on the file system using the Apache web server in order to reduce the load on the application server, refer to Section 3.12.14, “Serving media files using the Apache web server” here below. 3.12.13.1 How to configure the Portal to store media on the local file system By default the Portal stores media items (pictures, videos, music and files) on the server's local file system. If you wish to change the path where the files are stored, follow these steps: 1. Open the configuration file / config/portal/portal-ext.properties 2. Change the value of the storage.filesystem-path property from ../../../ds-server/ db to the new path: storage.filesystem-path=../../../ds-server/db 3. Change the symbolic link / tools/tomcat/webapps/ROOT/picture: rm /tools/tomcat/ webapps/ROOT/picture ln -s path_to_the_storage_file_system_root_path /tools/tomcat/webapps/ROOT/picture 55 Installation and configuration 4. Change the symbolic link / tools/tomcat/webapps/ROOT/file: rm /tools/tomcat/ webapps/ROOT/file ln -s path_to_the_storage_file_system_root_path /tools/tomcat/webapps/ROOT/file 5. Change the symbolic link / tools/tomcat/webapps/ROOT/video: rm /tools/tomcat/ webapps/ROOT/video ln -s path_to_the_storage_file_system_root_path /tools/tomcat/webapps/ROOT/video In fact, the root path from which the user's nested directories are created is given by the concatenation of the storage.filesystem-path the media source type, e.g. ../../../ds-server/db/ picture 3.12.14 Serving media files using the Apache web server Media files and thumbnails are stored on the file system (see Section 2.6.1, “File system structure”) and serving them using the Apache web server reduces the load on the application server. Warning Serving the files using the Apache web server also avoids remote code execution vulnerabilities. This is therefore mandatory if files are stored on the local file system. The Content-Disposition header should be set in the response to support the download of the file and set a different filename for the picture, video, or file items. This step is not required if the media files are stored on Amazon S3. To configure it, follow these steps: 1. Open the Apache httpd.conf configuration file 2. Check that the module rewrite is loaded: LoadModule rewrite_module modules/mod_rewrite.so 3. Add the following rules to the VirtualHost section: Alias /picture /dsserver/db/picture Alias /video /dsserver/db/video Alias /file /dsserver/db/file RewriteCond %{QUERY_STRING} (^|&)filename=([^&]*) RewriteRule .* - [E=FILENAME:%2] 56 Installation and configuration Header set "Content-disposition" "attachment; filename=%{FILENAME}e" env=FILENAME UnsetEnv FILENAME 4. Add the following directives at the end of /conf/omh-modcluster.conf: ProxyPassMatch ^/picture ! ProxyPassMatch ^/video ! ProxyPassMatch ^/file ! To test that serving pictures with Apache is correctly configured you can: • Add logs to the Apache server: RewriteLog /tmp/rewrite.log RewriteLogLevel 2 Warning According to Apache documentation, using a high value for RewriteLogLevel will slow down your Apache server dramatically. Remove it once testing is over. • Try to download a picture or a video from the Portal user interface and check response headers using a tool for Firefox such as Firebug. If the Content-Disposition is set with the correct filename, the configuration is working. Important Be sure to disable autoindex for the media files. You can simply do it disabling mod_autoindex in your httpd installation commenting the following line in your httpd configuration file: LoadModule autoindex_module modules/mod_autoindex.so 3.12.15 How to configure the Portal to store items on an online file storage provider 3.12.15.1 How to configure the Portal to store Media items on an online file storage provider If you wish to store media files (pictures, videos, music and files) on an online storage provider, follow these steps: 1. Open the configuration file / config/portal/porta-ext.properties 2. Change the value of the storage.provider property from filesystem to s3: # # Set it to 'filesystem' for local file system or to 's3' for Amazon S3 57 Installation and configuration # storage.provider=s3 3. Assign a value to the storage.container-name property: note that this property is not present when using the filesystem as storage provider, so it is possible that you need to add it. The value should be the container name assigned to the customer (e.g. fdo-container-funambol): storage.container-name=fdo-container-funambol 4. Add the value provided by your storage provider to the storage.identity property: # # Set properties with identity and credential specific for the storage. # They are not needed using 'filesystem' provider. # storage.identity=68E6NMRPOF673B4R09FN 5. Add the value provided by your storage provider to the storage.credential property: storage.credential=hgG56ds7JGHJDF5T65G6gU77h8JHUT6gj78N45dh7iIU 6. Change the value of the sp.mediaserver.url property from http:// : to http:// .s3.amazonaws.com, where is the container name chosen by the customer and unique to Amazon S3, e.g. http://fdo#container-funambol.s3.amazonaws.com. The container must be created in advance and must already be present on the provider before the Portal can start using it: sp.mediaserver.url=http://fdo-container#funambol.s3.amazonaws.com 7. Change the value of the sapi.picture.path property from picture to empty: sapi.picture.path= 8. Change the value of the sapi.video.path property from video to empty: sapi.video.path= 9. Change the value of the sapi.file.path property from file to empty: sapi.file.path= 3.12.16 How to configure the max item size allowed by the DS Server 3.12.16.1 How to configure the max picture size allowed by the DS Server The DS Server allows pictures coming from a client or as result of an API call with a size limit. This limit is configurable and is set by default to 2 GB. To change the size limit, edit the picture sync source configuration file /config/foundation/foundation/fdo#foundation/ PictureSource.xml and modify the value of the maxSize property: 58 Installation and configuration Note For the upload via Portal user interface this limit is the same (2 GB) and is configurable, but the maximum value guaranteed is 2 GB. 3.12.16.2 How to configure the max video size allowed by the DS Server The DS Server allows videos coming from a client or as result of an API call with a size limit. This limit is configurable and is set by default to 2 GB. To change the size limit, edit the video sync source configuration file 250M /config/foundation/foundation/fdo#foundation/ VideoSource.xml and modify the value of the maxSize property: Note For the upload via Portal user interface this limit is the same (2 GB) and is configurable, but the maximum value guaranteed is 2 GB. 3.12.16.3 How to configure the max file and music size allowed by the DS Server The DS Server allows music items and generic files coming from a client or as result of an API call with a size limit. This limit is configurable and is set by default to 2 GB. To change the size limit, edit the file sync source configuration file 250M /config/foundation/foundation/fdo#foundation/ FileSource.xml and modify the value of the maxSize property: Note For the upload via Portal user interface this limit is the same (2 GB) and is configurable, but the maximum value guaranteed is 2 GB. 3.12.17 How to configure the Server API base URL If a system administrator wants to configure the base URL of the Server API (by default /sapi) so that a customer can differentiate the URL from other customers or services (for example, /myapi), the following two files should be modified: • 250M /conf/portal/portalext.properties # configurable base url for SAPI. It must match the servlet-mapping value in web.xml 59 Installation and configuration sapi.baseurl=myapi • /tools/tomcat/ webapps/ROOT/web-inf/web.xml The following lines should be added as mappings for the SAPI servlet and the Impersonate filter: ... SAPIServlet /myapi/* A simpler way to achieve the same result is to change the ${api.baseurl}=sapi in the Quick Configuration as described in Section 3.3, “Quick configuration”. Note Even if SAPI default URL (/sapi) is not used by the AJAX UI or the customer does not implement any API call using the /sapi path, the default /sapi url-pattern for the API servlet in web.xml should not be disabled: doing so will compromise some core client functionalities, such as the media and files sync. OneMediaHub client Apps rely on Server API as well for certain advanced features. Note The name of the new base URL cannot match one of the already existing web applications in the product (so it cannot be for example /funambol, /ROOT, /content) or any already defined path in the ROOT webapp or in the web.xml such as /c, /me or /bb. 3.12.18 Help link OneMediaHub does not provide the content for the Help section out of the box. Every Portal installation should point to an external site or page where the content for the Help section is provided. The easiest way to achieve this is mapping the /help path used in the Portal to an external resource, adding a corresponding directive in the configuration file of the Apache HTTP Server. For example, to redirect /help to http://help.onemediahub.com: RedirectMatch /help http://help.onemediahub.com/ 3.12.19 Contact Us link OneMediaHub doesn't provide a landing page for the contact us link, so every installation needs to open an external page that contains contact information. It is possible to map the /contactus path used by 60 Installation and configuration the Portal to an external page, adding a directive in the configuration file of the Apache HTTP Server. For example, to redirect /contactus to http://funambol.com/contact.html: RedirectMatch /contactus http://funambol.com/contact.html 3.12.20 How to configure Google Analytics It is possible to include Google Analytics in both the Portal and Mobile portal to generate detailed statistics about the visitors. The Impersonate Filter /myapi/* /conf/ portal/portal-ext.properties file should be modified: # Google Analytics web property ID # if not provided, no Google Analytics code in the Portal and Mobile Portal sp.google-analytics.account= To generate a Google Analytics web property ID, check the Google Analytics website at [37] 3.12.21 How to enable iOS Push If a system administrator wants to enable the iOS Push, the following file should be modified: • /config/com/funambol/ server/plugin/IOSFeedbackServiceTask.xml 3.12.22 How to enable/disable music management In order to enable or disable the music management in the portal (by default is enabled), it's possible to modify the file true /config/ portal/portal-ext.properties editing the value of the following property: # Set to true to enable the audio. Otherwise, the audio items will be # managed as file and no metadata will be stored into db. audio.enabled= 3.12.23 How to enable media transcoding If you want to enable the media transcoding, follow these steps: 1. Set the property media.transcoding-service-class to com.funambol.transcoding.impl.ets.ETSTranscodingServiceImpl in the configuration file / config/portal/portal-ext.properties 2. If the storage is an S3 service, you should set the properties ets.pipeline-id, ets.clientregion, and ets.s3-container-name according to the AWS setup. The properties storage.container-name and ets.s3-container-name must have the same value 3. If the storage is a filesystem, the properties storage.identity, storage.credential, ets.pipeline-id, ets.client-region, and ets.s3-container-name must be set according to the AWS setup 61 Installation and configuration 4. Enable the task for deleting the transcoding jobs table, editing the configuration file /config/com/funambol/ server/plugin/TranscodingJobDeleteScheduledTask.xml and setting the property enabled to true: 3.12.24 How to customize the Privacy Policy and Term of Use pages redirecting the OneMediaHub links to external pages. 1. Open the Apache httpd.conf configuration file 2. Check that the mod_rewrite module is loaded: LoadModule rewrite_module modules/mod_rewrite.so 3. In the VirtualHost section check that the RewriteEngine is on: RewriteEngine on 4. Add the following lines setting the desired URL: RedirectMatch /ui/jsp/privacy.jsp https://www.example.com/ privacy_policy RedirectMatch /ui/jsp/terms.jsp https://www.example.com/ terms_of_use RedirectMatch /ui/mobile/jsp/pp.jsp https://www.example.com/ privacy_policy RedirectMatch /ui/mobile/jsp/toc.jsp https://www.example.com/ terms_of_use 3.12.25 How to configure user-level communication channel The user-level communication channel allows the configuration of the desired communication channel to sent user communications, at a user level, where two distinct users can receive notifications via E-Mail or SMS. This feature requires a correct configuration of the OneMediaHub E-Mail and SMS providers. 62 Installation and configuration If you wish to use this feature, you must configure the subscription notification sender and builder, by modifying the file /config/ portal/portal-ext.properties: subscription.notification-senderclass=com.funambol.subscriptions.notifications. usercommunicationchannel.UserCommunicationChannelNotificationSender subscription.notification-builderclass=com.funambol.subscriptions.notifications. usercommunicationchannel.UserCommunicationChannelNotificationBuilder The default user communication channel is configurable in the file /config/portal/portal-ext.properties. You can use the values email or sms. # Default communication channel for sending user notifications user-communication.default-channel= 3.13 Server URL configuration Change the serverURI property in the /config/Funambol.xml file: 3.13.1 Changing the default server port The default port is 8080, but you may choose to use a different port. In that case, you must modify the following files, substituting your preferred port number to 8080: • http://:/sync /tools/tomcat/conf/ server.xml • /config/portal/ portal-ext.properties (see Section 3.12.3, “portal-ext.properties”) • /tools/tomcat/ webapps/ROOT/html/devices/content.properties • /config/com/funambol/ pimlistener/PIMListenerConfiguration.xml 3.13.2 Supporting multiple server URL If you need to support multiple server URLs, follow these steps: 1. edit the /config/portal/ portal-ext.properties file and set the property ${portal.server} to one of the desired URLs (see Section 3.3, “Quick configuration”), or: 63 Installation and configuration edit the /config/ Funambol.xml file (see Section 3.13, “Server URL configuration”) 2. configure your web server (e.g. Apache) to forward the other URLs that you wish to support to the previously defined URL 3.14 Email configuration The OneMediaHub Portal can send several types of Email notifications, provided that an SMTP server is configured. This section describes how to customize the notification messages and how to configure the portal to send Emails. 3.14.1 New user messages It is possible to customize the text of the Email messages sent to the users. To configure the registration Email messages, edit the files active_user_mail_body.txt or create_user_mail_subject.txt under /tools/tomcat/webapps/ROOT/template/mail/. Note The files are under different folders according to the languages defined in the portalext.config file, e.g. /tools/tomcat/ webapps/ROOT/template/mail/it /tools/tomcat/ webapps/ROOT/template/mail/en The changes are applied at runtime, so there is no need to restart the server. 3.14.2 Email counter configuration for messages containing the app download URL It is possible to limit the number of Email messages for download links sent by the OneMediaHub Portal to a predefined value. You can customize this behavior by editing the following properties in the file /config/portal/portal-ext.properties: sp.syncportal.email-counter=true sp.syncportal.email-counter.messages=10 To disable the counter: sp.syncportal.email-counter=false In the OneMediaHub Portal you cannot set a different number of download messages for each user. The counter controls all the email messages used for sending the download link on a monthly basis. The configuration of the Email counter is immediately effective, with no need to restart the server. 64 Installation and configuration Important The configuration defined in portal-ext.properties for the Email counter applies to new users only, or when existing users are reset, or automatically every month. 3.15 Reminder Emails The OneMediaHub periodically sends, to all users who have not opted out of receiving reminders, a summary of all the digital content they have secured to OneMediaHub. This feature can be configured by editing the following file: /config/com/funambol/ server/plugin/ReminderEmailScheduledTask.xml It is possible to customize the layout of the Email messages sent to the users. To configure the reminder Email messages, edit the files reminder_body.ftl or reminder_thumbnails.ftl under /tools/tomcat/webapps/ ROOT/ui/notifications. 3.16 SMS Sender configuration To receive the SMS notification message, configure the SMS Service by editing the following file: /config/com/funambol/ server/sms/SMSProvider.xml with the user and password to access the service provider: By default, the option to limit the OneMediaHub Portal to a predefined number of SMS messages for download links or for OTA configuration messages is enabled. You can customize this behavior by editing the following properties in the file /config/portal/portal-ext.properties: sp.syncportal.sms-counter=true sp.syncportal.sms-counter.messages=10 To disable the counter, change the sp.syncportal.sms-counter property to false: sp.syncportal.sms-counter=false Note In the OneMediaHub Portal you cannot set a different number of download messages for each user. The counter controls all the SMS messages (text SMS and OTA configuration SMS) for the given user in a month. 65 Installation and configuration The configuration of the SMS counter in the product is automatically reloaded, there is no need to restart the service. The configuration defined in portal-ext.properties for the SMS counter applies to new users only or when existing users are reset either automatically (monthly), or by an administrator. The administrator can reset the SMS counter from the user interface. The number of SMS's left for existing users is recorded in the database and changes according to usage. The product tracks the number of SMS requests sent to the SMS service (OneMediaHub, by default, supports SubitoSMS, see below.) The number of SMS's used can be different if any text or OTA message requires concatenated SMS. Important OneMediaHub, by default, supports SubitoSMS, which is an SMS provider (for more information, please see [30]). OneMediaHub uses SubitoSMS's APIs for all services that require sending SMS messages: OTA configuration, SMS push, Clients download, etc. Unless you received SubitoSMS credentials from OneMediaHub, you will need to sign up for the service at [30], purchase some credit and configure your username and password. If you prefer to use a different SMS provider, please contact the Funambol support [31], since it will require further customization and configuration. 3.17 OTA settings provisioning configuration Edit the file /config/ portal/portal-ext.properties; OTA works with the following settings by default: # # Funambol OTA settings # sp.syncportal.ota.account.name=onemediahub sp.syncportal.ota.account.pin=1010 # set sp.syncportal.sms-counter to 'true' to limit the number of SMS messages per month sp.syncportal.sms-counter=false sp.syncportal.sms-counter.messages=10 The option to limit OTA to a predefined number of SMS messages for each user/device is disabled by default: sp.syncportal.sms-counter=false sp.syncportal.sms-counter.messages=10 To enable the counter, change the sp.syncportal.sms-counter property to: sp.syncportal.sms-counter=true 66 Installation and configuration and set the desired value. The counter controls all monthly SMS messages (text SMS's and OTA configuration SMS's) for the given user. To change the name of the OTA profile, change the value of sp.syncportal.ota.account.name in the file /config/portal/ portal-ext.properties: # # Funambol OTA settings # sp.syncportal.ota.account.name=funambol To change the name of the sender change the value of the property sender in the file /config/com/funambol/server/ sms/SMSService.xml: Note In the OneMediaHub Portal you cannot set a different number of OTA messages for each user. 3.18 Forgot password The forgot password can be sent to the user via Email or via SMS. It is possible to customize the texts of both the SMS and Email message sent to the user. To configure the “forgot password” SMS message, edit the file send_password_sms_body.txt under /tools/tomcat/ webapps/ROOT/template/sms/. To configure the “forgot password” Email message, edit the files send_password_mail_subject.txt and send_password_mail_body.txt under /tools/tomcat/webapps/ROOT/ template/mail/. Note The files are under different folders according to the languages defined in the portalext.config file, e.g. /tools/tomcat/ webapps/ROOT/template/mail/it /tools/tomcat/ webapps/ROOT/template/mail/en or 67 Installation and configuration /tools/tomcat/ webapps/ROOT/template/sms/pt /tools/tomcat/ webapps/ROOT/template/sms/de The changes are applied at runtime, so it is not needed to restart the server. By default, the sending is set to use the Email. You can force the sending via SMS by editing the sp.syncportal.pwd-via-sms property in the file /config/portal/portal-ext.properties: sp.syncportal.pwd-via-sms=true 3.19 Push policy configuration With OneMediaHub it is possible to configure different push policies for different device types. In particular, the following policies are defined and can be customized editing the configuration files listed below (currently just one policy does apply): 1. SMS service configuration (see the section called “SMS Service”): /config/com/funambol/ server/sms/SMSProvider.xml This file is used to configure the SMS service. The following sections describe the format of a *PushSender.xml configuration file and the default values for an out-of-the-box installation of the OneMediaHub. 3.19.1 The PushSender configuration file A PushSender configuration file is an XML file that allows the customization of the properties listed below, together with the configuration file's relevant fragment. Note More than one source can be specified. The available source names are: card, scard, cal, scal, event, task, and picture. Enabling/disabling TCP push Note This applies to both connection-less and connection-oriented push. Enabling/disabling SMS push [true|false] Notification archiving (for both TCP and SMS push) Push notifications can be archived in the database (in the table fnbl_push_sender_notification) if specified. This is enabled by the properties archiveTCPIPNotification and archiveSMSNotification: [true|false] 68 Installation and configuration[true|false] By default, archiveTCPIPNotification is set to false and archiveSMSNotification to true. Resending interval for SMS push In the case of SMS push, a situation that should be avoided is the continuous sending of SMS messages when these are not being received by the device. In fact, GSM/CDMA networks queue SMS messages until a phone joins the network again. A phone could be disconnected from a network if for example it is turned off or in an area without GSM/CDMA coverage. Since the phone will receive the SMS later on, the system should not send other notifications until the previous one is received by the phone. However, if the system stops sending SMS notifications until the previous notification has been received and for some reason one of the SMS notifications is lost, the server will simply not push the device any more. For this reason, if the OneMediaHub does not receive a status indicating that a notification has been received, after a while it sends another notification anyway. This tolerance can be configured by changing the property maxDeliveryDelayHours: [true|false] Note If the parameter archiveSMSNotification is set to false, SMS messages are sent without checking whether the previous one was received. Filtering Different types of filtering are available. For example, device filtering allows you to specify a particular device or group of devices that should be excluded by the push, while source filtering lets you specify which data sources should be pushed. In certain situations it is useful to ban some phone numbers so that no SMS messages are sent to them. For example, in case of test phones or to selectively avoid some phones being pushed. This can be done by banning one or more phone numbers with the deviceFilter.bannedMsisdn property: [number of hours] Note You can change the length attribute and have more ... sections to specify multiple regular expressions that will be matched. If the phone number to which the notification should be sent matches one of the given regular expressions, the push is discarded. For more information on the regular expression grammar used by this property, see [21]. A sender is also configured to notify changes to specific sources only. This is done by the property syncsourceFilter:Note More than one source can be specified. The available source names are: card, scard, cal, scal, event, stask, picture. 3.20 OneMediaHub SNMP Appender configuration The OneMediaHub SNMP Appender enables the Data Synchronization Service to send SNMP notifications (traps) as part of the server logging facility. 3.20.1 Configuration parameters The OneMediaHub SNMP Appender sends its traps using SNMPv2c and according to the MIB defined in FUNAMBOL-LOGGING-MIB.txt (see Appendix D, FUNAMBOL-LOGGING-MIB.txt). All SNMP parameters and OIDs are highly configurable; the parameters are described in the following table: 70 Installation and configuration Property Default values Description managerAddress 127.0.0.1 The manager's IP address managerPort 162 The UDP port the manager listens to for traps trapOID 1.3.6.1.4.1.27219.2.2.0.1 Enterprise OID to be specified in the SNMP trap community public Community string ("public", "private") thresholdLevel ERROR Logging level over which a log record should be notified (including the specified level variables Hash map of OID-format entries. Each format entry specifies the layout for the associated OID. The following substitutions are performed: {1.3.6.1.4.1.27219.2.1.1=%t, 1.3.6.1.4.1.27219.2.1.2=%L, 1.3.6.1.4.1.27219.2.1.3=%l, 1.3.6.1.4.1.27219.2.1.4=%s, 1.3.6.1.4.1.27219.2.1.5=%d, 1.3.6.1.4.1.27219.2.1.6=%u, 1.3.6.1.4.1.27219.2.1.7=%S, 1.3.6.1.4.1.27219.2.1.8=%m} • %t – timestamp in DateAndTime format (see [22]) • %tt – timestamp in the yyyy-MM-dd HH:mm:ss format • %L – log level as integer value (0: FATAL, 1: ERROR, 2: WARN, 3: INFO, 4: DEBUG, 5: TRACE) • %LL – log level as string (FATAL, ERROR, WARN, INFO, DEBUG, TRACE) • %l – logger name • %s – session id • %d – device id • %u – user name • %S – source • %m – log message 3.20.2 Configuration file OneMediaHub SNMP Appender's configuration file is funambol.snmp-appender.xml, copied by the installation procedure in the server appender config path: /config/com/funambol/ server/logging/appender Below is an example of the configuration file: 71 Installation and configuration The OneMediaHub SNMP Appender can be attached to any OneMediaHub logger by simply adding it to the list of the appenders associated with the logger. In order to enable it, you can add the following lines to the configuration file/config/com/funambol/server/logging/ logger/funambol.xml: If you do not have an SNMP server, it is still possible to verify that the SNMP Appender is working properly using the Unix command snmptrapd. For an example, please see Appendix F, Examples of sent SNMP traps. 3.20.3 Log records filtering The OneMediaHub SNMP Appender only notifies log records that have a logging level higher than the specified threshold. It is possible to apply an additional filter to the given logging event, based on the property filterCriticalMessages and on the set of criticalMessages. The property filterCriticalMessages can be set to true in order to verify if the log event is included in the set of criticalMessages. The property criticalMessages contains a list of exception class names considered critical. When the filterCriticalMessages is true, the SNMP Appender parses all incoming log events to see if the stack trace contained in the event description matches any of the exceptions listed in criticalMessages. In this case, a new logging event is created. If the level of the event is FATAL and the event is not critical, a logging event will be generated with a message starting with "Unknown fatal error". Note The criticalMessages and filterCriticalMessages parameters can be configured in the Appender configuration file (see Section 3.20.2, “Configuration file”). 3.20.4 OneMediaHub SNMP error trap For more details on SNMP errors, please see Section 7.6, “SNMP errors”. 3.21 Location from IP address and Accept-Language header OneMediaHub uses multiple services to determine the country from which the request originates, and, for example, to automatically set the country in the AJAX Portal or Mobile Portal signup. 74 Installation and configuration 3.21.1 IP address and IP2Location IP2Location offers an external service supported by OneMediaHub, used to determine the country from which the request originates based on the user's IP address, so that the appropriate country and timezone can automatically be set. If you have a valid IP2Location IP-Country Database license (see [32]) for your server, please contact their support for information on how to use this service. 3.21.2 Accept-Language header If an IP2Location IP-Country Database license is not present and the ipcountry table is not populated or the user's country cannot be decoded from the IP address, OneMediaHub uses the Accept-Language header as a fallback for determining the country from which the request originates. The feature can be disabled, modified or extended, tuning the content of the fnbl_country_language table where different locales from the Accept-Language header are mapped to their countries. To disable the feature, delete every record in the table. 3.22 Collecting client logs The server is able to receive client log files and save them on the file system for future issues, for investigation/troubleshooting purposes. The URL used by the client for sending the request is http://server:port/client-log The client should send the log as text/plain or in a compressed format, but in any case, the server will store it as a compressed file in order to save file system space. By default, client logs are stored in funambol.snmp-appender /logs/clients using the following structure and naming convention: / _ _