Spirent Communications NOMADHD-01 Table Top HW device for Voice Quality and Call Performance User Manual Part 1

Spirent Communications Inc Table Top HW device for Voice Quality and Call Performance Users Manual Part 1

Contents

Users Manual Part 1

Spirent Communications Nomad User’s Manual Spirent Communications 7340 Executive Way Suite A Frederick, MD 21704 Document Version: 3.2
Nomad User’s Manual Copyright © Spirent Communications, Inc. 2013 2 Notice: The Bluetooth word mark and logos are owned by the Bluetooth® SIG, Inc. and any use of such marks by Spirent Communications, Inc. is under license. Other trademarks and trade names are those of their respective owners. Notice: NOTE: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference in a residential installation. This equipment generates, uses and can radiate radio frequency energy and, if not installed in accordance with the instructions, may cause harmful interference to radio communications. However, there is no guarantee that interference will not occur in a particular installation. If this equipment does cause interference to radio or television reception, which can be determined by turning the equipment off and on, the user is encouraged to try to correct the interference by one or more of the following measures:  Reorient or relocate the receiving antenna  Increase the separation between the equipment and the receiver  Connect the equipment into an outlet on a circuit different from that to which the receiver is connected  Consult the dealer or an experienced radio/TV technician for help Changes or modifications to the Nomad hardware not expressly approved by Spirent Communications, Inc. may void the user’s authority to operate the equipment. Notice: Spirent’s Nomad ME hardware contains FCC certified transmitter modules FCC ID R47F2M03MLA. Spirent’s Nomad HD ME hardware is FCC certified under the FCC ID WR2-NOMADHD-01. Notice: This Class A digital apparatus complies with Canadian ICES-003. Cet appareil numérique de la Classe A est conforme à la norme NMB-003 du Canada.
Nomad User’s Manual Copyright © Spirent Communications, Inc. 2013 3 Contents 1 Introduction ..................................................................................................................................................................... 7 2 Nomad System Configuration ................................................................................................................................... 9 2.1 System Component Verification ..................................................................................................................... 9 2.1.1 Microsoft .NET Framework 3.5 SP1 .................................................................................................. 11 2.1.2 Disable User Account Control (Windows Vista and Windows 7) ......................................... 12 2.2 Software Installation ......................................................................................................................................... 15 2.3 Hardware Installation ...................................................................................................................................... 16 3 Application Workflow ................................................................................................................................................ 17 4 Specifying Nomad Settings ....................................................................................................................................... 18 4.1 Specifying General Settings ............................................................................................................................ 19 4.2 Specifying GPS/Time Settings ....................................................................................................................... 20 4.2.1 Time Synchronization ............................................................................................................................. 20 4.2.2 GPS Settings ................................................................................................................................................ 21 4.3 Specifying Call Performance Settings ........................................................................................................ 22 4.3.1 Call Server .................................................................................................................................................... 22 4.3.2 Logging Session Auto-Stop ................................................................................................................... 23 4.3.3 Audio Verification ..................................................................................................................................... 23 4.3.4 CDR Lookup ................................................................................................................................................ 23 4.3.5 Call Result Warnings ............................................................................................................................... 25 4.4 Specifying Voice Quality Settings ................................................................................................................. 26 4.4.1 MOS Thresholds ........................................................................................................................................ 26 4.4.2 Events ............................................................................................................................................................ 28 4.4.3 Bluetooth Auto-Dial ................................................................................................................................. 28 4.4.4 Audio Server ............................................................................................................................................... 29 4.4.5 MOS Correction ......................................................................................................................................... 29 4.4.6 IMS Setup ..................................................................................................................................................... 29 4.4.7 Default Scoring Algorithm..................................................................................................................... 30 4.5 Specifying Email Campaign Settings .......................................................................................................... 30 5 Defining a Test .............................................................................................................................................................. 32 5.1 Assigning Tasks to Channels.......................................................................................................................... 33 5.1.1 Voice Quality Tasks .................................................................................................................................. 34 5.1.2 Call Performance Tasks .......................................................................................................................... 34 5.1.3 Delay Task ................................................................................................................................................... 35 5.2 Modifying Task Settings .................................................................................................................................. 36 5.2.1 Device Info ................................................................................................................................................... 36
Nomad User’s Manual Copyright © Spirent Communications, Inc. 2013 4 5.2.2 Session........................................................................................................................................................... 37 5.2.2.1 IMS Client Settings ............................................................................................................................... 40 5.2.3 Channel Settings ........................................................................................................................................ 41 5.2.4 Call Campaign ............................................................................................................................................. 44 5.2.5 Auto-Level Assist ...................................................................................................................................... 46 5.2.6 Email Campaign ......................................................................................................................................... 50 5.2.7 Bluetooth Connection ............................................................................................................................. 51 5.2.8 Reconnect All Bluetooth Channels .................................................................................................... 52 5.3 Saving Test Settings .......................................................................................................................................... 53 5.3.1 Working with Configured Tasks ......................................................................................................... 54 5.3.2 Working with Profiles ............................................................................................................................. 56 6 Initiating a Test ............................................................................................................................................................. 58 6.1 Reviewing the Current Channel Configuration ...................................................................................... 58 6.2 Device Setup ......................................................................................................................................................... 58 6.3 Placing Calls on Mobile Handsets ................................................................................................................ 59 6.4 Selecting the Log File Storage Location .................................................................................................... 60 6.5 Time Synchronization ...................................................................................................................................... 60 7 Monitoring a Test in Progress ................................................................................................................................ 62 7.1 Nomad Test Window Orientation ............................................................................................................... 62 7.2 Manual Volume Adjustment (Voice Quality Testing) .......................................................................... 74 7.2.1 Manual Downlink Volume Adjustment ............................................................................................ 74 7.2.1.1 Downlink Volume Adjustment for Bluetooth Connections ................................................ 74 7.2.1.2 Downlink Volume Adjustment for Analog Connections ...................................................... 75 7.2.2 Manual Uplink Volume Adjustment .................................................................................................. 75 7.3 Event Logging and Event Detection (Voice Quality Testing) ........................................................... 75 7.4 Interpreting MOS Results (Voice Quality Testing) ............................................................................... 78 8 Analyzing the Results ................................................................................................................................................. 79 8.1 Generating a Report .......................................................................................................................................... 80 8.2 Displaying a Formatted Report .................................................................................................................... 82 8.3 Report Template Customization .................................................................................................................. 89 8.4 Interactive Viewer ............................................................................................................................................ 90 8.5 Visualizing Data Geographically ................................................................................................................... 92 8.6 Reporting IP Analytics ...................................................................................................................................... 94 8.7 Collecting Random Audio Sample Files (Voice Quality Testing) .................................................... 99 8.8 Viewing the Logs ............................................................................................................................................. 100 8.9 Waveform Analysis with the PESQ Tools GUI (Voice Quality Testing) ..................................... 100
Nomad User’s Manual Copyright © Spirent Communications, Inc. 2013 5 8.10 Troubleshooting Incomplete Data ........................................................................................................... 103 8.11 Offline Scoring (Voice Quality Testing) .................................................................................................. 106 8.11.1 Creating a New Logging Session Using Offline Scoring .......................................................... 107 8.11.2 Retrieving Complementary Uplink Data for an Existing Logging Session ..................... 108 8.12 Scoring PESQ and POLQA............................................................................................................................. 109 8.12.1 Re-Score Files .......................................................................................................................................... 109 8.12.2 Batch Scoring ........................................................................................................................................... 111 8.12.3 Offline Scoring ......................................................................................................................................... 113 9 Voice Quality Configuration Options ................................................................................................................ 114 9.1 Voice Quality Test Calibration ................................................................................................................... 115 9.2 Audio Server Testing ..................................................................................................................................... 116 9.3 IMS Client Testing ........................................................................................................................................... 117 9.4 Mobile-to-Mobile Testing ............................................................................................................................. 117 9.5 Remote Unit Testing ...................................................................................................................................... 120 9.6 Landline Module Testing .............................................................................................................................. 122 9.7 Base Station Simulator Testing.................................................................................................................. 123 9.8 Head and Torso Simulator ........................................................................................................................... 125 9.8.1 HATS Three Channel Configuration ............................................................................................... 125 9.8.2 HATS Two Channel Configuration .................................................................................................. 127 9.9 Wideband Testing ........................................................................................................................................... 128 9.10 Multi-RAB Testing ........................................................................................................................................... 129 9.11 Voice Delay Testing ........................................................................................................................................ 131 10 Call Performance Configuration Options ................................................................................................... 135 10.1 Mobile Originated Testing ........................................................................................................................... 135 10.2 Mobile Terminated Testing ......................................................................................................................... 138 10.3 Mobile-to-Mobile Testing ............................................................................................................................. 141 Appendix A – Glossary ...................................................................................................................................................... 144 General Testing Terms ................................................................................................................................................ 144 Voice Quality Testing Terms ..................................................................................................................................... 144 Call Performance Testing Terms ............................................................................................................................. 145 Appendix B – Call Performance Events ..................................................................................................................... 146 Call Outcome Events ..................................................................................................................................................... 146 Device State Events ....................................................................................................................................................... 147 Appendix C – Confidence Interval Calculation ....................................................................................................... 148 Appendix D – PESQ Tools GUI Case Study ................................................................................................................ 149 Appendix E – Introduction to POLQA ......................................................................................................................... 153
Nomad User’s Manual Copyright © Spirent Communications, Inc. 2013 6 Appendix F – Nomad HD Hardware ............................................................................................................................ 156 Important Safety Note ................................................................................................................................................. 156 Nomad HD LEDs ............................................................................................................................................................. 156 Base Unit Physical Interfaces .................................................................................................................................... 157 Physical Specifications................................................................................................................................................. 157 Power Specifications .................................................................................................................................................... 157 Environmental Requirements .................................................................................................................................. 157
Nomad User’s Manual Chapter 1 – Introduction Copyright © Spirent Communications, Inc. 2013 7 1 Introduction Spirent Communications Nomad provides a comprehensive and efficient measure of real-world mobile device performance by automating high-volume call control testing sequences and voice quality characterization on virtually any Bluetooth-enabled handset, regardless of wireless technology or device model. Spirent Nomad allows you to:  Reduce the manpower cost of collecting both voice quality and call performance data for testing through automation  Provide statistically repeatable and reliable results to impact device changes  Capture poorly performing devices prior to market launch  Improve device quality, user experience and brand perception In the Nomad system, mobile handsets make calls to and receive calls from centralized voice and call servers. All servers are managed through a PC-based interface. The servers collect and store call record and degraded audio information which is merged with locally stored results at test completion for a comprehensive view of device performance. Figure 1-1 - Nomad System Overview
Nomad User’s Manual Chapter 1 – Introduction Copyright © Spirent Communications, Inc. 2013 8 The metrics available as outputs from the Nomad system include:  Voice Quality (Uplink and Downlink)  Voice Delay  Call Initiation Performance (Mobile Originated and Mobile Terminated Calls)  Call Retention Performance (Mobile Originated and Mobile Terminated Calls)  Call Setup Time  Loss of Network  Bluetooth Voice Performance This document describes the setup and operation of the Spirent Nomad system for all voice quality, Bluetooth and call performance functionality. All testing is performed by controlling the Spirent Mobile Equipment (ME) using the Nomad software interface. This document describes the setup and operation of the Spirent Nomad system for all test types.
Nomad User’s Manual Chapter 2 – Nomad System Configuration Copyright © Spirent Communications, Inc. 2013 9 2 Nomad System Configuration The Spirent Communications Nomad system contains both hardware and software for mobile performance evaluation. Prior to installing Nomad for the first time, please verify that all hardware components are present, including a laptop meeting the Nomad computing requirements. Also note that when setting up the Nomad system for the first time, the software must be installed prior to connecting the hardware. The following sections describe the system configuration process. 2.1 System Component Verification The standard hardware components shipped with Nomad systems are:  Spirent Nomad HD ME–or- Spirent Nomad ME hardware unit  GPS receiver  USB cable  Six-foot TRS 2.5 mm audio cables – one for each licensed channel (up to four narrowband cables and four HD cables)  AC adapter  Cigarette lighter adapter  Calibration cable The Bluetooth profiles used by Nomad are:  Hands-Free Profile (HFP) 1.5  Hands-Free Profile (HFP) 1.6 (For Nomad HD ME hardware only)  Headset Profile (HSP) 1.1 A laptop computer and up to four test mobiles, not supplied, are also required to work with the Nomad system. Any computer used with the Nomad system must meet the Nomad minimum computing requirements:
Nomad User’s Manual Chapter 2 – Nomad System Configuration Copyright © Spirent Communications, Inc. 2013 10 Computing Requirement Optimal Performance Minimal Performance Processor Intel® CoreTM i7 Intel® CoreTM i5 RAM 8 GB 4 GB Free Disk Space 20 GB 10 GB Operating System Windows XP, Vista or 7 Windows XP USB Drive N/A USB 2.0 Drive Software N/A  Microsoft .NET Framework 3.5 SP1 (See Section 2.1.1)  Microsoft Excel 2007 or higher (for the Report Generator); alternatively, download Microsoft Excel Viewer to view the .XLSX output reports  User Account Control setting disabled on Windows Vista and 7 machines (See Section 2.1.2)  User must be logged in as a Windows Administrator. Nomad will not run under a non-admin login. Dedicated High Speed Internet Connection N/A Sustained internet connection of at least 50 kbps is required for the following situations:  Determining optimal volume levels for a voice quality test using Automatic Level Assist  Viewing live uplink data during a voice quality test  Starting a Mobile Terminated call campaign  Obtaining the report for a Mobile Terminated or Voice Quality call campaign  Automatic internet-based time sync during testing (GPS time sync is recommended)
Nomad User’s Manual Chapter 2 – Nomad System Configuration Copyright © Spirent Communications, Inc. 2013 11 2.1.1 Microsoft .NET Framework 3.5 SP1 The Nomad software application requires Microsoft .NET Framework 3.5 SP1. Although the Nomad software installation includes Microsoft .NET Framework 3.5 SP1, Spirent Communications recommends performing a manual check for this component. To verify the current version of Microsoft .NET:  From the Windows Start Menu select Settings → Control Panel → Add or Remove Programs (Programs and Features in Microsoft Vista)  Verify that Microsoft .NET Framework 3.5 SP1 is installed If Microsoft .NET Framework 3.5 SP1 is required:  Browse to the Microsoft .NET Framework download site: http://www.microsoft.com/downloads/details.aspx?FamilyID=ab99342f-5d1a-413d-8319-81da479ab0d7&DisplayLang=en  Follow the instructions to install
Nomad User’s Manual Chapter 2 – Nomad System Configuration Copyright © Spirent Communications, Inc. 2013 12 2.1.2 Disable User Account Control (Windows Vista and Windows 7) The Nomad software application requires that the User Account Control (UAC) setting be disabled on Windows Vista and Windows 7 machines. To disable UAC in Windows Vista:  From the Windows Start menu select Settings → Control Panel.  Under User Accounts and Family Safety select Add or remove user accounts. Figure 2-1 - User Accounts and Family Safety Area  Click on any user account (i.e. the Guest account).  In the list of user account options, select Go to the main User Accounts page. Figure 2-2 - User Account Options  Under Make changes to your user account, select Change security settings.
Nomad User’s Manual Chapter 2 – Nomad System Configuration Copyright © Spirent Communications, Inc. 2013 13 Figure 2-3 - Make changes to your user account  Deselect Use User Account Control (UAC) to help protect your computer. Figure 2-4 - User Account Control Screen  Click OK to confirm your selection.  When prompted, restart your computer to apply the changes. To disable UAC in Windows 7:  From the Windows Start menu select Settings → Control Panel.  Under User Accounts and Family Safety select Add or remove user accounts. Figure 2-5 - User Accounts and Family Safety Area
Nomad User’s Manual Chapter 2 – Nomad System Configuration Copyright © Spirent Communications, Inc. 2013 14  Click the User Accounts link. Figure 2-6 - User Accounts Area  In the list of user account options, select Go to the main User Accounts page  Select the Change User Account Control Settings option. Figure 2-7 – Make changes to your user account Screen  Lower the notification bar from Always notify to Never notify. Figure 2-8 - User Account Control Screen  Click OK to confirm your selection.  When prompted, restart your computer to apply the changes.
Nomad User’s Manual Chapter 2 – Nomad System Configuration Copyright © Spirent Communications, Inc. 2013 15 2.2 Software Installation The Nomad installation file may be obtained from your Spirent Communications representative. Once you have the Nomad installation package:  Unzip the package (if applicable) and click on Setup.  Follow the instructions to complete the software installation. Figure 2-9 - Nomad Installation Complete After Nomad has been installed, the software will inform you automatically when upgrades are available provided that:  The computer running the Nomad software is connected to the internet.  The Nomad hardware is attached.  Support and maintenance is up-to-date for the Nomad license in use.
Nomad User’s Manual Chapter 2 – Nomad System Configuration Copyright © Spirent Communications, Inc. 2013 16 2.3 Hardware Installation Figure 2-10 - Spirent Communications ME and HD ME Hardware Units Configure the Spirent ME or Spirent HD ME hardware by making the following connections:  Plug in the AC Adapter and connect to the +12V port on the Spirent Nomad unit.  Connect the GPS Receiver to the GPS port.  For analog voice quality testing, connect the audio cables to the desired licensed channel ports on the front of the unit (Channel 1, Channel 2, etc.). The test handsets will be connected later during the data collection process.  Connect the USB laptop cable to the PC port on the Spirent ME unit.  With the laptop turned on, connect the other end of the USB laptop cable to the USB port on the computer. Take note of this port so that you always connect the hardware unit to the same port.  Make sure that test handsets are available. The handsets will be connected during the session configuration process. The USB hardware detection will take place upon the first connection. A notification will be displayed when the automatic driver installation procedure is complete. To confirm that the driver has been installed, look for the Metrico Devices → Metrico ME Device(for ME devices) or Spirent Communications Devices → Spirent HD ME Device (for Spirent HD ME devices) in the Windows Device Manager. At this time the system is configured and ready for mobile performance testing.
Nomad User’s Manual Chapter 3 – Application Workflow Copyright © Spirent Communications, Inc. 2013 17 3 Application Workflow This section provides an overview of the Nomad workflow:  Open the program  Specify Nomad settings  Define the test by assigning tasks to channels and modifying task settings  Initiate the test  Monitor the test in progress  Analyze the results The testing process is illustrated in Figure 3-1. The following sections of this document describe the detailed procedures used in working with Nomad. Run the TestDefine the TestOpen the ProgramSpecify Nomad SettingsAssign Tasks to ChannelsAnalyze the ResultsModify Task SettingsMonitor the Test in ProgressInitiate the Test Figure 3-1 - Nomad Application Workflow
Nomad User’s Manual Chapter 4 – Specifying Nomad Settings Copyright © Spirent Communications, Inc. 2013 18 4 Specifying Nomad Settings Prior to first use, Nomad must be configured with user preferences. Configuring the program consists of:  Specifying General Settings  Specifying GPS / Time Settings  Specifying Call Performance Settings  Specifying Voice Quality Settings  Specifying Email Campaign Settings Each of these configuration options is accessible from the Nomad Settings tab. Figure 4-1 – Locating the Nomad Settings Tab
Nomad User’s Manual Chapter 4 – Specifying Nomad Settings Copyright © Spirent Communications, Inc. 2013 19 4.1 Specifying General Settings To specify general settings for Nomad on the General settings tab:  Choose whether to Check for updates on application startup to determine whether a newer version of Nomad is available (recommended).  Choose whether to Check for new utilities on application startup to check for newer versions of installed utilities and to find new utilities (recommended).  Choose whether to Output local time when generating reports. If this option is left unchecked, time will be listed in UTC in Nomad output reports.  The Enable Remote Control via TCP socket 7072 option will be visible only to customers who have purchased this optional command set for controlling Nomad remotely via any Socket Client. This option should be selected in order to access Remote Control functionality. Remote functionality includes: o Time synchronization o Specification of output file name and location o Starting / stopping a recording session o Displaying MOS results on a remote client  The Require Hardware Identifiers option, when checked, will prevent launching a test if the Hardware Identifier field on any enabled channel’s Device Info dialog is not filled in. Please contact a Spirent representative if you are interested in learning more about Nomad Remote Control. Figure 4-2 - General Settings
Nomad User’s Manual Chapter 4 – Specifying Nomad Settings Copyright © Spirent Communications, Inc. 2013 20 4.2 Specifying GPS/Time Settings Specify settings related to time synchronization and GPS on the GPS/Time tab. The Automatic Time Synchronization and GPS areas are discussed in detail below. Figure 4-3 - GPS/Time Settings 4.2.1 Time Synchronization The successful use of the Nomad system is strictly dependent on consistent timing between the test laptop and the Audio or Call Server. Time synchronization is critical in the case of a Voice Quality or Mobile Terminated Call Performance campaign. To specify time synchronization options:  Select the time synchronization Source. The time synchronization source is used as the method of automatic time synchronization and the first method attempted in the initial time synchronization performed prior to testing. o Internet Time Server – Nomad will synchronize the PC’s clock to internet time on demand. In order to use this method, you must have a dedicated internet connection.  Note: Spirent does not recommend using this method for Voice Quality testing performed while drive testing. Substantial latency may be introduced when the internet time protocol operates in a mobile environment. If possible, use GPS synchronization in this case. o GPS – Nomad will synchronize to the appropriate server using GPS time. Spirent recommends using this time synchronization method when GPS is available, particularly in the case of Voice Quality tests performed while drive testing.
Nomad User’s Manual Chapter 4 – Specifying Nomad Settings Copyright © Spirent Communications, Inc. 2013 21  Check Enable automatic time synchronization to prevent time drift in synchronization between the test laptop and the Voice or Call Server. This option requires that you have access to the time synchronization source (internet or GPS) during testing.  Set the period between time synchronization updates in minutes. Spirent recommends setting this to 5 minutes for frequent updates. Note that time synchronization during testing has no impact on data collection or performance. 4.2.2 GPS Settings Nomad can record GPS information during testing. Location information may be used to display data testing results graphically in MapInfoTM and similar mapping applications (see Section 8.4). GPS may also be used as the Time Synchronization method as described in Section 4.2.1. To capture GPS information with Nomad:  Connect the GPS device and enable the device to acquire a signal.  In the GPS area of the GPS/Time dialog, confirm that the Enable GPS box is selected.  If desired, enable the Audible alarm on failure of the GPS.  Select the GPS Type as Garmin or Serial NMEA0183 (for DeLorme or any device that supports this standard). For a serial device, select the corresponding COM Port. If available, live GPS data will be displayed here:  The current latitude and longitude are displayed.  The GPS FIX status is given by the colored circle in this area. Hover over the circle for the status tooltip. The status is provided as NO FIX, 2-D DIFF, 3-D or 3-D DIFF. DIFF status refers to a lock on differential GPS sensors, providing a slightly more accurate location reading than standard GPS.  The blinking heart indicates that Nomad is receiving messages from the GPS. The heart blinks each time a GPS message is received.  The yellow triangle indicates that an error has occurred. Click the Restart GPS button to activate (in some cases it may be necessary to click this button twice).
Nomad User’s Manual Chapter 4 – Specifying Nomad Settings Copyright © Spirent Communications, Inc. 2013 22 4.3 Specifying Call Performance Settings Specify settings related to Call Performance testing on the Call Performance tab. The different areas of this tab are discussed in detail below. Figure 4-4 - Call Performance Settings 4.3.1 Call Server In the Call Server area, select the geographically relevant Call Server. By dialing and receiving calls, and by collecting and storing call record information, the Call Server facilitates both Mobile Originated and Mobile Terminated call performance sequences. The Allow client override of call server results option was implemented for a small group of international customers facing an unreliable landline infrastructure. This option should be left unchecked unless otherwise instructed by the Spirent Product Support Team. Figure 4-5 - Call Server Area
Nomad User’s Manual Chapter 4 – Specifying Nomad Settings Copyright © Spirent Communications, Inc. 2013 23 4.3.2 Logging Session Auto-Stop Spirent provides the ability to automatically stop the entire logging session if the number of consecutive Bluetooth connection errors on any test device exceeds the provided threshold. This check ensures that a user does not conduct an extended test using a non-functioning device, possibly invalidating results for the entire test array. To configure Logging Session Auto-Stop:  Choose whether to enable this functionality (recommended).  Set the Threshold for the number of consecutive Bluetooth errors which constitutes stopping a test. Figure 4-6 - Logging Session Auto-Stop Area 4.3.3 Audio Verification Nomad provides Audio Verification data for Call Performance tasks by determining the correlation value of audio sent to the Call Server from the test mobile and audio echoed back. The correlation coefficient is a threshold at which audio will be considered matching. This number should remain unchanged, unless directed by Spirent. Figure 4-7 - Audio Verification Area 4.3.4 CDR Lookup In the CDR Lookup area, select the part of the dialed number (from the viewpoint of the call server that Nomad is calling) that should be excluded when Nomad contacts the call server to retrieve Call Detail Records. This is an advanced setting that can be used when the Call Performance testing is performed in one country and the call server resides in another – the Caller ID of the device that is calling the call server can under certain circumstances look different than the number that was dialed. The CDR Lookup setting can be used to indicate that prefixes (international dialing prefixes or similar) should be excluded when matching telephone numbers for uplink score retrieval.
Nomad User’s Manual Chapter 4 – Specifying Nomad Settings Copyright © Spirent Communications, Inc. 2013 24 Figure 4-8 – CDR Lookup area
Nomad User’s Manual Chapter 4 – Specifying Nomad Settings Copyright © Spirent Communications, Inc. 2013 25 4.3.5 Call Result Warnings Select whether or not to Enable call result warnings and to Enable audible alarms for these warnings. Warnings will occur when the consecutive number of events exceeds the user-defined threshold for each of these event types: Call Outcome Definition Failed Attempt  A Mobile Originated Call was placed and an outgoing call was established followed by the phone’s return to the call placement state. No Service  There was no service available when the call was attempted. Access Timeout  Mobile Originated Call – A call was placed but no state change occurred before the Access Timeout time.  Mobile Terminated Call – The connection did not take place before the Access Time. Dropped Call  The call was ended before the expected duration had elapsed. Physical Link Error  The Bluetooth connection between the test mobile and the Nomad hardware unit was not active at the time a call was supposed to occur. Logical Device Error  The test mobile failed to respond properly to a command issued by Nomad via the Bluetooth connection. BC Error (BlueCore Error)  The Bluetooth module was not in a nominal state at the time a call was supposed to occur. Figure 4-9 - Call Result Warnings Area
Nomad User’s Manual Chapter 4 – Specifying Nomad Settings Copyright © Spirent Communications, Inc. 2013 26 4.4 Specifying Voice Quality Settings Specify settings related to Voice Quality testing on the Voice Quality tab. The different areas of this tab are discussed in detail below. Figure 4-10 - Voice Quality Settings 4.4.1 MOS Thresholds Use the settings in the MOS Thresholds area to customize data views according to local performance criteria:
Nomad User’s Manual Chapter 4 – Specifying Nomad Settings Copyright © Spirent Communications, Inc. 2013 27 Figure 4-11 - MOS Thresholds Area
Nomad User’s Manual Chapter 4 – Specifying Nomad Settings Copyright © Spirent Communications, Inc. 2013 28  Use the Color Bar to set the thresholds that determine Great (green), Good (yellow) and Bad (red) voice quality performance.  Check Collect X random WAV sample(s) from each MOS range above at the end of each session to set aside a random sample of audio files within each MOS performance range. This allows the audible verification of Great, Good and Bad audio samples at the conclusion of the test. See Section 8.5 for detailed information on collecting random audio sample files.  Set the Low MOS score alert threshold to a MOS value between 0 and 4.5. When a MOS reading falls below this threshold, Nomad will present either a Visual Notification or Visual & Audio Notification to the user. Effectively disable this functionality by setting the Low MOS score alert threshold to 0. 4.4.2 Events Events that occur during a test session complement voice quality to provide an overall indication of handset performance. Nomad provides the ability to note events of interest on both a per-channel and per-cycle basis during voice quality testing. Nomad also provides the ability to automatically detect a sequence of null MOS values, often indicative of a dropped call.  Select Enable Event Detection to automatically detect and log null MOS events.  Select the number of consecutive null MOS scores that constitute a Null MOS Event and customize the label for this event.  Select the Edit Event Classifications button to specify custom Event Classifications by event names. These events must be manually recorded in the Events tab of the MOS Trends Window upon occurrence during testing. See Section 7.2 for more information about event logging and event detection during Voice Quality testing. Figure 4-12 - Events Area 4.4.3 Bluetooth Auto-Dial Check Enable auto-dial on Bluetooth audio channels when starting session to allow Nomad to automatically dial the audio server for channels connected using Bluetooth. Spirent recommends that international users dialing a non-US audio server should disable this option as local dialing conventions may be best handled manually. Figure 4-13 - Bluetooth Auto-Dial Area
Nomad User’s Manual Chapter 4 – Specifying Nomad Settings Copyright © Spirent Communications, Inc. 2013 29 4.4.4 Audio Server The Audio Server facilitates voice quality testing by answering calls, sending and recording .WAV files, storing the recorded .WAV files and handling uplink data collection. The Audio Server is an extremely high quality Interactive Voice Response (IVR) service provided by Spirent Communications to answer calls, send and record speech, and store the recorded samples for access to uplink results. In this area:  Select the geographically relevant Audio Server.  Enter your Audio Server Username and Password.  Validate Settings to ensure that the settings are correct for access throughout the test process. Figure 4-14 - Audio Server Area 4.4.5 MOS Correction Invalid MOS scores, such as those obtained by comparing an empty .WAV file to a reference file, may drastically skew performance statistics. Invalid MOS values may also be obtained when DTMF (push-button) tones are audible in the recorded speech sample. Selecting Enable MOS Correction prevents invalid MOS scores from being reported. This option is enabled by default. Figure 4-15 - MOS Correction Area 4.4.6 IMS Setup This setting is currently inapplicable to most users. It is utilized for configuring a local IMS client that will act as the server side in Voice Quality testing. For users configuring the local IMS client, this field should point to the boghe-remote.dll that is located on the Nomad system. This feature is part of the CS8 Nomad lab integration. For more information please contact your Spirent Sales representative.
Nomad User’s Manual Chapter 4 – Specifying Nomad Settings Copyright © Spirent Communications, Inc. 2013 30 4.4.7 Default Scoring Algorithm This setting controls the default scoring algorithm for each new voice quality task added to a Nomad channel. Either PESQ or POLQA may be selected. See Appendix E for an introduction to POLQA including a comparison between PESQ and POLQA. The default scoring algorithm may be overridden for any individual task when modifying the task settings. Figure 4-16 - Default Scoring Algorithm Area 4.5 Specifying Email Campaign Settings An e-mail campaign may be launched during a Voice Quality or Call Performance task to test Multi-RAB performance. An e-mail campaign tests whether e-mail sent to a phone during a call disrupts performance as compared to phones not receiving e-mail. Email campaign settings are specified from the Email Campaigns tab. Figure 4-17 - Email Campaign Settings
Nomad User’s Manual Chapter 4 – Specifying Nomad Settings Copyright © Spirent Communications, Inc. 2013 31  Check the Override default email content box to specify the content of the body of the emails to be sent to the phone during an email campaign. If this box is left unchecked, Spirent will automatically generate the content for the messages sent in the email campaign.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 32 5 Defining a Test Nomad provides testing flexibility with a fully configurable Test Setup utility. Users may customize the type of task to be performed on each test channel (per availability based on licensing) as well as task details. When defining a test it is necessary to:  Assign tasks to channels  Modify task settings  Save test settings All test definition options are available from the Test Setup tab and are discussed in the following sections. This chapter provides an introduction to the Nomad Test Setup interface. For instructions on configuring specific types of tests, see Chapter 9 – Voice Quality Configuration Options and Chapter 10 – Call Performance Configuration Options. Figure 5-1 - Test Setup Tab
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 33 5.1 Assigning Tasks to Channels Three different methods are available for assigning a new task to a channel: 1. Drag and drop any Voice Quality Task or Call Performance Task onto an available Test Channel. 2. Right-click on an individual channel and select Add Task. Then select the type of task to add from the sub-menu. 3. Right-click on a task name and select the channel(s) to apply to. Repeat the selected process for all channels to be active during testing. Figure 5-2 - Dragging a Task to a Test Channel The task types available for selection are arranged on three different tabs:  Voice Quality  Call Performance  Delay Once a task has been assigned to each desired channel, continue to Section 5.2 to modify the task settings.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 34 5.1.1 Voice Quality Tasks The following voice quality task types are available for selection: Figure 5-3 - Voice Quality Task Types  Mobile o The test device on a Mobile channel serves as the downlink device in the mobile communication path. This option is used for:  Standard bi-directional testing using the Audio Server. (See Section 9.2)  One of the test handsets in a bi-directional Mobile-to-Mobile test scenario (the other handset will be set as a Base). (See Section 9.3)  One of the test handsets in a bi-directional Remote Unit test scenario (the other handset will be set as a Base). (See Section 9.4)  A mobile acting as the downlink device in a test using the Landline Module. (See Section 9.5)  Base o The test device on a Base channel serves as the uplink device in the mobile communication path. This option should be selected for:  One of the test handsets in a bi-directional Mobile-to-Mobile test scenario (the other handset will be set as a Mobile). (See Section 9.3)  One of the test handsets in a bi-directional Remote Unit test scenario (the other handset will be set as a Mobile). (See Section 9.4)  The uplink end of a Landline Module test. (See Section 9.5)  Source Only o Select this option to perform uplink testing only. With this setting, Nomad will send speech every ten seconds but will not record anything. This setting can be used to shorten the test cycle time in the event that only uplink data is required.  Record Only o Select this option to perform downlink testing only. With this setting, Nomad will record speech every ten seconds but will not source speech. This setting can be used to shorten the test cycle time in the event that only downlink data is required. 5.1.2 Call Performance Tasks The following call performance task types are available for selection: Figure 5-4 - Call Performance Task Types
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 35  Mobile Originated o The mobile test device makes calls to the Call Server for call control testing. (See Section 10.1)  Mobile Terminated o The Call Server makes calls to the test mobile for call control testing. (See Section 10.2)  Idle o The test device makes no calls. This type of task only reports changes in signal strength and battery level.  Mobile to Mobile Originate o This device makes calls to a receiving test device on another channel or on a remote unit. This option allows for call control testing without a call server. (See Section 10.3)  Mobile to Mobile Receive o This device receives calls from an originating test device on another channel or on a remote unit. This option allows for call control testing without a call server. (See Section 10.3) 5.1.3 Delay Task One task type is available on this tab. Figure 5-5 - Delay Task Type  Voice Delay o This task allows for precise measurement of speech delay between two mobile devices connected to the same Spirent ME unit. The measurement includes delay introduced by both handsets and the time it takes to traverse the network. (See Section 9.10)
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 36 5.2 Modifying Task Settings Each assigned task must be configured with specific task settings. Click any available channel to modify its configuration. Different types of settings area available for different task types. Each Task Settings dialog is detailed in the sections below, including minor differences in content for different task types:  Device Info  Session  Channel Settings  Call Campaign  Email Campaign  Auto-Level Assist  Bluetooth Connection  Reconnect All Bluetooth Channels 5.2.1 Device Info The fields in the Device Info dialog provide optional metadata for the session log file and help identify the test devices in session outputs. All fields in this dialog are optional:  Make – The test device manufacturer.  Model – The test device model name.  Role – The role this device plays in the current test (i.e. “Reference”, “DUT”, etc.).  Hardware Identifier – The device’s serial number (i.e. “ESN”, “IMEI”, etc.).  Adapter – If an adapter was needed to interface to Nomad, note the model used here. Figure 5-6 - Device Info Dialog
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 37 5.2.2 Session The fields in the Session dialog specify connection settings for a Voice Quality Task. This dialog is not used for Call Performance Tasks. Some settings available in this dialog depend on the type of Voice Quality Task to be performed: Figure 5-7 - Session Dialog  Device Selection Area – Any Nomad Voice Quality Task is dependent on the transmission of speech between two communication nodes. In each task type, one node will be the mobile device connected to the Spirent ME hardware unit. The other end of the communication path may be selected for the task at hand: o Uplink Device – In a Mobile task, the test device serves as the downlink device in the mobile communication path. Select the Uplink Device to handle uplink data collection and downlink audio sourcing:  IMS Client – Testing utilizing a local IMS Client as the uplink device. This feature is part of the CS8 Nomad lab integration. For more information please contact your Spirent Sales representative. (See section for 5.2.2.1 more information)  Audio Server – For all Audio Server tests.  Landline Module - For all Landline Module tests.  Base Task on Remote Unit – Use this option to configure bi-directional Remote Unit testing. In this test case, the mobile on one Spirent ME unit is configured with a Mobile task while the mobile on the other unit is configured with a Base task.  Base Task on Channel X – Use this option to configure bi-directional Mobile-to-Mobile testing. In this test case, one channel is configured as a Mobile task while the other is configured as a Base task.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 38 o Downlink Device – In a Base task, the test device serves as the uplink device in the mobile communication path. Select the Downlink Device to handle downlink data collection and uplink audio sourcing.  Mobile Task on Remote Unit – Use this option to configure bi-directional Remote Unit testing. In this test case, the mobile on one Spirent ME unit is configured with a Mobile task while the mobile on the other unit is configured with a Base task.  Mobile Task on Channel X – Use this option to configure bi-directional Mobile-to-Mobile testing. In this test case, one channel is configured as a Mobile task while the other is configured as a Base task. o Recording Device – In a Source Only task, only uplink testing is performed for that channel. Select the Recording Device to capture uplink data from the test mobile.  Record Only task on remote unit – Use this option to configure uni-directional Remote Unit testing. In this test case, the mobile on one Spirent ME unit is configured with a Source Only task while the other is configured with a Record Only task.  Record Only task on channel X – Use this option to configure unidirectional Mobile-to-Mobile testing. In this test case, one channel is configured as a Source Only task while the other is configured as Record Only. o Sourcing Device – In a Record Only task, only downlink testing is performed for that channel. Select the Sourcing Device to source speech to the test mobile.  Source Only task on remote unit - Use this option to configure uni-directional Remote Unit testing. In this test case, the mobile on one Spirent ME unit is configured with a Source Only task while the other is configured with a Record Only task.  Source Only task on channel X – Use this option to configure unidirectional mobile-to-mobile testing. In this test case, one channel is configured as a Source Only task while the other is configured as Record Only.  Phone Number Settings – This area appears in the Session dialog only for Mobile tasks. When the uplink device is set to Audio Server, the Number to call and Number calling from fields are mandatory. Although these fields may be entered manually, Spirent recommends using the Connection Assistant for highest accuracy. To use the Connection Assistant: o Click the Connection Assistant button to start the utility. o Click the Edit Audio Server Settings button. In the Edit Audio Server Settings dialog, select the Audio Server Name and enter the Username and Password. Click OK to accept the changes.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 39 Figure 5-8 – Edit Audio Server Settings Dialog o Dial the Audio Server using the handset to be connected to this channel. The handset does not need to be connected at this time. o All active calls to the Audio Server will be represented in the Connection Assistant drop-down window using the {4 Digit DNIS}-{10 Digit ANI} convention:  {4 Digit DNIS} –The last four digits of the Audio Server phone number called by the mobile.  {10 Digit ANI} – The phone number of the mobile being tested. o If the test handset does not appear in the list, use the Refresh button to update. It may take up to one minute for the handset to appear. o Use the drop-down to select the correct number combination from the list. o Click OK. Figure 5-9 - Connection Assistant o The Number to call and Number calling from fields will be populated based on your selection, removing the risk of manually entering this information incorrectly. o Note: Spirent strongly recommends that international users outside of North America use the Connection Assistant to ensure that phone numbers are entered in the correct format.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 40 Figure 5-10 – Phone Number Settings Populated by the Connection Assistant  Reference file o This setting controls the reference file to be used for this voice quality task. This is an advanced setting which, when changed from the default value, will not work with any of Spirent’s default call servers, only when a corresponding ‘base’ task hase been setup up on a second Nomad unit.  Session Ends – In this area, select the method for ending the test session: o Manually – The user ends the test session by pressing the Stop button. o After X Cycles – The session ends after the system has run through X sourcing and recording cycles. Figure 5-11 – Reference file Settings 5.2.2.1 IMS Client Settings When selecting IMS Client as the Uplink Device, the options for configuring the IMS Client will appear along with some of the normal session options described above.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 41 Figure 5-12 – Reference file Settings  IMS Server Information – The following fields are related to the server on which the Device Under Test is registered. For more information on these settings, see the configuration of the IMS Client: o IMS Server Address o IMS Server Port o Public Identity o Private Identity o Password o Realm o IP Version o Transport – TCP or UDP o Codecs – Select the codecs that you would like as options for the negotiated communication between the server and the client. 5.2.3 Channel Settings The fields in the Channel Settings dialog specify connection settings for a Voice Quality Task. This dialog is not used for Call Performance Tasks. Figure 5-13 - Channel Settings Dialog  Audio interface for this channel – A Voice Quality test mobile may be connected to the o Spirent ME hardware unit using either the Analog Interface (audio cable) or Bluetooth Interface (see Section 5.2.7 for Bluetooth connection instructions) o Spirent HD ME hardware unit using either Narrowband (audio cable on the NBx interface), High Definition (audio cable on the HDx interface) or Bluetooth Interface (see Section 5.2.7 for Bluetooth connection instructions). Spirent HD ME hardware, for non-Bluetooth tests, has the ability to run tests with different
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 42 sampling rates per channel pairs. The channel pairs are 1&2 and 3&4. Thus, if channel 1 is set to Narrowband, channel 2 cannot be set to High Definition.  Sampling Rate – The Sampling rate for audio associated with this channel.  Scoring Model – The MOS scoring algorithm used to score audio. When the Audio interface is set to High Definition, Channels 1 and 2 must be set to the same scoring algorithm, and channels 3 and 4 must be set to the same scoring algorithm. Depending on the licensing and hardware combination, the following scoring/audio interface combinations are possible: o PESQ Scoring Algorithm  For Nomad ME: The Narrowband, Wideband and Bluetooth options are available.  For Nomad HD ME: The Narrowband option is available through the NBx port and over the air through the Bluetooth interface. o POLQA Scoring algorithm  For Nomad ME: The Narrowband and Wideband options are available.  For Nomad HD ME: The Narrowband option is available through the NBx port, the High Definition option is available through the HDx port and over the air through the Bluetooth interface.  Channel X – The Channel Audio area is used to manually adjust volume settings. If the Input Level (downlink volume) and Output Level (uplink volume) are known for the device on this test channel, use the sliders to set those values here. Otherwise, see Section 5.2.5 to automatically optimize volume settings prior to testing or Section 7.2 to manually adjust volume settings during testing.  Microphone Detect Mode – Enable this mode to support any handset requiring recognition of an external audio input.  Enable Bluetooth auto redial – This option allows Nomad to automatically redial the last number called in the case of a dropped connection. o In the case of a Bluetooth connection:  Auto redial is available for all devices.  Set the Audio Interface to Bluetooth.  Check the Enable Bluetooth audio redial box and specify the number of seconds to pause before redialing.  With this configuration, a call that is dropped during testing will be automatically redialed. o In the case of an analog connection:  Auto redial is available for phones which support the following options:  Ability to maintain an analog and Bluetooth connection simultaneously.  Ability to transfer from a Bluetooth connection to an analog connection.  Ability for the Bluetooth connection to be re-established while the phone is on the call.  To utilize auto redial with a qualified device:  Connect the test handset to the Spirent ME hardware using Bluetooth (see Section 5.2.7 for Bluetooth connection instructions)  Next, attach the test handset to the same channel using an analog connection.  Verify that the Bluetooth connection is still active.  Set the Audio interface for this channel as Analog.  Check the Enable Bluetooth auto redial option.  Specify the number of seconds to Pause before redialing.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 43  With this configuration, a call that is dropped during testing will be redialed using the Bluetooth connection. After three cycles of silence, Nomad will then issue a command to transfer the audio from the Bluetooth to the analog connection.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 44 5.2.4 Call Campaign The fields in the Call Campaign dialog are used to specify duration and sequencing of a Call Performance Task. This dialog is not used for Voice Quality Tasks. All of the fields in this dialog must be completed. Figure 5-14 - Call Campaign Dialog (Mobile Originated Task)  Enter the Phone Number Settings. o For a Mobile Originated call campaign, enter the Number to call as the call server number provided by Spirent Communications to be dialed by the mobile, as well as the Number calling from. o For a Mobile Terminated call campaign, enter the Number to dial as the mobile phone number to be dialed by the Call Server. o For a Mobile to Mobile Originate task, use the drop-down to select the location of the Receiving Device. Enter the Number to dial for the receiving device. o For a Mobile to Mobile Receive task, use the drop-down to select the location of the Calling Device. Enter the Calling number of the originating device.  Select the Call Initiation Mode. The Call Initiation Mode is applicable only when the same Call Performance Task is applied to multiple channels during a test. This option does not apply to a Mobile-to-Mobile Call Performance task. o Synchronous Testing – Calls start simultaneously on all devices regardless of call outcomes. For example, if one device drops a call, it will remain idle until the next time all mobiles are scheduled to start a new call. Note that Synchronous Testing may not be maintained for Mobile Terminated call campaigns. o Asynchronous Testing – Each device follows its own call sequence without regard to other device progress. In this mode, if one device drops a call, it will wait for the specified time and then start a new call, even as the other devices continue their first call.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 45  Specify the values for the task parameters. In the Mobile-to-Mobile case, these parameters will be set in the Mobile-to-Mobile Originate task. o Access Timeout – A call attempt that has not connected within this amount of time will be classified as an Access Timeout event. o Duration – The length of each call in the test sequence.  In a Synchronous campaign, this refers to the total attempt duration including access time and connected time.  In an Asynchronous campaign, this refers to the connected time only. o Wait Time – The amount of idle time between the end of one call in the sequence and the start of the next call. o Attempts – The number of calls to attempt in this task sequence.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 46 5.2.5 Auto-Level Assist The Auto-Level Assist Wizard provides an automatic method for determining optimal downlink and uplink volume for Voice Quality Tasks running on mobiles connected via the analog audio interface. The Auto-Level Assist Wizard has no bearing on Call Performance Tasks or on Voice Quality Tasks using a Bluetooth connection. The Auto-Level Assist procedure bypasses the need for manual adjustments in determining the optimal input level (downlink volume) and output level (uplink volume) for testing. Before beginning the Auto-Level Assist process, please take note of these guidelines recommended by and followed by Spirent staff:  Confirm that the test mobile is in a good coverage area.  Confirm the presence of a high-speed internet connection required for auto-calibration with the Auto-Level Assist Wizard. To run the Auto-Level Assist Wizard:  From the Test Setup tab, select the Auto Level Assist button found under any Voice Quality task using the Analog Audio Interface (see Section 5.2.3 to select the Channel Audio Interface).  Confirm that the Audio Server settings are correct using the Validate Settings button. It is not possible to continue beyond this screen until the Audio Server settings have been validated. Figure 5-15 - Auto-Level Assist Audio Server Validation  Click Next to continue.  Confirm that the Handset Phone # and Audio Server # are correct for all channels to be optimized. Use the Connection Assistant for the highest accuracy when entering the phone numbers (see Section 5.2.2 for more information on the Connection Assistant).
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 47 o Note: Spirent strongly recommends that international users outside of North America use the Connection Assistant to ensure that phone numbers are entered in the correct format.  Click Next to continue. Figure 5-16 - Auto-Level Assist Phone Number Validation  You will be asked whether you wish to check for uplink connectivity to verify communication with the server. Click Yes to verify.  Place the calls on the mobile handsets and connect the handsets to the Spirent ME hardware using the audio cables. See Section 6.3 for instructions on placing calls.  Click Next to continue. Figure 5-17 - Auto-Level Assist Prompt to Check for Uplink
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 48  The system will make three attempts to contact the Audio Server. When connectivity is verified, the level assist automation will begin. Figure 5-18 - Auto-Level Assist Audio Server Connectivity Verification  The auto-calibration process will run for several minutes displaying MOS progress as volume is adjusted. Slightly different algorithms are used to determine optimal downlink and uplink volume: o The downlink algorithm closely follows Spirent’s recommendation for manual volume adjustment (see Section 7.1.1.2). The algorithm increases the input level until the Overdrive condition is detected. From there, the input level is decreased for several cycles. The level producing the best MOS result from the ramp-down procedure is stored. o The uplink algorithm uses a form of binary search logic. The algorithm finds the lowest output volume producing a valid uplink score. The volume is then adjusted to the highest nominal operating level. The algorithm then uses combinations of average level to zero in on the level producing the best MOS result. Figure 5-19 - Auto-Level Assist Automation
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 49  After the Auto-Level Assist Wizard has run its course, the Results Review dialog will appear. Examine the downlink and uplink results for each channel. Select whether to use the saved level setting or the Level Assist recommendation.  Click Next to store selections. Figure 5-20 - Auto-Level Assist Results Review The Auto-Level Assist process should be repeated if, during testing, the input and output levels do not provide:  Maximum MOS values and/or  Downlink and uplink insertion gain values corresponding to Spirent’s recommendations (downlink gain close to 0, uplink gain between -12 and -3 dB) If the Auto-Level Assist process is unable to provide appropriate results, the input and output volume settings should be manually overridden during testing using the procedure described in Section 7.2.1.2.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 50 5.2.6 Email Campaign An Email Campaign may be launched during a Voice Quality or Call Performance task to test Multi-RAB performance. An Email Campaign tests whether email sent to the phone during a call disrupts performance as compared to phones not receiving email. An email campaign may be launched from the Email Campaign dialog. Figure 5-21 - Email Campaign Dialog  If an email campaign is desired during testing, select the Initiate email campaign when logging starts option.  Enter an email address accessible on the test mobile in the Recipient email address (To:) field.  Enter the # of emails to send to the mobile.  Enter the Interval between emails in seconds.  If desired, Generate Test Email to be sent to the mobile device. Note: In the event that Nomad becomes inaccessible during an Email Campaign (i.e. due to PC crash, etc.), it is possible to stop email messages from being sent to the handset(s). Simply reply to any message generated by the Email Campaign to stop unwanted messages from being sent to the phone.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 51 5.2.7 Bluetooth Connection Use the Bluetooth Connection screen to perform pairing of the test device with the Spirent ME hardware unit. The Bluetooth Connection dialog will be available for any Voice Quality Task with the Audio Interface set to Bluetooth. The Bluetooth Connection dialog will be available for all Call Performance Tasks. Figure 5-22 - Bluetooth Connection Dialog To pair a device:  Click the Start Pairing button. The corresponding LED will flash blue on the front of the hardware unit.  Set the mobile to search for Bluetooth devices. This procedure varies among devices, but the option is generally found under the Settings or similar menu.  The mobile may detect multiple Nomad channels, but will only be pairable with the channel that is in pairing mode. Select the appropriate channel based on the Nomad Hardware Name field.  When prompted for the Bluetooth passkey, enter 0000.  When pairing is complete, the hardware will display a solid blue LED at the paired channel. The software will show the paired channel as CONNECTED. o Note: The connection process must be handled manually for some handsets. If the blue light does not appear on the Spirent ME hardware unit and the software does not show the CONNECTED state, the phone must be connected manually. This is typically accomplished through the phone’s Bluetooth menu system. Note that if a device has previously been paired with this Nomad channel, the above procedure may be bypassed by selecting the Connect To Last Handset option. Selecting this option will automatically connect the channel to the last device to which it was paired. To reconnect all previously connected Bluetooth channels to the last paired handset, used the Reconnect All Bluetooth Channels button found on the Test Setup tab. The CLI area at the bottom of the Bluetooth Connection screen displays Bluetooth log messages. The command line below the message viewer is used for advanced diagnostic purposes only. Please use this area only if directed by a Spirent representative.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 52 5.2.8 Reconnect All Bluetooth Channels The Reconnect All Bluetooth Channels button will be available for voice quality tasks with the Audio Interface set to Bluetooth and for all Call Performance tasks. Click this button to reconnect all previously connected Bluetooth channels to the last paired handset.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 53 5.3 Saving Test Settings Once tasks have been assigned to all test channels and settings for all tasks have been configured to meet testing requirements, it may be desirable to save these settings for future use. The two options available for saving task settings are:  Configured Tasks  Session Profiles All options for working with Configured Tasks and Session Profiles are available using the buttons in the lower right-hand corner of the Test Setup screen. Figure 5-23 - Locating the Nomad File Options
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 54 5.3.1 Working with Configured Tasks A Configured Task saves all of the settings for the currently selected task. A Configured Task may then be applied to any test channel in the current or future tests. This option is useful for replicating a single task across multiple test channels. To save a Configured Task:  Configure all task settings as desired.  Select the Save As Configured Task option on the Test Setup tab.  Name the Configured Task and then Save. Figure 5-24 - Save Configured Task Dialog To apply a Configured Task to a test channel:  Click the Load Configured Task button on the Test Setup tab.  Click to select any Configured Task shown. Figure 5-25 - Select a Task from the List
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 55  Two methods are available to apply the task to any specific channel or All Channels: 1. Use the Apply selected task to: drop-down at the bottom of the dialog. Then click OK to confirm the selection. 2. Right-click on the selected task and select Apply To and the channel number. The Configured Tasks dialog will close automatically. Figure 5-26 - Applying the Selected Task A Configured Task may be exported as an XML file to share with other users. To export a Configured Task, click the Export Selected Tasks button in the Configured Tasks dialog. The saved XML may be imported into Nomad using the Import Tasks button in the Configured Tasks dialog. A task may be removed from a channel using the Remove Current Task button on the Test Setup tab.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 56 5.3.2 Working with Profiles A Session Profile defines the tasks and settings for all available test channels. Unlike a Configured Task which defines a task type and settings for an individual task, a Session Profile defines the task type and settings applied to each test channel. To save a Session Profile:  Configure all desired test channels with tasks and settings to meet testing requirements.  Click the Save Session Profile button on the Test Setup tab.  Name the Session Profile.  Choose whether to include: o All global settings included in the Settings section o Channel metadata (make, model, etc.) o Number to dial and number dialing from Figure 5-27 - Save Session Profile As Dialog To assign tasks to channels by loading a Session Profile:  Click the Load Session Profile button on the Test Setup tab.  Select any Saved Session Profile shown.  Choose whether to load the channel metadata and number to dial / number dialing from that was saved with the profile.
Nomad User’s Manual Chapter 5 – Defining a Test Copyright © Spirent Communications, Inc. 2013 57 Figure 5-28 - Select a Saved Session Profile  Click the Load Selected Session Profile button in the bottom right corner of the dialog. Each test channel configured in the Session Profile will be populated on the Test Setup tab. A Session Profile may be exported as an XML file in order to share with other users using the Export Session Profile button in the Saved Session Profiles dialog. The saved XML file may be imported into Nomad using the Import Session Profile button in the Saved Session Profiles dialog. Remove the tasks from all channels to build a new Session Profile from scratch using the New Session Profile button.
Nomad User’s Manual Chapter 6 – Initiating a Test Copyright © Spirent Communications, Inc. 2013 58 6 Initiating a Test Once a task has been assigned to each Nomad test channel and task settings have been configured, a test session may be initiated. To start a test:  Click the Start Logging Session button on the Test Setup tab. The Start Logging Session Wizard will appear. The following sections describe each step in the Wizard. 6.1 Reviewing the Current Channel Configuration On the first screen of the Start Logging Session Wizard:  Confirm the Audio Configuration and Email Campaign Configuration settings for each test channel.  If any changes are required, press Cancel to exit the Wizard and return to the Test Setup tab. Otherwise, click Next to proceed to the next screen. Figure 6-1 - Reviewing the Current Channel Configuration 6.2 Device Setup If device pairing was not completed for any Bluetooth channel when task settings were modified, the pairing may be completed on the Bluetooth Module Device Control screen. This screen will not appear if all test devices use the Analog audio interface.  The options on this screen are identical to those in the task settings Bluetooth Connection dialog. See Section 5.2.7 for instructions on pairing each Bluetooth device with the Spirent ME hardware unit.  Click Next when all Bluetooth devices have been paired.
Nomad User’s Manual Chapter 6 – Initiating a Test Copyright © Spirent Communications, Inc. 2013 59 Figure 6-2 - Bluetooth Module Device Control 6.3 Placing Calls on Mobile Handsets At the conclusion of the Start Logging Wizard, Nomad will automatically place test calls for:  All devices running Call Performance tasks.  Devices running Voice Quality tasks which are connected via Bluetooth and which have been configured for auto-dial on the Settings → Voice Quality tab (see Section 4.4.3). All other calls must be manually placed to the Audio Server. To place each call:  Dial the handset using the Audio Server phone numbers supplied with Nomad.  Following the voice prompt, enter the Access PIN (if applicable).  Verify from the voice prompt that the session has started.  Connect the handset to the licensed channel audio cable (Channel 1, Channel 2, etc.) as appropriate.
Nomad User’s Manual Chapter 6 – Initiating a Test Copyright © Spirent Communications, Inc. 2013 60 6.4 Selecting the Log File Storage Location To specify the log file storage location and other session information:  In the File Path area use the Browse button to select a log file location and file name. This field is required.  Complete the User Name, Route/Location and Description fields as desired. This information serves as optional metadata written to the session log file.  Click Next to continue. Figure 6-3 - Specifying the Log File Storage Location and Session Information 6.5 Time Synchronization The successful use of the Nomad system is strictly dependent on consistent timing between the test laptop and the Audio or Call Server. Time synchronization is critical in the case of a Voice Quality or Mobile Terminated Call Performance campaign. Nomad will automatically attempt to synchronize the system clock to atomic time, the standard used by both the Audio and Call Server. Nomad will first attempt synchronization using the Source method specified in the GPS/Time area of the Settings tab. If unavailable, Nomad will attempt synchronization using the other method.
Nomad User’s Manual Chapter 6 – Initiating a Test Copyright © Spirent Communications, Inc. 2013 61 Figure 6-4 - Attempting to Synchronize Time If neither GPS nor internet connectivity is available, synchronization will fail. Although the Start Logging Session button becomes available on the Failed Synchronization screen, Spirent recommends troubleshooting GPS and/or internet connectivity and then retrying. Running a logging session without verifying time synchronization is not recommended and may result in invalid data. Figure 6-5 - Time Synchronization Failed Once time synchronization is complete, the logging session will begin automatically.
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 62 7 Monitoring a Test in Progress Topics of interest while a test is in progress include:  Nomad Test Window Orientation  Manual Volume Adjustment (Voice Quality Testing)  Event Logging and Event Detection (Voice Quality Testing)  Interpreting MOS Results (Voice Quality Testing) 7.1 Nomad Test Window Orientation Figure 7-1 shows a Nomad test in progress including both voice quality and call performance tasks: Figure 7-1 - Monitoring a Test in Progress Key areas of interest in the Nomad test window include: A. Menu Bar Figure 7-2 - Menu Bar
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 63 B. Current Time Indicator  The radio button selected here is used to control the time display during testing. o Select UTC to display data collection time in UTC (Coordinated Universal Time), also known as GMT (Greenwich Mean Time). o Select Local to display data collection time according to the test laptop’s local time.  The time format selected here is for display purposes only. The Output local time when generating reports checkbox on the Settings → General tab controls the time format for output reports. If this option is left unchecked, time will be listed in UTC in Nomad output reports. Figure 7-3 - Current Time Indicator C. GPS Information  If GPS is available, the current latitude and longitude are displayed.  The GPS FIX status is given by the colored circle in this area. Hover over the circle for the status tooltip. The status is provided as NO FIX, 2-D, 2-D DIFF, 3-D or 3-D DIFF. DIFF status refers to a lock on differential GPS sensors, providing a slightly more accurate location reading than standard GPS.  The blinking heart indicates that Nomad is receiving messages from the GPS. The heart blinks each time a GPS message is received.  The yellow triangle indicates that an error has occurred. Click the Restart GPS button once to activate GPS (in some cases it may be necessary to click this button twice). Figure 7-4 - GPS Information
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 64 D. Session Duration  The elapsed time since the start of the session is displayed here. Figure 7-5 - Session Duration E. Stop  Use this button to manually end a test session.  Note: Losing internet connectivity during a Mobile Terminated Call Performance campaign may cause the Stop button to cease working. In this case, wait for a call to come to the phone, answer the call and dial 9999. This will manually stop the Call Server from repeatedly calling the phone through the scheduled end of the test. Figure 7-6 - Stop Button F. Time Sync  This dialog allows the user to manually synchronize the test laptop’s clock with the Audio and/or Call Server.  To manually synchronize time: o Select the Time Sync button during testing. o Select the Time Synchronization Source. o Click the Sync Now button.  Check Enable automatic time synchronization to prevent time drift. Set the period between time synchronization updates in minutes. Spirent recommends setting this to 5 minutes for frequent updates. Note that time synchronization during testing has no impact on data collection or performance. Figure 7-7 - Time Sync Button
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 65 Figure 7-8 - Time Sync Dialog G. Settings Figure 7-9 - Settings Button  Levels – Use this tab to manually adjust uplink and downlink volume settings for each channel. See Section 7.2 for details on manual volume adjustment. Figure 7-10 - Levels Tab of Test Settings Dialog
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 66  Bluetooth – Use this tab to pair, adjust or view information for Bluetooth-connected test devices. See Section 5.2.7 for more information on using the Bluetooth dialog. Figure 7-11 - Bluetooth Tab of Test Settings Dialog  Log – This tab provides a log of system events during a Voice Quality task. Figure 7-12 - Log Tab of Test Settings Dialog H. Help  This option opens the Online Help file for Nomad. Figure 7-13 - Help Button
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 67 I. Call Performance Task Status Window (Mobile Originated Task) Figure 7-14 - Call Performance Task Status Window (Mobile Originated Task)  Call Performance test status items displayed in this area: o State – The state of the call, typically listed as Waiting, Connecting or Active Call. o Actual Wait Duration – The amount of time the phone has spent in Wait mode. o Connected Duration – The amount of time spent on the call. o Setup Time – The amount of time between the start of the call attempt and the call connection. o Wait Time Remaining – The amount of idle time until the next call attempt.  Call Performance indicators displayed in this area: o Device State Indicator - The device icon will appear when the mobile is actively attempting or connected to a call. The device tooltip provides the actual call state. o Bluetooth State Indicator - This indicator will appear in gray when Bluetooth is OFF, and in blue when Bluetooth is ON.  Note: If the Bluetooth connection drops during the test, Nomad will attempt to automatically reconnect. This feature is device dependent and only possible on phones which support automatic Bluetooth reconnection. Phones that do not support automatic reconnection will prompt the user to manually reconnect when Bluetooth is lost. Other models give the user the option to override manual reconnection and take advantage of Nomad’s automatic reconnection. o Signal Strength Indicator - The signal strength indicator mimics the mobile’s signal strength bar display. The mouse-over tooltip provides the number of bars reported. Note that signal strength reporting varies widely by device and is not supported by all device types. o Battery Indicator - The battery indicator mimics the mobile’s battery bar display. The mouse-over tooltip provides the number of bars reported. Note that battery performance reporting varies widely by device and is not supported by all device types. Charging the battery during testing may cause misleading battery performance results for some phones. o RSSI Indicator - This indicator appears as a gauge measuring the RSSI reported by the handset. The mouse-over tooltip provides the actual RSSI level. This feature is device-dependent and only appears for phones that support RSSI reporting.
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 68 J. Call Performance Task Status Window (Mobile Terminated Task) Figure 7-15 - Call Performance Task Status Window (Mobile Terminated Task)  Call Performance test status items displayed in this area: o State – The state of the call, typically listed as Waiting for Next Call (Idle), Ringing or Active Call o Connected Duration – The amount of time spent on the call.  Call Performance indicators displayed in this area: o Device State Indicator - The device icon will appear when the mobile is actively attempting or connected to a call. The device tooltip provides the actual call state. o Bluetooth State Indicator - This indicator will appear in gray when Bluetooth is OFF, and in blue when Bluetooth is ON.  Note: If the Bluetooth connection drops during the test, Nomad will attempt to automatically reconnect. This feature is device dependent and only possible on phones which support automatic Bluetooth reconnection. Phones that do not support automatic reconnection will prompt the user to manually reconnect when Bluetooth is lost. Other models give the user the option to override manual reconnection and take advantage of Nomad’s automatic reconnection. o Signal Strength Indicator - The signal strength indicator mimics the mobile’s signal strength bar display. The mouse-over tooltip provides the number of bars reported. Note that signal strength reporting varies widely by device and is not supported by all device types. o Battery Indicator - The battery indicator mimics the mobile’s battery bar display. The mouse-over tooltip provides the number of bars reported. Note that battery performance reporting varies widely by device and is not supported by all device types. Charging the battery during testing may cause misleading battery performance results for some phones. o RSSI Indicator - This indicator appears as a gauge measuring the RSSI reported by the handset. The mouse-over tooltip provides the actual RSSI level. This feature is device-dependent and only appears for phones that support RSSI reporting.
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 69 K. Call Performance Statistics Window  For Mobile Originated calls, this window displays the current test progress along with Call Performance statistics.  For Mobile Terminated calls, only those fields for which data is available at the mobile end will be populated in this window during testing. The remaining fields will display as PENDING during the test. The statistics for these fields will be compiled in the formatted output report generated at the conclusion of testing. The affected fields are: o Average Access Time o Failed Attempts o No Service o Access Timeout o Dropped (Count) Figure 7-16 - Call Performance Statistics Window
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 70 L. Voice Quality Task Status Window  Status for the current Voice Quality cycle is noted here as one of the following options: o Waiting – This state indicates that the call has been placed midway through one of the Sourcing/Recording cycles. Nomad will remain idle until the start of the next full Sourcing/Recording cycle, at which point data capture will begin. o Recording – In this state, Nomad is currently recording the .WAV file that the mobile receives from the Audio Server to be using for downlink MOS scoring. o Sourcing – In this state, Nomad is sourcing the reference audio file to the mobile to be transmitted to the Audio Server and used for uplink MOS scoring. Note that the waveform and audio playing during this state represent the downlink audio recorded in the previous 10 second window. In parallel with the audio playback, Nomad sources a ‘perfect’ audio clip for uplink transmission that is inaudible to the user.  The waveform playback seen here represents the audio recorded by Nomad in the previous 10 second time window. This playback occurs in parallel with audio clip sourcing. Audio sourcing is not audible to the user, but is noted as Sourcing in this window.  MOS represents the score of the current speech waveform being displayed. A MOS value of 3.500 or higher generally indicates good voice quality. The color of the MOS area represents the MOS Thresholds configured on the Settings → Voice Quality tab. Below the MOS score is a label of either ‘HD’ or ‘NB’ that is based on the spectral energy measured within the downlink wav file. If the ratio of spectral energy above 4k in the wav file is greater than or equal to .09, this is labeled as ‘HD’, otherwise it is labeled as ‘NB’.  Ins Gain (PESQ) or Attenuation (POLQA) o Insertion Gain provides the amplitude of the current waveform when the PESQ scoring algorithm is in use. Ins Gain measures the gain that Nomad has applied to the downlink voice signal. For the best results, Spirent Communications recommends maintaining downlink insertion gain between -7 and 0 dB, ideally as close to 0 dB as possible. Increase Input Level to increase Ins Gain toward 0 dB. o Attenuation provides the amplitude of the current waveform when the POLQA scoring algorithm is in use. Attenuation measures the downlink signal gain reduced by Nomad. For the best results, Spirent Communications recommends maintaining downlink attenuation between 0 and 7 dB, ideally as close to 0 dB as possible. Increase Input Level to decrease Attenuation toward 0 dB. Note that compared to PESQ, POLQA is more forgiving when it comes to attenuation adjustment.  Avg MOS indicates the average of all MOS values collected during the current test session.  Std Dev indicates the spread of the MOS values about the average.  Algorithm displays the scoring algorithm as PESQ or POLQA. See Appendix E for an introduction to POLQA including a comparison between PESQ and POLQA.  Select Monitor Audio to listen to waveform playback for the current test channel. Audio may be monitored for one test channel at a time.
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 71 Figure 7-17 - Voice Quality Task Status Window M. MOS Trends Window  Select Session to view the downlink or uplink measurement history for all handsets performing Voice Quality testing. Figure 7-18 - MOS Session Window  Select an individual channel (Channel 1 / Channel 2 / Channel 3 / Channel 4) to view the measurement history for that handset for the current test session. Both the downlink and uplink history will be displayed here. Click on any data point to view and replay the .WAV file associated with that point. Hover over any data point for statistics for that test cycle. The Number Calling From, Number Called and Avg. Uplink MOS are displayed below the chart.
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 72 Figure 7-19 - Individual MOS Trend Window  Select Distribution to view the spread of downlink or uplink MOS values for all handsets in the current test session. Figure 7-20 - MOS Distribution Window  The Events tab displays null MOS events automatically detected by Nomad as well as events manually recorded by the user. See Section 7.3 for details on event logging and event detection.
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 73 Figure 7-21 - Events Window
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 74 7.2 Manual Volume Adjustment (Voice Quality Testing) Although Spirent recommends running the Auto-Level Assist Wizard (see Section 5.2.5) prior to Voice Quality testing to find optimal downlink and uplink volume settings, manual volume adjustment may be necessary to fine-tune the results. Manual volume adjustment may be used in lieu of Auto Level Assist for analog connections, and must be used in the case of a Bluetooth connection. Auto Level Assist is not available for Bluetooth1. 7.2.1 Manual Downlink Volume Adjustment Downlink volume can be adjusted manually in two fashions:  Using the handset’s volume control.  Using the Settings → Levels tab during testing to adjust the Input Level for the channel of interest. Spirent Communications strongly recommends testing with the handset volume set one level below the maximum, and making further adjustments using the Input Level controls. 7.2.1.1 Downlink Volume Adjustment for Bluetooth Connections In most cases, volume adjustment is not necessary for a channel using a Bluetooth connection.  In cases where the downlink insertion gain or attenuation for a Bluetooth channel is very far from 0 (roughly less than -8 dB for insertion gain or greater than 8 dB for attenuation) the Input Level may be adjusted on the Levels tab by moving the slider one or two notches to the right.  The slider should be kept within the Normal Operating Range.  If MOS deteriorates after the adjustment, return the Input Level to the original value. Figure 7-22 - Bluetooth Volume Controls 1 Volume adjustment for a Bluetooth connection operates differently that for an analog connection. Over a Bluetooth connection, Nomad captures a digital representation of what the handset manufacturer intended to be sent rather than the incoming voice sample. Instead of having an analog signal that can be amplified directly, the digital signal must be amplified by changing the received bit representation. The leading bits of the 16-bit representation are removed to make the remaining signal “louder”. This logarithmic scaling of the digital signal does not provide the precision available when fine-tuning an analog signal.
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 75 7.2.1.2 Downlink Volume Adjustment for Analog Connections To manually find the optimal downlink volume for any analog channel using the Levels dialog:  For the channel of interest, move the Input Level slider to the far right such that the Overdrive light illuminates in the GUI and a red light appears for that channel on the hardware unit.  Move the Input Level slider to the left one notch at a time and observe the Overdrive indicator during the Recording state.  The first position at which the Overdrive light stays off for the current and next recording cycle should be used as the Input setting. For most handsets, the Input Level will fall within the Normal Operating Range of 170-225 shown on the slider. The downlink Insertion Gain or Attenuation shown in the Task Status Window will likely read around 0. If the Overdrive Indicator illuminates for any channel, decrease the Input Level control gradually until it goes off. Note that any adjustment will take effect in the next audio cycle. Figure 7-23 - Analog Volume Controls 7.2.2 Manual Uplink Volume Adjustment In both the analog and Bluetooth connection cases, uplink volume adjustment is necessary only if uplink MOS values are unexpectedly poor. If this is the case, move the Output Level slider in the Settings → Levels dialog within the Normal Operating Range. If uplink MOS deteriorates after the adjustment, return the Output Level to the original value. 7.3 Event Logging and Event Detection (Voice Quality Testing) Events that occur during a Voice Quality test call complement MOS measurements to provide an overall indication of handset performance. Unlike RF collection tools, Nomad’s Voice Quality task does not have access to protocol layer information traditionally used to detect handset events such as dropped calls. Nomad does provide the ability to note events of interest on both a per-channel and per-cycle basis during a Voice Quality task. Nomad also provides the ability to automatically detect a sequence of null MOS scores, often indicative of a dropped call. Event logging and event detection takes place in the Events tab of the MOS Trends Window during Voice Quality testing. To manually note a channel-specific event of interest:  View the Events tab of the MOS Trends Window.
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 76 Figure 7-24 - Events Tab  Click on the cell corresponding to the cycle and channel of interest.  In the Event Classification dialog, select an event to add. Figure 7-25 - Event Classification Dialog  The selected event will appear in the cell. Any channel-specific event entry in the event log will replace the MOS entry for that position in the merged output (see Section 8.1). Affected data points will not be used to calculate MOS summary statistics.  The events in the Event Classification dialog may be customized using the Settings → Voice Quality tab when no test is in progress. See Section 4.4.2 to learn how to customize the Event Classification options.
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 77 To manually note an event that pertains to all channels at a given timestamp (cycle-specific event):  In the Events tab of the MOS Trends Window, click on the Notes column at the timestamp of interest.  Enter notes in the Cycle Notes dialog and click OK.  The recorded notes will appear in the Event Window and will be added to the summary report created with the merge utility. These notes will not replace MOS values for the affected cycles. MOS results for these cycles will be written to the report and included in all summary calculations. Figure 7-26 - Cycle Notes Dialog To automatically detect null MOS events:  While no test is in progress, configure the Events area of the Settings → Voice Quality tab to Enable Event Detection as described in Section 4.4.2.  During subsequent testing, null events will be automatically entered in the Events tab of the MOS Trends Window as channel-specific events. Like channel-specific events that are manually entered, these events will replace the corresponding MOS entry in the summary report and will not count toward calculation of results.
Nomad User’s Manual Chapter 7 – Monitoring a Test in Progress Copyright © Spirent Communications, Inc. 2013 78 7.4 Interpreting MOS Results (Voice Quality Testing) Nomad provides audio quality scores using either PESQ P862.1 MOS or POLQA P.863 MOS as selected by the user2. Both standardized models for predictive speech quality utilize speech signals that are subjected to the communications link under test and compare the degraded speech to the original speech. Both models provide speech quality on a 5-point scale: Score Quality of Speech 5 Excellent 4 Good 3 Fair 2 Poor 1 Bad See Appendix E for an introduction to POLQA including a comparison between PESQ and POLQA. When interpreting results during live collection or in a summary report (see Section 8.1), the highest MOS values represent the best quality of speech. Although the determination of “acceptable” or “poor” scores is somewhat subjective, the following guidelines represent a general rule of thumb:  A MOS value of 4.5 or higher represents virtually perfect audio.  A MOS value of 3.500 or higher generally indicates good voice quality.  A MOS value of 3.000 or higher generally indicates poor voice quality with degradation audible to the end user. 2 POLQA scoring is available as an optional upgrade. Please contact your Spirent Communications representative for more information about this option.

Navigation menu