SARA Reference Manual
User Manual:
Open the PDF directly: View PDF 
.
Page Count: 65
| Download | |
| Open PDF In Browser | View PDF |
HEALTH FACILITY ASSESSMENT OF SERVICE AVAILABILITY AND READINESS
Service Availability and Readiness
Assessment (SARA)
An annual monitoring
system for service delivery
Reference Manual
WHO/HIS/HSI/2014.5 Rev.1
© World Health Organization 2015
All rights reserved. Publications of the World Health Organization are available on the WHO website
(www.who.int) or can be purchased from WHO Press, World Health Organization, 20 Avenue Appia, 1211
Geneva 27, Switzerland (tel.: +41 22 791 3264; fax: +41 22 791 4857; e-mail: bookorders@who.int).
Requests for permission to reproduce or translate WHO publications –whether for sale or for non-commercial
distribution– should be addressed to WHO Press through the WHO website
(www.who.int/about/licensing/copyright_form/en/index.html).
The designations employed and the presentation of the material in this publication do not imply the expression
of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any
country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries.
Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full
agreement.
The mention of specific companies or of certain manufacturers’ products does not imply that they are
endorsed or recommended by the World Health Organization in preference to others of a similar nature that
are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by
initial capital letters.
All reasonable precautions have been taken by the World Health Organization to verify the information
contained in this publication. However, the published material is being distributed without warranty of any
kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the
reader. In no event shall the World Health Organization be liable for damages arising from its use.
Cover photo credit: WHO/Evelyn Hockstein
Service Availability and Readiness
Assessment (SARA)
An annual monitoring
system for service delivery
Reference Manual
Version 2.2
Revised July 2015
Acknowledgements
The service availability and readiness assessment (SARA) methodology was developed through a joint World
Health Organization (WHO) – United States Agency for International Development (USAID) collaboration. The
methodology builds upon previous and current approaches designed to assess service delivery including the
service availability mapping (SAM) tool developed by WHO, and the service provision assessment (SPA) tool
developed by ICF International under the USAID-funded MEASURE DHS project (monitoring and evaluation to
assess and use results, demographic and health surveys) project, among others. It draws on best practices and
lessons learned from the many countries that have implemented health facility assessments as well as
guidelines and standards developed by WHO technical programmes and the work of the International Health
Facility Assessment Network (IHFAN).
Particular thanks are extended to all those who contributed to the development of the service readiness
indicators, indices, and questionnaires during the workshop on "Strengthening Monitoring of Health Services
Readiness" held in Geneva, 22–23 September 2010.
Many thanks to The Norwegian Agency for Development Cooperation (Norad) whom has supported Statistics
Norway to take part in the development of the SARA tools. The support has contributed to the development
and implementation of a new electronic questionnaire in CSPro and data verification guidelines.
A special thanks to the Medicines Information and Evidence for Policy unit at WHO for their contribution to the
SARA training materials and to the Unidad de Calidad y Seguridad de la Atención Médica-Hospital General de
México for their contribution of photographs to the SARA data collectors' guide.
Project Management Group
The SARA methodology and tool were developed under the direction and management of Kathy O’Neill and
Ashley Sheffel with valuable inputs from Ties Boerma and Marina Takane.
Project Advisory Group
Carla AbouZahr, Maru Aregawi Weldedawit, Sisay Betizazu, Paulus Bloem, Krishna Bose, Maurice Bucagu,
Alexandra Cameron, Daniel Chemtob, Meena Cherian, Richard Cibulskis, Mario Dal Poz, Sergey Eremin, Jesus
Maria Garcia Calleja, Sandra Gove, Neeru Gupta, Teena Kunjumen, Thierry Lambrechts, Richard Laing, Blerta
Maliqi, Shanthi Mendis, Claire Preaud, Andrew Ramsay, Leanne Riley, Cathy Roth, Willy Urassa, Adriana
Velasquez Berumen, Junping Yu, Nevio Zagaria, and Evgeny Zheleznyakov.
2
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
Table of contents
Acknowledgements ...........................................................................................................2
Abbreviations ....................................................................................................................4
CHAPTER 1 | OVERVIEW ....................................................................................................5
1.1 Background ....................................................................................................................................7
1.2 Survey overview .............................................................................................................................9
1.3 Pre-survey preparation ................................................................................................................16
1.4 Planning the survey ......................................................................................................................22
1.5 Training field supervisors, data collectors and data entry personnel..........................................25
1.6 Preparing for data collection in the field .....................................................................................30
1.7 Data collection in the field ...........................................................................................................36
1.8 Data entry and processing ...........................................................................................................40
1.9 Data analysis ................................................................................................................................46
1.10 Data archiving ..............................................................................................................................55
References ..............................................................................................................................................63
CHAPTER 2 | CORE INSTRUMENT ..................................................................................... 65
CHAPTER 3 | INDICATORS INDEX.................................................................................... 125
3.1
3.2
3.3
3.4
Indicators ID numbers ................................................................................................................127
SARA general service availability indicators...............................................................................127
SARA general service readiness indicators ................................................................................131
SARA service specific availability and readiness indicators .......................................................136
3
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
Abbreviations
4
AIDS
acquired immunodeficiency syndrome
ALT
alanine aminotransferase
CBR
crude birth rate
CSV
comma-separated values
DBS
dried blood spot
DCMI
Dublin Core Metadata Initiative
DDI
Data Documentation Initiative
DQRC
Data quality report card
DV
Data verification
EDC
electronic data collection device
FBO
faith-based organization
GIS
geographical information system
GPS
global positioning system
HIV
human immunodeficiency virus
HMIS
health management information system
HRIS
human resources information system
ID
identification
IHFAN
International Health Facility Assessment Network
IHP+
IHSN
M&E
International Health Partnership and related initiatives
International Household Survey Network
monitoring and evaluation
MDG
Millennium Development Goal
MFL
master facility list
MNCH
maternal, newborn and child health
MoH
ministry of health
NADA
national data archive
NGO
nongovernmental organization
OECD
Organisation for Economic Co-operation and Development
PMTCT
prevention of mother-to-child transmission (of HIV)
RDT
rapid diagnostic test
SAM
service availability mapping
SARA
service availability and readiness assessment
SPA
service provision assessment
UNAIDS
Joint United Nations Programme on HIV/AIDS
UNDP
United Nations Development Programme
UNICEF
United Nations Children’s Fund
USAID
WHO
XML
United States Agency for International Development
World Health Organization
extensible markup language
1. Overview
5
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
1.1
Background
1.1.1 Why measure service availability and readiness?
Sound information on the supply and quality of health services is necessary for health systems management,
monitoring and evaluation. Efforts to achieve the Millennium Development Goals (MDGs) and to scale up
interventions for HIV/AIDS, malaria, safe motherhood and child health through global health partnerships, have
drawn attention to the need for strong country monitoring of health services, covering the public, private-forprofit and private not-for-profit sectors, and their readiness to deliver key interventions.
With the increased demand for accountability and the need to demonstrate results at country and global levels,
information is needed to track how health systems respond to increased inputs and improved processes over
time, and the impact such inputs and processes have on improved health outcomes and better health status.
However, despite heightened investments in health systems, few countries have up-to-date information on the
availability of health systems that covers both the public and private sectors. Fewer still have accurate, up-todate information required to assess and monitor the "readiness" of health facilities to provide quality services.
Ensuring access to quality health services is one of the main functions of a health system. Service access
includes different components: availability, which refers to the physical presence or reach of the facilities;
affordability, which refers to the ability of the client to pay for the services; and acceptability, which refers to
the sociocultural dimension.
The quality of services is yet another dimension. A prerequisite to service quality is service readiness, i.e. the
health facilities should have the capacity to deliver the services offered. This capacity includes the presence of
trained staff, guidelines, infrastructure, equipment, medicines and diagnostic tests. Service availability and
readiness are prerequisites to quality services, but do not guarantee the delivery of quality services.
7
1. Overview
1.1.2
The global and country context
Building upon principles derived from the Paris Declaration on Aid Effectiveness and the International Health
Partnership and related initiatives (IHP+), global partners and countries have developed a general framework
for the monitoring and evaluation (M&E) of health system strengthening (1). This framework centres on
country health strategies and related M&E processes such as annual health sector reviews, and at its core is the
strengthening of a common monitoring and review platform to improve the availability, quality and use of data
to inform health sector review processes and global monitoring (2).
Within this context, WHO has been working with USAID, MEASURE Evaluation, MEASURE DHS, ICF International,
and other country and global partners to develop tools to fill critical data gaps in measuring and tracking
progress in health systems strengthening. Service availability and readiness assessment (SARA) is one tool
available to fill data gaps on service delivery.
SARA relies on a rapid data collection and analysis methodology, and can be combined with a record review to
assess data quality of the facility reporting system. Ideally, SARA is conducted approximately three to five
months prior to a health sector review to allow for the results to feed into the health sector review process.
1.1.3 Related surveys and initiatives
The service availability and readiness assessment (SARA) effort builds on previous and current approaches
designed to assess health facility service delivery including the service availability mapping (SAM) tool
developed by WHO (3), and the service provision assessment (SPA) tool developed by ICF International under
the USAID-funded MEASURE DHS project (4).
The SARA methodology takes into account best practices and lessons learned from the many countries that
have implemented health facility assessments of service availability and readiness. It also draws heavily on the
work of the International Health Facility Assessment Network (IHFAN) and on experiences from programmeand service-specific facility assessment work.
The training materials for SARA draw on best practices and materials developed for survey methods such as the
SPA and the WHO/Health Action International (HAI) methodology for measuring medicine prices, availability,
affordability and price components (5).
8
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
1.2
1.2.1
Survey overview
Survey objectives
SARA is designed as a systematic survey to assess health facility service delivery. The objective of the survey is
to generate reliable and regular information on service delivery including service availability, such as the
availability of key human and infrastructure resources, and on the readiness of health facilities to provide basic
health-care interventions relating to family planning, child health services, basic and comprehensive obstetric
care, HIV/AIDS, tuberculosis, malaria and noncommunicable diseases.
The SARA survey generates a set of tracer indicators of service availability and readiness that can be used to:
• detect change and measure progress in health system strengthening over time;
• plan and monitor the scale-up of interventions that are key to achieving the MDGs, such as
implementing interventions to reduce child and maternal mortality, HIV/AIDS, tuberculosis and malaria,
and to respond to the increasing burden of noncommunicable diseases;
• generate the evidence base to feed into country annual health reviews, to better inform the
development of annual operational plans and to guide more effective country and partner investments;
• support national planners in planning and managing health systems (e.g. assessing equitable and
appropriate distribution of services, human resources and availability of medicines and supplies).
Key outputs from SARA form the basis for national and subnational monitoring systems of general service
availability and readiness, and service-specific readiness (maternal and child health, HIV/AIDS, tuberculosis,
malaria, noncommunicable diseases, surgical care, etc.). SARA products include a regularly updated national
database of public and private facilities, and an analytical report of core indicators to assess and monitor
availability of health services and readiness to provide services.
QUESTIONS ANSWERED BY SERVICE AVAILABILITY AND READINESS ASSESSMENT (SARA)
•
What is the availability of basic packages of essential health services offered by public and private health facilities?
•
Is there an adequate level of qualified staff?
•
Are resources and support systems available to assure a certain quality of services?
•
How well prepared are facilities to provide high-priority services such as reproductive health services, maternal and
child health services, and infectious disease diagnosis and treatment (e.g. HIV, sexually transmitted infections,
tuberculosis and malaria)?
•
Are facilities ready to respond to the increasing burden of noncommunicable diseases?
•
What are the strengths and weaknesses in the delivery of key services at health-care facilities?
9
1. Overview
1.2.2 Key topics, indicators and indices
The SARA survey is designed to generate a set of core indicators on key inputs and outputs of the health system,
which can be used to measure progress in health system strengthening over time (6). Tracer indicators aim to
provide objective information about whether or not a facility meets the required conditions to support
provision of basic or specific services with a consistent level of quality and quantity. Summary or composite
indicators, also called indices, can be used to summarize and communicate information about multiple
indicators and domains of indicators. Indices can be used for general and service-specific availability and
readiness.
There are three main focus areas of SARA.
I.
Service availability refers to the physical presence of the delivery of services and encompasses health
infrastructure, core health personnel and aspects of service utilization. This does not include more
complex dimensions such as geographical barriers, travel time and user behaviour, which require
more complex input data. Service availability is described by an index using the three areas of tracer
indicators (see Table 1.2.1). This is made possible by expressing the indicators as a percentage score
compared with a target or benchmark, then taking the mean of the area scores.
II.
General service readiness refers to the overall capacity of health facilities to provide general health
services. Readiness is defined as the availability of components required to provide services, such as
basic amenities, basic equipment, standard precautions for infection prevention, diagnostic capacity
and essential medicines. General service readiness is described by an index using the five general
service readiness domains (see Table 1.2.1). A score is generated per domain based on the number of
domain elements present, then an overall general readiness score is calculated based on the mean of
the five domains.
III.
Service-specific readiness refers to the ability of health facilities to offer a specific service, and the
capacity to provide that service measured through consideration of tracer items that include trained
staff, guidelines, equipment, diagnostic capacity, and medicines and commodities.
TABLE 1.2.1: SUMMARY OF TRACER INDICATORS, ITEMS AND SERVICES FOR SERVICE AVAILABILITY AND SERVICE
READINESS
Domain
Tracer indicators, items or services
I. Service availability
1. Health infrastructure
10
•
Number of health facilities per 10 000 population
•
Number of inpatient beds per 10 000 population
•
Number of maternity beds per 1000 pregnant women
2. Health workforce
•
Number of health workers per 10 000 population
3. Service utilization
•
Outpatient visits per capita per year
•
Hospital discharges per 100 population per year
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
Domain
Tracer indicators, items or services
II. General service readiness
1. Basic amenities
Mean availability of seven basic amenities items (%): power, improved water source,
room with privacy, adequate sanitation facilities, communication equipment, access to
computer with Internet, emergency transportation
2. Basic equipment
Mean availability of six basic equipment items (%): adult scale, child scale, thermometer,
stethoscope, blood pressure apparatus, light source
3. Standard precautions for
infection prevention
Mean availability of 9 standard precautions items (%): safe final disposal of sharps, safe
final disposal of infectious wastes, appropriate storage of sharps waste, appropriate
storage of infectious waste, disinfectant, single-use disposable/auto-disable syringes, soap
and running water or alcohol-based hand rub, latex gloves and guidelines for standard
precautions
4. Diagnostic capacity
Mean availability of 8 laboratory tests available on-site and with appropriate equipment
(%): haemoglobin, blood glucose, malaria diagnostic capacity, urine dipstick for protein,
urine dipstick for glucose, HIV diagnostic capacity, syphilis RDT and urine pregnancy test
5. Essential medicines
Mean availability of 25 essential medicines (%):Amlodipine tablet or alternative calcium
channel blocker, amoxicillin (syrup/suspension or dispersible tablets), amoxicillin tablet,
ampicillin powder for injection, aspirin (capsules/tablets), beclometasone inhaler, beta
blocker (e.g.bisoprolol, metaprolol, carvedilol, atenolol), carbamazepine tablet,
ceftriaxone injection, diazepam injection, enalapril tablet or alternative ACE inhibitor (e.g.
lisonopril, Ramipril, perindopril), fluoxetine tablet, gentamicin injection, glibenclamide
tablet, haloperidol tablet, insulin regular injection, magnesium sulfate injectable,
metformin tablet, omeprazole tablet or alternative (e.g. pantoprazole, rabeprazole), oral
rehydration solution, oxytocin injection, salbutamol inhaler, simvastatin tablet or other
statin (e.g. atorvastatin, pravastatin, fluvastatin), thiazide (e.g. hydrochlorothiazide) and
zinc sulphate (tablet or syrup).
III. Service-specific readiness
For each service, the
readiness score is computed
as the mean availability of
service-specific tracer items
in four domains: staff and
training, equipment,
diagnostics, and medicines
and commodities
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Family planning
Antenatal care
Basic obstetric care
Comprehensive obstetric and neonatal care
Child health immunization
Child health preventative and curative care
Adolescent health services
Lifesaving commodities for women and children
Malaria diagnosis or treatment
Tuberculosis services
HIV counselling and testing
HIV/AIDS care and support services
Antiretroviral prescription and client management
Prevention of mother-to-child transmission (PMTCT) of HIV
Sexually transmitted infections diagnosis or treatment
Noncommunicable diseases diagnosis or management: diabetes, cardiovascular
disease, chronic respiratory disease and cervical cancer screening
Basic and comprehensive surgical care
Blood transfusion
Laboratory capacity
11
1. Overview
1.2.3 Core instrument
The basic approach to SARA is to collect data that are comparable both across countries and within countries
(i.e. across regions and/or districts). To achieve this, a standard core questionnaire has been developed. The
core questionnaire was pretested in a variety of settings in two countries. The first pretest occurred in Sierra
Leone in April, 2011. A second pretest occurred in Kenya in June, 2011. This second pretest was part of a larger
pretest of the revised MEASURE DHS Service Provision Assessment (SPA) questionnaire, which includes all core
SARA questions as they are embedded in the SPA Inventory questionnaire. Following the pilot test experience,
adjustments were made to the questionnaire to account for the information gained, resulting in the standard
core questionnaire.
Typically, a country adopts the core questionnaire with adaptations to certain elements such as:
•
•
•
•
•
types of facilities
managing authority of facilities
national guidelines for services
staffing categories
national policies for medicines (e.g. for tuberculosis, HIV/AIDS).
The questionnaire does not attempt to measure the quality of services or resources, but it can be used in
conjunction with additional modules such as management assessment or quality of care.
1.2.4 Survey design methodology
The SARA survey requires visits to health facilities with data collection based on key informant interviews and
observation of key items. The survey can either be carried out as a sample or a census; the choice between
these methodologies will depend on a number of elements including the country's resources, the objectives of
the survey and the availability of a master facility list (MFL). For example, if the objective of the survey is to
have nationally representative estimates, a sample survey would be appropriate. However, if the objective is to
have district estimates, the sampling methodology must be adjusted to either a larger sample or in some cases
a full census.
Service availability
The recommended data source for information on service availability is a national master facility list (MFL) of
all public and private facilities (7). A facility census is usually required to establish and maintain a national MFL.
A facility census aims to cover ALL public and private health facilities in a country. The census is designed to
form the basis for a national and subnational monitoring system of service delivery, which can be
supplemented by quality ascertainment through facility surveys and further in-depth assessments. A census is
the recommended methodology for forming the baseline of service availability and readiness data. Service
availability data should be updated annually through routine, facility-based reporting and validated
approximately every five years through a facility census.
Service readiness
The recommended design methodology for measuring service readiness is a sample survey. Sampling is done in
a systematic way to ensure that the findings are representative of the country and region/district in which the
survey is being conducted. Drawing a random sample of health facilities is much more complicated if the
country does not have a comprehensive and up-to-date MFL. Therefore, it is highly recommended to invest in
establishing a MFL that includes all public and private facilities. In cases where a national list of facilities is not
available or up-to-date, the service readiness survey can be carried out at the same time as the facility census
for service availability.
12
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
Master facility list (MFL)
Regardless of the method selected, a complete MFL is required. Therefore, it is highly recommended to invest
in establishing a MFL that includes all public and private health facilities. In many countries there are already
lists of public facilities and sometimes also nongovernmental facilities. However, private facilities are often
excluded or only partially included in these lists. WHO and partners have developed a guide to support
countries in creating a MFL. Please refer to the document Creating a master facility list (7) for more
information on best practices in establishing a MFL.
Data quality assessment
The service availability and readiness assessment can be used for to assess data quality of the routine system
by comparing results with aggregated routine health information data at district, provincial and national level.
In addition, the service readiness assessment can be combined with a record review for data verification
purposes, to ascertain the completeness and quality of facility reporting. The data quality review (DQR) (8) can
be used to verify the quality of routinely reported data for some key coverage indicators, quantifying problems
of data completeness accuracy and external consistency.
1.2.5 Timeline of implementation
Service availability and readiness assessments should be planned on a yearly or biennial basis to coincide with
and feed into national health planning cycles. Sample surveys should be organized every year about three to
five months in advance of the annual review. The national MFL should be used to provide the sampling frame
(see Figure 1.2.1).
FIGURE 1.2.1: TIMELINE OF IMPLEMENTATION FOR SERVICE AVAILABILITY AND READINESS ASSESSMENT
Facility assessment (sample survey)
Service
readiness
Service
availability
(with data verification)
Facility
census
Updating from districts
Facility
census
Master facility list
Year
0……….1……….2………..3……….4………..5
The time needed to complete a service availability and readiness assessment depends on the size of the
country and whether or not there is a need for a full facility census. From the initial country-adaptation of the
assessment tool to the dissemination of data and production of country reports, the entire process generally
takes from three to six months.
13
1. Overview
Table 1.2.2 below provides an overview of the survey steps and the activities to be undertaken at each step.
TABLE 1.2.2: SUMMARY OF SURVEY STEPS AND ACTIVITIES
Steps
Survey activities
1. Survey planning and
preparation
•
•
•
•
•
•
•
•
•
•
2. Data collection in the
field
•
•
•
•
•
•
•
•
•
Plan the data collection visits (prepare a letter of introduction, contact each site,
prepare a schedule of visits)
Prepare materials and tools for data collectors
Arrange for transport and regular communications during fieldwork
Assemble materials necessary for local data collection
Confirm appointments with health facilities
Visit health facilities and collect SARA data in teams (usually two interviewers and a
driver)
At the end of the interview, check questionnaire and resolve missing/unreliable
information
Return completed forms and/or transfer electronic files to field supervisor at the
conclusion of each day
Return forms (paper and/or electronic) to survey manager when data collection is
complete
3. Data entry, analysis and
interpretation
•
•
•
•
Enter data using the CSPro application
Edit, validate and clean data set, checking for consistency and accuracy
Export the data set for analysis (SARA indicators)
Conduct analyses of SARA data using the standard core indicators (SARA automated
tool for results graphs and tables) as well as any country-specific indicators of
interest
4. Results dissemination
•
Meet with survey coordinating group to analyze and interpret survey results and to
finalize recommendations
Prepare the final report
Plan and implement dissemination activities. The results should be used to support
annual health reviews and feed into the M&E platform for the national health plan
Document and archive the survey using metadata standards
•
•
•
14
Establish a survey coordinating group of country stakeholders to oversee and
facilitate the objectives, scope, design, implementation and analysis
Obtain a list of all health facility sites (public, private, nongovernmental organizations
(NGOs) and faith-based organizations (FBOs)), including country facility registry codes
Determine appropriate design methodology (census or sample), develop an
implementation plan and budget, and secure funding
Adapt questionnaires to meet country-specific needs
Recruit survey personnel (survey manager, field supervisors, data collectors, data
entry/processing personnel, data analysts)
Prepare a survey schedule
Identify the survey sites (sampling frame). Select the sample size and sample of
health facilities (if sampling methodology is chosen)
Procure logistics including equipment and transport, taking into consideration the
number of sites to be visited, the number of data collection teams, drivers, vehicles,
petrol, etc.
Plan and conduct training courses for interviewers and field supervisors
Pilot test the survey in a selected number of health facilities, evaluate results and
make amendments if necessary
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
1.2.6 Roles and responsibilities
The survey is usually undertaken under the overall leadership of the Ministry of Health. The following section
briefly outlines the roles and responsibilities of the key parties involved in the implementation of SARA and
data quality activities.
Ministry of Health (MoH): Will have overall responsibility for the coordination of this process. Will coordinate
and provide support to get permission to conduct data collection activities, help with the coordination of
analysis and results dissemination meetings by inviting all the appropriate governmental departments, key
non-governmental and development partners. Will also promote the use of this data for policy and planning.
Survey Coordinating Group: The Coordinating Group, led by Ministry of Health should include national
institutes and other key stakeholders in the health services sector. This core group, will provide leadership and
oversight throughout the whole process from questionnaire to dissemination of results.
Implementation agency: Will be responsible for conducting field data collection for SARA and the data
verification component of the Data Quality Review. Details of the composition of the implementation agency
team is given in a separate document.
Agency providing quality assurance and technical support: It is recommended that an independent party be
involved in the implementation process. This support can be provided by a separate national institute or
independent consultant. He/she will be responsible for providing support to the implementation team on
planning and implementing SARA; provide a quality assurance role to ensure due processes are followed during
training, data collection, cleaning and analyses stages (including validation visits in 10% of the facilities); to
provide assistance and oversight to the implementing team on the production of the SARA and data quality
assessment report.
15
1. Overview
1.3
1.3.1
Pre-survey preparation
Establishing a survey coordinating group
Bringing partners together and mobilizing them around the survey is a critical first step towards successful
implementation. One of the first activities is to identify and establish a group of core stakeholders at country
level to oversee, coordinate and facilitate the planning, implementation and follow-up of the facility
assessment process. In general, partners include those groups, individuals, and/or organizations that are
carrying out or planning similar efforts as well as those for whom the outputs of the health facility assessment
will be of interest. These often include:
• ministries of health (as well as national institutes of statistics, geographical information system (GIS)
units, health management information systems (HMIS) units, health services and other public research
institutions);
• universities and other academic institutions involved in research;
• NGOs and other organizations involved in data collection;
• United Nations health-related organizations present in the country (e.g. WHO, UNICEF, UNDP, UNAIDS);
• international funders active in the country (i.e. the Global Fund to Fight AIDS, Tuberculosis and Malaria,
government agencies for international development).
The role of the survey coordinating group should include:
• clarifying the objectives of the survey;
• supporting the survey manager in planning, preparing and conducting the study, and identifying
important policy issues that should inform the survey protocol;
• advising on any matters that arise during survey preparation, fieldwork and data analysis;
• assisting in interpreting data and developing policy recommendations;
• promoting the findings of the survey and advocating for appropriate policy recommendations.
It is important to hold regular meetings with the survey coordinating group throughout the survey process. At
least one meeting should be held to support the planning and preparation of the SARA survey, and one
meeting should be held post-survey for interpreting survey results and developing recommendations. A second
post-survey meeting may be beneficial to discuss the results and their policy implications, consolidate all survey
results and finalize recommendations.
1.3.2 Identifying entities for survey implementation and quality assurance
Once the coordinating group has been established, it is important to define who will be in charge of the survey
field implementation. It is recommended to identify and work with a national institute (e.g. National Statistical
Office, School of Public Health, etc.) or an entity used to conducting such field assessments. The selection is
done in agreement with the Ministry of Health.
The institute in charge of the survey implementation works closely with the coordinating group for the
preparation, implementation and results dissemination of the survey. It is also recommended that a third party
(different from the implementing agency – consultant, regional institute, etc.) ensures the quality of the survey
(including organizing visits in 10% of the sites). This entity can also provide technical backstopping as required.
It works closely with the coordinating group and implementation institute throughout the process, ensuring
that survey procedures are followed properly and as per the defined methodology.
16
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
1.3.3 Compiling a master facility list (MFL)
Before beginning a health facility assessment, a situation analysis assessing the availability of health facility
information must be carried out. An important prerequisite for conducting a SARA survey is the existence of a
MFL. The analysis should therefore aim to ascertain the existence and reliability of an official MFL.
Regardless of the survey methodology (census or sample), a complete master list of facilities is required.
Therefore, it is highly recommended to invest in establishing a MFL that includes public, private for-profit, and
NGO facilities. In many countries there are already lists of public facilities and sometimes also
nongovernmental facilities. However, the private facilities are often excluded or only partially included in these
lists.
Before a health facility assessment can be implemented, ALL health facilities in a country must be identified
and a health facility list created. This list must include health facilities in all sectors including the public sector,
the private sector, FBOs and NGOs. In some countries, a MFL may be available containing all the required
information. In most cases however, this information is not readily available and must be compiled. The
ministry of health (MoH) generally maintains information on public health facilities and can serve as the basis
for the MFL. Other contacts will need to be identified to retrieve information on private, FBO, NGO and other
facilities.
All available health facility listings have to be reconciled to identify a single, comprehensive list, with each
facility assigned a unique identification (ID) code. Facilities should be classified by level of service provision
(from hospital at the highest level through clinic at the lowest level) and by ownership (MoH, mission, NGO or
private). Locational information should be included in the MFL when available. The geographical coordinate
collection method should also be recorded (i.e. global positioning system (GPS) remote device, digital place
names, gazetteers, etc.).
A key component of the MFL is the unique ID code assigned to each facility. A set of data must be gathered
with the specific purpose of uniquely identifying each survey site. In database terminology this set of identifier
data is referred to as a "primary key" or a "unique key": a code uniquely identifying a row or column of a
database. Without specific ID attached to each survey site, there is a risk of duplicate data collection. In
addition to greatly lessening the risk of data duplication, site ID fields allow for cross-survey comparisons as
well as comparisons over time.
Please refer to the document Creating a master facility list (7) for more information on best practices in
establishing a MFL.
1.3.4 Designing a methodology and implementation plan
Design methodology
There are two potential design methodologies for the SARA survey:
• a facility census (i.e. assessment of all health facilities)
• a sample survey (i.e. a representative sample of facilities).
Service availability requires a denominator that includes all public and private health facilities in the country (i.e.
a census). Service readiness can be measured through a representative sample of facilities.
17
1. Overview
Facility census
The recommended data source for information on service availability is a national MFL of all public and private
facilities. A facility census is usually required to establish and maintain the MFL. Service availability data should
be updated annually through routine, facility-based reporting, and data should be validated approximately
every five years through a facility census.
A facility census aims to cover ALL public and private health facilities in a country. The census is designed to
form the basis for a national and subnational monitoring system of service delivery, which can be
supplemented by quality ascertainment through facility surveys and further in-depth assessments. A census is
the recommended methodology for forming the baseline of service availability and readiness data.
Sample survey
The recommended design methodology for measuring service readiness is a sample survey. Sampling is done in
a systematic way to ensure that the findings are representative of the country or state/province in which the
survey is being conducted. Drawing a random sample of health facilities will be much more complicated if the
country does not have a comprehensive and up-to-date MFL. Therefore, it is highly recommended to invest in
establishing a MFL that includes public, private for-profit, and nongovernmental facilities. If a fairly complete
master list of facilities already exists, a sampling approach can be used.
Implementation plan and budget
An implementation plan should be drafted based on the objectives of the survey and the results of the
situation analysis of health facility information. The implementation plan serves as a comprehensive outline of
the operational plan for implementing a SARA survey and is key to ensuring the success of the survey. The plan
must lay out the reason for carrying out the survey, how the survey will be executed and how to oversee the
survey to ensure that it will be completed on time and within budget. The objectives of the survey will help to
determine the design methodology, which will in turn drive much of the operational plan and budget for the
survey. When designing the budget, it is essential to ensure that the following items are accounted for.
Financial and human resources
• human resources
− survey manager
− TA/QA entity
− field supervisors
− data collectors
− data entry personnel
− data analysts
− transport
− materials
− expenses related to
pilot testing
− materials (paper,
pens, etc.)
− photocopying
− communication (e.g.
telephone charges)
• training
− training venue
− daily allowance and
accommodation
• data collection and validation
− daily field allowance
and accommodation
for data collectors
− transport
• data cleaning, processing and analysis
• meetings of the survey coordinating group
• report production and dissemination
• advocacy and communications
• overheads
• contingency.
18
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
Technical resources
• mobile electronic data collection devices (EDCs) e.g. personal digital assistants (PDAs), tablet computers
or laptop computers: one for each data collection team
• GPS devices (if the facility coordinates need to be taken): one for each data collection team
• batteries for GPS devices
• memory cards for EDCs
• computer(s) for data entry
1
• data entry application (CSPro recommended)
• data analysis programme.
Once a comprehensive budget has been developed, funding should be secured to cover all survey costs (a
standard budget template is available in the SARA Implementation Guide—Chapter 1: Planning).
1.3.5 Adapting the SARA instrument to country-specific needs
A standard core questionnaire for measuring service availability and readiness is available. However, the
questionnaire must be adapted for country use to reflect the needs of each country and specificities of each
health-care system. When adapting the health facility questionnaire, consideration should be given to how
changes will affect data collection, and adjustments should be made to ascertain that definitions are specific
enough to assure comparability across the country and within districts. Training, data collection and analysis
are carried out, even in larger countries, within one month, and adding more to the tool will make it slower and
could create problems at the analysis stage if not carefully considered. SARA is not intended to provide
comprehensive data on all aspects of health system functioning. Rather, it focuses on key "tracer" elements
that are critical to programmes that are scaling up or that are indicative of the essential health system
underpinnings or "readiness" to do so. This should be kept in mind while adapting the questionnaire and
adding additional modules or questions.
The following areas of the SARA tool must always be adapted to the country context:
• types of facilities
• managing authority of facilities
• national guidelines for services
• staffing categories
• tuberculosis medicines
• HIV/AIDS medicines
• Routine immunization
• other country-specific medicines.
The questionnaire can be implemented as either a paper questionnaire or an electronic questionnaire.
Paper questionnaire: any changes should be made according to the country adaptation process prepared for
the survey training.
Electronic questionnaire: once a mobile EDC has been selected, the appropriate software can be chosen. This
software generally comprises a desktop forms designer and database, a synchronization conduit and the
handheld forms application. Once the software is uploaded, the survey form can be designed on a desktop
For information about the Census and Survey Processing System (CSPro), including free download, visit:
http://www.census.gov/population/international/software/cspro/
1
19
1. Overview
computer and then synchronized with the handheld device. For the SARA survey, the recommended software
for electronic questionnaires is CSPro.
1.3.6 Recruiting survey personnel
The SARA survey will require involvement of the following personnel:
• survey manager
• field supervisors
• data collectors
• data entry personnel
• data analysts.
Survey manager
The survey manager plans and coordinates the survey at the central (national) level. This includes planning the
survey’s technical and logistical aspects, recruiting and training survey personnel, supervising data collection
and data entry, conducting data quality assurance and data analysis, interpreting results and preparing a survey
report.
Wherever possible, the survey manager should have experience in conducting surveys and should be very
familiar with the health-care system. The survey manager should be familiar with basic statistics and
interpreting data. Successful communication of the survey results also requires an understanding of the policymaking process and different advocacy strategies. Where the survey manager does not possess all of these
qualities, he or she should select the survey coordinating group members to ensure that the survey
management team includes the necessary health, surveying, statistics, policy and advocacy skills.
Field supervisors
Field supervisors are responsible for overseeing all aspects of data collection in the survey area(s) for which
they are responsible. In a small country or in a survey that is conducted in a single region of a country, it may
be possible for all fieldwork to be undertaken by a single team. Experience has shown that in larger-scale
studies, however, it is advisable to designate a field supervisor in each of the geographical areas that will be
surveyed.
Field supervisors have a crucial role to play in ensuring data quality and consistency. They should be
experienced in data collection and be familiar with health terminology. They are also instrumental in gaining
access to facilities; if any field supervisor is unfamiliar with their designated area, a local contact may be
needed to assist. Field supervisors may also be responsible for choosing local data collectors when they are not
sent from the central level.
Data collectors
Data collectors are responsible for visiting health facilities and collecting SARA data with a high degree of
accuracy. The survey methodology has been designed to minimize as far as possible the need for a high level of
technical expertise.
However, data collectors should, wherever possible, have the following qualifications, skills and capabilities:
• a health qualification (nurse, midwife or medical student) and familiarity with the organization and
functioning of health facilities;
• some understanding of the principles of sample surveys, ideally with some previous experience in
conducting surveys;
• an appreciation of the logistics requirements for carrying out field studies;
20
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
• post-secondary school education as a minimum;
• familiarity with the locality and local language or dialect.
Data collection requires an aptitude for concentration and attention to detail. The best data collectors combine
the discipline of collecting data in a standardized way with the ability to identify unusual situations that require
advice from the field supervisor or survey manager. Data collectors must be available to work full time for the
duration of the fieldwork. They should be willing to work extended hours if necessary and be able to stay away
from their homes for extended periods of time.
The number of data collectors required depends on the sample size of the survey. Data collectors should work
in pairs. Each visit to a health facility is likely to require about two hours plus transport time. In practice, this
means that a team of two data collectors can survey two to four facilities per day. The number of data
collectors will also depend on the budget available, the locations of the survey areas, the travel conditions and
the number of health facilities to be surveyed. It is better to have a smaller number of better qualified data
collectors than to have a large team where some data collectors lack the necessary skills.
Data entry and data processing personnel
Accurate data entry is vital to ensure the reliability of the results. Two data processing personnel with
experience in using the selected data entry software are required: one to enter the data, and the other to reenter the same data to check that the entries are correct. If data are being entered from paper questionnaires,
double entry is essential to ensuring the accuracy of the data entry process. If data are collected both
electronically and on paper, then the first instance of data entry has already occurred during the electronic
data collection and the data entry personnel would only be responsible for the second entry of data for
validation purposes. In some cases, it may be possible to use the same personnel for both data collection and
data entry, provided they have the necessary expertise to undertake both functions.
Data analysts
The primary tasks of the data analyst(s) are to inspect, clean, transform, analyse and visualize data with the
goal of highlighting useful information, suggesting conclusions and supporting decision-making. It is vital that
the data analyst has an advanced knowledge of the chosen analysis software for the SARA survey. A working
knowledge of health service delivery and the specific country's health system is important for interpretation of
the results and is required of at least one member of the data analysis team.
1.3.7
Preparing the survey schedule
The complete survey should generally take between three and six months to complete, including survey
preparation, data collection, data entry, data analysis and report writing. Further time should be allotted for
dissemination and follow-up activities. Given that the information gathered from SARA should be used to
inform decision-making, it is important that data collection be conducted rapidly and the report generated as
soon as possible once data collection is complete. This will ensure that the survey results are relevant and
informative for decision-makers. A survey schedule should be developed and consulted regularly to ensure that
activities are proceeding according to plan. This schedule should detail the amount of time allotted for each
step in the survey process, and should serve as a timeline for all survey activities.
21
1. Overview
1.4
1.4.1
Planning the survey
Selecting the sample size and sample
Determining the sample size and selecting the sample for a facility survey is a complex subject, which will vary
considerably from case to case depending on the desired precision and type of estimates, the number of
facilities in the country as well as the specific objectives of the assessment. For example, a SARA conducted to
produce national estimates will require a much smaller sample size than if district-level estimates are desired.
In order to ensure that the sample is representative, it is best to consult with a sampling expert or a statistician
to select an appropriate sampling methodology. For the SARA, the most common sampling strategy is Option 1
in the table below—a nationally representative sample obtained by taking a simple random sample of facilities
within each stratum (facility type and managing authority) at the national level. The table below presents
different sampling options that could be used to conduct a SARA based on the desired level of estimates:
Domains of estimation
Sampling method
Sample size
2
(estimate)
Approximate cost
Option 1: National estimates
only
National estimates with
disaggregation by facility
type (3 levels) and managing
authority (public/private)
Small country
Stratification by facility type and
managing authority, simple/systematic
random sampling within each stratum
with census or oversampling of hospitals
(design effect = 1)
150 – 250 facilities
$60K-100K
250 – 500 facilities
$100K-200K
Small country
Stratification by region, facility type and
managing authority, simple/systematic
random sampling within each stratum,
with census or oversampling of hospitals
(deff = 1)
5 regions: 250 – 500
facilities
10 regions: 500 – 800
facilities
$100K-130K
$130K-180K
Medium/large country
Blend of list and area sampling: list
sampling for large health facilities, and
area sampling for small facilities (census
2
of facilities in sampled area PSUs )
(deff = 1.2)
Medium country
4 regions: 300 – 500
facilities
Large country
4 regions: 400 – 800
facilities
Large country
Purposive sample of regions,
simple/systematic random sample with
oversampling of hospitals for each
region
4 regions (150 facilities
per region): 600
facilities
Medium country
Blend of list and area sampling: list
sampling for large health facilities, and
area sampling for small facilities (census
3
of facilities in sampled area PSUs )
(deff = 1.2)
Option 2: Subnational
estimates
Regional and national
estimates with
disaggregation by facility
type (3 levels) and managing
authority (public/private)
Option 3: Subnational
estimates
Regional estimates for a
subset of regions, with
disaggregation by facility
type (3 levels) and managing
authority (public/private) for
selected regions; no national
estimates
$180K-360K
$60-100K per
region
Sample size estimates assume a margin of error of 0.1 and 95% level of confidence
Administrative units that form the PSUs (Primary Sampling Units) for the area sample should contain approximately
1-5 health facilities each (communes, sub-counties, villages)
2
3
22
$120K-200K
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
Domains of estimation
Sampling method
Option 4: District sample
District estimates for
sampled districts; national
estimates if sufficiently many
facilities are sampled
Small, medium and large countries
List sampling for regional and national
hospitals plus sampling of districts (twolevel cluster sample: selection of districts
as first level, selection of facilities within
these districts as the second level)
(deff = 2)
Option 5: Facility census
All possible domains of
estimation
Sample size
2
(estimate)
Small country
300-500 facilities (104
30 districts )
Medium country
400-800 facilities (20+
districts)
Large country
600-1000 facilities (30+
districts)
Small, medium and large countries
Census of all facilities
Approximate cost
$100K-200K
$160K-320K
$270K-470K
Very expensive
Small country: 50 – 100 hospitals, 1000 – 2000 health facilities total, 10 – 80 districts (e.g. Sierra Leone, Togo, Burkina Faso)
Medium country: 100-500 hospitals, 2000 – 5000 health facilities total, 80 – 500 districts (e.g. Uganda, Tanzania)
Large country: 500 – 1000 hospitals, 5000 – 10000 health facilities total, 500 – 1000 districts (e.g. DRC, Nigeria)
1.4.2
Procuring logistics
Planning for data collection requires consideration of the logistics needs for data collection teams as well as an
assessment of the hardware and software needs for data collection. Equipment should be considered for a
base camp as well as for fieldwork, and for operations as well as for training. The guiding principle that should
be kept in mind when compiling equipment for the field is redundancy, i.e. to have backup components and a
contingency plan in case equipment fails, breaks or is lost. All equipment should have one or more backups,
depending on the equipment type and survey requirements. If feasible, paper forms and printing capabilities
provide a viable contingency plan for the worst-case scenario of mobile device failure. Equipment requirements
are also determined according to country-specific needs, as well as the availability of resources and budget.
Assigning facilities to teams
It is recommended to map all
facilities in the survey sample
to assist with logistics
planning for the data
collection. This map can be
made either on paper or
electronically. The map should
include information such as
roads, topography, basic
geographical features,
elevation and location of
health facilities, which are
useful in determining survey
areas. Teams should be
assigned to facilities based on
the geographical distribution
of the selected health facilities.
Figure 1.4.1 gives an example
of a map that would be useful
for SARA logistics planning.
4
FIGURE 1.4.1 SERVICE AVAILABILITY AND READINESS ASSESSMENT (SARA)
EXAMPLE MAP
Number of districts in sample depends on the number of facilities per district
23
1. Overview
Survey team requirements
The duration of the field survey depends on the availability of resources, the number of teams, the number of
health facilities to be visited, and the size of the country and population.
As a general guide, data collection teams consist of two interviewers/data collectors plus a driver. On average,
one team can cover at least two health facilities per day.
The estimated duration of the survey is calculated during the planning phase and is unique to the needs and
resources available in the country. The following examples illustrate the planning that is required.
Example 1. A country consisting of 50 districts with on average 40 health facilities per district.
One team covers one district (40 health facilities) over 20 days (two facilities per day), and so 10 teams
cover 10 districts over 20 days.
Therefore 10 teams will cover one country (50 districts) over 100 days (or three months).
Example 2. An urban area with an average of 200 health facilities.
One team covers 200 health facilities over 100 days (two facilities per day), or 10 teams cover 200
health facilities over 10 days.
For all surveys, logistics planning needs to take into account the following:
• car hire and fuel for the duration of the survey
• per diem for the driver(s)
• per diem for the data collectors.
Equipment requirements
Equipment requirements are also determined according to country-specific needs, as well as the availability of
resources and budget.
24
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
1.5
Training field supervisors
data collectors and data
entry personnel
This chapter provides practical guidance on conducting a training workshop for field supervisors, data
collectors and data entry personnel. Training is an important element of survey preparation because it helps to
ensure the accuracy and reliability of the data gathering and data entry procedures. Consequently, this chapter
also covers the issue of ensuring data quality. This chapter has been developed to assist survey managers in
conducting training workshops for their survey personnel, regardless of whether they have attended any
previous training.
1.5.1
The importance of data quality
It is important to ensure data quality for several reasons:
• solid data support conclusions and recommendations;
• future policy decisions may rely on the evidence generated in the survey;
• critics and opponents will look for weaknesses in the survey methods and results;
• results will be publicly accessible and may be used by others, e.g. in conducting international
comparisons.
There are several reasons for data problems commonly encountered in a survey:
• field supervisors, data collectors and data entry personnel receive insufficient or poor-quality training;
• the pilot survey is not conducted properly;
• work in the field is of poor quality (insufficient supervision, no quality control for submission of
completed forms, misunderstanding of instructions, etc.);
• data are not checked at every stage of the survey process;
• data are entered incorrectly;
• there are problems with uniquely identifying facilities;
• there are problems of human error;
• there is non-response to questions.
Data problems can therefore be avoided by:
• carefully studying the survey manual and accompanying materials at every step, and following
instructions;
• selecting capable and reliable personnel and ensuring they are well trained in the survey methodology;
• encouraging personnel to communicate openly about uncertainties in survey procedures and
questionable data;
• double-checking data collection forms for accuracy and completeness after each data collection visit, at
the end of each day of fieldwork, and prior to data entry;
• conducting double entry of the survey data – data are entered twice by different people and then crosschecked.
25
1. Overview
Thorough training of survey personnel is one of the most important ways of ensuring accurate data collection
and good-quality data. Experience from previous surveys has shown that poor survey preparation, including
inadequate training of survey personnel, results in onerous and time-consuming data checking that can
significantly delay the survey’s completion. It is therefore more effective and efficient to apply rigorous data
collection methods than to try to clean or correct data once they are already collected.
1.5.2 Overview of training
All personnel involved in data collection, supervision and data entry require training to ensure reliable and
accurate data collection, completion of questionnaires and data entry.
A training workshop for survey personnel should be held as part of the survey preparation. The overall
objective of the training workshop is to provide field supervisors, data collectors and data entry personnel with
the knowledge and skills required to carry out a SARA survey in an accurate and reliable manner.
Upon completion of the training, participants should:
• be familiar with the key aspects of the survey and how it is conducted;
• understand their roles and responsibilities in the survey, including specific tasks, timelines and reporting
requirements;
• understand the critical content required to do their job effectively and possess the skills required to
undertake each of their activities;
• be aware of common issues that may arise during survey activities, and understand
troubleshooting/problem-solving strategies to address these issues;
• recognize the intrinsic value of good-quality data and be motivated to ensure data quality as part of
their activities.
Training should therefore focus on teaching the following to the participants:
• the overall purpose of the survey;
• the consequences of poor-quality data;
• how to administer and record responses using the SARA questionnaire, the purpose and meaning of
each question, and how to develop good rapport with the respondent;
• ethical issues involved in conducting a health facility survey, the importance of administering the
informed consent statement, and how to maintain the privacy and confidentiality of the respondent;
• problem-solving in the field;
• how to enter data for both paper and electronic questionnaires;
• how to collect geographical coordinates of visited sites using GPS;
• common data collection and data entry mistakes.
It is recommended that the duration of a training workshop, which covers both data collection and data entry,
last between 8 to 10 days (a sample agenda for the training of supervisors and data collectors is available in the
SARA implementation guide – Chapter 1: Planning).
Training should include a data collection pilot test in which survey personnel visit public and private sector
health facilities and collect data in the same way they would during actual fieldwork. This will not only provide
survey personnel with practical experience in collecting data, but will also serve as a check of the
appropriateness of the SARA questionnaire.
26
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
The trainer is usually the survey manager but could be a resource person with technical assistance from partner
implementing agencies. The participants should include all field supervisors, data collectors and data entry
personnel. For paper-based data collection, training on data entry is required. This can be held as a separate
workshop or session for data entry personnel, however, there may be some advantage in holding a combined
training session on data collection and data entry, since it will sensitize field supervisors and data collectors to
the difficulties in entering poor-quality data. It is also recommended that the members of the survey
coordinating group be invited to the introductory session of the training workshop to meet survey personnel
and discuss the survey methodology.
The training workshop should be held as close as possible to the initiation of data collection – immediate
departure for data collection can be scheduled if the survey manager has prepared well. Time lags between
training and data collection should be avoided so that survey personnel have better recall of the data collection
protocol.
1.5.3
Preparing for the training workshop
Planning the training workshop can require substantial time and preparation. Workshop preparations should
begin early in the survey development process and should run in parallel to other survey planning and
preparation activities. In preparing the training, it is essential to ensure that there is an adequate budget to
cover costs for the training venue, transport, materials, and a daily allowance and accommodation for
participants.
Select a training venue
A training venue should be selected based on the following criteria:
• availability of a room of appropriate size to hold the workshop;
• availability of essential technical resources (printer, photocopier, projector for presentations, electricity
to charge mobile EDCs, etc.);
• proximity to health facilities that can be surveyed during the data collection pilot test;
• accessibility by routine modes of transport;
• on-site or nearby refreshments and accommodation for out-of-town participants;
• reasonable cost.
It is useful to check with survey coordinating group members to see if a meeting room can be made available
for the training workshop at low or no cost.
Schedule dates of the training workshop
The training workshop should be scheduled close to the anticipated start of data collection. Do not plan the
workshop during a time when weather or other conditions may delay the initiation of data collection. All survey
personnel must attend the workshop and should be advised of the dates as early as possible. Invitations to
attend the introductory session of the workshop should also be sent to survey coordinating group members.
Plan data collection pilot test
During the data collection pilot test, data collection team will visit at least three health facility (from different
types) and collect data by following the survey procedures. It is recommended that each team visit one public
health facility and one private health facility during the pilot test. The participation of pilot sites should be
secured well in advance of the training workshops. The appointments should be made in advance and
reconfirmed before the training session, avoiding peak periods when health facilities may be busy with patients.
27
1. Overview
Prior to the training workshop a written schedule should be prepared for each data collection team,
indicating the time and location of each health facility visit, and including the name and contact details of
the person in charge at the facility. The schedule should also contain the survey manager’s telephone
number so that survey personnel can call if there is a query or problem.
Secure equipment
All necessary equipment should be procured prior to the training session. This includes:
• projector, computer, etc. for the training session;
• pens, notepads, clipboards;
• mobile EDCs loaded with software and electronic forms;
• GPS devices for data collection teams;
• mobile phones for data collection teams to carry during the pilot test;
• access to a printer and photocopier for reproducing the SARA questionnaire.
Prepare training materials
Each training participant should receive:
• one copy of the SARA questionnaire;
• one copy of the SARA data collectors’ guide;
• training hand-outs.
In addition, sufficient copies of the SARA questionnaire should be available for use in the pilot test.
1.5.4 Conducting the training workshop, including the data collection
pilot test
The SARA data collectors’ guide is available in the SARA Implementation Guide – Chapter 5: Data collector’s
guide. The guide provides:
• an overview of data collection processes;
• general guidance on interviewing practices and techniques;
• detailed explanations and definitions for each question in the questionnaire to provide a uniform
understanding of the meaning of each question and response choices, and to improve the consistency of
the data collected by different data collectors in different facilities.
This manual should be used during the training of all data collectors. In addition, slide presentations and
accompanying hand-outs to complement the SARA data collectors’ guide are available as tools for trainers to
use during the training workshop.
The quality of data collection is controlled at several points in the data collection process. The first point of
quality control is the thorough training of data collectors and the exclusion from fieldwork of any trainees
who do not exhibit competency in applying the data collection questionnaires at the end of training.
Conducting the data collection pilot test
During the pilot test, data collection teams and their field supervisors will visit health facilities and collect data
in the same way they would during the actual survey. Each field supervisor and data collector should complete
their own SARA questionnaire to gain hands-on experience. Field supervisors should also supervise and watch
28
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
out for common mistakes. It may be necessary to hold a preliminary pilot test with field supervisors to ensure
that they are sufficiently knowledgeable about the survey protocol to supervise data collectors and identify
mistakes. During the pilot test, any questions or uncertainties should be noted for clarification during the
training workshop.
The data collection pilot test also serves as a pre-test of the questions in the SARA questionnaire and should
help to highlight any country-specific adaptations that should be made to the survey including issues such as
question format, wording and order. The pilot test allows for an opportunity to uncover any defects in the
questions, glitches in wording of questions, lack of clarity of instructions, etc. The survey questionnaire should
be piloted in all languages in which it will be administered. In addition to testing the paper questionnaire, the
pilot test also tests field logistics, supervisory capacity and the application functionality for electronic data
entry.
1.5.5
Finalizing the questionnaire
After piloting the SARA questionnaire, changes should be made to its format and/or content based upon any
issues discovered during the piloting phase. All changes must be made to both paper and electronic versions of
the questionnaire.
29
1. Overview
1.6
Preparing for data
Collection in the field
The success of the SARA survey depends largely on the data collectors in the field, who are gathering and
recording accurate, reliable data. Data collection requires careful planning and preparation, involving the
following activities:
• planning the data collection visits
• preparing materials and tools for data collectors
• arranging transport and regular communications.
1.6.1
Planning the data collection visits
Who? Survey manager
The survey manager is responsible for planning the data collection visits. Before data collection starts, a
schedule of visits to health facilities should be prepared for each survey area. The number of days required to
collect the data can be estimated on the basis of the number of facilities to be visited in each geographical area,
the distance between them and the mode of transport available. In general, two data collectors will require
two hours plus travelling time for data collection in each facility.
Prepare a letter of introduction
Who? Survey manager
A letter of introduction from the survey manager is invaluable in introducing field supervisors – and later data
collectors – to staff in the health facilities being surveyed. The survey manager should prepare a letter of
introduction containing the following information:
• the name of the organization conducting the survey and the name of the survey manager
• contact details
• the purpose of the study
• the names of the data collectors who will visit the facility
• the time required for data collection in each facility.
The letter should also provide reassurance that the anonymity of the respondent will be maintained. The
survey manager should provide field supervisors with sufficient signed copies for use during both the
scheduling of field visits and the data collection visits.
Make initial contact with health facilities
Who? Field supervisors
It is essential that good relations be established with the person in charge of each facility to be surveyed, since
they will have to set aside considerable time to provide information for the survey. Ideally, field supervisors
should visit the heads of facilities personally, in advance, to seek their permission for data collection in their
facility. Field supervisors should show them the letter of and introduction, and make an appointment for data
collection on a date and at a time that is convenient for the head of the health facility, avoiding peak periods
30
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
when he or she may be busy with patients. Field supervisors should note the contact person’s name and
telephone number at each health facility. If visits are not possible, then those in charge of the facility should be
contacted by phone. The day before the scheduled data collection visit, field supervisors should telephone the
health facility to confirm the appointment.
The following checklist should be used by field supervisors when contacting health facilities.
Contact each health facility (sample and backup) to introduce the survey.
Introduce the survey using the letter of introduction
Make an appointment for data collection at a date and time that is convenient for the facility, avoiding
peak hours. Allow two hours for data collection at a primary level facility, plus travel time. For larger
facilities and hospitals, allow for additional time.
Note the name and telephone number of the contact person at each health facility.
Explain about the possibility of a second visit for 'validation', which should ideally take place in 10% of
the sampled health facilities.
Before data collection starts, telephone each health facility to confirm the appointment.
Prepare a schedule of data collection visits
Who? Field supervisors
Field supervisors are responsible for preparing a written schedule for each data collection team. For each
facility, the schedule should include the following:
• date and time of appointment
• name of facility
• contact person
• location
• administrative unit
• unique ID number for the facility (provided by survey manager)
• name and contact details of a backup facility.
EXAMPLE OF A SCHEDULE FOR DATA COLLECTION VISITS
Survey area: Region 1
Data collection Team 1
Date/time of
Name of
appointment
facility
20 April 2012
ABC
10:00
health
centre
Data collection Team 2
Date/time of
Name of
appointment
facility
Contact
person
Mrs Nguyen
Contact
person
Location
45 Main
Street
Eastern
City
Tel: +22
414 000
Location
Managing
authority
Private
Managing
authority
ID number
01234
ID number
Backup site
contact details
XYZ health
centre
59 Main Street
Eastern City
Mr Shah
Backup site
contact details
31
1. Overview
1.6.2
Preparing materials and tools for data collectors
Finalize and print questionnaire
Who? Survey manager
Following the data collection pilot test conducted as part of the training workshop, the survey manager should
review and, if necessary, revise the SARA questionnaire. Both the paper and electronic versions of the
questionnaire will need to be updated. Once the questionnaire has been finalized, the survey manager will
need to print sufficient copies and also deploy the electronic forms to the mobile data collection devices.
Prepare data collection forms for each facility to be visited
Who? Field supervisor
The survey manager should provide the field supervisor with a separate questionnaire (data collection form)
for:
• each sample health facility in the assigned survey area
• each backup facility
• each validation visit.
The survey manager should also provide the field supervisor with a list of the sample facilities in the survey
area. Ideally, about 10% of the sampled facilities should be visited a second time for validation. The field
supervisor will identify the validation sites by randomly selecting at least one public facility and one private
facility from the list of sample facilities.
The field supervisor should prepare the data collection forms for each facility by completing the front page of
the form (see Figure 1.6.1) with the identifying information of each sample facility, backup facility and
validation facility, i.e. completing the following fields:
• name of health facility
• health facility unique ID
• name of town/village
• region and district
• type of facility
• managing authority
The following fields should not be completed by the field supervisor, as these will be completed by data
collectors during the facility visits:
• date;
• name(s) of person(s) who provided information;
• name(s) of data collectors.
The verification at the top of the page should only be completed once the data collection form has been
completed.
32
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
FIGURE 1.6.1: FRONT PAGE OF THE SARA DATA COLLECTION FORM
The field
supervisor
should
complete this
section before
distributing
to data
collectors
Arrange for storage of completed questionnaires
Who? Field supervisor and Survey manager
Field supervisors should arrange to store completed questionnaires until all fieldwork is completed, at which
time they are transferred to the survey manager. A copy of all paper forms should be made by the field
supervisor, and all paper forms should be stored in sealed plastic bags to prevent damage. Electronic forms
should be synchronized daily to a central computer and a copy of all records should be stored on a memory
card as backup. Field supervisors should always keep a copy of all data collection forms, in case those sent to
the survey manager are lost or damaged. The survey manager should arrange for the safe storage of all
completed forms in secure conditions for an indefinite period, in the event that data need to be checked at a
later date.
33
1. Overview
Prepare materials and tools for data collectors
Who? Field supervisor
Data collectors need to bring tools and information with them on each day of data collection. Field supervisors
should prepare resource kits containing all needed items for each data collection team. Before each day of data
collection, the field supervisor should ensure that the data collectors have all the necessary tools and
information with them including the following.
A list of data collection teams and contact information.
Contact details of the field supervisor, including a mobile
phone number to call in case of difficulty in the field.
A schedule of visits to survey sites.
Contact details of the sites to be visited.
Details of backup facilities to be visited if scheduled visits are not possible.
Copies of letter of introduction.
Data collector’s guide and relevant hand-outs.
A SARA data collection form for each health facility to be visited that day.
A SARA data collection form for each backup site that may need to be visited that day.
An EDC (fully charged and loaded with the SARA questionnaire), batteries and power cable
A memory card for data backup (if applicable, depending on EDC selected) or USB key.
A fully charged and accurately configured GPS unit.
Pens (pencils should not be used to record data), a clipboard and other supplies.
A notebook to record any significant events or findings.
A field allowance for local expenses.
An identity document with a photograph for each data collector.
A mobile phone for each team and credit.
Where feasible, each data collection team should also be equipped with a mobile phone and credit to contact
the field supervisor. Additional supplies may include a local map and extra batteries.
1.6.3 Arranging transport and regular communications
Arranging transport
Who? Survey manager or Field supervisor
Once all the survey sites are known, the survey manager or field supervisor should arrange transportation
according to the number of sites to be visited, the number of teams going into the field, and the number of
people per team.
34
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
Arranging regular communications
Who? Survey manager and Field supervisor
Throughout the data collection process, field supervisors should be available to provide advice to data
collectors and answer any questions they may have. Providing data collectors with their field supervisor’s
mobile phone number, when feasible, is one way of ensuring timely communication.
Data collectors should also meet with their field supervisor on a regular basis so that completed forms can be
checked and any issues can be resolved. Ideally, this should occur at the end of each day of data collection so
that errors do not carry over into future data collection visits. In addition, data collectors will be better able to
recall the data collection visit, which may be useful in clarifying erroneous or illegible data. During data
collection, data collectors should record how problems were solved or how data collection was simplified.
These notes should be reviewed with the field supervisor during the debriefing.
The survey manager should also be available throughout the data collection process to respond to questions
from field supervisors, and the survey manager should provide field supervisors with his/her mobile phone
number for this purpose. Ideally, the survey manager should visit each survey area during data collection to
supervise activities. If this is not possible, he or she should arrange for regular communications with each field
supervisor to receive updates on the data collection process.
35
1. Overview
1.7
Data collection in the field
This chapter describes procedures for data collection in the field. Table 1.7.1 shows the activities involved for
each day of data collection.
TABLE 1.7.1: DAILY ACTIVITIES FOR DATA COLLECTION
When?
What?
Who?
Before going out
to collect data
each day
Check that the data collection teams have all the
materials necessary for field visits and confirm
transport arrangements
Field supervisors/data collectors
Call each facility to be visited and confirm
appointment
Field supervisors
Introduce survey team and remind facility staff of the
purpose of the visit
Data collectors
Verify and complete the SARA questionnaire
Data collectors
Check that all data are entered on the SARA
questionnaire before leaving the facility.
Data collectors
Conduct meeting between field supervisors and their
data collectors, and discuss any difficulties
Field supervisors/data collectors
Review each SARA questionnaire and clarify
missing/unreliable information
Field supervisors
Sign, copy and store all checked data collection forms
Field supervisors
On arrival at the
facility
At the end of each
day
Each step of data collection is described below according to the personnel responsible, namely field supervisors
and data collectors.
1.7.1
Field supervisors: fieldwork responsibilities
Field supervisors are responsible for ensuring the accuracy and reliability of data collection. This involves the
following activities.
Field supervision
Field supervisors should meet with their data collectors at the end of each day to check completed data
collection forms, get feedback on the data collection process and resolve any problems. They should visit the
health facilities regularly with the data collection teams to ensure that the agreed procedures are being
followed.
Daily check of completed SARA questionnaires
It is important that field supervisors review completed SARA questionnaires at the end of each day to check
that the data are complete, consistent and legible. Once the team has left the field, it becomes difficult to
verify information that may be missing or incomplete.
The supervisors should highlight any missing or unreliable information on the form and identify the source of
the problem. If necessary, data collectors should return to the facility to collect any further data required. Once
the field supervisor is satisfied with the completeness and reliability of a SARA questionnaire, he or she should
36
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
sign the form in the designated place to record that it has been checked. Forms should then be safely stored
until completion of data collection, at which time they are transferred to the survey manager.
Validation of data collection
Field supervisors should validate data collection by repeating the survey, using a “light” version of the
questionnaire (sub-selection of sections) at selected health facilities (about 10%) and checking their results
against those of their data collectors. Where possible, health facilities visited for validation should be selected
at random. Ideally, the validation should be done on the same day as data collection (soon after the data
collectors have left the facility) to avoid changes in the availability of the items. Any discrepancies between the
results of the field supervisor and those of their data collectors should be discussed with the data collectors,
and the data collection protocol should be clarified as necessary. Any problems that cannot be resolved in the
field should be discussed with the survey manager.
This validation can also be conducted by the entity in charge of the survey quality assurance. The supervisors
ease access to the data collected by the data collection teams in view of the data files comparisons.
Storing of completed SARA questionnaires
Completed paper questionnaires should be copied and stored in sealed waterproof plastic bags, in a location
that is protected from moisture, direct sunlight, rodents and insects. Originals should be stored in a separate
location from copies. Electronic questionnaires should be synchronized with a central computer and saved both
on the computer hard drive and on an external memory card for safe keeping. All original data collection
questionnaires, including those for validation visits, should be transferred to the survey manager upon
completion of fieldwork. Field supervisors should retain the copies for use in the event that the originals
become lost or damaged.
In order to accomplish these tasks, each field supervisor should have the following materials:
A full list of sample sites (and backup sites) for survey area and contact details.
An assignment of sites by data collection team.
A list of data collection teams and contact information when in the field.
A schedule of visits to survey sites and contact details of the sites.
Copies of letter of introduction.
Copies of the supervisor and data collector’s guides and other relevant documents/material.
Extra copies of the SARA data collection form.
A data collection form for data validation at each facility that may need to be visited that day.
A fully-charged laptop computer with appropriate software (CSPro)
Extra EDCs as backup (fully charged and loaded with the SARA questionnaire) in case of loss or damage,
with extra batteries and power cables.
Extra memory cards for data backup or USB keys (depending on the EDC used).
Extra GPS units as backup (fully charged and accurately configured).
Pens (pencils should not be used to record data), a clipboard and other supplies.
A notebook to record any significant events or findings.
A field allowance for local expenses.
An identity document with a photograph.
A cell phone with credit
37
1. Overview
1.7.2 Data collectors: fieldwork responsibilities
Before visiting the facilities each day
Before visiting the facilities each day, data collectors should check that they have all the materials they will
need for data collection.
A list of data collection teams and contact information.
Contact details of the field supervisor, including a mobile
phone number to call in case of difficulty in the field.
A schedule of visits to survey sites.
Contact details of the sites to be visited.
Details of backup facilities to be visited if scheduled visits are not possible.
Copies of letter of introduction.
Data collector’s guide and relevant hand-outs.
A SARA data collection form for each health facility to be visited that day.
A SARA data collection form for each backup site that may need to be visited that day.
An EDC (fully charged and loaded with the SARA questionnaire), batteries and power cable
A memory card for data backup (if applicable, depending on EDC selected) or USB key.
A fully charged and accurately configured GPS unit.
Pens (pencils should not be used to record data), a clipboard and other supplies.
A notebook to record any significant events or findings.
A field allowance for local expenses.
An identity document with a photograph for each data collector.
A mobile phone for each team and credit.
Where feasible, each data collection team should also be equipped with a mobile phone and credit to contact
the field supervisor. Additional supplies may include a local map and extra batteries.
On arrival at the facility
On arrival at the health facility, data collectors should do the following.
• Introduce themselves and remind health facility staff of the survey’s purpose as well as the scheduled
data collection visit. Data collectors should also thank the staff for their cooperation and, if necessary,
remind them that the respondents' identity will be kept confidential.
• Check that the facility information on the first page of the SARA questionnaire is complete and correct,
informing the field supervisor at the end of the day if there were any inaccuracies.
• Fill in the date and names of the data collectors on the cover page.
• Take the GPS coordinates of the health facility.
• Obtain informed consent to begin the survey.
• Fill out the SARA questionnaire making sure to speak to the most knowledgeable person in the health
facility for each section of the questionnaire. One data collector should complete the SARA data
collection paper form and another should complete the SARA electronic form, paying close attention to
the instructions on the forms. Data collectors should not leave the SARA data collection form at the
facility to be filled in later. A separate SARA data collection form should be completed at each facility.
38
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
Before leaving the facility
Before leaving the health facility, data collectors should do the following.
• Double-check that the data collection form is legible, accurate and complete.
NOTE: Backup facilities to be visited are identified in the schedule. The field supervisor should determine
when it is necessary to visit a backup facility. When visiting a backup facility, questionnaires should be
completed in exactly the same way as in other facilities, making sure to complete the SARA data
collection form that corresponds to that facility.
• Thank staff at the facility for their participation.
At the daily meeting with the field supervisor
At the end of each day, data collectors should meet with the field supervisor and do the following:
1.
2.
3.
4.
5.
6.
7.
back-up electronic data on the memory card in the EDC
submit the data collection forms and files completed that day
transfer data from the EDC to the central computer
report on the activities of the day
recharge the battery of the EDC to be ready for the next day
check the battery life of the GPS unit and get a second set of batteries if necessary
recharge mobile phone if necessary.
Data collectors should alert their field supervisor of any problems or uncertainties regarding data collection
procedures. They should also report any problems with electronic equipment and arrange to get replacements
if necessary.
1.7.3
Ensuring data quality
The quality of the information that the SARA survey generates depends on the accuracy of data collection. The
survey manager has overall responsibility for the quality of the data, although all survey personnel have a role
to play in ensuring the accuracy of the data collected. The field supervisors and data collectors should receive
regular supervision. Rigorous enforcement of data collection procedures will pay off with the ease with which
data entry and analysis occur. The following steps will also help to ensure greater accuracy of data collection.
1.
Ensure that there is thorough preparation and training as a first step in minimizing errors.
2.
Establish procedures to check for data completeness, consistency, plausibility and legibility in the field
when it is still possible to correct errors or to fill in missing information. Field supervisors should
review data collection forms every day after completion of the fieldwork and resolve any problems
before the next day of data collection.
3.
Plan random checks to ensure the quality of data collection. It is recommended that an entity returns
to randomly-selected health facilities (10%) to collect the same data so as to check the accuracy of the
first data set. Ideally, the validation should be done on the same day as data collection (soon after the
data collectors have left the facility) to avoid changes in the availability of survey items.
4.
Double-check all completed SARA questionnaires; verify any suspicious, incomplete or illegible data
prior to the initiation of data entry.
39
1. Overview
1.8
Data entry and processing
If data are collected on paper forms, they must be entered electronically before proceeding with data
processing and analysis. If data are collected in electronic forms, then one can proceed directly to the data
processing step.
Once in electronic form, the data need to be checked for accuracy, completeness and consistency before the
data set can be finalized. Any errors or inconsistencies must be flagged and resolved prior to analysis. The
purpose of editing is to eliminate omissions and invalid entries, e.g. by changing inconsistent entries, and
should be kept to a minimum: data should never be changed to conform to expectations. It is good practice to
always preserve an unedited copy of the data set and to document in detail the data editing process.
Finally, once the data have been checked and verified, it is customary to export the final data set in some
commonly-used file format, such as a spreadsheet file format or CSV (comma-separated values). This is useful
for sharing the data with other parties, and to perform analysis in other statistical software packages.
1.8.1 Data entry
Any data collected on paper must be entered electronically before it can be processed and analysed. With
input from technical members of the survey coordinating group, the survey manager selects the appropriate
data entry software and sets up a data entry operation. Transferring the data from paper to electronic form can
be a source of error; therefore, it is important to have the appropriate data validation processes in place to
ensure accurate data entry. If electronic data collection has been used, the data already exist in an electronic
format and this step is complete.
Selecting data entry software
When selecting data entry software, there are two main principles to consider:
1.
use software that speeds up data entry and minimizes errors
2.
have a thorough knowledge of the software selected.
Keeping these principles in mind when thinking about data entry software options helps to narrow down the
potential options and results in selection of an appropriate solution.
While it is possible to use many types of software for data entry (including statistical programs, database
management systems and spreadsheets), it is recommended that a specialized data entry software such as
CSPro be used to minimize the possibility of entry errors and to facilitate validation.
Statistical software
Statistical software package programs are software packages that are specialized for data analysis. Some
include data entry and data checking functions in addition to data analysis (e.g. CSPro), while others are useful
primarily for data analysis and visualization. Some advantages of using a software package with built-in data
entry and verification functionality are (1) data entry clerks are less likely to make mistakes when entering data,
and (2) mistakes are much easier to identify and fix. In particular, the software can be programmed to provide
5
a highly-structured data entry environment so that only valid values are accepted and skip patterns are
automatically integrated. In addition, such software facilitates independent data verification, in which the data
are entered manually twice and differences are later reconciled. Once the data have been entered, it may be
Skip patterns are a particular type of survey branching logic that will jump a respondent over a group of questions
that isn’t relevant to them.
5
40
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
necessary to use a different statistical software package to perform analysis, depending on whether the
package offers the desired data analysis and graphing functions. These types of tools require some advanced
technical knowledge, but overall result in improved data quality.
For SARA, the recommended data entry software is CSPro: a statistical software package with built-in data
collection/entry functionality that allows for speedy data entry while also providing sufficient checks and data
validation to ensure quality data. CSPro includes all the necessary functionality for SARA and can be
downloaded free of cost.
Database management systems
A database management system is an application that allows the creation and management of databases,
including storage and retrieval of data. There are different types of databases, but the most popular is a
relational database that stores data in tables where each row ("record") in the table holds the same sort of
information. Each record has a unique identifier ("key"), which allows retrieval of information from related
tables. Databases are more difficult to set up than spreadsheets, but they allow more sophisticated data
retrieval and search. In addition, it is possible to write scripts using a query language such as SQL to perform
simple data checking and other functions. However, these programs are designed mainly for data storage and
retrieval, and are usually not designed to facilitate manual data entry. For the SARA survey, it is recommended
that a database system be used to store the data once it has been entered, but not for data entry itself.
Spreadsheets
Spreadsheet programs offer the most basic option for data entry. Although spreadsheets are easy to use and
many people are knowledgeable about how to use the software programs, there are many disadvantages of
using spreadsheets for data entry: it is very easy to make a mistake, data entry is slow, and there is no built-in
checking for valid values. As a result, for the SARA survey, it is not recommended to use a spreadsheet for data
entry unless there are no other viable options. Spreadsheet software is often useful to view data once it has
been digitized and stored in a database.
Preparing for data entry
The data entry application must be designed using the selected data entry software. Valid values should be
defined for certain responses and entry of data should be restricted to these values alone. Furthermore, special
keys for missing data should be included in the value set and may use a standard identifying digit. Open-ended
questions or the selection of the broad category of "other" can also be programmed to allow for entering the
written response. This keying of open-ended questions will require the manual coding of these responses at
some future date.
A centralized system for data entry should be set up, with one or more groups of data entry clerks managed by
a supervisor. The number of data entry clerks will depend on a number of factors, including (1) budget, (2)
timeline, (3) the availability of qualified personnel, and (4) the availability of computers and other equipment
for data entry. Generally, the more data entry clerks there are, the quicker the data can be entered.
Part of the management and organization of a data entry operation requires establishing a specified work
schedule. Monitoring the productivity of the individual data entry clerks should be part of a data entry system
as well. Like other process, the data entry process requires good organizational and project management skills.
41
1. Overview
Entering data
Data should be entered using the software that has been selected.
In the data entry process, it is important to consider the following issues.
Missing data
In general, it is not good practice to use blanks as missing data codes. Missing data can arise in a number of
ways, and it is important to distinguish among these different instances. There are at least five missing data
situations, each of which should have a distinct missing data code.
• Refusal/no answer. The subject explicitly refused to answer the question or did not answer the question
when he or she should have.
• Don’t know. The subject was unable to answer the question, either because he or she had no opinion or
because the required information was not available (e.g. a respondent could not provide information on
the functionality of equipment due to inaccessibility).
• Processing error. For some reason, there is no answer to the question although the subject provided
one. This can result from interviewer error, incorrect coding, machine failure or other problems.
• Not applicable. For one reason or another, the subject was never asked the question. Sometimes this
6
results from “skip patterns” that occur (e.g. for facilities that do not have a generator, questions
regarding generator functionality and availability of fuel would be not applicable).
• No match. This situation may arise when data are drawn from different sources (e.g. a survey
questionnaire and an administrative database), and information from one source cannot be located.
Selecting missing data codes
Missing data codes should always match the content of the field. If the field is numeric, the codes should be
numeric, and if the field is alphabetic, the codes may be numeric or alphabetic. Most researchers use codes for
missing data that are above the maximum valid value for the variable (e.g. 97, 98 and 99). Missing data codes
should be standardized so that only one code is used for each missing data type across all variables in the data
file or across the entire collection if the study produced multiple data files.
Not applicable and skip patterns
Handling skip patterns is a constant source of error in both data management and analysis. On the
management side, deciding what to do about codes for respondents who are not asked certain questions is
crucial. "Not applicable" codes, as noted above, should be distinct from other missing data codes. It is not good
practice to leave the record blank. Data set documentation should clearly show for every item exactly who was
asked and who was not asked the question. At the data cleaning stage, all “filter items” should be checked
against items that follow to make sure that no one provides answers to the item who should not, and that
those who did not answer the item have the correct kind of missing data code.
Skip patterns are a particular type of survey branching logic that will jump a respondent over a group of questions
that isn’t relevant to them.
6
42
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
1.8.2 Data processing
After data entry, data should be checked for inconsistencies and possible errors. If the data are collected
electronically, field supervisors should, for the duration of the data collection phase, check data for all health
facilities that were visited each day. If the data are transferred from paper to electronic versions, the data
should be checked for entry errors. It is particularly important to check that the facility ID items such as the
facility number, name, location, facility type and managing authority have been entered correctly, and that
there are no inconsistent or missing data. Usually errors can be resolved by reviewing all of the information
provided by a respondent or by referring to the paper copy of the questionnaire responses.
Edit and correct data
The purpose of editing is to make the data as representative of the real life situation as possible; this can be
done by eliminating omissions and invalid entries, and by changing inconsistent entries. Below are some
important principles that should be followed.
• The fewest number of changes should be made to the originally recorded data. The goal is to make a
record or questionnaire acceptable, not to make it conform to what one thinks should be acceptable.
• For certain items it may be acceptable to have a "not reported [NR]" or “not stated [NS]" category. Thus,
in case of an omission or an inconsistent, impossible or unreasonable entry, a code for "NR" or "NS" can
be assigned.
• Obvious inconsistencies among the entries should be eliminated.
• Providing corrected values for erroneous or missing items should be supplied by using other values as a
guide, and always in accordance with specified procedures.
• Specifications for editing the questionnaire data should be developed at the same time as the
questionnaire itself.
Remove any duplicate records
It is possible that a facility has been entered in the database twice and thus the duplicate record must be
removed. For any records that are identical, one should be removed. If two records appear to be duplicates
according to facility name, but do not contain the same data, a list of criteria must be used to determine if it is
a true duplicate. The following data elements could be used as the criteria for determining duplicates:
• district
• facility code/name
• GPS coordinates
• facility type
• managing authority
• interviewer's name.
If these are all the same it is safe to consider the records as duplicates. At this point, the most complete record
should stay in the data set. If both records are complete, the record with latest time stamp should be kept.
Check validity of GPS coordinates
GPS coordinates should be checked to ensure that they fall within the boundaries for the country and region.
Sometimes latitude and longitude coordinates can be entered incorrectly (they can be inversed and +/- signs
can be reversed, or an incorrect format can be entered). All GPS coordinates should be double-checked to
ensure they are valid for the area being surveyed.
43
1. Overview
Check validity of responses
Data entry software often has built-in functionality to check data as it is being entered, such as range checks
and within-record consistency checks. Data editing programs can be written to check the validity of responses
after data entry, including whether the data follow the appropriate skip patterns.
Recode values for “other”
Questions where "other" is a possible response option should be checked, and the written responses reviewed
to determine if the response actually corresponds with one of the pre-coded options. If this is the case, these
responses should be recoded to the appropriate response category.
Review comments sections
At the end of the survey, there are several questions allowing for the interviewer to provide comments. Please
review these sections for any relevant information.
Data validation and verification
Verification is a process of double entry of the same questionnaire and comparing the responses. This can
either be the paper questionnaire entered twice or validation between paper and electronic versions if
electronic data collection is used with paper questionnaires as a backup. Differences in keyed data of the same
questionnaire need to be reconciled. A system of verification can virtually assure that the information
presented in the questionnaire is faithfully keyed. Verification can be dependent or independent. Dependent
data entry uses one data file and reconciles any identified error with the original data file. Independent
verification is the process of keying to fully independent data files of the same questionnaire or cluster and
comparing the two files. A report of inconsistencies is issued and the differences between the two data files
must be fully reconciled.
Data clean-up
Before finalizing and exporting the data set, the following steps should be taken to clean the data set as
applicable.
Rename the variables
The variable should be named according to the corresponding question number in the survey. This may already
be the case if the database is set up in this way. If electronic software is used, variables are often assigned
names based on category headings, which are sometimes long and cumbersome to use and do not provide a
good description of the variable and thus require renaming. For example, the variable “_2_001_date” which
corresponds to question 001 in the survey will be renamed “q001”.
Label the variables
Adding a label to a variable allows a text description to be associated with the variable name. For example, the
new variable “q001” can have a label called "date." This enables the user to more easily identify what each
variable represents.
Remove variables for which no data exist
If data are collected using electronic software, there may be variables in the data set which are actually
instructions from the questionnaire and do not include any data. These variables must be removed from the
data set.
44
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
Define the data type associated with variables
There are generally two data types associated with variables: numeric and string. Numeric variables are simple:
they contain numbers. String variables contain text that can contain any characters on the keyboard: letters,
numbers and special characters. It is important to define each variable according to the appropriate type in
order for statistical analysis to be carried out on the data.
Adjust variables in which two numeric responses have been chosen
All variables with numeric responses should contain only one response. It is possible that a single select
numeric variable is erroneously assigned two values. In some programs, this is represented in the data set as
#;# and causes the variable to be categorized as a string variable due to the non-numeric character. These
values must be imputed so that only one numeric value is recorded and then the data type of the variable must
be converted from a string to a numeric value.
1.8.3
Exporting the data set
Once the data set has been processed and verified, it is good practice to export the finalized data set into some
commonly used file format such as a spreadsheet format or CSV (.csv). This is useful when sharing the data and
for analysis of the data using statistical software packages.
45
1. Overview
1.9
Data analysis
Once data have been verified, data analysis can begin. There are many different types of results that can be
obtained from surveys. The types of analysis used depend to a large extent on the design determined in the
planning phase of the SARA survey. Some data analyses are standard and are included in most survey reports.
However, not all of the analyses of the survey data need to be included in the final report, as the focus should
be on the most important and relevant results. Therefore, survey managers should generate the full range of
survey results, and together with the survey coordinating group, select the most significant findings for
inclusion in the final report. It is only by conducting a complete analysis of the survey data that it can be
assured that important findings have not been overlooked. Based on the initial set of results from the standard
analyses, there is often further analysis in areas of interest. Following data analysis, a meeting with the survey
coordinating group should be held to assist in interpreting the results and developing recommendations.
Survey indicators are important in providing crucial information for informed policy choices, especially to
decision-makers, programme planners and policy-makers. Serving as baselines, indicators are important for
setting goals and targets for the future and allow for a certain level of comparability between surveys of
different location and time period. Moreover, indicators help place focus on predetermined areas of a survey
that are deemed to be most useful, relevant and important to the current health system. Having a consistent
indicator set also contributes to standardized analytical reporting.
SARA uses both tracer indicators and composite indicators in data analysis. Tracer indicators aim to provide
objective information about whether or not a facility meets the required conditions to support provision of
basic or specific services with a consistent level of quality and quantity. Summary or composite indicators, also
called indices, are a useful means to summarize and communicate information about multiple indicators and
domains of indicators. Composite indices are useful to help get an overall view of the situation and to
summarize multiple pieces of information. For SARA, composite indices are useful to compare districts or
regions or to look at change over time. However, composite indices also have limitations. It can be difficult to
understand the individual factors contributing to an index score, and thus it is important to have information
on individual indicator items in addition to composite index scores.
The following sections provide an overview of how to calculate SARA indicators and indices.
1.9.1 Calculating the service availability indicators and index
Overview
An important note regarding service availability: although this information is collected through the SARA
questionnaire, these indicators should not be calculated for a sample of facilities. Data must be available for
ALL facilities in an administrative unit in order to calculate service availability. All service availability measures
require data that link the numerator (e.g. number of facilities) to the denominator (e.g. population size). A
sample survey would not allow computation of the service availability indicators as it is not clear what the
corresponding population size to be used as the denominator should be.
The information needed to calculate service availability can be gathered from multiple sources in addition to
the SARA questionnaire, namely the HMIS and other routine information systems, and should be collated for all
facilities before calculating the service availability indicators. If SARA is implemented as a census, then it can be
used to calculate service availability.
Service availability is described by three domains of tracer indicators: health infrastructure, health workforce
and service utilization.
46
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
Health infrastructure indicators
• Facility density (number per 10 000 population): the facility density is primarily an indicator of
outpatient service access.
• Inpatient bed density (number per 10 000 population): inpatient bed density provides an indicator of
the inpatient services access. Paediatric beds (cots) are included, but maternity beds are excluded.
• Maternity bed density (number per 1000 pregnant women): maternity bed density provides an
indicator of access to delivery services. Data on maternity beds can be used calculate the density of
maternal beds per 1000 pregnant women per year. The denominator is estimated from the population
data. The indicator does not include delivery beds.
Health workforce indicator
• Health workforce density (number per 10 000 population): the health workforce density is the number
of core medical professionals per 10 000 population: physicians, non-physician clinicians, registered
nurses and midwives. This includes part-time physicians who are given the value of 0.5 in the scoring.
Service utilization indicators
In populations with poor or suboptimal health infrastructure, the service utilization rate is an indicator of
access.
• Outpatient service utilization (number of outpatient visits per capita per year): the number of visits for
ambulant care, not including immunization, over the total population.
• Inpatient service utilization (number of hospital discharges per 100 population per year, excluding
deliveries): this indicator provides additional information on the availability and access to inpatient
services.
These indicators must all be expressed as a percentage score compared with a target or benchmark. Table 1.9.1
shows the target and computation of each indicator. If the tracer indicator score exceeds the target, it is scored
as 100%.
TABLE 1.9.1: SERVICE AVAILABILITY INDICATORS
Domain
Indicator
Target*
Score (%)
(n / target, maximum 100)
Health infrastructure
a
Facility density
Number per 10 000 population (n)
2
n / 2 × 100
b
Inpatient bed density
Number per 10 000 population (n)
25
n / 25 × 100
c
Maternity bed density
Number per 1000 pregnant women (n)
10
n / 10 × 100
Number per 10 000 population (n)
23
n / 23 × 100
Health workforce
d
Core health workforce density
Service utilization
e
Outpatient service utilization
Outpatient visits per person per year (n)
5
n / 5 × 100
f
Inpatient service utilization
Hospital discharges per 100 per year (n)
10
n / 10 × 100
47
1. Overview
Health infrastructure targets and scores
The rationale for the targets can be summarized as follows.
Facility density (a): usually there is a country target, such as at least one facility per 5000 population, or two
facilities per 10 000 population. A major limitation is that this indicator does not take into account the size of
the facilities. The indicator is scored as n / 2 × 100% (maximum 100), where n is the number of facilities per 10
000 population.
Inpatient bed density (b): the global average is 27 per 10 000 (10). Lower- and upper-middle-income countries
have 18 and 39 hospital beds per 10 000, respectively (10). For SARA, an arbitrary benchmark of 25 per 10 000
is selected. The indicator is scored as n / 25 × 100% (maximum 100), where n is the number of inpatient beds
per 10 000 population.
Maternity bed density (c): under the assumption that there should be sufficient beds for all pregnant women
with an occupancy rate of 80% (to account for the uneven spread of demand over time) and a mean duration
of stay of 3 days, the target should be (1000 / 0.8) × (3 / 365) = 10 per 1000 pregnant women. The indicator is
scored as n / 10 × 100% (maximum 100), where n is the number of maternity beds per 1000 pregnant women.
An estimation for the number of pregnant women in the population can be derived from the CBR (crude birth
7
rate) for the country of interest and the following equations:
A = estimated number of live births = (CBR per 1000 × total population)
B = estimated live births expected per month = (A / 12)
C = estimated number of pregnancies ending in stillbirths or miscarriages = (A × 0.15)
D = estimated pregnancies expected in the year = (A + C)
E = estimated number of women pregnant in a given month = (0.70 × D)
F = estimated % of total population who are pregnant at a given period = (E / total population × 100).
Health workforce target and score
Health worker density (d): The published figure by WHO is 23 per 10 000 population (9). The indicator is scored
as n / 23 × 100% (maximum 100), where n is the number of core health workers per 10 000 population.
Service utilization targets and scores
Outpatient service utilization (e): in countries of the Organisation for Economic Co-operation and
Development (OECD), the average number of physician consultations per person per year is about six (10). For
SARA, the proposed benchmark is five visits per person per year. The indicator is scored as n / 5 × 100%
(maximum 100), where n is the number of outpatient visits per person per year.
Inpatient service utilization (f): in OECD countries, which have an ageing population, there are about 15
discharges per 100 population per year (11). For SARA, the proposed benchmark is 10 discharges per 100
people per year. The indicator is scored as n / 10 × 100% (maximum 100), where n is the number of hospital
discharges per 100 people per year.
The service availability index is calculated using the six above mentioned indicators. First, indices are calculated
for health services infrastructure, health workforce and service utilization. The calculations for creating those
indices are shown in Table 1.9.2 (please refer Table 1.9.1 for the definitions of indicators a–f). The service
availability index is the unweighted average of the three areas: infrastructure, health workforce and utilization:
[((a + b + c) / 3) + d + ((e + f) / 2)] / 3, and is a percentage score.
These equations can be found at:
http://www.who.int/reproductivehealth/publications/emergencies/field_manual_rh_humanitarian_settings.pdf,
Chapter 5, Annex 3.
7
48
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
TABLE 1.9.2: SERVICE AVAILABILITY INDICES
Index
Indicator
Target
Score
Average score of the three indicators:
facility density, inpatient bed density,
maternity bed density
100
(a + b + c) / 3
Health workforce index
Core health worker density
100
d
Service utilization index
Average score of the two indicators:
outpatient visits, hospital discharges
100
(e + f) / 2
Service availability index
Unweighted average of the three
areas: infrastructure, workforce and
utilization
100
[((a + b + c) /3) + d + ((e + f) / 2)] / 3
Health infrastructure index
Required data sources
Table 1.9.3 shows the required information and potential data sources for calculating service availability.
TABLE 1.9.3: DATA SOURCES
Information needed
Potential data source
List of all health facilities
MFL
Service utilization data
HMIS
Health workforce data
Human resources information system (HRIS)
Inpatient and maternity beds data
Varies by country
Population data (national and regional/district depending on
how results will be reported)
National Bureau of Statistics
Example calculation
Table 1.9.4 shows the data used for this example.
TABLE 1.9.4: EXAMPLE DATA
Data item
Value
Number of facilities
400
Number of inpatient beds
5500
Number of maternity beds
800
Number of core health workers
4600
Number of outpatient visits per year
Number of hospital discharges per
9 000 000
Population
3 000 000
Crude birth rate (CBR)
40
225 000
There are three main steps to calculate the service availability index.
49
1. Overview
Step 1. Calculate service availability indicators
The first step is to calculate the six service availability indicators. The following example (Table 1.9.5) shows the
equations used to calculate each of the six indicators using the example data values.
TABLE 1.9.5: CALCULATING THE INDICATORS
Indicator
Facility density (number per 10 000 population)
Value
number of facilities / population = n / 10 000
400 / 3 000 000 = n / 10 000
n = 1.33
Inpatient bed density (number per 10 000
population)
number of inpatient beds / population = n / 10 000
5500 / 3 000 000 = n / 10 000
n = 18.33
number of maternity beds / pregnant population* = n / 1000
Maternity bed density (number per 1000 pregnant
women)
800 / 96 600 = n / 1000
n = 8.28
*see Table 1.9.6 for how to calculate number of pregnant women
Health workforce density (number per 10 000
population)
number of core health workers / population = n / 10 000
4600 / 3 000 000 = n / 10 000
n = 15.33
Outpatient service utilization (outpatient visits per
capita per year)
number of outpatient visits per year / population = n
9 000 000 / 3 000 000 = n
n = 3.00
Inpatient service utilization (hospital discharges
per 100 population, excluding deliveries)
number of hospital discharges per year / population = n / 100
225 000 / 3 000 000 = n / 100
n = 7.50
TABLE 1.9.6: CALCULATING THE NUMBER OF PREGNANT WOMEN
50
A = Estimated number of live births = (CBR per 1000 ×
total population)
(40 / 1000) x 3 000 000 = 120 000
B = Estimated live births expected per month = (A / 12)
120 000 / 12 = 10 000
C = Estimated number of pregnancies ending in
stillbirths or miscarriages = (A × 0.15)
120 000 x 0.15 = 18 000
D = Estimated pregnancies expected in the year = (A +
C)
120 000 + 18 000 = 138 000
E = Estimated number of women pregnant in a given
month = (0.70 × D)
0.7 x 138 000 = 96 600
F = Estimated % of total population who are pregnant
at a given period = (E / total population × 100)
(96 600 / 3 000 000) x 100 = 3.22
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
Service availability indicators can each be displayed in a graph such as the one for health workforce density in
Figure 1.9.1.
40
32
30
20
26
20
15
Target
17
15
9
10
8
8
7
Overall
Urban
4
Peri-urban
0
Rural
Ov
e
Di ral
str l
i
Di c t 1
str
i
Di c t 2
str
i
Di c t 3
str
i
Di c t 4
str
i
Di c t 5
str
i
Di c t 6
str
i
Di c t 7
str
i
Di c t 8
str
Di ict
str 9
ic t
10
Core health workers per 10 000 pop
FIGURE 1.9.1: CORE HEALTH WORKERS PER 10 000 POPULATION
District
Step 2. Calculate service availability indicator scores
Next, use the values obtained from Step one to calculate the service availability indicator scores. The scores
compare the indicator to a target and are expressed as a percentage. Table 1.9.7shows the calculations for
each of the six service availability indicator scores.
TABLE 1.9.7: CALCULATING THE SERVICE AVAILABILITY INDICATOR SCORES
Domain
n
Target
Score (%)
(n / target) x 100 (maximum 100)
Health infrastructure
a
Facility density
1.33
2
(1.33 / 2) x 100
66.5
b
Inpatient bed density
18.33
25
(18.33 / 25) x 100
73.3
c
Maternity bed density
8.28
10
(8.28 / 10) x 100
82.8
15.33
23
(15.33 / 23) x 100
66.7
Health workforce
d
Core health workforce density
Service utilization
e
Outpatient service utilization
3.00
5
(3 / 5) x 100
60.0
f
Inpatient service utilization
7.50
10
(7.5 / 10) x 100
75.0
Step 3. Calculate service availability indices
Lastly, use the service availability indicator scores to create the health infrastructure index, the health
workforce index, the service utilization index and the overall service availability index. Table 1.9.8 shows these
four index calculations using the example data.
51
1. Overview
TABLE 1.9.8: CALCULATING THE SERVICE AVAILABILITY INDEX
Index
Indicator
Score (%)
Health infrastructure
index
Average score of the three indicators:
facility density, inpatient bed density,
maternity bed density
(a + b + c) / 3
(66.5 + 73.3 + 82.8) / 3 = 74.2
Health workforce
index
Core health worker density
d
66.7
Service utilization
index
Average score of the two indicators:
outpatient visits, hospital discharges
(e + f) / 2
(60.0 + 75.0) / 2 = 67.5
Service availability
index
Unweighted average of the three
areas: infrastructure, workforce and
utilization
[((a + b + c)/3) + d +
((e + f) / 2)] / 3
(74.2 + 66.7 + 67.5) / 3 = 69.5
The service availability indices can be displayed in a graph such as the one in Figure 1.9.2.
FIGURE 1.9.2: SERVICE AVAILABILITY INDICES
100
Score
80
69.5
74.2
66.7
67.5
Health
workforce
Service
utilization
60
40
20
0
General service
Health
availability infrastructure
Index
1.9.2 Calculating the general service readiness indicators and index
Overview
General service readiness is described by the following five domains of tracer indicators:
• Basic amenities
• Basic equipment
• Standard precautions for infection prevention
• Diagnostic capacity
• Essential medicines.
52
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
Each domain consists of a set of tracer items. Table 1.9.9 lists the tracer indicators for each domain.
TABLE 1.9.9: GENERAL SERVICE READINESS ITEMS AND INDEX
General service
domains
Tracer items
(a) Basic amenities
•
Power
•
Improved water source facility premises
•
Room with auditory and visual privacy for patient
consultations
•
Access to adequate sanitation facilities for clients
•
Communication equipment (phone or short-wave radio)
•
Access to computer with e-mail and Internet
•
Emergency transportation
•
Adult scale
•
Child scale
•
Thermometer
•
Stethoscope
•
Blood pressure apparatus
•
Light source
•
Safe final disposal of sharps
•
Safe final disposal of infectious wastes
•
Appropriate storage of sharps waste (sharps
box/container)
•
Appropriate storage of infectious waste (waste
receptacle with lid and plastic bin liner)
•
Disinfectant
•
Single-use, standard disposable or auto-disable syringes
•
Soap and running water or alcohol-based hand rub
•
Latex gloves
•
Guidelines for standard precautions
•
Haemoglobin
•
Blood glucose
•
Malaria diagnostic capacity
•
Urine dipstick - protein
•
Urine dipstick - glucose
•
HIV diagnostic capacity
•
Syphilis RDT
•
Urine pregnancy test
•
Amlodipine tablet or alternative calcium channel blocker
•
Amoxicillin (syrup/suspension or dispersible tablets)
(b) Basic equipment
(c) Standard
precautions for
infection prevention
(d) Diagnostic capacity
(e) Essential medicines
Domain score
(mean availability of items)
n / 7 × 100, where n is the total
number of items available in the
domain
n / 6 × 100 where n is the total
number of items available in the
domain
n / 9 × 100 where n is the total
number of items available in the
domain
n / 8× 100 where n is the total
number of items available in the
domain
n / 25× 100 where n is the total
number of items available in the
domain
53
1. Overview
General service
domains
Tracer items
Domain score
(mean availability of items)
•
amoxicillin tablet
•
Ampicillin powder for injection
•
Aspirin (capsules/tablets)
•
Beclometasone inhaler
•
Beta blocker (e.g.bisoprolol, metaprolol, carvedilol,
atenolol)
•
Carbamazepine tablet
•
Ceftriaxone injection
•
Diazepam injection
•
Enalapril tablet or alternative ACE inhibitor (e.g.
lisonopril, Ramipril, perindopril)
•
Fluoxetine tablet
•
Gentamicin injection
•
Glibenclamide tablet
•
Haloperidol tablet
•
Insulin regular injection
•
Magnesium sulfate injectable
•
Metformin tablet
•
Omeprazole tablet or alternative (e.g. pantoprazole,
rabeprazole)
•
Oral rehydration solution
•
Oxytocin injection
•
Salbutamol inhaler
•
Simvastatin tablet or other statin (e.g. atorvastatin,
pravastatin, fluvastatin)
•
Thiazide (e.g. hydrochlorothiazide)
•
Zinc sulphate (tablet or syrup)
General service readiness index
(Mean score of the five domains)
(a + b + c + d + e ) / 5
Required data source
Facility assessment information is needed to calculate general service readiness; the source for this information
is the SARA survey.
54
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
1.10 Data archiving
Data archiving includes the acquisition, preservation, documentation, cataloguing and dissemination of
microdata. 8 Archives are useful for promoting research and instruction in the social sciences; ensuring the
continued viability and usability of microdata in the future; and providing equitable access to these data within
the framework of the national legislation in the interest of all citizens, by protecting confidentiality and
following international recommendations and good practices.
Fully documenting and archiving data sets helps ensure that important survey data and metadata are
preserved for future reference and analysis. The data documentation, or metadata, helps researchers and
other audiences to find the data, understand what the data are measuring and assess the quality of the data.
• Finding the data. Names, abstracts, keywords and other important metadata elements help individuals
and organizations locate the data sets and variables that meet their needs.
• Understanding what the data are measuring and how the data have been created. Descriptions of the
survey design and the methods used when collecting and processing the data, allow users to fully
comprehend the context of the data.
• Assessing the quality of the data. Information about the data collection standards, as well as any
deviations from the planned standards, is important for gauging whether the data are useful for specific
uses.
1.10.1
Elements of data documentation
There are three main types of material that constitute ideal documentation for a data set: explanatory material,
contextual information and cataloguing material. This represents the minimum to create and preserve a data
set, and can be described as the material required to ensure the long-term viability and functionality of a data
set. Full understanding of the data set and its contents cannot be achieved without this material.
Explanatory material
Information about the data collection methods
This information describes the data collection process, whether it is a survey; the collection of administrative
information; or the transcription of a document source. It should describe the questionnaires used, the
methods employed and how these were developed. If applicable, details of the sampling design and sampling
frames should be included. It is also useful to include information on any monitoring process undertaken during
the data collection as well as details of quality controls.
Information about the structure of the data set
Key to this type of information is a detailed document describing the structure of the data set and including
information about relationships between individual files or records within the study. It should include, for
example, key variables required for unique identification of subjects across files. It should also include the
number of cases and variables in each file and the number of files in the data set. For relational models, a
diagram showing the structure and the relations between the records and elements of the data set should be
constructed.
Microdata refers to data on the characteristics of units of a population, such as individuals, households, facilities, or
establishments, collected by census, survey or experiment.
8
55
1. Overview
Technical information
This information relates to the technical framework and should include:
• the computer system used to generate the files
• the software packages with which the files were created
• the medium on which the data were stored
• a complete list of all data files present in the data set.
Variables, values, coding, and classification schemes
The documentation should contain a full list describing all variables (or fields) in the data set, including a
complete explanation and full details about the coding and classifications used for the information allocated to
those fields. It is especially important to have blank and missing fields explained and accounted for. It is helpful
to identify variables to which standard coding classifications apply, and to record the version of the
classification scheme used – preferably with a bibliographic reference to that code.
Information about derived variables
Many data producers derive new variables from original data. This may be as simple as grouping raw age data
(age in years) according to groups of years appropriate for the needs of the survey, or it may be much more
complex and require the use of sophisticated algorithms. When grouped or derived variables are created, it is
important that the logic for the grouping or derivation be clear. Simple grouping, such as for age, can be
included within the data dictionary. More complex derivations require other means of recording the
information. The best method of describing these is by using flow charts or accurate Boolean statements. It is
recommended that sufficient supporting information be provided to allow an easy link between the core
variables used and the resultant, derived variables. It is also recommended that the computer algorithms used
to create the derivations be saved together with information about the software.
Weighting
The weighting of variables needs to be fully documented, explaining the construction of the variables with a
clear indication of the circumstances in which weights should be used. This is particularly important when
different weights need to be applied for different purposes.
Data source
Details about the source the data is derived from should be included. For example, when the data source is
made up of responses to survey questionnaires, each question should be carefully recorded in the
documentation. Ideally, the text will include a reference to the generated variable(s). It is also useful to explain
the conditions under which a question would be asked to a respondent, including if possible, the cases to which
it applies, and ideally, a summary of response statistics.
Confidentiality and anonymization
It is important to note if the data contain any confidential information on individuals, households, organizations
or institutions. Whenever this occurs, it is recommended to record such information together with any
agreement on how to use the data, for example, with survey respondents. Issues of confidentiality may restrict
the analyses to be undertaken or the results to be published, particularly if the data are to be made available
for secondary use. If the data were anonymized to prevent subjects' identification, it is recommended to record
the anonymization procedure and its impact on the data, as such modification may restrict subsequent analysis.
56
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
Contextual information
Contextual information provides users with material about the context in which the data were collected, and
how data were put to use. This type of information adds richness and depth to the documentation, and enables
the secondary user to fully understand the background and processes behind the data collection exercise. This
also forms a vital historical record for future researchers.
Description of the originating project
Details should be provided about the history of the project or about the process that gave rise to the data set.
This should offer information on the intellectual and substantive framework. For example, the description
could cover topics such as:
• why the data collection was felt necessary
• the aims and objectives of the project
• who or what was being studied
• the geographical and temporal coverage
• publications or policy developments it contributed to or that arose as a response
• any other relevant information.
Provenance of the data set
Information on the origin of the data set relates to aspects such as the history of the data collection process,
changes and developments that occurred in the data themselves and the methodology, or any adjustments
made. The following can also be provided:
• details of data errors
• problems encountered in the process of data collection, data entry, data checking and cleaning
• conversion to a different software or operating system
• bibliographic references to reports or publications that stem from the study
• any other useful information on the life-cycle of the data set.
Serial and time-series data sets, new editions
For repeated cross-sectional, panel or time-series data sets, it is helpful to obtain additional information
describing, for example, changes in the question text, variable labelling or sampling procedures.
Cataloguing material
Cataloguing material serves two purposes. First, it serves as a bibliographic record of the data set. This allows
for the data set to be properly acknowledged and cited in publications, and for the material to act as a formal
record for long preservation purposes. Second, it is the basic instrument used for resource discovery, allowing
the data set to be uniquely identified within the collection by providing appropriate information to help
secondary users identify the study as being useful to their purpose.
1.10.2 Metadata standards
Traditionally, data producers and archivists produced expansive, text-based codebooks. Today, various
metadata alternatives, such the Data Documentation Initiative (DDI) and the Dublin Core Metadata Initiative
(DCMI), have been developed for the documentation and cataloguing of microdata and related materials
57
1. Overview
according to international standards. These new type of 'codebooks' are based on Extensible Markup Language
(XML), a type of regular text file that tags for meaning – rather than appearance – and can be viewed and
edited using any standard text editor. XML files can be searched and queried like a regular database and can be
edited.
Data Documentation Initiative (DDI)
The Data Documentation Initiative (DDI) is an effort to establish an international XML-based standard for
microdata documentation. Its aim is to provide a straightforward means to record and communicate to others
all the salient characteristics of microdata sets. By creating a consistent framework for microdata
documentation, the DDI has several key features: interoperability, richer content, multi-purpose
documentation, online analytical capability and search capability.
The DDI elements are organized in five sections.
Section 1.0. Document description
A study (survey, census or other) is not always documented and disseminated by the same agency as the
one that produced the data. It is therefore important to provide information (metadata) not only on the
study itself, but also on the documentation process. The document description consists of overview
information describing the DDI-compliant XML document, or, in other words, "metadata about the
metadata".
Section 2.0. Study description
The study description consists of overview information about the study. This section includes information
about how the study should be cited; who collected, compiled and distributes the data; a summary (abstract)
of the content of the data; and information on data collection methods and processing.
Section 3.0. Data file description
This section is used to describe each data file in terms of content; record and variable counts; version; producer;
and so on.
Section 4.0. Variable description
This section presents detailed information on each variable, including literal question text; universe, variable
and value labels; and derivation and imputation methods.
Section 5.0. Other material
This section allows for the description of other materials related to the study. These can include resources such
as documents (e.g. questionnaires, coding information, technical and analytical reports, interviewer's manuals),
data processing and analysis programs, photos and maps. However, the DCMI (described below) provides a
standard for documenting digital resources such as questionnaires and reports.
58
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
Dublin Core Metadata Initiative (DCMI)
The Dublin Core Metadata Initiative (DCMI) is an open forum to develop the Dublin Core metadata standard,
which is a simple set of elements for describing digital resources. This standard is particularly useful to describe
resources related to microdata such as questionnaires, reports, manuals, data processing scripts and programs.
A major reason behind the success of the Dublin Core metadata standard is its simplicity. From the outset it has
been the goal of the designers to keep the element set as small and simple as possible to allow the standard to
be used by non-specialists. In its simplest form the Dublin Core consists of 15 metadata elements, all of which
are optional and repeatable. The 15 elements are:
1.
title
2.
subject (topic)
3.
description: an abstract, a table of contents, or a free-text account of the content
4.
type: the nature or genre of the content of the resource
5.
source
6.
relation: a reference to a related resource (rarely used)
7.
coverage: the extent or scope of the content of the resource (e.g. spatial location or time period)
8.
creator
9.
publisher
10. contributor
11. rights: a rights management statement for the resource
12. date
13. format
14. identifier
15. language.
1.10.3 Creating metadata for SARA
Metadata can be created through a multitude of media including simple word processing programs and
software application programs. This section provides guidance on creating metadata for SARA by identifying
key elements to be included and by providing information on tools available to assist in creation of metadata.
Required elements
When creating a metadata document using a simple word processing program, the following elements need to
be included. Much of this information will have been generated as part of the data processing steps.
Survey description
DOCUMENT DESCRIPTION
The document description serves as an introduction to the metadata as a whole. It provides background
information such as the study title, document producer(s), date of production and version number.
STUDY DESCRIPTION
The study description serves to identify the study itself and to provide overview information, as well as the
project scope, coverage and sampling, and information on data collection, editing, appraisal and access. This
section also names producers and sponsors, and describes points of contact, and disclaimers and copyrights.
59
1. Overview
Data set(s)
FILE DESCRIPTION
The file description of a data set provides the data set contents, its producer and the version. It should also
include an explanation of how missing data are coded or accounted for, as well as any other relevant notes.
When applicable, a section on processing checks should be included. This element serves to provide
information about the types of checks and operations that have been performed on the data file to make sure
that the data are as correct as possible, e.g. consistency checking.
VARIABLES
The variables section of an archive consists of detailed descriptions of the actual data.
The variables list is typically a table listing every variable in the data set and providing for each the variable
number, name and label. This list also provides the literal question associated with the variable, the variable
format (character or numeric, number of units), and the number of valid and invalid cases (see Table 1.10.3).
TABLE 1.10.1: VARIABLES LIST
#
1
Name
V_001
Label
Facility
Name
Type
Discrete
Format
Character-12
Valid
97
Invalid
0
Question
Record the name
of the facility
The variables description is more detailed than the variable list. It includes variable information (type,
format, missing value coding), statistics (valid and invalid), literal question, and any notes (see Table
1.10.4).
TABLE 1.10.2: VARIABLES DESCRIPTION
#1 V_001: Facility name
Information
[Type= discrete] [Format=character] [Missing=*]
Statistics
[Valid=97 /-] [Invalid=0 /-]
Literal question
Record the name of the facility
Notes
External resources
TYPES OF RESOURCES
External resources encompass all of the documents contributing to the implementation of the survey or
stemming from the results of the survey. Examples include:
• questionnaires
• reports
• databases
• photos, videos, etc.
• maps or geospatial data
• technical documents
• analytical or administrative documents.
60
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
RESOURCE INFORMATION
Each external resource should be accompanied by relevant descriptive information.
Identification
• type of resource
• title
• authors: the individuals or organization primarily responsible for creating the resource
• date: the date on which the resource was created or last modified
• country: all countries within the scope of a resource
• language
• format
• an ID number, if applicable: an unambiguous reference to the resource.
Contributor and rights
• contributor(s): individuals or organizations who have supported or contributed to the development of
the resource (including funding agencies)
• publisher(s): individuals or organizations responsible for disseminating the resource
• rights: a clear and complete description of the usage rights, if relevant.
Content
• description: an account of the content of the resource
• abstract
• table of contents: a listing of all sections of the resource
• subjects: key topics discussed in the resource.
Available tools
The Microdata Management Toolkit 9 developed by the World Bank Data Group is designed to address the
technical issues facing data producers. It provides one of the most straightforward ways to create
comprehensive metadata that adhere to international standards. The aim in developing the Toolkit was to
promote the adoption of standards for international microdata documentation, dissemination and
preservation, as well as to foster best practices by data producers in developing countries.
The Toolkit consists of:
• a Metadata Editor, which documents data in accordance with international standards;
• an International Household Survey Network (IHSN) Report Center, which generates metadata reports
from inputs into the Metadata Editor;
• an Explorer, which allows users to view metadata and to re-export data in common formats;
• a CD-Rom Builder, which generates user-friendly outputs (CD-ROM, web) for dissemination and
archiving.
Templates for SARA survey archiving are publicly available through the IHFAN web site at
http://www.ihfan.org/home/index.php?editable=no&page_type=catalog.
The Microdata Management Toolkit is free and available for download along with a user manual at:
http://www.ihsn.org/home/index.php?q=tools/toolkit
9
61
1. Overview
1.10.4 Data archiving
Today, data archives are most always digital and are ideally web-based or are made publicly available through
the Internet. While this can be accomplished through many different types of media, SARA makes use of the
National Data Archive (NADA) which is a free, standardized application for publishing data archives. 10
National Data Archive (NADA) tool
The International Household Survey Network (IHSN) developed the national data archive (NADA) as a
complement to the Microdata Management Toolkit. NADA is a web-based survey cataloguing system that
serves as a portal for researchers to browse, search, apply for access, and download relevant census or survey
data and metadata.
NADA makes use of the XML-based international standards such as the DDI and Dublin Core and is a powerful
instrument that facilitates the process of releasing study metadata and microdata to the user community.
NADA is a tool for informing users about the existence and characteristics of survey, census or other microdata
sets, and for sharing metadata and (optional) disseminating microdata files. NADA does not provide tools for
data tabulation or analysis. It aims to provide users with detailed and searchable documentation of microdata
sets, along with information on policies and procedures for their access and use. NADA comes as a prepackaged but fully customizable web site. At the core of NADA is the data catalogue, which:
• provides summary information on each survey;
• provides access to reports, tables and other analytical output;
• provides data access policies to the user community and facilitates access by serving as an implementing
tool of the data access policy;
• provides links to related survey metadata;
• facilitates searches at the variable level and displays variable-level information;
• provides authorized users with access to the data (via direct access or through online forms), with
conditions for access clearly stated;
• keeps a log of user requests;
• links to the HTML output as provided by the CD-ROM Builder of the Microdata Management Toolkit;
• includes an automatically-generated history of added/updated data sets via an RSS feed;
• is easy to maintain and use.
The data catalogue interface is interactive, allowing users to sort and search the catalogue by study elements
and/or data variables, or find out detailed information through the survey's metadata.
WHO has created a national data archive for SARA surveys, which can be
http://apps.who.int/healthinfo/systems/datacatalog/index.php/catalog.
This site serves as an example of how a data archive can be created using the NADA software.
10
62
NADA is available to download free of charge at: http://www.ihsn.org/home/index.php?q=tools/nada.
located
at
Service Availability and Readiness Assessment (SARA) | Reference Manual, Version 2.2
References
1. International Health Partnership and related initiatives (IHP+). Geneva, World Health Organization and
Washington DC, The World Bank (http://www.internationalhealthpartnership.net/en/home, accessed 17
December 2011).
2. Monitoring, evaluation and review of national health strategies. A country-led platform for information and
accountability. Geneva, World Health Organization, 2011.
3. Service availability mapping (SAM). Geneva, World Health Organization
(http://www.who.int/healthinfo/systems/samintro/en/index.html, accessed 17 December 2011).
4. Service provision assessment (SPA) overview. Maryland, MEASURE DHS, ICF International
(http://www.measuredhs.com/aboutsurveys/spa/start.cfm, accessed 17 December 2011).
5. Measuring medicine prices, availability, affordability and price components, 2nd ed. Geneva, World Health
Organization and Health Action International, 2008
(http://www.haiweb.org/medicineprices/manual/documents.html and
http://www.who.int/medicines/areas/access/medicines_prices08/en/, accessed 17 December 2011).
6. Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies.
Geneva, World Health Organization, 2010 (http://www.who.int/healthinfo/systems/monitoring/en/index.htm,
accessed 17 December 2011).
7. Creating a master facility list. Draft document. Geneva, World Health Organization, 2012
8. Data Quality Review (DQR): A toolkit for facility data quality assessment. Working document. Geneva, World
Health Organization, 2015
9. Health workforce target reference
10. Outpatient service utilization target reference
11. Inpatient service utilization target reference
63
Source Exif Data:
File Type : PDF File Type Extension : pdf MIME Type : application/pdf PDF Version : 1.6 Linearized : No Author : sheffela Company : World Health Organization Create Date : 2015:08:06 14:33:50+02:00 Modify Date : 2015:08:26 10:51:37+02:00 Title : Tag New Review Cycle : Has XFA : No Tagged PDF : Yes XMP Toolkit : Adobe XMP Core 5.2-c001 63.139439, 2010/09/27-13:37:26 Metadata Date : 2015:08:26 10:51:37+02:00 Creator Tool : Acrobat PDFMaker 10.1 for Word Format : application/pdf Creator : sheffela Document ID : uuid:3590bcaf-c9c9-4442-8a0d-14214f0c8d78 Instance ID : uuid:2e9e2b1e-80f4-4b4d-a19b-d75d0a99de33 Producer : Adobe PDF Library 10.0 Page Count : 65EXIF Metadata provided by EXIF.tools