Power Analysis Guide

User Manual:

Open the PDF directly: View PDF PDF.
Page Count: 4

DownloadPower Analysis Guide
Open PDF In BrowserView PDF
PowerAnalysis.m Guide
Roger Strong
Harvard University

General Notes
•
•

PowerAnalysis.m does most the work, and is called in the example scripts
NOTE: This version only simulates t-tests between within subject conditions

•

Key Components:
– prefs.data:
•
•
•

either a #subjects (rows) x #conditions (columns) array, or a string file name of an excel or .csv file with data
listed as #subjects x #conditions.
Data can be listed as either decimal (.5) or percentage (50), although you will get a warning for the later (as
data will be converted to decimal)
If using excel or csv file, there should NOT be a header row

– prefs.N_range
•

Range of number of participants to simulate. E.g., 10:10:50 will simulate with 10, 20, 30, 40, and 50
participants

– prefs.trial_range
•

Range of number of trials per condition to simulate. E.g., 8:4:24 will simulate with 8, 12, 16, 20, and 24
trials per condition

– prefs.alpha
•

p-value to use in power simulations

– prefs.nSims
•

How many simulations to use for every particpant/trial number combination. 10,000 is a decent estimate
and runs pretty quickly, 100,000 is slower but a more stable estimate.

– prefs.comps
•

Which comparisons to test for significance. Each row is a comparison, with the condition expected to be
higher magnitude listed in the first column, and the condition expected to have lower magnitude in the
second column. A study will be classified as “successful” only if all listed comparisons are significant (see
examples).

Example 1
Power Analysis Settings

Pilot Data
-

I decided to simulate N from 10-100 by 10

Experiment 1

1
N = 97

0.9

Accuracy

File name as string (can also do data directly in matlab)

97 subjects, 2 conditions
Excel file is 97 rows x 2 columns

I decided to simulate trial number per condition from 8-24 by 4

p < .001, *dx = 0.56

P-value of .05 used in simulation

0.8

10,000 sims per N x num_trials combo (sims per cell in
output graph)

0.7
0.6
0.5

Only comparison I was interested in was condition
1 being larger than condition 2
Within

Across

Run power analysis using these settings

Power Analysis Output
Power by N and # of Trials

# of Trials Per Condition

Exp1_Data.xlsx

24

0.24

0.45

0.61

0.74

0.84

0.9

0.94

0.96

0.98

0.99

20

0.23

0.41

0.59

0.71

0.8

0.87

0.91

0.95

0.97

0.98

16

0.2

0.38

0.54

0.66

0.76

0.83

0.89

0.92

0.95

0.96

12

0.18

0.33

0.46

0.58

0.68

0.78

0.83

0.87

0.91

0.93

8

0.14

0.27

0.37

0.48

0.56

0.65

0.72

0.77

0.82

0.86

10

20

30

40

50

60

70

80

90

100

# of Subjects

Simulated power for each N X number or trials per
condition combo we specified in settings. Looking
at this, I know I could achieve > 90% power by
running 90 subjects with 12 trials per condition, for
example

Example 2
Power Analysis Settings

Pilot Data

File name as string (can also do data directly in matlab).

-

41 subjects, 4 conditions
Excel file is 41 rows x 4 columns

I decided to simulate trial number per condition from 8-20 by 4

N = 41

100

Percent Correct

I decided to simulate N from 50-300 by 25

P-value of .05 used in simulation

90

10,000 sims per N x num_trials combo (sims per cell in output
graph)

80
70

This time, I had 5 comparisons I am interested in. Specifically, I only want to call the study a
“success” if condition 1 >2, 1 >3, 1>4, 3>2, and 4>2. Each comparison specified as a separate row.

60
Run power analysis using these settings

50

W-W

B-B

W-B

B-W

Power Analysis Output
Power by N and # of Trials

Exp2_Data.xlsx

# of Trials Per Condition

20

Data is in percent, so
script will convert to
decimal and give a
warning that this has
occurred.

0.1

0.25 0.41 0.56 0.68 0.76 0.82 0.87

0.9

0.93 0.95

16

0.08 0.21 0.37 0.51 0.62 0.71 0.79 0.84 0.88

12

0.06 0.16

0.3

0.03

0.1

0.19 0.31 0.43 0.52 0.61 0.68 0.74 0.79 0.82

50

75

8

100

0.9

0.93

0.43 0.55 0.65 0.72 0.78 0.83 0.88

0.9

125

150

175

200

# of Subjects

225

250

275

300

Simulated power for each N X number or trials per
condition combo we specified in settings. Looking
at this, I know I could achieve > 90% power by
running 300 subjects with 12 trials per condition,
for example. Note that this is power for ALL 5
comparisons of interest being significant
Source Exif Data: 
File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
Linearized                      : No
Page Count                      : 4
PDF Version                     : 1.4
Producer                        : Mac OS X 10.12.6 Quartz PDFContext
Creator                         : PowerPoint
Create Date                     : 2019:02:18 20:44:12Z
Modify Date                     : 2019:02:18 20:44:12Z
EXIF Metadata provided by EXIF.tools

Navigation menu