Defending Quantitative Research: Viva Scripts for Eight Challenges

When it comes to defending quantitative research, viva answers are strongest when every result is stated as three things: the size of the effect, how uncertain it is, and what it means for your question. A p-value on its own answers none of those. Learn to say all three for each key result, and most statistical challenges become manageable.

This guide gives a three-part answer pattern, eight challenges with exact wording, a worked example with numbers you can check, and a sheet for preparing your key results. Universities and examiners differ, so treat it as preparation, not a forecast of your viva.

What examiners are testing

They are testing whether you understand your own numbers, not whether you can recite statistics theory. Expect to be asked why you chose your tests and what your results do and do not show.

The University of Edinburgh's viva preparation guide (PDF) lists "Why did you choose to use this statistical test rather than others?" among its example questions. It also lists "Why did you choose the method / model for this research?" For the general "why this design" answers, see our guide on how to defend your methodology in a viva. This guide covers what is particular to numbers.

Defending quantitative research: viva answers in three parts

Use three parts for any result: estimate, uncertainty, meaning. This is the shape that the standards below point towards.

1. Estimate. "The effect was [effect size, in its units or a standard measure]."

2. Uncertainty. "The 95% confidence interval runs from [lower] to [upper], and p was [exact value]."

3. Meaning. "In practical terms, that means [what it changes], and I'd claim [scope], not [beyond]."

The American Psychological Association's reporting standards for quantitative research (PDF) ask for "effect-size estimates and confidence intervals on estimates that correspond to each inferential test conducted, when possible". They also ask for "exact p values" when significance testing is used. If your thesis follows those standards, the three parts are already on your pages. You only need to say them aloud.

Eight challenges, with exact wording

The wording of each challenge is ours, written to show the kind of question you could hear. Each answer uses the three parts, or explains why a part is missing.

1. "What does your p-value actually tell you?"

Say what it measures and what it does not. The American Statistical Association's statement on p-values (PDF), published in 2016 with Wasserstein and Lazar's account of it, says "P-values can indicate how incompatible the data are with a specified statistical model". It also says "P-values do not measure the probability that the studied hypothesis is true".

"It tells me how incompatible my data are with a model of no relationship, given the assumptions of my test. A p below .001 says my data would be very unusual under that model. It doesn't tell me the probability that my hypothesis is true, and it doesn't tell me how big the effect is. For that I'd point you to the effect size and interval in Table 4.2."

If you only learn one answer in this guide, learn that one. It shows the examiner you know the limit of the number you are most likely to have leaned on.

2. "It's significant, but does it matter?"

Answer with the effect size and what it means in the real world. The ASA's fifth principle says "A p-value, or statistical significance, does not measure the size of an effect or the importance of a result."

"Statistically, the relationship is clear, but it's modest. A correlation of .21 means shift length accounts for about 4% of the variation in burnout. That matters for rota design at scale, but I wouldn't claim shift length is the main driver of burnout."

Greenland and colleagues (2016) list "Statistical significance indicates a scientifically or substantively important relation has been detected" as a misinterpretation. They note that large studies can make trivial effects statistically significant.

3. "So your non-significant result shows there's no difference?"

No, and say so plainly. Greenland and colleagues list this as a misinterpretation: "A nonsignificant test result (P > 0.05) means that the test hypothesis is true or should be accepted."

"No. A p of .22 means I couldn't distinguish the groups with this sample. The interval for the difference runs from slightly negative to a small-to-moderate positive effect, so my data fit no difference and fit a small one. What I can say is that a large difference is unlikely."

The interval is what lets you say something useful about a null result. If you only reported a p-value for that test, say what the interval would be, or that you would add it in corrections.

4. "Why did you use this test and not another?"

Name what your data and question needed, and how you checked the test's assumptions. The APA standards ask authors to "Report any problems with statistical assumptions and/or data distributions that could affect the validity of findings."

"My outcome was continuous and I had two groups, so I used an independent-samples t-test. I checked normality and equal variances, which you'll find in Appendix D. The variances were similar, so I didn't need a correction. If they hadn't been, I'd planned a version of the test that doesn't assume equal variances."

If an assumption failed and you went ahead anyway, say so first. Then say what you did about it, such as a robustness check, and whether the result changed.

5. "How many tests did you run?"

Give the number, and say which were planned and which were exploratory. The ASA's fourth principle says "Proper inference requires full reporting and transparency." The APA standards also ask authors to describe their "protection against experimentwise error".

"I ran three planned tests, one per hypothesis, and they're all reported in Chapter 4. The subgroup comparisons in Section 4.5 were exploratory, and I've labelled them that way. I didn't correct those for multiple comparisons, so I treat them as ideas for future work, not findings."

Do not round this answer down. If you ran more tests than you report, the viva is a better place to say so than a correction request later.

6. "What did you do about missing data?"

Give the amount, the method and the effect on your results. The APA standards ask for the "frequency or percentages of missing data" and the "methods actually used for addressing missing data, if any".

"About 6% of responses had at least one missing item. I used the approach described in Section 3.8 and ran the main analysis both ways. The result was the same direction and similar size, which is in Appendix E."

If you simply dropped incomplete cases, say so. Then say whether the people you dropped looked different from the rest.

7. "Can you really claim that X causes Y?"

Match your claim to your design. A survey at one point in time can show an association, so say "is associated with" and mean it.

"No, and I've tried not to. My design was a cross-sectional survey, so I can say longer shifts are associated with higher burnout. I can't rule out that burned-out staff choose longer shifts. Section 6.3 suggests a longitudinal design that could test the direction."

Check your abstract and conclusion for causal verbs such as "leads to", "drives" or "improves". If they are there and your design cannot support them, offer to reword them before the examiner asks.

8. "Was your study big enough?"

Say how you decided your sample size and what it lets you claim. The APA standards ask for a "power analysis, or methods used to determine precision of parameter estimates". Our guide on defending sample size in a viva covers power calculations and what to say when a sample falls short, so we do not repeat it here.

A worked example (illustration only)

The thesis, the numbers and the page references below are invented for this guide. The statistics are internally consistent, so you can check them yourself.

Imagine a thesis that surveys 312 hospital nurses about shift length and burnout. The main result is r(310) = .21, p < .001, 95% CI [.10, .31]. A secondary comparison of night and day staff (150 and 162 nurses) gives d = 0.14, t(310) = 1.24, p = .22, 95% CI [-0.08, 0.36].

Examiner: "Your abstract says shift length predicts burnout. How strong is that?"

Candidate: "It's a modest relationship. The correlation is .21, with an interval from .10 to .31, so even the upper end is moderate. It accounts for about 4% of the variance in burnout. I'd also change 'predicts' to 'is associated with' in the abstract, because the design is cross-sectional. The night and day comparison wasn't significant, but the interval includes a small effect, so I wouldn't claim the groups are the same."

The answer gives the estimate, the uncertainty and the meaning, then corrects an overclaim before the examiner has to. It also handles the null result without claiming "no difference".

A results sheet to prepare before the viva

Fill in one row for each result your conclusions depend on. Say each row out loud as a three-part answer.

Result	Test and statistic	Exact p	Effect size	95% CI	What it means in plain words	Page
Main finding
Second finding
Key null result

While you fill it in, recompute each p-value from its test statistic and degrees of freedom in your statistics software. A typo in a reported p-value is easy to make and awkward to explain under questioning. Also check the numbers in your abstract match the numbers in your results tables.

Mistakes that weaken a quantitative defence

Three habits make sound results sound weak. Each is easy to fix before the day.

Answering with a p-value alone. "It was significant" invites "how big was it?" Lead with the effect size and interval.

Treating p = .06 and p = .04 as opposites. The ASA's third principle warns against basing conclusions only on whether a p-value passes a specific threshold. Describe both results by their size and interval.

Defending a word you did not need. If your thesis says "proves" or "causes" and your design does not support it, concede the word and defend the finding.

If you face a US dissertation defense

The challenges are the same, but a US defense can be public, so you may explain your statistics to a mixed audience. At the University of Michigan, Rackham's policy says "Oral defenses are public". It also requires a cognate member from a doctoral program "other than the student's home department/program".

Prepare a version of each three-part answer with no jargon, in case the question comes from a committee member outside your field. In Canada, universities such as McGill spell it "defence", and McGill's proceedings open with a 20-minute presentation. A slide that shows effect sizes with intervals, not just stars for significance, answers challenges 1 to 3 before they are asked.

If your study mixes methods, our guide on defending qualitative research in a viva covers the challenges your interview strand may face. It uses the same idea of judging each strand by its own standards.

What Thesisroom does with this

The Statistics check recalculates the p-value from the test statistic and degrees of freedom you report, for t, F, r, z and chi-square results written in your text (not in tables). It shows where a reported figure cannot be right, so you can fix it before an examiner finds it. Defense Practice then writes questions an examiner or committee could ask, each tied to a specific passage of your own thesis, in three styles: Practice, Rapid round and Mock panel.

You answer in writing or out loud. Thesisroom does not write your answers or hand you a script. You can find both in the rooms, and Defense Practice directly in the Defense Practice room.

Frequently asked questions

Do I need to know the formula for every test I used?

You need to know what each test assumes, what it compares and why it suited your data. Being able to explain those in plain words matters more than reciting a formula. If you are unsure of a detail, say so and explain where you would check it.

What if an examiner finds an error in my statistics?

Look at it on the page, and if it is wrong, say so. Then say whether it changes your conclusion. If the finding survives the fix, say so calmly and offer to correct it.

Should I report exact p-values or p < .05?

The APA reporting standards ask for exact p values when significance testing is used. Check what your department or style guide requires, and use the same format throughout.

How do I explain a result that went against my hypothesis?

Report it with the same three parts as any other result: estimate, uncertainty, meaning. Then say what it changes in your argument. A clearly explained unexpected result shows you followed your data.

Is it fine to say "trend towards significance"?

Avoid it. The ASA statement warns against conclusions based only on whether a p-value passes a threshold. Give the effect size and interval instead, and say what the data are compatible with.

What if my supervisor chose the statistical tests?

You still need to explain and defend them as your own. Learn why each test suits your data and what its assumptions are. Saying that your supervisor chose it is not an answer an examiner can accept.

Sources

• University of Edinburgh Doctoral College: Viva Preparation (PDF)
• American Psychological Association: JARS-Quant Table 1, Information Recommended for Inclusion in Manuscripts That Report New Data Collections Regardless of Research Design (PDF)
• American Statistical Association: Statement on Statistical Significance and P-Values, 7 March 2016 (PDF)
• Wasserstein, R. L., and Lazar, N. A. (2016). The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician, 70(2), 129 to 133
• Greenland, S., Senn, S. J., Rothman, K. J., et al. (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology, 31, 337 to 350
• University of Michigan Rackham Graduate School: Academic Policies, Section 4
• McGill University Graduate and Postdoctoral Studies: Doctoral Oral Defence Proceedings

How this guide was made: Thesisroom drafted this guide with the help of AI writing tools, then checked every fact, figure and source by hand against the references linked above.