Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Testing Hypotheses of No Meaningful Effect
Search
Carlisle Rainey
January 04, 2013
Research
1
400
Testing Hypotheses of No Meaningful Effect
Carlisle Rainey
January 04, 2013
Tweet
Share
More Decks by Carlisle Rainey
See All by Carlisle Rainey
Topic Sampling @ SPSA 2024
carlislerainey
0
160
Topic Sampling
carlislerainey
1
330
ggcarly:theme_carly_presents()
carlislerainey
0
75
Unnecessary
carlislerainey
0
500
When BLUE Is Not Best: Non-Normal Errors and the Linear Model
carlislerainey
0
190
Dealing with Separation in Logistic Regression Models
carlislerainey
0
120
Logit Models in Small Samples
carlislerainey
0
100
When BLUE is Not Best
carlislerainey
0
430
Dealing with Separation in Logistic Regression Models
carlislerainey
0
440
Other Decks in Research
See All in Research
【論文紹介】Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions
ichiroex
0
110
10-ot-generic-bio.pdf
gpeyre
0
150
20240209 データを肴に熊本の交通を考える会「車1割削減、渋滞半減、公共交通2倍」をめざし世界に学ぼう
trafficbrain
0
950
ランサーズエージェント_フリーランスエンジニアの年収・キャリアの実態調査2024
lancers_pr
0
140
First Authorに俺はなるっ!! IROS’23 CCC2023 FY
shota_nishiyama
0
190
訓練データ作成のためのCloudCompareを利用した点群の手動ラベリング
kentaitakura
0
590
FMP L3 Year 1 Project Proposal
haiinya
0
150
継続的な研究費獲得のための考え方
moda0
2
470
メタ動画データセットによる動作認識の現状と可能性
yuyay
0
240
生成AIを用いたText to SQLの最前線
masatoto
1
2.8k
CARMUI:自動運転車遠隔監視システムのUI検討のためのバーチャル都市プラットフォームの開発 / CARMUI-i2024
yumulab
0
120
Refactoring Mining - The key to unlock software evolution
tsantalis
0
310
Featured
See All Featured
Fantastic passwords and where to find them - at NoRuKo
philnash
39
2.5k
BBQ
matthewcrist
80
8.8k
Robots, Beer and Maslow
schacon
PRO
155
8k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
245
20k
Ruby is Unlike a Banana
tanoku
96
10k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
126
32k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
501
140k
Practical Orchestrator
shlominoach
183
9.8k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
221
21k
How STYLIGHT went responsive
nonsquared
92
4.8k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
21
1.6k
A Modern Web Designer's Workflow
chriscoyier
689
190k
Transcript
Testing Hypotheses of No Meaningful Effect Carlisle Rainey
I'd like to convince you of three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
Important Step 1:
Hypotheses of no meaningful effect are crucial to complete evaluation
of theories
Interaction
Social heterogeneity increases the number of parties, but only when
electoral institutions are sufficiently permissive. “ ” Clark and Golder (2006)
Adjudication
None
None
How often do these examples occur?
30%
Our arguments are not compelling Step 2:
No American president since FDR has won a second term
when the unemployment rate topped 7.2 percent... Obama must defy that trend to keep his job. “ ” New York Times June 1, 2011 “ ”
None
None
The nation's unemployment status by itself is not going to
affect Obama's. “ ” Seth Masket June 2, 2011
My critique
Rule out implausible relationships
None
None
None
None
None
None
Insignificance can't be used to argue for “no effect.”
But doesn't everyone already know that? “ ” A skeptic
Political scientists draw strong conclusions from insignificance.
Recessions have no effect on whether a democracy is consolidated.
“ ” Svolik (2008)
Our arguments can be compelling Step 3:
just like any other hypothesis
Argue against relationships inconsistent with the hypothesis
Define substantively meaningful Step 1:
Check the 90% CI Step 2:
Only negativity about the respondent’s preferred candidate should have a
significant demobilizing effect on voter turnout. “ ” Krupnikov (2011)
What's a meaningful effect? 1% 3% 5% 7% 9%
The demobilizing effect could be as large as 9%
Those are the three things
Important 1
Our arguments are not compelling 2
Our arguments can be compelling 3
But what should I do about this? “ ” You
Go read the details crain.co/nme 1
Think about your own work. 2
Keep in mind when reviewing others' work 3
None