How to Teach Reproducibility in Classwork

class: center, middle, inverse, title-slide

# How to Teach Reproducibility in Classwork
## Western Economic Association International
### Fernando Hoces, Berkeley Initiative for Transparency in the Social Sciences
### 27 June 2020 | <a href="https://github.com/BITSS/ACRE/tree/master/slides">slides</a> .font80[(note: use the left and right arrow keys to navigate through slides)]

---

<style>

.center2 {
  margin: 0;
  position: absolute;
  top: 50%;
  left: 50%;
  -ms-transform: translate(-50%, -50%);
  transform: translate(-50%, -50%);
}

pre.sourceCode {
    max-height: 200px;
    overflow-y: auto;
}

/*
.remark-slide-number {
  position: inherit;
}

.remark-slide-number .progress-bar-container {
  position: absolute;
  bottom: 0;
  height: 4px;
  display: block;
  left: 0;
  right: 0;
}

.remark-slide-number .progress-bar {
 height: 100%;
 background-color: blue;
}
*/
</style>

</style>

# Today's Presentation

.pull-left[
### Part I: Accelerating Computational Reproducibility in Economics (X)

[10 min break]

### Part II: Hands-on Practices for Computational Reproducibility (90' - X)
]

.pull-right[

### Target Audience:
 - Instructors of Empirical/Applied Courses in Economics (and related) PhDs 
 - Advisers of undergraduate students 
 - Researchers interested in conducting reproductions

]

---
class: inverse, center, middle
name: prologue

# Part I: Accelerating Computational Reproducibility in Economics (ACRE)

---
# Table of Contents for Part I

.font130[

1. [BITSS](#about-bitss)

2. [Reproducibility](#motivation)

3. [ACRE Guidelines](#guidelines)

4. [ACRE Platform](#platform)
]

---
count: false
# Table of Contents for Part I 

.font130[

1. [**BITSS**](#about-bitss)

2. [Reproducibility](#motivation)

3. [ACRE Guidelines](#guidelines)

4. [ACRE Platform](#platform)
]
---
background-image: url("Images/BITSSlogo.png"), url(Images/cega.png)
background-size: contain, 200px
background-position: 50% 100% , 0% 100% 
name: about-bitss
# About Us
  
  
### [BITSS](https://bitss.org)

.pull-left[
The Berkeley Initiative for Transparency in the Social Sciences works to improve the credibility of science by 
advancing transparency, reproducibility, rigor, and ethics in research.

### ACRE

Aleks Bogdanoski

Joel Ferguson

Fernando Hoces  
 
 Katie Hoeberling

]
.pull-right[ .right[
We are part of the Center for Effective Global Action ([CEGA](https://cega.berkeley.edu/)). 
 
 Edward Miguel (PI)

Emma Ng

Lars Vilhuber (Co-PI)

]]

---
count: false
# Table of Contents for Part I 

.font130[

1. [BITSS](#about-bitss)

2. [**Reproducibility**](#motivation)

3. [ACRE Guidelines](#guidelines)

4. [ACRE Platform](#platform)
]
---
name: motivation
# Motivation 1: "Reproducibility Crisis"

.font130[
| Replication in Social Sciences (same method, different sample) 	| Reproduction in Economics (same data and methods) 	|
|-------------------------------------------------------------------	|------------------------------------------------------	|
| OSC ([2015](https://docs.google.com/document/d/1mm_4HZnEz_2Bh8XuiS2tpqCH08MFPyqUhi1baKPqR8Y/edit#heading=h.7vqf2cziid7z)): 30%-60% 	 | Chang & Li ([2015](https://www.nowpublishers.com/article/Details/CFR-0053)): 43% 	|
| Camerer et. al. ([2016](https://science.sciencemag.org/content/351/6280/1433)): ~60% 	| Gertler et. al. ([2017](https://www.nature.com/articles/d41586-018-02108-9)): 14% 	|
| Nosek & Camerer et. al. ([2018](https://www.nature.com/articles/s41562-018-0399-z)): ~60% 	| Kingi et. al. ([2018](https://hautahi.com/static/docs/Replication_aejae.pdf)): 43% 	|
| Klein et. al. ([2018](https://journals.sagepub.com/doi/10.1177/2515245918810225)): 50% 	| Wood et. al. ([2018](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0209416#abstract0)): 25% 	|
]
---

# .font90[M2: More Inclusive Concept Scholarly Output]

.font120[
 <blockquote>
 Clarebout Principle:
 
 “An article about computational science in a scientific publication is not the scholarship itself, it’s merely scholarship advertisement. The actual scholarship is the complete software development environment and the complete set of instructions which generated the figures.”
 
.right[ <cite> Buckheit and D.L. Donoho (1995, [2009](https://academiccommons.columbia.edu/doi/10.7916/D8QZ2MDS/download))</cite> ]
 </blockquote>
]

---

# .font90[M2: More Inclusive Concept Scholarly Output]

.font120[
 <blockquote>
 Clarebout Principle:
 
 “An article about computational science in a scientific publication is not the scholarship itself, it’s merely scholarship advertisement. The actual scholarship is the complete software development environment and the complete set of instructions which generated the figures.”
 
.right[ <cite> Buckheit and D.L. Donoho (1995, [2009](https://academiccommons.columbia.edu/doi/10.7916/D8QZ2MDS/download))</cite> ]
 </blockquote>
]

---
# .font90[M2: More Inclusive Concept Scholarly Output]

.font130[
Potential benefits of following the Clarebout Principle

Well discussed potential positive effects on:
- Pedagogy  
- Incremental generation of knowledge 
]
--

.font130[
Under discussed:

- Possible positive effect on diversity, equity and inclusion: no connections or language skills ("appropriate politeness") required to obtain materials 
]

???
An under discussed benefit of this principle is that increases access to knowledge for students who are outside of elite schools (no connections or "appropriate politeness" required to obtain materials).

---