Augmenting Intelligence
Augmenting Intelligence is a multidisciplinary course at CMU’s Human-Computer Interaction Institute. We explore how to design and evaluate technologies aimed at augmenting & amplifying human cognitive capabilities.
Oh! Hey there! I’m so glad you could join us!
Course resources
Prof. Ken Holstein
Course calendar
Week 1
Welcome!
Please review the course syllabus. The syllabus includes directions to the classroom + information about the course structure and policies.
This week, you and your classmates will collaboratively develop an additional course policy for the coming semester!
(Aug 26 - 30)
INTRO TO THE COURSE
+
COLLABORATIVE COURSE
POLICY DESIGN
Assignments
(due before class on
Wednesday, Aug 28)
As homework, participate in the collaborative course policy design activity each day (i.e., on Monday after class, on Tuesday, and then one more time before Wednesday’s class).
Week 2
(Sep 3 - 6)
Required Readings:
Early Visions, Branching Paths
(due by Tuesday night, Sep 3)
A BRIEF HISTORY OF THE FUTURE OF “AUGMENTED INTELLIGENCE”
How to read a paper — Keshav, 2007
suggested reading/skimming time: 20 min
AI and HCI: Two fields divided by a common focus — Grudin, 2009
suggested reading/skimming time: 45 min
Man-computer symbiosis — Licklider, 1960
suggested reading/skimming time: 45 min
Augmenting human intellect:
A conceptual framework – Engelbart, 1962
suggested reading/skimming time: 1 hr, 15 min
meta-skills
context
No class on Monday
(Labor Day)
concepts
concepts
Submit your reading
responses here
Other assignments
(due before class on Wednesday, Sep 4)
Sign up in groups of 3-4 students to present readings for one upcoming class, using this sign-up sheet
Optional Additional Readings
As we may think — Bush, 1945
The intertwined histories of artificial intelligence and education — Doroudi, 2023
Human Factors, CHI, and MIS – Grudin, 2015
Week 3
(Sep 9 - 13)
Required Readings:
HCI, AI, & Human Factors
(due by Sunday night, Sep 8)
A BRIEF HISTORY OF THE FUTURE OF “AUGMENTED INTELLIGENCE”
(continued)
The computer for the 21st century — Weiser, 1991
suggested reading/skimming time: 45 min
Principles of mixed-initiative user interfaces — Horvitz, 1999
suggested reading/skimming time: 1 hr
A model for types and levels of human interaction with automation — Parasuraman & Sheridan, 2000
suggested reading/skimming time: 1 hr
concepts
concepts
concepts
Submit your reading
responses here
Required Readings:
Human-Human & Human-AI Augmentation
(due by Tuesday night, Sep 10)
Augmenting social cognition — Chi et al., 2008
suggested reading/skimming time: 20 min
Hybrid intelligence — Dellerman, 2019
suggested reading/skimming time: 45 min
Designing for human–AI complementarity in K-12 education — Holstein & Aleven, 2021
suggested reading/skimming time: 45 min
A taxonomy of human and ML strengths in decision-making to investigate human–ML complementarity – Rastogi, Liu et al., 2023
read Sections 1-3 only
suggested reading/skimming time: 1 hr, 15 min
concepts
concepts
case studies
concepts
Submit your reading
responses here
Optional Additional Readings
Direct manipulation vs. interface agents — Shneiderman & Maes, 1997
A literature review on the levels of automation during the years. What are the different taxonomies that have been proposed? — Vagia et al., 2016
Human factors in automation design — Lee & Seppelt, 2009
Hybrid workplaces of the future — Kamar, 2016
Hybrid-augmented intelligence: Collaboration and cognition — Zheng et al., 2017
Artificial intelligence, human intelligence and hybrid intelligence based on mutual augmentation – Jarrahi et al., 2022
Cognitive science of augmented intelligence — Dubova et al., 2022
Week 4
(Sep 16 - 20)
Required Readings:
(due by Sunday night, Sep 15)
AUGMENTING
DECISION-MAKING
Wearable reasoner: Towards enhanced human rationality through a wearable device with an explainable AI assistant — Danry, Pataranutaporn et al., 2020
suggested reading/skimming time: 1 hr
Unremarkable AI: Fitting intelligent decision support into critical, clinical decision-making processes — Yang et al., 2019
suggested reading/skimming time: 1 hr
Improving human-AI partnerships in child welfare: Understanding worker practices, challenges, and desires for algorithmic decision-support — Kawakami et al., 2022
suggested reading/skimming time: 1 hr
case studies
case studies
case studies
Submit your reading
responses here
Required Readings
(due by Tuesday night, Sep 17)
Other assignments
(due before class on Wednesday, Sep 18)
Add a note in the course project brainstorming doc with at least one idea for a project direction you’d be interested in exploring.
Any notes you add at this stage are tentative. So don’t worry: you are not committing to pursuing a particular direction for your actual project.
Trust in automation: Designing for appropriate reliance — Lee & See, 2004
suggested reading/skimming time: 1 hr, 15 min
Don't just tell me, ask me: AI systems that intelligently frame explanations as questions improve human logical discernment accuracy over causal AI explanations — Danry et al., 2023
suggested reading/skimming time: 45 min
Towards human-AI deliberation: Design and evaluation of LLM-empowered deliberative AI for AI-assisted decision-making — Ma et al., 2024
suggested reading/skimming time: 45 min
concepts
case studies
case studies
Submit your reading
responses here
Optional Additional Readings
In search of verifiability: Explanations rarely enable complementary performance in AI-advised decision making — Fok & Weld, 2023
Cognitive challenges in human–artificial intelligence collaboration: Investigating the path toward productive delegation — Fügener et al., 2021
Ground(less) truth: A causal framework for proxy labels in human-algorithm decision-making — Guerdan et al., 2022
Manipulating and measuring model interpretability — Poursabzi-Sangdeh et al., 2021
The placebo effect of human augmentation: Anticipating cognitive augmentation increases risk-taking behavior — Villa et al., 2023
Human-algorithmic interaction using a large language model-augmented artificial intelligence clinical decision support system — Rajashekar, Shin et al., 2024
Week 5
(Sep 23 - 27)
AUGMENTING SENSEMAKING & PERCEPTION
Required Readings:
(due by Sunday night, Sep 22)
Accelerating scientific paper skimming with augmented intelligence through customizable faceted highlights — Fok et al., 2024
suggested reading/skimming time: 1 hr
Synergi: A mixed-initiative system for scholarly synthesis and sensemaking — Kang et al., 2023
suggested reading/skimming time: 1 hr
CollabCoder: A lower-barrier, rigorous workflow for inductive collaborative qualitative analysis with LLMs — Gao et al., 2024
suggested reading/skimming time: 1 hr
Guest lecture on Wednesday:
case studies
case studies
case studies
Submit your reading
responses here
Other assignments
(due before class on Monday, Sep 23)
If you haven’t done so already, look over the notes your classmates have added in the course project brainstorming doc. Use the commenting feature in Google Docs to indicate any of the project directions that you would be interested in potentially collaborating on! In your comments, please briefly note what skills you could contribute to the given project.
During Monday’s class, we will use this information to form project teams!
Required Readings
(due by Tuesday night, Sep 24)
RealityReplay: Detecting and replaying temporal changes in situ using mixed reality — Cho et al., 2023
suggested reading/skimming time: 45 min
A design space for vision augmentations and augmented human perception using digital eyewear — Langlotz et al., 2024
suggested reading/skimming time: 1 hr
case studies
concepts
Submit your reading
responses here
Optional Additional Readings
Supporting serendipity: Opportunities and challenges for human-AI collaboration in qualitative analysis — Jiang et al., 2021
Selenite: Scaffolding online sensemaking with comprehensive overviews elicited from large language models — Liu et al., 2024
Paper Plain: Making medical research papers approachable to healthcare consumers with natural language processing — August et al., 2023
Investigating use cases of AI-powered scene description applications for blind and low vision people — Gonzalez, Collins, et al., 2024
Look once to hear: Target speech hearing with noisy examples – Veluri, Itani et al., 2024
Week 6
(Sep 30 - Oct 4)
Required Readings:
(due by Sunday night, Sep 29)
AUGMENTING
LEARNING & MEMORY
Mutual learning in human-AI interaction — Østerlund et al., 2024
suggested reading/skimming time: 30 min
Flock: Hybrid crowd-machine learning classifiers — Cheng & Bernstein, 2015
suggested reading/skimming time: 1 hr
Rehearsal: Simulating conflict to teach conflict resolution — Shaikh et al., 2024
suggested reading/skimming time: 45 min
Memoro: Using large language models to realize a concise interface for real-time memory augmentation — Zulfikar et al., 2024
suggested reading/skimming time: 45 min
concepts
case studies
case studies
case studies
Submit your reading
responses here
Other assignments
(due before class on Wednesday, Oct 2)
1) Each project team should submit a brief blurb about their planned project focus in this folder. Please feel free to keep it short for now (e.g., 4 to 6 sentences). We will have the next couple of weeks to brainstorm further!
2) In addition, all students should nominate at least one reading for our upcoming ‘student-selected topics’ weeks, in this doc. You can add a reading that you would be interested in discussing and/or add a ‘+1’ to at least one of the readings others have added.
Optional Additional Readings
A conceptual framework for human–AI hybrid adaptivity in education — Holstein et al., 2020
Systematic review of augmented reality training systems — Butaslac et al., 2023
The TA framework: Designing real-time teaching augmentation for K-12 classrooms – An, Holstein et al., 2020
Improving student learning with hybrid human-AI tutoring: A three-study quasi-experimental investigation – Thomas et al., 2024
A piece of theatre: Investigating how teachers design LLM chatbots to assist adolescent cyberbullying education — Hedderich et al., 2024
Week 7
(Oct 7 - 11)
AUGMENTING CREATIVITY
+
COURSE PROJECT STUDIO
Required Readings:
(due by Sunday night, Oct 6)
Augmenting scientific creativity with an analogical search engine — Kang et al., 2022
suggested reading/skimming time: 1 hr, 15 min
AI-assisted causal pathway diagram for human-centered design — Zhong, Shin et al., 2024
suggested reading/skimming time: 1 hr
Exploring the potential for generative AI-based conversational cues for real-time collaborative ideation – Rayan et al., 2024
suggested reading/skimming time: 45 min
case studies
case studies
case studies
Other assignments
(due before class on Monday, Oct 7)
Sign up to give an upcoming Student-Selected Topics presentation on one of the selected papers (in groups of up to 5 students), using this sign-up sheet.
Submit your reading
responses here
Required Readings:
(due by Tuesday night, Oct 8)
Homogenization effects of large language models on human creative ideation — Chong et al., 2024
suggested reading/skimming time: 1 hr
Luminate: Structured generation and exploration of design space with large language models for human-AI co-creation — Suh, Chen et al., 2024
suggested reading/skimming time: 1 hr
case studies
case studies
Submit your reading
responses here
Optional Additional Readings
Generative AI enhances individual creativity but reduces the collective diversity of novel content – Doshi & Hauser, 2024
Exploring challenges and opportunities to support designers in learning to co-create with AI-based manufacturing design tools — Gmeiner et al., 2023
BioSpark: An end-to-end generative system for biological-analogical inspirations and ideation — Kang et al., 2024
Supermind Ideator: How scaffolding human-AI collaboration can increase creativity — Heyman, Rick et al., 2024
The knowledge accelerator: Big picture thinking in small pieces — Hahn et al., 2016
Intermission:
Fall Break!
Week 8
(Oct 21 - 25)
Required Readings:
(due by Sunday night, Oct 20)
AUGMENTING COLLECTIVE INTELLIGENCE
+
COURSE PROJECT STUDIO
Judgment Sieve: Reducing uncertainty in group judgments through interventions targeting ambiguity versus disagreement — Chen & Zhang, 2023
suggested reading/skimming time: 1 hr
Opportunities and risks of LLMs for scalable deliberation with Polis — Small et al., 2023
suggested reading/skimming time: 45 min
A framework for Open Civic Design: Integrating public participation, crowdsourcing, and design thinking – Reynante et al., 2021
suggested reading/skimming time: 1 hr
case studies
case studies
case studies
case studies
concepts
Other assignments
(due by Tuesday night, Oct 22)
It’s time to propose!
Submit an initial draft of your group’s course project proposal in this folder. We will discuss your proposals in class on Wednesday, and iterate as needed!
Submit your reading
responses here
Optional Additional Readings
Supporting reflective public thought with ConsiderIt — Kriplean et al., 2012
Using AI to enhance collective intelligence in virtual teams: Augmenting cognition with technology to help teams adapt to complexity — Woolley et al., 2023
Wikibench: Community-driven data curation for AI evaluation on Wikipedia — Kuo et al., 2024
Soylent: A word processor with a crowd inside — Bernstein et al., 2010
Mapping citizen science through the lens of human-centered AI — Rafner et al., 2022
Human-agent collectives – Jennings et al., 2014
Week 9
(Oct 28 - Nov 1)
Required Readings:
(due by Tuesday night, Oct 29)
COURSE PROJECT STUDIO
+
STUDENT-SELECTED TOPICS:
- Augmenting writing
- Augmenting empathy
A design space for intelligent and interactive writing assistants — Lee et al, 2024
suggested reading/skimming time: 1 hr
Toward machine therapy: Parapraxis of machine design and use — Dobson, 2007
suggested reading/skimming time: 1 hr
Generative AI tools can increase empathy in users — Lau et al, 2024
suggested reading/skimming time: 30 min
concepts
concepts
case studies
Submit your reading
responses here
Week 10
(Nov 4 - 8)
Required Readings:
(due by Tuesday night, Nov 5)
COURSE PROJECT STUDIO
+
STUDENT-SELECTED TOPICS:
- Conceptions of “intelligence”
- Ethics of cognitive augmentation
Unsocial intelligence: An investigation of the assumptions of AGI discourse — Blili-Hamelin et al, 2024
suggested reading/skimming time: 1 hr
Human development or human enhancement? A methodological reflection on capabilities and the evaluation of information technologies — Coeckelbergh, 2011
suggested reading/skimming time: 1 hr
AI extenders: The ethical and societal implications of humans cognitively extended by AI — Hernández-Orallo & Vold, 2019
suggested reading/skimming time: 40 min
concepts
concepts
concepts
Submit your reading
responses here
Week 11
Note:
Class will meet via Zoom on Monday.
There will be no class on Wednesday this week, given the CSCW conference.
(Nov 11 - 15)
CROWD-AUGMENTED COGNITION
Guest lecture on Monday:
Week 12
(Nov 18 - 22)
COURSE PROJECT STUDIO
Week 13
(Nov 25 - 26)
COURSE PROJECT STUDIO
No class on Wednesday
(Thanksgiving)
Week 14
(Dec 2 - 4)
Assignments
(due by Monday night, Dec 2)
Submit the materials for your team’s project presentation (e.g., a pdf of your team’s poster) in this folder.
Please remember to practice giving your presentations / demos!
COURSE PROJECT EXHIBITION
(public exhibition is on Dec 4)
