Identifying Learning Outcomes
After studying this material, a student should be able to:
- Define ontological boundaries in the context of designed systems
- Explain how user-authored systems challenge or negotiate those boundaries
- Identify the four sites of ontological boundary negotiation
- Understand Wizard of Oz as a research methodology
- Evaluate why usability/usefulness metrics alone are insufficient for assessing such systems
Step-by-Step Teaching
Step 1: Foundation — What is "Ontology" in System Design?
Core Concept:
Ontology = what we believe exists and matters
When designers build a system, they make hidden decisions about reality:
| Design Decision | Ontological Assumption |
|---|---|
| A fitness app tracks steps | Movement = health |
| A mood app uses emoji scales | Emotions are discrete categories |
| A calendar uses hourly blocks | Time is uniform and measurable |
Key Insight: These decisions shape what users can even think about within the system.
"Designed artifacts are ontological, shaping, and at times limiting, what becomes possible or imaginable."
Step 2: The Problem — Ontological Foreclosure
What is Ontological Foreclosure?
When a system's design closes off possibilities before a user ever interacts with it.
Example:
Traditional sleep tracker
├── Measures: duration, movement, heart rate
├── Assumes: sleep quality = these measurable signals
└── Forecloses: emotional context, environment, relationships
A user who believes their sleep is affected by social anxiety cannot express that within a system that only measures physical signals.
Why This Matters:
- Users are forced to adopt the designer's worldview
- Personal, cultural, or contextual meanings get erased
- The system becomes less personal despite being called "personal sensing"
Step 3: The Proposed Solution — User Authorship
Core Idea: Give people power to define the phenomena themselves rather than accepting pre-built categories.
Two Levels of Authorship:
Level 1 (Traditional): User configures existing categories
→ "Track my sleep from 10pm–6am"
Level 2 (This Paper): User defines what phenomena even means
→ "Track what I call 'mental heaviness'"
Why This Is Difficult:
- Requires machine learning systems that are trainable on user-defined concepts
- Requires researchers to study meaning-making, not just task completion
Step 4: The Methodology — Wizard of Oz Probes
What is Wizard of Oz (WoZ)?
A research technique where:
- The user believes a system is automated/intelligent
- A human researcher secretly operates the system behind the scenes
User Experience: Reality:
[I train my ML model] → [Researcher manually processes inputs]
[System responds] → [Researcher crafts the response]
Why Use WoZ Here?
| Challenge | WoZ Solution |
|---|---|
| Real ML systems are rigid | WoZ allows flexible, human-interpreted responses |
| Building custom ML is expensive | WoZ simulates the experience cheaply |
| We want to study behavior, not technology | WoZ separates the two |
The Two Probes: The researchers designed two open-ended probes — tools given to participants to use freely in their daily lives for one week, allowing participants to define and train their own sensing concepts.
Step 5: Core Findings — The Four Sites of Boundary Negotiation
This is the central contribution of the paper. Participants didn't just use the system — they actively negotiated what the system could mean.
Site 1: Boundaries of a Phenomenon
What happened: Users struggled to define where a phenomenon begins and ends.
Example:
Is "stress" the moment I feel tense? Or does it include the hour before a deadline? Or the whole day?
Phenomenon: "Stress"
├── Narrow boundary: physical tension in shoulders
├── Medium boundary: includes anticipatory anxiety
└── Wide boundary: entire high-demand periods
Design Implication: Systems should allow flexible, evolving definitions rather than fixed labels.
Site 2: The Subject as Part of Relations
What happened: Users recognized that they are not isolated — their states exist in relationship to others, environments, and contexts.
Example:
"My energy level isn't just mine — it depends on who I'm with"
Traditional model: Relational model:
[Individual] → [Data] [Individual ↔ Environment ↔ Others] → [Data]
Design Implication: Personal sensing systems should support relational and contextual data, not just individual metrics.
Site 3: What is Signal vs. What is Noise
What happened: Users challenged the system's (and researcher's) assumptions about what data counts as meaningful.
Example:
A user might consider background music as signal for their emotional state, while the system treats it as noise to be filtered out.
System's assumption: User's reality:
Noise → [filter out] Noise → [meaningful context]
Signal → [process] Signal → [may be irrelevant]
Design Implication: The user should have agency in deciding what counts as relevant data, not just the algorithm.
Site 4: The Objectivity of Data
What happened: Users questioned whether data could ever be truly objective or neutral.
Example:
"When I label this moment as 'calm,' that label is already shaped by my mood right now"
Key Tension:
| Assumption | Challenge |
|---|---|
| Data is neutral and objective | Labeling is always interpretive |
| More data = more truth | Data reflects the moment of collection |
| ML finds patterns in reality | ML finds patterns in labeled reality |
Design Implication: Systems should acknowledge and surface the interpretive nature of data collection rather than presenting outputs as objective truth.
Step 6: Why Traditional Evaluation Misses This
Most systems are evaluated on:
Traditional Metrics:
├── Usability: "Can users figure out how to use it?"
├── Usefulness: "Does it help users accomplish tasks?"
└── Technical Feasibility: "Does the technology work?"
What's Missing:
"Leaving questions of ontological boundary negotiation unexamined"
These metrics assume the system's ontology is correct and ask only whether users can operate within it.
Better Questions to Ask:
- Can users redefine what the system tracks?
- Does the system expand or constrain how users think about themselves?
- Who has power over what counts as meaningful data?
Step 7: Synthesis — The Bigger Picture
Connecting to Broader Themes:
Participatory Design ──────────────────────────────┐
↓
User Authorship → Ontological Negotiation → More Equitable Systems
↑
Critical Data Studies ─────────────────────────────┘
Key Takeaway Framework:
| Concept | Simple Definition | Why It Matters |
|---|---|---|
| Ontological boundary | What a system treats as real/relevant | Shapes user experience fundamentally |
| Foreclosure | When design eliminates possibilities | Limits self-expression and meaning |
| User authorship | Users define the system's categories | Restores agency |
| WoZ probe | Simulated intelligent system | Enables studying behavior before building |
| Boundary negotiation | Active process of redefining limits | The real work users do |
Quick Self-Assessment Questions
- What does it mean for a designed artifact to be "ontological"?
- Why is giving users authorship over system design a response to ontological foreclosure?
- Describe each of the four sites of boundary negotiation in your own words
- Explain why a Wizard of Oz technique was appropriate for this study
- Critique a familiar app (fitness tracker, mood journal) using the four sites framework