After studying this material, you should be able to:

1. Explain what a Data Abstraction Layer (DAL) is and why it exists
2. Describe the core components of a DAL and how they interact
3. Understand the table resolution process and its phases
4. Explain how query generation, execution, and result assembly work
5. Evaluate the business value and trade-offs of using a DAL

---

Step-by-Step Study Material

Step 1: Understanding the Problem (Why DAL Exists)

The Core Pain Point

Imagine you are a developer querying data. You receive this instruction:

"Use widget_metrics_v2 for data after 2025. Use widget_metrics_v1 before 2025. Also, the column was renamed."

This creates several serious problems:

The Programming Analogy

In software engineering, an abstraction hides complexity behind a simple interface.

Without abstraction:  Consumer must know HOW data is stored
With abstraction:     Consumer only specifies WHAT data they want

Key Insight: The DAL applies this same principle to data access.

---

Step 2: What Is a Data Abstraction Layer?

Definition

A Data Abstraction Layer (DAL) is an RPC (Remote Procedure Call) service that sits between data consumers and data producers.

Visual Mental Model

┌──────────────────┐         ┌─────────────────┐         ┌──────────────────┐
│  DATA CONSUMER   │ ──────► │       DAL       │ ──────► │  DATA PRODUCER   │
│ (e.g., front end)│ ◄────── │  (the service)  │ ◄────── │ (databases/tables│
└──────────────────┘         └─────────────────┘         └──────────────────┘
   Asks WHAT it wants          Figures out HOW            Stores actual data

DAL Responsibilities

1. Accept requests from consumers
2. Determine where to source data
3. Orchestrate queries across multiple databases
4. Assemble a coherent, unified response

---

Step 3: How You Request Data (The Interface)

The FetchData Endpoint

The DAL exposes one primary endpoint: FetchData

A request contains:

• Table name — which logical table to query
• Schema — which dimensions and metrics you want
• Time range — the period of interest
• Filters — conditions to narrow results

Dimensions vs. Metrics

These are two fundamental concepts you must understand:

Practical Example

Request: "Give me clicks and impressions, broken down by campaign, for the last 7 days"

Table:      ads.advertiser_metrics
Schema:     dimensions: [ad_campaign_uuid]
            metrics:    [clicks, impressions]
Time Range: last 7 days
Filter:     account_uuid = "ABC123"

Key Design Principle:

The consumer specifies WHAT they want. The DAL determines HOW to get it.

Automatic Rollups

If you remove a dimension from your request, the DAL automatically aggregates the data to the appropriate level.

With ad_campaign_uuid    →  Data broken down per campaign
Without ad_campaign_uuid →  Data rolled up to account level (automatically)

The client does nothing extra. The DAL handles it.

---

Step 4: Logical Tables vs. Physical Tables

This distinction is central to understanding the DAL.

Logical Table

A logical table defines the shape of data without describing where or how to get it.

Think of it as a contract or interface:

• Here are the dimensions available
• Here are the metrics available
• Each metric knows which physical tables can provide it

ads.advertiser_metrics (LOGICAL TABLE)
├── Dimensions: account_uuid, ad_campaign_uuid, day_of
└── Metrics:
    ├── clicks        → source: [realtime_metrics, daily_metrics]
    ├── impressions   → source: [realtime_metrics, daily_metrics]
    └── avg_score     → source: [daily_metrics only]

Physical Table

A physical table is the actual database table where data lives. It has:

• A specific schema
• An availability window (freshness + retention)
• Aggregation rules for each metric

Why This Separation Matters

Consumer queries logical table  →  Always the same interface
Physical tables change          →  Consumer is unaffected
New table added                 →  DAL routes automatically
Old table deprecated            →  DAL stops using it silently

---

Step 5: Table Resolution (The Core Intelligence)

Table resolution is the process of generating a query plan — deciding exactly which physical tables to query, for which time ranges, and for which columns.

The Three Phases

Phase 1: Schema Eligibility

"Can this table even answer the question?"

• Discard any physical table that is missing a required dimension
• Discard any physical table that has none of the requested metrics

Request asks for: clicks, impressions, ad_campaign_uuid

Table A has: clicks, impressions, ad_campaign_uuid  ✅ Keep
Table B has: clicks only, no ad_campaign_uuid       ❌ Discard
Table C has: avg_score only                         ❌ Discard

Phase 2: Dataset Availability

"Does this table cover the time range I need?"

Each physical table defines an availability window:

• Freshness — how recent is the data?
• Retention — how far back does it go?

Request: last 7 days

Realtime table:  covers last 0-3 days    ✅ Partially covers
Daily table:     covers last 3-90 days   ✅ Partially covers
Archive table:   covers 1-5 years ago    ❌ Discard (no overlap)

Phase 3: Column Continuity

"When multiple tables overlap, which one is better?"

• For each metric, map out which table covers which time range
• When tables overlap, prefer the table whose cardinality (granularity) is closer to what was requested

Days 1-3:   Use Realtime table (more granular, fresher)
Days 4-7:   Use Daily table (only option for older data)

Table Resolution Output

The result is a concrete execution plan:

Query 1: realtime_metrics  → metrics: [clicks, impressions] → days 1-3
Query 2: hourly_metrics    → metrics: [clicks]              → days 1-3
Query 3: daily_metrics     → metrics: [clicks, impressions] → days 4-7

---

Step 6: Query Generation and Execution

Per-Table Query Generation

For each table in the execution plan, the DAL generates a database-specific query that:

• Selects the right columns
• Applies filters (predicates)
• Performs aggregations
• Runs transformations

Multi-Database Support

The DAL supports multiple database technologies simultaneously:

Single consumer request might trigger:
├── Query to OLAP database  (for recent metrics)
├── Query to Apache Hive    (for historical data)
└── Query to Docstore       (for reference data)

The consumer never needs to know which database is being used.

Parallel Execution

All queries execute in parallel to minimize latency. The DAL waits for all responses before proceeding to result assembly.

---

Step 7: Result Assembly

Once all query responses return, the DAL assembles them into a single coherent result.

Four Assembly Operations

Assembly Example

Query 1 result: [days 1-3, clicks + impressions]
Query 2 result: [days 4-7, clicks + impressions]

Assembly step: CONCATENATE
Final result:  [days 1-7, clicks + impressions]  ✅ Sent to consumer

---

Step 8: Real-World Impact and Value

Before DAL vs. After DAL

Why Dynamic Experiences Become Easy

A pivot table or report builder lets users choose their own dimensions and metrics. With traditional approaches, this is nearly impossible because queries are hardcoded.

With the DAL:

User selects: [campaign, region] as dimensions + [clicks, spend] as metrics
DAL receives: a FetchData request with that schema
DAL handles:  all routing, querying, and assembly automatically

---

Step 9: Summary and Mental Model

The Big Picture in One Diagram

                    CONSUMER
                       │
              "I want clicks by campaign
                  for the last 7 days"
                       │
                       ▼
              ┌────────────────┐
              │  LOGICAL TABLE │  ← Stable interface (never changes)
              │ advertiser_    │
              │   metrics      │
              └────────┬───────┘
                       │
              TABLE RESOLUTION
              (3 phases: eligibility,
               availability, continuity)
                       │
              ┌────────┴────────┐
              ▼                 ▼
        realtime_table    daily_table
        (days 1-3)        (days 4-7)
              │                 │
         PARALLEL QUERIES       │
              └────────┬────────┘
                       │
              RESULT ASSEMBLY
              (concatenate, join,
               rollup, transform)
                       │
                       ▼
              UNIFIED RESPONSE
              sent to consumer

Three Core Principles to Remember

1. Separation of concerns — Consumers say WHAT, DAL figures out HOW
2. Logical vs. physical — Stable interfaces hide unstable implementations
3. Declarative configuration — Behavior is defined in config files, not hardcoded logic

---

Quick Knowledge Check

Test yourself with these questions:

1. What problem does a DAL solve that traditional query approaches cannot?
2. What is the difference between a logical table and a physical table?
3. Name the three phases of table resolution and what each one eliminates.
4. Why does the DAL execute queries in parallel?
5. If a user removes a dimension from their request, what does the DAL do automatically?
6. How does the DAL support multiple database technologies in a single request?

---

Mastering these concepts gives you a strong foundation for understanding modern data infrastructure design, abstraction patterns in distributed systems, and how large-scale platforms like Uber manage data complexity at scale.