delta-io/delta

[Kernel] Validate readSchema is a subset of the table schema in ScanBuilder

Closed

#2,149 opened on Oct 6, 2023

View on GitHubContributor guide
 (6 comments) (0 reactions) (1 assignee)Scala (2,153 forks)batch import
enhancementgood first issuekernelstale

Repository metrics

Stars
 (8,939 stars)
PR merge metrics
 (Avg merge 7d 1h) (142 merged PRs in 30d)

Description

Feature request

Which Delta project/connector is this regarding?

  • Spark
  • Standalone
  • Flink
  • Kernel
  • Other (fill in here)

Overview

Validate that the read schema provided in ScanBuilder is a subset of the snapshot schema. https://github.com/delta-io/delta/blob/master/kernel/kernel-api/src/main/java/io/delta/kernel/internal/ScanBuilderImpl.java#L80

Motivation

Better validation / error messages.

Contributor guide

Research direction
Examine ScanBuilderImpl.java to understand how readSchema is currently set and validate it as a subset of the table snapshot schema using existing schema utilities.
Tech stack
java
Domain
backenddatabase
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
JavaDelta Kernel concepts
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60