apache/airflow

Last automated data interval is always available in custom timetable

Closed

#27,672 opened on Nov 14, 2022

 (6 comments) (0 reactions) (0 assignees)Python (16,781 forks)batch import
area:coregood first issuekind:documentation

Repository metrics

Stars
 (44,809 stars)
PR merge metrics
 (Avg merge 7d 18h) (834 merged PRs in 30d)

Description

Apache Airflow version

Other Airflow 2 version (please specify below)

What happened

I'm writing an example custom timetable. Implemented next_dagrun_info. From the docs and examples the parameter last_automated_data_interval should be None if there are no previous runs.

However, when I start up the example:

  1. I can confirm that the table dag_run is empty.
  2. when starting (unpausing the DAG) for the first time, the last_automated_data_interval is a data interval and not None as specified by documentation.

This raises the question of how to determine the first DAG run (probably could subtract the DataInterval start and start_date from the DAG (if possible).

Here is an example from the logs: airflow-feat-scheduler | [2022-11-14 19:57:58,934] {WorkDayTimetable.py:28} INFO - last_automated_data_interval: DataInterval(start=DateTime(2022, 11, 10, 0, 0, 0, tzinfo=Timezone('UTC')), end=DateTime(2022, 11, 11, 0, 0, 0, tzinfo=Timezone('UTC')))

I'm using Airflow 2.4.2.

What you think should happen instead

The value of the parameter should be None as specified in the docs.

How to reproduce

Should be reproducible by running the example given in the docs and logging the value of the parameter: last_automated_data_interval. Should appear in the logs.

Operating System

macOs Ventura

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

No response

Anything else

The problem occurs every time.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Contributor guide