hacktoberfesttype: bug
Repository metrics
- Stars
- (9,441 stars)
- PR merge metrics
- (Avg merge 51d 21h) (8 merged PRs in 30d)
Description
What's the issue?
This is a collection of issues with Dagster that I personally find important. cc @cmpadden
Blocking development
- Allow calling IOManager dynamically -- big issue in the era of Components, you basically can't load user-defined dependencies via
IOManagerat all. Implemented in 1.11.3 - PartitionedConfig with Pythonic (pydantic) config
Really annoying, causing sub-optimal patterns
- Support generic type hints -- that's kinda a big deal since generics are very common in modern Python, but there is a work around (omitting them and lying to your type checker)
- Partitioned asset checks -- the most serious issue right now is
blockingbehavior. It blocks all downstream partitions, which is just plain wrong. -
dg devuses hard-codedvenvand is not compatible withuvworkspaces - Allow mapping unpartitioned assets into partitions
- should support mapping different partitioned assets into a single partition asset while combining partitions Mapping unpartitioned assets into partitions:
flowchart LR
up1("A") --> down("[A, B]")
up2("B") --> down("[A, B]")
Currently each of the downstream partitions would depend on every upstream asset. This means that if one of them changes, all the downstream partitions will be recomputed (by automation conditions).
Partitions union:
flowchart LR
up1("[A, B]") --> down("[A, B, C]")
up2("[C]") --> down("[A, B, C]")
This is currently not possible at all.
Would kill for this
- Add "re-execute with config" button to the Run page -- the current manual flow is very time-consuming
- Configurables backfills -- it should be easy to implement at this point!
- Support ForwardRef type hints -- also a big deal, they are very common and unlock the ability to instantly derive configs from third party packages which provide them (e.g.
obstore)
What did you expect to happen?
No response
How to reproduce?
No response
Dagster version
1.10.19
Deployment type
Dagster Cloud
Deployment details
No response
Additional information
No response
Message from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.