kestra-io/kestra

Caching example from the documentation fails in the latest version

Closed

#8,540 opened on Apr 27, 2025

View on GitHubContributor guide
 (13 comments) (0 reactions) (1 assignee)Java (2,579 forks)batch import
area/docsgood first issue

Repository metrics

Stars
 (26,854 stars)
PR merge metrics
 (Avg merge 1d 19h) (495 merged PRs in 30d)

Description

Describe the issue

The Python example from the Caching section:

id: python_cached_dependencies
namespace: company.team

tasks:
  - id: working_dir
    type: io.kestra.plugin.core.flow.WorkingDirectory
    tasks:
      - id: python_script
        type: io.kestra.plugin.scripts.python.Script
        taskRunner:
          type: io.kestra.plugin.core.runner.Process
        warningOnStdErr: false
        beforeCommands:
          - python -m venv venv
          - source venv/bin/activate
          - pip install pandas
        script: |
          import pandas as pd
          print(pd.__version__)
    cache:
      patterns:
        - venv/**
      ttl: PT24H

... fails in the latest Kestra version.

The first error is due to the source binary not being available.

ERROR /bin/sh: 3: source: not found

Replacing source with a dot doesn't seem to work, as the log starts showing a warning recommending to use a virtual environment.

ERROR WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

ERROR ModuleNotFoundError: No module named 'pandas'

Environment

  • Kestra Version: 0.22.4 (Docker)

Contributor guide

Research direction
The issue is that the caching example in the documentation uses 'source venv/bin/activate' which fails because the shell (likely /bin/sh) does not support 'source'. The fix should replace 'source' with '. venv/bin/activate' or adjust the command to work with /bin/sh. Additionally, the error about running pip as root suggests the virtual environment may not be activated properly. Investigate the correct sequence of commands for the Script task to ensure the virtual environment is activated before pip install. Update the documentation example accordingly and verify it works with the latest Kestra version (0.22.4) in a Docker environment.
Tech stack
pythonshell
Domain
documentationbackend
Issue type
Documentation
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Basic PythonUnderstanding of Kestra flows
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70