tree-sitter/py-tree-sitter

Release Python GIL when possible

Open

#111 opened on May 23, 2022

View on GitHub
 (10 comments) (2 reactions) (0 assignees)C (192 forks)github user discovery
help wanted

Repository metrics

Stars
 (1,487 stars)
PR merge metrics
 (Avg merge 4d 14h) (7 merged PRs in 30d)

Description

I'm looking to search several million source files using the tree-sitter query interface. I'd love to use a thread pool in Python for this. Would it be possible to make a thread-safe Query.captures method that released the GIL?

I tried doing it in https://github.com/grantjenks/py-tree-sitter-languages but couldn't figure it out this evening (not sure it's possible with the current interface).

Contributor guide

Research direction
Study CPython's GIL release macros (Py BEGIN ALLOW THREADS / Py END ALLOW THREADS) and apply them around the tree sitter query operations in the Query.captures method. Ensure thread safety of tree sitter's query API. Test with a Python thread pool.
Tech stack
pythonc
Domain
backendperformance
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonC
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40