cython/cython

Optimize list comprehension with known length

Open

#2,844 opened on Feb 15, 2019

 (2 comments) (1 reaction) (0 assignees)Python (1,517 forks)batch import
Optimizationenhancementhelp wanted

Repository metrics

Stars
 (8,663 stars)
PR merge metrics
 (Avg merge 45d 12h) (52 merged PRs in 30d)

Description

Many list comprehensions are of the form

[FOO for i in range(n)]

but Cython does not take advantage of the fact that the length is known to be n in advance. In this example

from cpython.object cimport PyObject

cdef extern from "Python.h":
    void Py_INCREF(PyObject *)
    PyObject * PyInt_FromLong(long ival)
    list PyList_New(Py_ssize_t size)
    void PyList_SET_ITEM(list l, Py_ssize_t, PyObject *)

def listcomp1(long n):
    cdef long i
    return [i*i for i in range(n)]

def listcomp2(long n):
    cdef long i
    cdef list T = PyList_New(n)
    for i in range(n):
        PyList_SET_ITEM(T, i, PyInt_FromLong(i*i))
    return T

the second variant can be 25% faster for a moderately-sized list.

Contributor guide