triton-lang/triton

Use atomic_cas inside a loop

Open

#1,421 opened on Mar 27, 2023

 (1 comment) (0 reactions) (0 assignees)MLIR (3,127 forks)github user discovery
help wanted

Repository metrics

Stars
 (19,982 stars)
PR merge metrics
 (Avg merge 2d 18h) (185 merged PRs in 30d)

Description

Hi, I'm experimenting with a simple kernel that reduces a rank-3 tensor on its last dimension. The kernel uses atomics to sum across one tensor row (I know it's not efficient, this is just a simple reproducer). I'm using two versions of the atomics, the first using atomic_add, and the second using the lock-like mechanism with atomic_cas and atomic_xchg.

@triton.jit
def _sum_kernel(X, Locks, O, width: tl.constexpr, use_atomic_add: tl.constexpr):
  in_off = tl.program_id(0) * width
  out_off = tl.program_id(0)

  for w in range(0, width):
    x = tl.load(X + in_off + w)
    if use_atomic_add:
      tl.atomic_add(O + out_off, x)
    else:
      while tl.atomic_cas(Locks + out_off, 0, 1) == 1:
        pass
      tl.store(O + out_off, tl.load(O + out_off) + x)
      tl.atomic_xchg(Locks + out_off, 0)


def _sum_fn(a, use_atomic_add):
  b, h, w = a.shape
  locks = torch.zeros([b, h], dtype=torch.int32, device='cuda')
  out = torch.zeros([b, h], dtype=a.dtype, device='cuda')
  _sum_kernel[(b * h,)](a, locks, out, width=w, use_atomic_add=use_atomic_add)
  return out

if __name__ == "__main__":
  x = torch.empty((8, 64, 128), dtype=torch.float32, device="cuda").normal_()
  ref = torch.sum(x, axis=-1)
  rtol = 1e-03

  print(torch.allclose(ref, _sum_fn(x, True), rtol=rtol))
  print(torch.allclose(ref, _sum_fn(x, False), rtol=rtol))

Unfortunately for the second version I get


  File "/usr/local/home/giorgio/triton/python/triton/compiler.py", line 741, in visit_For
    assert for_op_region.size() == 1, "We use SCF, so the loop body should only have one block"
AssertionError: We use SCF, so the loop body should only have one block

Any help would be appreciated, thanks.

Contributor guide