[Bug]: RuntimeError: Placeholder storage has not been allocated on MPS device!

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

When using controlnet with QR Monster, this error happens: RuntimeError: Placeholder storage has not been allocated on MPS device!

I have found solutions here, but the solution is removing this line of the args --use-cpu-all and that makes everything soooooo much slower, with the argument it takes 1 min to generate an image, without it takes almost 2 hours.

There isn't a way of solving the issue without removing that argument?

(I'm using a Mac with m2 processor running macOS Sonoma, I also use Arc Browser, but that's not an option, so I selected Chrome cuz it's Chromium based anyway)

I'm not an expert, so pls don't hate

Steps to reproduce the problem

Try generating an image using controlnet QR Monster enabled.

What should have happened?

Generated the image normally.

Commit where the problem happens

webui: https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/5ef669de080814067961f28357256e8fe27544f4 controlnet: https://github.com/Mikubill/sd-webui-controlnet/commit/7a4805c8ea3256a0eab3512280bd4f84ca0c8182

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

"--skip-torch-cuda-test --no-half --use-cpu all"

List of enabled extensions

Console logs

Traceback (most recent call last):
      File "/Users/-/stable-diffusion-webui/modules/call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "/Users/-/stable-diffusion-webui/modules/call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "/Users/-/stable-diffusion-webui/modules/txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "/Users/-/stable-diffusion-webui/modules/processing.py", line 732, in process_images
        res = process_images_inner(p)
      File "/Users/-/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/batch_hijack.py", line 42, in processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "/Users/-/stable-diffusion-webui/modules/processing.py", line 867, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "/Users/-/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/hook.py", line 451, in process_sample
        return process.sample_before_CN_hack(*args, **kwargs)
      File "/Users/-/stable-diffusion-webui/modules/processing.py", line 1140, in sample
        samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
      File "/Users/-/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 235, in sample
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "/Users/-/stable-diffusion-webui/modules/sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "/Users/-/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 235, in <lambda>
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "/opt/homebrew/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "/Users/-/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/sampling.py", line 594, in sample_dpmpp_2m
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "/opt/homebrew/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/Users/-/stable-diffusion-webui/modules/sd_samplers_cfg_denoiser.py", line 188, in forward
        x_out[a:b] = self.inner_model(x_in[a:b], sigma_in[a:b], cond=make_condition_dict(c_crossattn, image_cond_in[a:b]))
      File "/opt/homebrew/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/Users/-/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/external.py", line 112, in forward
        eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
      File "/Users/-/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/external.py", line 138, in get_eps
        return self.inner_model.apply_model(*args, **kwargs)
      File "/Users/-/stable-diffusion-webui/modules/sd_hijack_utils.py", line 17, in <lambda>
        setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
      File "/Users/-/stable-diffusion-webui/modules/sd_hijack_utils.py", line 28, in __call__
        return self.__orig_func(*args, **kwargs)
      File "/Users/-/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 858, in apply_model
        x_recon = self.model(x_noisy, t, **cond)
      File "/opt/homebrew/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1538, in _call_impl
        result = forward_call(*args, **kwargs)
      File "/Users/-/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 1335, in forward
        out = self.diffusion_model(x, t, context=cc)
      File "/opt/homebrew/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1538, in _call_impl
        result = forward_call(*args, **kwargs)
      File "/Users/-/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/hook.py", line 858, in forward_webui
        raise e
      File "/Users/-/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/hook.py", line 855, in forward_webui
        return forward(*args, **kwargs)
      File "/Users/-/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/hook.py", line 592, in forward
        control = param.control_model(x=x_in, hint=hint, timesteps=timesteps, context=context, y=y)
      File "/opt/homebrew/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/Users/-/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/cldm.py", line 31, in forward
        return self.control_model(*args, **kwargs)
      File "/opt/homebrew/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/Users/-/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/cldm.py", line 298, in forward
        emb = self.time_embed(t_emb)
      File "/opt/homebrew/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/opt/homebrew/lib/python3.10/site-packages/torch/nn/modules/container.py", line 217, in forward
        input = module(input)
      File "/opt/homebrew/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "/Users/-/stable-diffusion-webui/extensions-builtin/Lora/networks.py", line 429, in network_Linear_forward
        return originals.Linear_forward(self, input)
      File "/opt/homebrew/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
        return F.linear(input, self.weight, self.bias)
    RuntimeError: Placeholder storage has not been allocated on MPS device!

---

Additional information

No response