[Bug]: when hires fix&controlnet all on, batch num ≥ 2 will cause error

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

Steps to reproduce the problem

1. Go to t2i use an sdxl checkpoint
2. set batch num > 2
3. take on hires fix set an sd1.5 checkpoint
4. set controlnet
5. run, and the 2nd picture will crash

What should have happened?

2nd picture output normal picture

Commit where the problem happens

webui: 1.6.0 controlnet: 1.1.411

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--listen --port 7861 --xformers --enable-insecure-extension-access --lyco-dir 'E:\stable-diffusion-webui\models\Lora\Lycoris'

List of enabled extensions

Console logs

cmd logs no error message

Additional information

img info： realistic 3D style Gaming CG, 3d movie cg, character design, fullbody shot, full length shot, wide-angel lens shot, 18mm f1.4, 1lady, vector art, very long dress, solo, (looking at viewer), jewelry, necklace, bare shoulders, A ancient Egyptian looking lady in American landscapes wearing shiny Crinoline gown made of Silk with Veil and Train , big breasts, standing, strapless, strapless dress, cleavage, collarbone, (complex crinoline long dress:1.4), lora:add-detail-xl:6, lora:teengirls_reg:0.3, perfect extremities, detailed background, (detailed fingers, detailed hands, wearing high heels) Negative prompt: (asian:1.5), negativeXL_D, simple background, extra hands, extra fingers, grey background, depth of field, blurry, blurry background, cowboy shot, cut out of legs, standing, Back view, elderly, wrinkles, sorrowful, sad Steps: 20, Sampler: Euler a, CFG scale: 6.5, Seed: 1250908535, Size: 1024x1024, Model hash: 0538f9319d, Model: protovisionXLHighFidelity3D_beta0520Bakedvae, Denoising strength: 0.3, ADetailer model: person_yolov8m-seg.pt, ADetailer confidence: 0.6, ADetailer mask max ratio: 0.5, ADetailer dilate/erode: 4, ADetailer mask blur: 3, ADetailer denoising strength: 0.35, ADetailer inpaint only masked: True, ADetailer inpaint padding: 32, ADetailer use inpaint width/height: True, ADetailer inpaint width: 768, ADetailer inpaint height: 768, ADetailer version: 23.10.1, ControlNet 0: "Module: none, Model: thibaud_xl_openpose [c7b9cadd], Weight: 1.15, Resize Mode: Just Resize, Low Vram: False, Guidance Start: 0, Guidance End: 0.4, Pixel Perfect: True, Control Mode: Balanced"