Fix Pinned Memory Errors with GGUF Models in ComfyUI

torch.AcceleratorError: CUDA error: invalid argument
Search for 'cudaErrorInvalidValue' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information.

!!! Exception during processing !!! CUDA error: out of memory
Search for `cudaErrorMemoryAllocation' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information.

How to Fix It

Solution 1: Disable Pinned Memory (Most Effective)

This is the fix confirmed by multiple users and recommended by the community. Start ComfyUI with the --disable-pinned-memory flag. This flag tells ComfyUI to avoid using page-locked memory for CUDA tensors, which eliminates the conflict with GGUF model operations.

Steps:

1. Locate your ComfyUI startup script or command. This is typically a batch file (Windows), shell script (Linux/macOS), or a launcher like Stability Matrix.

2. Add the flag to the command. For example, if you normally run:

python main.py

Change it to:

python main.py --disable-pinned-memory

If you use Stability Matrix, edit the package settings or command-line arguments to include --disable-pinned-memory.

3. Restart ComfyUI and run your workflow again.

What to expect: The error should no longer appear. EkstraTuta reports that after adding this flag, the CUDA out-of-memory error with a Qwen Q8 GGUF model was fixed. The trade-off is a potential minor performance decrease in data transfers, but for most users this is negligible compared to the stability gain.

Solution 2: Reduce Resolution or Model Size

If you cannot or prefer not to disable pinned memory, try lowering the video resolution or using a smaller GGUF quantization (e.g., Q4 instead of Q8). The issue reporter notes the bug is more likely at higher resolutions. Reducing the resolution reduces the number of tensor transfers and the VRAM pressure, which may avoid the pinned memory conflict.

Steps:

1. In your workflow, reduce the width and height of the latent image or video input.
2. If using a GGUF model, try a lower quantization (e.g., Q4_K_M instead of Q8_0).
3. Re-run the workflow.

What to expect: The error may stop occurring, but this is a workaround, not a fix. You may lose output quality or resolution.

Solution 3: Update or Reinstall ComfyUI-GGUF Node

The issue may be caused by a bug in the ComfyUI-GGUF custom node itself. The error originates in ops.py and nodes.py of that node. Check for updates to the node via ComfyUI Manager or by reinstalling it from the repository.

Steps:

1. Open ComfyUI Manager (if installed) and check for updates to ComfyUI-GGUF.
2. If no update is available, remove the node from custom_nodes/ComfyUI-GGUF and reinstall it from the official repository: https://github.com/ComfyUI-GGUF/ComfyUI-GGUF
3. Restart ComfyUI.

What to expect: If the node maintainer has patched the pinned memory issue, the error will be resolved. As of the sources, no official fix has been released, so this is a hopeful measure.

Solution 4: Use CUDA_LAUNCH_BLOCKING=1 for Debugging

Both error messages suggest setting CUDA_LAUNCH_BLOCKING=1 to get more accurate stack traces. While this does not fix the error, it can help identify the exact operation that fails, which may be useful for reporting the bug.

Steps:

1. Set the environment variable before starting ComfyUI:

set CUDA_LAUNCH_BLOCKING=1
python main.py

Or on Linux/macOS:

CUDA_LAUNCH_BLOCKING=1 python main.py

2. Re-run the workflow and note the more precise error location.

What to expect: The error will still occur, but the stack trace will point to the exact line in the GGUF node code, which can be shared with developers.

If Nothing Works

If none of the above solutions resolve the error, consider the following escalation paths:

1. Report the issue on the ComfyUI GitHub repository: The original issue is at https://github.com/Comfy-Org/ComfyUI/issues/10662. Add your debug logs, workflow (if possible), and system details (GPU, VRAM, ComfyUI version, GGUF node version).

2. Report to the ComfyUI-GGUF node maintainer: Since the error originates in the custom node, the maintainer may need to fix the ops.py or nodes.py code to handle pinned memory correctly.

3. Check the Reddit thread: EkstraTuta references a Reddit discussion at https://www.reddit.com/r/comfyui/comments/1opqlxv/new_update/ where other users may have additional workarounds.

4. Temporary workaround: Use a non-GGUF version of the model (e.g., full FP16 or safetensors) if available, as the pinned memory issue appears specific to GGUF quantizations.

How to Prevent It

1. Always start ComfyUI with --disable-pinned-memory when using GGUF models, especially for high-resolution video generation or when using multiple samplers in a workflow. This is the most reliable prevention method.

2. Monitor VRAM usage: If you see "loaded partially" messages with high offloaded memory (e.g., "7658.84 MB offloaded" as in the logs), consider reducing model size or resolution to stay within your GPU's VRAM.

3. Keep ComfyUI and custom nodes updated: Regularly check for updates to ComfyUI and the ComfyUI-GGUF node, as future releases may include a proper fix for pinned memory compatibility.

4. Avoid workflows that unload models between samplers: If possible, restructure your workflow to use a single sampler or keep the model loaded throughout. The error is triggered during model unloading, so minimizing unload operations can help.