[RTX/A6000 GPUs] NaNs in backward pass when training with the huggingface diffusers-style trainer and unet. · Issue #631 · facebookresearch/xformers

Sorry if this bug report is sub-optimal, I'm going to append additional information. When training a diffusion model using a training script similar to/exact with the huggingface diffusers u-net. S...