How to fix "Provisioning of VM extension vmssCSE has timed out"

Hello Bob

Thank you for contacting us about the VMSS issue.

The error “Provisioning of VM extension vmssCSE has timed out” occurs when the Custom Script Extension (CSE) reports Plugin enabled but does not complete execution within the allowed time. This typically indicates that the script is running but unable to finish, rather than an extension installation failure.

This issue has been resolved in earlier cases by addressing script execution time, outbound connectivity, or extension state corruption.

Cause : -
Long‑running or blocking scripts

Scripts waiting indefinitely for package installation, user input, or external resources

Outbound connectivity issues

VMSS instances cannot reach required endpoints (storage, package repos, external URLs)

Stale or stuck CSE state

Extension reports Plugin enabled but execution never completes

Script re‑execution during scale‑out

Same script behaves differently when triggered on newly added instances

Your VMSS instance does not have a GPU available or the GPU driver cannot attach, so the NVIDIA GRID driver installer fails and that causes the vmssCSE extension to fail and time out.

Steps to perform:

1.Ensure VMSS instances can reach:

• Storage account hosting the script
• OS package repositories (apt/yum)
• Any external URLs referenced in the script

Use Network Watcher → Connection troubleshoot to validate connectivity.

2.Review logs to identify where execution is stuck:

Power shell Commands :

Earlier cases showed scripts stuck during package install or waiting on network calls.

3.If the extension state is corrupted, removing and re‑adding it resolves the timeout:

1. Remove vmssCSE
2. Reapply the extension with updated script or timeout

4.Ensure the script:

• Does not prompt for input
• Can run safely multiple times
• Exits cleanly with proper status

This is critical during scale‑out events where the script runs on new instances.

5**.Reboot – Sometimes. Restart extension / Redeploy**

1. Reboot only restarts the guest OS.
2. It does NOT restart the Custom Script Extension execution.
3. Helped only when the script was stuck due to a temporary OS‑level hang.

If the issue still appears pls help us with this questions:

• Are there any specific changes to the VMSS or its configuration that occurred before this timeout issue started happening?
• Have you checked the outbound connectivity rules and if any recent updates might have affected network access?
• Can you capture and share the specific error messages from the extension logs, specifically those related to the script execution or network errors?
• Could you please share the error logs ?

Thanks,
Manish Deshpande.

Some tries I get this logging:

VMSS Custom Script Extension failure (vmssCSE)
VM has reported a failure when processing extension vmssCSE
Publisher: Microsoft.Azure.Extensions
Type: CustomScript
Enable failed: failed to execute command
Command terminated with exit status 84
ExitCode: 84
Output: 64
Execution log:
2026-01-05T15:27:40 uname -m
2026-01-05T15:27:40 uname -m
2026-01-05T15:27:40 mount -t overlay overlay -o lowerdir=/usr/lib/x86_64-linux-gnu,upperdir=/tmp/overlay/lib64,workdir=/tmp/overlay/workdir /usr/lib/x86_64-linux-gnu
2026-01-05T15:27:40 grid == cuda
2026-01-05T15:27:40 grid == grid
2026-01-05T15:27:40 uname -m
2026-01-05T15:27:40 x86_64 != x86_64
2026-01-05T15:27:40 RUNFILE=NVIDIA-Linux-x86_64-550.144.06-grid-azure
2026-01-05T15:27:40 pushd /opt/gpu
/opt/gpu
2026-01-05T15:27:40 /opt/gpu/NVIDIA-Linux-x86-64-[version]-grid-azure/nvidia-installer -s -k=5.15.0-[version]-azure --log-file-name=[logfile] -a --no-drm --dkms
Warnings from NVIDIA installer:
The nvidia-drm module will not be installed. DRM-KMS will not function.
The installer guessed X library path /usr/lib64 and X module path /usr/lib64/xorg/modules.
Unable to find a suitable destination for 32 bit compatibility libraries.
Vulkan components included but no Vulkan ICD loader detected.
Post install check:
2026-01-05T15:28:32 nvidia-smi
No devices were found
Extension execution details:
exitStatus=6
check_cse_timeout=true
shouldLog=true
maxDurationSeconds=780
elapsedSeconds=494
Executed command:
bash -c ctr -n k8s.io run –privileged –rm –net-host –with-ns pid:/proc/1/ns/pid –mount type=bind,src=/opt/gpu,dst=/mnt/gpu,options=rbind –mount type=bind,src=/opt/actions,dst=/mnt/actions,options=rbind mcr.microsoft.com/aks/aks-gpu-grid:550.144.06-20251021155159 gpuinstall /entrypoint.sh install
Command retried 5 times, last exit status 6
Final error:
Failed to install GPU driver
Timing:
ExecDuration: 494
KernelStartTime: Mon 2026-01-05 15:20:20 UTC
CSEStartTime: Mon 2026-01-05 15:20:38 UTC
Troubleshooting reference:
https://aka.ms/VMExtensionCSELinuxTroubleshoot