> ## Documentation Index
> Fetch the complete documentation index at: https://docs.coreweave.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Workload Scheduling

> [card] [Can I run workloads directly on the login node?](https://docs.coreweave.com/support/sunk/articles/can-i-run-workloads-directly-on-the-login-node)
>
> No. The login node is a shared resource for submitting jobs, managing files, and running lightweight commands. Compute-h ...

> [card] [Can I submit Slurm jobs from Kubernetes Pods using slurmrestd?](https://docs.coreweave.com/support/sunk/articles/can-i-submit-slurm-jobs-from-kubernetes-pods-using-slurmrestd)
>
> Yes. SUNK can deploy slurmrestd as an in-cluster Service so authenticated Pods can submit jobs through its REST API inst ...

> [card] [How do I fix torch.distributed rendezvous and TCPStore failures?](https://docs.coreweave.com/support/sunk/articles/how-do-i-fix-torch-distributed-rendezvous-failures)
>
> Your torch.distributed job fails during startup or rendezvous with a c10d or TCPStore error such as sendBytes failed ... ...

> [card] [How do I run containers with Pyxis and Enroot in Slurm?](https://docs.coreweave.com/support/sunk/articles/how-do-i-run-containers-with-pyxis-and-enroot-in-slurm)
>
> Pyxis and Enroot are NVIDIA tools that let you run OCI-compatible container images inside Slurm jobs. SUNK includes Pyxi ...

> [card] [How do I set up the SUNK Pod Scheduler to share Nodes between Pods and Slurm jobs?](https://docs.coreweave.com/support/sunk/articles/how-do-i-set-up-the-sunk-pod-scheduler-to-share-nodes-between-pods-and-slurm-jobs)
>
> The SUNK Pod Scheduler routes Kubernetes Pod placement through Slurm, so Pods and Slurm jobs can share the same nodes. C ...

> [card] [How do I submit my first training job?](https://docs.coreweave.com/support/sunk/articles/how-do-i-submit-my-first-training-job)
>
> After SSH-ing into the login node, use srun for interactive jobs and sbatch for batch scripts. For a quick smoke test, r ...

> [card] [What do NCCL errors mean and how do I diagnose them?](https://docs.coreweave.com/support/sunk/articles/what-do-nccl-errors-mean)
>
> Most NCCL errors aren't NCCL bugs. The message you see usually names the layer where NCCL gave up, not the layer that fa ...

> [card] [Why are CUDA toolkit libraries missing in a SUNK container?](https://docs.coreweave.com/support/sunk/articles/why-are-cuda-toolkit-libraries-missing-in-sunk-containers)
>
> SUNK compute nodes provide NVIDIA GPU drivers to Pyxis containers. They don't install the full CUDA toolkit. Libraries s ...

> [card] [Why are Slurm commands not found inside a Pyxis container?](https://docs.coreweave.com/support/sunk/articles/why-are-slurm-commands-not-found-inside-a-pyxis-container)
>
> Custom container images don't include Slurm CLI tools. sbatch, srun, squeue, sinfo, sacct, and scancel work on the login ...

> [card] [Why do sudo, chown, and chmod fail inside a Pyxis container?](https://docs.coreweave.com/support/sunk/articles/why-do-permission-errors-occur-in-pyxis-enroot-containers)
>
> By default, Enroot remaps container root to your real user ID. That isolation is expected on SUNK, and it's why sudo, ta ...

> [card] [Why does Pyxis report File already exists after a job is preempted?](https://docs.coreweave.com/support/sunk/articles/why-does-pyxis-report-file-already-exists-after-preemption)
>
> After a job is preempted or requeued, Pyxis can fail to start with ERROR File already exists if a leftover container dir ...

> [card] [Why does my SUNK-scheduled Kubernetes Pod fail to start?](https://docs.coreweave.com/support/sunk/articles/why-does-my-sunk-scheduled-pod-fail-to-start)
>
> A SUNK-scheduled Pod fails to start when Slurm and Kubernetes account for a node's resources independently, or when a sc ...

> [card] [Why does my exclusive Slurm job use only one CPU core?](https://docs.coreweave.com/support/sunk/articles/why-does-my-exclusive-slurm-job-use-only-one-cpu-core)
>
> srun --exclusive gives your job the whole node, but when the same srun command also starts the job step and you set --nt ...

> [card] [Why is my Slurm job stuck in PENDING?](https://docs.coreweave.com/support/sunk/articles/why-is-my-slurm-job-stuck-in-pending)
>
> A Slurm job stays in PENDING (PD) when the scheduler has accepted the job but can't start it yet. Run squeue -u USER and ...
