Differences

This shows you the differences between two versions of the page.

--- tech:slurm [2020/02/07 12:55] – [Controller] kohofer
+++ tech:slurm [2020/04/24 15:09] – [GCC] kohofer
@@ Line 15: / Line 15: @@
 ===== Installation =====
-==== Controller ====
+===== Controller name: slurm-ctrl =====
-Controller name: slurm-ctrl
 Install slurm-wlm and tools
@@ Line 113: / Line 111: @@
 NodeName=linux1 NodeAddr=10.7.20.98 CPUs=1 State=UNKNOWN
 </code>
+Copy slurm.conf to compute nodes!
   root@slurm-ctrl# scp /etc/slurm-llnl/slurm.conf csadmin@10.7.20.109:/tmp/.; scp /etc/slurm-llnl/slurm.conf csadmin@10.7.20.110:/tmp/.
+  vi /lib/systemd/system/slurmctld.service
+<code>
+[Unit]
+Description=Slurm controller daemon
+After=network.target munge.service
+ConditionPathExists=/etc/slurm-llnl/slurm.conf
+Documentation=man:slurmctld(8)
+[Service]
+Type=forking
+EnvironmentFile=-/etc/default/slurmctld
+ExecStart=/usr/sbin/slurmctld $SLURMCTLD_OPTIONS
+ExecStartPost=/bin/sleep 2
+ExecReload=/bin/kill -HUP $MAINPID
+PIDFile=/var/run/slurm-llnl/slurmctld.pid
+[Install]
+WantedBy=multi-user.target
+</code>
+  vi /lib/systemd/system/slurmd.service
+<code>
+[Unit]
+Description=Slurm node daemon
+After=network.target munge.service
+ConditionPathExists=/etc/slurm-llnl/slurm.conf
+Documentation=man:slurmd(8)
+[Service]
+Type=forking
+EnvironmentFile=-/etc/default/slurmd
+ExecStart=/usr/sbin/slurmd $SLURMD_OPTIONS
+ExecStartPost=/bin/sleep 2
+ExecReload=/bin/kill -HUP $MAINPID
+PIDFile=/var/run/slurm-llnl/slurmd.pid
+KillMode=process
+LimitNOFILE=51200
+LimitMEMLOCK=infinity
+LimitSTACK=infinity
+[Install]
+WantedBy=multi-user.target
+</code>
   root@slurm-ctrl# systemctl daemon-reload
   root@slurm-ctrl# systemctl enable slurmdbd
@@ Line 192: / Line 241: @@
   debug*       up   infinite      1   idle linux1
-==== Compute Nodes ====
+If computer node is down
+<code>
+sinfo -a
+PARTITION AVAIL  TIMELIMIT  NODES  STATE NODELIST
+debug*       up   infinite      2   down gpu[02-03]
+</code>
+  scontrol update nodename=gpu02 state=idle
+  scontrol update nodename=gpu03 state=idle
+  scontrol update nodename=gpu02 state=resume
+<code>
+sinfo -a
+PARTITION AVAIL  TIMELIMIT  NODES  STATE NODELIST
+debug*       up   infinite      2   idle gpu[02-03]
+</code>
+===== Compute Nodes =====
 A compute node is a machine which will receive jobs to execute, sent from the Controller, it runs the slurmd service.
@@ Line 202: / Line 271: @@
   ssh -l csadmin <compute-nodes> 10.7.20.109 10.7.20.110
   sudo apt install slurm-wlm libmunge-dev libmunge2 munge
+  sudo vi /lib/systemd/system/slurmd.service
+<code>
+[Unit]
+Description=Slurm node daemon
+After=network.target munge.service
+ConditionPathExists=/etc/slurm-llnl/slurm.conf
+Documentation=man:slurmd(8)
+[Service]
+Type=forking
+EnvironmentFile=-/etc/default/slurmd
+ExecStart=/usr/sbin/slurmd $SLURMD_OPTIONS
+ExecStartPost=/bin/sleep 2
+ExecReload=/bin/kill -HUP $MAINPID
+PIDFile=/var/run/slurm-llnl/slurmd.pid
+KillMode=process
+LimitNOFILE=51200
+LimitMEMLOCK=infinity
+LimitSTACK=infinity
+[Install]
+WantedBy=multi-user.target
+</code>
   sudo systemctl enable slurmd
   sudo systemctl enable munge
@@ Line 212: / Line 307: @@
   ssh-keygen
-Copy ssh-keys to slurm-ctrl (using IP, because no DNS in place)
+Copy ssh-keys to slurm-ctrl
-  ssh-copy-id -i ~/.ssh/id_rsa.pub csadmin@10.7.20.97:
+  ssh-copy-id -i ~/.ssh/id_rsa.pub csadmin@slurm-ctrl.inf.unibz.it:
 Become root to do important things:
@@ Line 258: / Line 353: @@
 {{ :tech:9-slurm.pdf |Linux Clusters Institute: Scheduling and Resource Management 2017}}
+====== Modules ======
+===== GCC =====
+This take a long time!
+Commands to run to compile gcc-6.1.0
+  wget https://ftp.gnu.org/gnu/gcc/gcc-6.1.0/gcc-6.1.0.tar.bz2
+  tar xfj gcc-6.1.0.tar.bz2
+  cd gcc-6.1.0
+  ./contrib/download_prerequisites
+  ./configure --prefix=/opt/package/gcc/6.1.0 --disable-multilib
+  make
+After some time an error occurs, and the make process stops!
+<code>
+...
+In file included from ../.././libgcc/unwind-dw2.c:401:0:
+./md-unwind-support.h: In function ‘x86_64_fallback_frame_state’:
+./md-unwind-support.h:65:47: error: dereferencing pointer to incomplete type ‘struct ucontext’
+       sc = (struct sigcontext *) (void *) &uc_->uc_mcontext;
+                                               ^~
+../.././libgcc/shared-object.mk:14: recipe for target 'unwind-dw2.o' failed
+</code>
+To fix do: [[https://stackoverflow.com/questions/46999900/how-to-compile-gcc-6-4-0-with-gcc-7-2-in-archlinux|solution]]
+  vi /opt/packages/gcc-6.1.0/x86_64-pc-linux-gnu/libgcc/md-unwind-support.h
+and replace/comment out line 61 with this:
+<code>
+struct ucontext_t *uc_ = context->cfa;
+</code>
+old line: /* struct ucontext *uc_ = context->cfa; */
+  make
+Next error:
+<code>
+../../.././libsanitizer/sanitizer_common/sanitizer_stoptheworld_linux_libcdep.cc:270:22: error: aggregate ‘sigaltstack handler_stack’ has incomplete type and cannot be defined
+   struct sigaltstack handler_stack;
+</code>
+To fix see: [[https://github.com/llvm-mirror/compiler-rt/commit/8a5e425a68de4d2c80ff00a97bbcb3722a4716da?diff=unified|solution]]
+or [[https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81066]]
+Modify the files.
+===== Links =====
+http://www.walkingrandomly.com/?p=5680
+https://modules.readthedocs.io/en/latest/index.html