This requires an appropriate group project at MetaCentrum's OpenStack.
One-shot setup of everything within MetaCentrum's OpenStack. This creates the virtual network, etc.
openstack subnet pool create private-172-16-202 --pool-prefix 172.16.202.0/23 --default-prefix-length 23 openstack network create net1 openstack subnet create --network net1 --subnet-pool private-172-16-202 subnet1 openstack router create router1 openstack router set --external-gateway public-cesnet-78-128-251-GROUP router1 openstack port create --network net1 --disable-port-security --fixed-ip ip-address=$(openstack subnet show subnet1 -c gateway_ip -f value) gw1 openstack router add port router1 gw1 openstack floating ip create public-cesnet-78-128-251-GROUP openstack floating ip create public-cesnet-78-128-251-GROUP
Copy Zuul's existing pubkey (extract from Zuul's private key in ci/ansible-cesnet-secrets) and create a keypair in OpenStack:
openstack keypair create --public-key zuul-id_rsa.pub zuul
IMPORTANT: make sure that the build host has HW virtualization supported. This means activating nested virtualization in KVM when building in a VM.
I decided not to use OpenStack's disk-image-builder because that one apparently requires sudo, and I don't think image building should need sudo, sorry.
[root@fedora-32 ~]# time virt-builder fedora-31 --format raw --size 6G --smp 4 --memsize 4096 --no-sync --arch x86_64 --update --install cloud-init,qemu-guest-agent,cloud-utils-growpart,bash,coreutils,procps-ng,systemd,sudo,dnf,rsync,python3-libselinux,openssh-server,hostname,iproute,jq,cmake,git,gcc-c++,make,ninja-build,tree,protobuf-c-devel,flex,bison,pcre-devel,libev-devel,libcmocka-devel,libssh-devel,openssl-devel,systemd-devel,gdb,strace,swig,c-ares-devel,autoconf,automake,libtool,net-snmp-devel,libasan,liblsan,libtsan,libubsan,clang,libcxx-devel,libcxx-static,clang-analyzer,git-clang-format,llvm,doxygen,patch,wget,bzip2,bc,fakeroot,python3-devel,python3-pytest,gcovr,lcov,zstd,expat-devel,clang-tools-extra --timezone Europe/Prague --selinux-relabel --root-password disabled --copy-in /root/cloud-init-default-user.cfg:/etc/cloud/cloud.cfg.d/ --run-command 'dnf debuginfo-install -y glibc libgcc gcc libasan libubsan libstdc++ sssd sssd-client libev libssh pcre protobuf-c' --run-command 'systemctl enable cloud-init; systemctl disable firewalld' -o /tmp/cloud-image-f31 [ 1.0] Downloading: http://builder.libguestfs.org/fedora-31.xz [ 2.2] Planning how to build this image [ 2.2] Uncompressing [ 10.6] Opening the new disk [ 29.8] Setting a random seed [ 29.9] Updating packages [ 590.8] Installing packages: cloud-init qemu-guest-agent cloud-utils-growpart bash coreutils procps-ng systemd sudo dnf rsync python3-libselinux openssh-server hostname iproute jq cmake git gcc-c++ make ninja-build tree protobuf-c-devel flex bison pcre-devel libev-devel libcmocka-devel libssh-devel openssl-devel systemd-devel gdb strace swig c-ares-devel autoconf automake libtool net-snmp-devel libasan liblsan libtsan libubsan clang libcxx-devel libcxx-static clang-analyzer git-clang-format llvm doxygen patch wget bzip2 bc fakeroot python3-devel python3-pytest gcovr lcov zstd expat-devel clang-tools-extra [ 656.4] Setting the timezone: Europe/Prague [ 656.4] Copying: /root/cloud-init-default-user.cfg to /etc/cloud/cloud.cfg.d/ [ 656.5] Running: dnf debuginfo-install -y glibc libgcc gcc libasan libubsan libstdc++ sssd sssd-client libev libssh pcre protobuf-c [ 732.7] Running: systemctl enable cloud-init; systemctl disable firewalld [ 733.2] Setting passwords [ 736.2] SELinux relabelling [ 809.0] Finishing off Output file: /tmp/cloud-image-f31 Output size: 6.0G Output format: raw Total usable space: 5.4G Free space: 1.7G (32%) real 13m30.336s user 0m58.303s sys 0m8.796s
[centos@vm-el8 ~]$ time virt-builder fedora-32 --format raw --size 6G --memsize 4096 --no-sync --arch x86_64 --update --install cloud-init,qemu-guest-agent,cloud-utils-growpart,bash,coreutils,procps-ng,systemd,sudo,dnf,rsync,python3-libselinux,openssh-server,hostname,iproute,jq,cmake,git,gcc-c++,make,ninja-build,tree,protobuf-c-devel,flex,bison,pcre-devel,libev-devel,libcmocka-devel,libssh-devel,openssl-devel,systemd-devel,gdb,strace,swig,c-ares-devel,autoconf,automake,libtool,net-snmp-devel,libasan,liblsan,libtsan,libubsan,clang,libcxx-devel,libcxx-static,clang-analyzer,git-clang-format,llvm,doxygen,patch,wget,bzip2,bc,fakeroot,python3-devel,python3-pytest,gcovr,lcov,zstd,expat-devel,clang-tools-extra,spdlog-devel,boost-devel,python3-pybind11 --timezone Europe/Prague --selinux-relabel --root-password disabled --copy-in /home/centos/cloud-init-default-user.cfg:/etc/cloud/cloud.cfg.d/ --run-command 'dnf debuginfo-install -y glibc libgcc gcc libasan libubsan libstdc++ sssd sssd-client libev libssh pcre protobuf-c libev boost' --run-command 'systemctl enable cloud-init; systemctl disable firewalld' -o cloud-image-f32 [ 1.2] Downloading: http://libguestfs.org/download/builder/fedora-32.xz [ 2.3] Planning how to build this image [ 2.3] Uncompressing [ 11.7] Opening the new disk [ 35.7] Setting a random seed [ 35.7] Updating packages [ 201.3] Installing packages: cloud-init qemu-guest-agent cloud-utils-growpart bash coreutils procps-ng systemd sudo dnf rsync python3-libselinux openssh-server hostname iproute jq cmake git gcc-c++ make ninja-build tree protobuf-c-devel flex bison pcre-devel libev-devel libcmocka-devel libssh-devel openssl-devel systemd-devel gdb strace swig c-ares-devel autoconf automake libtool net-snmp-devel libasan liblsan libtsan libubsan clang libcxx-devel libcxx-static clang-analyzer git-clang-format llvm doxygen patch wget bzip2 bc fakeroot python3-devel python3-pytest gcovr lcov zstd expat-devel clang-tools-extra spdlog-devel boost-devel python3-pybind11 [ 264.5] Setting the timezone: Europe/Prague [ 266.8] Copying: /home/centos/cloud-init-default-user.cfg to /etc/cloud/cloud.cfg.d/ [ 267.0] Running: dnf debuginfo-install -y glibc libgcc gcc libasan libubsan libstdc++ sssd sssd-client libev libssh pcre protobuf-c libev boost [ 317.9] Running: systemctl enable cloud-init; systemctl disable firewalld [ 318.3] Setting passwords [ 321.6] SELinux relabelling [ 358.1] Finishing off Output file: cloud-image-f32 Output size: 6.0G Output format: raw Total usable space: 5.4G Free space: 1.5G (27%) real 5m59.490s user 0m36.237s sys 0m1.535s
The virt-builder templates for F33 are FUBAR, so after wasting a day on that, I settled on another script: files/nodepool/install-f33.sh:
[ci@f33 ~]$ time bash install-f33.sh ... [ 0.0] Examining the guest ... [ 6.2] Setting a random seed [ 6.2] Setting the machine ID in /etc/machine-id [ 6.2] Installing packages: cloud-init qemu-guest-agent cloud-utils-growpart bash coreutils procps-ng systemd sudo dnf rsync python3-libselinux openssh-server hostname iproute jq cmake git gcc-c++ make ninja-build tree flex bison pcre-devel libev-devel libcmocka-devel libssh-devel openssl-devel systemd-devel gdb strace swig c-ares-devel autoconf automake libtool libasan liblsan libtsan libubsan clang libcxx-devel libcxx-static clang-analyzer git-clang-format llvm doxygen patch wget bzip2 bc unzip zip perl-ExtUtils-MakeMaker python3-devel python3-pytest gcovr lcov zstd expat-devel clang-tools-extra spdlog-devel boost-devel python3-pybind11 npm hugo [ 101.0] Editing: /etc/sysconfig/selinux [ 101.1] Copying: /home/ci/cloud-init-default-user.cfg to /etc/cloud/cloud.cfg.d/ [ 101.1] Running: dnf debuginfo-install -y glibc libgcc gcc libstdc++ [ 136.7] Running: systemctl enable cloud-init; systemctl disable firewalld [ 136.9] Running: update-crypto-policies --set DEFAULT:FEDORA32 [ 137.2] Setting passwords [ 138.7] SELinux relabelling [ 170.1] Finishing off + qemu-img convert -f qcow2 -O raw f33.qcow2 f33.raw real 3m10.681s user 0m3.677s sys 0m7.905s
Same as Fedora 33, via a script: files/nodepool/install-f34.sh:
jkt@chobotnice ~/work/cesnet/gerrit/ci/ansible-cesnet[master] $ time bash files/nodepool/install-f34.sh + rm -f f34.qcow2 f34.raw + wget https://download.fedoraproject.org/pub/fedora/linux/releases/34/Cloud/x86_64/images/Fedora-Cloud-Base-34-1.2.x86_64.qcow2 -O f34.qcow2 ... + cat ++ pwd + virt-customize -a f34.qcow2 --install cloud-init,qemu-guest-agent,cloud-utils-growpart,bash,coreutils,procps-ng,systemd,sudo,dnf,rsync,python3-libselinux,openssh-server,hostname,iproute,jq,cmake,git,gcc-c++,make,ninja-build,tree,flex,bison,pcre-devel,libev-devel,libcmocka-devel,libssh-devel,openssl-devel,systemd-devel,gdb,strace,swig,c-ares-devel,autoconf,automake,libtool,libasan,liblsan,libtsan,libubsan,clang,libcxx-devel,libcxx-static,clang-analyzer,git-clang-format,llvm,doxygen,patch,wget,bzip2,bc,unzip,zip,perl-ExtUtils-MakeMaker,python3-devel,python3-pytest,gcovr,lcov,zstd,expat-devel,clang-tools-extra,spdlog-devel,boost-devel,python3-pybind11,npm,hugo --edit '/etc/sysconfig/selinux:s/SELINUX=.*/SELINUX=disabled/' --selinux-relabel --root-password disabled --copy-in /home/jkt/work/cesnet/gerrit/ci/ansible-cesnet/cloud-init-default-user.cfg:/etc/cloud/cloud.cfg.d/ --run-command 'dnf debuginfo-install -y glibc libgcc gcc libstdc++' --run-command 'systemctl enable cloud-init; systemctl disable firewalld' --run-command 'update-crypto-policies --set DEFAULT:FEDORA32' [ 0.0] Examining the guest ... [ 5.0] Setting a random seed [ 5.0] Setting the machine ID in /etc/machine-id [ 5.0] Installing packages: cloud-init qemu-guest-agent cloud-utils-growpart bash coreutils procps-ng systemd sudo dnf rsync python3-libselinux openssh-server hostname iproute jq cmake git gcc-c++ make ninja-build tree flex bison pcre-devel libev-devel libcmocka-devel libssh-devel openssl-devel systemd-devel gdb strace swig c-ares-devel autoconf automake libtool libasan liblsan libtsan libubsan clang libcxx-devel libcxx-static clang-analyzer git-clang-format llvm doxygen patch wget bzip2 bc unzip zip perl-ExtUtils-MakeMaker python3-devel python3-pytest gcovr lcov zstd expat-devel clang-tools-extra spdlog-devel boost-devel python3-pybind11 npm hugo [ 133.2] Editing: /etc/sysconfig/selinux [ 133.3] Copying: /home/jkt/work/cesnet/gerrit/ci/ansible-cesnet/cloud-init-default-user.cfg to /etc/cloud/cloud.cfg.d/ [ 133.3] Running: dnf debuginfo-install -y glibc libgcc gcc libstdc++ [ 174.9] Running: systemctl enable cloud-init; systemctl disable firewalld [ 175.1] Running: update-crypto-policies --set DEFAULT:FEDORA32 [ 175.4] Setting passwords [ 176.8] SELinux relabelling [ 223.6] Finishing off + qemu-img convert -f qcow2 -O raw f34.qcow2 f34.raw real 4m10.395s user 0m23.500s sys 3m27.572s
jkt@chobotnice ~/work/cesnet/gerrit/ci/ansible-cesnet[master*] $ time bash ./files/nodepool/install-f38.sh + URL=https://download.fedoraproject.org/pub/fedora/linux/releases/38/Cloud/x86_64/images/Fedora-Cloud-Base-38-1.6.x86_64.raw.xz ++ basename -s .xz https://download.fedoraproject.org/pub/fedora/linux/releases/38/Cloud/x86_64/images/Fedora-Cloud-Base-38-1.6.x86_64.raw.xz + UNCOMPRESSED_ORIGINAL=Fedora-Cloud-Base-38-1.6.x86_64.raw + IMAGE=f38.raw + [[ ! -f Fedora-Cloud-Base-38-1.6.x86_64.raw ]] + rm -f f38.raw + truncate -s +1G -r Fedora-Cloud-Base-38-1.6.x86_64.raw f38.raw + virt-resize --expand /dev/sda5 Fedora-Cloud-Base-38-1.6.x86_64.raw f38.raw [ 0.0] Examining Fedora-Cloud-Base-38-1.6.x86_64.raw ********** Summary of changes: virt-resize: /dev/sda1: This partition will be left alone. virt-resize: /dev/sda2: This partition will be left alone. virt-resize: /dev/sda3: This partition will be left alone. virt-resize: /dev/sda4: This partition will be left alone. virt-resize: /dev/sda5: This partition will be resized from 3.9G to 4.9G. The filesystem btrfs on /dev/sda5 will be expanded using the ‘btrfs-filesystem-resize’ method. ********** [ 3.1] Setting up initial partition table on f38.raw [ 20.4] Copying /dev/sda1 [ 20.4] Copying /dev/sda2 [ 20.9] Copying /dev/sda3 [ 21.0] Copying /dev/sda4 [ 21.0] Copying /dev/sda5 [ 23.9] Expanding /dev/sda5 using the ‘btrfs-filesystem-resize’ method virt-resize: Resize operation completed with no errors. Before deleting the old disk, carefully check that the resized disk boots and works correctly. + cat ++ pwd + virt-customize -a f38.raw --install cloud-init,qemu-guest-agent,cloud-utils-growpart,bash,coreutils,procps-ng,systemd,sudo,dnf,rsync,python3-libselinux,openssh-server,hostname,iproute,jq,cmake,git,gcc-c++,make,ninja-build,tree,flex,bison,pcre-devel,libev-devel,libcmocka-devel,libssh-devel,openssl-devel,systemd-devel,gdb,strace,swig,c-ares-devel,autoconf,automake,libtool,libasan,liblsan,libtsan,libubsan,clang,libcxx-devel,libcxx-static,clang-analyzer,git-clang-format,llvm,doxygen,patch,wget,bzip2,bc,unzip,zip,perl-ExtUtils-MakeMaker,python3-devel,python3-pytest,gcovr,lcov,zstd,expat-devel,clang-tools-extra,spdlog-devel,boost-devel,python3-pybind11,npm,hugo,pcre2-devel,pam-devel,openssl-perl,json-devel,libnl3-devel,libxml2-devel,date-devel,libdate-tz --edit '/etc/sysconfig/selinux:s/SELINUX=.*/SELINUX=disabled/' --selinux-relabel --root-password disabled --copy-in /home/jkt/work/cesnet/gerrit/ci/ansible-cesnet/cloud-init-default-user.cfg:/etc/cloud/cloud.cfg.d/ --run-command 'dnf debuginfo-install -y glibc libgcc gcc libstdc++' --run-command 'systemctl enable cloud-init' --run-command 'update-crypto-policies --set DEFAULT:FEDORA32' --run-command 'git config --global protocol.file.allow always' [ 0.0] Examining the guest ... [ 4.1] Setting a random seed [ 4.1] Setting the machine ID in /etc/machine-id [ 4.1] Installing packages: cloud-init qemu-guest-agent cloud-utils-growpart bash coreutils procps-ng systemd sudo dnf rsync python3-libselinux openssh-server hostname iproute jq cmake git gcc-c++ make ninja-build tree flex bison pcre-devel libev-devel libcmocka-devel libssh-devel openssl-devel systemd-devel gdb strace swig c-ares-devel autoconf automake libtool libasan liblsan libtsan libubsan clang libcxx-devel libcxx-static clang-analyzer git-clang-format llvm doxygen patch wget bzip2 bc unzip zip perl-ExtUtils-MakeMaker python3-devel python3-pytest gcovr lcov zstd expat-devel clang-tools-extra spdlog-devel boost-devel python3-pybind11 npm hugo pcre2-devel pam-devel openssl-perl json-devel libnl3-devel libxml2-devel date-devel libdate-tz [ 101.8] Editing: /etc/sysconfig/selinux [ 101.9] Copying: /home/jkt/work/cesnet/gerrit/ci/ansible-cesnet/cloud-init-default-user.cfg to /etc/cloud/cloud.cfg.d/ [ 101.9] Running: dnf debuginfo-install -y glibc libgcc gcc libstdc++ [ 127.8] Running: systemctl enable cloud-init [ 127.8] Running: update-crypto-policies --set DEFAULT:FEDORA32 [ 128.2] Running: git config --global protocol.file.allow always [ 128.2] Setting passwords [ 129.0] SELinux relabelling [ 154.2] Finishing off real 2m58.718s user 1m44.830s sys 0m31.977s ...
[centos@vm-el8 ~]$ time openstack image create --property hw_scsi_model=virtio-scsi --property hw_disk_bus=scsi --property hw_rng_model=virtio --property hw_qemu_guest_agent=yes --property os_require_quiesce=yes --property os_type=linux --property os_distro=fedora --private --file cloud-image-f31 ci-zuul-f31--2020-04-20--1 ... real 4m54.106s user 1m5.272s sys 0m22.973s
$ OS_CLOUD=metacentrum openstack server create --flavor hpc.18core-64ram-dukan --network net1 --security-group default --key-name jkt-1 --image ci-zuul-XYZ wtfwtfwtf $ ssh -AC -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o IdentityFile=/home/jkt/work/cesnet/gerrit/ci/ansible-cesnet-secrets/zuul/id_rsa ci@78.128.251.139