libyang v3
This is a huge commit which changes a lot of things in the system,
including the boot process and configuration handling. There are also
massive updates all over the stack which have accumulated in the
upstream projects. In no particular order:
- The configuration schema has changed in an incompatible way. That's
due to an update of some drafts related to the ietf-netconf-server
YANG model and its dependencies; as a result, the way how SSH keys and
SSH listening configuration is managed in YANG data is incompatible
with the previous approach, and that means that we *cannot* import the
previous config into sysrepo which already has new YANG modules.
"Solve" that by simply dropping the entire YANG config (!) because we
decided that it's sometimes better to start from scratch. Yay.
- Config migrations now work at the JSON level, not through sysrepo.
There is currently no real "migration" apart from the initial one (the
one that brings the config schema from the implicit/empty "v0" to
"v9"). The TL;DR version is that future migrations will work on an
JSON input, producing some new JSON output, and only after the
migration is done, we push stuff into sysrepo. Also, the migrations
will (likely) not be performed in a sequence, on a cumulative basis,
but it is now a one-shot process with no intermediate steps. This
should hopefully simplify the launching/wrapping code. From the
developer's standpoint, any migration should focus on bringing the
system to the "final state" (v_new), not to v_old + 1.
These migrations will use `jq` heavily. So far, we only have some
"simple" code which merges objects (and arrays!) recursively, courtesy
of Peter Koppstein (https://stackoverflow.com/a/53666584/2245623).
- Some Buildroot options which should have been hardcoded (e.g.,
persisting of OpenSSH server keys) are now unconditional. Once the
czechlight-cfg-fs package is selected, these cannot be disabled.
- We no longer nuke the entire sysrepo SW stack upon a failure. If,
e.g., cla-sysrepo fails, this does not bring the NETCONF server down
anymore, for example. (We might restore the PartOf=... if this proves
to be a problem.)
- There's a bunch of YANG rules which try to make sure that the NETCONF
server is reasonably configured. Previously, we would simply
reprovision the server config at the next boot, now we have nice error
messages at the YANG level already.
- The RESTCONF server has been vastly improved, and some changes all
over that SW stack were made. Also, cleaned up the packaging a bit
(especially those parts which would be a PITA to split into extra
commits due to conflicts everywhere -- sorry).
- Use `jq` patterns for JSON filtering (to make sure that we can provide
an exclusion filter for the crypto material). Also, since we drop
anything but v9 configs now, let's drop the existing migration tests
(there's nothing to test, really).
Change-Id: Id1bb5b9ee66c6deb7a886289e4d768ce3ff7c9b2
Depends-on: https://gerrit.cesnet.cz/plugins/gitiles/github/buildroot/buildroot/+/refs/heads/cesnet/2024-09-06
Depends-on: https://gerrit.cesnet.cz/c/CzechLight/gammarus/+/7570
diff --git a/package/czechlight-cfg-fs/50-czechlight.preset b/package/czechlight-cfg-fs/50-czechlight.preset
index 6b8c73e..2df38e0 100644
--- a/package/czechlight-cfg-fs/50-czechlight.preset
+++ b/package/czechlight-cfg-fs/50-czechlight.preset
@@ -1 +1,5 @@
enable systemd-journal-upload.service
+
+# this is managed by cfg-yang.service
+disable netopeer2-install-yang.service
+disable netopeer2-setup.service
diff --git a/package/czechlight-cfg-fs/CURRENT_CONFIG_VERSION b/package/czechlight-cfg-fs/CURRENT_CONFIG_VERSION
new file mode 100644
index 0000000..ec63514
--- /dev/null
+++ b/package/czechlight-cfg-fs/CURRENT_CONFIG_VERSION
@@ -0,0 +1 @@
+9
diff --git a/package/czechlight-cfg-fs/Config.in b/package/czechlight-cfg-fs/Config.in
index 40e6409..0f7add3 100644
--- a/package/czechlight-cfg-fs/Config.in
+++ b/package/czechlight-cfg-fs/Config.in
@@ -2,6 +2,8 @@
bool "Prepare persistent /cfg partition and /etc overlay. Install required YANG models."
depends on BR2_INIT_SYSTEMD
depends on BR2_PACKAGE_NETOPEER2
+ select BR2_PACKAGE_HOST_JQ
+ select BR2_PACKAGE_JQ
help
This is required for RAUC to work properly. It creates a blank FS
image, configures systemd to mount it, and ensures that its contents
@@ -20,34 +22,4 @@
accommodate all configuration, but small enough to fit within the
corresponding partition.
-if BR2_PACKAGE_SYSREPO
-
-config CZECHLIGHT_CFG_FS_PERSIST_SYSREPO
- bool "Persist sysrepo configuration into /cfg"
- default Y
- help
- Save sysrepo's YANG files into /cfg upon changes
-
-endif # BR2_PACKAGE_SYSREPO
-
-if BR2_PACKAGE_NETOPEER2
-
-config CZECHLIGHT_CFG_FS_PERSIST_KEYS
- bool "Persist host keys for OpenSSH and Netopeer2"
- default Y
- help
- Save OpenSSH's key material and netopeer2's SSH keys into /cfg
-
-endif # BR2_PACKAGE_NETOPEER2
-
-if BR2_PACKAGE_SYSTEMD
-
-config CZECHLIGHT_CFG_FS_PERSIST_NETWORK
- bool "Persist network configuration for eth1"
- default Y
- help
- Save network configuration file for eth1 into /cfg
-
-endif # BR2_PACKAGE_SYSTEMD
-
endif # BR2_PACKAGE_CZECHLIGHT_CFG_FS
diff --git a/package/czechlight-cfg-fs/cfg-filter-key.sh b/package/czechlight-cfg-fs/cfg-filter-key.sh
new file mode 100755
index 0000000..860ad7b
--- /dev/null
+++ b/package/czechlight-cfg-fs/cfg-filter-key.sh
@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+
+# This is a standalone script because it's dealing with cleartexts of crypto keys.
+# The outer wrapper might run with `set -x` to log each command; this separation ensures
+# that we won't leak cleartext keys into the log file.
+
+set -e
+
+PRIVKEY=$(openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -outform PEM 2>/dev/null | grep -v -- "-----" | tr -d "\n")
+sed -e "s|CLEARTEXT_PRIVATE_KEY|\"${PRIVKEY}\"|"
diff --git a/package/czechlight-cfg-fs/cfg-migrate.service b/package/czechlight-cfg-fs/cfg-migrate.service
new file mode 100644
index 0000000..83b7834
--- /dev/null
+++ b/package/czechlight-cfg-fs/cfg-migrate.service
@@ -0,0 +1,14 @@
+[Unit]
+Description=Prepare initial sysrepo configuration
+After=cfg.mount
+Requires=cfg.mount
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+ExecStart=/bin/bash /usr/libexec/czechlight-cfg-fs/cfg-migrate.sh
+Group=sysrepo
+PrivateTmp=yes
+
+[Install]
+WantedBy=multi-user.target
diff --git a/package/czechlight-cfg-fs/cfg-migrate.sh b/package/czechlight-cfg-fs/cfg-migrate.sh
new file mode 100755
index 0000000..9547fdb
--- /dev/null
+++ b/package/czechlight-cfg-fs/cfg-migrate.sh
@@ -0,0 +1,111 @@
+#!/usr/bin/env bash
+
+set -ex
+
+SCRIPT_ROOT=$(dirname $(realpath -s $0))
+CFG_SYSREPO_DIR="${CFG_SYSREPO_DIR:-/cfg/sysrepo}"
+CFG_VERSION_FILE=${CFG_SYSREPO_DIR}/version
+CFG_STARTUP_FILE=${CFG_SYSREPO_DIR}/startup.json
+CFG_STATIC_DATA="${CFG_STATIC_DATA:-/usr/share/yang/static-data/czechlight-cfg-fs}"
+VELIA_STATIC_DATA="${VELIA_STATIC_DATA:-/usr/share/yang/static-data/velia}"
+CLA_STATIC_DATA="${CLA_STATIC_DATA:-/usr/share/yang/static-data/cla-sysrepo}"
+PROC_CMDLINE="${PROC_CMDLINE:-/proc/cmdline}"
+CURRENT_VERSION_FILE="${CURRENT_VERSION_FILE:-/usr/libexec/czechlight-cfg-fs/CURRENT_CONFIG_VERSION}"
+
+if [[ -r "${CFG_VERSION_FILE}" && -f "${CFG_STARTUP_FILE}" ]]; then
+ OLD_VERSION="$(cat ${CFG_VERSION_FILE})"
+else
+ OLD_VERSION=0
+fi
+
+if [[ ! "$OLD_VERSION" =~ ^[0-9]+$ ]]; then
+ echo "Invalid version '${OLD_VERSION}'"
+ exit 1
+fi
+
+NEW_VERSION=$(cat ${CURRENT_VERSION_FILE})
+if (( ${OLD_VERSION} == ${NEW_VERSION} )); then
+ exit
+elif (( ${OLD_VERSION} > ${NEW_VERSION} )); then
+ echo "Attempted to downgrade from ${OLD_VERSION} to ${NEW_VERSION}, that's not supported"
+ exit 1
+fi
+
+rm -rf ${CFG_SYSREPO_DIR}/old/${OLD_VERSION}
+if [[ -f "${CFG_STARTUP_FILE}" ]]; then
+ mkdir -p ${CFG_SYSREPO_DIR}/old/${OLD_VERSION}
+ cp ${CFG_STARTUP_FILE} /etc/os-release ${CFG_SYSREPO_DIR}/old/${OLD_VERSION}/
+fi
+
+# determine which hardware model/variety we're on from /proc/cmdline,
+# e.g., there's a "czechlight=sdn-roadm-line-g2" flag passed from the bootloader
+for ARG in $(cat "$PROC_CMDLINE"); do
+ case "${ARG}" in
+ czechlight=*)
+ CZECHLIGHT="${ARG##czechlight=}"
+ ;;
+ esac
+done
+
+# busybox' mktemp doesn't know --suffix
+DATA_FILE=${DATA_FILE:-$(mktemp -t sr-new-XXXXXX)}
+
+if (( ${OLD_VERSION} < 9 )); then
+ V9_MERGE=(
+ # NACM rules for anonymous access via RESTCONF and for DWDM permissions
+ "${CFG_STATIC_DATA}/nacm.json"
+ # do not treat failures in journal upload as system failures
+ "${CFG_STATIC_DATA}/alarms-shelve-journal-upload.json"
+ # changing one's own passwords/keys
+ "${VELIA_STATIC_DATA}/czechlight-authentication.json"
+ )
+
+ # NETCONF server configuration
+ NETOPEER2_CONFIG=$(mktemp -t sr-nc-XXXXXX)
+ ${SCRIPT_ROOT}/cfg-filter-key.sh < ${CFG_STATIC_DATA}/netopeer2.json.in > ${NETOPEER2_CONFIG}
+ V9_MERGE+=($NETOPEER2_CONFIG)
+
+ # network configuration as well as optical-specific default config
+ case "${CZECHLIGHT}" in
+ sdn-roadm-line*)
+ V9_MERGE+=(
+ "${CLA_STATIC_DATA}/sdn-roadm-line.json"
+ "${CFG_STATIC_DATA}/ietf-interfaces-roadm-line.json"
+ )
+ ;;
+ sdn-roadm-add-drop*)
+ ;& # fallthrough
+ sdn-roadm-hires-add-drop*)
+ V9_MERGE+=(
+ "${CLA_STATIC_DATA}/sdn-roadm-add-drop.json"
+ "${CFG_STATIC_DATA}/ietf-interfaces-roadm-add-drop.json"
+ )
+ ;;
+ sdn-roadm-coherent-a-d*)
+ V9_MERGE+=(
+ "${CLA_STATIC_DATA}/sdn-roadm-coherent-a-d.json"
+ "${CFG_STATIC_DATA}/ietf-interfaces-roadm-add-drop.json"
+ )
+ ;;
+ sdn-inline*)
+ V9_MERGE+=(
+ "${CLA_STATIC_DATA}/sdn-inline.json"
+ "${CFG_STATIC_DATA}/ietf-interfaces-inline-amp.json"
+ )
+ ;;
+ calibration-box)
+ V9_MERGE+=(
+ "${CLA_STATIC_DATA}/calibration-box.json"
+ # no network data on this box
+ )
+ ;;
+ esac
+
+ # libyang v3 mass "migration" means dropping everything, so there's no ${DATA_FILE} as an input
+ jq -f ${SCRIPT_ROOT}/meld.jq ${V9_MERGE[@]} > ${DATA_FILE}
+else
+ cp ${CFG_STARTUP_FILE} ${DATA_FILE}
+fi
+
+cp ${DATA_FILE} ${CFG_STARTUP_FILE}
+echo "${NEW_VERSION}" > ${CFG_VERSION_FILE}
diff --git a/package/czechlight-cfg-fs/cfg-restore-sysrepo.service b/package/czechlight-cfg-fs/cfg-restore-sysrepo.service
deleted file mode 100644
index d289933..0000000
--- a/package/czechlight-cfg-fs/cfg-restore-sysrepo.service
+++ /dev/null
@@ -1,17 +0,0 @@
-[Unit]
-Description=Restore sysrepo startup datastore from /cfg
-After=netopeer2-install-yang.service czechlight-install-yang.service cfg.mount
-Requires=netopeer2-install-yang.service czechlight-install-yang.service cfg.mount
-Before=netopeer2-setup.service netopeer2.service sysrepo-persistent-cfg.service
-ConditionPathExists=/cfg/sysrepo/startup.json
-
-[Service]
-Type=oneshot
-RemainAfterExit=yes
-ExecStart=/bin/sysrepocfg -d startup -f json --import=/cfg/sysrepo/startup.json
-ExecStart=/bin/sysrepocfg -C startup
-Group=sysrepo
-StandardOutput=journal+console
-
-[Install]
-WantedBy=multi-user.target
diff --git a/package/czechlight-cfg-fs/cfg-yang.service b/package/czechlight-cfg-fs/cfg-yang.service
new file mode 100644
index 0000000..becef34
--- /dev/null
+++ b/package/czechlight-cfg-fs/cfg-yang.service
@@ -0,0 +1,16 @@
+[Unit]
+Description=Install YANG modules and persistent data
+After=cfg-migrate.service run-sysrepo.mount
+Requires=cfg-migrate.service
+Before=netopeer2.service
+BindsTo=run-sysrepo.mount
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+ExecStart=/bin/bash /usr/libexec/czechlight-cfg-fs/cfg-yang.sh
+Group=sysrepo
+PrivateTmp=yes
+
+[Install]
+WantedBy=multi-user.target
diff --git a/package/czechlight-cfg-fs/cfg-yang.sh b/package/czechlight-cfg-fs/cfg-yang.sh
new file mode 100755
index 0000000..f437d59
--- /dev/null
+++ b/package/czechlight-cfg-fs/cfg-yang.sh
@@ -0,0 +1,139 @@
+#!/usr/bin/env bash
+
+set -ex
+
+LN2_MODULE_DIR="${LN2_MODULE_DIR:-/usr/share/yang/modules/libnetconf2}"
+NP2_MODULE_DIR="${NP2_MODULE_DIR:-/usr/share/yang/modules/netopeer2}"
+NETOPEER2_SETUP_DIR="${NETOPEER2_SETUP_DIR:-/usr/libexec/netopeer2}"
+CLA_YANG="${CLA_YANG:-/usr/share/yang/modules/cla-sysrepo}"
+VELIA_YANG="${VELIA_YANG:-/usr/share/yang/modules/velia}"
+ALARMS_YANG="${ALARMS_YANG:-/usr/share/yang/modules/sysrepo-ietf-alarms}"
+ROUSETTE_YANG="${ROUSETTE_YANG:-/usr/share/yang/modules/rousette}"
+CFG_FS_YANG="${CFG_FS_YANG:-/usr/share/yang/modules/czechlight-cfg-fs}"
+PROC_CMDLINE="${PROC_CMDLINE:-/proc/cmdline}"
+CFG_SYSREPO_DIR="${CFG_SYSREPO_DIR:-/cfg/sysrepo}"
+
+source ${NETOPEER2_SETUP_DIR}/yang.sh
+
+ROUSETTE_MODULES=(
+ "--install ${ROUSETTE_YANG}/ietf-restconf@2017-01-26.yang"
+ "--install ${ROUSETTE_YANG}/ietf-restconf-monitoring@2017-01-26.yang"
+ "--install ${ROUSETTE_YANG}/ietf-yang-patch@2017-02-22.yang"
+)
+ALARM_MODULES=(
+ "--install ${ALARMS_YANG}/ietf-alarms@2019-09-11.yang"
+ "--enable-feature alarm-shelving"
+ "--enable-feature alarm-summary"
+ "--install ${ALARMS_YANG}/sysrepo-ietf-alarms@2022-02-17.yang"
+)
+VELIA_MODULES=(
+ "--install ${VELIA_YANG}/ietf-system@2014-08-06.yang"
+ "--install ${VELIA_YANG}/czechlight-lldp@2020-11-04.yang"
+ "--install ${VELIA_YANG}/czechlight-system@2022-07-08.yang"
+ "--install ${VELIA_YANG}/iana-if-type@2017-01-19.yang"
+ # sysrepoctl doesn't like duplicates, and the ietf-interfaces and
+ # ietf-ip modules are now dependencies of ietf-netconf-server
+ # "--install ${VELIA_YANG}/ietf-interfaces@2018-02-20.yang"
+ # "--install ${VELIA_YANG}/ietf-ip@2018-02-22.yang"
+ "--install ${VELIA_YANG}/ietf-routing@2018-03-13.yang"
+ "--install ${VELIA_YANG}/ietf-ipv4-unicast-routing@2018-03-13.yang"
+ "--install ${VELIA_YANG}/ietf-ipv6-unicast-routing@2018-03-13.yang"
+ "--install ${VELIA_YANG}/czechlight-network@2021-02-22.yang"
+ "--install ${VELIA_YANG}/ietf-access-control-list@2019-03-04.yang"
+ "--enable-feature match-on-eth"
+ "--enable-feature eth"
+ "--enable-feature match-on-ipv4"
+ "--enable-feature ipv4"
+ "--enable-feature match-on-ipv6"
+ "--enable-feature ipv6"
+ "--enable-feature mixed-eth-ipv4-ipv6"
+ "--install ${VELIA_YANG}/czechlight-firewall@2021-01-25.yang"
+ "--install ${VELIA_YANG}/velia-alarms@2022-07-12.yang"
+)
+CFG_FS_MODULES=(
+ "--install ${CFG_FS_YANG}/czechlight-netconf-server@2024-09-04.yang"
+)
+CLA_MODULES=(
+ "--install ${CLA_YANG}/iana-hardware@2018-03-13.yang"
+ "--install ${CLA_YANG}/ietf-hardware@2018-03-13.yang"
+ "--enable-feature hardware-sensor"
+ "--enable-feature hardware-state"
+)
+
+# determine which hardware model/variety we're on from /proc/cmdline,
+# e.g., there's a "czechlight=sdn-roadm-line-g2" flag passed from the bootloader
+for ARG in $(cat "$PROC_CMDLINE"); do
+ case "${ARG}" in
+ czechlight=*)
+ CZECHLIGHT="${ARG##czechlight=}"
+ ;;
+ esac
+done
+
+case "${CZECHLIGHT}" in
+ "")
+ # no device model set -> do nothing
+ ;;
+ sdn-roadm-line*)
+ CLA_MODULES+=(
+ "--install ${CLA_YANG}/czechlight-roadm-device@2021-03-05.yang"
+ "--enable-feature hw-line-9"
+ )
+ ;;
+ sdn-roadm-add-drop*)
+ CLA_MODULES+=(
+ "--install ${CLA_YANG}/czechlight-roadm-device@2021-03-05.yang"
+ "--enable-feature hw-add-drop-20"
+ )
+ ;;
+ sdn-roadm-hires-add-drop*)
+ CLA_MODULES+=(
+ "--install ${CLA_YANG}/czechlight-roadm-device@2021-03-05.yang"
+ "--enable-feature hw-add-drop-20"
+ "--enable-feature pre-wss-ocm"
+ )
+ ;;
+ sdn-roadm-coherent-a-d*)
+ CLA_MODULES+=(
+ "--install ${CLA_YANG}/czechlight-coherent-add-drop@2021-03-05.yang"
+ )
+ ;;
+ sdn-inline*)
+ CLA_MODULES+=(
+ "--install ${CLA_YANG}/czechlight-inline-amp@2021-03-05.yang"
+ )
+ ;;
+ calibration-box)
+ CLA_MODULES+=(
+ "--install ${CLA_YANG}/czechlight-calibration-device@2019-06-25.yang"
+ )
+ ;;
+ sdn-bidi-cplus1572-g2)
+ CLA_MODULES+=(
+ "--install ${CLA_YANG}/czechlight-bidi-amp@2022-03-22.yang"
+ "--enable-feature dualband-c-plus-1572"
+ )
+ ;;
+ sdn-bidi-cplus1572-ocm-g2)
+ CLA_MODULES+=(
+ "--install ${CLA_YANG}/czechlight-bidi-amp@2022-03-22.yang"
+ "--enable-feature dualband-c-plus-1572"
+ "--enable-feature c-band-ocm"
+ )
+ ;;
+ *)
+ echo "Error: unsupported CzechLight device model ${CZECHLIGHT}"
+ exit 1
+ ;;
+esac
+
+sysrepoctl \
+ -v2 \
+ --search-dirs ${NP2_MODULE_DIR}:${CLA_YANG}:${VELIA_YANG}:${ALARMS_YANG}:${ROUSETTE_YANG} \
+ ${NETOPEER2_YANG_SETUP[@]} \
+ ${ROUSETTE_MODULES[@]} \
+ ${ALARM_MODULES[@]} \
+ ${VELIA_MODULES[@]} \
+ ${CFG_FS_MODULES[@]} \
+ ${CLA_MODULES[@]} \
+ --init-data ${CFG_SYSREPO_DIR}/startup.json
diff --git a/package/czechlight-cfg-fs/czechlight-cfg-fs.mk b/package/czechlight-cfg-fs/czechlight-cfg-fs.mk
index 8ac58dd..6590873 100644
--- a/package/czechlight-cfg-fs/czechlight-cfg-fs.mk
+++ b/package/czechlight-cfg-fs/czechlight-cfg-fs.mk
@@ -25,20 +25,11 @@
endef
CZECHLIGHT_CFG_FS_SYSTEMD_FOR_MULTIUSER = \
- czechlight-install-yang.service \
- czechlight-migrate.service
-
-$(ifeq ($(CZECHLIGHT_CFG_FS_PERSIST_SYSREPO),y))
- CZECHLIGHT_CFG_FS_SYSTEMD_FOR_MULTIUSER += \
- sysrepo-persistent-cfg.service \
- cfg-restore-sysrepo.service
-$(endif)
-$(ifeq ($(CZECHLIGHT_CFG_FS_PERSIST_KEYS),y))
- CZECHLIGHT_CFG_FS_SYSTEMD_FOR_MULTIUSER += openssh-persistent-keys.service
-$(endif)
-$(ifeq ($(CZECHLIGHT_CFG_FS_PERSIST_NETWORK),y))
- CZECHLIGHT_CFG_FS_SYSTEMD_FOR_MULTIUSER += cfg-restore-systemd-networkd.service
-$(endif)
+ cfg-yang.service \
+ cfg-migrate.service \
+ sysrepo-persistent-cfg.service \
+ openssh-persistent-keys.service \
+ cfg-restore-systemd-networkd.service
define CZECHLIGHT_CFG_FS_INSTALL_TARGET_CMDS
mkdir -p $(TARGET_DIR)/cfg
@@ -50,12 +41,20 @@
$(@D)/czechlight-random-seed
$(INSTALL) -D -m 0755 -t $(TARGET_DIR)/usr/libexec/czechlight-cfg-fs \
- $(BR2_EXTERNAL_CZECHLIGHT_PATH)/package/czechlight-cfg-fs/czechlight-install-yang.sh \
- $(BR2_EXTERNAL_CZECHLIGHT_PATH)/package/czechlight-cfg-fs/czechlight-migrate.sh \
- $(BR2_EXTERNAL_CZECHLIGHT_PATH)/package/czechlight-cfg-fs/czechlight-migration-list.sh
+ $(BR2_EXTERNAL_CZECHLIGHT_PATH)/package/czechlight-cfg-fs/cfg-yang.sh \
+ $(BR2_EXTERNAL_CZECHLIGHT_PATH)/package/czechlight-cfg-fs/cfg-migrate.sh \
+ $(BR2_EXTERNAL_CZECHLIGHT_PATH)/package/czechlight-cfg-fs/cfg-filter-key.sh
- $(INSTALL) -D -m 0644 -t $(TARGET_DIR)/usr/libexec/czechlight-cfg-fs/migrations \
- $(BR2_EXTERNAL_CZECHLIGHT_PATH)/package/czechlight-cfg-fs/migrations/*
+ $(INSTALL) -D -m 0644 -t $(TARGET_DIR)/usr/libexec/czechlight-cfg-fs \
+ $(BR2_EXTERNAL_CZECHLIGHT_PATH)/package/czechlight-cfg-fs/meld.jq \
+ $(BR2_EXTERNAL_CZECHLIGHT_PATH)/package/czechlight-cfg-fs/CURRENT_CONFIG_VERSION
+
+ $(INSTALL) -D -m 0644 -t $(TARGET_DIR)/usr/share/yang/static-data/czechlight-cfg-fs \
+ $(BR2_EXTERNAL_CZECHLIGHT_PATH)/package/czechlight-cfg-fs/static-data/*.json \
+ $(BR2_EXTERNAL_CZECHLIGHT_PATH)/package/czechlight-cfg-fs/static-data/*.json.in
+
+ $(INSTALL) -D -m 0644 -t $(TARGET_DIR)/usr/share/yang/modules/czechlight-cfg-fs \
+ $(BR2_EXTERNAL_CZECHLIGHT_PATH)/package/czechlight-cfg-fs/yang/*.yang
for UNIT in $(CZECHLIGHT_CFG_FS_SYSTEMD_FOR_MULTIUSER); do \
@@ -66,6 +65,32 @@
$(INSTALL) -D -m 644 -t $(TARGET_DIR)/usr/lib/systemd/system-preset \
$(BR2_EXTERNAL_CZECHLIGHT_PATH)/package/czechlight-cfg-fs/50-czechlight.preset
+ $(INSTALL) -D -m 0644 --target-directory $(TARGET_DIR)/usr/lib/systemd/system/ \
+ $(BR2_EXTERNAL_CZECHLIGHT_PATH)/package/czechlight-cfg-fs/run-sysrepo.mount
+ $(INSTALL) -d -m 0755 $(TARGET_DIR)/usr/lib/systemd/system/run-sysrepo.mount.d/
+ for UNIT in \
+ cla-sdn-inline.service \
+ cla-sdn-roadm-add-drop.service \
+ cla-sdn-roadm-coherent-a-d.service \
+ cla-sdn-roadm-hires-drop.service \
+ cla-sdn-roadm-line.service \
+ cla-sdn-bidi-cplus1572.service \
+ cla-sdn-bidi-cplus1572-ocm.service \
+ netopeer2.service \
+ sysrepo-ietf-alarms.service \
+ sysrepo-persistent-cfg.service \
+ sysrepo-plugind.service \
+ velia-firewall.service \
+ velia-health.service \
+ velia-system.service \
+ velia-hardware-g1.service \
+ velia-hardware-g2.service \
+ rousette.service \
+ ; do \
+ $(INSTALL) -d -m 0755 $(TARGET_DIR)/usr/lib/systemd/system/$${UNIT}.d/ ; \
+ echo -e "[Unit]\nBindsTo=run-sysrepo.mount\nAfter=run-sysrepo.mount" \
+ > $(TARGET_DIR)/usr/lib/systemd/system/$${UNIT}.d/reset-sysrepo.conf ; \
+ done
endef
# Configure OpenSSH to look for *user* keys in the /cfg
@@ -74,9 +99,27 @@
endef
OPENSSH_POST_INSTALL_TARGET_HOOKS += CZECHLIGHT_CFG_FS_OPENSSH_AUTH_PATH_PATCH
-NETOPEER2_CONF_OPTS += \
- -DNP2SRV_SSH_AUTHORIZED_KEYS_PATTERN="/cfg/ssh-user-auth/%s" \
- -DNP2SRV_SSH_AUTHORIZED_KEYS_ARG_IS_USERNAME=ON
+NETOPEER2_CONF_OPTS += -DSSH_AUTHORIZED_KEYS_FORMAT="/cfg/ssh-user-auth/%u"
+
+VELIA_CONF_OPTS += \
+ -DVELIA_BACKUP_ETC_SHADOW=/cfg/etc/shadow \
+ -DVELIA_BACKUP_ETC_HOSTNAME=/cfg/etc/hostname \
+ -DVELIA_AUTHORIZED_KEYS_FORMAT="/cfg/ssh-user-auth/{USER}"
+
+# Do not use buildroot's stock installation scripts
+define CZECHLIGHT_CFG_FS_OVERRIDE_NETOPEER_UNITS
+ $(SED) 's|netopeer2-setup.service|cfg-yang.service|g' $(TARGET_DIR)/usr/lib/systemd/system/netopeer2.service
+endef
+NETOPEER2_POST_INSTALL_TARGET_HOOKS += CZECHLIGHT_CFG_FS_OVERRIDE_NETOPEER_UNITS
+
+# Do not clutter /dev/shm, use a proper prefix for sysrepo
+define RESET_SYSREPO_PATCH_DEV_SHM
+ sed -i \
+ 's|^#define SR_SHM_DIR .*|#define SR_SHM_DIR "/run/sysrepo"|' \
+ $(@D)/src/config.h.in
+endef
+SYSREPO_PRE_PATCH_HOOKS += RESET_SYSREPO_PATCH_DEV_SHM
+SYSREPO_POST_RSYNC_HOOKS += RESET_SYSREPO_PATCH_DEV_SHM
.PHONY: czechlight-cfg-fs-test-migrations
czechlight-cfg-fs-test-migrations: PKG=czechlight-cfg-fs
@@ -87,7 +130,13 @@
VELIA_SRCDIR=$(VELIA_SRCDIR) \
SYSREPO_IETF_ALARMS_SRCDIR=$(SYSREPO_IETF_ALARMS_SRCDIR) \
ROUSETTE_SRCDIR=$(ROUSETTE_SRCDIR) \
+ LIBNETCONF2_SRCDIR=$(LIBNETCONF2_SRCDIR) \
NETOPEER2_SRCDIR=$(NETOPEER2_SRCDIR) \
- pytest -vv $(BR2_EXTERNAL_CZECHLIGHT_PATH)/tests/czechlight-cfg-fs/migrations.py
+ PYTHONDONTWRITEBYTECODE=1 \
+ pytest \
+ -vv \
+ --basetemp $(BUILD_DIR)/czechlight-cfg-fs/pytest \
+ -o tmp_path_retention_count=1 \
+ $(BR2_EXTERNAL_CZECHLIGHT_PATH)/tests/czechlight-cfg-fs/migrations.py
$(eval $(generic-package))
diff --git a/package/czechlight-cfg-fs/czechlight-install-yang.service b/package/czechlight-cfg-fs/czechlight-install-yang.service
deleted file mode 100644
index 80e89fc..0000000
--- a/package/czechlight-cfg-fs/czechlight-install-yang.service
+++ /dev/null
@@ -1,14 +0,0 @@
-[Unit]
-Description=Install CzechLight YANG models
-After=netopeer2-install-yang.service cfg.mount
-Requires=netopeer2-install-yang.service cfg.mount
-Before=netopeer2.service cfg-restore-sysrepo.service czechlight-migrate.service
-
-[Service]
-Type=oneshot
-RemainAfterExit=yes
-ExecStart=/bin/bash /usr/libexec/czechlight-cfg-fs/czechlight-install-yang.sh
-Group=sysrepo
-
-[Install]
-WantedBy=multi-user.target
diff --git a/package/czechlight-cfg-fs/czechlight-install-yang.sh b/package/czechlight-cfg-fs/czechlight-install-yang.sh
deleted file mode 100755
index df1a828..0000000
--- a/package/czechlight-cfg-fs/czechlight-install-yang.sh
+++ /dev/null
@@ -1,77 +0,0 @@
-#!/bin/bash
-
-set -ex
-
-CLA_YANG="${CLA_YANG:-/usr/share/cla-sysrepo/yang}"
-VELIA_YANG="${VELIA_YANG:-/usr/share/velia/yang}"
-ALARMS_YANG="${ALARMS_YANG:-/usr/share/sysrepo-ietf-alarms/yang}"
-ROUSETTE_YANG="${ROUSETTE_YANG:-/usr/share/rousette/yang}"
-PROC_CMDLINE="${PROC_CMDLINE:-/proc/cmdline}"
-
-for ARG in $(cat "$PROC_CMDLINE"); do
- case "${ARG}" in
- czechlight=*)
- CZECHLIGHT="${ARG:11}"
- ;;
- esac
-done
-
-case "${CZECHLIGHT}" in
- sdn-roadm-line*)
- DEVICE_YANG="--install ${CLA_YANG}/czechlight-roadm-device@2021-03-05.yang --enable-feature hw-line-9"
- ;;
- sdn-roadm-add-drop*)
- DEVICE_YANG="--install ${CLA_YANG}/czechlight-roadm-device@2021-03-05.yang --enable-feature hw-add-drop-20"
- ;;
- sdn-roadm-hires-add-drop*)
- DEVICE_YANG="--install ${CLA_YANG}/czechlight-roadm-device@2021-03-05.yang --enable-feature hw-add-drop-20 --enable-feature pre-wss-ocm"
- ;;
- sdn-roadm-coherent-a-d*)
- DEVICE_YANG="--install ${CLA_YANG}/czechlight-coherent-add-drop@2021-03-05.yang"
- ;;
- sdn-inline*)
- DEVICE_YANG="--install ${CLA_YANG}/czechlight-inline-amp@2021-03-05.yang"
- ;;
- calibration-box)
- DEVICE_YANG="--install ${CLA_YANG}/czechlight-calibration-device@2019-06-25.yang"
- ;;
- sdn-bidi-cplus1572-g2)
- DEVICE_YANG="--install ${CLA_YANG}/czechlight-bidi-amp@2022-03-22.yang --enable-feature dualband-c-plus-1572"
- ;;
- sdn-bidi-cplus1572-ocm-g2)
- DEVICE_YANG="--install ${CLA_YANG}/czechlight-bidi-amp@2022-03-22.yang --enable-feature dualband-c-plus-1572 --enable-feature c-band-ocm"
- ;;
-esac
-
-sysrepoctl \
- --search-dirs ${CLA_YANG}:${VELIA_YANG}:${ALARMS_YANG}:${ROUSETTE_YANG} \
- --install ${CLA_YANG}/iana-hardware@2018-03-13.yang \
- --install ${CLA_YANG}/ietf-hardware@2018-03-13.yang \
- --enable-feature hardware-sensor \
- --enable-feature hardware-state \
- --install ${ALARMS_YANG}/ietf-alarms@2019-09-11.yang \
- --enable-feature alarm-shelving \
- --enable-feature alarm-summary \
- --install ${ALARMS_YANG}/sysrepo-ietf-alarms@2022-02-17.yang \
- --install ${VELIA_YANG}/ietf-system@2014-08-06.yang \
- --install ${VELIA_YANG}/czechlight-lldp@2020-11-04.yang \
- --install ${VELIA_YANG}/czechlight-system@2022-07-08.yang \
- --install ${VELIA_YANG}/iana-if-type@2017-01-19.yang \
- --install ${VELIA_YANG}/ietf-interfaces@2018-02-20.yang \
- --install ${VELIA_YANG}/ietf-ip@2018-02-22.yang \
- --install ${VELIA_YANG}/ietf-routing@2018-03-13.yang \
- --install ${VELIA_YANG}/ietf-ipv4-unicast-routing@2018-03-13.yang \
- --install ${VELIA_YANG}/ietf-ipv6-unicast-routing@2018-03-13.yang \
- --install ${VELIA_YANG}/czechlight-network@2021-02-22.yang \
- --install ${VELIA_YANG}/ietf-access-control-list@2019-03-04.yang \
- --enable-feature match-on-eth \
- --enable-feature eth \
- --enable-feature match-on-ipv4 \
- --enable-feature ipv4 \
- --enable-feature match-on-ipv6 \
- --enable-feature ipv6 \
- --enable-feature mixed-eth-ipv4-ipv6 \
- --install ${VELIA_YANG}/czechlight-firewall@2021-01-25.yang \
- --install ${VELIA_YANG}/velia-alarms@2022-07-12.yang \
- --install ${ROUSETTE_YANG}/ietf-restconf@2017-01-26.yang \
- ${DEVICE_YANG}
diff --git a/package/czechlight-cfg-fs/czechlight-migrate.service b/package/czechlight-cfg-fs/czechlight-migrate.service
deleted file mode 100644
index 7e58d4d..0000000
--- a/package/czechlight-cfg-fs/czechlight-migrate.service
+++ /dev/null
@@ -1,14 +0,0 @@
-[Unit]
-Description=Migrate CzechLight YANG models startup data
-After=netopeer2-install-yang.service czechlight-install-yang.service cfg-restore-sysrepo.service cfg.mount
-Requires=netopeer2-install-yang.service czechlight-install-yang.service cfg-restore-sysrepo.service cfg.mount
-Before=netopeer2-setup.service netopeer2.service sysrepo-persistent-cfg.service
-
-[Service]
-Type=oneshot
-RemainAfterExit=yes
-ExecStart=/bin/bash /usr/libexec/czechlight-cfg-fs/czechlight-migrate.sh
-Group=sysrepo
-
-[Install]
-WantedBy=multi-user.target
diff --git a/package/czechlight-cfg-fs/czechlight-migrate.sh b/package/czechlight-cfg-fs/czechlight-migrate.sh
deleted file mode 100755
index 2046010..0000000
--- a/package/czechlight-cfg-fs/czechlight-migrate.sh
+++ /dev/null
@@ -1,88 +0,0 @@
-#!/bin/bash
-
-set -ex
-
-SCRIPT_ROOT=$(dirname $(realpath -s $0))
-MIGRATIONS=$SCRIPT_ROOT/czechlight-migration-list.sh
-export MIGRATIONS_DIRECTORY=${SCRIPT_ROOT}/migrations
-CFG_VERSION_FILE="${CFG_VERSION_FILE:-/cfg/sysrepo/version}"
-CFG_STARTUP_FILE="${CFG_STARTUP_FILE:-/cfg/sysrepo/startup.json}"
-PROC_CMDLINE="${PROC_CMDLINE:-/proc/cmdline}"
-
-export CLA_YANG="${CLA_YANG:-/usr/share/cla-sysrepo/yang}"
-export VELIA_YANG="${VELIA_YANG:-/usr/share/velia/yang}"
-export ALARMS_YANG="${ALARMS_YANG:-/usr/share/sysrepo-ietf-alarms/yang}"
-
-# load migrations and perform a sanity check (filename's numerical prefix corresponds to the order in the MIGRATIONS array)
-source $MIGRATIONS
-for i in $(seq ${#MIGRATION_FILES[@]}); do
- FILENAME=${MIGRATION_FILES[$(($i - 1))]}
-
- if ! [[ "$FILENAME" =~ ^[0-9]{4}_.*.sh$ ]]; then
- echo "Migration file '$FILENAME' has unexpected name"
- exit 1
- fi
-
- FILE_REV=$(echo "$FILENAME" | grep -Eo "^[0-9]{4}")
- if [[ $((FILE_REV + 0)) != $i ]]; then
- echo "Migration filename '$FILENAME' hints revision $(($FILE_REV + 0)) but it is at position $i of the migration array"
- exit 1
- fi
-done
-
-for ARG in $(cat "$PROC_CMDLINE"); do
- case "${ARG}" in
- czechlight=*)
- export CZECHLIGHT="${ARG:11}"
- ;;
- esac
-done
-
-case "${CZECHLIGHT}" in
- sdn-roadm-line*)
- export YANG_ROADM=1
- ;;
- sdn-roadm-add-drop*)
- export YANG_ROADM=1
- ;;
- sdn-roadm-hires-add-drop*)
- export YANG_ROADM=1
- ;;
- sdn-roadm-coherent-a-d*)
- export YANG_COHERENT=1
- ;;
- sdn-inline*)
- export YANG_INLINE=1
- ;;
- calibration-box)
- export YANG_CALIBRATION=1
- ;;
-esac
-
-
-# we might end up on the system
-# * that was created before the migrations were introduced; such system does not have ${CFG_VERSION_FILE}
-# * that was just created and freshly initialized with firmware; it has ${CFG_VERSION_FILE} but startup.json does not exist
-if [[ -r "$CFG_VERSION_FILE" && -f "$CFG_STARTUP_FILE" ]]; then
- CURRENT_VERSION="$(cat ${CFG_VERSION_FILE})"
-else
- CURRENT_VERSION=0
-fi
-
-if [[ ! "$CURRENT_VERSION" =~ ^[0-9]+$ ]]; then
- echo "Invalid version '$CURRENT_VERSION'"
- exit 1
-fi
-
-while [[ $CURRENT_VERSION -lt ${#MIGRATION_FILES[@]} ]]; do
- /bin/bash ${SCRIPT_ROOT}/migrations/${MIGRATION_FILES[$CURRENT_VERSION]}
- CURRENT_VERSION=$(($CURRENT_VERSION + 1))
-done
-
-# store current version and save startup.json
-mkdir -p $(dirname ${CFG_STARTUP_FILE}) $(dirname ${CFG_VERSION_FILE})
-sysrepocfg -d startup -f json -X > "$CFG_STARTUP_FILE"
-echo "$CURRENT_VERSION" > "$CFG_VERSION_FILE"
-
-# If not do not copy here from startup -> running, running might be stale.
-sysrepocfg -C startup
diff --git a/package/czechlight-cfg-fs/czechlight-migration-list.sh b/package/czechlight-cfg-fs/czechlight-migration-list.sh
deleted file mode 100644
index fcaf04f..0000000
--- a/package/czechlight-cfg-fs/czechlight-migration-list.sh
+++ /dev/null
@@ -1,10 +0,0 @@
-MIGRATION_FILES=(
- '0001_initial-data.sh'
- '0002_default_startup_configuration_for_ietf-interfaces.sh'
- '0003_shelve_alarms.sh'
- '0004_nacm.sh'
- '0005_nacm_anonymous_user.sh'
- '0006_nacm_authentication_rpcs.sh'
- '0007_nacm_anonymous_user_monitoring.sh'
- '0008_bidi_amp_nacm.sh'
-)
diff --git a/package/czechlight-cfg-fs/meld.jq b/package/czechlight-cfg-fs/meld.jq
new file mode 100644
index 0000000..be30657
--- /dev/null
+++ b/package/czechlight-cfg-fs/meld.jq
@@ -0,0 +1,14 @@
+# https://stackoverflow.com/a/53666584/2245623
+# Recursively meld a and b, concatenating arrays and favoring b when there is a conflict
+def meld(a; b):
+ a as $a | b as $b
+ | if ($a|type) == "object" and ($b|type) == "object"
+ then reduce ([$a,$b]|add|keys_unsorted[]) as $k ({};
+ .[$k] = meld( $a[$k]; $b[$k]) )
+ elif ($a|type) == "array" and ($b|type) == "array"
+ then $a+$b
+ elif $b == null then $a
+ else $b
+ end;
+
+reduce inputs as $i (.; meld(.; $i))
diff --git a/package/czechlight-cfg-fs/migrations/0001_initial-data.sh b/package/czechlight-cfg-fs/migrations/0001_initial-data.sh
deleted file mode 100644
index 8b81ccb..0000000
--- a/package/czechlight-cfg-fs/migrations/0001_initial-data.sh
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/bin/bash
-
-# Load initial data
-# -----------------
-# These data are imported into the sysrepo's startup datastore only once. This happens either when the box is restored to
-# its factory settings (the box is new and boots for the first time or someone deletes the startup.json backup in /cfg)
-# or when the box is upgraded from the state before the migrations were introduced (versions released before July 2022).
-#
-# It's OK for user to remove these settings from sysrepo startup DS.
-# However, the data will NEVER get restored by us (unless somebody deletes /cfg/startup.json, see above).
-
-case "${CZECHLIGHT}" in
- sdn-roadm-line*)
- sysrepocfg --datastore=startup --format=json --module=czechlight-roadm-device --import="${CLA_YANG}/sdn-roadm-line.json"
- ;;
- sdn-roadm-add-drop*)
- ;& # fallthrough
- sdn-roadm-hires-add-drop*)
- sysrepocfg --datastore=startup --format=json --module=czechlight-roadm-device --import="${CLA_YANG}/sdn-roadm-add-drop.json"
- ;;
- sdn-roadm-coherent-a-d*)
- sysrepocfg --datastore=startup --format=json --module=czechlight-coherent-add-drop --import="${CLA_YANG}/sdn-roadm-coherent-a-d.json"
- ;;
- sdn-inline*)
- sysrepocfg --datastore=startup --format=json --module=czechlight-inline-amp --import="${CLA_YANG}/sdn-inline.json"
- ;;
- calibration-box)
- sysrepocfg --datastore=startup --format=json --module=czechlight-calibration-device --import="${CLA_YANG}/calibration-box.json"
- ;;
-esac
diff --git a/package/czechlight-cfg-fs/migrations/0002_default_startup_configuration_for_ietf-interfaces.sh b/package/czechlight-cfg-fs/migrations/0002_default_startup_configuration_for_ietf-interfaces.sh
deleted file mode 100644
index 341bb8c..0000000
--- a/package/czechlight-cfg-fs/migrations/0002_default_startup_configuration_for_ietf-interfaces.sh
+++ /dev/null
@@ -1,23 +0,0 @@
-#!/bin/bash
-
-# setup default configuration for ietf-interfaces in startup DS
-# This overwrites current networking setup
-
-set -x
-set -e
-
-case "${CZECHLIGHT}" in
- sdn-roadm-add-drop*)
- ;&
- sdn-roadm-hires-add-drop*)
- ;&
- sdn-roadm-coherent-a-d*)
- sysrepocfg --datastore=startup --format=json --module=ietf-interfaces --import="${MIGRATIONS_DIRECTORY}/0002_ietf-interfaces_default-startup-config_add-drop.json"
- ;;
- sdn-inline*)
- sysrepocfg --datastore=startup --format=json --module=ietf-interfaces --import="${MIGRATIONS_DIRECTORY}/0002_ietf-interfaces_default-startup-config_sdn-inline.json"
- ;;
- sdn-roadm-line*)
- sysrepocfg --datastore=startup --format=json --module=ietf-interfaces --import="${MIGRATIONS_DIRECTORY}/0002_ietf-interfaces_default-startup-config.json"
- ;;
-esac
diff --git a/package/czechlight-cfg-fs/migrations/0003_shelve_alarms.sh b/package/czechlight-cfg-fs/migrations/0003_shelve_alarms.sh
deleted file mode 100644
index c679ebb..0000000
--- a/package/czechlight-cfg-fs/migrations/0003_shelve_alarms.sh
+++ /dev/null
@@ -1,9 +0,0 @@
-#!/bin/bash
-
-# Ignore failures from systemd-journal-upload.service
-# ---------------------------------------------------
-# After migration to ietf-alarms based health state reporting we should
-# keep the current settings, i.e., ignore alarms coming from this particular
-# systemd-journal-upload.service
-#
-sysrepocfg --datastore=startup --format=json --module=ietf-alarms --import="${MIGRATIONS_DIRECTORY}/0003_ietf-alarms_shelve-journal-upload.json"
diff --git a/package/czechlight-cfg-fs/migrations/0004_nacm.json b/package/czechlight-cfg-fs/migrations/0004_nacm.json
deleted file mode 100644
index 59d413b..0000000
--- a/package/czechlight-cfg-fs/migrations/0004_nacm.json
+++ /dev/null
@@ -1,32 +0,0 @@
-{
- "ietf-netconf-acm:nacm": {
- "rule-list": [
- {
- "name": "Allow DWDM control to the optics group",
- "group": ["optics"],
- "rule": [
- {
- "name": "czechlight-roadm-device",
- "module-name": "czechlight-roadm-device",
- "action": "permit"
- },
- {
- "name": "czechlight-inline-amp",
- "module-name": "czechlight-inline-amp",
- "action": "permit"
- },
- {
- "name": "czechlight-coherent-add-drop",
- "module-name": "czechlight-coherent-add-drop",
- "action": "permit"
- },
- {
- "name": "czechlight-calibration-device",
- "module-name": "czechlight-calibration-device",
- "action": "permit"
- }
- ]
- }
- ]
- }
-}
diff --git a/package/czechlight-cfg-fs/migrations/0004_nacm.sh b/package/czechlight-cfg-fs/migrations/0004_nacm.sh
deleted file mode 100644
index 3d7a899..0000000
--- a/package/czechlight-cfg-fs/migrations/0004_nacm.sh
+++ /dev/null
@@ -1,8 +0,0 @@
-#!/bin/bash
-
-# Import CzechLight-specific NACM rules for DWDM modules
-# ------------------------------------------------------
-# Before this we restored these NACM rules from our "factory-default" on every boot, overwriting whatever was in the ietf-netconf-acm module.
-# Since config v4, the users are free to modify NACM rules as they wish.
-
-sysrepocfg -d startup -m ietf-netconf-acm -f json --import="${MIGRATIONS_DIRECTORY}/0004_nacm.json"
diff --git a/package/czechlight-cfg-fs/migrations/0005_nacm_anonymous_user.sh b/package/czechlight-cfg-fs/migrations/0005_nacm_anonymous_user.sh
deleted file mode 100644
index 0e45c05..0000000
--- a/package/czechlight-cfg-fs/migrations/0005_nacm_anonymous_user.sh
+++ /dev/null
@@ -1,7 +0,0 @@
-#!/bin/bash
-
-# Introduce rules for NACM anonymous access user
-# ----------------------------------------------
-# Adds rules for the anonymous user access to the front of the ietf-netconf-acm:nacm/rule-list.
-
-sysrepocfg --datastore=startup --format=json --module=ietf-netconf-acm --edit="${MIGRATIONS_DIRECTORY}/0005_nacm_anonymous_user.json"
diff --git a/package/czechlight-cfg-fs/migrations/0006_nacm_authentication_rpcs.sh b/package/czechlight-cfg-fs/migrations/0006_nacm_authentication_rpcs.sh
deleted file mode 100644
index 869aa53..0000000
--- a/package/czechlight-cfg-fs/migrations/0006_nacm_authentication_rpcs.sh
+++ /dev/null
@@ -1,8 +0,0 @@
-#!/bin/bash
-
-# Enable users change their own pubkeys and password
-# --------------------------------------------------
-# All users should be able to change their own pubkeys or password.
-# We have actions/RPCs for that under /czechlight-system:authentication/users: change-password, add-authorized-key, and authorized-keys/remove
-
-sysrepocfg --datastore=startup --format=json --module=ietf-netconf-acm --edit="${VELIA_YANG}/czechlight-authentication.json"
diff --git a/package/czechlight-cfg-fs/migrations/0007_nacm_anonymous_user_monitoring.json b/package/czechlight-cfg-fs/migrations/0007_nacm_anonymous_user_monitoring.json
deleted file mode 100644
index 7d507ca..0000000
--- a/package/czechlight-cfg-fs/migrations/0007_nacm_anonymous_user_monitoring.json
+++ /dev/null
@@ -1,24 +0,0 @@
-{
- "ietf-netconf-acm:nacm": {
- "rule-list": [
- {
- "name": "Permit yangnobody user/group to read only some modules",
- "group": [
- "yangnobody"
- ],
- "rule": [
- {
- "@": {
- "yang:insert": "before",
- "yang:key": "[name='wildcard-deny']"
- },
- "name": "ietf-restconf-monitoring",
- "module-name": "ietf-restconf-monitoring",
- "action": "permit",
- "access-operations": "read"
- }
- ]
- }
- ]
- }
-}
diff --git a/package/czechlight-cfg-fs/migrations/0007_nacm_anonymous_user_monitoring.sh b/package/czechlight-cfg-fs/migrations/0007_nacm_anonymous_user_monitoring.sh
deleted file mode 100644
index 99606f9..0000000
--- a/package/czechlight-cfg-fs/migrations/0007_nacm_anonymous_user_monitoring.sh
+++ /dev/null
@@ -1,17 +0,0 @@
-#!/bin/bash
-
-# Allow NACM anonymous access user to access ietf-restconf-monitoring module
-# --------------------------------------------------------------------------
-# Adds rules for the anonymous user access before wildcard deny rule. This runs
-# after "0004_nacm" and "0005_nacm_anonymous_user", so if the rule-list itself
-# does not exist, we assume that it's a deliberate configuration, and nothing
-# gets created. If the rule-list exists, but the deny-all rule does not exist,
-# then ietf-restconf-monitoring is simply added as the very last rule.
-
-if RULE=$(sysrepocfg -d startup -G "/ietf-netconf-acm:nacm/rule-list[name='Permit yangnobody user/group to read only some modules']/name") && [ -n "$RULE" ]; then
- if RULE=$(sysrepocfg -d startup -G "/ietf-netconf-acm:nacm/rule-list[name='Permit yangnobody user/group to read only some modules']/rule[name='wildcard-deny']/name") && [ -n "$RULE" ]; then
- sysrepocfg --datastore=startup --format=json --module=ietf-netconf-acm --edit="${MIGRATIONS_DIRECTORY}/0007_nacm_anonymous_user_monitoring.json"
- else
- sysrepocfg --datastore=startup --format=json --module=ietf-netconf-acm --edit="${MIGRATIONS_DIRECTORY}/0007_nacm_anonymous_user_monitoring_append.json"
- fi
-fi
diff --git a/package/czechlight-cfg-fs/migrations/0007_nacm_anonymous_user_monitoring_append.json b/package/czechlight-cfg-fs/migrations/0007_nacm_anonymous_user_monitoring_append.json
deleted file mode 100644
index 2daa6b0..0000000
--- a/package/czechlight-cfg-fs/migrations/0007_nacm_anonymous_user_monitoring_append.json
+++ /dev/null
@@ -1,23 +0,0 @@
-{
- "ietf-netconf-acm:nacm": {
- "rule-list": [
- {
- "name": "Permit yangnobody user/group to read only some modules",
- "group": [
- "yangnobody"
- ],
- "rule": [
- {
- "@": {
- "yang:insert": "last"
- },
- "name": "ietf-restconf-monitoring",
- "module-name": "ietf-restconf-monitoring",
- "action": "permit",
- "access-operations": "read"
- }
- ]
- }
- ]
- }
-}
diff --git a/package/czechlight-cfg-fs/migrations/0008_bidi_amp_nacm.sh b/package/czechlight-cfg-fs/migrations/0008_bidi_amp_nacm.sh
deleted file mode 100644
index b136ea4..0000000
--- a/package/czechlight-cfg-fs/migrations/0008_bidi_amp_nacm.sh
+++ /dev/null
@@ -1,23 +0,0 @@
-#!/bin/bash
-set -ex
-
-# Configure access to the czechlight-bidi-amp module
-# --------------------------------------------------
-#
-# The rules are added right after those for the inline amplifier. Since this script only runs after "0004_nacm"
-# and "0005_nacm_anonymous_user", we can assume that if the outer rule-list is not present at all, that must be
-# a deliberate configuration. In that case, we probably should not add any rules. Similarly, if there's no rule
-# for the "czechlight-inline-amp", let's take a guess and assume that the operator does not want to allow access
-# to amplifiers -- again, it can only happen due to an explicit configuration.
-#
-# The first rule is for authenticated users, default group "dwdm".
-if RULE=$(sysrepocfg -d startup -G "/ietf-netconf-acm:nacm/rule-list[name='Allow DWDM control to the optics group']/rule[name='czechlight-inline-amp']/name") \
- && [ "$RULE" == "czechlight-inline-amp" ]; then
- sysrepocfg --datastore=startup --format=json --module=ietf-netconf-acm --edit="${MIGRATIONS_DIRECTORY}/0008_bidi_amp_nacm_optics.json"
-fi
-
-# The second rule allows anonymous read-only access via RESTCONF.
-if RULE=$(sysrepocfg -d startup -G "/ietf-netconf-acm:nacm/rule-list[name='Permit yangnobody user/group to read only some modules']/rule[name='czechlight-inline-amp']/name") \
- && [ "$RULE" == "czechlight-inline-amp" ]; then
- sysrepocfg --datastore=startup --format=json --module=ietf-netconf-acm --edit="${MIGRATIONS_DIRECTORY}/0008_bidi_amp_nacm_anonymous_user.json"
-fi
diff --git a/package/czechlight-cfg-fs/migrations/0008_bidi_amp_nacm_anonymous_user.json b/package/czechlight-cfg-fs/migrations/0008_bidi_amp_nacm_anonymous_user.json
deleted file mode 100644
index d72d359..0000000
--- a/package/czechlight-cfg-fs/migrations/0008_bidi_amp_nacm_anonymous_user.json
+++ /dev/null
@@ -1,24 +0,0 @@
-{
- "ietf-netconf-acm:nacm": {
- "rule-list": [
- {
- "name": "Permit yangnobody user/group to read only some modules",
- "group": [
- "yangnobody"
- ],
- "rule": [
- {
- "@": {
- "yang:insert": "after",
- "yang:key": "[name='czechlight-inline-amp']"
- },
- "name": "czechlight-bidi-amp",
- "module-name": "czechlight-bidi-amp",
- "action": "permit",
- "access-operations": "read"
- }
- ]
- }
- ]
- }
-}
diff --git a/package/czechlight-cfg-fs/migrations/0008_bidi_amp_nacm_optics.json b/package/czechlight-cfg-fs/migrations/0008_bidi_amp_nacm_optics.json
deleted file mode 100644
index 9be0846..0000000
--- a/package/czechlight-cfg-fs/migrations/0008_bidi_amp_nacm_optics.json
+++ /dev/null
@@ -1,23 +0,0 @@
-{
- "ietf-netconf-acm:nacm": {
- "rule-list": [
- {
- "name": "Allow DWDM control to the optics group",
- "group": [
- "optics"
- ],
- "rule": [
- {
- "@": {
- "yang:insert": "after",
- "yang:key": "[name='czechlight-inline-amp']"
- },
- "name": "czechlight-bidi-amp",
- "module-name": "czechlight-bidi-amp",
- "action": "permit"
- }
- ]
- }
- ]
- }
-}
diff --git a/package/czechlight-cfg-fs/run-sysrepo.mount b/package/czechlight-cfg-fs/run-sysrepo.mount
new file mode 100644
index 0000000..d0a9b9c
--- /dev/null
+++ b/package/czechlight-cfg-fs/run-sysrepo.mount
@@ -0,0 +1,10 @@
+[Unit]
+Description=Prepare a fake /dev/shm for sysrepo
+
+[Mount]
+What=tmpfs
+Where=/run/sysrepo
+Type=tmpfs
+DirectoryMode=0000
+LazyUnmount=yes
+ForceUnmount=yes
diff --git a/package/czechlight-cfg-fs/migrations/0003_ietf-alarms_shelve-journal-upload.json b/package/czechlight-cfg-fs/static-data/alarms-shelve-journal-upload.json
similarity index 100%
rename from package/czechlight-cfg-fs/migrations/0003_ietf-alarms_shelve-journal-upload.json
rename to package/czechlight-cfg-fs/static-data/alarms-shelve-journal-upload.json
diff --git a/package/czechlight-cfg-fs/migrations/0002_ietf-interfaces_default-startup-config_sdn-inline.json b/package/czechlight-cfg-fs/static-data/ietf-interfaces-inline-amp.json
similarity index 100%
rename from package/czechlight-cfg-fs/migrations/0002_ietf-interfaces_default-startup-config_sdn-inline.json
rename to package/czechlight-cfg-fs/static-data/ietf-interfaces-inline-amp.json
diff --git a/package/czechlight-cfg-fs/migrations/0002_ietf-interfaces_default-startup-config_add-drop.json b/package/czechlight-cfg-fs/static-data/ietf-interfaces-roadm-add-drop.json
similarity index 100%
rename from package/czechlight-cfg-fs/migrations/0002_ietf-interfaces_default-startup-config_add-drop.json
rename to package/czechlight-cfg-fs/static-data/ietf-interfaces-roadm-add-drop.json
diff --git a/package/czechlight-cfg-fs/migrations/0002_ietf-interfaces_default-startup-config.json b/package/czechlight-cfg-fs/static-data/ietf-interfaces-roadm-line.json
similarity index 100%
rename from package/czechlight-cfg-fs/migrations/0002_ietf-interfaces_default-startup-config.json
rename to package/czechlight-cfg-fs/static-data/ietf-interfaces-roadm-line.json
diff --git a/package/czechlight-cfg-fs/migrations/0005_nacm_anonymous_user.json b/package/czechlight-cfg-fs/static-data/nacm.json
similarity index 68%
rename from package/czechlight-cfg-fs/migrations/0005_nacm_anonymous_user.json
rename to package/czechlight-cfg-fs/static-data/nacm.json
index b370ba7..689a5cd 100644
--- a/package/czechlight-cfg-fs/migrations/0005_nacm_anonymous_user.json
+++ b/package/czechlight-cfg-fs/static-data/nacm.json
@@ -2,47 +2,20 @@
"ietf-netconf-acm:nacm": {
"rule-list": [
{
- "@": {
- "yang:insert": "first"
- },
"name": "Permit yangnobody user/group to read only some modules",
"group": [
"yangnobody"
],
"rule": [
{
- "name": "czechlight-roadm-device",
- "module-name": "czechlight-roadm-device",
- "action": "permit",
- "access-operations": "read"
- },
- {
- "name": "czechlight-inline-amp",
- "module-name": "czechlight-inline-amp",
- "action": "permit",
- "access-operations": "read"
- },
- {
- "name": "czechlight-coherent-add-drop",
- "module-name": "czechlight-coherent-add-drop",
- "action": "permit",
- "access-operations": "read"
- },
- {
"name": "ietf-yang-library",
"module-name": "ietf-yang-library",
"action": "permit",
"access-operations": "read"
},
{
- "name": "ietf-hardware",
- "module-name": "ietf-hardware",
- "action": "permit",
- "access-operations": "read"
- },
- {
- "name": "ietf-interfaces",
- "module-name": "ietf-interfaces",
+ "name": "ietf-restconf-monitoring",
+ "module-name": "ietf-restconf-monitoring",
"action": "permit",
"access-operations": "read"
},
@@ -82,6 +55,18 @@
"access-operations": "read"
},
{
+ "name": "ietf-hardware",
+ "module-name": "ietf-hardware",
+ "action": "permit",
+ "access-operations": "read"
+ },
+ {
+ "name": "ietf-interfaces",
+ "module-name": "ietf-interfaces",
+ "action": "permit",
+ "access-operations": "read"
+ },
+ {
"name": "czechlight-lldp",
"module-name": "czechlight-lldp",
"action": "permit",
@@ -102,12 +87,75 @@
"access-operations": "read"
},
{
+ "name": "czechlight-roadm-device",
+ "module-name": "czechlight-roadm-device",
+ "action": "permit",
+ "access-operations": "read"
+ },
+ {
+ "name": "czechlight-inline-amp",
+ "module-name": "czechlight-inline-amp",
+ "action": "permit",
+ "access-operations": "read"
+ },
+ {
+ "name": "czechlight-bidi-amp",
+ "module-name": "czechlight-bidi-amp",
+ "action": "permit",
+ "access-operations": "read"
+ },
+ {
+ "name": "czechlight-coherent-add-drop",
+ "module-name": "czechlight-coherent-add-drop",
+ "action": "permit",
+ "access-operations": "read"
+ },
+ {
+ "name": "czechlight-calibration-device",
+ "module-name": "czechlight-calibration-device",
+ "action": "permit",
+ "access-operations": "read"
+ },
+ {
"name": "wildcard-deny",
"module-name": "*",
"action": "deny",
"access-operations": "*"
}
]
+ },
+ {
+ "name": "Allow DWDM control to the optics group",
+ "group": [
+ "optics"
+ ],
+ "rule": [
+ {
+ "name": "czechlight-roadm-device",
+ "module-name": "czechlight-roadm-device",
+ "action": "permit"
+ },
+ {
+ "name": "czechlight-inline-amp",
+ "module-name": "czechlight-inline-amp",
+ "action": "permit"
+ },
+ {
+ "name": "czechlight-bidi-amp",
+ "module-name": "czechlight-bidi-amp",
+ "action": "permit"
+ },
+ {
+ "name": "czechlight-coherent-add-drop",
+ "module-name": "czechlight-coherent-add-drop",
+ "action": "permit"
+ },
+ {
+ "name": "czechlight-calibration-device",
+ "module-name": "czechlight-calibration-device",
+ "action": "permit"
+ }
+ ]
}
]
}
diff --git a/package/czechlight-cfg-fs/static-data/netopeer2.json.in b/package/czechlight-cfg-fs/static-data/netopeer2.json.in
new file mode 100644
index 0000000..59cf52d
--- /dev/null
+++ b/package/czechlight-cfg-fs/static-data/netopeer2.json.in
@@ -0,0 +1,42 @@
+{
+ "ietf-keystore:keystore": {
+ "asymmetric-keys": {
+ "asymmetric-key": [
+ {
+ "name": "genkey",
+ "public-key-format": "ietf-crypto-types:ssh-public-key-format",
+ "private-key-format": "ietf-crypto-types:rsa-private-key-format",
+ "cleartext-private-key": CLEARTEXT_PRIVATE_KEY
+ }
+ ]
+ }
+ },
+ "ietf-netconf-server:netconf-server": {
+ "listen": {
+ "endpoints": {
+ "endpoint": [
+ {
+ "name": "default-ssh",
+ "ssh": {
+ "tcp-server-parameters": {
+ "local-address": "::"
+ },
+ "ssh-server-parameters": {
+ "server-identity": {
+ "host-key": [
+ {
+ "name": "default-key",
+ "public-key": {
+ "central-keystore-reference": "genkey"
+ }
+ }
+ ]
+ }
+ }
+ }
+ }
+ ]
+ }
+ }
+ }
+}
diff --git a/package/czechlight-cfg-fs/sysrepo-persistent-cfg.service b/package/czechlight-cfg-fs/sysrepo-persistent-cfg.service
index b8f5788..4c9418f 100644
--- a/package/czechlight-cfg-fs/sysrepo-persistent-cfg.service
+++ b/package/czechlight-cfg-fs/sysrepo-persistent-cfg.service
@@ -1,7 +1,7 @@
[Unit]
Description=Persisting persistent sysrepo datastores to /cfg
-After=netopeer2.service cfg.mount czechlight-migrate.service
-Requires=netopeer2.service cfg.mount czechlight-migrate.service
+After=cfg-yang.service
+Requires=cfg-yang.service
[Service]
Type=simple
diff --git a/package/czechlight-cfg-fs/yang/czechlight-netconf-server@2024-09-04.yang b/package/czechlight-cfg-fs/yang/czechlight-netconf-server@2024-09-04.yang
new file mode 100644
index 0000000..bdb8dce
--- /dev/null
+++ b/package/czechlight-cfg-fs/yang/czechlight-netconf-server@2024-09-04.yang
@@ -0,0 +1,34 @@
+module czechlight-netconf-server {
+ yang-version 1.1;
+ namespace "http://czechlight.cesnet.cz/yang/czechlight-netconf-server";
+ prefix "czechlight-netconf-server";
+
+ import ietf-netconf-server {
+ prefix ncs;
+ revision-date '2023-12-28';
+ }
+
+ organization "CESNET";
+ contact "photonic@cesnet.cz";
+ description "Failsafes for NETCONF server configuration";
+
+ revision 2024-09-04 {
+ description "Initial release";
+ }
+
+ deviation /ncs:netconf-server {
+ deviate add {
+ must "count(listen) = 1" {
+ error-message "The NETCONF server must be activated for listening at some endpoint";
+ }
+ }
+ }
+
+ deviation /ncs:netconf-server/ncs:listen/ncs:endpoints {
+ deviate add {
+ must "count(endpoint/ssh) >= 1" {
+ error-message "The NETCONF server must enable at least one SSH endpoint";
+ }
+ }
+ }
+}