Start watchdog pinging only once everything is up

Previously, systemd started pinging the HW watchdog too early. It was
possible to arrive at a system which is more or less FUBAR, yet the WS
was happily being periodically reset by systemd's event loop.

This change ensures that the watchdog resetting only begins once the
rest of the system is stable -- in other words, once "everything else"
is OK.

One cannot simply add another systemd target for this which would
Require=multi-user.target. Failures can occur at any level of the stack,
and essentially all units specify WantedBy, not RequiredBy, so their
respective failures do not propagate.

I was also trying to simply check the result of `systemctl
is-system-running`. That doesn't work because "starting" takes
precedence over `degraded", and we're still "starting" when checking
from within a unit, obviously.

This hack therefore looks at the list of failed units. If it isn't
empty, then we have a problem, and we won't activate neither the
watchdog, nor the RAUC good-marking thingy.

The "running" timeout is now set to 30 seconds. This means that systemd
will ping the HW once every 15s, and that a failure to do so for half a
minute results in an immediate reboot. There's no shutdown timer because
we don't really care about what happens once we started to shutdown. Our
only desire is to reboot, eventually, and this one looks like it can do
it :).

Change-Id: Ib94992814705b4a6d4d712db9e443d6a784cec7f
Fixes: https://tree.taiga.io/project/jktjkt-czechlight/issue/130
5 files changed
tree: d58a34ea9420d63965234ab282672a8641e66e36
  1. .gitmodules
  2. Config.in
  3. README.md
  4. board/
  5. configs/
  6. crypto/
  7. dev-setup-git.sh
  8. external.desc
  9. external.mk
  10. package/
  11. submodules/
README.md

How to use this

This repository contains CzechLight-specific bits for Buildroot. Buildroot is a tool which produces system images for flashing to embedded devices. They have a nice documentation which explains everything that one might need.

Quick Start

Everything is in Gerrit. One should not need to clone anything from anywhere else. The build will download source tarballs of various open source components, though.

TODO: Automate this via the CI system. I want to get the .img files for testing of each change, eventually.

git clone ssh://$YOUR_LOGIN@cesnet.cz@gerrit.cesnet.cz:29418/CzechLight/br2-external czechlight
pushd czechlight
git submodule update --init --recursive
popd
mkdir build-clearfog
cd build-clearfog
../czechlight/dev-setup-git.sh
make czechlight_clearfog_defconfig
make

A full rebuild takes between 30 and 45 minutes on a T460s laptop.

WARNING: Buildroot is fragile. It is not safe to perform incremental builds after changing an "important" setting. Please check their manual for details.

Hack: parallel build

A significant amount of time is wasted in configure steps which are not parallelized :( as of November 2017. This can be hacked by patching Buildroot's top-level Makefile, but note that one cannot easily debug stuff afterwards.

diff --git a/Makefile b/Makefile
index 79db7fe..905099a 100644
--- a/Makefile
+++ b/Makefile
@@ -114,7 +114,7 @@ endif
 # this top-level Makefile in parallel comment the ".NOTPARALLEL" line and
 # use the -j<jobs> option when building, e.g:
 #      make -j$((`getconf _NPROCESSORS_ONLN`+1))
-.NOTPARALLEL:
 
 # absolute path
 TOPDIR := $(CURDIR)

Installing

Updates via RAUC

Apart from the traditional way of re-flashing the SD card or the eMMC from scratch, it's also possible to use RAUC to update. This method preserves the U-Boot version and the U-Boot's environment. Apart from that, everything starting with the kernel and the DTB file and including the root FS is updated. Configuration stored in /cfg is brought along and preserved as well.

To install an update:

# build node
make
rsync -avP images/update.raucb somewhere.example.org:path/to/web/root

# target, perhaps via an USB console
wget http://somewhere.example.org/update.raucb -O /tmp/update.raucb
rauc install /tmp/update.raucb
reboot

Initial installation

Clearfog

On development boards with a µSD card slot, simply dd the images/sdcard.img to the SD card and boot from there.

On a regular Clearfog Base with an eMMC, one has to bootstrap the device first. If recovering a totally bricked board, one can use the kwboot command to upload the initial U-Boot via the console:

./tools/kwboot -b ./u-boot-spl.kwb -t -p /dev/ttyUSB0

Once in U-Boot (a stock factory image is OK as well), plug a USB flash disk which contains images/usb-flash.img and execute:

usb start; fatload usb 0:1 00800000 boot.scr; source 00800000

A Linux session will start. Run the following from the shell prompt:

mount /dev/sda1 /mnt; sh /mnt/usb-reflash-factory.sh