apertis-update-manager-ota-diskfull manual

medium

Image Types:
minimal-armhf
Image Deployment:
OSTree
Type:
functional

Description

Ensure that failures due to disk full errors during network updates have no effect on the system


Resources

  • The DUT u-boot environment must be clean: in u-boot, run: `env default -a` followed by `saveenv`
  • A PC must be connected to DUT serial port
  • The DUT must be connected to network

Pre Conditions

  1. This test requires a properly configured time source: when testing devices not carrying a battery-backed real time clock in a network which prevents connections to NTP servers (and only in that case) manually ensure that the time on the device is set appropriately and that it is propagated to the hardware clock so it is stay set on the next reboot if the power is not plugged off.
  2. $ sudo timedatectl --adjust-system-clock set-time "2019-08-21 18:49:03"

Execution Steps

  1. Check the initial deployment
  2. $ sudo ostree admin status
  3. Prepare the copy of commit and deploy to allow the upgrade to the same version
  4. Command below shows you an initial commit ID, for instance
  5. $ export BOOTID=$(sudo ostree admin status | sed -n -e 's/^\* apertis \([0-9a-f]*\)\.[0-9]$/\1/p'); echo $BOOTID
  6. Get the Collection ID and ref
  7. $ export CID=$(sudo ostree refs -c | head -n 1 | tr -d '(),' | cut -f 1 -d ' '); echo COLLECTION_ID=$CID
    $ export REF=$(sudo ostree refs -c | head -n 1 | tr -d '(),' | cut -f 2 -d ' '); echo REF=$REF
  8. Create the list of files to skip and enshure there are some files in these directories
  9. $ ls -1d /usr/share/locale /usr/share/man /usr/share/zoneinfo > /tmp/skip
    $ du -sh /usr/share/locale /usr/share/man /usr/share/zoneinfo
  10. Create the commit with changed timestamp and skipped list from above to allow upgrade with recent update file
  11. $ export NEWID=$(sudo ostree commit --orphan --tree=ref=$BOOTID --add-metadata-string=ostree.collection-binding=$CID --bind-ref=$REF --timestamp="1 year ago" --skip-list=/tmp/skip); echo "New commit: $NEWID"
  12. Deploy the prepared commit
  13. $ sudo ostree admin upgrade --allow-downgrade --deploy-only --override-commit=$NEWID --reboot
  14. Wait until the system is booted again and check the deployment
  15. $ sudo ostree admin status
  16. The booted commit (started with '*') must have ID which we prepare and the initial commit ID should be marked as '(rollback)'
  17. Check booted deployment have no file objects which we skip
  18. $ du -sh /usr/share/locale /usr/share/man /usr/share/zoneinfo
  19. Remove the initial deployment
  20. $ sudo ostree admin undeploy 1
  21. Reboot the system
  22. Check the current deployment
  23. $ sudo ostree admin status
  24. Need to monitor the journal log
  25. $ sudo journalctl -ef --unit apertis-update-manager &
  26. Check the free space on rootfs, calculate the size of file to leave insufficient free space for upgrade to happen (this prevents the unexpected reboot during the test caused by other services). For instance, in the case below, NUM value would be 1451. It is the result of the calculation 1823-348-24, there 24 is a magic constant, so that almost, but not all free space is used. The free space can be calculated with command line `sudo df -B 1M / | grep / | awk '{print $2-$3-24}'`
  27. $ sudo df -B 1M /

    Filesystem           1M-blocks      Used Available Use% Mounted on

    /dev/mmcblk0p2            1823       348      1365  21% /

    $ sudo dd if=/dev/zero of=/var/bigfile bs=1M count=<NUM> ; echo "dd completed!"
  28. Wait until 'dd' completes (it should take time), you will see the output similar to
  29. 1521483776 bytes (1.5 GB) copied, 82.1661 s, 18.5 MB/s

    dd completed!

  30. Enable network updates with CLI tool
  31. $ sudo updatectl --check-network-updates
  32. The update should fail with error message
  33. ** Message: 20:59:57.249: Network connected: Yes

    ** Message: 20:59:57.251: Auto update status: active

    ** Message: 20:59:57.251: Ostree upgrade poll starting

    ** Message: 20:59:58.697: Ostree upgrade failed: min-free-space-percent '3%' would be exceeded

  34. Set reserved space for ostree to '0'
  35. $ sudo ostree config set core.min-free-space-percent "0"
  36. Restart the Apertis Update Manager
  37. $ sudo systemctl restart apertis-update-manager
  38. Enable network updates with CLI tool
  39. $ sudo updatectl --check-network-updates
  40. The update should fail with some error message related to disk space, for instance:
  41. ** Message: 21:02:31.972: Network connected: Yes

    ** Message: 21:02:31.974: Auto update status: active

    ** Message: 21:02:31.974: Ostree upgrade poll starting

    ** Message: 21:02:33.800: Ostree upgrade progress: 2 metadata, 0 content objects fetched; 284 B transferred in 1 seconds

    ** (process:532): WARNING **: 21:02:33.801: Cannot check the ID in black list: No such file or directory

    ** Message: 21:02:41.054: Ostree upgrade failed: Error writing to file descriptor: No space left on device

  42. Remove temporary file
  43. $ sudo rm -f /var/bigfile
  44. Reboot the system
  45. $ sudo reboot
  46. Check that there is only one deployment
  47. $ sudo ostree admin status

Expected

The update wasn't applied

System boots using the initial deployment