use OpenWrt 21.02.7

this was leading to the OOM killer killing dd.

.gitignore

@@ -1,2 +1,2 @@
 notes.txt
-authorized_keys
+build/

README.md

@@ -1,16 +1,19 @@
 # What
 
-It's a script to build a customized [LEDE](https://lede-project.org/)
+It's a script to build a customized
-firmware image using a Linux x86_64 host (basic familiarity with
+[OpenWRT](https://openwrt.org/docs/guide-user/start)
-[LEDE](https://lede-project.org/) is assumed). LEDE is a fork of
+firmware image using
-[OpenWRT](https://openwrt.org/).
+[ImageBuilder](https://openwrt.org/docs/guide-user/additional-software/imagebuilder).
 
-If the generated image is flashed on a device it will try to automatically
+If the generated image is flashed on a device it will try to
-set up [extroot](http://wiki.openwrt.org/doc/howto/extroot) on **any
+automatically set up
-(!)** storage device plugged into the USB port (`/dev/sda`). Keep in
+[extroot](https://openwrt.org/docs/guide-user/additional-software/extroot_configuration)
-mind that **this will erase any inserted storage device while the
+on **any (!)** storage device plugged into the USB port (`/dev/sda`),
-router is in the initial setup phase**! Unfortunately there's little
+including your working extroot pendrive if you plug it in only later
-that can be done at that point to ask the user for confirmation.
+in the boot process. Keep in mind that **this will erase any inserted
+storage device while the router is in the initial setup phase**!
+Unfortunately there's little that can be done at that point to ask the
+user for confirmation.
 
 # Why
 
@@ -18,17 +21,37 @@ So that e.g. customers can buy a router on their own, flash our custom
 firmware, plug in a pendrive, and manage their SIP (telephony) node
 from our webapp.
 
+I've extracted the generic parts from the above mentioned auto-provision
+project because I thought it's useful enough for making it public.
+
 # How
+
+You can read more about the underlying technology on the OpenWRT wiki: see e.g. the
+[ImageBuilder](https://openwrt.org/docs/guide-user/additional-software/imagebuilder)
+page, or the page that lists some other
+[ImageBuilder frontends](https://openwrt.org/docs/guide-developer/imagebuilder_frontends).
+
+As for the actual mechanism: custom scripts are baked into the boot
+process of the flashed firmware. If the extroot overlay is properly
+set up, then these scripts get hidden by it; i.e. they will only run
+when the extroot has failed to mount early in the boot process.
+
 ### Building
 
-To build issue the following command: `./build.sh architecture variant device-profile`, e.g.:
+OpenWRT's ImageBuilder only works on Linux x86_64. To build a firmware, issue the following command:
-* `./build.sh ar71xx generic tl-wdr4300-v1`
+`./build.sh architecture variant device-profile`, e.g.:
 
-Results will be under `build/lede-imagebuilder-${release}-${architecture}-${variant}.Linux-x86_64/bin/`.
+* `./build.sh ath79 generic tplink_tl-wr1043nd-v1`
+* `./build.sh ath79 generic tplink_archer-c6-v2`
+* `./build.sh ath79 generic tplink_tl-wdr4300-v1`
+* `./build.sh bcm53xx generic dlink_dir-885l`
+
+Results will be under `build/openwrt-imagebuilder-${release}-${architecture}-${variant}.Linux-x86_64/bin/`.
 
 To see a list of available targets, run `make info` in the ImageBuilder dir.
 
-If you want to change which LEDE version is used, then edit the relevant variable(s) in `build.sh`.
+If you want to change which OpenWRT version is used, then edit the relevant variable(s)
+in `build.sh`.
 
 ### Setup stages
 
@@ -37,16 +60,21 @@ sources for details: [autoprovision-functions.sh](image-extras/common/root/autop
 
 #### Stage 1: setup extroot
 
-At the first boot after flashing the firmware the autoprovision script will
+When the custom firmware first boots, the autoprovision script will
 wait for anything (!) in `/dev/sda` to show up (that is >= 512M), then erase
 it and set up a `swap`, an `extroot`, and a `data`filesystem (for the remaining
 space), and then reboot.
 
 #### Stage 2: download and install some packages from the internet
 
-Once it booted into the new extroot, it will continuously attempt to install
+Once it rebooted into the new extroot, it will continuously keep trying to install
-some LEDE packages until an internet connection is set up on the router
+some OpenWRT packages until an internet connection is set up on the router. You
-(either by using ssh or LuCI if you could fit it into the firmware).
+need to do that manually either by using ssh or the web UI (LuCI).
+
+#### Stage 3, optional
+
+We also have a 3rd stage, written in Python, but it's commented out here.
+Search for `autoprovision-stage3.py` to see how it's done.
 
 ### Login
 
@@ -57,55 +85,51 @@ By default the root passwd is not set, so the router will start telnet with
 no password. If you want to set up a password, then edit the stage 2 script:
 [autoprovision-stage2.sh](image-extras/common/root/autoprovision-stage2.sh#L53).
 
-If a password is set, then telnet is disabled by LEDE and SSH will listen
+If a password is set, then telnet is disabled by OpenWRT and SSH will listen
 using the keys specified in [authorized_keys](image-extras/common/etc/dropbear/authorized_keys).
 
 Once connected, you can read the log with `logread -f`.
 
 # Status
 
-This is more of a template than something standalone. You most
+This is more of a template than something standalone, but I use it for
+my home routers as is. You most
 probably want to customize this script here and there; search for
 `CUSTOMIZE` for places of interest.
 
 Most importantly, **set up a password and maybe an ssh key**.
 
-I've extracted this from a project of mine where OpenWRT nodes auto-provision
+At the time of writing it only supports a few `ath79` routers out of
-themselves in 3 stages (stage 3 was a Python script for an app-level sync feature),
+the box, but it's easy to extend it. Support for a new router entails
-but I thought it's useful enough for making it public.
+looking up some led names for `setLedAttribute` so that there's
-
+feedback to the user through the blinking of the leds. It should work
-At the time of writing it only supports a few `ar71xx` routers out of the box,
+fine without that, but it will be less convenient to interact with
-but it's easy to extend it.
+your router in the initial setup phase.
-
-## Tested with
-
-[LEDE 17.01.1](https://downloads.lede-project.org/releases/17.01.1/)
-on a TP-Link WDR4300.
 
 # Troubleshooting
 
 ## Which file should I flash?
 
-You should consult the documentation at [LEDE](https://lede-project.org/docs/start) and/or at
+You should consult the [OpenWRT documentation](https://openwrt.org/docs/guide-user/start).
-[OpenWRT](https://wiki.openwrt.org/doc/howto/user.beginner).
+The produced firmware files should be somewhere around
-The produced firmware files should be somewhere around ```build/lede-imagebuilder-17.01.1-ar71xx-generic.Linux-x86_64/bin/ar71xx```.
+```./build/openwrt-imagebuilder-21.02.0-ath79-generic.Linux-x86_64/bin/targets/ath79/generic/```.
 
 In short:
 
 * You need a file with the name ```-factory.bin``` or ```-sysupgrade.bin```. The former is to
-  be used when you first install LEDE, the latter is when you upgrade an already installed
+  be used when you first install OpenWRT, the latter is when you upgrade an already installed
-  LEDE.
+  OpenWRT.
 
 * You must carefully pick the proper firmware file for your **hardware version**! I advise you
-  to look up the wiki page for your hardware on the [OpenWRT wiki](https://wiki.openwrt.org),
+  to look up the wiki page for your hardware on the [OpenWRT wiki](https://openwrt.org),
-  because most of them have a table of the released hardawre versions with comments on their
+  because most of them have a table of the released hardware versions with comments on their
-  status (sometimes new hardware revisions are only supported by the latest LEDE, which is
+  status (sometimes new hardware revisions are only supported by the latest OpenWRT, which is
   not released yet).
 
 ## Help! The build has finished but there's no firmware file!
 
 If the build doesn't yield a firmware file (```*-factory.bin``` and/or ```*-sysupgrade.bin```):
 when there's not enough space in the flash memory of the target device to install everything
-then the LEDE ImageBuilder prints a hardly visible error into its flow of output and
+then the OpenWRT ImageBuilder prints a hardly visible error into its flow of output and
 silently continues. Look into [build.sh](build.sh#L31) and try to remove some packages
 that you can live without.

build.sh

@@ -1,54 +1,44 @@
-#!/bin/bash
+#!/bin/sh
 
 set -e
 
-absolutize ()
-{
-  if [ ! -d "$1" ]; then
-    echo
-    echo "ERROR: '$1' doesn't exist or not a directory!"
-    kill -INT $$
-  fi
-
-  pushd "$1" >/dev/null
-  echo `pwd`
-  popd >/dev/null
-}
-
 TARGET_ARCHITECTURE=$1
 TARGET_VARIANT=$2
 TARGET_DEVICE=$3
 
 BUILD=`dirname "$0"`"/build/"
-BUILD=`absolutize $BUILD`
+BUILD=`readlink -f $BUILD`
 
 ###
 ### chose a release
 ###
-#RELEASE="15.05.1"
+RELEASE="21.02.7"
-RELEASE="17.01.1"
 
-IMGBUILDER_NAME="lede-imagebuilder-${RELEASE}-${TARGET_ARCHITECTURE}-${TARGET_VARIANT}.Linux-x86_64"
+IMGBUILDER_NAME="openwrt-imagebuilder-${RELEASE}-${TARGET_ARCHITECTURE}-${TARGET_VARIANT}.Linux-x86_64"
 IMGBUILDER_DIR="${BUILD}/${IMGBUILDER_NAME}"
 IMGBUILDER_ARCHIVE="${IMGBUILDER_NAME}.tar.xz"
 
-IMGTEMPDIR="${BUILD}/openwrt-build-image-extras"
+IMGTEMPDIR="${BUILD}/image-extras"
-#https://downloads.lede-project.org/snapshots/targets/ar71xx/generic/lede-imagebuilder-ar71xx-generic.Linux-x86_64.tar.xz
+# see this feature request:
-#https://downloads.lede-project.org/snapshots/targets/ar71xx/generic/lede-imagebuilder-ar71xx-generic.Linux-x86_64.tar.xz
+# FS#1670 - consistent naming convention for the imagebuilder.tar.xz URL
-IMGBUILDERURL="https://downloads.lede-project.org/releases/${RELEASE}/targets/${TARGET_ARCHITECTURE}/${TARGET_VARIANT}/${IMGBUILDER_ARCHIVE}"
+# https://bugs.openwrt.org/index.php?do=details&task_id=1670
+IMGBUILDERURL="https://downloads.openwrt.org/releases/${RELEASE}/targets/${TARGET_ARCHITECTURE}/${TARGET_VARIANT}/${IMGBUILDER_ARCHIVE}"
 
 if [ -z ${TARGET_DEVICE} ]; then
     echo "Usage: $0 architecture variant device-profile"
-    echo " e.g.: $0 ar71xx generic tl-wr1043nd-v2"
+    echo " e.g.: $0 ath79 generic tplink_tl-wr1043nd-v1"
-    echo "       $0 ramips mt7621 zbt-wg3526"
+    echo "       $0 ath79 generic tplink_archer-c6-v2"
+    echo "       $0 ath79 generic tplink_tl-wdr4300-v1"
+    echo "       $0 bcm53xx generic dlink_dir-885l"
+    echo "       (this last one will not work without editing build.sh, details: https://github.com/attila-lendvai/openwrt-auto-extroot/pull/15#issuecomment-405847440)"
     echo " to get a list of supported devices issue a 'make info' in the OpenWRT image builder directory:"
     echo "   '${IMGBUILDER_DIR}'"
     kill -INT $$
 fi
 
 # the absolute minimum for extroot to work at all (i.e. when the disk is already set up, for example by hand).
-# this list may be smaller and/or different for your router, but it works with my ar71xx.
+# this list may be smaller and/or different for your router, but it works with my ath79.
-PREINSTALLED_PACKAGES="block-mount kmod-usb2 kmod-usb-storage kmod-fs-ext4"
+PREINSTALLED_PACKAGES="block-mount kmod-fs-ext4 kmod-usb-storage"
 
 # some kernel modules may also be needed for your hardware
 #PREINSTALLED_PACKAGES+=" kmod-usb-uhci kmod-usb-ohci"
@@ -74,7 +64,7 @@ fi
 if [ ! -e ${IMGBUILDER_DIR} ]; then
     pushd ${BUILD}
     # --no-check-certificate if needed
-    wget  --continue ${IMGBUILDERURL}
+    wget --continue ${IMGBUILDERURL}
     xz -d <${IMGBUILDER_ARCHIVE} | tar vx
     popd
 fi
@@ -83,8 +73,8 @@ pushd ${IMGBUILDER_DIR}
 
 make image PROFILE=${TARGET_DEVICE} PACKAGES="${PREINSTALLED_PACKAGES}" FILES=${IMGTEMPDIR}
 
-pushd bin/${TARGET_ARCHITECTURE}/
+pushd bin/targets/${TARGET_ARCHITECTURE}/
-ln -s ../../packages .
+ln -s ../../../packages .
 popd
 
 popd

build/.gitignore

@@ -1,4 +0,0 @@
-# Ignore everything in this directory
-*
-# Except this file
-!.gitignore

default.nix

@@ -0,0 +1,19 @@
+{ pkgs ? import <nixpkgs> {} }:
+
+pkgs.mkShell {
+  buildInputs = with pkgs; [
+    coreutils posix_man_pages bash-completion less
+    gitFull diffutils
+    gnumake which
+    ncurses perl python2 python3
+
+    # keep this line if you use bash
+    bashInteractive
+  ];
+
+  shellHook =
+  ''
+    alias ..='cd ..'
+    alias ...='cd ../..'
+  '';
+}

image-extras/common/etc/dropbear/.gitignore

@@ -0,0 +1,3 @@
+# you can put your ssh public key into authorized_keys,
+# but we don't ever want it to be committed to the repo
+authorized_keys

image-extras/common/etc/dropbear/authorized_keys

@@ -1 +0,0 @@
-# this file may contain ssh public keys for passwordless ssh root login

image-extras/common/root/autoprovision-functions.sh

@@ -10,33 +10,34 @@ rootUUID=05d615b3-bef8-460c-9a23-52db8d09e000
 dataUUID=05d615b3-bef8-460c-9a23-52db8d09e001
 swapUUID=05d615b3-bef8-460c-9a23-52db8d09e002
 
-if [ -f /lib/ar71xx.sh ]; then
+. /lib/functions.sh
-   . /lib/ar71xx.sh
 
-   # let's attempt to define some defaults...
+# let's attempt to define some defaults...
-   autoprovisionUSBLed="tp-link:green:usb"
+autoprovisionUSBLed="green:usb"
-   autoprovisionStatusLed="tp-link:green:qss"
+autoprovisionStatusLed="green:qss"
+
+echo Board name is [$(board_name)]
+
+# CUSTOMIZE
+case $(board_name) in
+    *tl-wr1043nd*)
+        autoprovisionUSBLed="green:usb"
+        autoprovisionStatusLed="green:qss"
+        ;;
+    *tl-mr3020*)
+        autoprovisionUSBLed="green:wps"
+        autoprovisionStatusLed="green:wlan"
+        ;;
+    *tl-wr2543n*)
+        autoprovisionUSBLed="green:wps"
+        autoprovisionStatusLed="green:wlan5g"
+        ;;
+    *tl-wdr4300*)
+        autoprovisionUSBLed="green:wlan2g"
+        autoprovisionStatusLed="green:wlan5g"
+        ;;
+esac
 
-   # CUSTOMIZE
-   case $(ar71xx_board_name) in
-       "tl-wr1043nd")
-           autoprovisionUSBLed="tp-link:green:usb"
-           autoprovisionStatusLed="tp-link:green:qss"
-           ;;
-       "tl-mr3020")
-           autoprovisionUSBLed="tp-link:green:wps"
-           autoprovisionStatusLed="tp-link:green:wlan"
-           ;;
-       "tl-wr2543n")
-           autoprovisionUSBLed="tp-link:green:wps"
-           autoprovisionStatusLed="tp-link:green:wlan5g"
-           ;;
-       "tl-wdr4300")
-           autoprovisionUSBLed="tp-link:blue:wan"
-           autoprovisionStatusLed="tp-link:blue:qss"
-           ;;
-   esac
-fi
 
 log()
 {

image-extras/common/root/autoprovision-stage1.sh

@@ -34,7 +34,7 @@ hasBigEnoughPendrive()
 setupPendrivePartitions()
 {
     # erase partition table
-    dd if=/dev/zero of=/dev/sda bs=1M count=1
+    dd if=/dev/zero of=/dev/sda bs=1k count=256
 
     # sda1 is 'swap'
     # sda2 is 'root'
@@ -73,8 +73,8 @@ EOF
     done
 
     mkswap -L swap -U $swapUUID /dev/sda1
-    mkfs.ext4 -L root -U $rootUUID /dev/sda2
+    mkfs.ext4 -F -L root -U $rootUUID /dev/sda2
-    mkfs.ext4 -L data -U $dataUUID /dev/sda3
+    mkfs.ext4 -F -L data -U $dataUUID /dev/sda3
 
     log "Finished setting up filesystems"
 }

image-extras/common/root/autoprovision-stage2.sh

@@ -9,7 +9,7 @@ installPackages()
     signalAutoprovisionWaitingForUser
 
     until (opkg update)
-    do
+     do
         log "opkg update failed. No internet connection? Retrying in 15 seconds..."
         sleep 15
     done
@@ -50,6 +50,9 @@ autoprovisionStage2()
     else
         signalAutoprovisionWorking
 
+	echo Updating system time using ntp; otherwise the openwrt.org certificates are rejected as not yet valid.
+        ntpd -d -q -n -p 0.openwrt.pool.ntp.org
+
         # CUSTOMIZE: with an empty argument it will set a random password and only ssh key based login will work.
         # please note that stage2 requires internet connection to install packages and you most probably want to log in
         # on the GUI to set up a WAN connection. but on the other hand you don't want to end up using a publically