Update readme for LEDE

add note that it's linux x86_64 only

.gitignore

@@ -1 +1,2 @@
 notes.txt
+authorized_keys

README.md

@@ -1,56 +1,111 @@
 # What
 
-It's a script to build a customized OpenWRT firmware that will
-automatically set up
-[extroot](http://wiki.openwrt.org/doc/howto/extroot) on any (!)
-storage device plugged into the USB port (`/dev/sda`).
+It's a script to build a customized [LEDE](https://lede-project.org/)
+firmware image using a Linux x86_64 host (basic familiarity with
+[LEDE](https://lede-project.org/) is assumed). LEDE is a fork of
+[OpenWRT](https://openwrt.org/).
+
+If the generated image is flashed on a device it will try to automatically
+set up [extroot](http://wiki.openwrt.org/doc/howto/extroot) on **any
+(!)** storage device plugged into the USB port (`/dev/sda`). Keep in
+mind that **this will erase any inserted storage device while the
+router is in the initial setup phase**! Unfortunately there's little
+that can be done at that point to ask the user for confirmation.
 
 # Why
 
-So that e.g. customers can buy a router on their own, flash our
+So that e.g. customers can buy a router on their own, flash our custom
 firmware, plug in a pendrive, and manage their SIP (telephony) node
 from our webapp.
 
+# How
+### Building
+
+To build issue the following command: `./build.sh architecture variant device-profile`, e.g.:
+* `./build.sh ar71xx generic tl-wdr4300-v1`
+
+Results will be under `build/lede-imagebuilder-${release}-${architecture}-${variant}.Linux-x86_64/bin/`.
+
+To see a list of available targets, run `make info` in the ImageBuilder dir.
+
+If you want to change which LEDE version is used, then edit the relevant variable(s) in `build.sh`.
+
+### Setup stages
+
+Blinking leds show which phase the extroot setup scripts are in. Consult the
+sources for details: [autoprovision-functions.sh](image-extras/common/root/autoprovision-functions.sh#L49).
+
+#### Stage 1: setup extroot
+
+At the first boot after flashing the firmware the autoprovision script will
+wait for anything (!) in `/dev/sda` to show up (that is >= 512M), then erase
+it and set up a `swap`, an `extroot`, and a `data`filesystem (for the remaining
+space), and then reboot.
+
+#### Stage 2: download and install some packages from the internet
+
+Once it booted into the new extroot, it will continuously attempt to install
+some LEDE packages until an internet connection is set up on the router
+(either by using ssh or LuCI if you could fit it into the firmware).
+
+### Login
+
+After flashing the firmware the router will have the standard
+`192.168.1.1` IP address.
+
+By default the root passwd is not set, so the router will start telnet with
+no password. If you want to set up a password, then edit the stage 2 script:
+[autoprovision-stage2.sh](image-extras/common/root/autoprovision-stage2.sh#L53).
+
+If a password is set, then telnet is disabled by LEDE and SSH will listen
+using the keys specified in [authorized_keys](image-extras/common/etc/dropbear/authorized_keys).
+
+Once connected, you can read the log with `logread -f`.
+
 # Status
 
 This is more of a template than something standalone. You most
 probably want to customize this script here and there; search for
 `CUSTOMIZE` for places of interest.
 
-I've extracted this from a project where OpenWRT nodes auto-provision
-themselves in 3 stages, but I thought it's useful enough for making it
-public (stage 1: extroot setup; stage 2: install packages; stage 3: a
-Python script for app-level sync).
+Most importantly, **set up a password and maybe an ssh key**.
 
-At the time of writing it only supports a few `ar71xx` routers but
-it's easy to extend it.
+I've extracted this from a project of mine where OpenWRT nodes auto-provision
+themselves in 3 stages (stage 3 was a Python script for an app-level sync feature),
+but I thought it's useful enough for making it public.
+
+At the time of writing it only supports a few `ar71xx` routers out of the box,
+but it's easy to extend it.
 
 ## Tested with
 
-[OpenWRT Barrier Breaker 14.07](http://downloads.openwrt.org/barrier_breaker/14.07/)
+[LEDE 17.01.1](https://downloads.lede-project.org/releases/17.01.1/)
 on a TP-Link WDR4300.
 
-# Building
+# Troubleshooting
 
-e.g. `./build.sh TLWDR4300`
+## Which file should I flash?
 
-Results will be under `build/OpenWrt-ImageBuilder-ar71xx_generic-for-linux-x86_64`.
+You should consult the documentation at [LEDE](https://lede-project.org/docs/start) and/or at
+[OpenWRT](https://wiki.openwrt.org/doc/howto/user.beginner).
+The produced firmware files should be somewhere around ```build/lede-imagebuilder-17.01.1-ar71xx-generic.Linux-x86_64/bin/ar71xx```.
 
-To see a list of available targets, run this in the ImageBuilder dir: ```make info```.
+In short:
 
-# Usage
+* You need a file with the name ```-factory.bin``` or ```-sysupgrade.bin```. The former is to
+  be used when you first install LEDE, the latter is when you upgrade an already installed
+  LEDE.
 
-After flashing the firmware the router will have the standard
-`192.168.1.1` IP address, and SSH will listen there using the keys
-specified in `image-extras/etc/dropbear/authorized_keys`.
+* You must carefully pick the proper firmware file for your **hardware version**! I advise you
+  to look up the wiki page for your hardware on the [OpenWRT wiki](https://wiki.openwrt.org),
+  because most of them have a table of the released hardawre versions with comments on their
+  status (sometimes new hardware revisions are only supported by the latest LEDE, which is
+  not released yet).
 
-Once connected, you can read the log with `logread -f`.
+## Help! The build has finished but there's no firmware file!
 
-The autoprovision script will wait for any `/dev/sda` to show up, then
-erase it and set up a `swap`, an `extroot`, and a `data` filesystem,
-and then reboots.
-
-In stage 2 it will need an internet connection, so you should connect
-to its [LuCI interface](http://192.168.1.1) to set up an Internet
-upstream, and then it will automatically continue installing packages,
-finishing the whole process, and then do a final reboot.
+If the build doesn't yield a firmware file (```*-factory.bin``` and/or ```*-sysupgrade.bin```):
+when there's not enough space in the flash memory of the target device to install everything
+then the LEDE ImageBuilder prints a hardly visible error into its flow of output and
+silently continues. Look into [build.sh](build.sh#L31) and try to remove some packages
+that you can live without.

build.sh

@@ -15,44 +15,75 @@ absolutize ()
   popd >/dev/null
 }
 
-TARGET_PLATFORM=$1
-
-if [ -z ${TARGET_PLATFORM} ]; then
-    echo "Usage: $0 target-platform (e.g. 'TLWDR4300')"
-    kill -INT $$
-fi
+TARGET_ARCHITECTURE=$1
+TARGET_VARIANT=$2
+TARGET_DEVICE=$3
 
 BUILD=`dirname "$0"`"/build/"
 BUILD=`absolutize $BUILD`
+
+###
+### chose a release
+###
+#RELEASE="15.05.1"
+RELEASE="17.01.1"
+
+IMGBUILDER_NAME="lede-imagebuilder-${RELEASE}-${TARGET_ARCHITECTURE}-${TARGET_VARIANT}.Linux-x86_64"
+IMGBUILDER_DIR="${BUILD}/${IMGBUILDER_NAME}"
+IMGBUILDER_ARCHIVE="${IMGBUILDER_NAME}.tar.xz"
+
 IMGTEMPDIR="${BUILD}/openwrt-build-image-extras"
-IMGBUILDERDIR="${BUILD}/OpenWrt-ImageBuilder-ar71xx_generic-for-linux-x86_64"
-IMGBUILDERURL="https://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/OpenWrt-ImageBuilder-ar71xx_generic-for-linux-x86_64.tar.bz2"
+#https://downloads.lede-project.org/snapshots/targets/ar71xx/generic/lede-imagebuilder-ar71xx-generic.Linux-x86_64.tar.xz
+#https://downloads.lede-project.org/snapshots/targets/ar71xx/generic/lede-imagebuilder-ar71xx-generic.Linux-x86_64.tar.xz
+IMGBUILDERURL="https://downloads.lede-project.org/releases/${RELEASE}/targets/${TARGET_ARCHITECTURE}/${TARGET_VARIANT}/${IMGBUILDER_ARCHIVE}"
 
-PREINSTALLED_PACKAGES="wireless-tools firewall iptables"
-PREINSTALLED_PACKAGES+=" ppp ppp-mod-pppoe ppp-mod-pppol2tp ppp-mod-pptp kmod-ppp kmod-pppoe"
-PREINSTALLED_PACKAGES+=" fdisk blkid swap-utils mount-utils block-mount e2fsprogs kmod-fs-ext4 kmod-usb2 kmod-usb-uhci kmod-usb-ohci kmod-usb-storage kmod-usb-storage-extras kmod-mmc"
-PREINSTALLED_PACKAGES+=" luci"
-
-mkdir --parents ${BUILD}
-
-rm -rf $IMGTEMPDIR
-cp -r image-extras $IMGTEMPDIR
-if [ -e image-extras.$TARGET_PLATFORM/ ]; then
-    rsync -pr image-extras.$TARGET_PLATFORM/ $IMGTEMPDIR/
+if [ -z ${TARGET_DEVICE} ]; then
+    echo "Usage: $0 architecture variant device-profile"
+    echo " e.g.: $0 ar71xx generic tl-wr1043nd-v2"
+    echo "       $0 ramips mt7621 zbt-wg3526"
+    echo " to get a list of supported devices issue a 'make info' in the OpenWRT image builder directory:"
+    echo "   '${IMGBUILDER_DIR}'"
+    kill -INT $$
 fi
 
-if [ ! -e ${IMGBUILDERDIR} ]; then
+# the absolute minimum for extroot to work at all (i.e. when the disk is already set up, for example by hand).
+# this list may be smaller and/or different for your router, but it works with my ar71xx.
+PREINSTALLED_PACKAGES="block-mount kmod-usb2 kmod-usb-storage kmod-fs-ext4"
+
+# some kernel modules may also be needed for your hardware
+#PREINSTALLED_PACKAGES+=" kmod-usb-uhci kmod-usb-ohci"
+
+# these are needed for the proper functioning of the auto extroot scripts
+PREINSTALLED_PACKAGES+=" blkid mount-utils swap-utils e2fsprogs fdisk"
+
+# the following packages are optional, feel free to (un)comment them
+PREINSTALLED_PACKAGES+=" wireless-tools firewall iptables"
+PREINSTALLED_PACKAGES+=" kmod-usb-storage-extras kmod-mmc"
+PREINSTALLED_PACKAGES+=" ppp ppp-mod-pppoe ppp-mod-pppol2tp ppp-mod-pptp kmod-ppp kmod-pppoe"
+PREINSTALLED_PACKAGES+=" luci"
+
+mkdir -pv ${BUILD}
+
+rm -rf $IMGTEMPDIR
+cp -r image-extras/common/ $IMGTEMPDIR
+PER_PLATFORM_IMAGE_EXTRAS=image-extras/${TARGET_DEVICE}/
+if [ -e $PER_PLATFORM_IMAGE_EXTRAS ]; then
+    rsync -pr $PER_PLATFORM_IMAGE_EXTRAS $IMGTEMPDIR/
+fi
+
+if [ ! -e ${IMGBUILDER_DIR} ]; then
     pushd ${BUILD}
-    wget --continue ${IMGBUILDERURL}
-    tar jvxf OpenWrt-ImageBuilder*.tar.bz2
+    # --no-check-certificate if needed
+    wget  --continue ${IMGBUILDERURL}
+    xz -d <${IMGBUILDER_ARCHIVE} | tar vx
     popd
 fi
 
-pushd ${IMGBUILDERDIR}
+pushd ${IMGBUILDER_DIR}
 
-make image PROFILE=${TARGET_PLATFORM} PACKAGES="${PREINSTALLED_PACKAGES}" FILES=${IMGTEMPDIR}
+make image PROFILE=${TARGET_DEVICE} PACKAGES="${PREINSTALLED_PACKAGES}" FILES=${IMGTEMPDIR}
 
-pushd bin/ar71xx/
+pushd bin/${TARGET_ARCHITECTURE}/
 ln -s ../../packages .
 popd
 

image-extras.TLMR3020/etc/config/network

@@ -1,11 +0,0 @@
-
-config interface 'loopback'
-	option ifname 'lo'
-	option proto 'static'
-	option ipaddr '127.0.0.1'
-	option netmask '255.0.0.0'
-
-config interface 'lan'
-	option ifname 'eth0'
-	option type 'bridge'
-	option proto 'dhcp'

image-extras/TLMR3020/etc/config/placeholder

@@ -0,0 +1 @@
+# this is just a file to show how platform specific image extras can be specified

image-extras/common/etc/dropbear/authorized_keys

@@ -0,0 +1 @@
+# this file may contain ssh public keys for passwordless ssh root login

image-extras/common/root/autoprovision-functions.sh

@@ -10,31 +10,33 @@ rootUUID=05d615b3-bef8-460c-9a23-52db8d09e000
 dataUUID=05d615b3-bef8-460c-9a23-52db8d09e001
 swapUUID=05d615b3-bef8-460c-9a23-52db8d09e002
 
-. /lib/ar71xx.sh
+if [ -f /lib/ar71xx.sh ]; then
+   . /lib/ar71xx.sh
 
-# let's try some defaults...
-autoprovisionUSBLed="tp-link:green:usb"
-autoprovisionStatusLed="tp-link:green:qss"
+   # let's attempt to define some defaults...
+   autoprovisionUSBLed="tp-link:green:usb"
+   autoprovisionStatusLed="tp-link:green:qss"
 
-# CUSTOMIZE
-case $(ar71xx_board_name) in
-"tl-wr1043nd")
-        autoprovisionUSBLed="tp-link:green:usb"
-        autoprovisionStatusLed="tp-link:green:qss"
-	;;
-"tl-mr3020")
-        autoprovisionUSBLed="tp-link:green:wps"
-        autoprovisionStatusLed="tp-link:green:wlan"
-	;;
-"tl-wr2543n")
-        autoprovisionUSBLed="tp-link:green:wps"
-        autoprovisionStatusLed="tp-link:green:wlan5g"
-	;;
-"tl-wdr4300")
-        autoprovisionUSBLed="tp-link:blue:wan"
-        autoprovisionStatusLed="tp-link:blue:qss"
-	;;
-esac
+   # CUSTOMIZE
+   case $(ar71xx_board_name) in
+       "tl-wr1043nd")
+           autoprovisionUSBLed="tp-link:green:usb"
+           autoprovisionStatusLed="tp-link:green:qss"
+           ;;
+       "tl-mr3020")
+           autoprovisionUSBLed="tp-link:green:wps"
+           autoprovisionStatusLed="tp-link:green:wlan"
+           ;;
+       "tl-wr2543n")
+           autoprovisionUSBLed="tp-link:green:wps"
+           autoprovisionStatusLed="tp-link:green:wlan5g"
+           ;;
+       "tl-wdr4300")
+           autoprovisionUSBLed="tp-link:blue:wan"
+           autoprovisionStatusLed="tp-link:blue:qss"
+           ;;
+   esac
+fi
 
 log()
 {

image-extras/common/root/autoprovision-stage1.sh

@@ -10,7 +10,7 @@ getPendriveSize()
     # details: https://dev.openwrt.org/ticket/10716#comment:4
     if [ -e /dev/sda ]; then
         # force re-read of the partition table
-        head /dev/sda >/dev/null
+        head -c 1024 /dev/sda >/dev/null
     fi
 
     if (grep -q sda /proc/partitions) then
@@ -81,30 +81,29 @@ EOF
 
 setupExtroot()
 {
-    mkdir -p /mnt/extroot
-    # TODO they said on the wiki that it's optional, an empty overlay also works...
-    # we need to make the internal overlay read-only, otherwise the two md5's may be different
-    # due to writing to the internal overlay from this point until the reboot.
-    # files: /.extroot.md5sum (extroot) and /etc/extroot.md5sum (internal)
-    #mount -o remount,ro /
-    #log "Remounted / as read-only"
-
+    mkdir -p /mnt/extroot/
     mount -U $rootUUID /mnt/extroot
-    #tar -C /overlay -cvf - . | tar -C /mnt/extroot -xf -
 
-    # let's write a new rc.local on extroot which will shadow the one which is in the rom and runs stage1
-    mkdir -p /mnt/extroot/etc/
-    cat >/mnt/extroot/etc/rc.local <<EOF
+    overlay_root=/mnt/extroot/upper
+
+    # at this point we could copy the entire root (a previous version of this script did that), or just the overlay from the flash,
+    # but it seems to work fine if we just create an empty overlay that is only replacing the rc.local from the firmware.
+
+    # let's write a new rc.local on the extroot that will shadow the one which is in the rom (to run stage2 instead of stage1)
+    mkdir -p ${overlay_root}/etc/
+    cat >${overlay_root}/etc/rc.local <<EOF
 /root/autoprovision-stage2.sh
 exit 0
 EOF
 
-    # make sure that we shadow the /var -> /tmp symlink with the extroot, so that /var is permanent
-    mkdir -p /mnt/extroot/var
-    # KLUDGE: but /var/state is assumed to be transient, see https://dev.openwrt.org/ticket/12228
-    cd /mnt/extroot/var
-    ln -s /tmp state
-    cd -
+    # TODO FIXME when this below is enabled then Chaos Calmer doesn't turn on the network and the device remains unreachable
+
+    # make sure that we shadow the /var -> /tmp symlink in the new extroot, so that /var becomes persistent across reboots.
+#    mkdir -p ${overlay_root}/var
+    # KLUDGE: /var/state is assumed to be transient, so link it to tmp, see https://dev.openwrt.org/ticket/12228
+#    cd ${overlay_root}/var
+#    ln -s /tmp state
+#    cd -
 
     log "Finished setting up extroot"
 }

image-extras/common/root/autoprovision-stage2.sh

@@ -54,7 +54,7 @@ autoprovisionStage2()
         # please note that stage2 requires internet connection to install packages and you most probably want to log in
         # on the GUI to set up a WAN connection. but on the other hand you don't want to end up using a publically
         # available default password anywhere, therefore the random here...
-        setRootPassword ""
+        #setRootPassword ""
 
         installPackages
 

image-extras/etc/dropbear/authorized_keys

@@ -1 +0,0 @@
-ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAu9Nwb8tr91hvChHSjEbyS3P0c1+jKtKAdaFCRkyUjVwgCpuzDxiq0auuNulYIfD2oc+THJ6zymJUjWNrVipeUo8BmKkDSMgN0Qf5PlwcSiIj9vDbLqxmVnnvB6xGEROO215Y8XzMOgq8r3Z3WqRUZIeFDHC2sSwJKO3INgsLZd6IoDiM7Dza8pKzYPfY7jJ19JmK4S8lHG3YsoxTy2zkcwCI20sBekJU0iDGvOOJq5UbIumKsAm2uJkMKsKlxkDQr0Y+2J1l0iWBrUHonja6CieO5yNBWluA3DCqxa0pQW3dcOju3mGCQl0j8+3Iblu8lCGoQVSLQ3rUhekmz+cB2Q== alendvai laptop ssh key