Setting HypriotOS on a Raspberry PI 4 PicoCluster

A few weeks ago I ordered a PicoCluster for an upcoming learning project and blogpost series, to my surprise the entire setup wasn't straightforward. The bundled application set couldn't initially see each other node and neither connect to my Wi-Fi network, and after fiddling with the config I gave up on the default image. Having WiFi and Ethernet is important in the setup because I want download and run docker images, and each node can see each other through LAN.
Some weeks before buying the cluster I took an Akka Training at Lunatech where we saw how a similar cluster was used to show Akka Cluster, and having heard of HypriotOS and how easily it could run docker containers I decided to give a shot.

Flashing

The first step involves flashing the SD cards with Hypriot image, this guys provide a flash tool with convenient configuration parameters. For this step I'm using release 2.4.0, I'd to flash a card several times until I got the WLAN config correctly, maybe something to do with the version.

flash --userdata cloud-init.yaml \
      https://github.com/hypriot/image-builder-rpi/releases/download/v1.11.0/hypriotos-rpi-v1.11.0.img.zip

Where cloud-init.yaml is:

#cloud-config

# Set your hostname here, the manage_etc_hosts will update the hosts file entries as well
hostname: pc0
manage_etc_hosts: true

# You could modify this for your own user information
users:
  - name: pirate
    gecos: "Hypriot Pirate"
    sudo: ALL=(ALL) NOPASSWD:ALL
    shell: /bin/bash
    groups: users,docker,video
    plain_text_passwd: hypriot
    lock_passwd: false
    ssh_pwauth: true
    chpasswd: { expire: false }

package_upgrade: false

# # WiFi connect to HotSpot
# # - use `wpa_passphrase SSID PASSWORD` to encrypt the psk
write_files:
  - content: |
      allow-hotplug wlan0
      iface wlan0 inet dhcp
      wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
      iface default inet dhcp
    path: /etc/network/interfaces.d/wlan0
  - content: |
      country=de
      ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
      update_config=1
      network={
      ssid="<<Your WiFi SSID>>"
      psk="<<Your WiFi Password>>"
      proto=RSN
      key_mgmt=WPA-PSK
      pairwise=CCMP
      auth_alg=OPEN
      }
    path: /etc/wpa_supplicant/wpa_supplicant.conf


# These commands will be ran once on first boot only
runcmd:
  # Pickup the hostname changes
  - 'systemctl restart avahi-daemon'

  # Activate WiFi interface
  - 'ifup wlan0'

Notice on line 4, hostname is set to pc0, this should be difference for every node. Following the original PicoCluster setup, nodes are numbered from pc0 to pc4. After flashing the cards I could boot each of them and proceed to the network configuration.

Networking

After reading a few articles on Linux networking, and to be honest my networking knowledge was a bit rusty, I found that PicoCluster is setting static IPs by modifying the /etc/network/interfaces file directly, instead of the /etc/dhcpcd.conf as recommended in this StackOverflow reply. So I ssh-ed ¹ on each node to modify the configuration as follows:

# Append to /etc/dhcpcd.conf
interface eth0
static ip_address=10.1.10.240/24
static routers=192.168.1.1
static domain_name_servers=62.58.48.30 62.58.153.220 192.168.1.1

inteface wlan0
metric 200

Here we set the static IP for the first node (aka. pc0), as seen in the line 3. The routers and DNS correspond to my route configuration. Notice LAN and WLAN will work on different networks. Finally the last section is to make the OS prefer WLAN over LAN, honestly I think it could work without this, but I got an issue trying to pull a docker image from the Hub and this fixed it.
After setting this on each node I wanted to modify /etc/hosts so each node could know each other by using the PicoCluster hostname. We can't modify the aforementioned file directly since we specified manage_etc_hosts, for which we've to modify /etc/cloud/templates/hosts.tmpl with the following values:

10.1.10.240 pc0
10.1.10.241 pc1
10.1.10.242 pc2
10.1.10.243 pc3
10.1.10.244 pc4

I modified each node accordingly skipping localhost for the list.

Key-based authentication

As final step I wanted easy ssh login into each container, PicoCluster comes already with that setup, but I needed to replicate that here as well. Luckily someone else did before me here, so I'm just going to repeat the same steps (although credits goes to him).
SSH keys have to generated on the node you want to login from. For my case this meant pc0, although this could have been any node, but since this is the one that has HDMI connection it make sense to do it at least here.

# Enable SSH agent if not present
eval `ssh-agent -s`

# Create a new SSH key
# For this setup we can leave the passphrase empty, up to you
ssh-keygen -t rsa -b 4096 -o -a 100

# After creation we add the key to the agent
ssh-add ~/.ssh/id_rsa

# Finally we copy the public key into the other nodes
ssh-copy-id -i ~/.ssh/id_rsa.pub pirate@pc1
ssh-copy-id -i ~/.ssh/id_rsa.pub pirate@pc2
ssh-copy-id -i ~/.ssh/id_rsa.pub pirate@pc3
ssh-copy-id -i ~/.ssh/id_rsa.pub pirate@pc4

After this I was able to connect to each node from my computer and from node 0.

Appendix: Finding your Raspberry PI IPs

You will need to know which are your RPIs' IP addresses, for this you can login into your router's admin page and find them there. Another approach is using nmap:

nmap -sn 192.168.1.0/24

Provided that your DHCP server is giving addresses on that network, you can find which network are you on by using ifconfig.