docs: auto-sync wiki from main repository
Updated: 2025-12-09 00:01:08 UTC
Source commit: a22b1a7a56
Triggered by: push
github-actions[bot]
parent
53dd79922d
commit
58ca88f0a1
16 changed files with 4104 additions and 659 deletions
497
Architecture.md
Normal file
497
Architecture.md
Normal file
|
|
@ -0,0 +1,497 @@
|
||||||
|
# Architecture
|
||||||
|
|
||||||
|
Understanding how the Raspberry Pi Image Builder works under the hood.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The build system creates hybrid images by combining:
|
||||||
|
- **Raspberry Pi OS** - Boot partition with firmware, bootloader, and config
|
||||||
|
- **Debian ARM64** - Root filesystem with full userspace
|
||||||
|
- **RaspiOS Packages** - Kernel and firmware installed via APT (with auto-update support)
|
||||||
|
- **Custom Services** - Modular components composed during build
|
||||||
|
|
||||||
|
The result is a Debian system with full Raspberry Pi hardware support and automatic kernel/firmware updates.
|
||||||
|
|
||||||
|
## Design Philosophy
|
||||||
|
|
||||||
|
### Why Hybrid Images?
|
||||||
|
|
||||||
|
**Problem**: Raspberry Pi uses proprietary firmware and the RP1 chip for I/O (Ethernet, USB, GPIO). Standard Debian ARM64 kernels lack these drivers.
|
||||||
|
|
||||||
|
**Solution**: Keep Raspberry Pi OS boot partition and kernel packages, but use Debian for everything else.
|
||||||
|
|
||||||
|
**Benefits**:
|
||||||
|
- Full hardware support (RP1, WiFi, Bluetooth, GPIO)
|
||||||
|
- Automatic kernel/firmware updates via `apt upgrade`
|
||||||
|
- Debian's package ecosystem and stability
|
||||||
|
- No manual kernel compilation or firmware management
|
||||||
|
|
||||||
|
### Why Install Packages in QEMU?
|
||||||
|
|
||||||
|
**Traditional Approach**:
|
||||||
|
- Merge images
|
||||||
|
- Chroot into ARM64 rootfs from x86_64 host
|
||||||
|
- Use qemu-user-static for emulation
|
||||||
|
- Install packages
|
||||||
|
|
||||||
|
**Our Approach**:
|
||||||
|
- Install packages in native ARM64 QEMU VM **before** merge
|
||||||
|
- Merge pre-configured Debian image with RaspiOS boot
|
||||||
|
|
||||||
|
**Advantages**:
|
||||||
|
- **Simpler**: No complex chroot setup
|
||||||
|
- **Faster**: Native ARM64 execution, no user-mode overhead
|
||||||
|
- **Cleaner**: Merge script just copies files, no package management
|
||||||
|
- **Reproducible**: Same environment every build
|
||||||
|
|
||||||
|
## Build Pipeline
|
||||||
|
|
||||||
|
### 4-Stage Process
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ Stage 1: Download & Prepare │
|
||||||
|
├─────────────────────────────────────────────────────────────┤
|
||||||
|
│ • Download RaspiOS Lite image │
|
||||||
|
│ • Download Debian cloud/generic ARM64 image │
|
||||||
|
│ • Parse service configuration │
|
||||||
|
│ • Resolve service dependencies │
|
||||||
|
│ • Combine setup scripts from all services │
|
||||||
|
│ • Create setup.iso (contains setup scripts + files) │
|
||||||
|
│ • Generate cloud-init seed.img OR inject first-boot service │
|
||||||
|
└─────────────────────────────────────────────────────────────┘
|
||||||
|
↓
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ Stage 2: QEMU Setup (ARM64 VM) │
|
||||||
|
├─────────────────────────────────────────────────────────────┤
|
||||||
|
│ • Boot Debian ARM64 in QEMU │
|
||||||
|
│ • Cloud-init or first-boot service creates user │
|
||||||
|
│ • Mount setup.iso │
|
||||||
|
│ • Execute combined setup.sh: │
|
||||||
|
│ - Add RaspiOS APT repository + pinning │
|
||||||
|
│ - Install RaspiOS kernel packages (raspberrypi-kernel) │
|
||||||
|
│ - Install RaspiOS firmware packages │
|
||||||
|
│ - Install service packages (docker, incus, etc.) │
|
||||||
|
│ - Copy configuration files to /etc/setupfiles/ │
|
||||||
|
│ - Copy first-boot scripts │
|
||||||
|
│ • Auto-shutdown when complete │
|
||||||
|
└─────────────────────────────────────────────────────────────┘
|
||||||
|
↓
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ Stage 3: Merge │
|
||||||
|
├─────────────────────────────────────────────────────────────┤
|
||||||
|
│ • Call merge-debian-raspios.sh │
|
||||||
|
│ • Keep RaspiOS boot partition (FAT32): │
|
||||||
|
│ - bootloader, config.txt, cmdline.txt │
|
||||||
|
│ • Backup RaspiOS /etc/fstab │
|
||||||
|
│ • Replace root partition with Debian (ext4): │
|
||||||
|
│ - Delete RaspiOS rootfs │
|
||||||
|
│ - rsync Debian rootfs (with RaspiOS packages installed) │
|
||||||
|
│ - Restore RaspiOS fstab (correct partition UUIDs) │
|
||||||
|
│ - Create /boot/firmware mount point │
|
||||||
|
│ • Resize root partition to fill image │
|
||||||
|
└─────────────────────────────────────────────────────────────┘
|
||||||
|
↓
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ Stage 4: Compress │
|
||||||
|
├─────────────────────────────────────────────────────────────┤
|
||||||
|
│ • Run PiShrink to minimize filesystem │
|
||||||
|
│ • Compress with xz (parallel, level 6) │
|
||||||
|
│ • Generate checksums (SHA256) │
|
||||||
|
│ • Output: image-name.img.xz │
|
||||||
|
└─────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
## Partition Layout
|
||||||
|
|
||||||
|
### Before Merge
|
||||||
|
|
||||||
|
**RaspiOS Image**:
|
||||||
|
```
|
||||||
|
┌────────────────────┬──────────────────────┐
|
||||||
|
│ /dev/loop0p1 │ /dev/loop0p2 │
|
||||||
|
│ boot (FAT32) │ root (ext4) │
|
||||||
|
│ 512MB │ ~2GB │
|
||||||
|
│ ================== │ ==================== │
|
||||||
|
│ • bootloader │ • RaspiOS rootfs │
|
||||||
|
│ • kernel │ • (will be replaced) │
|
||||||
|
│ • firmware │ │
|
||||||
|
│ • config.txt │ │
|
||||||
|
└────────────────────┴──────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
**Debian Image**:
|
||||||
|
```
|
||||||
|
┌────────────────────────────────────────────┐
|
||||||
|
│ /dev/loop1p1 (or p2, auto-detected) │
|
||||||
|
│ root (ext4) │
|
||||||
|
│ ~2-4GB │
|
||||||
|
│ ========================================== │
|
||||||
|
│ • Debian userspace │
|
||||||
|
│ • RaspiOS kernel packages (from QEMU) │
|
||||||
|
│ • Service packages (from QEMU) │
|
||||||
|
│ • /etc/setupfiles/ (configs) │
|
||||||
|
│ • First-boot scripts │
|
||||||
|
└────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### After Merge
|
||||||
|
|
||||||
|
**Hybrid Image**:
|
||||||
|
```
|
||||||
|
┌────────────────────┬──────────────────────────────┐
|
||||||
|
│ /dev/mmcblk0p1 │ /dev/mmcblk0p2 │
|
||||||
|
│ boot (FAT32) │ root (ext4) │
|
||||||
|
│ 512MB │ 8GB (expanded) │
|
||||||
|
│ ================== │ ============================ │
|
||||||
|
│ • RaspiOS bootload │ • Debian userspace │
|
||||||
|
│ • RaspiOS kernel* │ • RaspiOS kernel packages │
|
||||||
|
│ • RaspiOS firmware │ • RaspiOS firmware packages │
|
||||||
|
│ • config.txt │ • RaspiOS fstab │
|
||||||
|
│ │ • /boot/firmware → p1 │
|
||||||
|
└────────────────────┴──────────────────────────────┘
|
||||||
|
(kept from RaspiOS) (replaced with Debian)
|
||||||
|
|
||||||
|
* Kernel files also in /boot/ and /lib/modules/ from APT packages
|
||||||
|
```
|
||||||
|
|
||||||
|
## Modular Service System
|
||||||
|
|
||||||
|
### Service Directory Structure
|
||||||
|
|
||||||
|
Each service is a self-contained module:
|
||||||
|
|
||||||
|
```
|
||||||
|
services/
|
||||||
|
└── <service-name>/
|
||||||
|
├── setup.sh # Runs in QEMU (package installation)
|
||||||
|
├── first-boot/
|
||||||
|
│ └── init.sh # Runs on first boot (runtime config)
|
||||||
|
├── setupfiles/ # Static files → /etc/setupfiles/
|
||||||
|
│ └── config.xyz
|
||||||
|
├── depends.sh # Optional: dependencies
|
||||||
|
└── motd.sh # Optional: MOTD content
|
||||||
|
```
|
||||||
|
|
||||||
|
### Service Lifecycle
|
||||||
|
|
||||||
|
```
|
||||||
|
BUILD TIME (QEMU):
|
||||||
|
┌─────────────────────────────────────────────────┐
|
||||||
|
│ setup.sh │
|
||||||
|
├─────────────────────────────────────────────────┤
|
||||||
|
│ • Install packages (apt install ...) │
|
||||||
|
│ • Configure system settings │
|
||||||
|
│ • Create users/groups │
|
||||||
|
│ • Set up repositories │
|
||||||
|
│ • Copy setupfiles/ to /etc/setupfiles/ │
|
||||||
|
│ • Install first-boot/init.sh │
|
||||||
|
└─────────────────────────────────────────────────┘
|
||||||
|
|
||||||
|
FIRST BOOT (Raspberry Pi):
|
||||||
|
┌─────────────────────────────────────────────────┐
|
||||||
|
│ rpi-first-boot.service (one-time) │
|
||||||
|
├─────────────────────────────────────────────────┤
|
||||||
|
│ • Expand root partition to fill SD card │
|
||||||
|
│ • Set persistent network interface names │
|
||||||
|
│ • Reboot │
|
||||||
|
└─────────────────────────────────────────────────┘
|
||||||
|
↓
|
||||||
|
┌─────────────────────────────────────────────────┐
|
||||||
|
│ services-first-boot.service (one-time) │
|
||||||
|
├─────────────────────────────────────────────────┤
|
||||||
|
│ • Execute base/first-boot/init.sh │
|
||||||
|
│ - Configure network bridges (br-wan, br-lan) │
|
||||||
|
│ - Set up DHCP server (if br-lan) │
|
||||||
|
│ • Execute service/first-boot/init.sh │
|
||||||
|
│ - Download images (HAOS, OpenWrt) │
|
||||||
|
│ - Create containers/VMs │
|
||||||
|
│ - Detect and configure hardware │
|
||||||
|
│ - Start services │
|
||||||
|
│ • Disable itself │
|
||||||
|
└─────────────────────────────────────────────────┘
|
||||||
|
|
||||||
|
RUNTIME:
|
||||||
|
┌─────────────────────────────────────────────────┐
|
||||||
|
│ Services running │
|
||||||
|
├─────────────────────────────────────────────────┤
|
||||||
|
│ • Docker containers │
|
||||||
|
│ • Incus VMs/containers │
|
||||||
|
│ • WiFi hotspot │
|
||||||
|
│ • OpenWrt router │
|
||||||
|
│ • etc. │
|
||||||
|
└─────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### Service Dependency Resolution
|
||||||
|
|
||||||
|
Services can declare dependencies:
|
||||||
|
|
||||||
|
**Example**: `services/haos/depends.sh`
|
||||||
|
```bash
|
||||||
|
DEPENDS_ON="qemu"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Build process**:
|
||||||
|
1. Parse requested services: `qemu+docker+haos`
|
||||||
|
2. Resolve dependencies:
|
||||||
|
- `qemu` (no dependencies)
|
||||||
|
- `docker` (no dependencies)
|
||||||
|
- `haos` → requires `qemu`
|
||||||
|
3. Build order: `base` → `qemu` → `docker` → `haos`
|
||||||
|
4. Combine setup.sh from all services
|
||||||
|
|
||||||
|
### Service Composition
|
||||||
|
|
||||||
|
**Dynamic image example**: `debian/qemu+docker+haos`
|
||||||
|
|
||||||
|
**Process**:
|
||||||
|
1. Create temporary directory: `images/debian-qemu-docker-haos/`
|
||||||
|
2. Copy base config: `images/debian/config.sh`
|
||||||
|
3. Override variables:
|
||||||
|
```bash
|
||||||
|
OUTPUT_IMAGE="debian-qemu-docker-haos.img"
|
||||||
|
SERVICES="base qemu docker haos"
|
||||||
|
```
|
||||||
|
4. Combine setup scripts:
|
||||||
|
```bash
|
||||||
|
cat services/base/setup.sh \
|
||||||
|
services/qemu/setup.sh \
|
||||||
|
services/docker/setup.sh \
|
||||||
|
services/haos/setup.sh \
|
||||||
|
> setup.sh
|
||||||
|
```
|
||||||
|
5. Merge setupfiles:
|
||||||
|
```bash
|
||||||
|
cp -r services/base/setupfiles/* setupfiles/
|
||||||
|
cp -r services/qemu/setupfiles/* setupfiles/
|
||||||
|
cp -r services/docker/setupfiles/* setupfiles/
|
||||||
|
cp -r services/haos/setupfiles/* setupfiles/
|
||||||
|
```
|
||||||
|
6. Inject first-boot scripts:
|
||||||
|
```bash
|
||||||
|
# Aggregate all init.sh into services-first-boot.sh
|
||||||
|
```
|
||||||
|
7. Build image
|
||||||
|
|
||||||
|
## APT Repository Management
|
||||||
|
|
||||||
|
### Repository Configuration
|
||||||
|
|
||||||
|
**File**: `/etc/apt/sources.list.d/raspi.sources`
|
||||||
|
```
|
||||||
|
Types: deb
|
||||||
|
URIs: http://archive.raspberrypi.com/debian/
|
||||||
|
Suites: trixie
|
||||||
|
Components: main
|
||||||
|
Signed-By: /usr/share/keyrings/raspberrypi-archive-keyring.pgp
|
||||||
|
```
|
||||||
|
|
||||||
|
**File**: `/etc/apt/preferences.d/raspi-pin`
|
||||||
|
```
|
||||||
|
# Pin RaspiOS kernel and firmware packages
|
||||||
|
Package: raspberrypi-kernel raspberrypi-bootloader libraspberrypi* firmware-brcm80211
|
||||||
|
Pin: release o=Raspberry Pi Foundation
|
||||||
|
Pin-Priority: 1001
|
||||||
|
|
||||||
|
# Default to Debian for everything else
|
||||||
|
Package: *
|
||||||
|
Pin: release o=Debian
|
||||||
|
Pin-Priority: 500
|
||||||
|
```
|
||||||
|
|
||||||
|
### Update Behavior
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt update
|
||||||
|
# Fetches package lists from:
|
||||||
|
# - Debian repositories (default)
|
||||||
|
# - RaspiOS repository (for kernel/firmware)
|
||||||
|
|
||||||
|
sudo apt upgrade
|
||||||
|
# Upgrades:
|
||||||
|
# - raspberrypi-kernel → from RaspiOS repo (priority 1001)
|
||||||
|
# - raspberrypi-bootloader → from RaspiOS repo (priority 1001)
|
||||||
|
# - libraspberrypi* → from RaspiOS repo (priority 1001)
|
||||||
|
# - firmware-brcm80211 → from RaspiOS repo (priority 1001)
|
||||||
|
# - All other packages → from Debian repos (priority 500)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Result**: Kernel and firmware stay in sync with RaspiOS, userspace packages track Debian.
|
||||||
|
|
||||||
|
## Boot Modes
|
||||||
|
|
||||||
|
### Cloud-Init Mode (CLOUD=true)
|
||||||
|
|
||||||
|
**Use when**:
|
||||||
|
- Using Debian cloud images
|
||||||
|
- Need cloud-init features (network config, metadata, etc.)
|
||||||
|
|
||||||
|
**Files**:
|
||||||
|
- `cloudinit/user-data` - User creation, SSH config, runcmd
|
||||||
|
- `cloudinit/meta-data` - Instance ID, hostname
|
||||||
|
- `cloudinit/seed.img` - Auto-generated ISO (CIDATA volume)
|
||||||
|
|
||||||
|
**Boot process**:
|
||||||
|
1. QEMU mounts seed.img (cloud-init config)
|
||||||
|
2. QEMU mounts setup.iso (build scripts)
|
||||||
|
3. Cloud-init creates user and runs runcmd
|
||||||
|
4. runcmd mounts setup.iso and executes setup.sh
|
||||||
|
5. VM shuts down after setup
|
||||||
|
|
||||||
|
### First-Boot Service Mode (CLOUD=false)
|
||||||
|
|
||||||
|
**Use when**:
|
||||||
|
- Using generic Debian images
|
||||||
|
- Don't need cloud-init
|
||||||
|
- Want minimal dependencies
|
||||||
|
|
||||||
|
**Files**:
|
||||||
|
- `first-boot/setup-runner.sh` - Creates user, runs setup
|
||||||
|
- `first-boot/setup-runner.service` - Systemd one-shot service
|
||||||
|
|
||||||
|
**Boot process**:
|
||||||
|
1. Autobuild injects files into Debian image (before QEMU)
|
||||||
|
2. Autobuild enables systemd service via chroot
|
||||||
|
3. QEMU boots Debian
|
||||||
|
4. Systemd starts setup-runner.service
|
||||||
|
5. Service creates user, mounts setup.iso, runs setup.sh
|
||||||
|
6. Service disables itself
|
||||||
|
7. VM shuts down after setup
|
||||||
|
|
||||||
|
## Hardware Detection
|
||||||
|
|
||||||
|
### Detection Points
|
||||||
|
|
||||||
|
**1. Build Time (QEMU)**:
|
||||||
|
- Install packages needed for hardware detection
|
||||||
|
- Copy detection scripts to /etc/setupfiles/
|
||||||
|
|
||||||
|
**2. First Boot (Raspberry Pi)**:
|
||||||
|
- Detect network interfaces (eth0, eth1)
|
||||||
|
- Detect WiFi adapters (wlan0, wlan1)
|
||||||
|
- Detect USB Zigbee dongles
|
||||||
|
- Configure services based on detected hardware
|
||||||
|
|
||||||
|
### Network Interface Detection
|
||||||
|
|
||||||
|
**File**: `services/base/first-boot/init.sh`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Detect eth1 (second NIC)
|
||||||
|
if ip link show eth1 >/dev/null 2>&1; then
|
||||||
|
# Dual NIC mode
|
||||||
|
# br-wan (eth0) - WAN with DHCP client
|
||||||
|
# br-lan (eth1) - LAN with DHCP server + NAT
|
||||||
|
else
|
||||||
|
# Single NIC mode
|
||||||
|
# br-wan (eth0) - WAN with DHCP client
|
||||||
|
fi
|
||||||
|
```
|
||||||
|
|
||||||
|
**Result**:
|
||||||
|
- Single NIC: WAN only
|
||||||
|
- Dual NIC: WAN + LAN with DHCP/NAT
|
||||||
|
|
||||||
|
### WiFi Detection
|
||||||
|
|
||||||
|
**File**: `services/hotspot/first-boot/init.sh`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Detect WiFi interfaces
|
||||||
|
wlan0_exists=$(ip link show wlan0 2>/dev/null)
|
||||||
|
wlan1_exists=$(ip link show wlan1 2>/dev/null)
|
||||||
|
|
||||||
|
if [[ -n "$wlan0_exists" && -n "$wlan1_exists" ]]; then
|
||||||
|
# Dual-band: 2.4GHz on wlan0, 5GHz on wlan1
|
||||||
|
elif [[ -n "$wlan0_exists" ]]; then
|
||||||
|
# Single-band: 5GHz on wlan0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Determine bridge
|
||||||
|
if ip link show br-lan >/dev/null 2>&1; then
|
||||||
|
BRIDGE="br-lan"
|
||||||
|
else
|
||||||
|
BRIDGE="br-wan"
|
||||||
|
fi
|
||||||
|
```
|
||||||
|
|
||||||
|
**Result**:
|
||||||
|
- Dual WiFi: 2.4GHz + 5GHz APs
|
||||||
|
- Single WiFi: 5GHz AP only
|
||||||
|
- Adapts to available bridge
|
||||||
|
|
||||||
|
### USB Zigbee Detection
|
||||||
|
|
||||||
|
**File**: `services/haos/first-boot/init.sh`
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Scan USB serial devices
|
||||||
|
for device in /dev/ttyUSB* /dev/ttyACM*; do
|
||||||
|
device_info=$(udevadm info -q property -n "$device")
|
||||||
|
|
||||||
|
# Check vendor for known Zigbee coordinators
|
||||||
|
if echo "$device_info" | grep -qiE "(FTDI|Silicon_Labs|Texas_Instruments|dresden_elektronik|ITead|Sonoff)"; then
|
||||||
|
# Extract USB IDs
|
||||||
|
USB_VENDOR=$(echo "$device_info" | grep "ID_VENDOR_ID=" | cut -d'=' -f2)
|
||||||
|
USB_PRODUCT=$(echo "$device_info" | grep "ID_MODEL_ID=" | cut -d'=' -f2)
|
||||||
|
|
||||||
|
# Pass through to Home Assistant VM
|
||||||
|
incus config device add haos zigbee-dongle usb \
|
||||||
|
vendorid="$USB_VENDOR" \
|
||||||
|
productid="$USB_PRODUCT" \
|
||||||
|
required=false
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
**Result**: Zigbee coordinators automatically available in Home Assistant.
|
||||||
|
|
||||||
|
## Image Size Management
|
||||||
|
|
||||||
|
### Size Calculation
|
||||||
|
|
||||||
|
**Autobuild logic**:
|
||||||
|
```bash
|
||||||
|
# Debian image size
|
||||||
|
DEBIAN_SIZE=$(qemu-img info --output=json debian.raw | jq -r '.["virtual-size"]')
|
||||||
|
|
||||||
|
# Add overhead (1-2GB for services, temp files, expansion)
|
||||||
|
FINAL_SIZE=$((DEBIAN_SIZE + 2GB))
|
||||||
|
|
||||||
|
# Override with config.sh IMAGE_SIZE if specified
|
||||||
|
```
|
||||||
|
|
||||||
|
### Partition Expansion
|
||||||
|
|
||||||
|
**During merge** (`merge-debian-raspios.sh`):
|
||||||
|
1. Create output image with `IMAGE_SIZE`
|
||||||
|
2. Resize partition table
|
||||||
|
3. Expand root partition to fill available space
|
||||||
|
4. Resize ext4 filesystem
|
||||||
|
|
||||||
|
**On first boot** (`rpi-first-boot.sh`):
|
||||||
|
1. Expand root partition to fill SD card
|
||||||
|
2. Resize ext4 filesystem to match
|
||||||
|
3. Reboot to apply changes
|
||||||
|
|
||||||
|
**Result**: Image expands to fill entire SD card, regardless of size.
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
The architecture uses:
|
||||||
|
- **Hybrid approach** - RaspiOS boot + Debian rootfs
|
||||||
|
- **QEMU ARM64** - Native package installation before merge
|
||||||
|
- **Modular services** - Composable image components
|
||||||
|
- **APT pinning** - Automatic kernel/firmware updates
|
||||||
|
- **Hardware detection** - Runtime configuration based on detected hardware
|
||||||
|
- **Two boot modes** - Cloud-init or first-boot service
|
||||||
|
|
||||||
|
This design provides:
|
||||||
|
- Full Raspberry Pi hardware support
|
||||||
|
- Safe automatic updates
|
||||||
|
- Easy customization
|
||||||
|
- Reproducible builds
|
||||||
|
- Team collaboration via version control
|
||||||
|
|
||||||
|
**Next**: [Learn about the build system](Build-System.md)
|
||||||
323
Available-Images.md
Normal file
323
Available-Images.md
Normal file
|
|
@ -0,0 +1,323 @@
|
||||||
|
# Available Images
|
||||||
|
|
||||||
|
Pre-configured images available for download and their use cases.
|
||||||
|
|
||||||
|
## Official Images
|
||||||
|
|
||||||
|
All images available in [GitHub Releases](https://github.com/Pikatsuto/raspberry-builds/releases).
|
||||||
|
|
||||||
|
### debian-base
|
||||||
|
|
||||||
|
**Description**: Minimal Debian 13 (Trixie) with RaspiOS kernel and firmware
|
||||||
|
|
||||||
|
**Services included**:
|
||||||
|
- base (RaspiOS kernel, NetworkManager, SSH)
|
||||||
|
|
||||||
|
**Size**: ~512MB compressed
|
||||||
|
|
||||||
|
**RAM required**: 2GB minimum
|
||||||
|
|
||||||
|
**Use cases**:
|
||||||
|
- Minimal Debian installation
|
||||||
|
- Custom development base
|
||||||
|
- Learning/testing Raspberry Pi
|
||||||
|
|
||||||
|
**Default login**:
|
||||||
|
- Username: `pi`
|
||||||
|
- Password: `raspberry`
|
||||||
|
|
||||||
|
**Download**:
|
||||||
|
```bash
|
||||||
|
wget https://github.com/Pikatsuto/raspberry-builds/releases/latest/download/debian-base.img.xz
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### debian-qemu-docker
|
||||||
|
|
||||||
|
**Description**: Debian with Incus virtualization and Docker Engine
|
||||||
|
|
||||||
|
**Services included**:
|
||||||
|
- base
|
||||||
|
- qemu (Incus + KVM)
|
||||||
|
- docker (Docker Engine + Portainer + Watchtower)
|
||||||
|
|
||||||
|
**Size**: ~4-5GB compressed
|
||||||
|
|
||||||
|
**RAM required**: 4GB minimum (8GB recommended)
|
||||||
|
|
||||||
|
**Use cases**:
|
||||||
|
- Container host (Docker + Incus)
|
||||||
|
- Development platform
|
||||||
|
- Microservices deployment
|
||||||
|
- VM/container testing
|
||||||
|
|
||||||
|
**Access**:
|
||||||
|
- Portainer: `https://raspberry-ip:9443`
|
||||||
|
- Incus: `incus list` (SSH)
|
||||||
|
|
||||||
|
**Download**:
|
||||||
|
```bash
|
||||||
|
wget https://github.com/Pikatsuto/raspberry-builds/releases/latest/download/debian-qemu-docker.img.xz
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### debian-qemu-openwrt-hotspot
|
||||||
|
|
||||||
|
**Description**: Debian with OpenWrt router and WiFi hotspot
|
||||||
|
|
||||||
|
**Services included**:
|
||||||
|
- base
|
||||||
|
- qemu (Incus)
|
||||||
|
- openwrt (OpenWrt container)
|
||||||
|
- hotspot (WiFi AP)
|
||||||
|
|
||||||
|
**Size**: ~3-4GB compressed
|
||||||
|
|
||||||
|
**RAM required**: 4GB minimum
|
||||||
|
|
||||||
|
**Hardware required**:
|
||||||
|
- Dual NIC (eth0 + eth1) or single NIC
|
||||||
|
- WiFi adapter(s) for hotspot
|
||||||
|
|
||||||
|
**Use cases**:
|
||||||
|
- Network router/firewall
|
||||||
|
- WiFi access point
|
||||||
|
- VPN gateway
|
||||||
|
- Network segmentation
|
||||||
|
|
||||||
|
**Access**:
|
||||||
|
- OpenWrt LuCI: `http://192.168.10.1`
|
||||||
|
- WiFi SSID: `RaspberryPi-5G` (password: `raspberry`)
|
||||||
|
|
||||||
|
**Download**:
|
||||||
|
```bash
|
||||||
|
wget https://github.com/Pikatsuto/raspberry-builds/releases/latest/download/debian-qemu-openwrt-hotspot.img.xz
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### debian-qemu-docker-openwrt-hotspot-haos (Full Stack)
|
||||||
|
|
||||||
|
**Description**: Complete home automation and network platform
|
||||||
|
|
||||||
|
**Services included**:
|
||||||
|
- base
|
||||||
|
- qemu (Incus + KVM)
|
||||||
|
- docker (Docker Engine + Portainer)
|
||||||
|
- openwrt (OpenWrt router)
|
||||||
|
- hotspot (WiFi AP)
|
||||||
|
- haos (Home Assistant OS)
|
||||||
|
|
||||||
|
**Size**: ~6-8GB compressed
|
||||||
|
|
||||||
|
**RAM required**: 8GB (Raspberry Pi 5 8GB recommended)
|
||||||
|
|
||||||
|
**Hardware required**:
|
||||||
|
- Dual NIC (eth0 + eth1) recommended
|
||||||
|
- WiFi adapter(s) for hotspot
|
||||||
|
- Optional: Zigbee/Z-Wave USB dongle
|
||||||
|
|
||||||
|
**Use cases**:
|
||||||
|
- Complete home automation gateway
|
||||||
|
- Network router + WiFi + smart home
|
||||||
|
- All-in-one Raspberry Pi solution
|
||||||
|
|
||||||
|
**Access**:
|
||||||
|
- Home Assistant: `http://raspberry-ip:8123`
|
||||||
|
- Portainer: `https://raspberry-ip:9443`
|
||||||
|
- OpenWrt LuCI: `http://192.168.10.1`
|
||||||
|
- WiFi SSID: `RaspberryPi-5G`
|
||||||
|
|
||||||
|
**Download**:
|
||||||
|
```bash
|
||||||
|
wget https://github.com/Pikatsuto/raspberry-builds/releases/latest/download/debian-qemu-docker-openwrt-hotspot-haos.img.xz
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Image Comparison
|
||||||
|
|
||||||
|
| Feature | base | qemu-docker | qemu-openwrt-hotspot | Full Stack |
|
||||||
|
|---------|------|-------------|----------------------|------------|
|
||||||
|
| Debian OS | ✅ | ✅ | ✅ | ✅ |
|
||||||
|
| RaspiOS Kernel | ✅ | ✅ | ✅ | ✅ |
|
||||||
|
| Network Bridges | ✅ | ✅ | ✅ | ✅ |
|
||||||
|
| SSH | ✅ | ✅ | ✅ | ✅ |
|
||||||
|
| Incus (VMs/containers) | ❌ | ✅ | ✅ | ✅ |
|
||||||
|
| Docker | ❌ | ✅ | ❌ | ✅ |
|
||||||
|
| Portainer | ❌ | ✅ | ❌ | ✅ |
|
||||||
|
| OpenWrt Router | ❌ | ❌ | ✅ | ✅ |
|
||||||
|
| WiFi Hotspot | ❌ | ❌ | ✅ | ✅ |
|
||||||
|
| Home Assistant | ❌ | ❌ | ❌ | ✅ |
|
||||||
|
| Zigbee Auto-detect | ❌ | ❌ | ❌ | ✅ |
|
||||||
|
| Minimum RAM | 2GB | 4GB | 4GB | 8GB |
|
||||||
|
| Compressed Size | ~512MB | ~700MB | ~750MB | ~750MB |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Release Channels
|
||||||
|
|
||||||
|
### Stable (main branch)
|
||||||
|
|
||||||
|
**Recommended for production use**
|
||||||
|
|
||||||
|
- Tested and verified
|
||||||
|
- No pre-release flag
|
||||||
|
- Downloaded from latest release
|
||||||
|
|
||||||
|
**Download**:
|
||||||
|
```bash
|
||||||
|
# Latest stable
|
||||||
|
wget https://github.com/Pikatsuto/raspberry-builds/releases/latest/download/<image-name>.img.xz
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test (test branch)
|
||||||
|
|
||||||
|
**For testing new features**
|
||||||
|
|
||||||
|
- Pre-release builds
|
||||||
|
- May contain bugs
|
||||||
|
- Use for testing only
|
||||||
|
|
||||||
|
**Download**:
|
||||||
|
```bash
|
||||||
|
# Browse releases and select test build
|
||||||
|
# URL: https://github.com/Pikatsuto/raspberry-builds/releases
|
||||||
|
```
|
||||||
|
|
||||||
|
### Preview (preview branch)
|
||||||
|
|
||||||
|
**Experimental builds**
|
||||||
|
|
||||||
|
- Bleeding-edge features
|
||||||
|
- Unstable
|
||||||
|
- Use at your own risk
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Daily Builds
|
||||||
|
|
||||||
|
**Automatic daily builds** at 2:00 AM UTC
|
||||||
|
|
||||||
|
**Tag format**: `daily-YYYY-MM-DD`
|
||||||
|
|
||||||
|
**Purpose**:
|
||||||
|
- Captures latest base image updates
|
||||||
|
- Tests build system
|
||||||
|
- Provides fresh images daily
|
||||||
|
|
||||||
|
**Download**:
|
||||||
|
```bash
|
||||||
|
# Today's build
|
||||||
|
wget https://github.com/Pikatsuto/raspberry-builds/releases/download/daily-2024-12-08/<image-name>.img.xz
|
||||||
|
```
|
||||||
|
|
||||||
|
**Note**: Daily builds overwrite previous daily release. Download and archive if needed.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
### 1. Download Image
|
||||||
|
|
||||||
|
Choose image from above, download via browser or wget.
|
||||||
|
|
||||||
|
**Verify checksum**:
|
||||||
|
```bash
|
||||||
|
sha256sum <image-name>.img.xz
|
||||||
|
# Compare with .sha256 file from release
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Flash to SD Card
|
||||||
|
|
||||||
|
**Linux**:
|
||||||
|
```bash
|
||||||
|
xz -dc <image-name>.img.xz | sudo dd of=/dev/sdX bs=4M status=progress conv=fsync
|
||||||
|
sync
|
||||||
|
```
|
||||||
|
|
||||||
|
**macOS**:
|
||||||
|
```bash
|
||||||
|
xz -dc <image-name>.img.xz | sudo dd of=/dev/rdiskX bs=4m
|
||||||
|
sync
|
||||||
|
```
|
||||||
|
|
||||||
|
**Windows**:
|
||||||
|
- Use [Balena Etcher](https://www.balena.io/etcher/)
|
||||||
|
- Select `.img.xz` file
|
||||||
|
- Select SD card
|
||||||
|
- Flash
|
||||||
|
|
||||||
|
### 3. Boot Raspberry Pi
|
||||||
|
|
||||||
|
- Insert SD card
|
||||||
|
- Connect Ethernet (recommended for first boot)
|
||||||
|
- Power on
|
||||||
|
- Wait 3-5 minutes for first-boot setup
|
||||||
|
- Find IP address from router or MOTD on login
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Customization After Installation
|
||||||
|
|
||||||
|
All images can be customized post-installation:
|
||||||
|
|
||||||
|
**Update system**:
|
||||||
|
```bash
|
||||||
|
sudo apt update && sudo apt upgrade -y
|
||||||
|
```
|
||||||
|
|
||||||
|
**Install additional packages**:
|
||||||
|
```bash
|
||||||
|
sudo apt install <package-name>
|
||||||
|
```
|
||||||
|
|
||||||
|
**Add Docker containers** (if Docker image):
|
||||||
|
```bash
|
||||||
|
docker run -d <container-image>
|
||||||
|
# Or use Portainer web UI
|
||||||
|
```
|
||||||
|
|
||||||
|
**Create Incus containers/VMs** (if qemu image):
|
||||||
|
```bash
|
||||||
|
incus launch images:alpine mycontainer
|
||||||
|
incus launch images:debian myvm --vm
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Building Custom Images
|
||||||
|
|
||||||
|
Don't see the image you need? Build your own!
|
||||||
|
|
||||||
|
**See**: [Creating Custom Images](Custom-Images.md)
|
||||||
|
|
||||||
|
**Quick example**:
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/Pikatsuto/raspberry-builds.git
|
||||||
|
cd raspberry-builds
|
||||||
|
|
||||||
|
# Build custom combination
|
||||||
|
./bin/autobuild --image debian/qemu+docker+haos
|
||||||
|
|
||||||
|
# Or create entirely custom image
|
||||||
|
cp -r images/debian images/myproject
|
||||||
|
vim images/myproject/config.sh
|
||||||
|
./bin/autobuild --image myproject
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Support
|
||||||
|
|
||||||
|
**Issues with images**:
|
||||||
|
- [Report bug](https://github.com/Pikatsuto/raspberry-builds/issues/new?template=bug_report.md)
|
||||||
|
- [Request feature](https://github.com/Pikatsuto/raspberry-builds/issues/new?template=feature_request.md)
|
||||||
|
- [Ask question](https://github.com/Pikatsuto/raspberry-builds/discussions)
|
||||||
|
|
||||||
|
**Image documentation**:
|
||||||
|
- [Getting Started](Getting-Started.md)
|
||||||
|
- [FAQ](FAQ.md)
|
||||||
|
- [Troubleshooting](Troubleshooting.md)
|
||||||
76
Available-Services.md
Normal file
76
Available-Services.md
Normal file
|
|
@ -0,0 +1,76 @@
|
||||||
|
# Available Services
|
||||||
|
|
||||||
|
Quick reference for service modules. See [Services Guide](Services.md) for creating custom services.
|
||||||
|
|
||||||
|
## Service List
|
||||||
|
|
||||||
|
### base
|
||||||
|
**Always included** - RaspiOS kernel, NetworkManager, SSH, network bridges
|
||||||
|
- Location: `images/debian/services/base/`
|
||||||
|
|
||||||
|
### qemu
|
||||||
|
**Incus virtualization** - Containers and VMs with KVM
|
||||||
|
- Dependencies: None
|
||||||
|
- Command: `./bin/autobuild --image debian/qemu`
|
||||||
|
|
||||||
|
### docker
|
||||||
|
**Docker Engine** - Portainer + Watchtower
|
||||||
|
- Dependencies: None
|
||||||
|
- Access: `https://raspberry-ip:9443` (Portainer)
|
||||||
|
- Command: `./bin/autobuild --image debian/docker`
|
||||||
|
|
||||||
|
### haos
|
||||||
|
**Home Assistant OS** - VM with Zigbee auto-detect
|
||||||
|
- Dependencies: qemu
|
||||||
|
- Access: `http://raspberry-ip:8123`
|
||||||
|
- Command: `./bin/autobuild --image debian/qemu+haos`
|
||||||
|
|
||||||
|
### openwrt
|
||||||
|
**OpenWrt router** - Container with advanced routing
|
||||||
|
- Dependencies: qemu
|
||||||
|
- Access: `http://192.168.10.1` (LuCI)
|
||||||
|
- Command: `./bin/autobuild --image debian/qemu+openwrt`
|
||||||
|
|
||||||
|
### hotspot
|
||||||
|
**WiFi access point** - Auto-detects wlan0/wlan1
|
||||||
|
- Dependencies: None
|
||||||
|
- SSID: `RaspberryPi-5G` (password: `raspberry`)
|
||||||
|
- Command: `./bin/autobuild --image debian/hotspot`
|
||||||
|
|
||||||
|
## Resource Requirements
|
||||||
|
|
||||||
|
| Service | RAM | Disk | Special Hardware |
|
||||||
|
|---------|-----|------|------------------|
|
||||||
|
| base | 500MB | 2GB | - |
|
||||||
|
| qemu | +100MB | +500MB | - |
|
||||||
|
| docker | +200MB | +1GB | - |
|
||||||
|
| haos | +4GB | +32GB | Optional: USB Zigbee |
|
||||||
|
| openwrt | +100MB | +500MB | Optional: Dual NIC |
|
||||||
|
| hotspot | +50MB | +100MB | WiFi adapter |
|
||||||
|
|
||||||
|
## Common Combinations
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Home server
|
||||||
|
./bin/autobuild --image debian/docker
|
||||||
|
|
||||||
|
# Virtualization platform
|
||||||
|
./bin/autobuild --image debian/qemu+docker
|
||||||
|
|
||||||
|
# Network router + WiFi
|
||||||
|
./bin/autobuild --image debian/qemu+openwrt+hotspot
|
||||||
|
|
||||||
|
# Home automation
|
||||||
|
./bin/autobuild --image debian/qemu+haos
|
||||||
|
|
||||||
|
# Full stack (requires 8GB RAM)
|
||||||
|
./bin/autobuild --image debian/qemu+docker+openwrt+hotspot+haos
|
||||||
|
```
|
||||||
|
|
||||||
|
## Service Details
|
||||||
|
|
||||||
|
For detailed information:
|
||||||
|
- [Service architecture](Services.md)
|
||||||
|
- [Creating custom services](Services/#creating-custom-services)
|
||||||
|
- [Hardware detection](Hardware-Detection.md)
|
||||||
|
- [Pre-built images](Available-Images.md)
|
||||||
565
Build-System.md
Normal file
565
Build-System.md
Normal file
|
|
@ -0,0 +1,565 @@
|
||||||
|
# Build System
|
||||||
|
|
||||||
|
Complete reference for the autobuild system and image creation.
|
||||||
|
|
||||||
|
## Autobuild Command
|
||||||
|
|
||||||
|
The `autobuild` script orchestrates the entire build process.
|
||||||
|
|
||||||
|
### Basic Usage
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/autobuild --image <image-name>
|
||||||
|
```
|
||||||
|
|
||||||
|
### Image Formats
|
||||||
|
|
||||||
|
**Physical directory format**:
|
||||||
|
```bash
|
||||||
|
./bin/autobuild --image debian
|
||||||
|
# Uses images/debian/config.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
**Dynamic service composition**:
|
||||||
|
```bash
|
||||||
|
./bin/autobuild --image debian/qemu+docker+haos
|
||||||
|
# Uses images/debian/config.sh + combines services
|
||||||
|
```
|
||||||
|
|
||||||
|
### Common Options
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build specific image
|
||||||
|
./bin/autobuild --image debian
|
||||||
|
|
||||||
|
# Build all images from .github/images.txt
|
||||||
|
./bin/autobuild --all-images
|
||||||
|
|
||||||
|
# List available images
|
||||||
|
./bin/autobuild --list-images
|
||||||
|
|
||||||
|
# Skip base image downloads (use cached)
|
||||||
|
./bin/autobuild --image debian --skip-download
|
||||||
|
|
||||||
|
# Skip QEMU setup (use existing Debian image)
|
||||||
|
./bin/autobuild --image debian --skip-qemu
|
||||||
|
|
||||||
|
# Skip compression
|
||||||
|
./bin/autobuild --image debian --skip-compress
|
||||||
|
|
||||||
|
# Clean previous builds
|
||||||
|
./bin/autobuild --image debian --clean
|
||||||
|
```
|
||||||
|
|
||||||
|
### Combined Options
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Quick rebuild without downloads or compression
|
||||||
|
./bin/autobuild --image debian --skip-download --skip-compress
|
||||||
|
|
||||||
|
# Rebuild with new setup scripts, skip downloads
|
||||||
|
./bin/autobuild --image debian --skip-download --clean
|
||||||
|
```
|
||||||
|
|
||||||
|
## Build Stages
|
||||||
|
|
||||||
|
### Stage 1: Download & Prepare
|
||||||
|
|
||||||
|
**Actions**:
|
||||||
|
- Download RaspiOS Lite image (if not cached)
|
||||||
|
- Download Debian ARM64 image (if not cached)
|
||||||
|
- Parse image configuration (config.sh)
|
||||||
|
- Resolve service dependencies
|
||||||
|
- Combine setup.sh from all services
|
||||||
|
- Merge setupfiles/ directories
|
||||||
|
- Create setup.iso with combined configuration
|
||||||
|
- Generate cloud-init seed.img OR inject first-boot service
|
||||||
|
|
||||||
|
**Outputs**:
|
||||||
|
- `raspios-lite.img` (in distro directory)
|
||||||
|
- `debian-arm64.raw` (in distro directory)
|
||||||
|
- `setup.iso` (in image directory)
|
||||||
|
- `seed.img` (in cloudinit/ directory, if CLOUD=true)
|
||||||
|
|
||||||
|
**Environment Variables**:
|
||||||
|
```bash
|
||||||
|
RASPIOS_URL="https://downloads.raspberrypi.org/..."
|
||||||
|
IMAGE_URL="https://cloud.debian.org/..." # from config.sh
|
||||||
|
CLOUD=true # or false, from config.sh
|
||||||
|
SERVICES="base qemu docker" # from config.sh or dynamic
|
||||||
|
```
|
||||||
|
|
||||||
|
### Stage 2: QEMU Setup
|
||||||
|
|
||||||
|
**Actions**:
|
||||||
|
- Convert Debian image to raw format (if qcow2)
|
||||||
|
- Launch QEMU ARM64 VM:
|
||||||
|
- RAM: QEMU_RAM (from config.sh)
|
||||||
|
- CPUs: QEMU_CPUS (from config.sh)
|
||||||
|
- Disk: Debian image
|
||||||
|
- CD-ROM 1: seed.img (if cloud-init mode)
|
||||||
|
- CD-ROM 2: setup.iso
|
||||||
|
- Wait for setup completion (VM auto-shutdown)
|
||||||
|
- Monitor progress via QEMU serial console
|
||||||
|
|
||||||
|
**QEMU Configuration**:
|
||||||
|
```bash
|
||||||
|
qemu-system-aarch64 \
|
||||||
|
-machine virt \
|
||||||
|
-cpu cortex-a72 \
|
||||||
|
-m $QEMU_RAM \
|
||||||
|
-smp $QEMU_CPUS \
|
||||||
|
-drive file=debian.raw,format=raw,if=virtio \
|
||||||
|
-drive file=seed.img,format=raw,if=virtio,readonly=on \ # cloud-init
|
||||||
|
-drive file=setup.iso,format=raw,if=virtio,readonly=on \
|
||||||
|
-bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd \
|
||||||
|
-nographic \
|
||||||
|
-netdev user,id=net0 \
|
||||||
|
-device virtio-net-pci,netdev=net0
|
||||||
|
```
|
||||||
|
|
||||||
|
**Inside QEMU**:
|
||||||
|
1. Cloud-init or first-boot service creates user
|
||||||
|
2. Setup script mounts setup.iso
|
||||||
|
3. Executes combined setup.sh:
|
||||||
|
- Adds RaspiOS repository
|
||||||
|
- Installs RaspiOS kernel/firmware
|
||||||
|
- Installs service packages
|
||||||
|
- Copies setupfiles to /etc/setupfiles/
|
||||||
|
- Installs first-boot scripts
|
||||||
|
4. Shuts down VM
|
||||||
|
|
||||||
|
**Timeout**: 30 minutes (configurable via `QEMU_TIMEOUT`)
|
||||||
|
|
||||||
|
### Stage 3: Merge
|
||||||
|
|
||||||
|
**Actions**:
|
||||||
|
- Call `merge-debian-raspios.sh`
|
||||||
|
- Keep RaspiOS boot partition (FAT32)
|
||||||
|
- Replace RaspiOS root with Debian root
|
||||||
|
- Restore RaspiOS fstab (for correct partition UUIDs)
|
||||||
|
- Resize root partition to IMAGE_SIZE
|
||||||
|
|
||||||
|
**Merge Process** (10 sub-stages):
|
||||||
|
1. Dependency verification
|
||||||
|
2. RaspiOS preparation (decompress if .xz)
|
||||||
|
3. Debian preparation (convert to raw if needed)
|
||||||
|
4. Size analysis
|
||||||
|
5. Output image creation (copy of RaspiOS)
|
||||||
|
6. Image resizing
|
||||||
|
7. Loop device mounting
|
||||||
|
8. Backup RaspiOS fstab
|
||||||
|
9. **Rootfs replacement**:
|
||||||
|
- Delete RaspiOS root
|
||||||
|
- Rsync Debian root with `-aAXv` (preserve all attributes)
|
||||||
|
- Create `/boot/firmware` mount point
|
||||||
|
- Restore fstab
|
||||||
|
10. Cleanup
|
||||||
|
|
||||||
|
**Output**: `<image-name>.img`
|
||||||
|
|
||||||
|
### Stage 4: Compress
|
||||||
|
|
||||||
|
**Actions**:
|
||||||
|
- Run PiShrink to minimize filesystem
|
||||||
|
- Compress with xz (parallel, level 6)
|
||||||
|
- Generate SHA256 checksum
|
||||||
|
|
||||||
|
**PiShrink**:
|
||||||
|
```bash
|
||||||
|
pishrink.sh -z <image>.img <image>.img.xz
|
||||||
|
# -z: Compress with xz after shrinking
|
||||||
|
```
|
||||||
|
|
||||||
|
**Output**: `<image-name>.img.xz`
|
||||||
|
|
||||||
|
## Configuration Files
|
||||||
|
|
||||||
|
### Image Configuration (config.sh)
|
||||||
|
|
||||||
|
**Location**: `images/<distro>/config.sh` or `images/<image-name>/config.sh`
|
||||||
|
|
||||||
|
**Required Variables**:
|
||||||
|
```bash
|
||||||
|
# Output filename
|
||||||
|
OUTPUT_IMAGE="debian-base.img"
|
||||||
|
|
||||||
|
# Final image size (supports K, M, G suffixes)
|
||||||
|
IMAGE_SIZE="8G"
|
||||||
|
|
||||||
|
# QEMU resources
|
||||||
|
QEMU_RAM="8G"
|
||||||
|
QEMU_CPUS="4"
|
||||||
|
|
||||||
|
# Boot mode (true = cloud-init, false = first-boot service)
|
||||||
|
CLOUD=true
|
||||||
|
|
||||||
|
# Base distribution image URL
|
||||||
|
IMAGE_URL="https://cloud.debian.org/images/cloud/trixie-backports/daily/latest/debian-13-backports-genericcloud-arm64-daily.raw"
|
||||||
|
|
||||||
|
# Services to include (space-separated)
|
||||||
|
SERVICES="base qemu docker"
|
||||||
|
|
||||||
|
# Description (optional, for documentation)
|
||||||
|
DESCRIPTION="Debian with Incus and Docker"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Optional Variables**:
|
||||||
|
```bash
|
||||||
|
# Custom RaspiOS URL (default: RaspiOS Lite trixie)
|
||||||
|
RASPIOS_URL="https://downloads.raspberrypi.org/..."
|
||||||
|
|
||||||
|
# QEMU timeout in seconds (default: 1800 = 30 minutes)
|
||||||
|
QEMU_TIMEOUT=3600
|
||||||
|
|
||||||
|
# Skip PiShrink compression
|
||||||
|
SKIP_PISHRINK=false
|
||||||
|
```
|
||||||
|
|
||||||
|
### Service Configuration
|
||||||
|
|
||||||
|
Each service directory contains:
|
||||||
|
|
||||||
|
**setup.sh** (required):
|
||||||
|
```bash
|
||||||
|
#!/bin/bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Install packages
|
||||||
|
apt update
|
||||||
|
apt install -y package1 package2
|
||||||
|
|
||||||
|
# Configure system
|
||||||
|
systemctl enable service1
|
||||||
|
```
|
||||||
|
|
||||||
|
**first-boot/init.sh** (optional):
|
||||||
|
```bash
|
||||||
|
#!/bin/bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Runtime configuration
|
||||||
|
# Detect hardware, create containers, etc.
|
||||||
|
```
|
||||||
|
|
||||||
|
**depends.sh** (optional):
|
||||||
|
```bash
|
||||||
|
# Declare dependencies
|
||||||
|
DEPENDS_ON="qemu"
|
||||||
|
```
|
||||||
|
|
||||||
|
**motd.sh** (optional):
|
||||||
|
```bash
|
||||||
|
# MOTD content
|
||||||
|
cat <<'EOF'
|
||||||
|
Service UI: https://raspberry-ip:9000
|
||||||
|
Username: admin
|
||||||
|
EOF
|
||||||
|
```
|
||||||
|
|
||||||
|
**setupfiles/** (optional):
|
||||||
|
- Static configuration files
|
||||||
|
- Copied to /etc/setupfiles/ during build
|
||||||
|
|
||||||
|
## Build Artifacts
|
||||||
|
|
||||||
|
### Directory Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
images/
|
||||||
|
└── debian/ # Distribution directory
|
||||||
|
├── config.sh # Base configuration
|
||||||
|
├── cloudinit/ # Cloud-init mode
|
||||||
|
│ ├── user-data
|
||||||
|
│ ├── meta-data
|
||||||
|
│ └── seed.img # Auto-generated
|
||||||
|
├── services/ # Service modules
|
||||||
|
│ ├── base/
|
||||||
|
│ ├── qemu/
|
||||||
|
│ └── docker/
|
||||||
|
├── raspios-lite.img # Downloaded RaspiOS (cached)
|
||||||
|
├── debian-arm64.raw # Downloaded Debian (cached)
|
||||||
|
└── debian-qemu-docker/ # Dynamic image (created during build)
|
||||||
|
├── config.sh # Generated from base + overrides
|
||||||
|
├── setup.sh # Combined from services
|
||||||
|
├── setup.iso # Generated
|
||||||
|
├── setupfiles/ # Merged from services
|
||||||
|
├── debian-qemu-docker.img # Final image
|
||||||
|
└── debian-qemu-docker.img.xz # Compressed
|
||||||
|
```
|
||||||
|
|
||||||
|
### Artifact Persistence
|
||||||
|
|
||||||
|
**Persistent** (cached between builds):
|
||||||
|
- `raspios-lite.img` - RaspiOS base image
|
||||||
|
- `debian-arm64.raw` - Debian base image (modified by QEMU)
|
||||||
|
- Final `.img` and `.img.xz` files
|
||||||
|
|
||||||
|
**Temporary** (cleaned with `--clean`):
|
||||||
|
- Dynamic image directories (e.g., `debian-qemu-docker/`)
|
||||||
|
- `setup.iso`
|
||||||
|
- `seed.img` (regenerated each build)
|
||||||
|
|
||||||
|
**Skip Downloads**:
|
||||||
|
```bash
|
||||||
|
# Use cached base images
|
||||||
|
./bin/autobuild --image debian --skip-download
|
||||||
|
```
|
||||||
|
|
||||||
|
**Skip QEMU**:
|
||||||
|
```bash
|
||||||
|
# Use existing configured Debian image (skip setup in QEMU)
|
||||||
|
./bin/autobuild --image debian --skip-qemu
|
||||||
|
```
|
||||||
|
|
||||||
|
## Service Dependency Resolution
|
||||||
|
|
||||||
|
### Dependency Declaration
|
||||||
|
|
||||||
|
**Example**: Home Assistant requires Incus
|
||||||
|
|
||||||
|
**File**: `images/debian/services/haos/depends.sh`
|
||||||
|
```bash
|
||||||
|
DEPENDS_ON="qemu"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Resolution Algorithm
|
||||||
|
|
||||||
|
**Input**: `debian/qemu+docker+haos`
|
||||||
|
|
||||||
|
**Process**:
|
||||||
|
1. Parse services: `qemu`, `docker`, `haos`
|
||||||
|
2. Resolve dependencies:
|
||||||
|
- `qemu`: no dependencies
|
||||||
|
- `docker`: no dependencies
|
||||||
|
- `haos`: depends on `qemu` (already in list)
|
||||||
|
3. Remove duplicates
|
||||||
|
4. Order by dependencies: `base` → `qemu` → `docker` → `haos`
|
||||||
|
|
||||||
|
**Output**: `SERVICES="base qemu docker haos"`
|
||||||
|
|
||||||
|
### Build Order
|
||||||
|
|
||||||
|
Services are built in dependency order:
|
||||||
|
|
||||||
|
1. **base** (always first)
|
||||||
|
2. Dependencies (e.g., `qemu` for `haos`)
|
||||||
|
3. Requested services
|
||||||
|
|
||||||
|
**Setup script combination**:
|
||||||
|
```bash
|
||||||
|
{
|
||||||
|
cat services/base/setup.sh
|
||||||
|
cat services/qemu/setup.sh
|
||||||
|
cat services/docker/setup.sh
|
||||||
|
cat services/haos/setup.sh
|
||||||
|
} > combined-setup.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
## Merge Process Details
|
||||||
|
|
||||||
|
### Partition Operations
|
||||||
|
|
||||||
|
**1. RaspiOS Boot Partition** (kept):
|
||||||
|
```bash
|
||||||
|
# Mount as read-only (no changes)
|
||||||
|
mount -o ro /dev/loop0p1 /mnt/raspios-boot
|
||||||
|
```
|
||||||
|
|
||||||
|
**2. RaspiOS Root Partition** (backed up, then replaced):
|
||||||
|
```bash
|
||||||
|
# Backup fstab only
|
||||||
|
cp /mnt/raspios-root/etc/fstab /tmp/raspios-fstab
|
||||||
|
|
||||||
|
# Delete entire rootfs
|
||||||
|
rm -rf /mnt/raspios-root/*
|
||||||
|
```
|
||||||
|
|
||||||
|
**3. Debian Root Partition** (source):
|
||||||
|
```bash
|
||||||
|
# Rsync to RaspiOS root
|
||||||
|
rsync -aAXv /mnt/debian-root/ /mnt/raspios-root/
|
||||||
|
|
||||||
|
# Preserve attributes:
|
||||||
|
# -a: archive mode (recursive, preserve permissions, times, symlinks)
|
||||||
|
# -A: preserve ACLs
|
||||||
|
# -X: preserve extended attributes
|
||||||
|
# -v: verbose
|
||||||
|
```
|
||||||
|
|
||||||
|
**4. Restore RaspiOS fstab**:
|
||||||
|
```bash
|
||||||
|
# RaspiOS fstab has correct partition UUIDs
|
||||||
|
cp /tmp/raspios-fstab /mnt/raspios-root/etc/fstab
|
||||||
|
```
|
||||||
|
|
||||||
|
**5. Create boot mount point**:
|
||||||
|
```bash
|
||||||
|
# Ensure /boot/firmware exists for boot partition
|
||||||
|
mkdir -p /mnt/raspios-root/boot/firmware
|
||||||
|
```
|
||||||
|
|
||||||
|
### Size Calculation
|
||||||
|
|
||||||
|
**Automatic sizing**:
|
||||||
|
```bash
|
||||||
|
# Get Debian image virtual size
|
||||||
|
DEBIAN_SIZE=$(qemu-img info --output=json debian.raw | jq -r '.["virtual-size"]')
|
||||||
|
|
||||||
|
# Add overhead (1-2GB)
|
||||||
|
AUTO_SIZE=$((DEBIAN_SIZE + 2 * 1024 * 1024 * 1024))
|
||||||
|
|
||||||
|
# Use IMAGE_SIZE from config if larger
|
||||||
|
FINAL_SIZE=$(max $AUTO_SIZE $IMAGE_SIZE)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Manual override**:
|
||||||
|
```bash
|
||||||
|
./bin/merge-debian-raspios.sh raspios.img debian.raw -s 16G
|
||||||
|
```
|
||||||
|
|
||||||
|
### Partition Expansion
|
||||||
|
|
||||||
|
**During merge**:
|
||||||
|
```bash
|
||||||
|
# Resize partition table
|
||||||
|
parted /dev/loop0 resizepart 2 100%
|
||||||
|
|
||||||
|
# Resize ext4 filesystem
|
||||||
|
e2fsck -f /dev/loop0p2
|
||||||
|
resize2fs /dev/loop0p2
|
||||||
|
```
|
||||||
|
|
||||||
|
**On first boot** (rpi-first-boot.sh):
|
||||||
|
```bash
|
||||||
|
# Expand to fill entire SD card
|
||||||
|
parted /dev/mmcblk0 resizepart 2 100%
|
||||||
|
resize2fs /dev/mmcblk0p2
|
||||||
|
reboot
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting Builds
|
||||||
|
|
||||||
|
### QEMU Won't Boot
|
||||||
|
|
||||||
|
**Symptoms**: QEMU hangs at boot
|
||||||
|
|
||||||
|
**Causes**:
|
||||||
|
- Missing UEFI firmware
|
||||||
|
- Wrong image format
|
||||||
|
- Insufficient RAM
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
```bash
|
||||||
|
# Install UEFI firmware
|
||||||
|
sudo apt install qemu-efi-aarch64
|
||||||
|
|
||||||
|
# Check image format
|
||||||
|
qemu-img info debian.raw
|
||||||
|
|
||||||
|
# Increase RAM in config.sh
|
||||||
|
QEMU_RAM="8G"
|
||||||
|
```
|
||||||
|
|
||||||
|
### QEMU Timeout
|
||||||
|
|
||||||
|
**Symptoms**: Build fails with "QEMU timeout"
|
||||||
|
|
||||||
|
**Causes**:
|
||||||
|
- Slow network (downloading packages)
|
||||||
|
- Insufficient resources
|
||||||
|
- Stuck on interactive prompt
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
```bash
|
||||||
|
# Increase timeout
|
||||||
|
QEMU_TIMEOUT=3600 # 1 hour
|
||||||
|
|
||||||
|
# Increase resources
|
||||||
|
QEMU_RAM="8G"
|
||||||
|
QEMU_CPUS="4"
|
||||||
|
|
||||||
|
# Check QEMU logs
|
||||||
|
cat qemu-*.log
|
||||||
|
```
|
||||||
|
|
||||||
|
### Merge Fails
|
||||||
|
|
||||||
|
**Symptoms**: Error during merge stage
|
||||||
|
|
||||||
|
**Causes**:
|
||||||
|
- Insufficient disk space
|
||||||
|
- Corrupted images
|
||||||
|
- Partition layout mismatch
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
```bash
|
||||||
|
# Check disk space
|
||||||
|
df -h
|
||||||
|
|
||||||
|
# Re-download base images
|
||||||
|
rm images/debian/raspios-lite.img images/debian/debian-arm64.raw
|
||||||
|
./bin/autobuild --image debian
|
||||||
|
|
||||||
|
# Check partition layout
|
||||||
|
fdisk -l raspios.img
|
||||||
|
fdisk -l debian.raw
|
||||||
|
```
|
||||||
|
|
||||||
|
### Image Won't Boot
|
||||||
|
|
||||||
|
**Symptoms**: Raspberry Pi won't boot image
|
||||||
|
|
||||||
|
**Causes**:
|
||||||
|
- Corrupted SD card
|
||||||
|
- Wrong fstab UUIDs
|
||||||
|
- Missing boot files
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
```bash
|
||||||
|
# Verify image integrity
|
||||||
|
sha256sum image.img.xz
|
||||||
|
|
||||||
|
# Check SD card
|
||||||
|
sudo badblocks -v /dev/sdX
|
||||||
|
|
||||||
|
# Re-flash image
|
||||||
|
xz -dc image.img.xz | sudo dd of=/dev/sdX bs=4M status=progress conv=fsync
|
||||||
|
```
|
||||||
|
|
||||||
|
## Advanced Usage
|
||||||
|
|
||||||
|
### Custom RaspiOS Version
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# In config.sh
|
||||||
|
RASPIOS_URL="https://downloads.raspberrypi.org/raspios_lite_arm64/images/raspios_lite_arm64-2024-11-24/2024-11-24-raspios-trixie-arm64-lite.img.xz"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Custom Debian Version
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# In config.sh
|
||||||
|
IMAGE_URL="https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-genericcloud-arm64.raw"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Parallel Builds
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build multiple images in parallel
|
||||||
|
./bin/autobuild --image debian &
|
||||||
|
./bin/autobuild --image debian/qemu+docker &
|
||||||
|
wait
|
||||||
|
```
|
||||||
|
|
||||||
|
**Warning**: Ensure sufficient RAM and disk space for parallel builds.
|
||||||
|
|
||||||
|
### CI/CD Integration
|
||||||
|
|
||||||
|
See [GitHub Actions](GitHub-Actions.md) for automated builds.
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
- [Learn about available services](Services.md)
|
||||||
|
- [Create a custom image](Custom-Images.md)
|
||||||
|
- [Set up CI/CD](GitHub-Actions.md)
|
||||||
90
Command-Reference.md
Normal file
90
Command-Reference.md
Normal file
|
|
@ -0,0 +1,90 @@
|
||||||
|
# Command Reference
|
||||||
|
|
||||||
|
Quick reference for build commands.
|
||||||
|
|
||||||
|
## autobuild
|
||||||
|
|
||||||
|
Main build script.
|
||||||
|
|
||||||
|
### Basic Usage
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/autobuild --image <name>
|
||||||
|
```
|
||||||
|
|
||||||
|
### Options
|
||||||
|
|
||||||
|
```bash
|
||||||
|
--image <name> Build specific image
|
||||||
|
--all-images Build all images from .github/images.txt
|
||||||
|
--list-images List available images
|
||||||
|
--skip-download Use cached base images
|
||||||
|
--skip-qemu Skip QEMU setup
|
||||||
|
--skip-compress Skip PiShrink compression
|
||||||
|
--clean Clean previous build artifacts
|
||||||
|
```
|
||||||
|
|
||||||
|
### Examples
|
||||||
|
|
||||||
|
See [Build System - Autobuild Command](Build-System/#autobuild-command) for comprehensive examples and options.
|
||||||
|
|
||||||
|
## merge-debian-raspios.sh
|
||||||
|
|
||||||
|
Low-level merge script (called by autobuild).
|
||||||
|
|
||||||
|
### Usage
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/merge-debian-raspios.sh <raspios-image> <debian-image> [options]
|
||||||
|
```
|
||||||
|
|
||||||
|
### Options
|
||||||
|
|
||||||
|
```bash
|
||||||
|
-o, --output <file> Output image name
|
||||||
|
-s, --size <size> Final image size (e.g., 16G)
|
||||||
|
-k, --keep-kernel Use Debian kernel (not recommended)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Examples
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Basic merge
|
||||||
|
./bin/merge-debian-raspios.sh raspios.img debian.raw
|
||||||
|
|
||||||
|
# Custom output and size
|
||||||
|
./bin/merge-debian-raspios.sh raspios.img debian.raw -o custom.img -s 16G
|
||||||
|
```
|
||||||
|
|
||||||
|
## Flashing Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Decompress and flash
|
||||||
|
xz -dc image.img.xz | sudo dd of=/dev/sdX bs=4M status=progress conv=fsync
|
||||||
|
|
||||||
|
# Flash uncompressed
|
||||||
|
sudo dd if=image.img of=/dev/sdX bs=4M status=progress conv=fsync
|
||||||
|
|
||||||
|
# Sync
|
||||||
|
sync
|
||||||
|
```
|
||||||
|
|
||||||
|
## System Commands (on Raspberry Pi)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Update system
|
||||||
|
sudo apt update && sudo apt upgrade -y
|
||||||
|
|
||||||
|
# Check first-boot logs
|
||||||
|
sudo journalctl -u services-first-boot
|
||||||
|
|
||||||
|
# Check service status
|
||||||
|
sudo systemctl status docker
|
||||||
|
incus list
|
||||||
|
|
||||||
|
# Network info
|
||||||
|
ip addr show
|
||||||
|
nmcli con show
|
||||||
|
```
|
||||||
|
|
||||||
|
See [Build System](Build-System.md) for detailed usage.
|
||||||
65
Configuration-Reference.md
Normal file
65
Configuration-Reference.md
Normal file
|
|
@ -0,0 +1,65 @@
|
||||||
|
# Configuration Reference
|
||||||
|
|
||||||
|
Quick reference for configuration files.
|
||||||
|
|
||||||
|
## config.sh
|
||||||
|
|
||||||
|
Image configuration file.
|
||||||
|
|
||||||
|
**Location**: `images/<name>/config.sh`
|
||||||
|
|
||||||
|
**Required**:
|
||||||
|
```bash
|
||||||
|
OUTPUT_IMAGE="image-name.img"
|
||||||
|
IMAGE_SIZE="8G"
|
||||||
|
QEMU_RAM="4G"
|
||||||
|
QEMU_CPUS="4"
|
||||||
|
CLOUD=true # or false
|
||||||
|
IMAGE_URL="https://..."
|
||||||
|
SERVICES="base service1 service2"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Optional**:
|
||||||
|
```bash
|
||||||
|
DESCRIPTION="Image description"
|
||||||
|
RASPIOS_URL="https://..."
|
||||||
|
QEMU_TIMEOUT=1800
|
||||||
|
```
|
||||||
|
|
||||||
|
## Service Files
|
||||||
|
|
||||||
|
See [Services - Creating Custom Services](Services/#creating-custom-services) for detailed templates and best practices.
|
||||||
|
|
||||||
|
**Quick reference**:
|
||||||
|
- `setup.sh`: Runs in QEMU, installs packages
|
||||||
|
- `first-boot/init.sh`: Runs on first boot, detects hardware
|
||||||
|
- `depends.sh`: Declares service dependencies
|
||||||
|
- `motd.sh`: MOTD banner content
|
||||||
|
|
||||||
|
## Network Configuration (on Raspberry Pi)
|
||||||
|
|
||||||
|
### Static IP
|
||||||
|
```bash
|
||||||
|
sudo nmcli con mod br-wan ipv4.addresses "192.168.1.100/24"
|
||||||
|
sudo nmcli con mod br-wan ipv4.gateway "192.168.1.1"
|
||||||
|
sudo nmcli con mod br-wan ipv4.dns "8.8.8.8"
|
||||||
|
sudo nmcli con mod br-wan ipv4.method manual
|
||||||
|
sudo nmcli con up br-wan
|
||||||
|
```
|
||||||
|
|
||||||
|
### WiFi Hotspot
|
||||||
|
|
||||||
|
See [Hardware Detection - WiFi Adapters](Hardware-Detection/#wifi-adapters) for detailed configuration.
|
||||||
|
|
||||||
|
**Quick edit**: `/etc/hostapd/hostapd-5ghz.conf` (SSID, password), then `sudo systemctl restart hostapd-5ghz`
|
||||||
|
|
||||||
|
## APT Configuration
|
||||||
|
|
||||||
|
See [Architecture - APT Repository Management](Architecture/#apt-repository-management) for full details on repository configuration and update behavior.
|
||||||
|
|
||||||
|
**Files**: `/etc/apt/sources.list.d/raspi.sources` and `/etc/apt/preferences.d/raspi-pin`
|
||||||
|
|
||||||
|
For detailed explanations see:
|
||||||
|
- [Build System](Build-System.md)
|
||||||
|
- [Services](Services.md)
|
||||||
|
- [Architecture](Architecture.md)
|
||||||
73
Custom-Images.md
Normal file
73
Custom-Images.md
Normal file
|
|
@ -0,0 +1,73 @@
|
||||||
|
# Creating Custom Images
|
||||||
|
|
||||||
|
Guide to creating your own image configurations.
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Copy base configuration
|
||||||
|
cp -r images/debian images/myproject
|
||||||
|
|
||||||
|
# 2. Edit config
|
||||||
|
vim images/myproject/config.sh
|
||||||
|
|
||||||
|
# 3. Customize setup
|
||||||
|
vim images/myproject/services/base/setup.sh
|
||||||
|
|
||||||
|
# 4. Build
|
||||||
|
./bin/autobuild --image myproject
|
||||||
|
```
|
||||||
|
|
||||||
|
## Configuration File (config.sh)
|
||||||
|
|
||||||
|
**Required variables**:
|
||||||
|
```bash
|
||||||
|
OUTPUT_IMAGE="myproject.img"
|
||||||
|
IMAGE_SIZE="8G"
|
||||||
|
QEMU_RAM="4G"
|
||||||
|
QEMU_CPUS="4"
|
||||||
|
CLOUD=true # or false
|
||||||
|
IMAGE_URL="https://cloud.debian.org/images/cloud/trixie-backports/daily/latest/debian-13-backports-genericcloud-arm64-daily.raw"
|
||||||
|
SERVICES="base qemu docker"
|
||||||
|
DESCRIPTION="My custom image"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Using Service Composition
|
||||||
|
|
||||||
|
**Dynamic images** - compose from existing services:
|
||||||
|
```bash
|
||||||
|
./bin/autobuild --image debian/qemu+docker+myservice
|
||||||
|
```
|
||||||
|
|
||||||
|
**Physical images** - full custom directory:
|
||||||
|
```bash
|
||||||
|
./bin/autobuild --image myproject
|
||||||
|
```
|
||||||
|
|
||||||
|
See [Services - Creating Custom Services](Services/#creating-custom-services) for adding new service modules.
|
||||||
|
|
||||||
|
## Examples
|
||||||
|
|
||||||
|
**Minimal Debian**:
|
||||||
|
```bash
|
||||||
|
SERVICES="base"
|
||||||
|
IMAGE_SIZE="4G"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Docker host**:
|
||||||
|
```bash
|
||||||
|
SERVICES="base docker"
|
||||||
|
IMAGE_SIZE="8G"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Custom service stack**:
|
||||||
|
```bash
|
||||||
|
SERVICES="base qemu docker myapp"
|
||||||
|
IMAGE_SIZE="16G"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
- [Build system options](Build-System.md)
|
||||||
|
- [Create custom service](Services/#creating-custom-services)
|
||||||
|
- [GitHub Actions CI/CD](GitHub-Actions.md)
|
||||||
70
Custom-Services.md
Normal file
70
Custom-Services.md
Normal file
|
|
@ -0,0 +1,70 @@
|
||||||
|
# Creating Custom Services
|
||||||
|
|
||||||
|
Guide to creating new service modules.
|
||||||
|
|
||||||
|
## Service Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
images/debian/services/myservice/
|
||||||
|
├── setup.sh # Package installation (runs in QEMU)
|
||||||
|
├── first-boot/init.sh # Runtime config (runs on first boot)
|
||||||
|
├── setupfiles/ # Static files → /etc/setupfiles/
|
||||||
|
├── depends.sh # Optional: dependencies
|
||||||
|
└── motd.sh # Optional: MOTD banner
|
||||||
|
```
|
||||||
|
|
||||||
|
## Quick Template
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Create service
|
||||||
|
mkdir -p images/debian/services/myservice/first-boot
|
||||||
|
|
||||||
|
# Package installation
|
||||||
|
cat > images/debian/services/myservice/setup.sh <<'EOF'
|
||||||
|
#!/bin/bash
|
||||||
|
set -e
|
||||||
|
apt update
|
||||||
|
apt install -y nginx
|
||||||
|
systemctl enable nginx
|
||||||
|
EOF
|
||||||
|
chmod +x images/debian/services/myservice/setup.sh
|
||||||
|
|
||||||
|
# Runtime configuration
|
||||||
|
cat > images/debian/services/myservice/first-boot/init.sh <<'EOF'
|
||||||
|
#!/bin/bash
|
||||||
|
set -e
|
||||||
|
systemctl start nginx
|
||||||
|
EOF
|
||||||
|
chmod +x images/debian/services/myservice/first-boot/init.sh
|
||||||
|
|
||||||
|
# Build
|
||||||
|
./bin/autobuild --image debian/myservice
|
||||||
|
```
|
||||||
|
|
||||||
|
## Script Guidelines
|
||||||
|
|
||||||
|
**setup.sh** (runs in QEMU):
|
||||||
|
- Install packages
|
||||||
|
- Configure system
|
||||||
|
- Enable services (don't start them)
|
||||||
|
|
||||||
|
**first-boot/init.sh** (runs on Raspberry Pi):
|
||||||
|
- Detect hardware
|
||||||
|
- Download large files
|
||||||
|
- Create containers/VMs
|
||||||
|
- Start services
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
Create `depends.sh`:
|
||||||
|
```bash
|
||||||
|
DEPENDS_ON="qemu" # This service requires qemu
|
||||||
|
```
|
||||||
|
|
||||||
|
## Advanced Topics
|
||||||
|
|
||||||
|
For detailed information see [Services Guide](Services.md):
|
||||||
|
- Service lifecycle
|
||||||
|
- Hardware detection
|
||||||
|
- Best practices
|
||||||
|
- Debugging
|
||||||
502
FAQ.md
Normal file
502
FAQ.md
Normal file
|
|
@ -0,0 +1,502 @@
|
||||||
|
# Frequently Asked Questions
|
||||||
|
|
||||||
|
## General Questions
|
||||||
|
|
||||||
|
### What is this project?
|
||||||
|
|
||||||
|
An automated build system for creating custom Raspberry Pi images with:
|
||||||
|
- Debian ARM64 userspace
|
||||||
|
- Raspberry Pi OS kernel and firmware
|
||||||
|
- Modular service composition
|
||||||
|
- Automatic hardware detection
|
||||||
|
- GitHub Actions CI/CD
|
||||||
|
|
||||||
|
### Why not use standard Raspberry Pi OS?
|
||||||
|
|
||||||
|
You should use standard RaspiOS if it meets your needs. Use this project if you want:
|
||||||
|
- Debian package ecosystem instead of RaspiOS customizations
|
||||||
|
- Modular service composition
|
||||||
|
- Version-controlled image configurations
|
||||||
|
- Automated builds and updates
|
||||||
|
- Custom base distributions (in development)
|
||||||
|
|
||||||
|
### Why not use standard Debian ARM64?
|
||||||
|
|
||||||
|
Standard Debian ARM64 lacks Raspberry Pi-specific drivers:
|
||||||
|
- RP1 chip drivers (Ethernet, USB, GPIO on Pi 5)
|
||||||
|
- Optimized WiFi/Bluetooth firmware
|
||||||
|
- Hardware acceleration
|
||||||
|
- Bootloader configuration
|
||||||
|
|
||||||
|
This project combines Debian with RaspiOS kernel/firmware for full hardware support.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Build Questions
|
||||||
|
|
||||||
|
### How long does a build take?
|
||||||
|
|
||||||
|
**Typical build times**:
|
||||||
|
- Base image: 10-15 minutes
|
||||||
|
- With services: 15-30 minutes
|
||||||
|
- Full stack (all services): 30-45 minutes
|
||||||
|
|
||||||
|
**Factors**:
|
||||||
|
- Internet speed (downloading base images, packages)
|
||||||
|
- CPU speed (QEMU emulation)
|
||||||
|
- RAM available (QEMU VM performance)
|
||||||
|
- Disk speed (image operations)
|
||||||
|
|
||||||
|
### Can I build on macOS or Windows?
|
||||||
|
|
||||||
|
Not directly. The build system requires Linux for:
|
||||||
|
- Loop device support (partition mounting)
|
||||||
|
- QEMU ARM64 with KVM (optional but faster)
|
||||||
|
- ext4 filesystem tools
|
||||||
|
|
||||||
|
**Options**:
|
||||||
|
- Use WSL2 on Windows
|
||||||
|
- Use a Linux VM on macOS
|
||||||
|
- Use GitHub Actions (no local build needed)
|
||||||
|
|
||||||
|
### Why does QEMU use so much RAM?
|
||||||
|
|
||||||
|
QEMU allocates RAM for:
|
||||||
|
- Guest VM (QEMU_RAM, typically 4-8GB)
|
||||||
|
- Host overhead (1-2GB)
|
||||||
|
- Disk cache (varies)
|
||||||
|
|
||||||
|
**Reduce RAM usage**:
|
||||||
|
```bash
|
||||||
|
# In config.sh
|
||||||
|
QEMU_RAM="2G" # Minimum for base builds
|
||||||
|
QEMU_CPUS="2" # Reduce CPU count
|
||||||
|
```
|
||||||
|
|
||||||
|
**Note**: Lower RAM may cause build failures for images with many packages.
|
||||||
|
|
||||||
|
### Can I build multiple images in parallel?
|
||||||
|
|
||||||
|
Yes, but consider resources:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Parallel builds
|
||||||
|
./bin/autobuild --image debian &
|
||||||
|
./bin/autobuild --image debian/qemu &
|
||||||
|
wait
|
||||||
|
```
|
||||||
|
|
||||||
|
**Requirements per build**:
|
||||||
|
- RAM: QEMU_RAM + 2GB overhead
|
||||||
|
- Disk: IMAGE_SIZE * 3 (source + output + temp)
|
||||||
|
- CPU: QEMU_CPUS cores
|
||||||
|
|
||||||
|
**Example**: 2 parallel builds with QEMU_RAM=4G requires 12GB total RAM.
|
||||||
|
|
||||||
|
### Why is PiShrink so slow?
|
||||||
|
|
||||||
|
PiShrink:
|
||||||
|
- Resizes ext4 filesystem
|
||||||
|
- Compresses with xz (CPU-intensive)
|
||||||
|
- Verifies integrity
|
||||||
|
|
||||||
|
**Speed up**:
|
||||||
|
```bash
|
||||||
|
# Skip PiShrink
|
||||||
|
./bin/autobuild --image debian --skip-compress
|
||||||
|
|
||||||
|
# Or use faster compression
|
||||||
|
xz -T0 -1 image.img # Parallel, lower compression
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Hardware Questions
|
||||||
|
|
||||||
|
### Which Raspberry Pi models are supported?
|
||||||
|
|
||||||
|
**Supported**:
|
||||||
|
- Raspberry Pi 4 (all RAM variants)
|
||||||
|
- Raspberry Pi 5 (all RAM variants)
|
||||||
|
|
||||||
|
**Not supported**:
|
||||||
|
- Raspberry Pi 3 and earlier (32-bit ARM)
|
||||||
|
- Raspberry Pi Zero (32-bit, insufficient resources)
|
||||||
|
- Raspberry Pi Compute Module (untested)
|
||||||
|
|
||||||
|
**Reason**: This project uses ARM64 (64-bit) Debian. Pi 3 and earlier are 32-bit ARMv7.
|
||||||
|
|
||||||
|
### Do I need a specific SD card?
|
||||||
|
|
||||||
|
**Recommended**:
|
||||||
|
- Class 10 or better
|
||||||
|
- U1/U3 for video applications
|
||||||
|
- A1/A2 for better random I/O
|
||||||
|
- Reputable brands (SanDisk, Samsung, Kingston)
|
||||||
|
|
||||||
|
**Minimum size**:
|
||||||
|
- Base image: 8GB
|
||||||
|
- With services: 16GB
|
||||||
|
- Full stack: 32GB
|
||||||
|
|
||||||
|
**Avoid**:
|
||||||
|
- Generic/cheap cards (reliability issues)
|
||||||
|
- Cards older than 5 years (wear)
|
||||||
|
|
||||||
|
### Can I use USB SSD instead of SD card?
|
||||||
|
|
||||||
|
Yes! Benefits:
|
||||||
|
- Faster I/O
|
||||||
|
- Better reliability
|
||||||
|
- Longer lifespan
|
||||||
|
|
||||||
|
**Process**:
|
||||||
|
```bash
|
||||||
|
# Flash to USB SSD (same as SD card)
|
||||||
|
xz -dc image.img.xz | sudo dd of=/dev/sdX bs=4M status=progress
|
||||||
|
|
||||||
|
# Boot from USB (Raspberry Pi 4/5 support USB boot)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Raspberry Pi 5**: Native USB boot support
|
||||||
|
**Raspberry Pi 4**: Update bootloader for USB boot
|
||||||
|
|
||||||
|
### Does WiFi work out of the box?
|
||||||
|
|
||||||
|
Yes, with `firmware-brcm80211` package (included in base service).
|
||||||
|
|
||||||
|
**Supported adapters**:
|
||||||
|
- Built-in WiFi on Raspberry Pi 4/5
|
||||||
|
- Most USB WiFi adapters (Realtek, Atheros, etc.)
|
||||||
|
|
||||||
|
**Check compatibility**:
|
||||||
|
```bash
|
||||||
|
# After boot, check WiFi interface
|
||||||
|
ip link show wlan0
|
||||||
|
|
||||||
|
# Check firmware
|
||||||
|
dmesg | grep brcmfmac
|
||||||
|
```
|
||||||
|
|
||||||
|
### Does Bluetooth work?
|
||||||
|
|
||||||
|
Yes, Bluetooth firmware is included.
|
||||||
|
|
||||||
|
**Usage**:
|
||||||
|
```bash
|
||||||
|
# Check Bluetooth
|
||||||
|
bluetoothctl
|
||||||
|
scan on
|
||||||
|
```
|
||||||
|
|
||||||
|
**Pairing**:
|
||||||
|
- Use `bluetoothctl` CLI
|
||||||
|
- Or install GUI: `apt install blueman`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Service Questions
|
||||||
|
|
||||||
|
### Can I run Home Assistant without Incus?
|
||||||
|
|
||||||
|
No. The `haos` service requires Incus to run Home Assistant OS as a VM.
|
||||||
|
|
||||||
|
**Alternative**: Use Home Assistant Container (Docker):
|
||||||
|
```bash
|
||||||
|
# Add to custom service
|
||||||
|
docker run -d --name homeassistant \
|
||||||
|
-v /home/pi/homeassistant:/config \
|
||||||
|
--network=host \
|
||||||
|
ghcr.io/home-assistant/home-assistant:stable
|
||||||
|
```
|
||||||
|
|
||||||
|
### How much RAM do services need?
|
||||||
|
|
||||||
|
**Service RAM usage**:
|
||||||
|
- base: ~500MB
|
||||||
|
- docker: ~200MB + containers
|
||||||
|
- qemu (Incus): ~100MB + VMs/containers
|
||||||
|
- haos VM: 4GB (configurable)
|
||||||
|
- openwrt container: ~100MB
|
||||||
|
- hotspot: ~50MB
|
||||||
|
|
||||||
|
**Total for full stack**:
|
||||||
|
- Minimum: 4GB (Pi 5 8GB recommended)
|
||||||
|
- Comfortable: 8GB
|
||||||
|
|
||||||
|
### Can I disable a service after building?
|
||||||
|
|
||||||
|
Yes:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Stop service
|
||||||
|
sudo systemctl stop service-name
|
||||||
|
|
||||||
|
# Disable service
|
||||||
|
sudo systemctl disable service-name
|
||||||
|
|
||||||
|
# Or remove completely
|
||||||
|
sudo apt remove package-name
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example**: Disable Home Assistant:
|
||||||
|
```bash
|
||||||
|
sudo incus stop haos
|
||||||
|
sudo incus delete haos
|
||||||
|
```
|
||||||
|
|
||||||
|
### How do I add my own service?
|
||||||
|
|
||||||
|
See [Services - Creating Custom Services](Services/#creating-custom-services).
|
||||||
|
|
||||||
|
Quick version:
|
||||||
|
1. Create `images/debian/services/myservice/`
|
||||||
|
2. Add `setup.sh` (package installation)
|
||||||
|
3. Add `first-boot/init.sh` (runtime config)
|
||||||
|
4. Build: `./bin/autobuild --image debian/myservice`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Network Questions
|
||||||
|
|
||||||
|
### What are br-wan and br-lan?
|
||||||
|
|
||||||
|
**Network bridges** for consistent service networking:
|
||||||
|
|
||||||
|
**br-wan** (always created):
|
||||||
|
- Connects to WAN (internet)
|
||||||
|
- DHCP client
|
||||||
|
- Attached to eth0
|
||||||
|
|
||||||
|
**br-lan** (created if eth1 exists):
|
||||||
|
- LAN for internal services
|
||||||
|
- DHCP server (192.168.10.0/24)
|
||||||
|
- NAT enabled
|
||||||
|
- Attached to eth1
|
||||||
|
|
||||||
|
**Why bridges?**
|
||||||
|
- Services attach to bridges, not physical interfaces
|
||||||
|
- Allows flexible network configuration
|
||||||
|
- Supports VMs/containers without network conflicts
|
||||||
|
|
||||||
|
### How do I use a single NIC?
|
||||||
|
|
||||||
|
It works automatically. If only eth0 detected:
|
||||||
|
- br-wan created (WAN)
|
||||||
|
- br-lan not created
|
||||||
|
- Services use br-wan
|
||||||
|
|
||||||
|
### How do I set static IP?
|
||||||
|
|
||||||
|
See [Configuration Reference - Static IP](Configuration-Reference/#static-ip).
|
||||||
|
|
||||||
|
### Can I use WiFi for WAN instead of Ethernet?
|
||||||
|
|
||||||
|
Yes, configure NetworkManager to bridge WiFi to br-wan:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Connect WiFi
|
||||||
|
sudo nmcli dev wifi connect "SSID" password "PASSWORD"
|
||||||
|
|
||||||
|
# Bridge WiFi to br-wan (advanced - requires network downtime)
|
||||||
|
sudo nmcli con add type bridge con-name br-wan ifname br-wan
|
||||||
|
sudo nmcli con add type wifi slave-type bridge con-name wlan0 ifname wlan0 master br-wan ssid "SSID"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Warning**: WiFi bridging may be unreliable. Ethernet recommended for WAN.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Update Questions
|
||||||
|
|
||||||
|
### How do I update the system?
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt update
|
||||||
|
sudo apt upgrade -y
|
||||||
|
```
|
||||||
|
|
||||||
|
This updates:
|
||||||
|
- Debian packages (from Debian repos)
|
||||||
|
- RaspiOS kernel/firmware (from RaspiOS repo)
|
||||||
|
|
||||||
|
### Will updates break hardware support?
|
||||||
|
|
||||||
|
No. APT pinning ensures RaspiOS kernel/firmware packages update from RaspiOS repository, maintaining hardware support.
|
||||||
|
|
||||||
|
**Pinning config**: `/etc/apt/preferences.d/raspi-pin`
|
||||||
|
|
||||||
|
### How do I update to a newer Debian version?
|
||||||
|
|
||||||
|
**Not recommended while Debian 13 (Trixie) is testing/unstable.**
|
||||||
|
|
||||||
|
When Debian 13 is stable:
|
||||||
|
```bash
|
||||||
|
# Standard Debian upgrade process
|
||||||
|
sudo apt update
|
||||||
|
sudo apt full-upgrade
|
||||||
|
sudo reboot
|
||||||
|
```
|
||||||
|
|
||||||
|
Or rebuild image with newer Debian.
|
||||||
|
|
||||||
|
### Can I upgrade the kernel manually?
|
||||||
|
|
||||||
|
Not recommended. Kernel updates via APT automatically.
|
||||||
|
|
||||||
|
**To force kernel update**:
|
||||||
|
```bash
|
||||||
|
sudo apt update
|
||||||
|
sudo apt install --reinstall raspberrypi-kernel
|
||||||
|
sudo reboot
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Troubleshooting Questions
|
||||||
|
|
||||||
|
### Image won't boot (rainbow screen)
|
||||||
|
|
||||||
|
**Causes**:
|
||||||
|
- Corrupted SD card
|
||||||
|
- Incomplete flash
|
||||||
|
- Incompatible Raspberry Pi model
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
1. Verify image integrity:
|
||||||
|
```bash
|
||||||
|
sha256sum image.img.xz
|
||||||
|
```
|
||||||
|
2. Re-flash:
|
||||||
|
```bash
|
||||||
|
xz -dc image.img.xz | sudo dd of=/dev/sdX bs=4M status=progress conv=fsync
|
||||||
|
sync
|
||||||
|
```
|
||||||
|
3. Try different SD card
|
||||||
|
4. Check UART output for errors
|
||||||
|
|
||||||
|
### Can't login (credentials don't work)
|
||||||
|
|
||||||
|
**Default credentials**:
|
||||||
|
- Username: `pi`
|
||||||
|
- Password: `raspberry`
|
||||||
|
|
||||||
|
**If still fails**:
|
||||||
|
- Caps Lock enabled?
|
||||||
|
- Keyboard layout (US by default)
|
||||||
|
- Wait for first-boot to complete (3-5 minutes)
|
||||||
|
|
||||||
|
**Reset password** (mount SD card on another Linux machine):
|
||||||
|
```bash
|
||||||
|
sudo mount /dev/sdX2 /mnt
|
||||||
|
sudo chroot /mnt
|
||||||
|
passwd pi
|
||||||
|
exit
|
||||||
|
sudo umount /mnt
|
||||||
|
```
|
||||||
|
|
||||||
|
### No network / can't get IP
|
||||||
|
|
||||||
|
**Troubleshooting**:
|
||||||
|
1. Check cable:
|
||||||
|
```bash
|
||||||
|
ip link show eth0
|
||||||
|
# Should show "state UP"
|
||||||
|
```
|
||||||
|
2. Check DHCP:
|
||||||
|
```bash
|
||||||
|
sudo nmcli con up br-wan
|
||||||
|
```
|
||||||
|
3. Check NetworkManager:
|
||||||
|
```bash
|
||||||
|
sudo systemctl status NetworkManager
|
||||||
|
```
|
||||||
|
4. Manual IP (see "How do I set static IP?" above)
|
||||||
|
|
||||||
|
### Service didn't start
|
||||||
|
|
||||||
|
**Check first-boot logs**:
|
||||||
|
```bash
|
||||||
|
sudo journalctl -u services-first-boot
|
||||||
|
```
|
||||||
|
|
||||||
|
**Check service status**:
|
||||||
|
```bash
|
||||||
|
sudo systemctl status docker
|
||||||
|
sudo incus list
|
||||||
|
```
|
||||||
|
|
||||||
|
**Common issues**:
|
||||||
|
- First-boot still running (wait 5-10 min)
|
||||||
|
- Insufficient disk space (`df -h`)
|
||||||
|
- Network unavailable during first-boot
|
||||||
|
|
||||||
|
### Build failed in GitHub Actions
|
||||||
|
|
||||||
|
**Check logs**:
|
||||||
|
1. Go to Actions tab
|
||||||
|
2. Click failed workflow
|
||||||
|
3. Click failed job
|
||||||
|
4. Review logs
|
||||||
|
|
||||||
|
**Common issues**:
|
||||||
|
- QEMU timeout (increase QEMU_TIMEOUT)
|
||||||
|
- Out of disk space (reduce IMAGE_SIZE)
|
||||||
|
- Network error (retry build)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Advanced Questions
|
||||||
|
|
||||||
|
### Can I use a different base distribution?
|
||||||
|
|
||||||
|
**Experimental support** for non-Debian distributions is in development.
|
||||||
|
|
||||||
|
**Current**: Only Debian 13 (Trixie) supported
|
||||||
|
|
||||||
|
**Future**: Ubuntu, Fedora, Alpine planned
|
||||||
|
|
||||||
|
**Why Debian-only currently?**
|
||||||
|
- Services use APT (Debian package manager)
|
||||||
|
- RaspiOS repository is Debian-based
|
||||||
|
|
||||||
|
### Can I cross-compile instead of using QEMU?
|
||||||
|
|
||||||
|
Theoretically yes, but:
|
||||||
|
- Complex to set up
|
||||||
|
- Requires ARM64 toolchain
|
||||||
|
- Package installation still needs emulation or chroot
|
||||||
|
|
||||||
|
**QEMU advantages**:
|
||||||
|
- Native ARM64 environment
|
||||||
|
- No cross-compilation issues
|
||||||
|
- Simpler workflow
|
||||||
|
|
||||||
|
### Can I build for Raspberry Pi 3 (32-bit)?
|
||||||
|
|
||||||
|
Not currently. This project uses ARM64 (64-bit) Debian.
|
||||||
|
|
||||||
|
**For Pi 3**:
|
||||||
|
- Use standard Raspberry Pi OS (32-bit)
|
||||||
|
- Or adapt this project for armhf (significant work)
|
||||||
|
|
||||||
|
### How do I contribute?
|
||||||
|
|
||||||
|
See [CONTRIBUTING.md](../CONTRIBUTING.md) for guidelines.
|
||||||
|
|
||||||
|
Quick start:
|
||||||
|
1. Fork repository
|
||||||
|
2. Create feature branch
|
||||||
|
3. Make changes
|
||||||
|
4. Test builds
|
||||||
|
5. Submit pull request
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Support
|
||||||
|
|
||||||
|
**Can't find your answer?**
|
||||||
|
|
||||||
|
- [Check Troubleshooting guide](Troubleshooting.md)
|
||||||
|
- [Open an issue](https://github.com/Pikatsuto/raspberry-builds/issues)
|
||||||
|
- [Start a discussion](https://github.com/Pikatsuto/raspberry-builds/discussions)
|
||||||
274
Getting-Started.md
Normal file
274
Getting-Started.md
Normal file
|
|
@ -0,0 +1,274 @@
|
||||||
|
# Getting Started
|
||||||
|
|
||||||
|
This guide will walk you through installing dependencies, building your first image, and flashing it to an SD card.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
### System Requirements
|
||||||
|
|
||||||
|
- **OS**: Linux (Debian/Ubuntu recommended)
|
||||||
|
- **RAM**: 8GB minimum (16GB recommended for QEMU builds)
|
||||||
|
- **Disk**: 20GB free space per image
|
||||||
|
- **CPU**: x86_64 with virtualization support (for QEMU ARM64 emulation)
|
||||||
|
|
||||||
|
### Required Packages
|
||||||
|
|
||||||
|
Install build dependencies:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt update
|
||||||
|
sudo apt install -y \
|
||||||
|
parted \
|
||||||
|
e2fsprogs \
|
||||||
|
dosfstools \
|
||||||
|
qemu-utils \
|
||||||
|
rsync \
|
||||||
|
xz-utils \
|
||||||
|
genisoimage \
|
||||||
|
qemu-system-aarch64 \
|
||||||
|
qemu-efi-aarch64
|
||||||
|
```
|
||||||
|
|
||||||
|
**Package purposes:**
|
||||||
|
- `parted`, `e2fsprogs`, `dosfstools` - Partition and filesystem tools
|
||||||
|
- `qemu-utils` - Image format conversion
|
||||||
|
- `rsync` - Efficient file copying
|
||||||
|
- `xz-utils` - Compression/decompression
|
||||||
|
- `genisoimage` - Create ISO images for setup scripts
|
||||||
|
- `qemu-system-aarch64` - ARM64 emulation for native package installation
|
||||||
|
|
||||||
|
### Optional: PiShrink
|
||||||
|
|
||||||
|
For image compression (automatically downloaded if not present):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
wget https://raw.githubusercontent.com/Drewsif/PiShrink/master/pishrink.sh
|
||||||
|
chmod +x pishrink.sh
|
||||||
|
sudo mv pishrink.sh /usr/local/bin/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
Clone the repository:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/Pikatsuto/raspberry-builds.git
|
||||||
|
cd raspberry-builds
|
||||||
|
```
|
||||||
|
|
||||||
|
## Building Your First Image
|
||||||
|
|
||||||
|
### Option 1: Base Debian Image
|
||||||
|
|
||||||
|
Build a minimal Debian image with RaspiOS kernel:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/autobuild --image debian
|
||||||
|
```
|
||||||
|
|
||||||
|
This will:
|
||||||
|
1. Download Raspberry Pi OS Lite and Debian 13 cloud images
|
||||||
|
2. Launch QEMU ARM64 VM
|
||||||
|
3. Install RaspiOS kernel and firmware packages
|
||||||
|
4. Merge RaspiOS boot with Debian rootfs
|
||||||
|
5. Compress the final image
|
||||||
|
|
||||||
|
**Output**: `debian-base.img.xz` (approximately 2-3GB compressed)
|
||||||
|
|
||||||
|
**Build time**: 15-30 minutes (depending on internet speed and CPU)
|
||||||
|
|
||||||
|
### Option 2: Image with Services
|
||||||
|
|
||||||
|
Build an image with Docker and Incus:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/autobuild --image debian/qemu+docker
|
||||||
|
```
|
||||||
|
|
||||||
|
Available service combinations:
|
||||||
|
- `debian/qemu+docker` - Incus + Docker Engine
|
||||||
|
- `debian/qemu+haos` - Incus + Home Assistant OS
|
||||||
|
- `debian/qemu+openwrt+hotspot` - Incus + OpenWrt + WiFi AP
|
||||||
|
- `debian/qemu+docker+openwrt+hotspot+haos` - Full stack
|
||||||
|
|
||||||
|
### Build Options
|
||||||
|
|
||||||
|
Speed up subsequent builds:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Skip base image downloads (use cached)
|
||||||
|
./bin/autobuild --image debian --skip-download
|
||||||
|
|
||||||
|
# Skip QEMU setup (use existing Debian image)
|
||||||
|
./bin/autobuild --image debian --skip-qemu
|
||||||
|
|
||||||
|
# Skip compression
|
||||||
|
./bin/autobuild --image debian --skip-compress
|
||||||
|
|
||||||
|
# Build all images defined in .github/images.txt
|
||||||
|
./bin/autobuild --all-images
|
||||||
|
|
||||||
|
# List available images
|
||||||
|
./bin/autobuild --list-images
|
||||||
|
```
|
||||||
|
|
||||||
|
## Flashing to SD Card
|
||||||
|
|
||||||
|
### Find Your SD Card Device
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Before inserting SD card
|
||||||
|
lsblk
|
||||||
|
|
||||||
|
# Insert SD card, then run again
|
||||||
|
lsblk
|
||||||
|
|
||||||
|
# Look for the new device (e.g., /dev/sdc, /dev/mmcblk0)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Warning**: Double-check the device name! Using the wrong device will destroy data.
|
||||||
|
|
||||||
|
### Flash the Image
|
||||||
|
|
||||||
|
Decompress and flash in one command:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# If image is compressed (.xz)
|
||||||
|
xz -dc debian-base.img.xz | sudo dd of=/dev/sdX bs=4M status=progress conv=fsync
|
||||||
|
|
||||||
|
# If image is already decompressed (.img)
|
||||||
|
sudo dd if=debian-base.img of=/dev/sdX bs=4M status=progress conv=fsync
|
||||||
|
```
|
||||||
|
|
||||||
|
Replace `/dev/sdX` with your SD card device.
|
||||||
|
|
||||||
|
Sync and eject:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sync
|
||||||
|
sudo eject /dev/sdX
|
||||||
|
```
|
||||||
|
|
||||||
|
## First Boot
|
||||||
|
|
||||||
|
### Default Credentials
|
||||||
|
|
||||||
|
**Username**: `pi`
|
||||||
|
**Password**: `raspberry`
|
||||||
|
|
||||||
|
**Important**: Change the default password immediately after first login!
|
||||||
|
|
||||||
|
```bash
|
||||||
|
passwd
|
||||||
|
```
|
||||||
|
|
||||||
|
### Network Configuration
|
||||||
|
|
||||||
|
**Default**: DHCP on br-wan bridge. IP displayed in MOTD on login.
|
||||||
|
|
||||||
|
**Static IP**: See [Configuration Reference - Static IP](Configuration-Reference/#static-ip)
|
||||||
|
|
||||||
|
### First Boot Process
|
||||||
|
|
||||||
|
**Duration**: 3-5 minutes for automatic setup (partition resize, hardware detection, service initialization).
|
||||||
|
|
||||||
|
**Details**: See [Hardware Detection - Detection Sequence](Hardware-Detection/#detection-sequence) for complete timeline and monitoring commands.
|
||||||
|
|
||||||
|
## Accessing Services
|
||||||
|
|
||||||
|
Services display access information in the MOTD (Message of the Day) on login.
|
||||||
|
|
||||||
|
### Docker + Portainer
|
||||||
|
|
||||||
|
```
|
||||||
|
Portainer UI: https://<raspberry-ip>:9443
|
||||||
|
Username: admin
|
||||||
|
Password: (set on first access)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Home Assistant OS
|
||||||
|
|
||||||
|
```
|
||||||
|
Home Assistant: http://<raspberry-ip>:8123
|
||||||
|
(First boot setup takes 5-10 minutes)
|
||||||
|
```
|
||||||
|
|
||||||
|
### OpenWrt
|
||||||
|
|
||||||
|
```
|
||||||
|
OpenWrt LuCI: http://192.168.10.1
|
||||||
|
Username: root
|
||||||
|
Password: (none - set on first access)
|
||||||
|
```
|
||||||
|
|
||||||
|
### WiFi Hotspot
|
||||||
|
|
||||||
|
If WiFi adapters detected:
|
||||||
|
|
||||||
|
```
|
||||||
|
SSID: RaspberryPi-5G (or RaspberryPi-2.4G)
|
||||||
|
Password: raspberry
|
||||||
|
```
|
||||||
|
|
||||||
|
## Verifying the Build
|
||||||
|
|
||||||
|
### Check Kernel
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uname -r
|
||||||
|
# Should show: 6.6.x-rpi-v8 or 6.6.x-rpi-2712
|
||||||
|
```
|
||||||
|
|
||||||
|
### Check RaspiOS Packages
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dpkg -l | grep raspberrypi
|
||||||
|
# Should show:
|
||||||
|
# - raspberrypi-kernel
|
||||||
|
# - raspberrypi-bootloader
|
||||||
|
# - libraspberrypi0
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test Hardware
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check WiFi/Bluetooth firmware
|
||||||
|
dmesg | grep brcmfmac
|
||||||
|
|
||||||
|
# Check RP1 drivers (Raspberry Pi 5)
|
||||||
|
lsmod | grep rp1
|
||||||
|
|
||||||
|
# List USB devices
|
||||||
|
lsusb
|
||||||
|
|
||||||
|
# Check network bridges
|
||||||
|
ip link show
|
||||||
|
# Should show: br-wan, and br-lan if dual NIC
|
||||||
|
```
|
||||||
|
|
||||||
|
## Updating the System
|
||||||
|
|
||||||
|
Safe system updates:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt update
|
||||||
|
sudo apt upgrade -y
|
||||||
|
```
|
||||||
|
|
||||||
|
RaspiOS kernel and firmware packages update from the RaspiOS repository, while all other packages update from Debian repositories. No special handling required.
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
- [Learn about the architecture](Architecture.md)
|
||||||
|
- [Explore available services](Available-Services.md)
|
||||||
|
- [Create a custom image](Custom-Images.md)
|
||||||
|
- [Set up GitHub Actions for automated builds](GitHub-Actions.md)
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
For common issues and solutions, see:
|
||||||
|
- [Troubleshooting - Build Issues](Troubleshooting/#build-issues)
|
||||||
|
- [Troubleshooting - Boot Issues](Troubleshooting/#boot-issues)
|
||||||
|
- [Troubleshooting - Service Issues](Troubleshooting/#service-issues)
|
||||||
|
- [FAQ](FAQ.md)
|
||||||
|
- [Open an issue](https://github.com/Pikatsuto/raspberry-builds/issues)
|
||||||
|
|
@ -1,529 +1,523 @@
|
||||||
# GitHub Actions - Automated Build System
|
# GitHub Actions CI/CD
|
||||||
|
|
||||||
This page documents the automated build system powered by GitHub Actions. The workflow automatically builds all Raspberry Pi images, compresses them, and creates GitHub releases with ready-to-flash `.img.xz` files.
|
Automated image builds and releases using GitHub Actions.
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
The GitHub Actions workflow (`.github/workflows/build-images.yml`) provides a complete CI/CD pipeline for building Raspberry Pi images. It automatically:
|
The project includes a sophisticated multi-stage CI/CD pipeline that:
|
||||||
|
- Builds images in parallel
|
||||||
|
- Creates GitHub releases automatically
|
||||||
|
- Uploads compressed images as release assets
|
||||||
|
- Supports multiple branch strategies (stable, test, preview)
|
||||||
|
- Runs daily builds
|
||||||
|
|
||||||
1. Detects all available images in the `images/` directory
|
## Workflow Architecture
|
||||||
2. Downloads base images (RaspiOS + Debian)
|
|
||||||
3. Executes setup scripts in QEMU ARM64
|
|
||||||
4. Creates hybrid images
|
|
||||||
5. Compresses images with PiShrink
|
|
||||||
6. Creates GitHub releases with downloadable assets
|
|
||||||
|
|
||||||
## Workflow Triggers
|
### Build Pipeline (`.github/workflows/build-images.yml`)
|
||||||
|
|
||||||
The workflow runs automatically on:
|
**6-job multi-stage pipeline**:
|
||||||
|
|
||||||
### 1. Push Events
|
|
||||||
- **Trigger**: Any push to any branch
|
|
||||||
- **Branches**: `**` (all branches)
|
|
||||||
- **Purpose**: Immediate build on code changes
|
|
||||||
- **Release Type**:
|
|
||||||
- `main` branch → Stable release
|
|
||||||
- Other branches → Pre-release
|
|
||||||
|
|
||||||
### 2. Scheduled Builds
|
|
||||||
- **Trigger**: Daily at 2:00 AM UTC
|
|
||||||
- **Schedule**: `cron: '0 2 * * *'`
|
|
||||||
- **Purpose**: Fetch latest Debian and RaspiOS base images
|
|
||||||
- **Benefit**: Images stay up-to-date with security patches and new features
|
|
||||||
|
|
||||||
### 3. Manual Dispatch
|
|
||||||
- **Trigger**: Manual workflow run via GitHub Actions UI
|
|
||||||
- **Purpose**: Build on-demand without pushing code
|
|
||||||
- **Usage**: Go to Actions tab → Select workflow → "Run workflow"
|
|
||||||
|
|
||||||
## Build Architecture
|
|
||||||
|
|
||||||
The workflow uses a **4-stage parallel build system** with artifact caching to optimize build time and resource usage.
|
|
||||||
|
|
||||||
```
|
```
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
detect-images
|
||||||
│ Job 1: detect-images │
|
|
||||||
│ └─ Scans images/ directory and outputs image list │
|
|
||||||
└─────────────────────────────────────────────────────────────┘
|
|
||||||
↓
|
↓
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
├─→ stage1-2-download-qemu (parallel matrix)
|
||||||
│ Job 2: stage1-download (Parallel per image) │
|
│ └─→ stage3-build (sequential)
|
||||||
│ ├─ Downloads RaspiOS and Debian base images │
|
│ └─→ stage4-compress (sequential)
|
||||||
│ ├─ Creates setup.iso from setup.sh + setupfiles/ │
|
│
|
||||||
│ └─ Uploads artifacts: *.img, *.raw, seed.img, setup.iso │
|
└─→ create-release (parallel)
|
||||||
└─────────────────────────────────────────────────────────────┘
|
└─→ cleanup-release (final)
|
||||||
↓
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ Job 3: stage2-qemu (Parallel per image) │
|
|
||||||
│ ├─ Downloads artifacts from stage1 │
|
|
||||||
│ ├─ Launches QEMU ARM64 with Debian + setup.iso │
|
|
||||||
│ ├─ Executes setup.sh in native ARM64 environment │
|
|
||||||
│ ├─ Installs RaspiOS kernel/firmware via APT │
|
|
||||||
│ └─ Uploads configured Debian image │
|
|
||||||
└─────────────────────────────────────────────────────────────┘
|
|
||||||
↓
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ Job 4: stage3-build (Parallel per image) │
|
|
||||||
│ ├─ Downloads RaspiOS from stage1 │
|
|
||||||
│ ├─ Downloads configured Debian from stage2 │
|
|
||||||
│ ├─ Runs merge-debian-raspios.sh │
|
|
||||||
│ └─ Uploads hybrid image (rpi-*.img) │
|
|
||||||
└─────────────────────────────────────────────────────────────┘
|
|
||||||
↓
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ Job 5: stage4-compress (Parallel per image) │
|
|
||||||
│ ├─ Downloads hybrid image from stage3 │
|
|
||||||
│ ├─ Compresses with PiShrink + xz │
|
|
||||||
│ └─ Uploads final image (rpi-*.img.xz) │
|
|
||||||
└─────────────────────────────────────────────────────────────┘
|
|
||||||
↓
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ Job 6: create-release │
|
|
||||||
│ ├─ Downloads all final images (rpi-*.img.xz) │
|
|
||||||
│ ├─ Generates release name and tag │
|
|
||||||
│ ├─ Creates GitHub release with build info │
|
|
||||||
│ └─ Uploads all .img.xz files as release assets │
|
|
||||||
└─────────────────────────────────────────────────────────────┘
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Detailed Job Breakdown
|
### Job Descriptions
|
||||||
|
|
||||||
### Job 1: detect-images
|
**1. detect-images**
|
||||||
|
- Reads `.github/images.txt`
|
||||||
|
- Outputs list of images to build
|
||||||
|
- Sets up matrix for parallel builds
|
||||||
|
|
||||||
**Purpose**: Dynamic image detection
|
**2. stage1-2-download-qemu**
|
||||||
|
- Downloads RaspiOS and Debian base images
|
||||||
|
- Runs QEMU setup (installs packages)
|
||||||
|
- Uploads artifacts: Debian image, setup files
|
||||||
|
|
||||||
**Steps**:
|
**3. create-release**
|
||||||
1. Checkout repository
|
- Determines release tag (daily vs push)
|
||||||
2. Scan `images/` directory
|
- Creates GitHub release (or updates existing)
|
||||||
3. Extract directory names (basename)
|
- Sets release as pre-release for non-main branches
|
||||||
4. Convert to JSON array for matrix strategy
|
|
||||||
5. Output image list to GitHub Actions outputs
|
|
||||||
|
|
||||||
**Output**: JSON array like `["raspivirt-incus", "raspivirt-incus+docker"]`
|
**4. stage3-build**
|
||||||
|
- Downloads artifacts from stage1-2
|
||||||
|
- Merges RaspiOS + Debian
|
||||||
|
- Uploads merged image artifact
|
||||||
|
|
||||||
|
**5. stage4-compress**
|
||||||
|
- Downloads merged image
|
||||||
|
- Compresses with PiShrink + xz
|
||||||
|
- Uploads compressed image to GitHub release
|
||||||
|
|
||||||
|
**6. cleanup-release**
|
||||||
|
- Deletes release if no assets uploaded (failure)
|
||||||
|
- Always runs, even on failure
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Setting Up CI/CD
|
||||||
|
|
||||||
|
### 1. Fork/Clone Repository
|
||||||
|
|
||||||
**Code**:
|
|
||||||
```bash
|
```bash
|
||||||
images=$(ls -d images/*/ | xargs -n 1 basename | jq -R -s -c 'split("\n")[:-1]')
|
git clone https://github.com/Pikatsuto/raspberry-builds.git
|
||||||
|
cd raspberry-builds
|
||||||
```
|
```
|
||||||
|
|
||||||
### Job 2: stage1-download
|
### 2. Configure Images to Build
|
||||||
|
|
||||||
**Purpose**: Download and prepare base images
|
Edit `.github/images.txt`:
|
||||||
|
|
||||||
**Strategy**: Matrix parallelization (one job per image)
|
|
||||||
|
|
||||||
**Steps**:
|
|
||||||
1. **Free disk space**: Remove unused packages to prevent disk full errors
|
|
||||||
- Removes: .NET, Android SDK, GHC, CodeQL, Docker images
|
|
||||||
- Frees ~40GB of space
|
|
||||||
2. **Install dependencies**: `wget`, `genisoimage`, `xz-utils`
|
|
||||||
3. **Run autobuild stage 1**: `./bin/autobuild --image <name> --stage 1`
|
|
||||||
- Downloads RaspiOS image (if missing)
|
|
||||||
- Downloads Debian image (if missing)
|
|
||||||
- Creates `setup.iso` from `setup.sh` + `setupfiles/`
|
|
||||||
- Regenerates cloud-init `seed.img`
|
|
||||||
4. **Upload artifacts**:
|
|
||||||
- RaspiOS image (`*.img`)
|
|
||||||
- Debian image (`*.raw`)
|
|
||||||
- Cloud-init seed (`seed.img`)
|
|
||||||
- Setup ISO (`setup.iso`)
|
|
||||||
- Retention: 1 day
|
|
||||||
|
|
||||||
**Artifact Size**: ~2-4GB per image
|
|
||||||
|
|
||||||
### Job 3: stage2-qemu
|
|
||||||
|
|
||||||
**Purpose**: Execute setup script in QEMU ARM64
|
|
||||||
|
|
||||||
**Strategy**: Matrix parallelization (one job per image)
|
|
||||||
|
|
||||||
**Timeout**: 60 minutes (safety limit for long-running setups)
|
|
||||||
|
|
||||||
**Steps**:
|
|
||||||
1. **Install QEMU dependencies**:
|
|
||||||
- `qemu-system-aarch64`: ARM64 system emulator
|
|
||||||
- `qemu-utils`: Image utilities
|
|
||||||
- `qemu-efi-aarch64`: UEFI firmware for ARM64
|
|
||||||
2. **Download artifacts from stage1**:
|
|
||||||
- RaspiOS image (needed for reference)
|
|
||||||
- Debian image (modified during setup)
|
|
||||||
- Cloud-init seed (first-boot configuration)
|
|
||||||
- Setup ISO (contains setup.sh + files)
|
|
||||||
3. **Run autobuild stage 2**: `./bin/autobuild --image <name> --stage 2`
|
|
||||||
- Creates working copy of Debian image
|
|
||||||
- Launches QEMU ARM64 with:
|
|
||||||
- Debian image as main disk
|
|
||||||
- Cloud-init seed for user configuration
|
|
||||||
- Setup ISO for package installation
|
|
||||||
- QEMU executes setup.sh in native ARM64:
|
|
||||||
- System update (`apt update && apt upgrade`)
|
|
||||||
- Essential packages (curl, wget, sudo, SSH, etc.)
|
|
||||||
- RaspiOS repository configuration + APT pinning
|
|
||||||
- RaspiOS kernel/firmware installation
|
|
||||||
- Image-specific software (Incus, Docker, etc.)
|
|
||||||
- Service configuration
|
|
||||||
- Waits for automatic poweroff signal
|
|
||||||
4. **Upload configured Debian image**:
|
|
||||||
- Debian image with RaspiOS kernel + custom software
|
|
||||||
- Retention: 1 day
|
|
||||||
- Compression disabled (speed optimization)
|
|
||||||
|
|
||||||
**Artifact Size**: ~3-6GB per image
|
|
||||||
|
|
||||||
**Key Feature**: Native ARM64 execution ensures compatibility and proper package installation
|
|
||||||
|
|
||||||
### Job 4: stage3-build
|
|
||||||
|
|
||||||
**Purpose**: Create hybrid Raspberry Pi image
|
|
||||||
|
|
||||||
**Strategy**: Matrix parallelization (one job per image)
|
|
||||||
|
|
||||||
**Steps**:
|
|
||||||
1. **Install build dependencies**:
|
|
||||||
- `parted`: Partition management
|
|
||||||
- `e2fsprogs`: ext4 filesystem tools
|
|
||||||
- `dosfstools`: FAT32 filesystem tools
|
|
||||||
- `rsync`: Efficient file copying
|
|
||||||
- `qemu-utils`: Image conversion
|
|
||||||
2. **Enable loop devices**: Load kernel module for disk image mounting
|
|
||||||
3. **Download artifacts**:
|
|
||||||
- RaspiOS image from stage1 (boot partition source)
|
|
||||||
- Configured Debian image from stage2 (rootfs source)
|
|
||||||
4. **Run autobuild stage 3**: `./bin/autobuild --image <name> --stage 3`
|
|
||||||
- Executes `merge-debian-raspios.sh`:
|
|
||||||
- Creates output image based on RaspiOS
|
|
||||||
- Resizes to configured IMAGE_SIZE
|
|
||||||
- Mounts both images via loop devices
|
|
||||||
- Deletes RaspiOS root partition content
|
|
||||||
- Copies Debian rootfs (with RaspiOS kernel pre-installed)
|
|
||||||
- Restores RaspiOS `/etc/fstab`
|
|
||||||
- Creates `/boot/firmware` mount point
|
|
||||||
- Unmounts and cleans up
|
|
||||||
5. **Upload hybrid image**:
|
|
||||||
- Final bootable image (`rpi-*.img`)
|
|
||||||
- Retention: 1 day
|
|
||||||
- Compression disabled (PiShrink does this in stage 4)
|
|
||||||
|
|
||||||
**Artifact Size**: Matches IMAGE_SIZE (e.g., 6GB for raspivirt-incus)
|
|
||||||
|
|
||||||
**Result**: Bootable Raspberry Pi image ready for compression
|
|
||||||
|
|
||||||
### Job 5: stage4-compress
|
|
||||||
|
|
||||||
**Purpose**: Shrink and compress final images
|
|
||||||
|
|
||||||
**Strategy**: Matrix parallelization (one job per image)
|
|
||||||
|
|
||||||
**Steps**:
|
|
||||||
1. **Install PiShrink**:
|
|
||||||
- Downloads from official repository
|
|
||||||
- Moves to `/usr/local/bin/`
|
|
||||||
2. **Download hybrid image from stage3**
|
|
||||||
3. **Run autobuild stage 4**: Compresses with PiShrink
|
|
||||||
- Executes: `sudo pishrink.sh -aZ <image>.img`
|
|
||||||
- `-a`: Aggressive compression
|
|
||||||
- `-Z`: Parallel xz compression
|
|
||||||
- Process:
|
|
||||||
- Shrinks filesystem to minimum size
|
|
||||||
- Removes unused space from partition
|
|
||||||
- Truncates image file
|
|
||||||
- Compresses to `.img.xz`
|
|
||||||
4. **Upload final image**:
|
|
||||||
- Compressed image (`rpi-*.img.xz`)
|
|
||||||
- Retention: 7 days
|
|
||||||
- Typical compression: 6GB → 1-2GB
|
|
||||||
|
|
||||||
**Artifact Size**: ~1-2GB per image (compressed)
|
|
||||||
|
|
||||||
**Compression Ratio**: Typically 60-80% size reduction
|
|
||||||
|
|
||||||
### Job 6: create-release
|
|
||||||
|
|
||||||
**Purpose**: Create GitHub release with all final images
|
|
||||||
|
|
||||||
**Permissions**: `contents: write` (required for release creation)
|
|
||||||
|
|
||||||
**Steps**:
|
|
||||||
1. **Download all compressed images**:
|
|
||||||
- Pattern: `final-image-*`
|
|
||||||
- Merges all image artifacts
|
|
||||||
2. **List built images**: Display file sizes for verification
|
|
||||||
3. **Generate release information**:
|
|
||||||
- **Release tag**: `v<YYYY-MM-DD>-<HHMM>` (e.g., `v2025-12-01-1430`)
|
|
||||||
- **Release name**:
|
|
||||||
- Main branch: `Release <YYYY-MM-DD>`
|
|
||||||
- Other branches: `Pre-release <YYYY-MM-DD> (<branch-name>)`
|
|
||||||
- **Pre-release flag**:
|
|
||||||
- Main branch: `false` (stable release)
|
|
||||||
- Other branches: `true` (experimental release)
|
|
||||||
4. **Create release body**: Markdown with:
|
|
||||||
- Build date
|
|
||||||
- Flash instructions
|
|
||||||
- Build information (branch, commit, workflow link)
|
|
||||||
- Documentation links
|
|
||||||
5. **Create GitHub release**:
|
|
||||||
- Uses `gh release create` CLI
|
|
||||||
- Uploads all `.img.xz` files as assets
|
|
||||||
- Sets pre-release flag based on branch
|
|
||||||
6. **Cleanup**: Remove large files to free disk space
|
|
||||||
|
|
||||||
**Release Example**:
|
|
||||||
```
|
|
||||||
Tag: v2025-12-01-1430
|
|
||||||
Name: Release 2025-12-01
|
|
||||||
Assets:
|
|
||||||
- rpi-raspivirt-incus.img.xz (1.2 GB)
|
|
||||||
- rpi-raspivirt-incus+docker.img.xz (1.5 GB)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Artifact Management
|
|
||||||
|
|
||||||
### Artifact Flow
|
|
||||||
|
|
||||||
```
|
```
|
||||||
stage1 → stage2: Base images (RaspiOS, Debian, seed.img, setup.iso)
|
debian
|
||||||
stage2 → stage3: Configured Debian image
|
debian/qemu+docker
|
||||||
stage3 → stage4: Hybrid image (rpi-*.img)
|
debian/qemu+haos
|
||||||
stage4 → release: Final compressed image (rpi-*.img.xz)
|
debian/qemu+docker+openwrt+hotspot+haos
|
||||||
```
|
```
|
||||||
|
|
||||||
### Retention Policy
|
One image per line. Comments with `#` supported.
|
||||||
|
|
||||||
- **Stage 1-4 artifacts**: 1 day (temporary build artifacts)
|
### 3. Push to GitHub
|
||||||
- **Final compressed images**: 7 days (backup before release)
|
|
||||||
- **Release assets**: Permanent (until manually deleted)
|
|
||||||
|
|
||||||
### Compression Strategy
|
```bash
|
||||||
|
git add .github/images.txt
|
||||||
|
git commit -m "Configure CI/CD images"
|
||||||
|
git push origin main
|
||||||
|
```
|
||||||
|
|
||||||
- **Stage 1-3**: No compression (speed optimization)
|
### 4. Enable GitHub Actions
|
||||||
- **Stage 4**: Full compression with PiShrink + xz
|
|
||||||
- **Rationale**: Intermediate artifacts are deleted quickly; only final images need compression
|
|
||||||
|
|
||||||
## Optimization Strategies
|
- Go to repository Settings → Actions → General
|
||||||
|
- Enable "Allow all actions and reusable workflows"
|
||||||
|
- Save
|
||||||
|
|
||||||
### 1. Parallel Execution
|
### 5. Enable GitHub Pages (for docs)
|
||||||
- All images build simultaneously (matrix strategy)
|
|
||||||
- Total build time = slowest image (not sum of all images)
|
|
||||||
|
|
||||||
### 2. Stage Separation
|
- Go to repository Settings → Pages
|
||||||
- Each stage uploads artifacts for the next stage
|
- Source: GitHub Actions
|
||||||
- Failed stages don't rebuild earlier stages
|
- Save
|
||||||
- Easy to retry individual stages
|
|
||||||
|
|
||||||
### 3. Artifact Caching
|
### 6. Trigger First Build
|
||||||
- Reuses artifacts across jobs
|
|
||||||
- Avoids redundant downloads and builds
|
|
||||||
- Reduces total workflow time
|
|
||||||
|
|
||||||
### 4. Disk Space Management
|
**Manual trigger**:
|
||||||
- Removes unused software before builds
|
- Go to Actions tab
|
||||||
- Cleans up artifacts after each stage
|
- Select "Build Raspberry Pi Images" workflow
|
||||||
- Prevents out-of-disk errors on GitHub runners
|
- Click "Run workflow"
|
||||||
|
- Select branch (main)
|
||||||
|
- Click "Run workflow"
|
||||||
|
|
||||||
### 5. Fail-Fast Disabled
|
**Automatic trigger**:
|
||||||
- `fail-fast: false` in stage2 (QEMU)
|
- Push changes to `images/**`, `bin/**`, or `.github/**`
|
||||||
- One failing image doesn't stop others
|
- Wait for daily cron (2:00 AM UTC)
|
||||||
- Maximizes successful builds
|
|
||||||
|
|
||||||
## Environment Variables
|
---
|
||||||
|
|
||||||
|
## Triggers
|
||||||
|
|
||||||
|
### Push Trigger
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches:
|
||||||
|
- main
|
||||||
|
- test
|
||||||
|
- preview
|
||||||
|
paths:
|
||||||
|
- 'images/**'
|
||||||
|
- 'bin/**'
|
||||||
|
- '.github/**'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Behavior**:
|
||||||
|
- Only triggers when image configs or build scripts change
|
||||||
|
- Ignores documentation-only changes
|
||||||
|
- Supports main, test, preview branches
|
||||||
|
|
||||||
|
### Schedule Trigger
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
on:
|
||||||
|
schedule:
|
||||||
|
- cron: '0 2 * * *' # Daily at 2:00 AM UTC
|
||||||
|
```
|
||||||
|
|
||||||
|
**Behavior**:
|
||||||
|
- Runs daily to capture latest base images
|
||||||
|
- Builds all images from `.github/images.txt`
|
||||||
|
- Creates `daily-YYYY-MM-DD` release
|
||||||
|
|
||||||
|
### Manual Trigger
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
on:
|
||||||
|
workflow_dispatch:
|
||||||
|
```
|
||||||
|
|
||||||
|
**Behavior**:
|
||||||
|
- Run from GitHub Actions UI
|
||||||
|
- Useful for testing or on-demand builds
|
||||||
|
- Supports branch selection
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Release Strategy
|
||||||
|
|
||||||
|
### Release Tagging
|
||||||
|
|
||||||
|
**Daily builds** (cron):
|
||||||
|
```
|
||||||
|
Tag: daily-YYYY-MM-DD
|
||||||
|
Example: daily-2024-12-08
|
||||||
|
```
|
||||||
|
- Overwrites existing daily release
|
||||||
|
- Always pre-release
|
||||||
|
|
||||||
|
**Push builds**:
|
||||||
|
```
|
||||||
|
Tag: vYYYY-MM-DD-HHMM
|
||||||
|
Example: v2024-12-08-1430
|
||||||
|
```
|
||||||
|
- Unique timestamp for each build
|
||||||
|
- Pre-release for test/preview branches
|
||||||
|
- Stable release for main branch
|
||||||
|
|
||||||
|
### Branch Strategy
|
||||||
|
|
||||||
|
**main branch**:
|
||||||
|
- Stable releases
|
||||||
|
- No pre-release flag
|
||||||
|
- Recommended for production
|
||||||
|
|
||||||
|
**test branch**:
|
||||||
|
- Pre-releases with warning banner
|
||||||
|
- For testing new features
|
||||||
|
- May contain bugs
|
||||||
|
|
||||||
|
**preview branch**:
|
||||||
|
- Experimental pre-releases
|
||||||
|
- Bleeding-edge features
|
||||||
|
- Use at your own risk
|
||||||
|
|
||||||
|
### Release Notes
|
||||||
|
|
||||||
|
Auto-generated release notes include:
|
||||||
|
- List of images built
|
||||||
|
- Download links
|
||||||
|
- SHA256 checksums
|
||||||
|
- Installation instructions
|
||||||
|
- Changelog (if commits since last release)
|
||||||
|
|
||||||
|
**Example**:
|
||||||
|
```markdown
|
||||||
|
## Raspberry Pi Images - v2024-12-08-1430
|
||||||
|
|
||||||
|
### Images Built
|
||||||
|
- debian-base.img.xz
|
||||||
|
- debian-qemu-docker.img.xz
|
||||||
|
- debian-qemu-haos.img.xz
|
||||||
|
|
||||||
|
### Installation
|
||||||
|
1. Download image
|
||||||
|
2. Verify checksum
|
||||||
|
3. Flash to SD card:
|
||||||
|
```bash
|
||||||
|
xz -dc image.img.xz | sudo dd of=/dev/sdX bs=4M status=progress
|
||||||
|
```
|
||||||
|
|
||||||
|
### Checksums (SHA256)
|
||||||
|
- debian-base.img.xz: abc123...
|
||||||
|
- debian-qemu-docker.img.xz: def456...
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Artifacts
|
||||||
|
|
||||||
|
### Uploaded Artifacts (Intermediate)
|
||||||
|
|
||||||
|
Used for passing data between stages:
|
||||||
|
|
||||||
|
**stage1-2-download-qemu**:
|
||||||
|
- `debian-<image>-image` - Configured Debian image
|
||||||
|
- `setup-iso-<image>` - Setup scripts
|
||||||
|
- `setupfiles-<image>` - Configuration files
|
||||||
|
|
||||||
|
**stage3-build**:
|
||||||
|
- `hybrid-<image>-image` - Merged image (uncompressed)
|
||||||
|
|
||||||
|
**Retention**: 1 day (deleted after stage4 completes)
|
||||||
|
|
||||||
|
### Release Assets (Final)
|
||||||
|
|
||||||
|
Uploaded to GitHub Releases:
|
||||||
|
|
||||||
|
- `<image-name>.img.xz` - Compressed image
|
||||||
|
- `<image-name>.img.xz.sha256` - Checksum
|
||||||
|
|
||||||
|
**Retention**: Permanent (until manually deleted)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Customizing Workflow
|
||||||
|
|
||||||
|
### Adjust Runner Resources
|
||||||
|
|
||||||
|
For faster builds, use larger runners:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
jobs:
|
||||||
|
stage1-2-download-qemu:
|
||||||
|
runs-on: ubuntu-latest # Change to ubuntu-latest-4-cores or custom runner
|
||||||
|
```
|
||||||
|
|
||||||
|
**Options**:
|
||||||
|
- `ubuntu-latest` - 2 cores, 7GB RAM (free)
|
||||||
|
- `ubuntu-latest-4-cores` - 4 cores, 16GB RAM (paid)
|
||||||
|
- Self-hosted runners
|
||||||
|
|
||||||
|
### Adjust QEMU Timeout
|
||||||
|
|
||||||
|
Increase timeout for slow builds:
|
||||||
|
|
||||||
### Global Environment
|
|
||||||
```yaml
|
```yaml
|
||||||
env:
|
env:
|
||||||
DEBIAN_FRONTEND: noninteractive
|
QEMU_TIMEOUT: 3600 # 1 hour (default: 1800 = 30 min)
|
||||||
```
|
```
|
||||||
- Prevents interactive prompts during apt operations
|
|
||||||
- Required for unattended package installation
|
|
||||||
|
|
||||||
### Job-Specific Variables
|
### Parallel Image Builds
|
||||||
- Passed via autobuild script stages
|
|
||||||
- Configured in `images/<name>/config.sh`:
|
|
||||||
- `OUTPUT_IMAGE`: Final filename
|
|
||||||
- `IMAGE_SIZE`: Target image size
|
|
||||||
- `QEMU_RAM`: RAM for QEMU
|
|
||||||
- `QEMU_CPUS`: CPU cores for QEMU
|
|
||||||
|
|
||||||
## Release Versioning
|
By default, images build in parallel via matrix strategy. To limit parallelism:
|
||||||
|
|
||||||
### Version Format
|
```yaml
|
||||||
- **Pattern**: `vYYYY-MM-DD-HHMM`
|
strategy:
|
||||||
- **Example**: `v2025-12-01-1430` (December 1, 2025 at 14:30)
|
matrix:
|
||||||
- **Uniqueness**: Timestamp ensures unique tags
|
image: ${{ fromJson(needs.detect-images.outputs.images) }}
|
||||||
|
max-parallel: 2 # Limit to 2 concurrent builds
|
||||||
|
```
|
||||||
|
|
||||||
### Release Types
|
### Skip Compression
|
||||||
|
|
||||||
#### Stable Release (main branch)
|
To speed up testing builds:
|
||||||
- **Name**: `Release YYYY-MM-DD`
|
|
||||||
- **Flag**: `prerelease: false`
|
|
||||||
- **Purpose**: Production-ready images
|
|
||||||
- **Visibility**: Featured on repository homepage
|
|
||||||
|
|
||||||
#### Pre-Release (other branches)
|
```yaml
|
||||||
- **Name**: `Pre-release YYYY-MM-DD (branch-name)`
|
- name: Build image
|
||||||
- **Flag**: `prerelease: true`
|
run: ./bin/autobuild --image ${{ matrix.image }} --skip-compress
|
||||||
- **Purpose**: Testing and development
|
```
|
||||||
- **Visibility**: Listed but marked as pre-release
|
|
||||||
|
|
||||||
## Build Duration
|
### Custom Base Images
|
||||||
|
|
||||||
Typical build times on GitHub runners (ubuntu-latest):
|
Override base image URLs via environment variables:
|
||||||
|
|
||||||
- **Stage 1 (Download)**: 3-5 minutes per image
|
```yaml
|
||||||
- **Stage 2 (QEMU)**: 15-30 minutes per image (most time-consuming)
|
env:
|
||||||
- **Stage 3 (Build)**: 5-10 minutes per image
|
RASPIOS_URL: https://example.com/custom-raspios.img.xz
|
||||||
- **Stage 4 (Compress)**: 3-5 minutes per image
|
DEBIAN_URL: https://example.com/custom-debian.raw
|
||||||
- **Stage 5 (Release)**: 1-2 minutes
|
```
|
||||||
|
|
||||||
**Total Time**: ~30-50 minutes for all images (parallel execution)
|
---
|
||||||
|
|
||||||
## Monitoring Builds
|
## Monitoring Builds
|
||||||
|
|
||||||
### Via GitHub UI
|
### GitHub Actions UI
|
||||||
1. Go to repository **Actions** tab
|
|
||||||
2. Select workflow run
|
|
||||||
3. View job progress and logs
|
|
||||||
4. Check artifacts and releases
|
|
||||||
|
|
||||||
### Via GitHub CLI
|
**View running builds**:
|
||||||
```bash
|
1. Go to Actions tab
|
||||||
# List recent workflow runs
|
2. Click on running workflow
|
||||||
gh run list --workflow=build-images.yml
|
3. Click on job to see logs
|
||||||
|
|
||||||
# Watch a running workflow
|
**Check build status**:
|
||||||
gh run watch <run-id>
|
- Green checkmark: Success
|
||||||
|
- Red X: Failure
|
||||||
|
- Yellow circle: In progress
|
||||||
|
|
||||||
# Download artifacts
|
### Logs
|
||||||
gh run download <run-id>
|
|
||||||
```
|
|
||||||
|
|
||||||
## Troubleshooting
|
**Download logs**:
|
||||||
|
1. Go to completed workflow run
|
||||||
|
2. Click "..." (three dots)
|
||||||
|
3. Select "Download log archive"
|
||||||
|
|
||||||
### Build Failures
|
**Key log sections**:
|
||||||
|
- **stage1-2**: QEMU boot, package installation
|
||||||
|
- **stage3**: Merge operation
|
||||||
|
- **stage4**: Compression, upload
|
||||||
|
|
||||||
#### Stage 1 Failures
|
### Debugging Failures
|
||||||
- **Symptom**: Download errors
|
|
||||||
- **Cause**: Network issues, broken URLs
|
|
||||||
- **Solution**: Check base image URLs in `autobuild` script
|
|
||||||
|
|
||||||
#### Stage 2 Failures (QEMU)
|
**QEMU timeout**:
|
||||||
- **Symptom**: QEMU timeout or setup.sh errors
|
- Increase `QEMU_TIMEOUT`
|
||||||
- **Cause**: Package installation failures, network issues in QEMU
|
- Check for interactive prompts in logs
|
||||||
- **Solution**: Check `setup.sh` syntax, verify package names
|
- Verify network connectivity
|
||||||
- **Debugging**: Add `set -x` to `setup.sh` for verbose output
|
|
||||||
|
|
||||||
#### Stage 3 Failures
|
**Merge failure**:
|
||||||
- **Symptom**: Merge errors, loop device issues
|
- Check disk space
|
||||||
- **Cause**: Insufficient permissions, partition layout issues
|
- Verify base image URLs
|
||||||
- **Solution**: Verify image formats, check merge script logic
|
- Review merge logs
|
||||||
|
|
||||||
#### Stage 4 Failures
|
**Upload failure**:
|
||||||
- **Symptom**: PiShrink errors
|
- Check GitHub token permissions
|
||||||
- **Cause**: Filesystem corruption, insufficient disk space
|
- Verify release exists
|
||||||
- **Solution**: Check stage 3 output, verify filesystem integrity
|
- Check artifact size limits (2GB per file)
|
||||||
|
|
||||||
### Disk Space Issues
|
---
|
||||||
- **Symptom**: "No space left on device"
|
|
||||||
- **Solution**: Already handled by "Free disk space" step
|
|
||||||
- **If persists**: Reduce IMAGE_SIZE in config.sh
|
|
||||||
|
|
||||||
### Release Creation Failures
|
|
||||||
- **Symptom**: Permission denied errors
|
|
||||||
- **Cause**: Missing `contents: write` permission
|
|
||||||
- **Solution**: Verify workflow permissions in YAML
|
|
||||||
|
|
||||||
## Customizing the Workflow
|
|
||||||
|
|
||||||
### Adding a New Build Stage
|
|
||||||
|
|
||||||
1. Add new job to `.github/workflows/build-images.yml`
|
|
||||||
2. Define dependencies with `needs: [previous-job]`
|
|
||||||
3. Add matrix strategy if parallel execution needed
|
|
||||||
4. Upload artifacts for next stage
|
|
||||||
5. Update artifact retention as needed
|
|
||||||
|
|
||||||
### Changing Build Frequency
|
|
||||||
|
|
||||||
Edit the `schedule` trigger:
|
|
||||||
```yaml
|
|
||||||
schedule:
|
|
||||||
# Build every 6 hours
|
|
||||||
- cron: '0 */6 * * *'
|
|
||||||
```
|
|
||||||
|
|
||||||
### Disabling Automatic Builds
|
|
||||||
|
|
||||||
Comment out unwanted triggers:
|
|
||||||
```yaml
|
|
||||||
# on:
|
|
||||||
# push:
|
|
||||||
# branches:
|
|
||||||
# - '**'
|
|
||||||
# schedule:
|
|
||||||
# - cron: '0 2 * * *'
|
|
||||||
workflow_dispatch: # Keep manual trigger only
|
|
||||||
```
|
|
||||||
|
|
||||||
## Best Practices
|
|
||||||
|
|
||||||
### For Image Developers
|
|
||||||
1. **Test locally first**: Run `./bin/autobuild --image <name>` before pushing
|
|
||||||
2. **Small iterations**: Commit small changes to debug issues faster
|
|
||||||
3. **Monitor logs**: Check Actions tab for build output
|
|
||||||
4. **Use branches**: Test in feature branches before merging to main
|
|
||||||
|
|
||||||
### For Repository Maintainers
|
|
||||||
1. **Review PRs carefully**: Malicious setup.sh could compromise runners
|
|
||||||
2. **Monitor disk usage**: Adjust retention policies if needed
|
|
||||||
3. **Update base images**: Keep RaspiOS and Debian URLs current
|
|
||||||
4. **Clean old releases**: Remove outdated releases periodically
|
|
||||||
|
|
||||||
## Security Considerations
|
## Security Considerations
|
||||||
|
|
||||||
### Runner Security
|
### GitHub Token
|
||||||
- Workflows run in isolated GitHub-hosted runners
|
|
||||||
- Each job runs in a fresh VM
|
|
||||||
- Artifacts are sandboxed
|
|
||||||
- No access to repository secrets (unless explicitly added)
|
|
||||||
|
|
||||||
### Image Security
|
The workflow uses `GITHUB_TOKEN` for:
|
||||||
- Setup scripts run with root privileges in QEMU
|
- Creating releases
|
||||||
- Only trusted contributors should modify setup.sh
|
- Uploading assets
|
||||||
- Review all package installations
|
- Downloading artifacts
|
||||||
- Avoid hardcoded credentials
|
|
||||||
|
|
||||||
### Release Security
|
**Permissions required**:
|
||||||
- Releases are public by default
|
```yaml
|
||||||
- Verify checksums before flashing images
|
permissions:
|
||||||
- Use HTTPS for all downloads
|
contents: write # Create releases, upload assets
|
||||||
|
actions: read # Download artifacts
|
||||||
|
```
|
||||||
|
|
||||||
## Performance Metrics
|
**Token is automatically provided by GitHub Actions** - no manual setup needed.
|
||||||
|
|
||||||
### Resource Usage (per image)
|
### Secrets
|
||||||
- **CPU**: 4 cores (QEMU)
|
|
||||||
- **RAM**: 8GB (QEMU)
|
|
||||||
- **Disk**: ~20GB peak usage
|
|
||||||
- **Network**: ~2GB download (base images)
|
|
||||||
|
|
||||||
### GitHub Actions Limits
|
No secrets required for basic builds. Optional secrets:
|
||||||
- **Concurrent jobs**: 20 (free tier)
|
|
||||||
- **Job timeout**: 6 hours (we set 60 min for QEMU)
|
|
||||||
- **Artifact size**: 10GB per file
|
|
||||||
- **Total storage**: 500MB artifacts (free tier)
|
|
||||||
|
|
||||||
## Future Improvements
|
**CUSTOM_REGISTRY_TOKEN**:
|
||||||
|
- For private Docker registries
|
||||||
|
- Add in Settings → Secrets → Actions
|
||||||
|
|
||||||
Potential enhancements:
|
**Usage**:
|
||||||
|
```yaml
|
||||||
|
env:
|
||||||
|
REGISTRY_TOKEN: ${{ secrets.CUSTOM_REGISTRY_TOKEN }}
|
||||||
|
```
|
||||||
|
|
||||||
1. **Caching base images**: Store RaspiOS/Debian images in GitHub cache
|
---
|
||||||
2. **Incremental builds**: Only rebuild changed images
|
|
||||||
3. **Build matrix**: Test multiple Debian versions
|
|
||||||
4. **Checksum verification**: Add SHA256 checksums to releases
|
|
||||||
5. **Build badges**: Display build status in README
|
|
||||||
6. **Multi-architecture**: Support x86_64 images for testing
|
|
||||||
|
|
||||||
## Related Documentation
|
## Cost Optimization
|
||||||
|
|
||||||
- **[Home](Home)**: Project overview
|
### Free Tier Limits
|
||||||
- **[Main README](../README.md)**: Complete documentation
|
|
||||||
- **[CLAUDE.md](../CLAUDE.md)**: Technical architecture details
|
**GitHub Actions free tier** (public repos):
|
||||||
- **GitHub Actions**: [Official documentation](https://docs.github.com/en/actions)
|
- Unlimited minutes
|
||||||
|
- 2 cores, 7GB RAM runners
|
||||||
|
- No artifact storage cost
|
||||||
|
|
||||||
|
**GitHub Actions free tier** (private repos):
|
||||||
|
- 2000 minutes/month
|
||||||
|
- Same runner specs
|
||||||
|
- Artifact storage counted
|
||||||
|
|
||||||
|
### Reduce Build Time
|
||||||
|
|
||||||
|
**Cache base images**:
|
||||||
|
- Base images cached between builds
|
||||||
|
- Use `--skip-download` when possible
|
||||||
|
|
||||||
|
**Reduce image count**:
|
||||||
|
- Remove unused images from `.github/images.txt`
|
||||||
|
- Build only on significant changes
|
||||||
|
|
||||||
|
**Use smaller images**:
|
||||||
|
- Reduce `IMAGE_SIZE` in config
|
||||||
|
- Skip unnecessary services
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Advanced Workflows
|
||||||
|
|
||||||
|
### Build on Pull Request
|
||||||
|
|
||||||
|
Add PR trigger to test changes before merge:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
on:
|
||||||
|
pull_request:
|
||||||
|
branches:
|
||||||
|
- main
|
||||||
|
paths:
|
||||||
|
- 'images/**'
|
||||||
|
- 'bin/**'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Behavior**:
|
||||||
|
- Builds images without creating release
|
||||||
|
- Uploads artifacts for testing
|
||||||
|
- Blocks merge if build fails
|
||||||
|
|
||||||
|
### Multi-Architecture Builds
|
||||||
|
|
||||||
|
Add ARM64 runner for native builds (no QEMU emulation):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
jobs:
|
||||||
|
build:
|
||||||
|
runs-on: [self-hosted, linux, arm64]
|
||||||
|
```
|
||||||
|
|
||||||
|
**Benefits**:
|
||||||
|
- Faster builds (no emulation overhead)
|
||||||
|
- Lower resource usage
|
||||||
|
|
||||||
|
**Requirements**:
|
||||||
|
- ARM64 GitHub runner (Raspberry Pi 5, AWS Graviton, etc.)
|
||||||
|
|
||||||
|
### Notification on Failure
|
||||||
|
|
||||||
|
Send notifications via email/Slack/Discord:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- name: Notify on failure
|
||||||
|
if: failure()
|
||||||
|
uses: actions/notify@v1
|
||||||
|
with:
|
||||||
|
webhook-url: ${{ secrets.SLACK_WEBHOOK }}
|
||||||
|
message: "Build failed: ${{ github.workflow }}"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Example: Custom Workflow
|
||||||
|
|
||||||
|
**Scenario**: Build only on weekends, upload to custom S3 bucket
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: Weekend Builds
|
||||||
|
|
||||||
|
on:
|
||||||
|
schedule:
|
||||||
|
- cron: '0 2 * * 6' # Saturday at 2 AM UTC
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
build:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Build images
|
||||||
|
run: ./bin/autobuild --all-images
|
||||||
|
|
||||||
|
- name: Upload to S3
|
||||||
|
uses: aws-actions/upload-s3@v1
|
||||||
|
with:
|
||||||
|
aws-access-key-id: ${{ secrets.AWS_KEY }}
|
||||||
|
aws-secret-access-key: ${{ secrets.AWS_SECRET }}
|
||||||
|
bucket: my-rpi-images
|
||||||
|
source: images/**/*.img.xz
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
- [Learn about hardware auto-detection](Hardware-Detection.md)
|
||||||
|
- [Create custom services](Services.md)
|
||||||
|
- [Troubleshoot build issues](Troubleshooting.md)
|
||||||
331
Hardware-Detection.md
Normal file
331
Hardware-Detection.md
Normal file
|
|
@ -0,0 +1,331 @@
|
||||||
|
# Hardware Auto-Detection
|
||||||
|
|
||||||
|
The build system automatically detects and configures hardware on first boot.
|
||||||
|
|
||||||
|
## Supported Hardware
|
||||||
|
|
||||||
|
### Network Interfaces
|
||||||
|
|
||||||
|
**Ethernet (eth0, eth1)**
|
||||||
|
|
||||||
|
**Detection**:
|
||||||
|
- Scans for eth0 and eth1 during first boot
|
||||||
|
- Executed by: `services/base/first-boot/init.sh`
|
||||||
|
|
||||||
|
**Configuration**:
|
||||||
|
|
||||||
|
| Configuration | Bridge Setup | IP Assignment |
|
||||||
|
|---------------|--------------|---------------|
|
||||||
|
| Single NIC (eth0 only) | br-wan on eth0 | DHCP client |
|
||||||
|
| Dual NIC (eth0 + eth1) | br-wan on eth0<br>br-lan on eth1 | br-wan: DHCP client<br>br-lan: 192.168.10.254/24 |
|
||||||
|
|
||||||
|
**br-lan features** (when created):
|
||||||
|
- DHCP server (192.168.10.1-100)
|
||||||
|
- DNS forwarder (dnsmasq)
|
||||||
|
- NAT to br-wan
|
||||||
|
- Gateway for internal services
|
||||||
|
|
||||||
|
**Example detection log**:
|
||||||
|
```bash
|
||||||
|
# Check first-boot logs
|
||||||
|
sudo journalctl -u services-first-boot | grep "Network"
|
||||||
|
|
||||||
|
# Output:
|
||||||
|
# Detected eth0 and eth1
|
||||||
|
# Created br-wan on eth0 (WAN)
|
||||||
|
# Created br-lan on eth1 (LAN)
|
||||||
|
# DHCP server enabled on br-lan
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### WiFi Adapters
|
||||||
|
|
||||||
|
**Wireless interfaces (wlan0, wlan1)**
|
||||||
|
|
||||||
|
**Detection**:
|
||||||
|
- Scans for wlan0 and wlan1 during first boot
|
||||||
|
- Executed by: `services/hotspot/first-boot/init.sh`
|
||||||
|
|
||||||
|
**Configuration**:
|
||||||
|
|
||||||
|
| Configuration | Access Point Setup |
|
||||||
|
|---------------|-------------------|
|
||||||
|
| No WiFi | Hotspot service disabled |
|
||||||
|
| Single WiFi (wlan0) | 5GHz AP on wlan0 |
|
||||||
|
| Dual WiFi (wlan0 + wlan1) | 2.4GHz AP on wlan0<br>5GHz AP on wlan1 |
|
||||||
|
|
||||||
|
**Access Point defaults**:
|
||||||
|
- SSID: `RaspberryPi-5G` or `RaspberryPi-2.4G`
|
||||||
|
- Password: `raspberry`
|
||||||
|
- Bridge: br-lan (if exists) or br-wan
|
||||||
|
|
||||||
|
**Bridge selection logic**:
|
||||||
|
```bash
|
||||||
|
if br-lan exists:
|
||||||
|
attach WiFi AP to br-lan (LAN network)
|
||||||
|
else:
|
||||||
|
attach WiFi AP to br-wan (WAN network)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example detection log**:
|
||||||
|
```bash
|
||||||
|
sudo journalctl -u services-first-boot | grep "WiFi"
|
||||||
|
|
||||||
|
# Output:
|
||||||
|
# Detected wlan0 and wlan1
|
||||||
|
# Configured 2.4GHz AP on wlan0 (RaspberryPi-2.4G)
|
||||||
|
# Configured 5GHz AP on wlan1 (RaspberryPi-5G)
|
||||||
|
# Attached to br-lan
|
||||||
|
```
|
||||||
|
|
||||||
|
**Customization**:
|
||||||
|
Edit `/etc/setupfiles/hostapd-5ghz.conf` or `hostapd-2.4ghz.conf`:
|
||||||
|
```ini
|
||||||
|
ssid=MyCustomSSID
|
||||||
|
wpa_passphrase=MySecurePassword
|
||||||
|
```
|
||||||
|
|
||||||
|
Then restart:
|
||||||
|
```bash
|
||||||
|
sudo systemctl restart hostapd-5ghz
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### USB Zigbee Coordinators
|
||||||
|
|
||||||
|
**USB serial devices for Zigbee/Z-Wave**
|
||||||
|
|
||||||
|
**Detection**:
|
||||||
|
- Scans `/dev/ttyUSB*` and `/dev/ttyACM*`
|
||||||
|
- Matches vendor IDs for known Zigbee coordinators
|
||||||
|
- Executed by: `services/haos/first-boot/init.sh`
|
||||||
|
|
||||||
|
**Supported vendors**:
|
||||||
|
- dresden elektronik (ConBee/ConBee II)
|
||||||
|
- Texas Instruments (CC2652, CC1352)
|
||||||
|
- Silicon Labs (EFR32)
|
||||||
|
- ITead (Sonoff Zigbee dongles)
|
||||||
|
- FTDI-based coordinators
|
||||||
|
|
||||||
|
**Configuration**:
|
||||||
|
- USB device passed through to Home Assistant VM
|
||||||
|
- Uses vendor/product ID matching (survives USB port changes)
|
||||||
|
- Non-required passthrough (VM starts even if dongle unplugged)
|
||||||
|
|
||||||
|
**Example detection log**:
|
||||||
|
```bash
|
||||||
|
sudo journalctl -u services-first-boot | grep -i zigbee
|
||||||
|
|
||||||
|
# Output:
|
||||||
|
# Detected Zigbee coordinator: ConBee II
|
||||||
|
# USB Vendor: 1cf1, Product: 0030
|
||||||
|
# Passed through to Home Assistant VM (haos)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Manual passthrough**:
|
||||||
|
```bash
|
||||||
|
# List USB devices
|
||||||
|
lsusb
|
||||||
|
|
||||||
|
# Pass through manually
|
||||||
|
incus config device add haos my-zigbee usb \
|
||||||
|
vendorid=1cf1 \
|
||||||
|
productid=0030
|
||||||
|
```
|
||||||
|
|
||||||
|
**Verification**:
|
||||||
|
```bash
|
||||||
|
# Check HAOS VM devices
|
||||||
|
incus config show haos
|
||||||
|
|
||||||
|
# Should show:
|
||||||
|
# devices:
|
||||||
|
# zigbee-dongle:
|
||||||
|
# productid: "0030"
|
||||||
|
# type: usb
|
||||||
|
# vendorid: 1cf1
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Detection Sequence
|
||||||
|
|
||||||
|
### First Boot Timeline
|
||||||
|
|
||||||
|
**1. rpi-first-boot.service** (runs once, before network)
|
||||||
|
- Expand root partition to fill SD card
|
||||||
|
- Set persistent network interface names (eth0, eth1, wlan0, wlan1)
|
||||||
|
- Reboot
|
||||||
|
|
||||||
|
**2. After reboot: services-first-boot.service** (runs once)
|
||||||
|
|
||||||
|
**Stage 1: Base network setup** (`base/first-boot/init.sh`)
|
||||||
|
- Detect eth0, eth1
|
||||||
|
- Create br-wan (always)
|
||||||
|
- Create br-lan (if eth1 exists)
|
||||||
|
- Configure DHCP server on br-lan
|
||||||
|
- Enable NAT
|
||||||
|
|
||||||
|
**Stage 2: Service initialization** (service-specific init.sh)
|
||||||
|
- qemu: Configure Incus networks
|
||||||
|
- docker: Start Portainer, Watchtower
|
||||||
|
- haos: Download HAOS image, create VM, detect Zigbee dongles
|
||||||
|
- openwrt: Download OpenWrt image, create container
|
||||||
|
- hotspot: Detect WiFi adapters, configure hostapd
|
||||||
|
|
||||||
|
**3. Normal boot**
|
||||||
|
- All services running
|
||||||
|
- Hardware configured
|
||||||
|
- Ready for use
|
||||||
|
|
||||||
|
**Total first-boot time**: 3-10 minutes (depends on services, network speed)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Manual Hardware Configuration
|
||||||
|
|
||||||
|
### Add Network Bridge Manually
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Create bridge
|
||||||
|
sudo nmcli con add type bridge con-name br-custom ifname br-custom
|
||||||
|
|
||||||
|
# Add interface to bridge
|
||||||
|
sudo nmcli con add type bridge-slave con-name eth2 ifname eth2 master br-custom
|
||||||
|
|
||||||
|
# Set IP
|
||||||
|
sudo nmcli con mod br-custom ipv4.addresses 192.168.20.1/24
|
||||||
|
sudo nmcli con mod br-custom ipv4.method manual
|
||||||
|
|
||||||
|
# Bring up
|
||||||
|
sudo nmcli con up br-custom
|
||||||
|
```
|
||||||
|
|
||||||
|
### Add WiFi AP Manually
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Create hostapd config
|
||||||
|
sudo tee /etc/hostapd/hostapd-custom.conf <<EOF
|
||||||
|
interface=wlan2
|
||||||
|
driver=nl80211
|
||||||
|
ssid=MyCustomAP
|
||||||
|
hw_mode=a
|
||||||
|
channel=36
|
||||||
|
ieee80211n=1
|
||||||
|
ieee80211ac=1
|
||||||
|
wmm_enabled=1
|
||||||
|
wpa=2
|
||||||
|
wpa_passphrase=MyPassword
|
||||||
|
wpa_key_mgmt=WPA-PSK
|
||||||
|
rsn_pairwise=CCMP
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# Create systemd service
|
||||||
|
sudo tee /etc/systemd/system/hostapd-custom.service <<EOF
|
||||||
|
[Unit]
|
||||||
|
Description=Custom WiFi AP
|
||||||
|
After=network.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
ExecStart=/usr/sbin/hostapd /etc/hostapd/hostapd-custom.conf
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# Enable and start
|
||||||
|
sudo systemctl enable hostapd-custom
|
||||||
|
sudo systemctl start hostapd-custom
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pass USB Device to VM Manually
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# List USB devices
|
||||||
|
lsusb
|
||||||
|
# Example output: Bus 001 Device 003: ID 10c4:ea60 Silicon Labs CP210x
|
||||||
|
|
||||||
|
# Pass through
|
||||||
|
incus config device add haos my-device usb \
|
||||||
|
vendorid=10c4 \
|
||||||
|
productid=ea60
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Hardware Detection Logs
|
||||||
|
|
||||||
|
### View Detection Logs
|
||||||
|
|
||||||
|
**First-boot service**:
|
||||||
|
```bash
|
||||||
|
sudo journalctl -u services-first-boot
|
||||||
|
```
|
||||||
|
|
||||||
|
**Specific hardware type**:
|
||||||
|
```bash
|
||||||
|
# Network detection
|
||||||
|
sudo journalctl -u services-first-boot | grep -i "network\|eth\|bridge"
|
||||||
|
|
||||||
|
# WiFi detection
|
||||||
|
sudo journalctl -u services-first-boot | grep -i "wifi\|wlan\|hostapd"
|
||||||
|
|
||||||
|
# USB detection
|
||||||
|
sudo journalctl -u services-first-boot | grep -i "usb\|zigbee\|dongle"
|
||||||
|
```
|
||||||
|
|
||||||
|
**All hardware events**:
|
||||||
|
```bash
|
||||||
|
sudo journalctl -u services-first-boot --no-pager
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Disabling Auto-Detection
|
||||||
|
|
||||||
|
### Disable Network Bridge Auto-Creation
|
||||||
|
|
||||||
|
Edit `/etc/setupfiles/rpi-first-boot.sh`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Comment out bridge creation
|
||||||
|
# create_bridges
|
||||||
|
```
|
||||||
|
|
||||||
|
Then delete and recreate first-boot service to apply changes (before first boot).
|
||||||
|
|
||||||
|
### Disable WiFi Hotspot
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Disable hostapd services
|
||||||
|
sudo systemctl disable hostapd-5ghz
|
||||||
|
sudo systemctl disable hostapd-2.4ghz
|
||||||
|
sudo systemctl stop hostapd-5ghz
|
||||||
|
sudo systemctl stop hostapd-2.4ghz
|
||||||
|
```
|
||||||
|
|
||||||
|
### Disable USB Passthrough
|
||||||
|
|
||||||
|
Edit `services/haos/first-boot/init.sh` and comment out the USB detection section before building.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Future Hardware Detection
|
||||||
|
|
||||||
|
**Planned** (in development):
|
||||||
|
- Storage devices (auto-mount USB drives)
|
||||||
|
- Audio devices (auto-configure ALSA/PulseAudio)
|
||||||
|
- Camera modules (CSI/USB cameras)
|
||||||
|
- GPIO devices (I2C, SPI peripherals)
|
||||||
|
|
||||||
|
**Contributions welcome!** See [CONTRIBUTING.md](../CONTRIBUTING.md).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
- [Learn about creating custom services](Services.md)
|
||||||
|
- [Troubleshoot hardware issues](Troubleshooting.md)
|
||||||
|
- [Configure GitHub Actions CI/CD](GitHub-Actions.md)
|
||||||
274
Home.md
274
Home.md
|
|
@ -1,225 +1,109 @@
|
||||||
# RPI-Dev - Hybrid Raspberry Pi Images Builder
|
# Raspberry Pi Image Builder - Documentation
|
||||||
|
|
||||||
Welcome to the RPI-Dev wiki! This project automates the creation of hybrid Raspberry Pi images combining Raspberry Pi OS hardware support with custom Debian ARM64 root filesystems.
|
Welcome to the Raspberry Pi Image Builder documentation! This system makes it easy to build, customize, and maintain production-ready Raspberry Pi images.
|
||||||
|
|
||||||
## Overview
|
## What is This?
|
||||||
|
|
||||||
RPI-Dev is an automated build system that creates ready-to-use Raspberry Pi images with full hardware compatibility and custom software configurations. The project addresses a common challenge: running pure Debian on Raspberry Pi while maintaining complete hardware support for critical components like the RP1 chip (Ethernet, USB, GPIO).
|
A modular automated build system that creates custom Raspberry Pi OS images by:
|
||||||
|
- Combining Raspberry Pi OS firmware with Debian ARM64
|
||||||
|
- Installing packages via QEMU ARM64 emulation
|
||||||
|
- Composing images from reusable service modules
|
||||||
|
- Automating builds via GitHub Actions CI/CD
|
||||||
|
- Auto-detecting and configuring hardware
|
||||||
|
|
||||||
### Key Features
|
## Quick Links
|
||||||
|
|
||||||
- **Automated Builds**: GitHub Actions automatically build all images daily and on every push
|
### Getting Started
|
||||||
- **Full Hardware Support**: Maintains Raspberry Pi OS kernel and firmware for complete hardware compatibility
|
- [Installation & First Build](Getting-Started.md)
|
||||||
- **Pure Debian Userspace**: Uses official Debian ARM64 root filesystems for a clean, standard environment
|
- [Flashing Images to SD Cards](Getting-Started/#flashing-to-sd-card)
|
||||||
- **Multi-Image Support**: Easy creation and management of multiple image configurations
|
- [First Boot & Login](Getting-Started/#first-boot)
|
||||||
- **Automatic Updates**: Built-in APT configuration enables safe kernel and firmware updates
|
|
||||||
- **QEMU Development**: Test and develop images in QEMU before flashing to hardware
|
|
||||||
- **CI/CD Integration**: Automated releases via GitHub Actions with compressed, ready-to-flash images
|
|
||||||
|
|
||||||
## Why This Project?
|
### Core Concepts
|
||||||
|
- [Architecture Overview](Architecture.md)
|
||||||
|
- [Build System](Build-System.md)
|
||||||
|
- [Service System](Services.md)
|
||||||
|
|
||||||
The Raspberry Pi uses specialized hardware that requires kernel drivers not yet available in mainline Linux:
|
### Advanced Topics
|
||||||
|
- [Creating Custom Images](Custom-Images.md)
|
||||||
|
- [Creating Custom Services](Custom-Services.md)
|
||||||
|
- [GitHub Actions CI/CD](GitHub-Actions.md)
|
||||||
|
- [Hardware Auto-Detection](Hardware-Detection.md)
|
||||||
|
|
||||||
- **RP1 Southbridge Chip**: Handles Ethernet, USB 2.0/3.0, GPIO, and other critical I/O
|
### Reference
|
||||||
- **VideoCore GPU**: Provides hardware acceleration and display output
|
- [Available Images](Available-Images.md)
|
||||||
- **WiFi/Bluetooth**: Requires specific firmware blobs
|
- [Available Services](Available-Services.md)
|
||||||
|
- [Command Reference](Command-Reference.md)
|
||||||
|
- [Configuration Reference](Configuration-Reference.md)
|
||||||
|
|
||||||
Using a standard Debian kernel results in non-functional hardware. This project solves that by:
|
### Help
|
||||||
|
- [FAQ](FAQ.md)
|
||||||
|
- [Troubleshooting](Troubleshooting.md)
|
||||||
|
- [GitHub Issues](https://github.com/Pikatsuto/raspberry-builds/issues)
|
||||||
|
|
||||||
1. Using Raspberry Pi OS boot partition and kernel
|
## Why Use This?
|
||||||
2. Replacing the root filesystem with pure Debian
|
|
||||||
3. Installing RaspiOS packages via APT for automatic updates
|
|
||||||
4. Preserving hardware compatibility while gaining Debian's advantages
|
|
||||||
|
|
||||||
## How It Works
|
### Easy to Build
|
||||||
|
|
||||||
### Architecture
|
|
||||||
|
|
||||||
The project uses a **partition-level merge approach**:
|
|
||||||
|
|
||||||
1. **Boot Partition (FAT32)**: Retained from Raspberry Pi OS for firmware compatibility
|
|
||||||
2. **Root Partition (ext4)**: Replaced with custom Debian ARM64 rootfs
|
|
||||||
3. **RaspiOS Packages**: Installed via APT with repository pinning:
|
|
||||||
- `raspberrypi-kernel` - Kernel, initramfs, and modules (including RP1 drivers)
|
|
||||||
- `raspberrypi-bootloader` - Bootloader and firmware files
|
|
||||||
- `libraspberrypi*` - VideoCore libraries
|
|
||||||
- `firmware-brcm80211` - WiFi/Bluetooth firmware
|
|
||||||
4. **APT Configuration**: RaspiOS repository with pinning enables safe `apt upgrade`
|
|
||||||
|
|
||||||
### Build Process
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ Stage 1: Download Base Images │
|
|
||||||
│ ├─ Raspberry Pi OS Lite ARM64 (.img.xz) │
|
|
||||||
│ └─ Debian Generic Cloud ARM64 (.raw) │
|
|
||||||
└─────────────────────────────────────────────────────────────┘
|
|
||||||
↓
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ Stage 2: QEMU Setup (Native ARM64 Execution) │
|
|
||||||
│ ├─ Create setup.iso from image/setup.sh + setupfiles/ │
|
|
||||||
│ ├─ Launch QEMU ARM64 with Debian + setup.iso │
|
|
||||||
│ ├─ Execute setup.sh (install packages, configure system) │
|
|
||||||
│ ├─ Install RaspiOS kernel/firmware via APT │
|
|
||||||
│ └─ Automatic shutdown when complete │
|
|
||||||
└─────────────────────────────────────────────────────────────┘
|
|
||||||
↓
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ Stage 3: Hybrid Image Creation │
|
|
||||||
│ ├─ Copy RaspiOS boot partition │
|
|
||||||
│ ├─ Replace root partition with configured Debian │
|
|
||||||
│ ├─ Preserve /etc/fstab from RaspiOS │
|
|
||||||
│ └─ Resize to final image size │
|
|
||||||
└─────────────────────────────────────────────────────────────┘
|
|
||||||
↓
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ Stage 4: Compression │
|
|
||||||
│ ├─ Shrink unused space with PiShrink │
|
|
||||||
│ └─ Compress to .img.xz (ready to flash) │
|
|
||||||
└─────────────────────────────────────────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
## Quick Start
|
|
||||||
|
|
||||||
### For Users: Download Pre-Built Images
|
|
||||||
|
|
||||||
The easiest way to use this project is to download pre-built images from [GitHub Releases](../../releases):
|
|
||||||
|
|
||||||
1. Go to the [Releases](../../releases) page
|
|
||||||
2. Download the `.img.xz` file for your desired image
|
|
||||||
3. Flash to SD card or SSD:
|
|
||||||
```bash
|
```bash
|
||||||
xz -dc rpi-*.img.xz | sudo dd of=/dev/sdX bs=4M status=progress conv=fsync
|
# One command to build a custom image
|
||||||
sync
|
./bin/autobuild --image debian/qemu+docker+haos
|
||||||
```
|
```
|
||||||
|
|
||||||
### For Developers: Fork and Customize
|
### Easy to Maintain
|
||||||
|
- Version control your image configurations
|
||||||
|
- Automated builds via GitHub Actions
|
||||||
|
- Safe `apt upgrade` - kernel and firmware update automatically
|
||||||
|
|
||||||
Want to create your own custom images? It's easy:
|
### Easy to Share
|
||||||
|
- Share image configurations with your team
|
||||||
|
- Reproducible builds across environments
|
||||||
|
- Pre-built images in GitHub Releases
|
||||||
|
|
||||||
1. **Fork this repository** on GitHub
|
### Easy to Customize
|
||||||
2. **Clone your fork**:
|
- Modular service system
|
||||||
```bash
|
- Add your own services
|
||||||
git clone https://github.com/Pikatsuto/raspberry-builds.git
|
- Full control over packages and configuration
|
||||||
cd rpi-dev
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Create a new image configuration**:
|
## Key Features
|
||||||
```bash
|
|
||||||
# Copy an existing image as a template
|
|
||||||
cp -r images/raspivirt-incus images/my-custom-image
|
|
||||||
```
|
|
||||||
|
|
||||||
4. **Edit the configuration** (`images/my-custom-image/config.sh`):
|
- **Modular Services** - Compose images from qemu, docker, haos, openwrt, hotspot
|
||||||
```bash
|
- **Auto-Detection** - Network interfaces, WiFi, Zigbee dongles
|
||||||
OUTPUT_IMAGE="rpi-my-custom-image.img"
|
- **CI/CD Ready** - GitHub Actions for automated builds
|
||||||
IMAGE_SIZE="8G"
|
- **Safe Updates** - RaspiOS kernel via APT with pinning
|
||||||
QEMU_RAM="8G"
|
- **Full Hardware Support** - RP1 drivers, WiFi, Bluetooth, GPIO
|
||||||
QEMU_CPUS="4"
|
|
||||||
DESCRIPTION="My custom Raspberry Pi image"
|
|
||||||
```
|
|
||||||
|
|
||||||
5. **Customize the setup script** (`images/my-custom-image/setup.sh`):
|
## What Can You Build?
|
||||||
- Add package installations
|
|
||||||
- Configure services
|
|
||||||
- Apply custom settings
|
|
||||||
|
|
||||||
6. **Add custom files** to `images/my-custom-image/setupfiles/`:
|
- **Home Server** - Docker + Portainer + Watchtower
|
||||||
- Configuration files
|
- **IoT Gateway** - Home Assistant + Zigbee auto-detection
|
||||||
- Scripts
|
- **Network Router** - OpenWrt + WiFi hotspot + dual NIC
|
||||||
- SSH keys
|
- **Virtualization Platform** - Incus containers and VMs
|
||||||
|
- **Custom Appliance** - Your own service modules
|
||||||
|
|
||||||
7. **Commit and push**:
|
## Architecture at a Glance
|
||||||
```bash
|
|
||||||
git add images/my-custom-image/
|
|
||||||
git commit -m "Add my custom image"
|
|
||||||
git push origin main
|
|
||||||
```
|
|
||||||
|
|
||||||
8. **GitHub Actions automatically builds your image!**
|
|
||||||
- Check the [Actions](../../actions) tab for build progress
|
|
||||||
- Download from [Releases](../../releases) when complete
|
|
||||||
|
|
||||||
### Building Locally
|
|
||||||
|
|
||||||
If you prefer to build locally:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Install dependencies
|
|
||||||
sudo apt install -y qemu-system-aarch64 qemu-utils parted \
|
|
||||||
e2fsprogs dosfstools rsync xz-utils genisoimage
|
|
||||||
|
|
||||||
# Build a specific image
|
|
||||||
./bin/autobuild --image my-custom-image
|
|
||||||
|
|
||||||
# Or build all images
|
|
||||||
./bin/autobuild --all-images
|
|
||||||
```
|
|
||||||
|
|
||||||
## Image Directory Structure
|
|
||||||
|
|
||||||
Each image is defined in `images/<image-name>/`:
|
|
||||||
|
|
||||||
```
|
```
|
||||||
images/my-image/
|
RaspiOS Boot Partition (FAT32) Debian Rootfs (ext4)
|
||||||
├── config.sh # Build configuration
|
├── bootloader ├── /bin, /usr, /etc (Debian)
|
||||||
│ # - OUTPUT_IMAGE: Final image filename
|
├── kernel ├── RaspiOS kernel packages (via APT)
|
||||||
│ # - IMAGE_SIZE: Final image size (e.g., "8G")
|
├── firmware ├── Your services
|
||||||
│ # - QEMU_RAM: RAM for QEMU (e.g., "8G")
|
└── config.txt └── Auto-hardware detection
|
||||||
│ # - QEMU_CPUS: CPU cores for QEMU (e.g., "4")
|
↓ ↓
|
||||||
│ # - DESCRIPTION: Image description
|
└────────── Merged Image ───────────┘
|
||||||
│
|
|
||||||
├── setup.sh # Setup script executed in QEMU ARM64
|
|
||||||
│ # - Installs packages (RaspiOS kernel, software)
|
|
||||||
│ # - Configures system (users, services, etc.)
|
|
||||||
│ # - Runs in native ARM64 environment
|
|
||||||
│
|
|
||||||
├── setupfiles/ # Files copied to /root/setupfiles/ in image
|
|
||||||
│ # - Config files, scripts, certificates, etc.
|
|
||||||
│ # - Available to setup.sh during QEMU execution
|
|
||||||
│
|
|
||||||
└── cloudinit/ # Cloud-init configuration (for first boot)
|
|
||||||
├── user-data # User configuration (users, SSH keys, passwords)
|
|
||||||
├── meta-data # Instance metadata (hostname, instance-id)
|
|
||||||
└── seed.img # Auto-generated ISO (don't edit manually)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Available Images
|
## Support
|
||||||
|
|
||||||
This repository includes the following pre-configured images:
|
- Raspberry Pi 4/5 (ARM64)
|
||||||
|
- Debian 13 (Trixie) ARM64
|
||||||
|
- Additional distributions in development
|
||||||
|
|
||||||
- **[RaspiVirt-Incus](Image-RaspiVirt-Incus)**: Raspberry Pi virtualization platform with Incus container/VM manager
|
## Next Steps
|
||||||
- **[RaspiVirt-Incus+Docker](Image-RaspiVirt-Incus-Docker)**: RaspiVirt-Incus plus Docker for container orchestration
|
|
||||||
- **[RaspiVirt-Incus+HAOS](Image-RaspiVirt-Incus-HAOS)**: Home automation platform with automatic Home Assistant OS deployment, adaptive dual-bridge networking, and Zigbee dongle support
|
|
||||||
|
|
||||||
See individual image pages for detailed documentation.
|
1. [Install dependencies and build your first image](Getting-Started.md)
|
||||||
|
2. [Understand how the build system works](Architecture.md)
|
||||||
|
3. [Explore available services](Available-Services.md)
|
||||||
|
4. [Create your own custom image](Custom-Images.md)
|
||||||
|
|
||||||
## Automatic Updates
|
---
|
||||||
|
|
||||||
Images built with this system support automatic kernel and firmware updates via APT:
|
**Happy Building!**
|
||||||
|
|
||||||
```bash
|
|
||||||
# On your Raspberry Pi
|
|
||||||
sudo apt update
|
|
||||||
sudo apt upgrade -y
|
|
||||||
```
|
|
||||||
|
|
||||||
The APT pinning configuration ensures RaspiOS packages (kernel, firmware) are updated from the RaspiOS repository while all other packages use Debian repositories. This maintains hardware compatibility while keeping the system up-to-date.
|
|
||||||
|
|
||||||
## Documentation
|
|
||||||
|
|
||||||
- **[GitHub Actions Workflow](GitHub-Actions)**: Detailed documentation of the automated build system
|
|
||||||
- **[RaspiVirt-Incus Image](Image-RaspiVirt-Incus)**: Virtualization platform with Incus
|
|
||||||
- **[RaspiVirt-Incus+Docker Image](Image-RaspiVirt-Incus-Docker)**: Incus + Docker platform
|
|
||||||
- **[RaspiVirt-Incus+HAOS Image](Image-RaspiVirt-Incus-HAOS)**: Home Assistant OS platform with adaptive networking
|
|
||||||
- **[Main README](../README.md)**: Complete project documentation with manual build instructions
|
|
||||||
|
|
||||||
## Getting Help
|
|
||||||
|
|
||||||
- **Issues**: Report bugs or request features via [GitHub Issues](../../issues)
|
|
||||||
- **Discussions**: Ask questions in [GitHub Discussions](../../discussions)
|
|
||||||
- **Documentation**: Check the [README](../README.md) and [CLAUDE.md](../CLAUDE.md) for technical details
|
|
||||||
|
|
||||||
## License
|
|
||||||
|
|
||||||
This project is provided "as is" without warranty. Use at your own risk.
|
|
||||||
490
Services.md
Normal file
490
Services.md
Normal file
|
|
@ -0,0 +1,490 @@
|
||||||
|
# Services
|
||||||
|
|
||||||
|
Complete guide to available services and creating custom ones.
|
||||||
|
|
||||||
|
## Available Services
|
||||||
|
|
||||||
|
### base (Always Included)
|
||||||
|
|
||||||
|
**Purpose**: Essential system configuration for all images
|
||||||
|
|
||||||
|
**Packages**:
|
||||||
|
- `raspberrypi-kernel` - RaspiOS kernel with RP1 drivers
|
||||||
|
- `raspberrypi-bootloader` - Boot firmware
|
||||||
|
- `firmware-brcm80211` - WiFi/Bluetooth firmware
|
||||||
|
- `NetworkManager` - Network management
|
||||||
|
- `openssh-server` - SSH access
|
||||||
|
- Essential utilities
|
||||||
|
|
||||||
|
**Configuration**:
|
||||||
|
- RaspiOS APT repository + pinning
|
||||||
|
- Network bridges (br-wan, br-lan if dual NIC)
|
||||||
|
- SSH enabled
|
||||||
|
- MOTD with IP addresses
|
||||||
|
|
||||||
|
**First-boot actions**:
|
||||||
|
- Partition resize to fill SD card
|
||||||
|
- Network interface name persistence
|
||||||
|
- Bridge configuration based on detected NICs
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### qemu
|
||||||
|
|
||||||
|
**Purpose**: Incus container/VM platform with KVM acceleration
|
||||||
|
|
||||||
|
**Dependencies**: None
|
||||||
|
|
||||||
|
**Packages**:
|
||||||
|
- `qemu-kvm` - KVM virtualization
|
||||||
|
- `incus` - Container/VM manager
|
||||||
|
- `incus-tools` - Management utilities
|
||||||
|
|
||||||
|
**Configuration**:
|
||||||
|
- Incus initialization with default profile
|
||||||
|
- Network bridges: br-wan (WAN), br-lan (LAN if dual NIC)
|
||||||
|
- Storage pool: default (dir backend)
|
||||||
|
|
||||||
|
**First-boot actions**:
|
||||||
|
- Create br-lan and br-wan bridges (if not exist)
|
||||||
|
- Configure Incus network integration
|
||||||
|
- Enable IP forwarding and NAT
|
||||||
|
|
||||||
|
**Use cases**:
|
||||||
|
- Run VMs (Home Assistant, OpenWrt)
|
||||||
|
- Create containers for isolated services
|
||||||
|
- Development environments
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### docker
|
||||||
|
|
||||||
|
**Purpose**: Docker Engine with Portainer and Watchtower
|
||||||
|
|
||||||
|
**Dependencies**: None
|
||||||
|
|
||||||
|
**Packages**:
|
||||||
|
- `docker-ce` - Docker Engine
|
||||||
|
- `docker-ce-cli` - Docker CLI
|
||||||
|
- `containerd.io` - Container runtime
|
||||||
|
|
||||||
|
**Configuration**:
|
||||||
|
- Docker installed from official repository
|
||||||
|
- User `pi` added to `docker` group
|
||||||
|
|
||||||
|
**First-boot actions**:
|
||||||
|
- Deploy Portainer (web UI for Docker)
|
||||||
|
- Deploy Watchtower (automatic container updates)
|
||||||
|
- Configure bridge networking
|
||||||
|
|
||||||
|
**Services deployed**:
|
||||||
|
- **Portainer**: `https://raspberry-ip:9443`
|
||||||
|
- **Watchtower**: Automatic updates for containers
|
||||||
|
|
||||||
|
**Use cases**:
|
||||||
|
- Run containerized applications
|
||||||
|
- Manage containers via web UI
|
||||||
|
- Auto-update containers
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### haos
|
||||||
|
|
||||||
|
**Purpose**: Home Assistant OS in an Incus VM
|
||||||
|
|
||||||
|
**Dependencies**: `qemu` (Incus required)
|
||||||
|
|
||||||
|
**Packages**: None (VM image downloaded at first boot)
|
||||||
|
|
||||||
|
**First-boot actions**:
|
||||||
|
- Download Home Assistant OS image (latest stable)
|
||||||
|
- Create Incus VM named `haos`
|
||||||
|
- Allocate resources (2 vCPUs, 4GB RAM, 24GB disk)
|
||||||
|
- Configure network bridge (br-lan or br-wan)
|
||||||
|
- Detect and pass through USB Zigbee coordinators
|
||||||
|
- Start VM
|
||||||
|
|
||||||
|
**Zigbee auto-detection**:
|
||||||
|
- Scans `/dev/ttyUSB*` and `/dev/ttyACM*`
|
||||||
|
- Detects ConBee, CC2652, Sonoff, Silicon Labs dongles
|
||||||
|
- Passes through via USB vendor/product ID
|
||||||
|
|
||||||
|
**Access**:
|
||||||
|
- Home Assistant: `http://raspberry-ip:8123`
|
||||||
|
- First boot setup takes 5-10 minutes
|
||||||
|
|
||||||
|
**Use cases**:
|
||||||
|
- Home automation platform
|
||||||
|
- Zigbee/Z-Wave integration
|
||||||
|
- IoT device management
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### openwrt
|
||||||
|
|
||||||
|
**Purpose**: OpenWrt router in an Incus container
|
||||||
|
|
||||||
|
**Dependencies**: `qemu` (Incus required)
|
||||||
|
|
||||||
|
**Packages**: None (container image downloaded at first boot)
|
||||||
|
|
||||||
|
**First-boot actions**:
|
||||||
|
- Download OpenWrt rootfs image
|
||||||
|
- Create Incus container named `openwrt`
|
||||||
|
- Allocate resources (1 vCPUs, 1GB RAM, 24GB disk)
|
||||||
|
- Configure network: eth0 → br-wan, eth1 → br-lan
|
||||||
|
- Set static IP: `192.168.10.1/24`
|
||||||
|
- Start container
|
||||||
|
|
||||||
|
**Network configuration**:
|
||||||
|
- WAN: br-wan (DHCP client)
|
||||||
|
- LAN: br-lan (192.168.10.1, DHCP server)
|
||||||
|
|
||||||
|
**Access**:
|
||||||
|
- LuCI web UI: `http://192.168.10.1`
|
||||||
|
- SSH: `ssh root@192.168.10.1`
|
||||||
|
|
||||||
|
**Use cases**:
|
||||||
|
- Advanced routing and firewall
|
||||||
|
- VPN server/client
|
||||||
|
- QoS and traffic shaping
|
||||||
|
- Network monitoring
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### hotspot
|
||||||
|
|
||||||
|
**Purpose**: WiFi access point (2.4GHz/5GHz)
|
||||||
|
|
||||||
|
**Dependencies**: None
|
||||||
|
|
||||||
|
**Packages**:
|
||||||
|
- `hostapd` - WiFi access point daemon
|
||||||
|
|
||||||
|
**Configuration files**:
|
||||||
|
- `hostapd-5ghz.conf` - 5GHz AP config
|
||||||
|
- `hostapd-2.4ghz.conf` - 2.4GHz AP config
|
||||||
|
|
||||||
|
**First-boot actions**:
|
||||||
|
- Detect WiFi interfaces (wlan0, wlan1)
|
||||||
|
- Configure hostapd based on detected adapters:
|
||||||
|
- Dual WiFi: 2.4GHz (wlan0) + 5GHz (wlan1)
|
||||||
|
- Single WiFi: 5GHz (wlan0)
|
||||||
|
- Attach to br-lan (if exists) or br-wan
|
||||||
|
- Disable NetworkManager management of WiFi
|
||||||
|
- Start hostapd services
|
||||||
|
|
||||||
|
**Default credentials**:
|
||||||
|
- SSID: `RaspberryPi-WIFI`
|
||||||
|
- Password: `raspberry`
|
||||||
|
|
||||||
|
**Use cases**:
|
||||||
|
- WiFi access for IoT devices
|
||||||
|
- Guest network
|
||||||
|
- Extend network coverage
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Service Combination Examples
|
||||||
|
|
||||||
|
### Minimal Development Platform
|
||||||
|
```bash
|
||||||
|
./bin/autobuild --image debian
|
||||||
|
```
|
||||||
|
- Base Debian with RaspiOS kernel
|
||||||
|
- SSH access
|
||||||
|
- Network bridges
|
||||||
|
|
||||||
|
### Docker Host
|
||||||
|
```bash
|
||||||
|
./bin/autobuild --image debian/docker
|
||||||
|
```
|
||||||
|
- Docker Engine
|
||||||
|
- Portainer web UI
|
||||||
|
- Watchtower auto-updates
|
||||||
|
|
||||||
|
### Virtualization Platform
|
||||||
|
```bash
|
||||||
|
./bin/autobuild --image debian/qemu
|
||||||
|
```
|
||||||
|
- Incus containers/VMs
|
||||||
|
- KVM acceleration
|
||||||
|
|
||||||
|
### Home Automation Gateway
|
||||||
|
```bash
|
||||||
|
./bin/autobuild --image debian/qemu+haos
|
||||||
|
```
|
||||||
|
- Home Assistant OS VM
|
||||||
|
- Zigbee dongle auto-detection
|
||||||
|
- Web UI on port 8123
|
||||||
|
|
||||||
|
### Network Router + WiFi AP
|
||||||
|
```bash
|
||||||
|
./bin/autobuild --image debian/qemu+openwrt+hotspot
|
||||||
|
```
|
||||||
|
- OpenWrt routing
|
||||||
|
- Dual-band WiFi hotspot
|
||||||
|
- Advanced firewall/QoS
|
||||||
|
|
||||||
|
### Full Stack
|
||||||
|
```bash
|
||||||
|
./bin/autobuild --image debian/qemu+docker+openwrt+hotspot+haos
|
||||||
|
```
|
||||||
|
- All services combined
|
||||||
|
- Ideal for Raspberry Pi 5 with 8GB RAM
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Creating Custom Services
|
||||||
|
|
||||||
|
### Service Directory Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
images/debian/services/myservice/
|
||||||
|
├── setup.sh # Required: package installation (runs in QEMU)
|
||||||
|
├── first-boot/
|
||||||
|
│ └── init.sh # Optional: runtime config (runs on first boot)
|
||||||
|
├── setupfiles/ # Optional: static files → /etc/setupfiles/
|
||||||
|
│ ├── config.conf
|
||||||
|
│ └── script.sh
|
||||||
|
├── depends.sh # Optional: dependencies
|
||||||
|
└── motd.sh # Optional: MOTD content
|
||||||
|
```
|
||||||
|
|
||||||
|
### setup.sh (Package Installation)
|
||||||
|
|
||||||
|
Executed in QEMU ARM64 during build.
|
||||||
|
|
||||||
|
**Template**:
|
||||||
|
```bash
|
||||||
|
#!/bin/bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Install packages
|
||||||
|
apt update
|
||||||
|
apt install -y package1 package2
|
||||||
|
|
||||||
|
# Configure system
|
||||||
|
systemctl enable my-service
|
||||||
|
|
||||||
|
# Copy files
|
||||||
|
cp /etc/setupfiles/config.conf /etc/my-service/
|
||||||
|
```
|
||||||
|
|
||||||
|
**Best practices**:
|
||||||
|
- Use `set -e` to exit on errors
|
||||||
|
- Install packages from Debian repositories when possible
|
||||||
|
- Add custom repositories if needed
|
||||||
|
- Enable systemd services that should start at boot
|
||||||
|
- Don't start services (no network in QEMU)
|
||||||
|
|
||||||
|
### first-boot/init.sh (Runtime Configuration)
|
||||||
|
|
||||||
|
Executed on first boot on Raspberry Pi.
|
||||||
|
|
||||||
|
**Template**:
|
||||||
|
```bash
|
||||||
|
#!/bin/bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Detect hardware
|
||||||
|
if ip link show eth1 >/dev/null 2>&1; then
|
||||||
|
BRIDGE="br-lan"
|
||||||
|
else
|
||||||
|
BRIDGE="br-wan"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Download resources
|
||||||
|
wget https://example.com/resource.tar.gz -O /tmp/resource.tar.gz
|
||||||
|
|
||||||
|
# Create containers/VMs
|
||||||
|
incus launch images:alpine mycontainer
|
||||||
|
|
||||||
|
# Configure services
|
||||||
|
systemctl start my-service
|
||||||
|
```
|
||||||
|
|
||||||
|
**Best practices**:
|
||||||
|
- Detect hardware and adapt configuration
|
||||||
|
- Download large files here (not in setup.sh)
|
||||||
|
- Create containers/VMs
|
||||||
|
- Start services
|
||||||
|
- Use `/etc/setupfiles/` for configuration files
|
||||||
|
|
||||||
|
### depends.sh (Dependencies)
|
||||||
|
|
||||||
|
Declare dependencies on other services.
|
||||||
|
|
||||||
|
**Example**:
|
||||||
|
```bash
|
||||||
|
# myservice depends on qemu
|
||||||
|
DEPENDS_ON="qemu"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Dependency resolution**:
|
||||||
|
- Dependencies are automatically included
|
||||||
|
- Build order is adjusted to satisfy dependencies
|
||||||
|
|
||||||
|
### motd.sh (Message of the Day)
|
||||||
|
|
||||||
|
Display service information on login.
|
||||||
|
|
||||||
|
**Example**:
|
||||||
|
```bash
|
||||||
|
cat <<'EOF'
|
||||||
|
My Service UI: https://raspberry-ip:8080
|
||||||
|
Username: admin
|
||||||
|
Password: (set on first access)
|
||||||
|
EOF
|
||||||
|
```
|
||||||
|
|
||||||
|
### setupfiles/ (Static Files)
|
||||||
|
|
||||||
|
Static configuration files copied to `/etc/setupfiles/`.
|
||||||
|
|
||||||
|
**Access in scripts**:
|
||||||
|
```bash
|
||||||
|
# In setup.sh or first-boot/init.sh
|
||||||
|
cp /etc/setupfiles/myconfig.conf /etc/my-service/
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Service Development Workflow
|
||||||
|
|
||||||
|
### 1. Create Service Directory
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mkdir -p images/debian/services/myservice/first-boot
|
||||||
|
mkdir -p images/debian/services/myservice/setupfiles
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Write setup.sh
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cat > images/debian/services/myservice/setup.sh <<'EOF'
|
||||||
|
#!/bin/bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
apt update
|
||||||
|
apt install -y nginx
|
||||||
|
|
||||||
|
systemctl enable nginx
|
||||||
|
EOF
|
||||||
|
|
||||||
|
chmod +x images/debian/services/myservice/setup.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Write first-boot/init.sh (Optional)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cat > images/debian/services/myservice/first-boot/init.sh <<'EOF'
|
||||||
|
#!/bin/bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Configure nginx with custom config
|
||||||
|
cp /etc/setupfiles/nginx.conf /etc/nginx/sites-available/default
|
||||||
|
systemctl restart nginx
|
||||||
|
EOF
|
||||||
|
|
||||||
|
chmod +x images/debian/services/myservice/first-boot/init.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Add Configuration Files (Optional)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cat > images/debian/services/myservice/setupfiles/nginx.conf <<'EOF'
|
||||||
|
server {
|
||||||
|
listen 80;
|
||||||
|
root /var/www/html;
|
||||||
|
index index.html;
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Add MOTD (Optional)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cat > images/debian/services/myservice/motd.sh <<'EOF'
|
||||||
|
cat <<'MOTD'
|
||||||
|
Nginx: http://raspberry-ip/
|
||||||
|
MOTD
|
||||||
|
EOF
|
||||||
|
|
||||||
|
chmod +x images/debian/services/myservice/motd.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6. Test Service
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build image with your service
|
||||||
|
./bin/autobuild --image debian/myservice
|
||||||
|
|
||||||
|
# Flash and boot
|
||||||
|
sudo dd if=debian-myservice.img of=/dev/sdX bs=4M status=progress
|
||||||
|
|
||||||
|
# Check logs on first boot
|
||||||
|
sudo journalctl -u services-first-boot
|
||||||
|
```
|
||||||
|
|
||||||
|
### 7. Debug Issues
|
||||||
|
|
||||||
|
**QEMU stage issues**:
|
||||||
|
```bash
|
||||||
|
# Check QEMU logs
|
||||||
|
cat images/debian-myservice/qemu-*.log
|
||||||
|
|
||||||
|
# Manually test in QEMU
|
||||||
|
./bin/autobuild --image debian/myservice --skip-compress
|
||||||
|
```
|
||||||
|
|
||||||
|
**First-boot stage issues**:
|
||||||
|
```bash
|
||||||
|
# On Raspberry Pi, check logs
|
||||||
|
sudo journalctl -u services-first-boot -f
|
||||||
|
|
||||||
|
# Check service status
|
||||||
|
sudo systemctl status myservice
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Service Best Practices
|
||||||
|
|
||||||
|
### Package Installation
|
||||||
|
- Install in `setup.sh`, not `first-boot/init.sh`
|
||||||
|
- Use official Debian repositories when possible
|
||||||
|
- Pin package versions if stability critical
|
||||||
|
|
||||||
|
### Resource Downloads
|
||||||
|
- Download large files in `first-boot/init.sh`
|
||||||
|
- Don't download in `setup.sh` (slows QEMU)
|
||||||
|
- Cache downloads in `/var/cache/` if appropriate
|
||||||
|
|
||||||
|
### Hardware Detection
|
||||||
|
- Always detect hardware in `first-boot/init.sh`
|
||||||
|
- Adapt configuration based on detection
|
||||||
|
- Fail gracefully if hardware missing
|
||||||
|
|
||||||
|
### Error Handling
|
||||||
|
- Use `set -e` to exit on errors
|
||||||
|
- Log errors to journald
|
||||||
|
- Provide helpful error messages
|
||||||
|
|
||||||
|
### Network Configuration
|
||||||
|
- Use existing bridges (br-wan, br-lan)
|
||||||
|
- Don't create new bridges unless necessary
|
||||||
|
- Check bridge existence before use
|
||||||
|
|
||||||
|
### Security
|
||||||
|
- Don't hardcode passwords in scripts
|
||||||
|
- Use strong default passwords (prompt user to change)
|
||||||
|
- Enable firewall if service exposed
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
- [Create a custom image with your service](Custom-Images.md)
|
||||||
|
- [Learn about hardware auto-detection](Hardware-Detection.md)
|
||||||
|
- [Set up CI/CD for automated builds](GitHub-Actions.md)
|
||||||
205
Troubleshooting.md
Normal file
205
Troubleshooting.md
Normal file
|
|
@ -0,0 +1,205 @@
|
||||||
|
# Troubleshooting
|
||||||
|
|
||||||
|
Common issues and solutions.
|
||||||
|
|
||||||
|
## Build Issues
|
||||||
|
|
||||||
|
### QEMU Won't Boot
|
||||||
|
**Symptoms**: QEMU hangs at boot
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
```bash
|
||||||
|
# Install UEFI firmware
|
||||||
|
sudo apt install qemu-efi-aarch64
|
||||||
|
|
||||||
|
# Increase timeout in config.sh
|
||||||
|
QEMU_TIMEOUT=3600
|
||||||
|
|
||||||
|
# Check logs
|
||||||
|
cat images/*/qemu-*.log
|
||||||
|
```
|
||||||
|
|
||||||
|
### QEMU Timeout
|
||||||
|
**Symptoms**: "QEMU timeout" error
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
- Increase `QEMU_TIMEOUT` in config.sh
|
||||||
|
- Increase `QEMU_RAM` and `QEMU_CPUS`
|
||||||
|
- Check network connectivity
|
||||||
|
- Review QEMU logs
|
||||||
|
|
||||||
|
### Merge Fails
|
||||||
|
**Symptoms**: Error during merge stage
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
```bash
|
||||||
|
# Check disk space
|
||||||
|
df -h
|
||||||
|
|
||||||
|
# Re-download base images
|
||||||
|
rm images/debian/*.img images/debian/*.raw
|
||||||
|
./bin/autobuild --image debian
|
||||||
|
|
||||||
|
# Check partition layout
|
||||||
|
fdisk -l raspios.img
|
||||||
|
fdisk -l debian.raw
|
||||||
|
```
|
||||||
|
|
||||||
|
### Out of Disk Space
|
||||||
|
**Symptoms**: "No space left on device"
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
- Free up space: `sudo apt clean`
|
||||||
|
- Use smaller `IMAGE_SIZE`
|
||||||
|
- Build fewer services
|
||||||
|
|
||||||
|
## Boot Issues
|
||||||
|
|
||||||
|
### Rainbow Screen
|
||||||
|
**Symptoms**: Raspberry Pi shows rainbow screen, won't boot
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
1. Verify image integrity: `sha256sum image.img.xz`
|
||||||
|
2. Re-flash SD card
|
||||||
|
3. Try different SD card
|
||||||
|
4. Check UART output
|
||||||
|
|
||||||
|
### Can't Login
|
||||||
|
**Symptoms**: Credentials don't work
|
||||||
|
|
||||||
|
**Default**: Username `pi`, password `raspberry`
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
- Check Caps Lock
|
||||||
|
- Wait for first-boot to complete (3-5 min)
|
||||||
|
- Reset password via SD card mount
|
||||||
|
|
||||||
|
### No Network
|
||||||
|
**Symptoms**: Can't get IP address
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
```bash
|
||||||
|
# Check interface
|
||||||
|
ip link show eth0
|
||||||
|
|
||||||
|
# Restart connection
|
||||||
|
sudo nmcli con up br-wan
|
||||||
|
|
||||||
|
# Check NetworkManager
|
||||||
|
sudo systemctl status NetworkManager
|
||||||
|
|
||||||
|
# Set static IP (see Configuration Reference)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Service Issues
|
||||||
|
|
||||||
|
### Service Not Starting
|
||||||
|
**Symptoms**: Docker/HAOS/OpenWrt not running
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
```bash
|
||||||
|
# Check first-boot logs
|
||||||
|
sudo journalctl -u services-first-boot
|
||||||
|
|
||||||
|
# Check service status
|
||||||
|
sudo systemctl status docker
|
||||||
|
incus list
|
||||||
|
|
||||||
|
# Check disk space
|
||||||
|
df -h
|
||||||
|
```
|
||||||
|
|
||||||
|
### Container/VM Won't Start
|
||||||
|
**Symptoms**: Incus container/VM fails to start
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
```bash
|
||||||
|
# Check Incus logs
|
||||||
|
incus info haos
|
||||||
|
incus info --show-log haos
|
||||||
|
|
||||||
|
# Check resources
|
||||||
|
free -h
|
||||||
|
df -h
|
||||||
|
|
||||||
|
# Restart Incus
|
||||||
|
sudo systemctl restart incus
|
||||||
|
```
|
||||||
|
|
||||||
|
## Hardware Issues
|
||||||
|
|
||||||
|
### WiFi Not Detected
|
||||||
|
**Symptoms**: wlan0 doesn't exist
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
```bash
|
||||||
|
# Check firmware
|
||||||
|
dmesg | grep brcmfmac
|
||||||
|
|
||||||
|
# Install firmware
|
||||||
|
sudo apt install firmware-brcm80211
|
||||||
|
|
||||||
|
# Reboot
|
||||||
|
sudo reboot
|
||||||
|
```
|
||||||
|
|
||||||
|
### Zigbee Not Passed to HAOS
|
||||||
|
**Symptoms**: Zigbee dongle not in Home Assistant
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
```bash
|
||||||
|
# Check detection
|
||||||
|
sudo journalctl -u services-first-boot | grep -i zigbee
|
||||||
|
|
||||||
|
# Manual passthrough
|
||||||
|
lsusb # Find vendor/product ID
|
||||||
|
incus config device add haos zigbee usb vendorid=XXXX productid=YYYY
|
||||||
|
|
||||||
|
# Restart VM
|
||||||
|
incus restart haos
|
||||||
|
```
|
||||||
|
|
||||||
|
## GitHub Actions Issues
|
||||||
|
|
||||||
|
### Build Timeout
|
||||||
|
**Symptoms**: CI/CD build fails with timeout
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
- Increase `QEMU_TIMEOUT` in workflow
|
||||||
|
- Reduce `IMAGE_SIZE`
|
||||||
|
- Use fewer services
|
||||||
|
|
||||||
|
### Asset Upload Fails
|
||||||
|
**Symptoms**: Image built but not in release
|
||||||
|
|
||||||
|
**Solutions**:
|
||||||
|
- Check GitHub token permissions
|
||||||
|
- Verify release was created
|
||||||
|
- Check asset size (<2GB per file)
|
||||||
|
|
||||||
|
## Getting Help
|
||||||
|
|
||||||
|
**Check logs**:
|
||||||
|
```bash
|
||||||
|
# Build logs
|
||||||
|
cat images/*/qemu-*.log
|
||||||
|
|
||||||
|
# First-boot logs
|
||||||
|
sudo journalctl -u rpi-first-boot
|
||||||
|
sudo journalctl -u services-first-boot
|
||||||
|
|
||||||
|
# Service logs
|
||||||
|
sudo journalctl -u docker
|
||||||
|
sudo journalctl -u incus
|
||||||
|
```
|
||||||
|
|
||||||
|
**Still stuck?**
|
||||||
|
- [Check FAQ](FAQ.md)
|
||||||
|
- [Open issue](https://github.com/Pikatsuto/raspberry-builds/issues)
|
||||||
|
- [Ask in discussions](https://github.com/Pikatsuto/raspberry-builds/discussions)
|
||||||
|
|
||||||
|
Include:
|
||||||
|
- Build command used
|
||||||
|
- Error messages
|
||||||
|
- Relevant logs
|
||||||
|
- Hardware details (Pi model, RAM, SD card size)
|
||||||
48
_Sidebar.md
48
_Sidebar.md
|
|
@ -1,26 +1,32 @@
|
||||||
## Navigation
|
## Raspberry Pi Image Builder
|
||||||
|
|
||||||
**Getting Started**
|
**[Home](Home)**
|
||||||
- [Home](Home)
|
|
||||||
- [Quick Start](Home#quick-start)
|
|
||||||
|
|
||||||
**Documentation**
|
### Getting Started
|
||||||
- [GitHub Actions](GitHub-Actions)
|
- [Installation & First Build](Getting-Started)
|
||||||
- [Build Process](GitHub-Actions#build-architecture)
|
- [Architecture Overview](Architecture)
|
||||||
|
|
||||||
**Images**
|
### Build System
|
||||||
- [RaspiVirt-Incus](Image-RaspiVirt-Incus)
|
- [Build System Guide](Build-System)
|
||||||
- [RaspiVirt-Incus+Docker](Image-RaspiVirt-Incus-Docker)
|
- [Service System](Services)
|
||||||
- [RaspiVirt-Incus+HAOS](Image-RaspiVirt-Incus-HAOS)
|
- [GitHub Actions CI/CD](GitHub-Actions)
|
||||||
|
|
||||||
**Resources**
|
### Reference
|
||||||
- [Main README](https://github.com/Pikatsuto/raspberry-builds/blob/main/README.md)
|
- [Available Images](Available-Images)
|
||||||
- [Releases](https://github.com/Pikatsuto/raspberry-builds/releases)
|
- [Available Services](Available-Services)
|
||||||
|
- [Hardware Detection](Hardware-Detection)
|
||||||
|
- [Command Reference](Command-Reference)
|
||||||
|
- [Configuration Reference](Configuration-Reference)
|
||||||
|
|
||||||
|
### Advanced
|
||||||
|
- [Creating Custom Images](Custom-Images)
|
||||||
|
- [Creating Custom Services](Custom-Services)
|
||||||
|
|
||||||
|
### Help
|
||||||
|
- [FAQ](FAQ)
|
||||||
|
- [Troubleshooting](Troubleshooting)
|
||||||
|
|
||||||
|
### Links
|
||||||
|
- [GitHub Repository](https://github.com/Pikatsuto/raspberry-builds)
|
||||||
- [Issues](https://github.com/Pikatsuto/raspberry-builds/issues)
|
- [Issues](https://github.com/Pikatsuto/raspberry-builds/issues)
|
||||||
|
- [Discussions](https://github.com/Pikatsuto/raspberry-builds/discussions)
|
||||||
---
|
|
||||||
|
|
||||||
**Quick Links**
|
|
||||||
- [Download Images](https://github.com/Pikatsuto/raspberry-builds/releases)
|
|
||||||
- [Report Bug](https://github.com/Pikatsuto/raspberry-builds/issues/new)
|
|
||||||
- [GitHub Actions](https://github.com/Pikatsuto/raspberry-builds/actions)
|
|
||||||
Loading…
Reference in a new issue