gabriel / raspberry-builds
Actions du dépôt
Watch
1
0
Fork
You've already forked raspberry-builds
 0

docs: auto-sync wiki from main repository 

Updated: 2025-12-09 00:01:08 UTC

Source commit: a22b1a7a56

Triggered by: push
github-actions[bot] 2025-12-09 00:01:08 +00:00
commit 58ca88f0a1
16 changed files with 4104 additions and 659 deletions
Download patch file Download diff file Expand all files Collapse all files

497
Architecture.md Normal file

@ -0,0 +1,497 @@
# Architecture
Understanding how the Raspberry Pi Image Builder works under the hood.
## Overview
The build system creates hybrid images by combining:
- **Raspberry Pi OS** - Boot partition with firmware, bootloader, and config
- **Debian ARM64** - Root filesystem with full userspace
- **RaspiOS Packages** - Kernel and firmware installed via APT (with auto-update support)
- **Custom Services** - Modular components composed during build
The result is a Debian system with full Raspberry Pi hardware support and automatic kernel/firmware updates.
## Design Philosophy
### Why Hybrid Images?
**Problem**: Raspberry Pi uses proprietary firmware and the RP1 chip for I/O (Ethernet, USB, GPIO). Standard Debian ARM64 kernels lack these drivers.
**Solution**: Keep Raspberry Pi OS boot partition and kernel packages, but use Debian for everything else.
**Benefits**:
- Full hardware support (RP1, WiFi, Bluetooth, GPIO)
- Automatic kernel/firmware updates via `apt upgrade`
- Debian's package ecosystem and stability
- No manual kernel compilation or firmware management
### Why Install Packages in QEMU?
**Traditional Approach**:
- Merge images
- Chroot into ARM64 rootfs from x86_64 host
- Use qemu-user-static for emulation
- Install packages
**Our Approach**:
- Install packages in native ARM64 QEMU VM **before** merge
- Merge pre-configured Debian image with RaspiOS boot
**Advantages**:
- **Simpler**: No complex chroot setup
- **Faster**: Native ARM64 execution, no user-mode overhead
- **Cleaner**: Merge script just copies files, no package management
- **Reproducible**: Same environment every build
## Build Pipeline
### 4-Stage Process
```
┌─────────────────────────────────────────────────────────────┐
│ Stage 1: Download & Prepare │
├─────────────────────────────────────────────────────────────┤
│ • Download RaspiOS Lite image │
│ • Download Debian cloud/generic ARM64 image │
│ • Parse service configuration │
│ • Resolve service dependencies │
│ • Combine setup scripts from all services │
│ • Create setup.iso (contains setup scripts + files) │
│ • Generate cloud-init seed.img OR inject first-boot service │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Stage 2: QEMU Setup (ARM64 VM) │
├─────────────────────────────────────────────────────────────┤
│ • Boot Debian ARM64 in QEMU │
│ • Cloud-init or first-boot service creates user │
│ • Mount setup.iso │
│ • Execute combined setup.sh: │
│ - Add RaspiOS APT repository + pinning │
│ - Install RaspiOS kernel packages (raspberrypi-kernel) │
│ - Install RaspiOS firmware packages │
│ - Install service packages (docker, incus, etc.) │
│ - Copy configuration files to /etc/setupfiles/ │
│ - Copy first-boot scripts │
│ • Auto-shutdown when complete │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Stage 3: Merge │
├─────────────────────────────────────────────────────────────┤
│ • Call merge-debian-raspios.sh │
│ • Keep RaspiOS boot partition (FAT32): │
│ - bootloader, config.txt, cmdline.txt │
│ • Backup RaspiOS /etc/fstab │
│ • Replace root partition with Debian (ext4): │
│ - Delete RaspiOS rootfs │
│ - rsync Debian rootfs (with RaspiOS packages installed) │
│ - Restore RaspiOS fstab (correct partition UUIDs) │
│ - Create /boot/firmware mount point │
│ • Resize root partition to fill image │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Stage 4: Compress │
├─────────────────────────────────────────────────────────────┤
│ • Run PiShrink to minimize filesystem │
│ • Compress with xz (parallel, level 6) │
│ • Generate checksums (SHA256) │
│ • Output: image-name.img.xz │
└─────────────────────────────────────────────────────────────┘
```
## Partition Layout
### Before Merge
**RaspiOS Image**:
```
┌────────────────────┬──────────────────────┐
│ /dev/loop0p1 │ /dev/loop0p2 │
│ boot (FAT32) │ root (ext4) │
│ 512MB │ ~2GB │
│ ================== │ ==================== │
│ • bootloader │ • RaspiOS rootfs │
│ • kernel │ • (will be replaced) │
│ • firmware │ │
│ • config.txt │ │
└────────────────────┴──────────────────────┘
```
**Debian Image**:
```
┌────────────────────────────────────────────┐
│ /dev/loop1p1 (or p2, auto-detected) │
│ root (ext4) │
│ ~2-4GB │
│ ========================================== │
│ • Debian userspace │
│ • RaspiOS kernel packages (from QEMU) │
│ • Service packages (from QEMU) │
│ • /etc/setupfiles/ (configs) │
│ • First-boot scripts │
└────────────────────────────────────────────┘
```
### After Merge
**Hybrid Image**:
```
┌────────────────────┬──────────────────────────────┐
│ /dev/mmcblk0p1 │ /dev/mmcblk0p2 │
│ boot (FAT32) │ root (ext4) │
│ 512MB │ 8GB (expanded) │
│ ================== │ ============================ │
│ • RaspiOS bootload │ • Debian userspace │
│ • RaspiOS kernel* │ • RaspiOS kernel packages │
│ • RaspiOS firmware │ • RaspiOS firmware packages │
│ • config.txt │ • RaspiOS fstab │
│ │ • /boot/firmware → p1 │
└────────────────────┴──────────────────────────────┘
(kept from RaspiOS) (replaced with Debian)
* Kernel files also in /boot/ and /lib/modules/ from APT packages
```
## Modular Service System
### Service Directory Structure
Each service is a self-contained module:
```
services/
└── <service-name>/
├── setup.sh # Runs in QEMU (package installation)
├── first-boot/
│ └── init.sh # Runs on first boot (runtime config)
├── setupfiles/ # Static files → /etc/setupfiles/
│ └── config.xyz
├── depends.sh # Optional: dependencies
└── motd.sh # Optional: MOTD content
```
### Service Lifecycle
```
BUILD TIME (QEMU):
┌─────────────────────────────────────────────────┐
│ setup.sh │
├─────────────────────────────────────────────────┤
│ • Install packages (apt install ...) │
│ • Configure system settings │
│ • Create users/groups │
│ • Set up repositories │
│ • Copy setupfiles/ to /etc/setupfiles/ │
│ • Install first-boot/init.sh │
└─────────────────────────────────────────────────┘
FIRST BOOT (Raspberry Pi):
┌─────────────────────────────────────────────────┐
│ rpi-first-boot.service (one-time) │
├─────────────────────────────────────────────────┤
│ • Expand root partition to fill SD card │
│ • Set persistent network interface names │
│ • Reboot │
└─────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────┐
│ services-first-boot.service (one-time) │
├─────────────────────────────────────────────────┤
│ • Execute base/first-boot/init.sh │
│ - Configure network bridges (br-wan, br-lan) │
│ - Set up DHCP server (if br-lan) │
│ • Execute service/first-boot/init.sh │
│ - Download images (HAOS, OpenWrt) │
│ - Create containers/VMs │
│ - Detect and configure hardware │
│ - Start services │
│ • Disable itself │
└─────────────────────────────────────────────────┘
RUNTIME:
┌─────────────────────────────────────────────────┐
│ Services running │
├─────────────────────────────────────────────────┤
│ • Docker containers │
│ • Incus VMs/containers │
│ • WiFi hotspot │
│ • OpenWrt router │
│ • etc. │
└─────────────────────────────────────────────────┘
```
### Service Dependency Resolution
Services can declare dependencies:
**Example**: `services/haos/depends.sh`
```bash
DEPENDS_ON="qemu"
```
**Build process**:
1. Parse requested services: `qemu+docker+haos`
2. Resolve dependencies:
- `qemu` (no dependencies)
- `docker` (no dependencies)
- `haos` → requires `qemu`
3. Build order: `base` → `qemu` → `docker` → `haos`
4. Combine setup.sh from all services
### Service Composition
**Dynamic image example**: `debian/qemu+docker+haos`
**Process**:
1. Create temporary directory: `images/debian-qemu-docker-haos/`
2. Copy base config: `images/debian/config.sh`
3. Override variables:
```bash
OUTPUT_IMAGE="debian-qemu-docker-haos.img"
SERVICES="base qemu docker haos"
```
4. Combine setup scripts:
```bash
cat services/base/setup.sh \
services/qemu/setup.sh \
services/docker/setup.sh \
services/haos/setup.sh \
> setup.sh
```
5. Merge setupfiles:
```bash
cp -r services/base/setupfiles/* setupfiles/
cp -r services/qemu/setupfiles/* setupfiles/
cp -r services/docker/setupfiles/* setupfiles/
cp -r services/haos/setupfiles/* setupfiles/
```
6. Inject first-boot scripts:
```bash
# Aggregate all init.sh into services-first-boot.sh
```
7. Build image
## APT Repository Management
### Repository Configuration
**File**: `/etc/apt/sources.list.d/raspi.sources`
```
Types: deb
URIs: http://archive.raspberrypi.com/debian/
Suites: trixie
Components: main
Signed-By: /usr/share/keyrings/raspberrypi-archive-keyring.pgp
```
**File**: `/etc/apt/preferences.d/raspi-pin`
```
# Pin RaspiOS kernel and firmware packages
Package: raspberrypi-kernel raspberrypi-bootloader libraspberrypi* firmware-brcm80211
Pin: release o=Raspberry Pi Foundation
Pin-Priority: 1001
# Default to Debian for everything else
Package: *
Pin: release o=Debian
Pin-Priority: 500
```
### Update Behavior
```bash
sudo apt update
# Fetches package lists from:
# - Debian repositories (default)
# - RaspiOS repository (for kernel/firmware)
sudo apt upgrade
# Upgrades:
# - raspberrypi-kernel → from RaspiOS repo (priority 1001)
# - raspberrypi-bootloader → from RaspiOS repo (priority 1001)
# - libraspberrypi* → from RaspiOS repo (priority 1001)
# - firmware-brcm80211 → from RaspiOS repo (priority 1001)
# - All other packages → from Debian repos (priority 500)
```
**Result**: Kernel and firmware stay in sync with RaspiOS, userspace packages track Debian.
## Boot Modes
### Cloud-Init Mode (CLOUD=true)
**Use when**:
- Using Debian cloud images
- Need cloud-init features (network config, metadata, etc.)
**Files**:
- `cloudinit/user-data` - User creation, SSH config, runcmd
- `cloudinit/meta-data` - Instance ID, hostname
- `cloudinit/seed.img` - Auto-generated ISO (CIDATA volume)
**Boot process**:
1. QEMU mounts seed.img (cloud-init config)
2. QEMU mounts setup.iso (build scripts)
3. Cloud-init creates user and runs runcmd
4. runcmd mounts setup.iso and executes setup.sh
5. VM shuts down after setup
### First-Boot Service Mode (CLOUD=false)
**Use when**:
- Using generic Debian images
- Don't need cloud-init
- Want minimal dependencies
**Files**:
- `first-boot/setup-runner.sh` - Creates user, runs setup
- `first-boot/setup-runner.service` - Systemd one-shot service
**Boot process**:
1. Autobuild injects files into Debian image (before QEMU)
2. Autobuild enables systemd service via chroot
3. QEMU boots Debian
4. Systemd starts setup-runner.service
5. Service creates user, mounts setup.iso, runs setup.sh
6. Service disables itself
7. VM shuts down after setup
## Hardware Detection
### Detection Points
**1. Build Time (QEMU)**:
- Install packages needed for hardware detection
- Copy detection scripts to /etc/setupfiles/
**2. First Boot (Raspberry Pi)**:
- Detect network interfaces (eth0, eth1)
- Detect WiFi adapters (wlan0, wlan1)
- Detect USB Zigbee dongles
- Configure services based on detected hardware
### Network Interface Detection
**File**: `services/base/first-boot/init.sh`
```bash
# Detect eth1 (second NIC)
if ip link show eth1 >/dev/null 2>&1; then
# Dual NIC mode
# br-wan (eth0) - WAN with DHCP client
# br-lan (eth1) - LAN with DHCP server + NAT
else
# Single NIC mode
# br-wan (eth0) - WAN with DHCP client
fi
```
**Result**:
- Single NIC: WAN only
- Dual NIC: WAN + LAN with DHCP/NAT
### WiFi Detection
**File**: `services/hotspot/first-boot/init.sh`
```bash
# Detect WiFi interfaces
wlan0_exists=$(ip link show wlan0 2>/dev/null)
wlan1_exists=$(ip link show wlan1 2>/dev/null)
if [[ -n "$wlan0_exists" && -n "$wlan1_exists" ]]; then
# Dual-band: 2.4GHz on wlan0, 5GHz on wlan1
elif [[ -n "$wlan0_exists" ]]; then
# Single-band: 5GHz on wlan0
fi
# Determine bridge
if ip link show br-lan >/dev/null 2>&1; then
BRIDGE="br-lan"
else
BRIDGE="br-wan"
fi
```
**Result**:
- Dual WiFi: 2.4GHz + 5GHz APs
- Single WiFi: 5GHz AP only
- Adapts to available bridge
### USB Zigbee Detection
**File**: `services/haos/first-boot/init.sh`
```bash
# Scan USB serial devices
for device in /dev/ttyUSB* /dev/ttyACM*; do
device_info=$(udevadm info -q property -n "$device")
# Check vendor for known Zigbee coordinators
if echo "$device_info" | grep -qiE "(FTDI|Silicon_Labs|Texas_Instruments|dresden_elektronik|ITead|Sonoff)"; then
# Extract USB IDs
USB_VENDOR=$(echo "$device_info" | grep "ID_VENDOR_ID=" | cut -d'=' -f2)
USB_PRODUCT=$(echo "$device_info" | grep "ID_MODEL_ID=" | cut -d'=' -f2)
# Pass through to Home Assistant VM
incus config device add haos zigbee-dongle usb \
vendorid="$USB_VENDOR" \
productid="$USB_PRODUCT" \
required=false
fi
done
```
**Result**: Zigbee coordinators automatically available in Home Assistant.
## Image Size Management
### Size Calculation
**Autobuild logic**:
```bash
# Debian image size
DEBIAN_SIZE=$(qemu-img info --output=json debian.raw | jq -r '.["virtual-size"]')
# Add overhead (1-2GB for services, temp files, expansion)
FINAL_SIZE=$((DEBIAN_SIZE + 2GB))
# Override with config.sh IMAGE_SIZE if specified
```
### Partition Expansion
**During merge** (`merge-debian-raspios.sh`):
1. Create output image with `IMAGE_SIZE`
2. Resize partition table
3. Expand root partition to fill available space
4. Resize ext4 filesystem
**On first boot** (`rpi-first-boot.sh`):
1. Expand root partition to fill SD card
2. Resize ext4 filesystem to match
3. Reboot to apply changes
**Result**: Image expands to fill entire SD card, regardless of size.
## Summary
The architecture uses:
- **Hybrid approach** - RaspiOS boot + Debian rootfs
- **QEMU ARM64** - Native package installation before merge
- **Modular services** - Composable image components
- **APT pinning** - Automatic kernel/firmware updates
- **Hardware detection** - Runtime configuration based on detected hardware
- **Two boot modes** - Cloud-init or first-boot service
This design provides:
- Full Raspberry Pi hardware support
- Safe automatic updates
- Easy customization
- Reproducible builds
- Team collaboration via version control
**Next**: [Learn about the build system](Build-System.md)

323
Available-Images.md Normal file

@ -0,0 +1,323 @@
# Available Images
Pre-configured images available for download and their use cases.
## Official Images
All images available in [GitHub Releases](https://github.com/Pikatsuto/raspberry-builds/releases).
### debian-base
**Description**: Minimal Debian 13 (Trixie) with RaspiOS kernel and firmware
**Services included**:
- base (RaspiOS kernel, NetworkManager, SSH)
**Size**: ~512MB compressed
**RAM required**: 2GB minimum
**Use cases**:
- Minimal Debian installation
- Custom development base
- Learning/testing Raspberry Pi
**Default login**:
- Username: `pi`
- Password: `raspberry`
**Download**:
```bash
wget https://github.com/Pikatsuto/raspberry-builds/releases/latest/download/debian-base.img.xz
```
---
### debian-qemu-docker
**Description**: Debian with Incus virtualization and Docker Engine
**Services included**:
- base
- qemu (Incus + KVM)
- docker (Docker Engine + Portainer + Watchtower)
**Size**: ~4-5GB compressed
**RAM required**: 4GB minimum (8GB recommended)
**Use cases**:
- Container host (Docker + Incus)
- Development platform
- Microservices deployment
- VM/container testing
**Access**:
- Portainer: `https://raspberry-ip:9443`
- Incus: `incus list` (SSH)
**Download**:
```bash
wget https://github.com/Pikatsuto/raspberry-builds/releases/latest/download/debian-qemu-docker.img.xz
```
---
### debian-qemu-openwrt-hotspot
**Description**: Debian with OpenWrt router and WiFi hotspot
**Services included**:
- base
- qemu (Incus)
- openwrt (OpenWrt container)
- hotspot (WiFi AP)
**Size**: ~3-4GB compressed
**RAM required**: 4GB minimum
**Hardware required**:
- Dual NIC (eth0 + eth1) or single NIC
- WiFi adapter(s) for hotspot
**Use cases**:
- Network router/firewall
- WiFi access point
- VPN gateway
- Network segmentation
**Access**:
- OpenWrt LuCI: `http://192.168.10.1`
- WiFi SSID: `RaspberryPi-5G` (password: `raspberry`)
**Download**:
```bash
wget https://github.com/Pikatsuto/raspberry-builds/releases/latest/download/debian-qemu-openwrt-hotspot.img.xz
```
---
### debian-qemu-docker-openwrt-hotspot-haos (Full Stack)
**Description**: Complete home automation and network platform
**Services included**:
- base
- qemu (Incus + KVM)
- docker (Docker Engine + Portainer)
- openwrt (OpenWrt router)
- hotspot (WiFi AP)
- haos (Home Assistant OS)
**Size**: ~6-8GB compressed
**RAM required**: 8GB (Raspberry Pi 5 8GB recommended)
**Hardware required**:
- Dual NIC (eth0 + eth1) recommended
- WiFi adapter(s) for hotspot
- Optional: Zigbee/Z-Wave USB dongle
**Use cases**:
- Complete home automation gateway
- Network router + WiFi + smart home
- All-in-one Raspberry Pi solution
**Access**:
- Home Assistant: `http://raspberry-ip:8123`
- Portainer: `https://raspberry-ip:9443`
- OpenWrt LuCI: `http://192.168.10.1`
- WiFi SSID: `RaspberryPi-5G`
**Download**:
```bash
wget https://github.com/Pikatsuto/raspberry-builds/releases/latest/download/debian-qemu-docker-openwrt-hotspot-haos.img.xz
```
---
## Image Comparison
| Feature | base | qemu-docker | qemu-openwrt-hotspot | Full Stack |
|---------|------|-------------|----------------------|------------|
| Debian OS | ✅ | ✅ | ✅ | ✅ |
| RaspiOS Kernel | ✅ | ✅ | ✅ | ✅ |
| Network Bridges | ✅ | ✅ | ✅ | ✅ |
| SSH | ✅ | ✅ | ✅ | ✅ |
| Incus (VMs/containers) | ❌ | ✅ | ✅ | ✅ |
| Docker | ❌ | ✅ | ❌ | ✅ |
| Portainer | ❌ | ✅ | ❌ | ✅ |
| OpenWrt Router | ❌ | ❌ | ✅ | ✅ |
| WiFi Hotspot | ❌ | ❌ | ✅ | ✅ |
| Home Assistant | ❌ | ❌ | ❌ | ✅ |
| Zigbee Auto-detect | ❌ | ❌ | ❌ | ✅ |
| Minimum RAM | 2GB | 4GB | 4GB | 8GB |
| Compressed Size | ~512MB | ~700MB | ~750MB | ~750MB |
---
## Release Channels
### Stable (main branch)
**Recommended for production use**
- Tested and verified
- No pre-release flag
- Downloaded from latest release
**Download**:
```bash
# Latest stable
wget https://github.com/Pikatsuto/raspberry-builds/releases/latest/download/<image-name>.img.xz
```
### Test (test branch)
**For testing new features**
- Pre-release builds
- May contain bugs
- Use for testing only
**Download**:
```bash
# Browse releases and select test build
# URL: https://github.com/Pikatsuto/raspberry-builds/releases
```
### Preview (preview branch)
**Experimental builds**
- Bleeding-edge features
- Unstable
- Use at your own risk
---
## Daily Builds
**Automatic daily builds** at 2:00 AM UTC
**Tag format**: `daily-YYYY-MM-DD`
**Purpose**:
- Captures latest base image updates
- Tests build system
- Provides fresh images daily
**Download**:
```bash
# Today's build
wget https://github.com/Pikatsuto/raspberry-builds/releases/download/daily-2024-12-08/<image-name>.img.xz
```
**Note**: Daily builds overwrite previous daily release. Download and archive if needed.
---
## Installation
### 1. Download Image
Choose image from above, download via browser or wget.
**Verify checksum**:
```bash
sha256sum <image-name>.img.xz
# Compare with .sha256 file from release
```
### 2. Flash to SD Card
**Linux**:
```bash
xz -dc <image-name>.img.xz | sudo dd of=/dev/sdX bs=4M status=progress conv=fsync
sync
```
**macOS**:
```bash
xz -dc <image-name>.img.xz | sudo dd of=/dev/rdiskX bs=4m
sync
```
**Windows**:
- Use [Balena Etcher](https://www.balena.io/etcher/)
- Select `.img.xz` file
- Select SD card
- Flash
### 3. Boot Raspberry Pi
- Insert SD card
- Connect Ethernet (recommended for first boot)
- Power on
- Wait 3-5 minutes for first-boot setup
- Find IP address from router or MOTD on login
---
## Customization After Installation
All images can be customized post-installation:
**Update system**:
```bash
sudo apt update && sudo apt upgrade -y
```
**Install additional packages**:
```bash
sudo apt install <package-name>
```
**Add Docker containers** (if Docker image):
```bash
docker run -d <container-image>
# Or use Portainer web UI
```
**Create Incus containers/VMs** (if qemu image):
```bash
incus launch images:alpine mycontainer
incus launch images:debian myvm --vm
```
---
## Building Custom Images
Don't see the image you need? Build your own!
**See**: [Creating Custom Images](Custom-Images.md)
**Quick example**:
```bash
git clone https://github.com/Pikatsuto/raspberry-builds.git
cd raspberry-builds
# Build custom combination
./bin/autobuild --image debian/qemu+docker+haos
# Or create entirely custom image
cp -r images/debian images/myproject
vim images/myproject/config.sh
./bin/autobuild --image myproject
```
---
## Support
**Issues with images**:
- [Report bug](https://github.com/Pikatsuto/raspberry-builds/issues/new?template=bug_report.md)
- [Request feature](https://github.com/Pikatsuto/raspberry-builds/issues/new?template=feature_request.md)
- [Ask question](https://github.com/Pikatsuto/raspberry-builds/discussions)
**Image documentation**:
- [Getting Started](Getting-Started.md)
- [FAQ](FAQ.md)
- [Troubleshooting](Troubleshooting.md)

76
Available-Services.md Normal file

@ -0,0 +1,76 @@
# Available Services
Quick reference for service modules. See [Services Guide](Services.md) for creating custom services.
## Service List
### base
**Always included** - RaspiOS kernel, NetworkManager, SSH, network bridges
- Location: `images/debian/services/base/`
### qemu
**Incus virtualization** - Containers and VMs with KVM
- Dependencies: None
- Command: `./bin/autobuild --image debian/qemu`
### docker
**Docker Engine** - Portainer + Watchtower
- Dependencies: None
- Access: `https://raspberry-ip:9443` (Portainer)
- Command: `./bin/autobuild --image debian/docker`
### haos
**Home Assistant OS** - VM with Zigbee auto-detect
- Dependencies: qemu
- Access: `http://raspberry-ip:8123`
- Command: `./bin/autobuild --image debian/qemu+haos`
### openwrt
**OpenWrt router** - Container with advanced routing
- Dependencies: qemu
- Access: `http://192.168.10.1` (LuCI)
- Command: `./bin/autobuild --image debian/qemu+openwrt`
### hotspot
**WiFi access point** - Auto-detects wlan0/wlan1
- Dependencies: None
- SSID: `RaspberryPi-5G` (password: `raspberry`)
- Command: `./bin/autobuild --image debian/hotspot`
## Resource Requirements
| Service | RAM | Disk | Special Hardware |
|---------|-----|------|------------------|
| base | 500MB | 2GB | - |
| qemu | +100MB | +500MB | - |
| docker | +200MB | +1GB | - |
| haos | +4GB | +32GB | Optional: USB Zigbee |
| openwrt | +100MB | +500MB | Optional: Dual NIC |
| hotspot | +50MB | +100MB | WiFi adapter |
## Common Combinations
```bash
# Home server
./bin/autobuild --image debian/docker
# Virtualization platform
./bin/autobuild --image debian/qemu+docker
# Network router + WiFi
./bin/autobuild --image debian/qemu+openwrt+hotspot
# Home automation
./bin/autobuild --image debian/qemu+haos
# Full stack (requires 8GB RAM)
./bin/autobuild --image debian/qemu+docker+openwrt+hotspot+haos
```
## Service Details
For detailed information:
- [Service architecture](Services.md)
- [Creating custom services](Services/#creating-custom-services)
- [Hardware detection](Hardware-Detection.md)
- [Pre-built images](Available-Images.md)

565
Build-System.md Normal file

@ -0,0 +1,565 @@
# Build System
Complete reference for the autobuild system and image creation.
## Autobuild Command
The `autobuild` script orchestrates the entire build process.
### Basic Usage
```bash
./bin/autobuild --image <image-name>
```
### Image Formats
**Physical directory format**:
```bash
./bin/autobuild --image debian
# Uses images/debian/config.sh
```
**Dynamic service composition**:
```bash
./bin/autobuild --image debian/qemu+docker+haos
# Uses images/debian/config.sh + combines services
```
### Common Options
```bash
# Build specific image
./bin/autobuild --image debian
# Build all images from .github/images.txt
./bin/autobuild --all-images
# List available images
./bin/autobuild --list-images
# Skip base image downloads (use cached)
./bin/autobuild --image debian --skip-download
# Skip QEMU setup (use existing Debian image)
./bin/autobuild --image debian --skip-qemu
# Skip compression
./bin/autobuild --image debian --skip-compress
# Clean previous builds
./bin/autobuild --image debian --clean
```
### Combined Options
```bash
# Quick rebuild without downloads or compression
./bin/autobuild --image debian --skip-download --skip-compress
# Rebuild with new setup scripts, skip downloads
./bin/autobuild --image debian --skip-download --clean
```
## Build Stages
### Stage 1: Download & Prepare
**Actions**:
- Download RaspiOS Lite image (if not cached)
- Download Debian ARM64 image (if not cached)
- Parse image configuration (config.sh)
- Resolve service dependencies
- Combine setup.sh from all services
- Merge setupfiles/ directories
- Create setup.iso with combined configuration
- Generate cloud-init seed.img OR inject first-boot service
**Outputs**:
- `raspios-lite.img` (in distro directory)
- `debian-arm64.raw` (in distro directory)
- `setup.iso` (in image directory)
- `seed.img` (in cloudinit/ directory, if CLOUD=true)
**Environment Variables**:
```bash
RASPIOS_URL="https://downloads.raspberrypi.org/..."
IMAGE_URL="https://cloud.debian.org/..." # from config.sh
CLOUD=true # or false, from config.sh
SERVICES="base qemu docker" # from config.sh or dynamic
```
### Stage 2: QEMU Setup
**Actions**:
- Convert Debian image to raw format (if qcow2)
- Launch QEMU ARM64 VM:
- RAM: QEMU_RAM (from config.sh)
- CPUs: QEMU_CPUS (from config.sh)
- Disk: Debian image
- CD-ROM 1: seed.img (if cloud-init mode)
- CD-ROM 2: setup.iso
- Wait for setup completion (VM auto-shutdown)
- Monitor progress via QEMU serial console
**QEMU Configuration**:
```bash
qemu-system-aarch64 \
-machine virt \
-cpu cortex-a72 \
-m $QEMU_RAM \
-smp $QEMU_CPUS \
-drive file=debian.raw,format=raw,if=virtio \
-drive file=seed.img,format=raw,if=virtio,readonly=on \ # cloud-init
-drive file=setup.iso,format=raw,if=virtio,readonly=on \
-bios /usr/share/qemu-efi-aarch64/QEMU_EFI.fd \
-nographic \
-netdev user,id=net0 \
-device virtio-net-pci,netdev=net0
```
**Inside QEMU**:
1. Cloud-init or first-boot service creates user
2. Setup script mounts setup.iso
3. Executes combined setup.sh:
- Adds RaspiOS repository
- Installs RaspiOS kernel/firmware
- Installs service packages
- Copies setupfiles to /etc/setupfiles/
- Installs first-boot scripts
4. Shuts down VM
**Timeout**: 30 minutes (configurable via `QEMU_TIMEOUT`)
### Stage 3: Merge
**Actions**:
- Call `merge-debian-raspios.sh`
- Keep RaspiOS boot partition (FAT32)
- Replace RaspiOS root with Debian root
- Restore RaspiOS fstab (for correct partition UUIDs)
- Resize root partition to IMAGE_SIZE
**Merge Process** (10 sub-stages):
1. Dependency verification
2. RaspiOS preparation (decompress if .xz)
3. Debian preparation (convert to raw if needed)
4. Size analysis
5. Output image creation (copy of RaspiOS)
6. Image resizing
7. Loop device mounting
8. Backup RaspiOS fstab
9. **Rootfs replacement**:
- Delete RaspiOS root
- Rsync Debian root with `-aAXv` (preserve all attributes)
- Create `/boot/firmware` mount point
- Restore fstab
10. Cleanup
**Output**: `<image-name>.img`
### Stage 4: Compress
**Actions**:
- Run PiShrink to minimize filesystem
- Compress with xz (parallel, level 6)
- Generate SHA256 checksum
**PiShrink**:
```bash
pishrink.sh -z <image>.img <image>.img.xz
# -z: Compress with xz after shrinking
```
**Output**: `<image-name>.img.xz`
## Configuration Files
### Image Configuration (config.sh)
**Location**: `images/<distro>/config.sh` or `images/<image-name>/config.sh`
**Required Variables**:
```bash
# Output filename
OUTPUT_IMAGE="debian-base.img"
# Final image size (supports K, M, G suffixes)
IMAGE_SIZE="8G"
# QEMU resources
QEMU_RAM="8G"
QEMU_CPUS="4"
# Boot mode (true = cloud-init, false = first-boot service)
CLOUD=true
# Base distribution image URL
IMAGE_URL="https://cloud.debian.org/images/cloud/trixie-backports/daily/latest/debian-13-backports-genericcloud-arm64-daily.raw"
# Services to include (space-separated)
SERVICES="base qemu docker"
# Description (optional, for documentation)
DESCRIPTION="Debian with Incus and Docker"
```
**Optional Variables**:
```bash
# Custom RaspiOS URL (default: RaspiOS Lite trixie)
RASPIOS_URL="https://downloads.raspberrypi.org/..."
# QEMU timeout in seconds (default: 1800 = 30 minutes)
QEMU_TIMEOUT=3600
# Skip PiShrink compression
SKIP_PISHRINK=false
```
### Service Configuration
Each service directory contains:
**setup.sh** (required):
```bash
#!/bin/bash
set -e
# Install packages
apt update
apt install -y package1 package2
# Configure system
systemctl enable service1
```
**first-boot/init.sh** (optional):
```bash
#!/bin/bash
set -e
# Runtime configuration
# Detect hardware, create containers, etc.
```
**depends.sh** (optional):
```bash
# Declare dependencies
DEPENDS_ON="qemu"
```
**motd.sh** (optional):
```bash
# MOTD content
cat <<'EOF'
Service UI: https://raspberry-ip:9000
Username: admin
EOF
```
**setupfiles/** (optional):
- Static configuration files
- Copied to /etc/setupfiles/ during build
## Build Artifacts
### Directory Structure
```
images/
└── debian/ # Distribution directory
├── config.sh # Base configuration
├── cloudinit/ # Cloud-init mode
│ ├── user-data
│ ├── meta-data
│ └── seed.img # Auto-generated
├── services/ # Service modules
│ ├── base/
│ ├── qemu/
│ └── docker/
├── raspios-lite.img # Downloaded RaspiOS (cached)
├── debian-arm64.raw # Downloaded Debian (cached)
└── debian-qemu-docker/ # Dynamic image (created during build)
├── config.sh # Generated from base + overrides
├── setup.sh # Combined from services
├── setup.iso # Generated
├── setupfiles/ # Merged from services
├── debian-qemu-docker.img # Final image
└── debian-qemu-docker.img.xz # Compressed
```
### Artifact Persistence
**Persistent** (cached between builds):
- `raspios-lite.img` - RaspiOS base image
- `debian-arm64.raw` - Debian base image (modified by QEMU)
- Final `.img` and `.img.xz` files
**Temporary** (cleaned with `--clean`):
- Dynamic image directories (e.g., `debian-qemu-docker/`)
- `setup.iso`
- `seed.img` (regenerated each build)
**Skip Downloads**:
```bash
# Use cached base images
./bin/autobuild --image debian --skip-download
```
**Skip QEMU**:
```bash
# Use existing configured Debian image (skip setup in QEMU)
./bin/autobuild --image debian --skip-qemu
```
## Service Dependency Resolution
### Dependency Declaration
**Example**: Home Assistant requires Incus
**File**: `images/debian/services/haos/depends.sh`
```bash
DEPENDS_ON="qemu"
```
### Resolution Algorithm
**Input**: `debian/qemu+docker+haos`
**Process**:
1. Parse services: `qemu`, `docker`, `haos`
2. Resolve dependencies:
- `qemu`: no dependencies
- `docker`: no dependencies
- `haos`: depends on `qemu` (already in list)
3. Remove duplicates
4. Order by dependencies: `base` → `qemu` → `docker` → `haos`
**Output**: `SERVICES="base qemu docker haos"`
### Build Order
Services are built in dependency order:
1. **base** (always first)
2. Dependencies (e.g., `qemu` for `haos`)
3. Requested services
**Setup script combination**:
```bash
{
cat services/base/setup.sh
cat services/qemu/setup.sh
cat services/docker/setup.sh
cat services/haos/setup.sh
} > combined-setup.sh
```
## Merge Process Details
### Partition Operations
**1. RaspiOS Boot Partition** (kept):
```bash
# Mount as read-only (no changes)
mount -o ro /dev/loop0p1 /mnt/raspios-boot
```
**2. RaspiOS Root Partition** (backed up, then replaced):
```bash
# Backup fstab only
cp /mnt/raspios-root/etc/fstab /tmp/raspios-fstab
# Delete entire rootfs
rm -rf /mnt/raspios-root/*
```
**3. Debian Root Partition** (source):
```bash
# Rsync to RaspiOS root
rsync -aAXv /mnt/debian-root/ /mnt/raspios-root/
# Preserve attributes:
# -a: archive mode (recursive, preserve permissions, times, symlinks)
# -A: preserve ACLs
# -X: preserve extended attributes
# -v: verbose
```
**4. Restore RaspiOS fstab**:
```bash
# RaspiOS fstab has correct partition UUIDs
cp /tmp/raspios-fstab /mnt/raspios-root/etc/fstab
```
**5. Create boot mount point**:
```bash
# Ensure /boot/firmware exists for boot partition
mkdir -p /mnt/raspios-root/boot/firmware
```
### Size Calculation
**Automatic sizing**:
```bash
# Get Debian image virtual size
DEBIAN_SIZE=$(qemu-img info --output=json debian.raw | jq -r '.["virtual-size"]')
# Add overhead (1-2GB)
AUTO_SIZE=$((DEBIAN_SIZE + 2 * 1024 * 1024 * 1024))
# Use IMAGE_SIZE from config if larger
FINAL_SIZE=$(max $AUTO_SIZE $IMAGE_SIZE)
```
**Manual override**:
```bash
./bin/merge-debian-raspios.sh raspios.img debian.raw -s 16G
```
### Partition Expansion
**During merge**:
```bash
# Resize partition table
parted /dev/loop0 resizepart 2 100%
# Resize ext4 filesystem
e2fsck -f /dev/loop0p2
resize2fs /dev/loop0p2
```
**On first boot** (rpi-first-boot.sh):
```bash
# Expand to fill entire SD card
parted /dev/mmcblk0 resizepart 2 100%
resize2fs /dev/mmcblk0p2
reboot
```
## Troubleshooting Builds
### QEMU Won't Boot
**Symptoms**: QEMU hangs at boot
**Causes**:
- Missing UEFI firmware
- Wrong image format
- Insufficient RAM
**Solutions**:
```bash
# Install UEFI firmware
sudo apt install qemu-efi-aarch64
# Check image format
qemu-img info debian.raw
# Increase RAM in config.sh
QEMU_RAM="8G"
```
### QEMU Timeout
**Symptoms**: Build fails with "QEMU timeout"
**Causes**:
- Slow network (downloading packages)
- Insufficient resources
- Stuck on interactive prompt
**Solutions**:
```bash
# Increase timeout
QEMU_TIMEOUT=3600 # 1 hour
# Increase resources
QEMU_RAM="8G"
QEMU_CPUS="4"
# Check QEMU logs
cat qemu-*.log
```
### Merge Fails
**Symptoms**: Error during merge stage
**Causes**:
- Insufficient disk space
- Corrupted images
- Partition layout mismatch
**Solutions**:
```bash
# Check disk space
df -h
# Re-download base images
rm images/debian/raspios-lite.img images/debian/debian-arm64.raw
./bin/autobuild --image debian
# Check partition layout
fdisk -l raspios.img
fdisk -l debian.raw
```
### Image Won't Boot
**Symptoms**: Raspberry Pi won't boot image
**Causes**:
- Corrupted SD card
- Wrong fstab UUIDs
- Missing boot files
**Solutions**:
```bash
# Verify image integrity
sha256sum image.img.xz
# Check SD card
sudo badblocks -v /dev/sdX
# Re-flash image
xz -dc image.img.xz | sudo dd of=/dev/sdX bs=4M status=progress conv=fsync
```
## Advanced Usage
### Custom RaspiOS Version
```bash
# In config.sh
RASPIOS_URL="https://downloads.raspberrypi.org/raspios_lite_arm64/images/raspios_lite_arm64-2024-11-24/2024-11-24-raspios-trixie-arm64-lite.img.xz"
```
### Custom Debian Version
```bash
# In config.sh
IMAGE_URL="https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-genericcloud-arm64.raw"
```
### Parallel Builds
```bash
# Build multiple images in parallel
./bin/autobuild --image debian &
./bin/autobuild --image debian/qemu+docker &
wait
```
**Warning**: Ensure sufficient RAM and disk space for parallel builds.
### CI/CD Integration
See [GitHub Actions](GitHub-Actions.md) for automated builds.
## Next Steps
- [Learn about available services](Services.md)
- [Create a custom image](Custom-Images.md)
- [Set up CI/CD](GitHub-Actions.md)

90
Command-Reference.md Normal file

@ -0,0 +1,90 @@
# Command Reference
Quick reference for build commands.
## autobuild
Main build script.
### Basic Usage
```bash
./bin/autobuild --image <name>
```
### Options
```bash
--image <name> Build specific image
--all-images Build all images from .github/images.txt
--list-images List available images
--skip-download Use cached base images
--skip-qemu Skip QEMU setup
--skip-compress Skip PiShrink compression
--clean Clean previous build artifacts
```
### Examples
See [Build System - Autobuild Command](Build-System/#autobuild-command) for comprehensive examples and options.
## merge-debian-raspios.sh
Low-level merge script (called by autobuild).
### Usage
```bash
./bin/merge-debian-raspios.sh <raspios-image> <debian-image> [options]
```
### Options
```bash
-o, --output <file> Output image name
-s, --size <size> Final image size (e.g., 16G)
-k, --keep-kernel Use Debian kernel (not recommended)
```
### Examples
```bash
# Basic merge
./bin/merge-debian-raspios.sh raspios.img debian.raw
# Custom output and size
./bin/merge-debian-raspios.sh raspios.img debian.raw -o custom.img -s 16G
```
## Flashing Commands
```bash
# Decompress and flash
xz -dc image.img.xz | sudo dd of=/dev/sdX bs=4M status=progress conv=fsync
# Flash uncompressed
sudo dd if=image.img of=/dev/sdX bs=4M status=progress conv=fsync
# Sync
sync
```
## System Commands (on Raspberry Pi)
```bash
# Update system
sudo apt update && sudo apt upgrade -y
# Check first-boot logs
sudo journalctl -u services-first-boot
# Check service status
sudo systemctl status docker
incus list
# Network info
ip addr show
nmcli con show
```
See [Build System](Build-System.md) for detailed usage.

65
Configuration-Reference.md Normal file

@ -0,0 +1,65 @@
# Configuration Reference
Quick reference for configuration files.
## config.sh
Image configuration file.
**Location**: `images/<name>/config.sh`
**Required**:
```bash
OUTPUT_IMAGE="image-name.img"
IMAGE_SIZE="8G"
QEMU_RAM="4G"
QEMU_CPUS="4"
CLOUD=true # or false
IMAGE_URL="https://..."
SERVICES="base service1 service2"
```
**Optional**:
```bash
DESCRIPTION="Image description"
RASPIOS_URL="https://..."
QEMU_TIMEOUT=1800
```
## Service Files
See [Services - Creating Custom Services](Services/#creating-custom-services) for detailed templates and best practices.
**Quick reference**:
- `setup.sh`: Runs in QEMU, installs packages
- `first-boot/init.sh`: Runs on first boot, detects hardware
- `depends.sh`: Declares service dependencies
- `motd.sh`: MOTD banner content
## Network Configuration (on Raspberry Pi)
### Static IP
```bash
sudo nmcli con mod br-wan ipv4.addresses "192.168.1.100/24"
sudo nmcli con mod br-wan ipv4.gateway "192.168.1.1"
sudo nmcli con mod br-wan ipv4.dns "8.8.8.8"
sudo nmcli con mod br-wan ipv4.method manual
sudo nmcli con up br-wan
```
### WiFi Hotspot
See [Hardware Detection - WiFi Adapters](Hardware-Detection/#wifi-adapters) for detailed configuration.
**Quick edit**: `/etc/hostapd/hostapd-5ghz.conf` (SSID, password), then `sudo systemctl restart hostapd-5ghz`
## APT Configuration
See [Architecture - APT Repository Management](Architecture/#apt-repository-management) for full details on repository configuration and update behavior.
**Files**: `/etc/apt/sources.list.d/raspi.sources` and `/etc/apt/preferences.d/raspi-pin`
For detailed explanations see:
- [Build System](Build-System.md)
- [Services](Services.md)
- [Architecture](Architecture.md)

73
Custom-Images.md Normal file

@ -0,0 +1,73 @@
# Creating Custom Images
Guide to creating your own image configurations.
## Quick Start
```bash
# 1. Copy base configuration
cp -r images/debian images/myproject
# 2. Edit config
vim images/myproject/config.sh
# 3. Customize setup
vim images/myproject/services/base/setup.sh
# 4. Build
./bin/autobuild --image myproject
```
## Configuration File (config.sh)
**Required variables**:
```bash
OUTPUT_IMAGE="myproject.img"
IMAGE_SIZE="8G"
QEMU_RAM="4G"
QEMU_CPUS="4"
CLOUD=true # or false
IMAGE_URL="https://cloud.debian.org/images/cloud/trixie-backports/daily/latest/debian-13-backports-genericcloud-arm64-daily.raw"
SERVICES="base qemu docker"
DESCRIPTION="My custom image"
```
## Using Service Composition
**Dynamic images** - compose from existing services:
```bash
./bin/autobuild --image debian/qemu+docker+myservice
```
**Physical images** - full custom directory:
```bash
./bin/autobuild --image myproject
```
See [Services - Creating Custom Services](Services/#creating-custom-services) for adding new service modules.
## Examples
**Minimal Debian**:
```bash
SERVICES="base"
IMAGE_SIZE="4G"
```
**Docker host**:
```bash
SERVICES="base docker"
IMAGE_SIZE="8G"
```
**Custom service stack**:
```bash
SERVICES="base qemu docker myapp"
IMAGE_SIZE="16G"
```
## Next Steps
- [Build system options](Build-System.md)
- [Create custom service](Services/#creating-custom-services)
- [GitHub Actions CI/CD](GitHub-Actions.md)

70
Custom-Services.md Normal file

@ -0,0 +1,70 @@
# Creating Custom Services
Guide to creating new service modules.
## Service Structure
```
images/debian/services/myservice/
├── setup.sh # Package installation (runs in QEMU)
├── first-boot/init.sh # Runtime config (runs on first boot)
├── setupfiles/ # Static files → /etc/setupfiles/
├── depends.sh # Optional: dependencies
└── motd.sh # Optional: MOTD banner
```
## Quick Template
```bash
# Create service
mkdir -p images/debian/services/myservice/first-boot
# Package installation
cat > images/debian/services/myservice/setup.sh <<'EOF'
#!/bin/bash
set -e
apt update
apt install -y nginx
systemctl enable nginx
EOF
chmod +x images/debian/services/myservice/setup.sh
# Runtime configuration
cat > images/debian/services/myservice/first-boot/init.sh <<'EOF'
#!/bin/bash
set -e
systemctl start nginx
EOF
chmod +x images/debian/services/myservice/first-boot/init.sh
# Build
./bin/autobuild --image debian/myservice
```
## Script Guidelines
**setup.sh** (runs in QEMU):
- Install packages
- Configure system
- Enable services (don't start them)
**first-boot/init.sh** (runs on Raspberry Pi):
- Detect hardware
- Download large files
- Create containers/VMs
- Start services
## Dependencies
Create `depends.sh`:
```bash
DEPENDS_ON="qemu" # This service requires qemu
```
## Advanced Topics
For detailed information see [Services Guide](Services.md):
- Service lifecycle
- Hardware detection
- Best practices
- Debugging

502
FAQ.md Normal file

@ -0,0 +1,502 @@
# Frequently Asked Questions
## General Questions
### What is this project?
An automated build system for creating custom Raspberry Pi images with:
- Debian ARM64 userspace
- Raspberry Pi OS kernel and firmware
- Modular service composition
- Automatic hardware detection
- GitHub Actions CI/CD
### Why not use standard Raspberry Pi OS?
You should use standard RaspiOS if it meets your needs. Use this project if you want:
- Debian package ecosystem instead of RaspiOS customizations
- Modular service composition
- Version-controlled image configurations
- Automated builds and updates
- Custom base distributions (in development)
### Why not use standard Debian ARM64?
Standard Debian ARM64 lacks Raspberry Pi-specific drivers:
- RP1 chip drivers (Ethernet, USB, GPIO on Pi 5)
- Optimized WiFi/Bluetooth firmware
- Hardware acceleration
- Bootloader configuration
This project combines Debian with RaspiOS kernel/firmware for full hardware support.
---
## Build Questions
### How long does a build take?
**Typical build times**:
- Base image: 10-15 minutes
- With services: 15-30 minutes
- Full stack (all services): 30-45 minutes
**Factors**:
- Internet speed (downloading base images, packages)
- CPU speed (QEMU emulation)
- RAM available (QEMU VM performance)
- Disk speed (image operations)
### Can I build on macOS or Windows?
Not directly. The build system requires Linux for:
- Loop device support (partition mounting)
- QEMU ARM64 with KVM (optional but faster)
- ext4 filesystem tools
**Options**:
- Use WSL2 on Windows
- Use a Linux VM on macOS
- Use GitHub Actions (no local build needed)
### Why does QEMU use so much RAM?
QEMU allocates RAM for:
- Guest VM (QEMU_RAM, typically 4-8GB)
- Host overhead (1-2GB)
- Disk cache (varies)
**Reduce RAM usage**:
```bash
# In config.sh
QEMU_RAM="2G" # Minimum for base builds
QEMU_CPUS="2" # Reduce CPU count
```
**Note**: Lower RAM may cause build failures for images with many packages.
### Can I build multiple images in parallel?
Yes, but consider resources:
```bash
# Parallel builds
./bin/autobuild --image debian &
./bin/autobuild --image debian/qemu &
wait
```
**Requirements per build**:
- RAM: QEMU_RAM + 2GB overhead
- Disk: IMAGE_SIZE * 3 (source + output + temp)
- CPU: QEMU_CPUS cores
**Example**: 2 parallel builds with QEMU_RAM=4G requires 12GB total RAM.
### Why is PiShrink so slow?
PiShrink:
- Resizes ext4 filesystem
- Compresses with xz (CPU-intensive)
- Verifies integrity
**Speed up**:
```bash
# Skip PiShrink
./bin/autobuild --image debian --skip-compress
# Or use faster compression
xz -T0 -1 image.img # Parallel, lower compression
```
---
## Hardware Questions
### Which Raspberry Pi models are supported?
**Supported**:
- Raspberry Pi 4 (all RAM variants)
- Raspberry Pi 5 (all RAM variants)
**Not supported**:
- Raspberry Pi 3 and earlier (32-bit ARM)
- Raspberry Pi Zero (32-bit, insufficient resources)
- Raspberry Pi Compute Module (untested)
**Reason**: This project uses ARM64 (64-bit) Debian. Pi 3 and earlier are 32-bit ARMv7.
### Do I need a specific SD card?
**Recommended**:
- Class 10 or better
- U1/U3 for video applications
- A1/A2 for better random I/O
- Reputable brands (SanDisk, Samsung, Kingston)
**Minimum size**:
- Base image: 8GB
- With services: 16GB
- Full stack: 32GB
**Avoid**:
- Generic/cheap cards (reliability issues)
- Cards older than 5 years (wear)
### Can I use USB SSD instead of SD card?
Yes! Benefits:
- Faster I/O
- Better reliability
- Longer lifespan
**Process**:
```bash
# Flash to USB SSD (same as SD card)
xz -dc image.img.xz | sudo dd of=/dev/sdX bs=4M status=progress
# Boot from USB (Raspberry Pi 4/5 support USB boot)
```
**Raspberry Pi 5**: Native USB boot support
**Raspberry Pi 4**: Update bootloader for USB boot
### Does WiFi work out of the box?
Yes, with `firmware-brcm80211` package (included in base service).
**Supported adapters**:
- Built-in WiFi on Raspberry Pi 4/5
- Most USB WiFi adapters (Realtek, Atheros, etc.)
**Check compatibility**:
```bash
# After boot, check WiFi interface
ip link show wlan0
# Check firmware
dmesg | grep brcmfmac
```
### Does Bluetooth work?
Yes, Bluetooth firmware is included.
**Usage**:
```bash
# Check Bluetooth
bluetoothctl
scan on
```
**Pairing**:
- Use `bluetoothctl` CLI
- Or install GUI: `apt install blueman`
---
## Service Questions
### Can I run Home Assistant without Incus?
No. The `haos` service requires Incus to run Home Assistant OS as a VM.
**Alternative**: Use Home Assistant Container (Docker):
```bash
# Add to custom service
docker run -d --name homeassistant \
-v /home/pi/homeassistant:/config \
--network=host \
ghcr.io/home-assistant/home-assistant:stable
```
### How much RAM do services need?
**Service RAM usage**:
- base: ~500MB
- docker: ~200MB + containers
- qemu (Incus): ~100MB + VMs/containers
- haos VM: 4GB (configurable)
- openwrt container: ~100MB
- hotspot: ~50MB
**Total for full stack**:
- Minimum: 4GB (Pi 5 8GB recommended)
- Comfortable: 8GB
### Can I disable a service after building?
Yes:
```bash
# Stop service
sudo systemctl stop service-name
# Disable service
sudo systemctl disable service-name
# Or remove completely
sudo apt remove package-name
```
**Example**: Disable Home Assistant:
```bash
sudo incus stop haos
sudo incus delete haos
```
### How do I add my own service?
See [Services - Creating Custom Services](Services/#creating-custom-services).
Quick version:
1. Create `images/debian/services/myservice/`
2. Add `setup.sh` (package installation)
3. Add `first-boot/init.sh` (runtime config)
4. Build: `./bin/autobuild --image debian/myservice`
---
## Network Questions
### What are br-wan and br-lan?
**Network bridges** for consistent service networking:
**br-wan** (always created):
- Connects to WAN (internet)
- DHCP client
- Attached to eth0
**br-lan** (created if eth1 exists):
- LAN for internal services
- DHCP server (192.168.10.0/24)
- NAT enabled
- Attached to eth1
**Why bridges?**
- Services attach to bridges, not physical interfaces
- Allows flexible network configuration
- Supports VMs/containers without network conflicts
### How do I use a single NIC?
It works automatically. If only eth0 detected:
- br-wan created (WAN)
- br-lan not created
- Services use br-wan
### How do I set static IP?
See [Configuration Reference - Static IP](Configuration-Reference/#static-ip).
### Can I use WiFi for WAN instead of Ethernet?
Yes, configure NetworkManager to bridge WiFi to br-wan:
```bash
# Connect WiFi
sudo nmcli dev wifi connect "SSID" password "PASSWORD"
# Bridge WiFi to br-wan (advanced - requires network downtime)
sudo nmcli con add type bridge con-name br-wan ifname br-wan
sudo nmcli con add type wifi slave-type bridge con-name wlan0 ifname wlan0 master br-wan ssid "SSID"
```
**Warning**: WiFi bridging may be unreliable. Ethernet recommended for WAN.
---
## Update Questions
### How do I update the system?
```bash
sudo apt update
sudo apt upgrade -y
```
This updates:
- Debian packages (from Debian repos)
- RaspiOS kernel/firmware (from RaspiOS repo)
### Will updates break hardware support?
No. APT pinning ensures RaspiOS kernel/firmware packages update from RaspiOS repository, maintaining hardware support.
**Pinning config**: `/etc/apt/preferences.d/raspi-pin`
### How do I update to a newer Debian version?
**Not recommended while Debian 13 (Trixie) is testing/unstable.**
When Debian 13 is stable:
```bash
# Standard Debian upgrade process
sudo apt update
sudo apt full-upgrade
sudo reboot
```
Or rebuild image with newer Debian.
### Can I upgrade the kernel manually?
Not recommended. Kernel updates via APT automatically.
**To force kernel update**:
```bash
sudo apt update
sudo apt install --reinstall raspberrypi-kernel
sudo reboot
```
---
## Troubleshooting Questions
### Image won't boot (rainbow screen)
**Causes**:
- Corrupted SD card
- Incomplete flash
- Incompatible Raspberry Pi model
**Solutions**:
1. Verify image integrity:
```bash
sha256sum image.img.xz
```
2. Re-flash:
```bash
xz -dc image.img.xz | sudo dd of=/dev/sdX bs=4M status=progress conv=fsync
sync
```
3. Try different SD card
4. Check UART output for errors
### Can't login (credentials don't work)
**Default credentials**:
- Username: `pi`
- Password: `raspberry`
**If still fails**:
- Caps Lock enabled?
- Keyboard layout (US by default)
- Wait for first-boot to complete (3-5 minutes)
**Reset password** (mount SD card on another Linux machine):
```bash
sudo mount /dev/sdX2 /mnt
sudo chroot /mnt
passwd pi
exit
sudo umount /mnt
```
### No network / can't get IP
**Troubleshooting**:
1. Check cable:
```bash
ip link show eth0
# Should show "state UP"
```
2. Check DHCP:
```bash
sudo nmcli con up br-wan
```
3. Check NetworkManager:
```bash
sudo systemctl status NetworkManager
```
4. Manual IP (see "How do I set static IP?" above)
### Service didn't start
**Check first-boot logs**:
```bash
sudo journalctl -u services-first-boot
```
**Check service status**:
```bash
sudo systemctl status docker
sudo incus list
```
**Common issues**:
- First-boot still running (wait 5-10 min)
- Insufficient disk space (`df -h`)
- Network unavailable during first-boot
### Build failed in GitHub Actions
**Check logs**:
1. Go to Actions tab
2. Click failed workflow
3. Click failed job
4. Review logs
**Common issues**:
- QEMU timeout (increase QEMU_TIMEOUT)
- Out of disk space (reduce IMAGE_SIZE)
- Network error (retry build)
---
## Advanced Questions
### Can I use a different base distribution?
**Experimental support** for non-Debian distributions is in development.
**Current**: Only Debian 13 (Trixie) supported
**Future**: Ubuntu, Fedora, Alpine planned
**Why Debian-only currently?**
- Services use APT (Debian package manager)
- RaspiOS repository is Debian-based
### Can I cross-compile instead of using QEMU?
Theoretically yes, but:
- Complex to set up
- Requires ARM64 toolchain
- Package installation still needs emulation or chroot
**QEMU advantages**:
- Native ARM64 environment
- No cross-compilation issues
- Simpler workflow
### Can I build for Raspberry Pi 3 (32-bit)?
Not currently. This project uses ARM64 (64-bit) Debian.
**For Pi 3**:
- Use standard Raspberry Pi OS (32-bit)
- Or adapt this project for armhf (significant work)
### How do I contribute?
See [CONTRIBUTING.md](../CONTRIBUTING.md) for guidelines.
Quick start:
1. Fork repository
2. Create feature branch
3. Make changes
4. Test builds
5. Submit pull request
---
## Support
**Can't find your answer?**
- [Check Troubleshooting guide](Troubleshooting.md)
- [Open an issue](https://github.com/Pikatsuto/raspberry-builds/issues)
- [Start a discussion](https://github.com/Pikatsuto/raspberry-builds/discussions)

274
Getting-Started.md Normal file

@ -0,0 +1,274 @@
# Getting Started
This guide will walk you through installing dependencies, building your first image, and flashing it to an SD card.
## Prerequisites
### System Requirements
- **OS**: Linux (Debian/Ubuntu recommended)
- **RAM**: 8GB minimum (16GB recommended for QEMU builds)
- **Disk**: 20GB free space per image
- **CPU**: x86_64 with virtualization support (for QEMU ARM64 emulation)
### Required Packages
Install build dependencies:
```bash
sudo apt update
sudo apt install -y \
parted \
e2fsprogs \
dosfstools \
qemu-utils \
rsync \
xz-utils \
genisoimage \
qemu-system-aarch64 \
qemu-efi-aarch64
```
**Package purposes:**
- `parted`, `e2fsprogs`, `dosfstools` - Partition and filesystem tools
- `qemu-utils` - Image format conversion
- `rsync` - Efficient file copying
- `xz-utils` - Compression/decompression
- `genisoimage` - Create ISO images for setup scripts
- `qemu-system-aarch64` - ARM64 emulation for native package installation
### Optional: PiShrink
For image compression (automatically downloaded if not present):
```bash
wget https://raw.githubusercontent.com/Drewsif/PiShrink/master/pishrink.sh
chmod +x pishrink.sh
sudo mv pishrink.sh /usr/local/bin/
```
## Installation
Clone the repository:
```bash
git clone https://github.com/Pikatsuto/raspberry-builds.git
cd raspberry-builds
```
## Building Your First Image
### Option 1: Base Debian Image
Build a minimal Debian image with RaspiOS kernel:
```bash
./bin/autobuild --image debian
```
This will:
1. Download Raspberry Pi OS Lite and Debian 13 cloud images
2. Launch QEMU ARM64 VM
3. Install RaspiOS kernel and firmware packages
4. Merge RaspiOS boot with Debian rootfs
5. Compress the final image
**Output**: `debian-base.img.xz` (approximately 2-3GB compressed)
**Build time**: 15-30 minutes (depending on internet speed and CPU)
### Option 2: Image with Services
Build an image with Docker and Incus:
```bash
./bin/autobuild --image debian/qemu+docker
```
Available service combinations:
- `debian/qemu+docker` - Incus + Docker Engine
- `debian/qemu+haos` - Incus + Home Assistant OS
- `debian/qemu+openwrt+hotspot` - Incus + OpenWrt + WiFi AP
- `debian/qemu+docker+openwrt+hotspot+haos` - Full stack
### Build Options
Speed up subsequent builds:
```bash
# Skip base image downloads (use cached)
./bin/autobuild --image debian --skip-download
# Skip QEMU setup (use existing Debian image)
./bin/autobuild --image debian --skip-qemu
# Skip compression
./bin/autobuild --image debian --skip-compress
# Build all images defined in .github/images.txt
./bin/autobuild --all-images
# List available images
./bin/autobuild --list-images
```
## Flashing to SD Card
### Find Your SD Card Device
```bash
# Before inserting SD card
lsblk
# Insert SD card, then run again
lsblk
# Look for the new device (e.g., /dev/sdc, /dev/mmcblk0)
```
**Warning**: Double-check the device name! Using the wrong device will destroy data.
### Flash the Image
Decompress and flash in one command:
```bash
# If image is compressed (.xz)
xz -dc debian-base.img.xz | sudo dd of=/dev/sdX bs=4M status=progress conv=fsync
# If image is already decompressed (.img)
sudo dd if=debian-base.img of=/dev/sdX bs=4M status=progress conv=fsync
```
Replace `/dev/sdX` with your SD card device.
Sync and eject:
```bash
sync
sudo eject /dev/sdX
```
## First Boot
### Default Credentials
**Username**: `pi`
**Password**: `raspberry`
**Important**: Change the default password immediately after first login!
```bash
passwd
```
### Network Configuration
**Default**: DHCP on br-wan bridge. IP displayed in MOTD on login.
**Static IP**: See [Configuration Reference - Static IP](Configuration-Reference/#static-ip)
### First Boot Process
**Duration**: 3-5 minutes for automatic setup (partition resize, hardware detection, service initialization).
**Details**: See [Hardware Detection - Detection Sequence](Hardware-Detection/#detection-sequence) for complete timeline and monitoring commands.
## Accessing Services
Services display access information in the MOTD (Message of the Day) on login.
### Docker + Portainer
```
Portainer UI: https://<raspberry-ip>:9443
Username: admin
Password: (set on first access)
```
### Home Assistant OS
```
Home Assistant: http://<raspberry-ip>:8123
(First boot setup takes 5-10 minutes)
```
### OpenWrt
```
OpenWrt LuCI: http://192.168.10.1
Username: root
Password: (none - set on first access)
```
### WiFi Hotspot
If WiFi adapters detected:
```
SSID: RaspberryPi-5G (or RaspberryPi-2.4G)
Password: raspberry
```
## Verifying the Build
### Check Kernel
```bash
uname -r
# Should show: 6.6.x-rpi-v8 or 6.6.x-rpi-2712
```
### Check RaspiOS Packages
```bash
dpkg -l | grep raspberrypi
# Should show:
# - raspberrypi-kernel
# - raspberrypi-bootloader
# - libraspberrypi0
```
### Test Hardware
```bash
# Check WiFi/Bluetooth firmware
dmesg | grep brcmfmac
# Check RP1 drivers (Raspberry Pi 5)
lsmod | grep rp1
# List USB devices
lsusb
# Check network bridges
ip link show
# Should show: br-wan, and br-lan if dual NIC
```
## Updating the System
Safe system updates:
```bash
sudo apt update
sudo apt upgrade -y
```
RaspiOS kernel and firmware packages update from the RaspiOS repository, while all other packages update from Debian repositories. No special handling required.
## Next Steps
- [Learn about the architecture](Architecture.md)
- [Explore available services](Available-Services.md)
- [Create a custom image](Custom-Images.md)
- [Set up GitHub Actions for automated builds](GitHub-Actions.md)
## Troubleshooting
For common issues and solutions, see:
- [Troubleshooting - Build Issues](Troubleshooting/#build-issues)
- [Troubleshooting - Boot Issues](Troubleshooting/#boot-issues)
- [Troubleshooting - Service Issues](Troubleshooting/#service-issues)
- [FAQ](FAQ.md)
- [Open an issue](https://github.com/Pikatsuto/raspberry-builds/issues)

902
GitHub-Actions.md

@ -1,529 +1,523 @@
# GitHub Actions - Automated Build System
# GitHub Actions CI/CD
This page documents the automated build system powered by GitHub Actions. The workflow automatically builds all Raspberry Pi images, compresses them, and creates GitHub releases with ready-to-flash `.img.xz` files.
Automated image builds and releases using GitHub Actions.
## Overview
The GitHub Actions workflow (`.github/workflows/build-images.yml`) provides a complete CI/CD pipeline for building Raspberry Pi images. It automatically:
The project includes a sophisticated multi-stage CI/CD pipeline that:
- Builds images in parallel
- Creates GitHub releases automatically
- Uploads compressed images as release assets
- Supports multiple branch strategies (stable, test, preview)
- Runs daily builds
1. Detects all available images in the `images/` directory
2. Downloads base images (RaspiOS + Debian)
3. Executes setup scripts in QEMU ARM64
4. Creates hybrid images
5. Compresses images with PiShrink
6. Creates GitHub releases with downloadable assets
## Workflow Architecture
## Workflow Triggers
### Build Pipeline (`.github/workflows/build-images.yml`)
The workflow runs automatically on:
### 1. Push Events
- **Trigger**: Any push to any branch
- **Branches**: `**` (all branches)
- **Purpose**: Immediate build on code changes
- **Release Type**:
- `main` branch → Stable release
- Other branches → Pre-release
### 2. Scheduled Builds
- **Trigger**: Daily at 2:00 AM UTC
- **Schedule**: `cron: '0 2 * * *'`
- **Purpose**: Fetch latest Debian and RaspiOS base images
- **Benefit**: Images stay up-to-date with security patches and new features
### 3. Manual Dispatch
- **Trigger**: Manual workflow run via GitHub Actions UI
- **Purpose**: Build on-demand without pushing code
- **Usage**: Go to Actions tab → Select workflow → "Run workflow"
## Build Architecture
The workflow uses a **4-stage parallel build system** with artifact caching to optimize build time and resource usage.
**6-job multi-stage pipeline**:
```
┌─────────────────────────────────────────────────────────────┐
│ Job 1: detect-images │
│ └─ Scans images/ directory and outputs image list │
└─────────────────────────────────────────────────────────────┘
detect-images
↓
┌─────────────────────────────────────────────────────────────┐
│ Job 2: stage1-download (Parallel per image) │
│ ├─ Downloads RaspiOS and Debian base images │
│ ├─ Creates setup.iso from setup.sh + setupfiles/ │
│ └─ Uploads artifacts: *.img, *.raw, seed.img, setup.iso │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Job 3: stage2-qemu (Parallel per image) │
│ ├─ Downloads artifacts from stage1 │
│ ├─ Launches QEMU ARM64 with Debian + setup.iso │
│ ├─ Executes setup.sh in native ARM64 environment │
│ ├─ Installs RaspiOS kernel/firmware via APT │
│ └─ Uploads configured Debian image │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Job 4: stage3-build (Parallel per image) │
│ ├─ Downloads RaspiOS from stage1 │
│ ├─ Downloads configured Debian from stage2 │
│ ├─ Runs merge-debian-raspios.sh │
│ └─ Uploads hybrid image (rpi-*.img) │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Job 5: stage4-compress (Parallel per image) │
│ ├─ Downloads hybrid image from stage3 │
│ ├─ Compresses with PiShrink + xz │
│ └─ Uploads final image (rpi-*.img.xz) │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Job 6: create-release │
│ ├─ Downloads all final images (rpi-*.img.xz) │
│ ├─ Generates release name and tag │
│ ├─ Creates GitHub release with build info │
│ └─ Uploads all .img.xz files as release assets │
└─────────────────────────────────────────────────────────────┘
├─→ stage1-2-download-qemu (parallel matrix)
│ └─→ stage3-build (sequential)
│ └─→ stage4-compress (sequential)
│
└─→ create-release (parallel)
└─→ cleanup-release (final)
```
## Detailed Job Breakdown
### Job Descriptions
### Job 1: detect-images
**1. detect-images**
- Reads `.github/images.txt`
- Outputs list of images to build
- Sets up matrix for parallel builds
**Purpose**: Dynamic image detection
**2. stage1-2-download-qemu**
- Downloads RaspiOS and Debian base images
- Runs QEMU setup (installs packages)
- Uploads artifacts: Debian image, setup files
**Steps**:
1. Checkout repository
2. Scan `images/` directory
3. Extract directory names (basename)
4. Convert to JSON array for matrix strategy
5. Output image list to GitHub Actions outputs
**3. create-release**
- Determines release tag (daily vs push)
- Creates GitHub release (or updates existing)
- Sets release as pre-release for non-main branches
**Output**: JSON array like `["raspivirt-incus", "raspivirt-incus+docker"]`
**4. stage3-build**
- Downloads artifacts from stage1-2
- Merges RaspiOS + Debian
- Uploads merged image artifact
**5. stage4-compress**
- Downloads merged image
- Compresses with PiShrink + xz
- Uploads compressed image to GitHub release
**6. cleanup-release**
- Deletes release if no assets uploaded (failure)
- Always runs, even on failure
---
## Setting Up CI/CD
### 1. Fork/Clone Repository
**Code**:
```bash
images=$(ls -d images/*/ | xargs -n 1 basename | jq -R -s -c 'split("\n")[:-1]')
git clone https://github.com/Pikatsuto/raspberry-builds.git
cd raspberry-builds
```
### Job 2: stage1-download
### 2. Configure Images to Build
**Purpose**: Download and prepare base images
**Strategy**: Matrix parallelization (one job per image)
**Steps**:
1. **Free disk space**: Remove unused packages to prevent disk full errors
- Removes: .NET, Android SDK, GHC, CodeQL, Docker images
- Frees ~40GB of space
2. **Install dependencies**: `wget`, `genisoimage`, `xz-utils`
3. **Run autobuild stage 1**: `./bin/autobuild --image <name> --stage 1`
- Downloads RaspiOS image (if missing)
- Downloads Debian image (if missing)
- Creates `setup.iso` from `setup.sh` + `setupfiles/`
- Regenerates cloud-init `seed.img`
4. **Upload artifacts**:
- RaspiOS image (`*.img`)
- Debian image (`*.raw`)
- Cloud-init seed (`seed.img`)
- Setup ISO (`setup.iso`)
- Retention: 1 day
**Artifact Size**: ~2-4GB per image
### Job 3: stage2-qemu
**Purpose**: Execute setup script in QEMU ARM64
**Strategy**: Matrix parallelization (one job per image)
**Timeout**: 60 minutes (safety limit for long-running setups)
**Steps**:
1. **Install QEMU dependencies**:
- `qemu-system-aarch64`: ARM64 system emulator
- `qemu-utils`: Image utilities
- `qemu-efi-aarch64`: UEFI firmware for ARM64
2. **Download artifacts from stage1**:
- RaspiOS image (needed for reference)
- Debian image (modified during setup)
- Cloud-init seed (first-boot configuration)
- Setup ISO (contains setup.sh + files)
3. **Run autobuild stage 2**: `./bin/autobuild --image <name> --stage 2`
- Creates working copy of Debian image
- Launches QEMU ARM64 with:
- Debian image as main disk
- Cloud-init seed for user configuration
- Setup ISO for package installation
- QEMU executes setup.sh in native ARM64:
- System update (`apt update && apt upgrade`)
- Essential packages (curl, wget, sudo, SSH, etc.)
- RaspiOS repository configuration + APT pinning
- RaspiOS kernel/firmware installation
- Image-specific software (Incus, Docker, etc.)
- Service configuration
- Waits for automatic poweroff signal
4. **Upload configured Debian image**:
- Debian image with RaspiOS kernel + custom software
- Retention: 1 day
- Compression disabled (speed optimization)
**Artifact Size**: ~3-6GB per image
**Key Feature**: Native ARM64 execution ensures compatibility and proper package installation
### Job 4: stage3-build
**Purpose**: Create hybrid Raspberry Pi image
**Strategy**: Matrix parallelization (one job per image)
**Steps**:
1. **Install build dependencies**:
- `parted`: Partition management
- `e2fsprogs`: ext4 filesystem tools
- `dosfstools`: FAT32 filesystem tools
- `rsync`: Efficient file copying
- `qemu-utils`: Image conversion
2. **Enable loop devices**: Load kernel module for disk image mounting
3. **Download artifacts**:
- RaspiOS image from stage1 (boot partition source)
- Configured Debian image from stage2 (rootfs source)
4. **Run autobuild stage 3**: `./bin/autobuild --image <name> --stage 3`
- Executes `merge-debian-raspios.sh`:
- Creates output image based on RaspiOS
- Resizes to configured IMAGE_SIZE
- Mounts both images via loop devices
- Deletes RaspiOS root partition content
- Copies Debian rootfs (with RaspiOS kernel pre-installed)
- Restores RaspiOS `/etc/fstab`
- Creates `/boot/firmware` mount point
- Unmounts and cleans up
5. **Upload hybrid image**:
- Final bootable image (`rpi-*.img`)
- Retention: 1 day
- Compression disabled (PiShrink does this in stage 4)
**Artifact Size**: Matches IMAGE_SIZE (e.g., 6GB for raspivirt-incus)
**Result**: Bootable Raspberry Pi image ready for compression
### Job 5: stage4-compress
**Purpose**: Shrink and compress final images
**Strategy**: Matrix parallelization (one job per image)
**Steps**:
1. **Install PiShrink**:
- Downloads from official repository
- Moves to `/usr/local/bin/`
2. **Download hybrid image from stage3**
3. **Run autobuild stage 4**: Compresses with PiShrink
- Executes: `sudo pishrink.sh -aZ <image>.img`
- `-a`: Aggressive compression
- `-Z`: Parallel xz compression
- Process:
- Shrinks filesystem to minimum size
- Removes unused space from partition
- Truncates image file
- Compresses to `.img.xz`
4. **Upload final image**:
- Compressed image (`rpi-*.img.xz`)
- Retention: 7 days
- Typical compression: 6GB → 1-2GB
**Artifact Size**: ~1-2GB per image (compressed)
**Compression Ratio**: Typically 60-80% size reduction
### Job 6: create-release
**Purpose**: Create GitHub release with all final images
**Permissions**: `contents: write` (required for release creation)
**Steps**:
1. **Download all compressed images**:
- Pattern: `final-image-*`
- Merges all image artifacts
2. **List built images**: Display file sizes for verification
3. **Generate release information**:
- **Release tag**: `v<YYYY-MM-DD>-<HHMM>` (e.g., `v2025-12-01-1430`)
- **Release name**:
- Main branch: `Release <YYYY-MM-DD>`
- Other branches: `Pre-release <YYYY-MM-DD> (<branch-name>)`
- **Pre-release flag**:
- Main branch: `false` (stable release)
- Other branches: `true` (experimental release)
4. **Create release body**: Markdown with:
- Build date
- Flash instructions
- Build information (branch, commit, workflow link)
- Documentation links
5. **Create GitHub release**:
- Uses `gh release create` CLI
- Uploads all `.img.xz` files as assets
- Sets pre-release flag based on branch
6. **Cleanup**: Remove large files to free disk space
**Release Example**:
```
Tag: v2025-12-01-1430
Name: Release 2025-12-01
Assets:
- rpi-raspivirt-incus.img.xz (1.2 GB)
- rpi-raspivirt-incus+docker.img.xz (1.5 GB)
```
## Artifact Management
### Artifact Flow
Edit `.github/images.txt`:
```
stage1 → stage2: Base images (RaspiOS, Debian, seed.img, setup.iso)
stage2 → stage3: Configured Debian image
stage3 → stage4: Hybrid image (rpi-*.img)
stage4 → release: Final compressed image (rpi-*.img.xz)
debian
debian/qemu+docker
debian/qemu+haos
debian/qemu+docker+openwrt+hotspot+haos
```
### Retention Policy
One image per line. Comments with `#` supported.
- **Stage 1-4 artifacts**: 1 day (temporary build artifacts)
- **Final compressed images**: 7 days (backup before release)
- **Release assets**: Permanent (until manually deleted)
### 3. Push to GitHub
### Compression Strategy
```bash
git add .github/images.txt
git commit -m "Configure CI/CD images"
git push origin main
```
- **Stage 1-3**: No compression (speed optimization)
- **Stage 4**: Full compression with PiShrink + xz
- **Rationale**: Intermediate artifacts are deleted quickly; only final images need compression
### 4. Enable GitHub Actions
## Optimization Strategies
- Go to repository Settings → Actions → General
- Enable "Allow all actions and reusable workflows"
- Save
### 1. Parallel Execution
- All images build simultaneously (matrix strategy)
- Total build time = slowest image (not sum of all images)
### 5. Enable GitHub Pages (for docs)
### 2. Stage Separation
- Each stage uploads artifacts for the next stage
- Failed stages don't rebuild earlier stages
- Easy to retry individual stages
- Go to repository Settings → Pages
- Source: GitHub Actions
- Save
### 3. Artifact Caching
- Reuses artifacts across jobs
- Avoids redundant downloads and builds
- Reduces total workflow time
### 6. Trigger First Build
### 4. Disk Space Management
- Removes unused software before builds
- Cleans up artifacts after each stage
- Prevents out-of-disk errors on GitHub runners
**Manual trigger**:
- Go to Actions tab
- Select "Build Raspberry Pi Images" workflow
- Click "Run workflow"
- Select branch (main)
- Click "Run workflow"
### 5. Fail-Fast Disabled
- `fail-fast: false` in stage2 (QEMU)
- One failing image doesn't stop others
- Maximizes successful builds
**Automatic trigger**:
- Push changes to `images/**`, `bin/**`, or `.github/**`
- Wait for daily cron (2:00 AM UTC)
## Environment Variables
---
## Triggers
### Push Trigger
```yaml
on:
push:
branches:
- main
- test
- preview
paths:
- 'images/**'
- 'bin/**'
- '.github/**'
```
**Behavior**:
- Only triggers when image configs or build scripts change
- Ignores documentation-only changes
- Supports main, test, preview branches
### Schedule Trigger
```yaml
on:
schedule:
- cron: '0 2 * * *' # Daily at 2:00 AM UTC
```
**Behavior**:
- Runs daily to capture latest base images
- Builds all images from `.github/images.txt`
- Creates `daily-YYYY-MM-DD` release
### Manual Trigger
```yaml
on:
workflow_dispatch:
```
**Behavior**:
- Run from GitHub Actions UI
- Useful for testing or on-demand builds
- Supports branch selection
---
## Release Strategy
### Release Tagging
**Daily builds** (cron):
```
Tag: daily-YYYY-MM-DD
Example: daily-2024-12-08
```
- Overwrites existing daily release
- Always pre-release
**Push builds**:
```
Tag: vYYYY-MM-DD-HHMM
Example: v2024-12-08-1430
```
- Unique timestamp for each build
- Pre-release for test/preview branches
- Stable release for main branch
### Branch Strategy
**main branch**:
- Stable releases
- No pre-release flag
- Recommended for production
**test branch**:
- Pre-releases with warning banner
- For testing new features
- May contain bugs
**preview branch**:
- Experimental pre-releases
- Bleeding-edge features
- Use at your own risk
### Release Notes
Auto-generated release notes include:
- List of images built
- Download links
- SHA256 checksums
- Installation instructions
- Changelog (if commits since last release)
**Example**:
```markdown
## Raspberry Pi Images - v2024-12-08-1430
### Images Built
- debian-base.img.xz
- debian-qemu-docker.img.xz
- debian-qemu-haos.img.xz
### Installation
1. Download image
2. Verify checksum
3. Flash to SD card:
```bash
xz -dc image.img.xz | sudo dd of=/dev/sdX bs=4M status=progress
```
### Checksums (SHA256)
- debian-base.img.xz: abc123...
- debian-qemu-docker.img.xz: def456...
```
---
## Artifacts
### Uploaded Artifacts (Intermediate)
Used for passing data between stages:
**stage1-2-download-qemu**:
- `debian-<image>-image` - Configured Debian image
- `setup-iso-<image>` - Setup scripts
- `setupfiles-<image>` - Configuration files
**stage3-build**:
- `hybrid-<image>-image` - Merged image (uncompressed)
**Retention**: 1 day (deleted after stage4 completes)
### Release Assets (Final)
Uploaded to GitHub Releases:
- `<image-name>.img.xz` - Compressed image
- `<image-name>.img.xz.sha256` - Checksum
**Retention**: Permanent (until manually deleted)
---
## Customizing Workflow
### Adjust Runner Resources
For faster builds, use larger runners:
```yaml
jobs:
stage1-2-download-qemu:
runs-on: ubuntu-latest # Change to ubuntu-latest-4-cores or custom runner
```
**Options**:
- `ubuntu-latest` - 2 cores, 7GB RAM (free)
- `ubuntu-latest-4-cores` - 4 cores, 16GB RAM (paid)
- Self-hosted runners
### Adjust QEMU Timeout
Increase timeout for slow builds:
### Global Environment
```yaml
env:
DEBIAN_FRONTEND: noninteractive
QEMU_TIMEOUT: 3600 # 1 hour (default: 1800 = 30 min)
```
- Prevents interactive prompts during apt operations
- Required for unattended package installation
### Job-Specific Variables
- Passed via autobuild script stages
- Configured in `images/<name>/config.sh`:
- `OUTPUT_IMAGE`: Final filename
- `IMAGE_SIZE`: Target image size
- `QEMU_RAM`: RAM for QEMU
- `QEMU_CPUS`: CPU cores for QEMU
### Parallel Image Builds
## Release Versioning
By default, images build in parallel via matrix strategy. To limit parallelism:
### Version Format
- **Pattern**: `vYYYY-MM-DD-HHMM`
- **Example**: `v2025-12-01-1430` (December 1, 2025 at 14:30)
- **Uniqueness**: Timestamp ensures unique tags
```yaml
strategy:
matrix:
image: ${{ fromJson(needs.detect-images.outputs.images) }}
max-parallel: 2 # Limit to 2 concurrent builds
```
### Release Types
### Skip Compression
#### Stable Release (main branch)
- **Name**: `Release YYYY-MM-DD`
- **Flag**: `prerelease: false`
- **Purpose**: Production-ready images
- **Visibility**: Featured on repository homepage
To speed up testing builds:
#### Pre-Release (other branches)
- **Name**: `Pre-release YYYY-MM-DD (branch-name)`
- **Flag**: `prerelease: true`
- **Purpose**: Testing and development
- **Visibility**: Listed but marked as pre-release
```yaml
- name: Build image
run: ./bin/autobuild --image ${{ matrix.image }} --skip-compress
```
## Build Duration
### Custom Base Images
Typical build times on GitHub runners (ubuntu-latest):
Override base image URLs via environment variables:
- **Stage 1 (Download)**: 3-5 minutes per image
- **Stage 2 (QEMU)**: 15-30 minutes per image (most time-consuming)
- **Stage 3 (Build)**: 5-10 minutes per image
- **Stage 4 (Compress)**: 3-5 minutes per image
- **Stage 5 (Release)**: 1-2 minutes
```yaml
env:
RASPIOS_URL: https://example.com/custom-raspios.img.xz
DEBIAN_URL: https://example.com/custom-debian.raw
```
**Total Time**: ~30-50 minutes for all images (parallel execution)
---
## Monitoring Builds
### Via GitHub UI
1. Go to repository **Actions** tab
2. Select workflow run
3. View job progress and logs
4. Check artifacts and releases
### GitHub Actions UI
### Via GitHub CLI
```bash
# List recent workflow runs
gh run list --workflow=build-images.yml
**View running builds**:
1. Go to Actions tab
2. Click on running workflow
3. Click on job to see logs
# Watch a running workflow
gh run watch <run-id>
**Check build status**:
- Green checkmark: Success
- Red X: Failure
- Yellow circle: In progress
# Download artifacts
gh run download <run-id>
```
### Logs
## Troubleshooting
**Download logs**:
1. Go to completed workflow run
2. Click "..." (three dots)
3. Select "Download log archive"
### Build Failures
**Key log sections**:
- **stage1-2**: QEMU boot, package installation
- **stage3**: Merge operation
- **stage4**: Compression, upload
#### Stage 1 Failures
- **Symptom**: Download errors
- **Cause**: Network issues, broken URLs
- **Solution**: Check base image URLs in `autobuild` script
### Debugging Failures
#### Stage 2 Failures (QEMU)
- **Symptom**: QEMU timeout or setup.sh errors
- **Cause**: Package installation failures, network issues in QEMU
- **Solution**: Check `setup.sh` syntax, verify package names
- **Debugging**: Add `set -x` to `setup.sh` for verbose output
**QEMU timeout**:
- Increase `QEMU_TIMEOUT`
- Check for interactive prompts in logs
- Verify network connectivity
#### Stage 3 Failures
- **Symptom**: Merge errors, loop device issues
- **Cause**: Insufficient permissions, partition layout issues
- **Solution**: Verify image formats, check merge script logic
**Merge failure**:
- Check disk space
- Verify base image URLs
- Review merge logs
#### Stage 4 Failures
- **Symptom**: PiShrink errors
- **Cause**: Filesystem corruption, insufficient disk space
- **Solution**: Check stage 3 output, verify filesystem integrity
**Upload failure**:
- Check GitHub token permissions
- Verify release exists
- Check artifact size limits (2GB per file)
### Disk Space Issues
- **Symptom**: "No space left on device"
- **Solution**: Already handled by "Free disk space" step
- **If persists**: Reduce IMAGE_SIZE in config.sh
### Release Creation Failures
- **Symptom**: Permission denied errors
- **Cause**: Missing `contents: write` permission
- **Solution**: Verify workflow permissions in YAML
## Customizing the Workflow
### Adding a New Build Stage
1. Add new job to `.github/workflows/build-images.yml`
2. Define dependencies with `needs: [previous-job]`
3. Add matrix strategy if parallel execution needed
4. Upload artifacts for next stage
5. Update artifact retention as needed
### Changing Build Frequency
Edit the `schedule` trigger:
```yaml
schedule:
# Build every 6 hours
- cron: '0 */6 * * *'
```
### Disabling Automatic Builds
Comment out unwanted triggers:
```yaml
# on:
# push:
# branches:
# - '**'
# schedule:
# - cron: '0 2 * * *'
workflow_dispatch: # Keep manual trigger only
```
## Best Practices
### For Image Developers
1. **Test locally first**: Run `./bin/autobuild --image <name>` before pushing
2. **Small iterations**: Commit small changes to debug issues faster
3. **Monitor logs**: Check Actions tab for build output
4. **Use branches**: Test in feature branches before merging to main
### For Repository Maintainers
1. **Review PRs carefully**: Malicious setup.sh could compromise runners
2. **Monitor disk usage**: Adjust retention policies if needed
3. **Update base images**: Keep RaspiOS and Debian URLs current
4. **Clean old releases**: Remove outdated releases periodically
---
## Security Considerations
### Runner Security
- Workflows run in isolated GitHub-hosted runners
- Each job runs in a fresh VM
- Artifacts are sandboxed
- No access to repository secrets (unless explicitly added)
### GitHub Token
### Image Security
- Setup scripts run with root privileges in QEMU
- Only trusted contributors should modify setup.sh
- Review all package installations
- Avoid hardcoded credentials
The workflow uses `GITHUB_TOKEN` for:
- Creating releases
- Uploading assets
- Downloading artifacts
### Release Security
- Releases are public by default
- Verify checksums before flashing images
- Use HTTPS for all downloads
**Permissions required**:
```yaml
permissions:
contents: write # Create releases, upload assets
actions: read # Download artifacts
```
## Performance Metrics
**Token is automatically provided by GitHub Actions** - no manual setup needed.
### Resource Usage (per image)
- **CPU**: 4 cores (QEMU)
- **RAM**: 8GB (QEMU)
- **Disk**: ~20GB peak usage
- **Network**: ~2GB download (base images)
### Secrets
### GitHub Actions Limits
- **Concurrent jobs**: 20 (free tier)
- **Job timeout**: 6 hours (we set 60 min for QEMU)
- **Artifact size**: 10GB per file
- **Total storage**: 500MB artifacts (free tier)
No secrets required for basic builds. Optional secrets:
## Future Improvements
**CUSTOM_REGISTRY_TOKEN**:
- For private Docker registries
- Add in Settings → Secrets → Actions
Potential enhancements:
**Usage**:
```yaml
env:
REGISTRY_TOKEN: ${{ secrets.CUSTOM_REGISTRY_TOKEN }}
```
1. **Caching base images**: Store RaspiOS/Debian images in GitHub cache
2. **Incremental builds**: Only rebuild changed images
3. **Build matrix**: Test multiple Debian versions
4. **Checksum verification**: Add SHA256 checksums to releases
5. **Build badges**: Display build status in README
6. **Multi-architecture**: Support x86_64 images for testing
---
## Related Documentation
## Cost Optimization
- **[Home](Home)**: Project overview
- **[Main README](../README.md)**: Complete documentation
- **[CLAUDE.md](../CLAUDE.md)**: Technical architecture details
- **GitHub Actions**: [Official documentation](https://docs.github.com/en/actions)
### Free Tier Limits
**GitHub Actions free tier** (public repos):
- Unlimited minutes
- 2 cores, 7GB RAM runners
- No artifact storage cost
**GitHub Actions free tier** (private repos):
- 2000 minutes/month
- Same runner specs
- Artifact storage counted
### Reduce Build Time
**Cache base images**:
- Base images cached between builds
- Use `--skip-download` when possible
**Reduce image count**:
- Remove unused images from `.github/images.txt`
- Build only on significant changes
**Use smaller images**:
- Reduce `IMAGE_SIZE` in config
- Skip unnecessary services
---
## Advanced Workflows
### Build on Pull Request
Add PR trigger to test changes before merge:
```yaml
on:
pull_request:
branches:
- main
paths:
- 'images/**'
- 'bin/**'
```
**Behavior**:
- Builds images without creating release
- Uploads artifacts for testing
- Blocks merge if build fails
### Multi-Architecture Builds
Add ARM64 runner for native builds (no QEMU emulation):
```yaml
jobs:
build:
runs-on: [self-hosted, linux, arm64]
```
**Benefits**:
- Faster builds (no emulation overhead)
- Lower resource usage
**Requirements**:
- ARM64 GitHub runner (Raspberry Pi 5, AWS Graviton, etc.)
### Notification on Failure
Send notifications via email/Slack/Discord:
```yaml
- name: Notify on failure
if: failure()
uses: actions/notify@v1
with:
webhook-url: ${{ secrets.SLACK_WEBHOOK }}
message: "Build failed: ${{ github.workflow }}"
```
---
## Example: Custom Workflow
**Scenario**: Build only on weekends, upload to custom S3 bucket
```yaml
name: Weekend Builds
on:
schedule:
- cron: '0 2 * * 6' # Saturday at 2 AM UTC
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build images
run: ./bin/autobuild --all-images
- name: Upload to S3
uses: aws-actions/upload-s3@v1
with:
aws-access-key-id: ${{ secrets.AWS_KEY }}
aws-secret-access-key: ${{ secrets.AWS_SECRET }}
bucket: my-rpi-images
source: images/**/*.img.xz
```
---
## Next Steps
- [Learn about hardware auto-detection](Hardware-Detection.md)
- [Create custom services](Services.md)
- [Troubleshoot build issues](Troubleshooting.md)

331
Hardware-Detection.md Normal file

@ -0,0 +1,331 @@
# Hardware Auto-Detection
The build system automatically detects and configures hardware on first boot.
## Supported Hardware
### Network Interfaces
**Ethernet (eth0, eth1)**
**Detection**:
- Scans for eth0 and eth1 during first boot
- Executed by: `services/base/first-boot/init.sh`
**Configuration**:
| Configuration | Bridge Setup | IP Assignment |
|---------------|--------------|---------------|
| Single NIC (eth0 only) | br-wan on eth0 | DHCP client |
| Dual NIC (eth0 + eth1) | br-wan on eth0<br>br-lan on eth1 | br-wan: DHCP client<br>br-lan: 192.168.10.254/24 |
**br-lan features** (when created):
- DHCP server (192.168.10.1-100)
- DNS forwarder (dnsmasq)
- NAT to br-wan
- Gateway for internal services
**Example detection log**:
```bash
# Check first-boot logs
sudo journalctl -u services-first-boot | grep "Network"
# Output:
# Detected eth0 and eth1
# Created br-wan on eth0 (WAN)
# Created br-lan on eth1 (LAN)
# DHCP server enabled on br-lan
```
---
### WiFi Adapters
**Wireless interfaces (wlan0, wlan1)**
**Detection**:
- Scans for wlan0 and wlan1 during first boot
- Executed by: `services/hotspot/first-boot/init.sh`
**Configuration**:
| Configuration | Access Point Setup |
|---------------|-------------------|
| No WiFi | Hotspot service disabled |
| Single WiFi (wlan0) | 5GHz AP on wlan0 |
| Dual WiFi (wlan0 + wlan1) | 2.4GHz AP on wlan0<br>5GHz AP on wlan1 |
**Access Point defaults**:
- SSID: `RaspberryPi-5G` or `RaspberryPi-2.4G`
- Password: `raspberry`
- Bridge: br-lan (if exists) or br-wan
**Bridge selection logic**:
```bash
if br-lan exists:
attach WiFi AP to br-lan (LAN network)
else:
attach WiFi AP to br-wan (WAN network)
```
**Example detection log**:
```bash
sudo journalctl -u services-first-boot | grep "WiFi"
# Output:
# Detected wlan0 and wlan1
# Configured 2.4GHz AP on wlan0 (RaspberryPi-2.4G)
# Configured 5GHz AP on wlan1 (RaspberryPi-5G)
# Attached to br-lan
```
**Customization**:
Edit `/etc/setupfiles/hostapd-5ghz.conf` or `hostapd-2.4ghz.conf`:
```ini
ssid=MyCustomSSID
wpa_passphrase=MySecurePassword
```
Then restart:
```bash
sudo systemctl restart hostapd-5ghz
```
---
### USB Zigbee Coordinators
**USB serial devices for Zigbee/Z-Wave**
**Detection**:
- Scans `/dev/ttyUSB*` and `/dev/ttyACM*`
- Matches vendor IDs for known Zigbee coordinators
- Executed by: `services/haos/first-boot/init.sh`
**Supported vendors**:
- dresden elektronik (ConBee/ConBee II)
- Texas Instruments (CC2652, CC1352)
- Silicon Labs (EFR32)
- ITead (Sonoff Zigbee dongles)
- FTDI-based coordinators
**Configuration**:
- USB device passed through to Home Assistant VM
- Uses vendor/product ID matching (survives USB port changes)
- Non-required passthrough (VM starts even if dongle unplugged)
**Example detection log**:
```bash
sudo journalctl -u services-first-boot | grep -i zigbee
# Output:
# Detected Zigbee coordinator: ConBee II
# USB Vendor: 1cf1, Product: 0030
# Passed through to Home Assistant VM (haos)
```
**Manual passthrough**:
```bash
# List USB devices
lsusb
# Pass through manually
incus config device add haos my-zigbee usb \
vendorid=1cf1 \
productid=0030
```
**Verification**:
```bash
# Check HAOS VM devices
incus config show haos
# Should show:
# devices:
# zigbee-dongle:
# productid: "0030"
# type: usb
# vendorid: 1cf1
```
---
## Detection Sequence
### First Boot Timeline
**1. rpi-first-boot.service** (runs once, before network)
- Expand root partition to fill SD card
- Set persistent network interface names (eth0, eth1, wlan0, wlan1)
- Reboot
**2. After reboot: services-first-boot.service** (runs once)
**Stage 1: Base network setup** (`base/first-boot/init.sh`)
- Detect eth0, eth1
- Create br-wan (always)
- Create br-lan (if eth1 exists)
- Configure DHCP server on br-lan
- Enable NAT
**Stage 2: Service initialization** (service-specific init.sh)
- qemu: Configure Incus networks
- docker: Start Portainer, Watchtower
- haos: Download HAOS image, create VM, detect Zigbee dongles
- openwrt: Download OpenWrt image, create container
- hotspot: Detect WiFi adapters, configure hostapd
**3. Normal boot**
- All services running
- Hardware configured
- Ready for use
**Total first-boot time**: 3-10 minutes (depends on services, network speed)
---
## Manual Hardware Configuration
### Add Network Bridge Manually
```bash
# Create bridge
sudo nmcli con add type bridge con-name br-custom ifname br-custom
# Add interface to bridge
sudo nmcli con add type bridge-slave con-name eth2 ifname eth2 master br-custom
# Set IP
sudo nmcli con mod br-custom ipv4.addresses 192.168.20.1/24
sudo nmcli con mod br-custom ipv4.method manual
# Bring up
sudo nmcli con up br-custom
```
### Add WiFi AP Manually
```bash
# Create hostapd config
sudo tee /etc/hostapd/hostapd-custom.conf <<EOF
interface=wlan2
driver=nl80211
ssid=MyCustomAP
hw_mode=a
channel=36
ieee80211n=1
ieee80211ac=1
wmm_enabled=1
wpa=2
wpa_passphrase=MyPassword
wpa_key_mgmt=WPA-PSK
rsn_pairwise=CCMP
EOF
# Create systemd service
sudo tee /etc/systemd/system/hostapd-custom.service <<EOF
[Unit]
Description=Custom WiFi AP
After=network.target
[Service]
ExecStart=/usr/sbin/hostapd /etc/hostapd/hostapd-custom.conf
[Install]
WantedBy=multi-user.target
EOF
# Enable and start
sudo systemctl enable hostapd-custom
sudo systemctl start hostapd-custom
```
### Pass USB Device to VM Manually
```bash
# List USB devices
lsusb
# Example output: Bus 001 Device 003: ID 10c4:ea60 Silicon Labs CP210x
# Pass through
incus config device add haos my-device usb \
vendorid=10c4 \
productid=ea60
```
---
## Hardware Detection Logs
### View Detection Logs
**First-boot service**:
```bash
sudo journalctl -u services-first-boot
```
**Specific hardware type**:
```bash
# Network detection
sudo journalctl -u services-first-boot | grep -i "network\|eth\|bridge"
# WiFi detection
sudo journalctl -u services-first-boot | grep -i "wifi\|wlan\|hostapd"
# USB detection
sudo journalctl -u services-first-boot | grep -i "usb\|zigbee\|dongle"
```
**All hardware events**:
```bash
sudo journalctl -u services-first-boot --no-pager
```
---
## Disabling Auto-Detection
### Disable Network Bridge Auto-Creation
Edit `/etc/setupfiles/rpi-first-boot.sh`:
```bash
# Comment out bridge creation
# create_bridges
```
Then delete and recreate first-boot service to apply changes (before first boot).
### Disable WiFi Hotspot
```bash
# Disable hostapd services
sudo systemctl disable hostapd-5ghz
sudo systemctl disable hostapd-2.4ghz
sudo systemctl stop hostapd-5ghz
sudo systemctl stop hostapd-2.4ghz
```
### Disable USB Passthrough
Edit `services/haos/first-boot/init.sh` and comment out the USB detection section before building.
---
## Future Hardware Detection
**Planned** (in development):
- Storage devices (auto-mount USB drives)
- Audio devices (auto-configure ALSA/PulseAudio)
- Camera modules (CSI/USB cameras)
- GPIO devices (I2C, SPI peripherals)
**Contributions welcome!** See [CONTRIBUTING.md](../CONTRIBUTING.md).
---
## Next Steps
- [Learn about creating custom services](Services.md)
- [Troubleshoot hardware issues](Troubleshooting.md)
- [Configure GitHub Actions CI/CD](GitHub-Actions.md)

274
Home.md

@ -1,225 +1,109 @@
# RPI-Dev - Hybrid Raspberry Pi Images Builder
# Raspberry Pi Image Builder - Documentation
Welcome to the RPI-Dev wiki! This project automates the creation of hybrid Raspberry Pi images combining Raspberry Pi OS hardware support with custom Debian ARM64 root filesystems.
Welcome to the Raspberry Pi Image Builder documentation! This system makes it easy to build, customize, and maintain production-ready Raspberry Pi images.
## Overview
## What is This?
RPI-Dev is an automated build system that creates ready-to-use Raspberry Pi images with full hardware compatibility and custom software configurations. The project addresses a common challenge: running pure Debian on Raspberry Pi while maintaining complete hardware support for critical components like the RP1 chip (Ethernet, USB, GPIO).
A modular automated build system that creates custom Raspberry Pi OS images by:
- Combining Raspberry Pi OS firmware with Debian ARM64
- Installing packages via QEMU ARM64 emulation
- Composing images from reusable service modules
- Automating builds via GitHub Actions CI/CD
- Auto-detecting and configuring hardware
### Key Features
## Quick Links
- **Automated Builds**: GitHub Actions automatically build all images daily and on every push
- **Full Hardware Support**: Maintains Raspberry Pi OS kernel and firmware for complete hardware compatibility
- **Pure Debian Userspace**: Uses official Debian ARM64 root filesystems for a clean, standard environment
- **Multi-Image Support**: Easy creation and management of multiple image configurations
- **Automatic Updates**: Built-in APT configuration enables safe kernel and firmware updates
- **QEMU Development**: Test and develop images in QEMU before flashing to hardware
- **CI/CD Integration**: Automated releases via GitHub Actions with compressed, ready-to-flash images
### Getting Started
- [Installation & First Build](Getting-Started.md)
- [Flashing Images to SD Cards](Getting-Started/#flashing-to-sd-card)
- [First Boot & Login](Getting-Started/#first-boot)
## Why This Project?
### Core Concepts
- [Architecture Overview](Architecture.md)
- [Build System](Build-System.md)
- [Service System](Services.md)
The Raspberry Pi uses specialized hardware that requires kernel drivers not yet available in mainline Linux:
### Advanced Topics
- [Creating Custom Images](Custom-Images.md)
- [Creating Custom Services](Custom-Services.md)
- [GitHub Actions CI/CD](GitHub-Actions.md)
- [Hardware Auto-Detection](Hardware-Detection.md)
- **RP1 Southbridge Chip**: Handles Ethernet, USB 2.0/3.0, GPIO, and other critical I/O
- **VideoCore GPU**: Provides hardware acceleration and display output
- **WiFi/Bluetooth**: Requires specific firmware blobs
### Reference
- [Available Images](Available-Images.md)
- [Available Services](Available-Services.md)
- [Command Reference](Command-Reference.md)
- [Configuration Reference](Configuration-Reference.md)
Using a standard Debian kernel results in non-functional hardware. This project solves that by:
### Help
- [FAQ](FAQ.md)
- [Troubleshooting](Troubleshooting.md)
- [GitHub Issues](https://github.com/Pikatsuto/raspberry-builds/issues)
1. Using Raspberry Pi OS boot partition and kernel
2. Replacing the root filesystem with pure Debian
3. Installing RaspiOS packages via APT for automatic updates
4. Preserving hardware compatibility while gaining Debian's advantages
## Why Use This?
## How It Works
### Architecture
The project uses a **partition-level merge approach**:
1. **Boot Partition (FAT32)**: Retained from Raspberry Pi OS for firmware compatibility
2. **Root Partition (ext4)**: Replaced with custom Debian ARM64 rootfs
3. **RaspiOS Packages**: Installed via APT with repository pinning:
- `raspberrypi-kernel` - Kernel, initramfs, and modules (including RP1 drivers)
- `raspberrypi-bootloader` - Bootloader and firmware files
- `libraspberrypi*` - VideoCore libraries
- `firmware-brcm80211` - WiFi/Bluetooth firmware
4. **APT Configuration**: RaspiOS repository with pinning enables safe `apt upgrade`
### Build Process
```
┌─────────────────────────────────────────────────────────────┐
│ Stage 1: Download Base Images │
│ ├─ Raspberry Pi OS Lite ARM64 (.img.xz) │
│ └─ Debian Generic Cloud ARM64 (.raw) │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Stage 2: QEMU Setup (Native ARM64 Execution) │
│ ├─ Create setup.iso from image/setup.sh + setupfiles/ │
│ ├─ Launch QEMU ARM64 with Debian + setup.iso │
│ ├─ Execute setup.sh (install packages, configure system) │
│ ├─ Install RaspiOS kernel/firmware via APT │
│ └─ Automatic shutdown when complete │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Stage 3: Hybrid Image Creation │
│ ├─ Copy RaspiOS boot partition │
│ ├─ Replace root partition with configured Debian │
│ ├─ Preserve /etc/fstab from RaspiOS │
│ └─ Resize to final image size │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ Stage 4: Compression │
│ ├─ Shrink unused space with PiShrink │
│ └─ Compress to .img.xz (ready to flash) │
└─────────────────────────────────────────────────────────────┘
```
## Quick Start
### For Users: Download Pre-Built Images
The easiest way to use this project is to download pre-built images from [GitHub Releases](../../releases):
1. Go to the [Releases](../../releases) page
2. Download the `.img.xz` file for your desired image
3. Flash to SD card or SSD:
### Easy to Build
```bash
xz -dc rpi-*.img.xz | sudo dd of=/dev/sdX bs=4M status=progress conv=fsync
sync
# One command to build a custom image
./bin/autobuild --image debian/qemu+docker+haos
```
### For Developers: Fork and Customize
### Easy to Maintain
- Version control your image configurations
- Automated builds via GitHub Actions
- Safe `apt upgrade` - kernel and firmware update automatically
Want to create your own custom images? It's easy:
### Easy to Share
- Share image configurations with your team
- Reproducible builds across environments
- Pre-built images in GitHub Releases
1. **Fork this repository** on GitHub
2. **Clone your fork**:
```bash
git clone https://github.com/Pikatsuto/raspberry-builds.git
cd rpi-dev
```
### Easy to Customize
- Modular service system
- Add your own services
- Full control over packages and configuration
3. **Create a new image configuration**:
```bash
# Copy an existing image as a template
cp -r images/raspivirt-incus images/my-custom-image
```
## Key Features
4. **Edit the configuration** (`images/my-custom-image/config.sh`):
```bash
OUTPUT_IMAGE="rpi-my-custom-image.img"
IMAGE_SIZE="8G"
QEMU_RAM="8G"
QEMU_CPUS="4"
DESCRIPTION="My custom Raspberry Pi image"
```
- **Modular Services** - Compose images from qemu, docker, haos, openwrt, hotspot
- **Auto-Detection** - Network interfaces, WiFi, Zigbee dongles
- **CI/CD Ready** - GitHub Actions for automated builds
- **Safe Updates** - RaspiOS kernel via APT with pinning
- **Full Hardware Support** - RP1 drivers, WiFi, Bluetooth, GPIO
5. **Customize the setup script** (`images/my-custom-image/setup.sh`):
- Add package installations
- Configure services
- Apply custom settings
## What Can You Build?
6. **Add custom files** to `images/my-custom-image/setupfiles/`:
- Configuration files
- Scripts
- SSH keys
- **Home Server** - Docker + Portainer + Watchtower
- **IoT Gateway** - Home Assistant + Zigbee auto-detection
- **Network Router** - OpenWrt + WiFi hotspot + dual NIC
- **Virtualization Platform** - Incus containers and VMs
- **Custom Appliance** - Your own service modules
7. **Commit and push**:
```bash
git add images/my-custom-image/
git commit -m "Add my custom image"
git push origin main
```
8. **GitHub Actions automatically builds your image!**
- Check the [Actions](../../actions) tab for build progress
- Download from [Releases](../../releases) when complete
### Building Locally
If you prefer to build locally:
```bash
# Install dependencies
sudo apt install -y qemu-system-aarch64 qemu-utils parted \
e2fsprogs dosfstools rsync xz-utils genisoimage
# Build a specific image
./bin/autobuild --image my-custom-image
# Or build all images
./bin/autobuild --all-images
```
## Image Directory Structure
Each image is defined in `images/<image-name>/`:
## Architecture at a Glance
```
images/my-image/
├── config.sh # Build configuration
│ # - OUTPUT_IMAGE: Final image filename
│ # - IMAGE_SIZE: Final image size (e.g., "8G")
│ # - QEMU_RAM: RAM for QEMU (e.g., "8G")
│ # - QEMU_CPUS: CPU cores for QEMU (e.g., "4")
│ # - DESCRIPTION: Image description
│
├── setup.sh # Setup script executed in QEMU ARM64
│ # - Installs packages (RaspiOS kernel, software)
│ # - Configures system (users, services, etc.)
│ # - Runs in native ARM64 environment
│
├── setupfiles/ # Files copied to /root/setupfiles/ in image
│ # - Config files, scripts, certificates, etc.
│ # - Available to setup.sh during QEMU execution
│
└── cloudinit/ # Cloud-init configuration (for first boot)
├── user-data # User configuration (users, SSH keys, passwords)
├── meta-data # Instance metadata (hostname, instance-id)
└── seed.img # Auto-generated ISO (don't edit manually)
RaspiOS Boot Partition (FAT32) Debian Rootfs (ext4)
├── bootloader ├── /bin, /usr, /etc (Debian)
├── kernel ├── RaspiOS kernel packages (via APT)
├── firmware ├── Your services
└── config.txt └── Auto-hardware detection
↓ ↓
└────────── Merged Image ───────────┘
```
## Available Images
## Support
This repository includes the following pre-configured images:
- Raspberry Pi 4/5 (ARM64)
- Debian 13 (Trixie) ARM64
- Additional distributions in development
- **[RaspiVirt-Incus](Image-RaspiVirt-Incus)**: Raspberry Pi virtualization platform with Incus container/VM manager
- **[RaspiVirt-Incus+Docker](Image-RaspiVirt-Incus-Docker)**: RaspiVirt-Incus plus Docker for container orchestration
- **[RaspiVirt-Incus+HAOS](Image-RaspiVirt-Incus-HAOS)**: Home automation platform with automatic Home Assistant OS deployment, adaptive dual-bridge networking, and Zigbee dongle support
## Next Steps
See individual image pages for detailed documentation.
1. [Install dependencies and build your first image](Getting-Started.md)
2. [Understand how the build system works](Architecture.md)
3. [Explore available services](Available-Services.md)
4. [Create your own custom image](Custom-Images.md)
## Automatic Updates
---
Images built with this system support automatic kernel and firmware updates via APT:
```bash
# On your Raspberry Pi
sudo apt update
sudo apt upgrade -y
```
The APT pinning configuration ensures RaspiOS packages (kernel, firmware) are updated from the RaspiOS repository while all other packages use Debian repositories. This maintains hardware compatibility while keeping the system up-to-date.
## Documentation
- **[GitHub Actions Workflow](GitHub-Actions)**: Detailed documentation of the automated build system
- **[RaspiVirt-Incus Image](Image-RaspiVirt-Incus)**: Virtualization platform with Incus
- **[RaspiVirt-Incus+Docker Image](Image-RaspiVirt-Incus-Docker)**: Incus + Docker platform
- **[RaspiVirt-Incus+HAOS Image](Image-RaspiVirt-Incus-HAOS)**: Home Assistant OS platform with adaptive networking
- **[Main README](../README.md)**: Complete project documentation with manual build instructions
## Getting Help
- **Issues**: Report bugs or request features via [GitHub Issues](../../issues)
- **Discussions**: Ask questions in [GitHub Discussions](../../discussions)
- **Documentation**: Check the [README](../README.md) and [CLAUDE.md](../CLAUDE.md) for technical details
## License
This project is provided "as is" without warranty. Use at your own risk.
**Happy Building!**

490
Services.md Normal file

@ -0,0 +1,490 @@
# Services
Complete guide to available services and creating custom ones.
## Available Services
### base (Always Included)
**Purpose**: Essential system configuration for all images
**Packages**:
- `raspberrypi-kernel` - RaspiOS kernel with RP1 drivers
- `raspberrypi-bootloader` - Boot firmware
- `firmware-brcm80211` - WiFi/Bluetooth firmware
- `NetworkManager` - Network management
- `openssh-server` - SSH access
- Essential utilities
**Configuration**:
- RaspiOS APT repository + pinning
- Network bridges (br-wan, br-lan if dual NIC)
- SSH enabled
- MOTD with IP addresses
**First-boot actions**:
- Partition resize to fill SD card
- Network interface name persistence
- Bridge configuration based on detected NICs
---
### qemu
**Purpose**: Incus container/VM platform with KVM acceleration
**Dependencies**: None
**Packages**:
- `qemu-kvm` - KVM virtualization
- `incus` - Container/VM manager
- `incus-tools` - Management utilities
**Configuration**:
- Incus initialization with default profile
- Network bridges: br-wan (WAN), br-lan (LAN if dual NIC)
- Storage pool: default (dir backend)
**First-boot actions**:
- Create br-lan and br-wan bridges (if not exist)
- Configure Incus network integration
- Enable IP forwarding and NAT
**Use cases**:
- Run VMs (Home Assistant, OpenWrt)
- Create containers for isolated services
- Development environments
---
### docker
**Purpose**: Docker Engine with Portainer and Watchtower
**Dependencies**: None
**Packages**:
- `docker-ce` - Docker Engine
- `docker-ce-cli` - Docker CLI
- `containerd.io` - Container runtime
**Configuration**:
- Docker installed from official repository
- User `pi` added to `docker` group
**First-boot actions**:
- Deploy Portainer (web UI for Docker)
- Deploy Watchtower (automatic container updates)
- Configure bridge networking
**Services deployed**:
- **Portainer**: `https://raspberry-ip:9443`
- **Watchtower**: Automatic updates for containers
**Use cases**:
- Run containerized applications
- Manage containers via web UI
- Auto-update containers
---
### haos
**Purpose**: Home Assistant OS in an Incus VM
**Dependencies**: `qemu` (Incus required)
**Packages**: None (VM image downloaded at first boot)
**First-boot actions**:
- Download Home Assistant OS image (latest stable)
- Create Incus VM named `haos`
- Allocate resources (2 vCPUs, 4GB RAM, 24GB disk)
- Configure network bridge (br-lan or br-wan)
- Detect and pass through USB Zigbee coordinators
- Start VM
**Zigbee auto-detection**:
- Scans `/dev/ttyUSB*` and `/dev/ttyACM*`
- Detects ConBee, CC2652, Sonoff, Silicon Labs dongles
- Passes through via USB vendor/product ID
**Access**:
- Home Assistant: `http://raspberry-ip:8123`
- First boot setup takes 5-10 minutes
**Use cases**:
- Home automation platform
- Zigbee/Z-Wave integration
- IoT device management
---
### openwrt
**Purpose**: OpenWrt router in an Incus container
**Dependencies**: `qemu` (Incus required)
**Packages**: None (container image downloaded at first boot)
**First-boot actions**:
- Download OpenWrt rootfs image
- Create Incus container named `openwrt`
- Allocate resources (1 vCPUs, 1GB RAM, 24GB disk)
- Configure network: eth0 → br-wan, eth1 → br-lan
- Set static IP: `192.168.10.1/24`
- Start container
**Network configuration**:
- WAN: br-wan (DHCP client)
- LAN: br-lan (192.168.10.1, DHCP server)
**Access**:
- LuCI web UI: `http://192.168.10.1`
- SSH: `ssh root@192.168.10.1`
**Use cases**:
- Advanced routing and firewall
- VPN server/client
- QoS and traffic shaping
- Network monitoring
---
### hotspot
**Purpose**: WiFi access point (2.4GHz/5GHz)
**Dependencies**: None
**Packages**:
- `hostapd` - WiFi access point daemon
**Configuration files**:
- `hostapd-5ghz.conf` - 5GHz AP config
- `hostapd-2.4ghz.conf` - 2.4GHz AP config
**First-boot actions**:
- Detect WiFi interfaces (wlan0, wlan1)
- Configure hostapd based on detected adapters:
- Dual WiFi: 2.4GHz (wlan0) + 5GHz (wlan1)
- Single WiFi: 5GHz (wlan0)
- Attach to br-lan (if exists) or br-wan
- Disable NetworkManager management of WiFi
- Start hostapd services
**Default credentials**:
- SSID: `RaspberryPi-WIFI`
- Password: `raspberry`
**Use cases**:
- WiFi access for IoT devices
- Guest network
- Extend network coverage
---
## Service Combination Examples
### Minimal Development Platform
```bash
./bin/autobuild --image debian
```
- Base Debian with RaspiOS kernel
- SSH access
- Network bridges
### Docker Host
```bash
./bin/autobuild --image debian/docker
```
- Docker Engine
- Portainer web UI
- Watchtower auto-updates
### Virtualization Platform
```bash
./bin/autobuild --image debian/qemu
```
- Incus containers/VMs
- KVM acceleration
### Home Automation Gateway
```bash
./bin/autobuild --image debian/qemu+haos
```
- Home Assistant OS VM
- Zigbee dongle auto-detection
- Web UI on port 8123
### Network Router + WiFi AP
```bash
./bin/autobuild --image debian/qemu+openwrt+hotspot
```
- OpenWrt routing
- Dual-band WiFi hotspot
- Advanced firewall/QoS
### Full Stack
```bash
./bin/autobuild --image debian/qemu+docker+openwrt+hotspot+haos
```
- All services combined
- Ideal for Raspberry Pi 5 with 8GB RAM
---
## Creating Custom Services
### Service Directory Structure
```
images/debian/services/myservice/
├── setup.sh # Required: package installation (runs in QEMU)
├── first-boot/
│ └── init.sh # Optional: runtime config (runs on first boot)
├── setupfiles/ # Optional: static files → /etc/setupfiles/
│ ├── config.conf
│ └── script.sh
├── depends.sh # Optional: dependencies
└── motd.sh # Optional: MOTD content
```
### setup.sh (Package Installation)
Executed in QEMU ARM64 during build.
**Template**:
```bash
#!/bin/bash
set -e
# Install packages
apt update
apt install -y package1 package2
# Configure system
systemctl enable my-service
# Copy files
cp /etc/setupfiles/config.conf /etc/my-service/
```
**Best practices**:
- Use `set -e` to exit on errors
- Install packages from Debian repositories when possible
- Add custom repositories if needed
- Enable systemd services that should start at boot
- Don't start services (no network in QEMU)
### first-boot/init.sh (Runtime Configuration)
Executed on first boot on Raspberry Pi.
**Template**:
```bash
#!/bin/bash
set -e
# Detect hardware
if ip link show eth1 >/dev/null 2>&1; then
BRIDGE="br-lan"
else
BRIDGE="br-wan"
fi
# Download resources
wget https://example.com/resource.tar.gz -O /tmp/resource.tar.gz
# Create containers/VMs
incus launch images:alpine mycontainer
# Configure services
systemctl start my-service
```
**Best practices**:
- Detect hardware and adapt configuration
- Download large files here (not in setup.sh)
- Create containers/VMs
- Start services
- Use `/etc/setupfiles/` for configuration files
### depends.sh (Dependencies)
Declare dependencies on other services.
**Example**:
```bash
# myservice depends on qemu
DEPENDS_ON="qemu"
```
**Dependency resolution**:
- Dependencies are automatically included
- Build order is adjusted to satisfy dependencies
### motd.sh (Message of the Day)
Display service information on login.
**Example**:
```bash
cat <<'EOF'
My Service UI: https://raspberry-ip:8080
Username: admin
Password: (set on first access)
EOF
```
### setupfiles/ (Static Files)
Static configuration files copied to `/etc/setupfiles/`.
**Access in scripts**:
```bash
# In setup.sh or first-boot/init.sh
cp /etc/setupfiles/myconfig.conf /etc/my-service/
```
---
## Service Development Workflow
### 1. Create Service Directory
```bash
mkdir -p images/debian/services/myservice/first-boot
mkdir -p images/debian/services/myservice/setupfiles
```
### 2. Write setup.sh
```bash
cat > images/debian/services/myservice/setup.sh <<'EOF'
#!/bin/bash
set -e
apt update
apt install -y nginx
systemctl enable nginx
EOF
chmod +x images/debian/services/myservice/setup.sh
```
### 3. Write first-boot/init.sh (Optional)
```bash
cat > images/debian/services/myservice/first-boot/init.sh <<'EOF'
#!/bin/bash
set -e
# Configure nginx with custom config
cp /etc/setupfiles/nginx.conf /etc/nginx/sites-available/default
systemctl restart nginx
EOF
chmod +x images/debian/services/myservice/first-boot/init.sh
```
### 4. Add Configuration Files (Optional)
```bash
cat > images/debian/services/myservice/setupfiles/nginx.conf <<'EOF'
server {
listen 80;
root /var/www/html;
index index.html;
}
EOF
```
### 5. Add MOTD (Optional)
```bash
cat > images/debian/services/myservice/motd.sh <<'EOF'
cat <<'MOTD'
Nginx: http://raspberry-ip/
MOTD
EOF
chmod +x images/debian/services/myservice/motd.sh
```
### 6. Test Service
```bash
# Build image with your service
./bin/autobuild --image debian/myservice
# Flash and boot
sudo dd if=debian-myservice.img of=/dev/sdX bs=4M status=progress
# Check logs on first boot
sudo journalctl -u services-first-boot
```
### 7. Debug Issues
**QEMU stage issues**:
```bash
# Check QEMU logs
cat images/debian-myservice/qemu-*.log
# Manually test in QEMU
./bin/autobuild --image debian/myservice --skip-compress
```
**First-boot stage issues**:
```bash
# On Raspberry Pi, check logs
sudo journalctl -u services-first-boot -f
# Check service status
sudo systemctl status myservice
```
---
## Service Best Practices
### Package Installation
- Install in `setup.sh`, not `first-boot/init.sh`
- Use official Debian repositories when possible
- Pin package versions if stability critical
### Resource Downloads
- Download large files in `first-boot/init.sh`
- Don't download in `setup.sh` (slows QEMU)
- Cache downloads in `/var/cache/` if appropriate
### Hardware Detection
- Always detect hardware in `first-boot/init.sh`
- Adapt configuration based on detection
- Fail gracefully if hardware missing
### Error Handling
- Use `set -e` to exit on errors
- Log errors to journald
- Provide helpful error messages
### Network Configuration
- Use existing bridges (br-wan, br-lan)
- Don't create new bridges unless necessary
- Check bridge existence before use
### Security
- Don't hardcode passwords in scripts
- Use strong default passwords (prompt user to change)
- Enable firewall if service exposed
---
## Next Steps
- [Create a custom image with your service](Custom-Images.md)
- [Learn about hardware auto-detection](Hardware-Detection.md)
- [Set up CI/CD for automated builds](GitHub-Actions.md)

205
Troubleshooting.md Normal file

@ -0,0 +1,205 @@
# Troubleshooting
Common issues and solutions.
## Build Issues
### QEMU Won't Boot
**Symptoms**: QEMU hangs at boot
**Solutions**:
```bash
# Install UEFI firmware
sudo apt install qemu-efi-aarch64
# Increase timeout in config.sh
QEMU_TIMEOUT=3600
# Check logs
cat images/*/qemu-*.log
```
### QEMU Timeout
**Symptoms**: "QEMU timeout" error
**Solutions**:
- Increase `QEMU_TIMEOUT` in config.sh
- Increase `QEMU_RAM` and `QEMU_CPUS`
- Check network connectivity
- Review QEMU logs
### Merge Fails
**Symptoms**: Error during merge stage
**Solutions**:
```bash
# Check disk space
df -h
# Re-download base images
rm images/debian/*.img images/debian/*.raw
./bin/autobuild --image debian
# Check partition layout
fdisk -l raspios.img
fdisk -l debian.raw
```
### Out of Disk Space
**Symptoms**: "No space left on device"
**Solutions**:
- Free up space: `sudo apt clean`
- Use smaller `IMAGE_SIZE`
- Build fewer services
## Boot Issues
### Rainbow Screen
**Symptoms**: Raspberry Pi shows rainbow screen, won't boot
**Solutions**:
1. Verify image integrity: `sha256sum image.img.xz`
2. Re-flash SD card
3. Try different SD card
4. Check UART output
### Can't Login
**Symptoms**: Credentials don't work
**Default**: Username `pi`, password `raspberry`
**Solutions**:
- Check Caps Lock
- Wait for first-boot to complete (3-5 min)
- Reset password via SD card mount
### No Network
**Symptoms**: Can't get IP address
**Solutions**:
```bash
# Check interface
ip link show eth0
# Restart connection
sudo nmcli con up br-wan
# Check NetworkManager
sudo systemctl status NetworkManager
# Set static IP (see Configuration Reference)
```
## Service Issues
### Service Not Starting
**Symptoms**: Docker/HAOS/OpenWrt not running
**Solutions**:
```bash
# Check first-boot logs
sudo journalctl -u services-first-boot
# Check service status
sudo systemctl status docker
incus list
# Check disk space
df -h
```
### Container/VM Won't Start
**Symptoms**: Incus container/VM fails to start
**Solutions**:
```bash
# Check Incus logs
incus info haos
incus info --show-log haos
# Check resources
free -h
df -h
# Restart Incus
sudo systemctl restart incus
```
## Hardware Issues
### WiFi Not Detected
**Symptoms**: wlan0 doesn't exist
**Solutions**:
```bash
# Check firmware
dmesg | grep brcmfmac
# Install firmware
sudo apt install firmware-brcm80211
# Reboot
sudo reboot
```
### Zigbee Not Passed to HAOS
**Symptoms**: Zigbee dongle not in Home Assistant
**Solutions**:
```bash
# Check detection
sudo journalctl -u services-first-boot | grep -i zigbee
# Manual passthrough
lsusb # Find vendor/product ID
incus config device add haos zigbee usb vendorid=XXXX productid=YYYY
# Restart VM
incus restart haos
```
## GitHub Actions Issues
### Build Timeout
**Symptoms**: CI/CD build fails with timeout
**Solutions**:
- Increase `QEMU_TIMEOUT` in workflow
- Reduce `IMAGE_SIZE`
- Use fewer services
### Asset Upload Fails
**Symptoms**: Image built but not in release
**Solutions**:
- Check GitHub token permissions
- Verify release was created
- Check asset size (<2GB per file)
## Getting Help
**Check logs**:
```bash
# Build logs
cat images/*/qemu-*.log
# First-boot logs
sudo journalctl -u rpi-first-boot
sudo journalctl -u services-first-boot
# Service logs
sudo journalctl -u docker
sudo journalctl -u incus
```
**Still stuck?**
- [Check FAQ](FAQ.md)
- [Open issue](https://github.com/Pikatsuto/raspberry-builds/issues)
- [Ask in discussions](https://github.com/Pikatsuto/raspberry-builds/discussions)
Include:
- Build command used
- Error messages
- Relevant logs
- Hardware details (Pi model, RAM, SD card size)

48
_Sidebar.md

@ -1,26 +1,32 @@
## Navigation
## Raspberry Pi Image Builder
**Getting Started**
- [Home](Home)
- [Quick Start](Home#quick-start)
**[Home](Home)**
**Documentation**
- [GitHub Actions](GitHub-Actions)
- [Build Process](GitHub-Actions#build-architecture)
### Getting Started
- [Installation & First Build](Getting-Started)
- [Architecture Overview](Architecture)
**Images**
- [RaspiVirt-Incus](Image-RaspiVirt-Incus)
- [RaspiVirt-Incus+Docker](Image-RaspiVirt-Incus-Docker)
- [RaspiVirt-Incus+HAOS](Image-RaspiVirt-Incus-HAOS)
### Build System
- [Build System Guide](Build-System)
- [Service System](Services)
- [GitHub Actions CI/CD](GitHub-Actions)
**Resources**
- [Main README](https://github.com/Pikatsuto/raspberry-builds/blob/main/README.md)
- [Releases](https://github.com/Pikatsuto/raspberry-builds/releases)
### Reference
- [Available Images](Available-Images)
- [Available Services](Available-Services)
- [Hardware Detection](Hardware-Detection)
- [Command Reference](Command-Reference)
- [Configuration Reference](Configuration-Reference)
### Advanced
- [Creating Custom Images](Custom-Images)
- [Creating Custom Services](Custom-Services)
### Help
- [FAQ](FAQ)
- [Troubleshooting](Troubleshooting)
### Links
- [GitHub Repository](https://github.com/Pikatsuto/raspberry-builds)
- [Issues](https://github.com/Pikatsuto/raspberry-builds/issues)
---
**Quick Links**
- [Download Images](https://github.com/Pikatsuto/raspberry-builds/releases)
- [Report Bug](https://github.com/Pikatsuto/raspberry-builds/issues/new)
- [GitHub Actions](https://github.com/Pikatsuto/raspberry-builds/actions)
- [Discussions](https://github.com/Pikatsuto/raspberry-builds/discussions)