NestBox firmware: Waveshare ESP32-S3-LCD-1.69

Status: the firmware builds, but has not yet run end to end on a board. BOARD.md lists what is still unverified.

The pet on the real board. It runs the same engine/ as the Mac simulator, serves the same HTTP API and announces itself the same way, so everything that works with the simulator works with the device. The board facts behind it, with sources, are in BOARD.md.

• Screen: used sideways, 280×240, 30 frames a second. Each frame goes to the panel by SPI DMA while the next one is drawn.
• Wi-Fi: station mode, set up over the air with a phone (or over the USB console) and remembered. It keeps retrying forever and shows "NO WI-FI" at the bottom of the screen while offline.
• Network: each pet has its own name, nestbox-<4 hex>.local (from its MAC), so several pets can share a network (see Names). It serves the HTTP API on port 80 and announces _nestbox._tcp over Bonjour. Request bodies can be up to 16 KB. It answers only to its own names and addresses and refuses writes from other sites' web pages; button presses are accepted only from the device itself (button, motion sensor, USB console).
• Internet time: once Wi-Fi is up, the device keeps its own clock, so the scene follows the time of day with no computer at all.
• USB: the pet appears as a serial console plus a small read-only drive, NESTBOX, holding the Mac app and a README. It also notices when the computer at the other end of the cable goes to sleep and wakes up.
• Eggs: a LittleFS data image, loaded into PSRAM at boot. The chosen pet is remembered.
• On the device: a tap on the case, a shake, and the PWR button go to the API as events.

Requirements

• PlatformIO: pipx install platformio, or see the installation guide. The first build downloads the ESP32 toolchain and framework (a few hundred MB).
• The Mac app for the drive: scripts/package_app.sh in the repo root makes build/NestBox.zip. Without it the drive carries a note saying it is missing; build it, then upload again.
• A USB-C data cable.

Build and flash

cd firmware/esp32
pio run                 # build the firmware and the NESTBOX drive image
pio run -t upload       # flash the firmware and the drive
pio run -t uploadfs     # flash the eggs: the data image, made from ../../pets

Flashing the eggs doesn't touch the firmware. Flashing the firmware doesn't touch the eggs or the saved settings (Wi-Fi, time zone, chosen pet).

Uploading: USB runs through TinyUSB, not the chip's built-in USB serial. PlatformIO asks the running firmware to restart into the ROM loader, which comes back as a different USB port, and waits for it (use_1200bps_touch and wait_for_upload_port in platformio.ini).

If an upload can't find the board (for example, a firmware that crashes before USB starts):

1. Unplug it, hold BOOT, plug it back in, and let go.
2. Run pio run -t upload again.
3. Press RST, or unplug and replug, to start the new firmware.

Close any open serial monitor before uploading.

First board revision: Waveshare moved the buzzer and power pins in V2 (the board has a "V2" QC sticker). For a board without it, use pio run -e nestbox-v1 -t upload. BOARD.md explains why the PWR button is off there.

Which pets go on it

By default only the goose. To add eggs you made yourself (keep them in a folder outside the repo):

NESTBOX_EGGS=~/my-eggs NESTBOX_PETS="goose kitchen-cat" pio run -t uploadfs

Or set custom_pets in platformio.ini. The build copies exactly those eggs (from NESTBOX_EGGS, else the repo's pets/) into data/ and removes any others. An egg whose art may not be redistributed (marked "private": true in its pet.json, or with no license) is left out unless NESTBOX_INCLUDE_PRIVATE=1.

The flash has 7.9 MB for eggs, and they are loaded whole into the 8 MB PSRAM at boot. Eggs store their frames cropped, so a pet is small (the goose is about 200 KB, a large pet about 600 KB). An egg that doesn't fit in PSRAM is skipped, and the console says so.

The NESTBOX drive

Plug the pet into a Mac and a read-only drive called NESTBOX appears, holding:

• NestBox.zip, the menu bar app with the companion inside.
• README.txt: how to install the app, find the pet's name, and connect a coding agent.

scripts/drive.py builds the drive from drive/ plus build/NestBox.zip, using scripts/fatimage.py (FAT12 with long names and no partition table). It goes into its own 2 MB flash partition and is flashed with every upload. Anything in drive/ goes on the drive; the build stops if the files don't fit.

Set up Wi-Fi and time

A pet with no Wi-Fi saved opens its own network and a phone sets it up (see Wi-Fi setup in the main README). The USB console does the same by hand:

pio device monitor      # the USB console; type `help`

wifi MyNetwork my-password
wifi "My Network" my-password      (quote a name with spaces)
tz CET-1CEST,M3.5.0,M10.5.0/3      (the time zone as a POSIX TZ rule; tz - clears it)

The device joins the network at once and remembers it. It needs a 2.4 GHz network. For 8 s after it connects, and for 8 s after each press of BOOT, the bottom of the screen shows its name and address, e.g. NESTBOX-3F2A.LOCAL 192.168.1.23. wifi shows the connection and forget clears it.

Internet time starts once Wi-Fi is up. Without a tz, the device uses the UTC offset the Mac companion last reported. With neither, it doesn't know the local time, so day and night follow the companion only.

Other console commands:

• name: the name in use, the pet's id, and its default name.
• name kitchen: renames it to kitchen.local (remembered). name - goes back to the default.
• tz: the time and zone in use.
• usb: what the port sees (a computer, asleep or awake).
• event {"event":"done"}: the same as POST /v1/event.
• state, pets, api: the GET pages.
• rotate: turns the picture upside down (remembered).
• bright 0-100: backlight brightness (remembered); digits only.
• imu: logs every bump and swing, to tune tap and shake.
• perf: frame times and free memory.
• reboot.

Use it

curl http://nestbox-3f2a.local/               # the API page (use your pet's name)
curl -d '{"source":"tests","event":"error"}' http://nestbox-3f2a.local/v1/event

Coding agents (Claude Code, Codex, Cursor…) can read http://<pet's name>.local/agents and connect themselves.

Names

• Id: every pet has a permanent id, the last 3 bytes of its factory MAC as 6 hex digits (for example a13f2a).
• Name: nestbox- plus the id's last 4 hex digits (nestbox-3f2a.local), unless renamed with name. The name is also the DHCP host name; after a rename the router learns it on the next connect.
• Bonjour instance: the pet's display name plus its host name, e.g. Goose (nestbox-3f2a).
• TXT record: pet=<name>, api=/v1/event, id=<6 hex> and host=<hostname>.local, published again when the pet or the name changes.
• **base_url:** the API's base address follows the name in use, so pages that print the pet's address give this pet's own.
• Name conflicts (two pets renamed alike, or another device on the network): ESP-IDF's mDNS (component 1.13.1) notices while claiming the name and moves this pet to the next free one. - kitchen becomes kitchen-2. - A name ending in -<number> gets the number raised: nestbox-1234 becomes nestbox-1235. - The device that already held the name keeps it.

The firmware reads back the name actually in use, republishes it, and shows it on screen and in name. The new name lasts until the next restart, which tries the wanted name again.

• Pairing: finders should pair by id, never by name.

The Mac companion

build/nestbox (or the menu bar app) finds the pet by Bonjour. It prefers a device over a simulator (any name containing "simulator" sorts last). To pick one, use --id a13f2a (the id nestbox find lists), the Bonjour instance name (--name "Goose (nestbox-3f2a)"), or --device http://<name>.local. With several pets, the menu bar app pairs by id.

Computer asleep and awake, over the cable

A computer that goes to sleep suspends its USB ports. The pet sees that after 3 s and sends itself {"event": "computer", "state": "asleep"}, so it goes to sleep too; when the bus resumes it sends "awake". This needs no software on the computer.

A phone charger never enumerates the pet as a USB device, so it never reads as asleep. Unplugging the cable while the pet runs on its battery reads as sleep. Activity from the companion over Wi-Fi wakes it either way.

On the device itself

On battery: hold PWR until the screen lights. The firmware holds the power on from then and never switches itself off; unplug the battery, or let it run down, to switch off.

Source layout

• **src/main.cpp:** setup(), the frame task (core 1) and the sensor task (core 0). - Each frame calls the engine's step_frame() (engine/host/frame.h, shared with the simulator) and hands the canvas to the LCD. It waits for the previous frame's DMA before it takes the lock. - App (src/app.h) holds the engine's objects behind one mutex; tasks reach them only through App::with_lock().
• **src/service.cpp:** the service task on core 0: the console, the log, setup's radio work, Wi-Fi, clock and USB upkeep, and every flash write.
• **src/console.cpp:** the console's commands; src/console_reply.h holds the replies the Mac app reads. **src/log.cpp:** log lines from any task, written to USB only by the service task.
• **src/settings.cpp:** the settings kept in NVS.
• **src/lcd.cpp:** a small ST7789V2 driver on ESP-IDF's spi_master (full frames by DMA, bytes swapped).
• **src/net.cpp:** Wi-Fi through Arduino, ESP-IDF's esp_http_server routing every method and path to the API, and ESP-IDF's mDNS.
• **src/esp_radio.cpp:** the radio, clock and storage behind setup over the air.
• **src/usb_port.cpp:** the TinyUSB composite device: CDC console plus the MSC drive, read from the memory-mapped drive partition; also tracks bus suspend and resume.
• **src/wall_clock.cpp:** SNTP (pool.ntp.org, time.google.com, time.apple.com), POSIX TZ, local time.
• **src/imu.cpp, src/button.cpp:** tap and shake from the QMI8658; single, double and long presses.
• Scripts: scripts/version.py takes the version from ../../CMakeLists.txt; scripts/engine.py compiles ../../engine unchanged; scripts/pets.py fills data/; scripts/drive.py and scripts/fatimage.py build the drive image.

Flash layout (partitions.csv, 16 MB):

The framework is Arduino core 3.3.12 on ESP-IDF 5.5, through PlatformIO's espressif32 8.0.0. The firmware uses about 60% of an app slot and a quarter of the internal RAM (pio run prints both).

At run time one frame buffer (134 KB) sits in internal DMA memory; the 134 KB canvases, the eggs and request bodies sit in PSRAM.

Crash messages and the boot log go to UART0 (GPIO43/44 on the header), not USB, because the USB port runs the firmware's own TinyUSB device.