tds3p-rsvp-reader/README.md

# T-Display S3 Pro RSVP Reader

An offline-first RSVP reader for the LilyGO T-Display S3 Pro. It renders one
white word on a black, landscape display and never starts Wi-Fi or Bluetooth.

## What you need

- A LilyGO T-Display S3 Pro and a **USB-C data cable**. A charge-only cable
  cannot upload firmware.
- A computer with internet access for the first build, Python 3, and a terminal.
- A microSD card formatted as FAT32. The reader uses this as its book library.
- This repository, cloned or downloaded to your computer.

The first build downloads the ESP32 toolchain and display library. The reader
does not need network access after it has been flashed.

## 1. Install PlatformIO Core

PlatformIO Core is the command-line build and upload tool used by this project.
Install it in your normal user account; do not use `sudo`.

### macOS or Linux

Confirm Python 3 is available:

```bash
python3 --version
```

Then use PlatformIO's installer:

```bash
curl -fsSL -o get-platformio.py https://raw.githubusercontent.com/platformio/platformio-core-installer/master/get-platformio.py
python3 get-platformio.py
```

The installer puts `pio` at `~/.platformio/penv/bin/pio`. Add that directory to
your shell `PATH`, or use the full path in every command below. For the current
shell only:

```bash
export PATH="$HOME/.platformio/penv/bin:$PATH"
pio --version
```

On Linux, if the serial device appears but uploads fail with a permission error,
add your user to the serial-device group and then sign out and back in:

```bash
sudo usermod -a -G dialout "$USER"
```

### Windows

Install Python 3, then follow PlatformIO's official installer instructions.
After installation, open a new PowerShell window and confirm:

```powershell
pio --version
```

PlatformIO's installer documentation is the authoritative reference for all
operating systems: <https://docs.platformio.org/en/latest/core/installation/methods/installer-script.html>.

## 2. Prepare a book card

1. Format the microSD card as **FAT32**.
2. Create a directory named `books` at the card's root.
3. Create one folder per book inside `books`. Each book folder needs a
   `TOC.txt`, UTF-8 text files for its sections, and matching `.rsvp` sidecars
   generated by the converter.

`TOC.txt` is the reader's source of truth for section order, membership, and
title-card text. Each non-empty line names a section and its file is that name
plus `.txt`. The reader reads and caches this file when it needs navigation
order; it opens only the selected section file.

The resulting card must look like this:

```text
microSD root/
└── books/
    └── Example Book/
        ├── TOC.txt
        ├── Dedication.txt
        ├── Dedication.rsvp
        ├── Chapter 1.txt
        ├── Chapter 1.rsvp
        ├── Chapter 2.txt
        └── Chapter 2.rsvp
```

For the example above, `TOC.txt` contains:

```text
Dedication
Chapter 1
Chapter 2
```

Generate this layout from an EPUB with the included converter:

```bash
./.venv/bin/python -m pip install -r book-processing/requirements.txt
./.venv/bin/python book-processing/epub_to_chapters.py book.epub "Example Book"
```

For a quick end-to-end check, the repository includes a two-chapter sample
with EPUB styles, punctuation, and repeated invented names:

```bash
./.venv/bin/python book-processing/epub_to_chapters.py book-processing/examples/rsvp-timing-sample.epub "Timing Sample"
```

The output directory must be empty unless `--overwrite` is supplied. Copy the
resulting `Example Book` directory into `/books` on the card. A section is
limited to 150,000 words; its source text and `.rsvp` sidecar may each be up
to 1 MiB. The converter writes
`CONVERSION_REPORT.txt` in the book directory. If unsupported non-ASCII
characters remain after normalizing common punctuation, ligatures, accents,
whitespace, fractions, and ornamental breaks, the converter asks once for an
ASCII replacement for each character. The replacement is used everywhere that
character occurs; press Enter for a blank replacement. The report records the
chosen substitutions. The included display font is best for printable ASCII.

The converter also writes a compact `<section>.rsvp` file alongside every
section. Its versioned, delta-encoded records preserve italic and bold EPUB
markup—including semantic tags, CSS classes, and inline CSS—and reading-timing
metadata. The fingerprint covers the source `.txt` bytes, before the device
applies display-only punctuation normalization. During conversion, the
desktop-only English vocabulary classifier marks each unfamiliar word. It
combines a dictionary with word-frequency data and recognizes normal English
inflections, contractions, possessives, and hyphenated compounds. The first
ten appearances of each marked word in
the book, plus its first appearance in every section, use a 3× delay. Later
appearances use a 2× delay. The reader uses the strongest applicable delay, so
sentence and paragraph pauses are never compounded with unfamiliar-word
timing. The reader validates the sidecar's fingerprint against its `.txt` file
and safely ignores it if it is missing, stale, or malformed. Valid italic and
bold records select matching reader-font variants; the device never performs
dictionary or SD-card lookups while advancing words.

The converter requires the desktop-only `english-words` and `wordfreq`
packages listed in `book-processing/requirements.txt`; the reader does not need either
dependency. Reconvert a book to create v2 sidecars with unfamiliar-word
timing. Existing v1 style-only sidecars remain readable but do not contain
timing metadata.

Eject the card safely and insert it in the board before powering the board.

## 3. Build the firmware

Open a terminal in the `firmware` directory:

```bash
cd /path/to/tds3p-rsvp-reader/firmware
pio run
```

On the first run, PlatformIO downloads the pinned ESP32 platform, compiler,
and the TFT_eSPI, SensorLib, and XPowersLib dependencies. A successful build
ends with `SUCCESS` and creates the firmware under `.pio/build/rsvp_reader/`.

The project targets the T-Display S3 Pro's 16 MB flash and 8 MB OPI PSRAM. It
defaults to the v1.1 16-step backlight driver. If your board is the original
v1.0 hardware, add this line under `build_flags` in `firmware/platformio.ini` before
building:

```ini
  -DTDS3P_V1_0
```

## 4. Connect and identify the board

1. Connect the board directly to the computer with the USB-C data cable.
2. Give the operating system a few seconds to enumerate it.
3. List serial ports:

```bash
pio device list
```

Typical ports are `/dev/ttyACM0` or `/dev/ttyUSB0` on Linux, `/dev/cu.usbmodem*`
on macOS, and `COM3` on Windows. Note the port belonging to the board.

## 5. Upload and start the reader

Let PlatformIO select the port automatically:

```bash
pio run -t upload
```

If it selects the wrong port or does not find one, supply it explicitly:

```bash
pio run -t upload --upload-port /dev/ttyACM0
```

Replace `/dev/ttyACM0` with the port from the previous step. On Windows, for
example, use `--upload-port COM3`.

After a successful upload, the display shows the current section title and is
**paused**. If it shows `NO BOOKS`, check that the card is inserted, FAT32, and
contains at least one `/books/<book>/TOC.txt` with a listed section.

If the board is not detected or an upload repeatedly fails, force download mode:

1. Hold **BOOT** (the lower-left button, GPIO0).
2. While holding BOOT, press and release **RESET** (upper-left).
3. Release BOOT.
4. Run the upload command again.

## 6. Read

| Control | Reading view | Menu or picker |
| --- | --- | --- |
| BOOT / GPIO0 | Play or pause | Select / confirm |
| Touchscreen | Open menu | No action |
| Upper-right / GPIO12 | Previous paragraph or section | Move selection backward |
| Lower-right / GPIO16 | Next paragraph or section | Move selection forward |

The touch-edge Home button has no action.

Do not hold BOOT while pressing RESET during normal use; that combination is
the manual download mode used for recovery above.

The menu opens with **Back** selected. BOOT on Back returns to reading. Pickers
start at the current setting and move forward when confirmed. The Font option
selects style first, then size; the size preview displays the word “size” in
the selected font and size. The selected font style also applies to the status,
footer, menus, and picker labels.

The default is 300 WPM. While paused, the backlight dims after 20 seconds,
switches off after one minute, and the ESP32 deep-sleeps after ten minutes.
GPIO12 wakes it.
The footer shows the chapter title, its fixed total word count, and reading
progress as a rounded percentage.

## Resume and power behavior

The reader saves the selected section, paragraph-level position, WPM,
brightness, and font settings in the ESP32's nonvolatile storage. On the next
boot or wake it opens the same section's title card and remains paused;
pressing BOOT begins at the saved paragraph. It opens the
saved section directly, then falls back to the first TOC-listed section if the
card has changed.

When continuous reading reaches the end of a section, it opens the next
section's title card and pauses. Press BOOT to begin that
section.

The physical power switch only disconnects the battery. USB power is sufficient
for development and flashing. The reader intentionally has no Wi-Fi, Bluetooth,
cloud account, or synchronization feature.

## Troubleshooting

- **No serial port:** use another USB-C cable; many included cables only charge.
- **Upload cannot connect:** use the BOOT + RESET sequence in step 5, then retry.
- **Permission denied on Linux:** complete the `dialout` group step and log out
  and back in.
- **`NO BOOKS`:** reformat the card FAT32, create `/books/<book>`, and add a
  `TOC.txt` plus the listed `.txt` section files. Confirm the card is fully
  seated. Dot-prefixed folders such as `.Trash` are ignored.
- **Black display after boot:** press GPIO12 to wake it if it has slept; otherwise
  reflash and check whether the v1.0 backlight build flag is needed.
- **Build fails before compilation:** rerun `pio run` with internet access so
  PlatformIO can download the required packages.

## Limits of the first MVP

- It reads sections up to 150,000 words and 1 MiB from
  `/books/<book>/<section>.txt`, in the order declared by that book's
  `TOC.txt`.
- The built-in display font is best for printable ASCII text. UTF-8 text files
  are accepted, but accented/non-Latin characters need a Unicode font addition
  before they will render correctly.
- The v1.1 backlight driver is the default. Define `TDS3P_V1_0` in
  `build_flags` for an original v1.0 board.