# GFX -- Terminal graphics, sprites, and animation ## Overview `term.gfx` is the Kitty-graphics-protocol layer plus a small base for terminal animations, sprites, pixel-art text, and games. It builds on the Kitty graphics protocol (image transmission, placement, deletion), the LEV async runtime (frame timing and input), and a fast C pixel surface. ## Architecture `term.gfx` is a thin aggregator that re-exports a set of submodules. Older code that called `require("term.gfx").send_png` / `.new_canvas` keeps working unchanged. | Module | Responsibility | | ------------------ | ----------------------------------------------------------------- | | `term.gfx_core` | C: raw RGBA `Surface` (alloc, fill, pixel, rect, blit, tostring) | | `term.gfx.proto` | Kitty protocol wire layer: transmit / place / delete commands | | `term.gfx.canvas` | Ergonomic Lua canvas over a surface (`{r,g,b,a}` colours, drawing) | | `term.gfx.sprite` | Sprites: views, sheets/atlases, pixel-art, animations | | `term.gfx.font` | Pixel-art text + a bundled default 5x7 ASCII font | | `term.gfx.display` | Double-buffered on-screen placement + image/placement id alloc | | `term.gfx.loop` | Fixed-timestep frame loop + ticker | Layering: everything draws into a **canvas** (RGBA pixels). A **display** hands the canvas pixels to the terminal and atomically replaces the previous frame. A **loop** drives update/render/input on a fixed timestep. Sprites and fonts are just blit sources for the canvas. Coordinates are **1-indexed** everywhere, matching the rest of the codebase. Colours are `{r, g, b, a}` arrays (each 0–255; alpha defaults to 255). ## Canvas ```lua local c = gfx.canvas({ width = 320, height = 200, colors = { bg = {0,0,0,0}, main = {252,252,252,255} } }) ``` `gfx.canvas(cfg)` (alias `gfx.new_canvas`) returns a canvas backed by a C surface. Methods: | Method | Description | | --------------------------------------- | ---------------------------------------------------- | | `c:width()` / `c:height()` | size in pixels | | `c:fill(col)` | flood with `col` (default bg) | | `c:clear()` | reset to fully transparent | | `c:pixel(x, y, col)` | set one pixel (clipped) | | `c:get(x, y)` | read a pixel -> `r, g, b, a` (nil out of bounds) | | `c:line(x1, y1, x2, y2, col)` | Bresenham line | | `c:circle(cx, cy, r, col)` | filled circle | | `c:rect(x1, y1, x2, y2, col, filled)` | rectangle (filled or 1px outline) | | `c:poly(pts, col)` | filled **convex** polygon (`pts` = `{{x=,y=},…}`; scan range clamped to the canvas) | | `c:ellipse(cx, cy, rx, ry, col, steps)` | ellipse outline (line segments; default 44 steps) | | `c:blit(src, dx, dy, opts)` | composite a sprite/canvas/surface (see below) | | `c:resize(w, h)` | resize, preserving the top-left overlap | | `c:tostring()` | raw RGBA bytes (`w*h*4`) for a Kitty `f=32` transmit | | `c:send(opts)` / `c:display(opts)` | low-level direct transmit (see `term.gfx.proto`) | ### Blitting `c:blit(src, dx, dy, opts)` composites another canvas/sprite/surface onto `c` with its top-left at `(dx, dy)`. It always clips against both surfaces, so drawing partly off-screen is safe. `opts`: - `sx, sy, sw, sh` — source sub-rectangle (for sprite sheets). - `flip` — `"h"`, `"v"`, or `"hv"`. - `tint` — `{r, g, b[, a]}` channel multiply (255 = unchanged). Recolour a white mask, flash a sprite on damage, etc. - `alpha` — 0–255 global opacity. - `blend` — `"alpha"` (default, straight-alpha over), `"replace"` (copy), or `"add"` (additive/glow). - `dw, dh` (or `scale`) — destination size; the source rect is nearest-neighbour scaled into it (default 1:1). - `rot` — rotation in radians around the (scaled) rect's centre; positive rotates **clockwise on screen** (y-down). Nearest-neighbour sampled at pixel centres; composes freely with `dw/dh`, `flip`, `tint`, and every blend mode. `rot = 0` takes the bit-for-bit identical axis-aligned fast path. Note for pixel art: arbitrary-angle rotation breaks the pixel grid (edge shimmer) — prefer pre-baked angle frames for small sprites; `rot` shines for large or smooth-shaded content and bake-time rotation. ## Sprites A sprite is any table with a `surface` and a `sx, sy, sw, sh` source rect — i.e. a blit source. ```lua -- a view of a region of a canvas/surface local spr = gfx.sprite(canvas, { sx = 1, sy = 1, sw = 16, sh = 16 }) -- a sprite sheet / atlas local sheet = gfx.sheet(canvas, { tile_w = 16, tile_h = 16, margin = 0, spacing = 0 }) canvas:blit(sheet:frame(3), x, y) -- or named frames: local s2 = gfx.sheet(canvas, { frames = { idle = {1,1,16,16}, run = {17,1,16,16} } }) canvas:blit(s2:frame("run"), x, y) ``` ### Pseudo-pixel-art `gfx.art{ rows, palette, scale }` rasterizes a character map into its own surface once and returns a sprite. Unmapped characters and spaces are transparent. ```lua local ship = gfx.art({ rows = { "..R..", ".RRR.", "RRWRR" }, palette = { R = {220,40,40,255}, W = {252,252,252,255} }, scale = 4, }) canvas:blit(ship, x, y) local glitched = ship:glitch(0.15) -- new sprite with random pixel flips ``` A monochrome mask (one white palette entry) can be recoloured per blit via `tint` — this is exactly how the font and the matrix rain work. ### Animated sprites ```lua local anim = gfx.anim({ frames = { gfx.sprite(a), gfx.sprite(b), gfx.sprite(c) }, -- or { sheet = sh, frames = {1,2,3,2} } durations = 0.1, -- seconds per frame (or a per-frame array) loop = true, -- true / false / a count on_end = function() end, -- called once when a non-looping anim finishes }) -- each tick: anim:update(dt) canvas:blit(anim, x, y) -- an anim is itself a blit source (current frame) ``` ## Pixel-art text ```lua local font = gfx.font({}) -- bundled default 5x7 ASCII local f2 = gfx.font({ cell_w = 5, cell_h = 7, glyphs = { A = {"#####", ...}, ... } }) font:draw(canvas, "SCORE 1234", 8, 8, { scale = 2, color = {0,255,128,255} }) local w, h = font:measure("SCORE 1234", { scale = 2 }) ``` Glyphs are monochrome bitmaps (rows of `#`/`.`). Each is baked once into a white mask sprite (cached per character + scale) and blitted with the draw colour applied as a `tint`, so the same cached mask renders in any colour. `font:draw` honours `\n`, lowercase falls back to uppercase, and unknown glyphs fall back to `fallback` (default space). It returns the pen x after the last glyph. ## On-screen display A `display` owns an image id and a placement id and atomically replaces the previous frame each `:update()`. Pixels travel via a fresh-per-frame POSIX shared-memory object (the terminal unlinks it after reading, so it never leaks), falling back to inline base64 if shm fails. ```lua local d = gfx.display(canvas, { anchor = "absolute", -- or "cursor", or omit for the current cursor row = 1, col = 1, -- cell position (absolute) / column (cursor anchor) z = 0, -- z-index; negative composites behind text cols = 40, rows = 12, -- scale into this cell box (omit for natural pixels) }) d:show() -- transmit + place d:update() -- re-transmit current pixels + re-place (atomic frame swap) d:hide() -- drop the placement, keep the stored image d:destroy() -- delete the image and free its data ``` `anchor = "cursor"` jumps up `up` lines (default `rows+1`) and to column `col` relative to the current cursor, then restores it — so the image rides behind the most recent output as it scrolls. `gfx.alloc_id()` / `gfx.alloc_placement()` hand out unique ids so independent graphics apps never collide. ## DRM console display `gfx.drm_display(canvas, opts)` is the console twin of `gfx.display` — the same duck-typed method contract, but each `:update()` blits the canvas into a DRM/KMS dumb buffer (`term.drm_core`) and presents it with a vsynced page flip instead of transmitting it over the Kitty protocol. No terminal is required: run from a bare console/VT. Opening takes DRM master, so it fails with a clean error under a running compositor. ```lua local d, err = gfx.drm_display(canvas, { path = "/dev/dri/card0", -- the default manage_console = true, -- the default: raw mode + hidden cursor, }) -- restored by :destroy() ``` The canvas is upscaled by the largest INTEGER factor that fits the display mode — nearest-neighbour, centered, black borders — so pixel art stays crisp (`gfx.drm_layout(dev_w, dev_h, cw, ch)` exposes the math). Returns `nil, err` (console untouched) when the device can't be opened, DRM master is unavailable, no connected output is found, or the mode is smaller than the canvas. Presentation is double-buffered with one flip in flight: `:update()` waits out the previous flip's completion event via the LEV loop (the wait yields, so the display paces the frame loop at the display's real refresh), then blits and queues the next flip. `:destroy()` restores the CRTC the console was scanning out along with the cursor and terminal mode. Drive it with the frame loop configured terminal-free: pass the drm display via `display`, set `alt_screen = false`, and give no `input` / `on_resize` callbacks (there is no KKBP on a raw console — input comes from elsewhere, e.g. the `evdev` module). While the display is live the process holds DRM master, so other VTs show a frozen frame until it exits. ## Frame loop ```lua local handle -- forward-declare: callbacks (e.g. input) handle = gfx.loop({ -- often call handle:stop(); `local handle =` fps = 60, -- would leave `handle` nil inside them canvas = gfx.canvas({ width = W, height = H }), display = true, -- auto-create an absolute top-left display update = function(dt, t) ... end, -- fixed-step simulation render = function(canvas) ... end, -- draw into the canvas input = function(key, mods, event, shifted, base) ... end, on_resize = function(cols, rows) ... end, }) handle:run() -- owns its own lev.run; blocks until stopped ``` The loop decouples the simulation rate from wall-clock frame time: `update` is called at the fixed `1/fps` step (possibly several times to catch up after a slow frame, clamped by `max_frame`), then `render`, then the display is updated. Input is read cooperatively via `term.read_key` over a `lev.fd_reader` — **only** when an `input` callback is given, so a pure overlay never grabs stdin. - `handle:run()` — owns `lev.run`, blocks until stopped (standalone game). - `handle:start()` — spawns into an already-running `lev.run`, returns immediately (embedding, e.g. the matrix rain inside the shell loop). - `handle:stop()` / `:pause()` / `:resume()` / `:is_running()`. - fields: `handle.canvas`, `handle.display`, `handle.fps_actual`. By default the loop enters the alternate screen with raw mode + KKBP and hides the cursor (`alt_screen = true`), all restored on stop. Set `alt_screen = false` for an overlay that draws behind normal output. `gfx.ticker(fps)` is the low-level alternative: `ticker:tick()` sleeps to the next frame boundary and returns the delta time, for code that runs its own loop. ## Protocol layer `term.gfx.proto` (re-exported on `term.gfx`) is the raw wire layer used by the higher layers and by direct image display: `build_data`/`send_data`, `send_data_shm`, `send_data_file`, `build_png`/`send_png`, `png_dimensions`, `build_place`/`place_image`, `build_delete`/`delete_images`, `send_cmd`. Most applications should prefer a canvas + display over calling these directly. ## A complete example ```lua local gfx = require("term.gfx") local W, H = 240, 160 local canvas = gfx.canvas({ width = W, height = H }) local font = gfx.font({}) local ball = gfx.art({ rows = { ".WW.", "WWWW", "WWWW", ".WW." }, palette = { W = {120,220,120,255} }, scale = 4 }) local x, y, vx, vy, score = 20, 20, 90, 70, 0 -- Forward-declare `handle` so the `input` closure below captures this local. -- `local handle = gfx.loop{...}` would NOT work: inside its own initializer the -- local is not yet in scope, so `handle` there would resolve to a nil global. local handle handle = gfx.loop({ fps = 60, canvas = canvas, display = true, update = function(dt) x = x + vx * dt; y = y + vy * dt if x < 1 or x + ball.w > W then vx = -vx; score = score + 1 end if y < 1 or y + ball.h > H then vy = -vy end end, render = function(c) c:clear() c:blit(ball, math.floor(x), math.floor(y)) font:draw(c, "SCORE " .. score, 4, 4, { scale = 1, color = {255,255,255,255} }) end, input = function(key) if key == "ESC" or key == "q" then handle:stop() end end, }) handle:run() ``` ## Notes & limits - The shm + `f=32` transport is kitty/ghostty-family; `display` falls back to inline base64 automatically. - Loading a PNG **into** a canvas (CPU decode to RGBA for blitting) is not yet supported; PNGs can still be displayed directly via `send_png`. - The C surface caps each dimension at 32767 and rejects sizes that would overflow.