DataTable render crashes with StringIndexError on multibyte cell text

kahliburke commented

2026-06-19 16:46:38 -07:00

(Migrated from github.com)

Summary

DataTable.render (and the paged render.jl equivalent) truncates overlong cell text by byte-indexing a String, but Julia strings index by byte while length/avail count characters. When a cell contains multibyte UTF-8 (e.g. em-dash —, 3 bytes) the slice boundary can land mid-character and throw StringIndexError.

Stack trace (reported via Kaimon)

StringIndexError: invalid index [45], valid nearby indices [44]=>'—', [47]=>' '
  [2] getindex(s::String, r::UnitRange{Int64})
    @ Base ./strings/string.jl:503
  [3] render(dt::Tachikoma.DataTable, rect::Tachikoma.Rect, buf::Tachikoma.Buffer)
    @ Tachikoma .../src/widgets/datatable.jl:534

Cause

src/widgets/datatable.jl:534 and src/paged/render.jl:124:

cell_text = avail > 1 ? cell_text[1:max(1, avail-1)] * "…" : string(cell_text[1])

cell_text[1:avail-1] treats avail-1 as a byte offset.

Fix

Use first(s, n), which counts characters and respects UTF-8 boundaries:

cell_text = avail > 1 ? first(cell_text, max(1, avail-1)) * "…" : string(first(cell_text, 1))

Reported downstream in kahliburke/Kaimon.jl#29.

## Summary `DataTable.render` (and the paged `render.jl` equivalent) truncates overlong cell text by **byte**-indexing a `String`, but Julia strings index by byte while `length`/`avail` count characters. When a cell contains multibyte UTF-8 (e.g. em-dash `—`, 3 bytes) the slice boundary can land mid-character and throw `StringIndexError`. ## Stack trace (reported via Kaimon) ``` StringIndexError: invalid index [45], valid nearby indices [44]=>'—', [47]=>' ' [2] getindex(s::String, r::UnitRange{Int64}) @ Base ./strings/string.jl:503 [3] render(dt::Tachikoma.DataTable, rect::Tachikoma.Rect, buf::Tachikoma.Buffer) @ Tachikoma .../src/widgets/datatable.jl:534 ``` ## Cause `src/widgets/datatable.jl:534` and `src/paged/render.jl:124`: ```julia cell_text = avail > 1 ? cell_text[1:max(1, avail-1)] * "…" : string(cell_text[1]) ``` `cell_text[1:avail-1]` treats `avail-1` as a byte offset. ## Fix Use `first(s, n)`, which counts characters and respects UTF-8 boundaries: ```julia cell_text = avail > 1 ? first(cell_text, max(1, avail-1)) * "…" : string(first(cell_text, 1)) ``` Reported downstream in kahliburke/Kaimon.jl#29.

kahliburke commented

2026-06-20 19:14:29 -07:00

(Migrated from github.com)

Full scope: same byte-index-vs-character bug found at 6 sites

A sweep for the truncate-by-byte pattern (length-guarded String slice with a width/char count used as a byte index) turned up the same class of bug in several widgets. All fixed by switching to character-safe first(s, n):

Site	Field
`src/widgets/datatable.jl:534`	data cell (original crash)
`src/paged/render.jl:124`	paged data cell
`src/widgets/datatable.jl:435`	column header (name + sort indicator)
`src/paged/render.jl:27`	paged column header (name + sort/filter indicators)
`src/paged/render.jl:279`	error overlay text
`src/widgets/table.jl:112`	Table cell
`src/widgets/barchart.jl:68`	BarChart label
`src/app.jl:669`	export-modal font name

Editor widgets (codeeditor.jl, textarea.jl) were checked and are not affected — their lines are Vector{Char}, so the slices are already character-indexed. layout.jl constraint deserialization is internal ASCII and safe.

Tests

Regression tests added that sweep buffer/column widths so the truncation boundary lands at every byte offset within a multibyte run (em-dash = 3 bytes), guaranteeing the mid-character cut is exercised:

test/test_widgets_extended.jl — DataTable cells + headers
test/test_paged_datatable.jl — paged cells, headers, error overlay
test/test_widgets_coverage.jl — Table cells
test/test_core.jl — BarChart labels

Each test was verified to fail on the pre-fix code and pass after. Full suite: 5074/5074 passing.

## Full scope: same byte-index-vs-character bug found at 6 sites A sweep for the truncate-by-byte pattern (`length`-guarded `String` slice with a width/char count used as a byte index) turned up the same class of bug in several widgets. All fixed by switching to character-safe `first(s, n)`: | Site | Field | |------|-------| | `src/widgets/datatable.jl:534` | data cell (original crash) | | `src/paged/render.jl:124` | paged data cell | | `src/widgets/datatable.jl:435` | column header (name + sort indicator) | | `src/paged/render.jl:27` | paged column header (name + sort/filter indicators) | | `src/paged/render.jl:279` | error overlay text | | `src/widgets/table.jl:112` | Table cell | | `src/widgets/barchart.jl:68` | BarChart label | | `src/app.jl:669` | export-modal font name | Editor widgets (`codeeditor.jl`, `textarea.jl`) were checked and are **not** affected — their lines are `Vector{Char}`, so the slices are already character-indexed. `layout.jl` constraint deserialization is internal ASCII and safe. ### Tests Regression tests added that **sweep buffer/column widths** so the truncation boundary lands at every byte offset within a multibyte run (em-dash = 3 bytes), guaranteeing the mid-character cut is exercised: - `test/test_widgets_extended.jl` — DataTable cells + headers - `test/test_paged_datatable.jl` — paged cells, headers, error overlay - `test/test_widgets_coverage.jl` — Table cells - `test/test_core.jl` — BarChart labels Each test was verified to fail on the pre-fix code and pass after. Full suite: 5074/5074 passing.

ShauryaAttreya37 commented

2026-06-23 01:35:51 -07:00

(Migrated from github.com)

Check your discord sometimes too 😄

ShauryaAttreya37 commented

2026-06-23 01:36:17 -07:00

(Migrated from github.com)

or do we have to talk here from now on?

ShauryaAttreya37 commented

2026-06-23 01:46:37 -07:00

(Migrated from github.com)

I'm doing so much work now!
learnt a lot

I'm doing so much work now! learnt a lot

ShauryaAttreya37 commented

2026-06-29 19:56:50 -07:00

(Migrated from github.com)

@kahliburke Dr KB !

kahliburke commented

2026-06-30 00:38:34 -07:00

(Migrated from github.com)

Fixed in v2.3.0 (released).

Root cause was truncating overlong text by byte-indexing a String while the length guard counts characters — so a cut inside a multibyte UTF-8 char (e.g. the 3-byte em-dash) threw StringIndexError. Fixed by switching to character-safe first(s, n) at every truncation site: DataTable (cell + header), PagedDataTable (cell + header + error overlay), Table, BarChart, and the export-modal font name. Editor widgets (codeeditor/textarea) operate on Vector{Char} and were unaffected.

Regression tests added that sweep buffer/column widths so the truncation boundary lands at every byte offset within a multibyte run (each verified to fail before the fix, pass after).

Fix: c82c78a..0a0e940 (see f180230)
Release: https://github.com/kahliburke/Tachikoma.jl/releases/tag/v2.3.0
Reported downstream in kahliburke/Kaimon.jl#29

Fixed in v2.3.0 (released). Root cause was truncating overlong text by **byte**-indexing a `String` while the length guard counts **characters** — so a cut inside a multibyte UTF-8 char (e.g. the 3-byte em-dash) threw `StringIndexError`. Fixed by switching to character-safe `first(s, n)` at every truncation site: `DataTable` (cell + header), `PagedDataTable` (cell + header + error overlay), `Table`, `BarChart`, and the export-modal font name. Editor widgets (`codeeditor`/`textarea`) operate on `Vector{Char}` and were unaffected. Regression tests added that sweep buffer/column widths so the truncation boundary lands at every byte offset within a multibyte run (each verified to fail before the fix, pass after). - Fix: c82c78a..0a0e940 (see f180230) - Release: https://github.com/kahliburke/Tachikoma.jl/releases/tag/v2.3.0 - Reported downstream in kahliburke/Kaimon.jl#29

kahliburke referenced this issue from a commit

2026-08-08 02:39:53 -07:00

Fix StringIndexError when truncating multibyte cell/header text

Rows
Columns

DataTable render crashes with StringIndexError on multibyte cell text #36

Summary

Stack trace (reported via Kaimon)

Cause

Fix

Full scope: same byte-index-vs-character bug found at 6 sites

Tests