strsplit

strsplit splits strings into bounded-size chunks without breaking Unicode characters, keeping human-readable boundaries (spaces, punctuation, and newlines).

Table of contents

Part of nurago, a collection of independent Go packages for backend services.

import "github.com/tecnickcom/nurago/pkg/strsplit"

Package strsplit splits strings into bounded-size chunks without breaking Unicode characters, keeping human-readable boundaries (spaces, punctuation, and newlines).

How It Works

The package exposes two functions:

  • [Chunk]: splits a full text block, prioritizing newline boundaries first, trimming whitespace per line, then delegating long lines to ChunkLine.
  • [ChunkLine]: splits a single line by maximum byte size, ensuring the split point is at a rune boundary and preferring the closest separator before the limit.

Separator preference order in [ChunkLine]:

  1. Unicode whitespace.
  2. Unicode punctuation (kept with the preceding chunk).
  3. Hard UTF-8-safe cut when no separator exists.

Sizing Semantics

size is a maximum byte length, not a rune count. Produced chunks never exceed size bytes, with one unavoidable exception: a single rune wider than size is emitted whole (splitting it would produce invalid UTF-8), so that one chunk may exceed size.

Line And Whitespace Handling

Chunk treats only the newline byte “\n” as a line boundary. A carriage return “\r” is not a boundary on its own; it is removed only when it is adjacent to a “\n” (or otherwise leading/trailing), because each line is whitespace-trimmed. Other Unicode line separators (U+2028, U+2029, U+0085, …) are likewise not treated as boundaries and may remain inside a chunk. Empty and whitespace-only lines are dropped, so blank lines between paragraphs never yield empty chunks and are not preserved.

Chunk Limit

Both functions support an optional chunk limit n:

  • n > 0: return at most n chunks.
  • n < 0: unlimited chunks.
  • n == 0: return nil.

Return Values

Both functions return nil only for invalid arguments (size < 1 or n == 0). Otherwise they return a non-nil slice that may be empty (for example when the input is empty or contains only whitespace). Produced chunks are always whitespace-trimmed and never empty.

Usage

chunks := strsplit.Chunk(text, 280, -1)    // split full text block
lineParts := strsplit.ChunkLine(line, 64, 3) // at most 3 chunks

When To Use

  • Text must fit a field or message size limit measured in bytes.
  • Splitting must not cut through a multi-byte UTF-8 character or an emoji.
  • Chunks should break at spaces, punctuation, or newlines where possible.

Example

str := "helloworld\nbellaciao"
d := strsplit.Chunk(str, 5, 3)

fmt.Println(d)

// Output:
// [hello world bella]

Full source is in example_strsplit_test.go. More runnable examples are on pkg.go.dev.

Dependencies

This package reaches no external module: it uses only the Go standard library.