#internal/util

#internal/util

Package util holds the small shared helpers the rest of selfdoc builds on: frontmatter parsing, project manifest and version detection, HTML escaping, path joining, date formatting, title casing, and the Python-compatible string, number and JSON spellings the emitted documents are pinned to.

Nothing here knows about a build, a page or a directive. A helper earns its place here by being needed in more than one package and by having no dependency on any other package of this module.

#UnknownField

const UnknownField = "unknown"

UnknownField is the value [ReadProjectField] returns when no manifest answers -- the sentinel string the Python surface returned, which templates and pages render verbatim.

#PythonSpaceChars

const PythonSpaceChars = `\t\n\v\f\r \x{001c}-\x{001f}\x{0085}\x{00a0}\x{1680}\x{2000}-\x{200a}\x{2028}\x{2029}\x{202f}\x{205f}\x{3000}`

PythonSpaceChars are the members of Python's \s, without the enclosing brackets: the ASCII whitespace characters, the four ASCII information separators, and the Unicode whitespace code points. A pattern that composes whitespace with further characters into one class needs the members, which a nested class cannot express.

#PythonWordChars

const PythonWordChars = `\p{L}\p{N}_`

PythonWordChars are the members of Python's \w where it matches an identifier character, without the enclosing brackets: a letter, a digit or an underscore, in any script.

#PythonSpaceClass

const PythonSpaceClass = "[" + PythonSpaceChars + "]"

PythonSpaceClass is Python's \s.

#PythonNonSpaceClass

const PythonNonSpaceClass = "[^" + PythonSpaceChars + "]"

PythonNonSpaceClass is Python's \S, the complement of [PythonSpaceClass].

#PythonWordClass

const PythonWordClass = "[" + PythonWordChars + "]"

PythonWordClass is Python's \w where it matches an identifier character.

#Frontmatter

type Frontmatter = map[string]any

Frontmatter is the parsed metadata block of a Markdown template, in the hand-rolled dialect selfdoc has always used. Values are one of string, bool, int64, float64 or []string.

#ParseFrontmatter

func ParseFrontmatter(text string) (Frontmatter, string, int)

ParseFrontmatter parses the YAML-like frontmatter of a Markdown document.

When text starts with "---", the key/value pairs up to the next line that is exactly "---" after trimming become the metadata and everything after it becomes the body, with the body's leading blank lines removed. When there is no frontmatter -- text does not start with "---", or the closing fence is missing -- the metadata is empty and the body is text unchanged.

The dialect is deliberately not YAML, and existing documents depend on every rule below:

- A line is split on its FIRST colon; a line with no colon is skipped, as is an empty line and a line starting with "#". - Key and value are both trimmed of surrounding whitespace. - One pair of wrapping quotes (either ' or ") is stripped from the value BEFORE any other interpretation, so tags: "[a, b]" is still a list and draft: "true" is still a boolean. - [a, b, c] becomes a []string of the trimmed, non-empty items. - Otherwise a value equal to "true" or "false" case-insensitively becomes a bool; else a Python-integer literal becomes an int64; else a Python-float literal becomes a float64; else the value stays a string. - There is no nesting, no multi-line value and no comment stripping inside a value.

The third return value is the number of source lines consumed before the body's first line -- the fence, the metadata lines, the closing fence and any blank lines stripped after it -- so a caller can map a body line number back to its line in text. It is zero when there is no frontmatter.

#ParsePythonInt

func ParsePythonInt(s string) (int64, bool)

ParsePythonInt parses s the way Python's int() does for a base-10 string, reporting whether it is a valid integer literal. Underscores between digits are accepted, as Python accepts them; a hexadecimal or octal prefix is not.

One bound Python does not have: a literal outside the int64 range is reported invalid, where Python's arbitrary-precision int would accept it. In frontmatter such a value then stays a string instead of becoming a number.

#ParsePythonFloat

func ParsePythonFloat(s string) (float64, bool)

ParsePythonFloat parses s the way Python's float() does, reporting whether it is a valid float literal. It accepts underscores between digits and the signed spellings of inf, infinity and nan, and rejects the hexadecimal float syntax Go's own parser would otherwise accept.

#DetectProjectVersion

func DetectProjectVersion(baseDir, fallback string) string

DetectProjectVersion reads the project's version out of its manifest files.

A source language declared in baseDir's selfdoc.json picks the manifest the version is read from (go -> VERSION, python -> pyproject.toml, js/node -> package.json), so a polyglot repository's incidental manifests -- a private browser-test harness's package.json at the root of a Go project, whose version field is conventionally 0.0.0 -- cannot win. That is the version counterpart of the rule [ReadProjectField] applies to the project name.

Without a declaration, or when the picked manifest is absent or carries no version, the original lookup chain applies: pyproject.toml's [project].version, then package.json's "version", then a plain-text VERSION file. It returns fallback when no version is found.

#ReadProjectField

func ReadProjectField(baseDir, field string) string

ReadProjectField reads a project metadata field from the project's manifest.

A source language declared in baseDir's selfdoc.json picks the manifest (go -> go.mod, python -> pyproject.toml, js/node -> package.json), so a polyglot repository's incidental manifests cannot win. Without a declaration, or when the picked manifest is absent or unreadable, the original lookup chain applies: pyproject.toml, then package.json, then go.mod.

The "version" field is answered by [DetectProjectVersion] with [UnknownField] as the fallback. Every other unanswerable field is [UnknownField] too -- this surface reports absence in band, as the templates consuming it expect, and never as an error.

#IsPythonSpace

func IsPythonSpace(r rune) bool

IsPythonSpace reports whether r is whitespace by Python's str.isspace rule, the predicate str.strip and str.split use.

It is [unicode.IsSpace] plus the four information separators U+001C through U+001F, which Python counts as whitespace and Go does not.

#PythonStrip

func PythonStrip(s string) string { return strings.TrimFunc(s, IsPythonSpace) }

PythonStrip reproduces Python's str.strip() with no argument: both ends trimmed of every character [IsPythonSpace] accepts.

[strings.TrimSpace] is close but not the same -- it trims what [unicode.IsSpace] accepts, which omits the four information separators Python trims.

#PythonLStrip

func PythonLStrip(s string) string { return strings.TrimLeftFunc(s, IsPythonSpace) }

PythonLStrip reproduces Python's str.lstrip() with no argument.

#PythonRStrip

func PythonRStrip(s string) string { return strings.TrimRightFunc(s, IsPythonSpace) }

PythonRStrip reproduces Python's str.rstrip() with no argument.

#PythonFields

func PythonFields(s string) []string { return strings.FieldsFunc(s, IsPythonSpace) }

PythonFields reproduces Python's str.split() with no argument: s split on runs of [IsPythonSpace] runes, with no empty items, so leading and trailing whitespace contribute nothing.

#PythonSplitLines

func PythonSplitLines(text string) []string

PythonSplitLines splits text the way Python's str.splitlines() does: on every character in pythonLineBreaks, counting "\r\n" once, and with a trailing terminator producing no final empty line.

#PythonRepr

func PythonRepr(v any) string

PythonRepr renders v the way Python's repr() does -- the spelling every ported diagnostic that interpolates {value!r} was written against.

A string is quoted by [pythonReprString]. A mapping's keys are rendered in sorted order rather than insertion order: Go maps carry no insertion order, and a diagnostic that names an object has to be reproducible. A value of a type Python has no counterpart for falls back to [fmt.Sprint].

#PythonStr

func PythonStr(v any) string

PythonStr renders v the way Python's str() -- and therefore an f-string interpolation -- renders it.

A container renders through [fmt.Sprint] rather than Python's own container repr, which is what every ported call site did: the containers that reach a str() interpolation are rejected by a type check before any message could quote one, so the difference cannot reach a document. A value that really is a container to be shown goes through [PythonRepr].

#PythonStrOrEmpty

func PythonStrOrEmpty(v any) string

PythonStrOrEmpty renders v the way Python's str(value or "") idiom does: a falsy value -- an absent key, a null, a false, a zero, an empty string, an empty container -- becomes the empty string, and anything else becomes its [PythonStr] rendering.

Every optional string key of a config or a frontmatter block was read through that idiom, so an absent key and a declared empty one are the same answer.

#PythonTypeName

func PythonTypeName(v any) string

PythonTypeName renders the name Python's type(value).__name__ gives a decoded JSON or TOML value, for a refusal that reports the type it was handed instead of the one it wanted.

#PythonJSON

func PythonJSON(v any) ([]byte, error)

PythonJSON encodes v the way Python's json.dumps(v, sort_keys=True, separators=(",", ":")) does, byte for byte.

That exact spelling is the hash-store's schema-hash input, so a divergence here silently invalidates every stored hash. The reproduced rules are:

- No whitespace anywhere: "," between items, ":" between a key and its value. - Object keys sorted by code point. Only string keys are accepted, because Python's sort_keys refuses a mixed-type key set. - ensure_ascii: every character outside printable ASCII is escaped as \uXXXX in lowercase hex, with a surrogate pair above the Basic Multilingual Plane. "<", ">" and "&" are NOT escaped -- Python's encoder does no HTML escaping, unlike Go's encoding/json. - Floats render as Python's repr does: shortest round-trip digits, a trailing ".0" on an integral value, and exponential notation only when the decimal point sits at or below position -4 or above position 16. Infinities and NaN render as Python's non-standard Infinity, -Infinity and NaN literals, which json.dumps emits by default.

A nil slice encodes as "[]" and a nil map as "{}" -- Python has no nil collection, so a Go port's unset slice stands for the empty list the Python it replaces would have built. Only a nil interface or nil pointer is "null".

#PythonJSONIndent2

func PythonJSONIndent2(v any) ([]byte, error)

PythonJSONIndent2 encodes v the way Python's json.dumps(v, indent=2, sort_keys=True) does, byte for byte -- the spelling the hash store's own file is written with.

Every rule of [PythonJSON] holds, except that an indent switches Python's separators to "," plus a newline and ": " between a key and its value. An empty object and an empty array still render as "{}" and "[]".

#PythonJSONString

func PythonJSONString(s string) string

PythonJSONString quotes s the way Python's json encoder does under ensure_ascii: the short escapes for backslash, quote, backspace, form feed, newline, carriage return and tab, a \uXXXX escape for every other character outside the printable ASCII range 0x20-0x7E, and a surrogate pair for a code point above the Basic Multilingual Plane.

#PythonFloatRepr

func PythonFloatRepr(f float64) string

PythonFloatRepr renders f the way Python's repr(float) does, which is also what json.dumps writes for a float.

The digits are the shortest decimal string that round-trips. Notation is chosen from the decimal point's position: exponential when it sits at or below -4 or above 16, fixed otherwise, and a fixed rendering always carries a fractional part (so 1.0 is "1.0", never "1"). Infinities and NaN render as json.dumps's non-standard Infinity, -Infinity and NaN literals.

#EscapeHTML

func EscapeHTML(s string) string

EscapeHTML escapes s for insertion into HTML text or a double-quoted attribute value: "&", "<", ">" and the double quote, in that order.

The apostrophe is deliberately NOT escaped, so the output matches Python's html.escape(s, quote=True) rather than Go's html.EscapeString, which also rewrites "'" to "'" and would change every rendered page.

#FormatDateLong

func FormatDateLong(t time.Time) string

FormatDateLong renders t as Python's strftime("%B %-d, %Y") does in the C locale: the English month name, the day of the month without a leading zero, a comma, and the four-digit year -- "September 1, 2026".

#ResolveDirectivePath

func ResolveDirectivePath(baseDir, path string) string

ResolveDirectivePath resolves a directive's path attribute against the project's base directory.

This is the one place filesystem directive paths are resolved, so every directive that reads a path attribute behaves identically and any future normalization or sandboxing has a single home.

#PathJoin

func PathJoin(parts ...string) string

PathJoin reproduces Python's posixpath.join, which is what every ported call site was written against.

It differs from [path/filepath.Join] in two ways that reach real documents: an absolute later element REPLACES everything before it instead of being appended, and the result is never cleaned, so "docs" joined with "../x" stays "docs/../x" rather than collapsing to "x". Use [path/filepath.Join] for a new path that no Python call site constrains.

#TitleCase

func TitleCase(s string) string

TitleCase reproduces Python's str.title().

Every cased character that follows an uncased one is title-cased and every other cased character is lower-cased, with "cased" meaning the Unicode Cased property -- not "alphabetic". An apostrophe is uncased, so "don't" becomes "Don'T", and a digit is uncased too, so "a1b" becomes "A1B". Both are the documented behavior of the Python method this replaces, not accidents.

The full (multi-character) case mappings Python applies are reproduced from the tables below, so a word-initial sharp s becomes "Ss" and a word-initial ff ligature becomes "Ff" as they do in Python, rather than staying put the way the single-rune unicode.ToTitle would leave them.

#DecodeTOML

func DecodeTOML(data []byte) (map[string]any, error)

DecodeTOML decodes a TOML document into the generic Go value every caller in this module validates by hand.

The shapes are the ones the hand-written validations and their pinned refusals were written against: a string, an int64, a float64, a bool, a time.Time for each of the four date-time flavors, a []any for an array, a map[string]any for a table, a []map[string]any for an array of tables, and nested maps for a dotted key. A caller reads them with a type assertion and refuses anything else by name, so a shape stated here is part of every one of those refusals.

#DecodeTOMLFile

func DecodeTOMLFile(path string) (map[string]any, error)

DecodeTOMLFile reads path and decodes it through [DecodeTOML].

#DecodeTOMLOrdered

func DecodeTOMLOrdered(data []byte) (map[string]any, [][]string, error)

DecodeTOMLOrdered decodes like [DecodeTOML] and additionally answers every key the document declares, as its path from the root, in document order.

A Go map has no order, so a renderer whose row order is document order reads the order from here instead. The list carries a table header before the keys written under it, one entry per array-of-tables element, and the keys of an inline table under the key it is bound to -- a sequence a caller replays over the decoded values to rebuild the document's own order.