Skip to content
Embedded Data
Edit
On this page

How dirstat classifies files as text or binary, how canonical format aliases merge group names, and which directories are excluded by default.

#Embedded Data

dirstat embeds several data files into the binary at build time via go:embed. These control file classification and default traversal behavior. There are no runtime config files for classification -- changing the lists requires a rebuild.

The embedded data lives under internal/config/data/ and is loaded by the internal/config package.

#Text vs. binary classification

Every file dirstat encounters is classified as either text or binary. This classification determines whether lines of code (LOC) are counted for the file (binary files have no LOC stats), how the file's row is colored in table output (text and binary rows use different theme colors), and whether the file appears in --type text or --type binary filtered output. The classification is strict: every file is one or the other, never "unknown." The mechanism depends on the grouping method:

Text vs. binary classification
MethodFiles with extensionFiles without extension
extExtension list lookupAlways binary
typeContent sniffingContent sniffing
hybrid (default)Extension list lookup, falling back to content sniffing when the extension is not in the listContent sniffing

A zero-byte file is text under every method: an empty file has no bytes that could make it binary, and it contributes 0 lines of code.

#Text extension list

The text extension list is embedded from internal/config/data/text_extensions.txt via go:embed and contains approximately 210 extensions covering programming languages, shell scripts, web technologies, data formats, documentation markup, build systems, game engines, and shaders. A file is text if its normalized extension (last-dot suffix, lowercased, dot stripped) appears in the list.

A miss is not a verdict. In ext mode the file is binary, but in hybrid mode the file is content-sniffed and the MIME rule decides, so text formats outside the list still count their lines of code. Deliberately ambiguous extensions are left out on purpose: .lock is TOML text for uv and Poetry but binary for other tools, so every .lock file is judged by its own content.

Programming

Text extension list
ExtensionFormat
.pyPython
.rbRuby
.jsJavaScript
.tsTypeScript
.jsxJSX
.tsxTSX
.mjsES module JS
.cjsCommonJS
.coffeeCoffeeScript
.javaJava
.ktKotlin
.ktsKotlin Script
.scalaScala
.groovyGroovy
.cljClojure
.cljsClojureScript
.cC
.hC/C++ header
.cppC++
.hppC++ header
.ccC++
.cxxC++
.hxxC++ header
.c++C++
.h++C++ header
.csC#
.fsF#
.vbVisual Basic
.goGo
.rsRust
.swiftSwift
.phpPHP
.plPerl
.pmPerl module
.perlPerl
.luaLua
.tclTcl
.rR
.RR
.rmdR Markdown
.mObjective-C
.mmObjective-C++
.asmAssembly
.sAssembly
.SAssembly
.dD
.nimNim
.zigZig
.vV
.odinOdin
.elmElm
.hsHaskell
.lhsLiterate Haskell
.mlOCaml/SML
.mliOCaml interface
.ocamlOCaml
.erlErlang
.hrlErlang header
.exElixir
.exsElixir script
.lispLisp
.clCommon Lisp
.lspLisp
.scmScheme
.ssScheme
.rktRacket
.fFortran
.f90Fortran 90
.f95Fortran 95
.forFortran
.ftnFortran
.pasPascal
.ppPascal
.incInclude file
.adaAda
.adbAda body
.adsAda spec
.cobCOBOL
.cblCOBOL
.cpyCOBOL copybook
.proProlog
.PProlog

Game/Shader

Text extension list
ExtensionFormat
.gdGDScript
.gdshaderGodot shader
.tscnGodot scene
.tresGodot resource
.godotGodot project
.uidGodot resource UID
.glslGLSL shader
.hlslHLSL shader
.wgslWGSL shader
.fragFragment shader
.vertVertex shader

Web

Text extension list
ExtensionFormat
.htmlHTML
.htmHTML
.xhtmlXHTML
.shtmlServer-side HTML
.cssCSS
.scssSCSS
.sassSass
.lessLess
.stylusStylus
.svgSVG
.vueVue
.svelteSvelte
.astroAstro
.svxSvelte Markdown

Data/Config

Text extension list
ExtensionFormat
.xmlXML
.xslXSLT
.xsltXSLT
.xsdXML Schema
.dtdDTD
.jsonJSON
.json5JSON5
.jsonlJSON Lines
.jsoncJSON with comments
.yamlYAML
.ymlYAML
.tomlTOML
.iniINI
.cfgConfig
.confConfig
.configConfig
.envEnv file
.htaccessApache config
.csvCSV
.tsvTSV
.sqlSQL
.graphqlGraphQL
.gqlGraphQL
.protoProtocol Buffers
.avscAvro schema
.prismaPrisma schema
.plistProperty list
.servicesystemd unit
.desktopDesktop entry
.nixNix
.dhallDhall
.jsonnetJsonnet
.hclHCL
.tfTerraform
.tfvarsTerraform vars

Docs

Text extension list
ExtensionFormat
.mdMarkdown
.markdownMarkdown
.mdownMarkdown
.mkdMarkdown
.mkdnMarkdown
.rstreStructuredText
.txtPlain text
.textPlain text
.texTeX
.latexLaTeX
.bibBibTeX
.adocAsciiDoc
.asciidocAsciiDoc
.orgOrg-mode
.podPerl POD
.rdocRDoc
.wikiWiki markup
.creoleCreole markup
.textileTextile
.logLog file
.diffDiff/patch
.patchPatch

Shell

Text extension list
ExtensionFormat
.awkAWK
.sedsed
.ps1PowerShell
.psm1PowerShell module
.psd1PowerShell data
.batBatch
.cmdWindows command
.shShell
.bashBash
.zshZsh
.fishFish
.kshKorn shell
.cshC shell
.tcshTENEX C shell
.vimVim script
.vimrcVim config
.elEmacs Lisp
.elispEmacs Lisp

Build

Text extension list
ExtensionFormat
.mkMakefile
.makefileMakefile
.cmakeCMake
.modGo module file
.sumGo checksum file
.gradleGradle
.sbtSBT
.cabalCabal
.cargoCargo
.gemfileGemfile
.gemspecGemspec
.podfileCocoaPods
.packagePackage file
.composerComposer
.pipfilePipfile
.pyprojectpyproject
.gitignore.gitignore
.gitattributes.gitattributes
.gitmodules.gitmodules
.gitconfig.gitconfig
.dockerfileDockerfile
.dockerignore.dockerignore
.editorconfigEditorConfig
.eslintrcESLint config
.prettierrcPrettier config
.babelrcBabel config
.npmrcnpm config
.yarnrcYarn config
.inputrcReadline config

#Text MIME type list

The text MIME type list is embedded from internal/config/data/text_mimetypes.txt via go:embed and enumerates non-text/ MIME types that should be treated as text for classification purposes. When content sniffing is performed (always in type mode; for extensionless files in hybrid mode), dirstat uses the github.com/gabriel-vasile/mimetype library to detect the file's MIME type. The file is classified as text if:

  1. The detected MIME type starts with text/ (e.g., text/plain, text/html), or
  2. The detected MIME type is in the text MIME type list

The second rule exists because many text-based formats have MIME types under application/ or other top-level types rather than text/. For example, application/json is clearly text but does not start with text/.

The list contains 48 MIME types:

Text MIME type list
MIME TypeCategory
application/jsonData interchange
application/xmlMarkup
application/javascriptProgramming
application/ecmascriptProgramming
application/x-javascriptProgramming (legacy)
application/x-shShell
application/x-shellscriptShell
application/x-perlProgramming
application/x-pythonProgramming
application/x-rubyProgramming
application/x-phpProgramming
application/x-httpd-phpProgramming
application/x-awkShell
application/x-gawkShell
application/x-nawkShell
application/x-sedShell
application/sqlData
application/graphqlData
application/ld+jsonData interchange
application/manifest+jsonData interchange
application/x-yamlData
application/yamlData
application/tomlData
application/x-tomlData
application/x-wine-extension-iniConfig
application/xhtml+xmlMarkup
application/rss+xmlMarkup
application/atom+xmlMarkup
application/soap+xmlMarkup
application/xslt+xmlMarkup
application/mathml+xmlMarkup
application/x-texDocumentation
application/x-latexDocumentation
application/rtfDocumentation
application/postscriptDocumentation
application/x-troffDocumentation
application/x-troff-manDocumentation
application/x-troff-meDocumentation
application/x-troff-msDocumentation
application/x-infoDocumentation
application/x-texinfoDocumentation
application/x-makerDocumentation
application/csvData
application/x-emptyEmpty file
inode/x-emptyEmpty file
image/svg+xmlVector graphics (text-based)

#Canonical format aliases

The alias table is embedded from internal/config/data/canonical_formats.txt via go:embed and is read only under --formats canonical. Each line is an old new pair: a raw group name -- a normalized extension or a sniffed MIME type -- and the format name that group is counted under. Lines starting with # are comments, and a line that is not a pair is a hard error rather than a skipped line.

The table merges module variants into their language (mjs and cjs into js, mts and cts into ts), headers into their language (h into c, hh and hpp into cpp), and the script MIME types the sniffer emits into format names (text/x-python into py, text/x-shellscript into sh, and the same for Perl, Ruby, PHP, Lua and Tcl). A group name the table does not list is counted under its raw name.

Canonical mode also names extensionless scripts by their shebang interpreter, before the MIME sniff gets a say -- the sniffer has no signature for most scripts and answers text/plain, which names nothing. The interpreter is read after stripping an env wrapper (its options, its -S/--split-string forms, and any NAME=value assignments), unwrapping the uv run X and uvx X runner forms, and dropping a trailing version suffix, so #!/usr/bin/env -S uv run python and #!/usr/bin/python3.12 both count under py. The sniff still runs: it decides text vs. binary, and the shebang only names the group. An interpreter outside the mapping falls through to the sniffed MIME type, which then goes through the alias table.

#Default excludes

When --exclude is not explicitly passed (and no config file sets exclude), dirstat skips these directories and files by exact base-name match during traversal. Excluded directories are pruned entirely -- their contents are not descended into, counted, or classified.

Default excludes
DirectoryDescription
.gitGit repository metadata
node_modulesnpm dependencies
.venvPython virtual environment
vendorVendored dependencies (Go, PHP, etc.)
buildBuild output
distDistribution/build output
targetBuild output (Rust, Maven, etc.)
zig-outZig build output
zig-pkgZig package cache
.nextNext.js build output
.svelte-kitSvelteKit build output
__pycache__Python bytecode cache
.mypy_cachemypy type-checker cache
.ruff_cacheRuff linter cache
.pytest_cachepytest cache
.hypothesisHypothesis test data
.gradleGradle build cache
.ideaJetBrains IDE config
.wranglerCloudflare Wrangler state
.vscodeVS Code config

#Overriding exclusions

The default exclude list is replaced entirely when you provide your own exclusion list, whether via the --exclude command-line flag or the exclude key in a TOML config file. There is no additive merging -- any explicit exclusion list completely supersedes all built-in defaults, so you must re-specify any default entries you still want excluded.

#Via the --exclude flag

Pass --exclude one or more times on the command line to set the exclusion list for a single invocation. Each --exclude value is an exact base-name match against directories and files encountered during traversal. When any --exclude flag is present, the entire built-in default list is replaced by the values you specify, so only those names are skipped.

dirstat scan --exclude node_modules --exclude .git

This scans everything except node_modules and .git. The 18 other built-in defaults are no longer excluded.

To scan with no exclusions at all, the config file approach is required (see below).

#Via a config file

Set the exclude key in a TOML config file passed with --config to control exclusions persistently across invocations. The config file uses a string array where each element is an exact base-name to skip during traversal. Setting exclude to an empty array scans everything with no exclusions at all.

TM toml
# scan.toml
exclude = ["node_modules", ".git", "dist"]
dirstat scan --config scan.toml

To scan everything with no exclusions:

TM toml
exclude = []

A key set both in the config file and on the command line is a hard error -- use one or the other. See the spec (R41-R47) for full config file semantics.

Search