How to Map an Unfamiliar Repository With find and Abbreviatory

The first thing anyone does in a new repository is look around. A person runs tree and gives up after a few screens. A coding agent runs find . -type f and gets every path, and on a large project that’s thousands of lines spent before it has read a single file. Most of them say the same thing: another test fixture, another Go file in the same package.

The listing in this example is find src -type f over Go’s own source tree, which is big enough to make the point: 10,703 paths in 1,097 directories, 454.6 kB.

A find listing of 10,703 paths in 1,097 directories is reduced to 1,484 bytes: totals, the most common extensions, then groups by directory and extension such as 867 txt files in src/cmd/go/testdata/script and 563 Go files in src/runtime.

The Map

find src -type f | sort | abbreviatory reduce -budget 1500

[abbreviatory] directory listing, 10,703 lines (454.6 kB), ref 6d668613b8a2
listing: 10,703 files in 1,097 directories; most: .go 7,117, .txt 1,276, .s 607, .crt 407
OTHER (1,408 kinds, 10,703 lines):
- x867 src/cmd/go/testdata/script/*.txt [ref 6d668613b8a2:g412]
- x563 src/runtime/*.go [ref 6d668613b8a2:g1288]
- x405 src/crypto/x509/testdata/nist-pkits/certs/*.crt [ref 6d668613b8a2:g904]
- x298 src/syscall/*.go [ref 6d668613b8a2:g1350]
- x280 src/cmd/vendor/golang.org/x/sys/unix/*.go [ref 6d668613b8a2:g617]
- x249 src/cmd/go/testdata/mod/*.txt [ref 6d668613b8a2:g410]
- x224 src/net/*.go [ref 6d668613b8a2:g1234]
...

Files are grouped by directory and extension, biggest groups first. The first line already tells you a lot: mostly Go, a large body of text test scripts, a fair amount of assembly, a pile of certificates. The groups say where the weight is. The go command’s tests are 867 script files in one folder, the runtime is 563 Go files and 185 assembly files, and 405 of the certificates are one test suite’s.

Then One Corner

Every group is a reference. To see the 224 files of the net package:

abbreviatory expand 6d668613b8a2:g1234

Or list just that corner with sizes, and let the report pick out the largest files:

ls -lR src/net/http src/crypto/tls | abbreviatory reduce

On those two packages the report opens with listing: 275 files in 18 directories, 3.9 MB and names the three largest files, led by net/http/h2_bundle.go at 394.1 kB, which is usually the file you want to know about before an agent tries to read it whole.

What It Reads

find and fd paths, ls -l with sizes, ls -R with its directory headings, tree with its own count at the end, and du, where the total is the largest row rather than the sum. “Permission denied” and “No such file” lines are errors and come first, so a listing that silently skipped a folder says so.

du -a can’t tell a file without an extension from a directory, and counts it as one. In tree output without its closing count, an empty directory counts as a file. Hidden folders such as .git are listed like any other unless you leave them out of the find, which you usually want: find . -path ./.git -prune -o -type f -print.