ci: compact older builds on the storage server #697

2026-09-16T05:39:57Z

Burer commented

2026-09-16 05:39:57 +00:00

Every push to main publishes 4 images per language plus the unpacked build tree, and nothing on the storage server ever shrinks. kolibri.raw is a fixed 128 MiB regardless of content, and kolibri.img ends up stored three times per language.

This adds a compaction step at the end of the deploy playbook. It runs on the storage server itself, so nothing extra crosses the network.

The build that was just published stays as it is: bare images and a browsable tree.
Every older build keeps its images zipped and loses the unpacked tree, which is rebuildable from the commit anyway.

*.distr.zip, sha256sums.txt and build.log are never touched.

Every push to main publishes 4 images per language plus the unpacked build tree, and nothing on the storage server ever shrinks. `kolibri.raw` is a fixed 128 MiB regardless of content, and `kolibri.img` ends up stored three times per language. This adds a compaction step at the end of the deploy playbook. It runs on the storage server itself, so nothing extra crosses the network. - The build that was just published stays as it is: bare images and a browsable tree. - Every older build keeps its images zipped and loses the unpacked tree, which is rebuildable from the commit anyway. `*.distr.zip`, `sha256sums.txt` and `build.log` are never touched.

CODEOWNERS rules requested review from devops 2026-09-16 05:39:58 +00:00

Sweetbread requested changes

2026-09-23 13:36:30 +00:00

Dismissed

.gitea/ansible/playbooks/deploy.yml Outdated

						
				@@ -57,0 +57,4 @@

				    # older builds shrink to zipped images and lose their unpacked tree; the newest keeps both

				    - name: Looking for zip on the storage server

				      ansible.builtin.shell: command -v zip

Sweetbread commented

2026-09-23 13:35:59 +00:00

Are you sure zip is better for compressing? Why not 7z?

Burer commented

2026-09-25 09:57:46 +00:00

Feels like zip is more common and supported out of box in different OS.
Also runs much faster on our server, without much bigger size.
Example for kolibri.raw:

method	size	from original	time
zip -9	56 270 041	41.9%	17 с
7z -tzip -mx9 (deflate)	54 260 068	40.4%	112 с
zstd -12	50 916 883	37.9%	7 с
zstd -19	46 533 976	34.7%	38 с
xz -6	44 025 416	32.8%	35 с
xz -9	43 146 400	32.1%	95 с
7z LZMA2 -mx9	43 116 123	32.1%	40 с

Feels like zip is more common and supported out of box in different OS. Also runs much faster on our server, without much bigger size. Example for `kolibri.raw`: method | size | from original | time -- | -- | -- | -- zip -9 | 56 270 041 | 41.9% | 17 с 7z -tzip -mx9 (deflate) | 54 260 068 | 40.4% | 112 с zstd -12 | 50 916 883 | 37.9% | 7 с zstd -19 | 46 533 976 | 34.7% | 38 с xz -6 | 44 025 416 | 32.8% | 35 с xz -9 | 43 146 400 | 32.1% | 95 с 7z LZMA2 -mx9 | 43 116 123 | 32.1% | 40 с

Sweetbread commented

2026-09-25 22:13:34 +00:00

Well, we have and had a lot of versions and I don't think it would be difficult to install some rare archiver to extract an old version for some reason

Well, we have and had a *lot* of versions and I don't think it would be difficult to install some rare archiver to extract an old version for some reason

Burer commented

2026-09-26 14:12:13 +00:00

Switched to zstd -12. Measured on the storage server, per build (680 MB of images, three languages):

	zip -9	zstd -12
size after	314 MB	290 MB
time	54 s	42 s

Switched to zstd -12. Measured on the storage server, per build (680 MB of images, three languages): | | zip -9 | zstd -12 | |---|---|---| | size after | 314 MB | 290 MB | | time | 54 s | 42 s |

Sweetbread marked this conversation as resolved

.gitea/utils/compact-builds.sh Outdated

						
				@@ -0,0 +24,4 @@

				# images and the unpacked tree both sit exactly two levels below <version>

				leftovers() {

				    find "$1" -mindepth 2 -maxdepth 2 \

Sweetbread commented

2026-09-23 13:30:14 +00:00

You defined root var, but still use positional argument

You defined `root` var, but still use positional argument

Burer commented

2026-09-25 12:32:10 +00:00

Fixed.

Sweetbread commented

2026-09-25 22:14:13 +00:00

No

Burer commented

2026-09-26 13:56:10 +00:00

leftovers is not called with the storage root - it takes one build directory (leftovers "$dir", where dir="$root/$version"), so $root would be the wrong path to search. The argument is now named:

    leftovers() {
        local build=$1
        find "$build" -mindepth 2 -maxdepth 2 ...

If you meant something else by "still use positional argument", say which line and I will change it.

`leftovers` is not called with the storage root - it takes one build directory (`leftovers "$dir"`, where `dir="$root/$version"`), so `$root` would be the wrong path to search. The argument is now named: ```sh leftovers() { local build=$1 find "$build" -mindepth 2 -maxdepth 2 ... ``` If you meant something else by "still use positional argument", say which line and I will change it.

Sweetbread marked this conversation as resolved

.gitea/utils/compact-builds.sh Outdated

						
				@@ -0,0 +52,4 @@

				    compacted=$((compacted + 1))

				    echo "compacted $version"

				done < <(find "$root" -mindepth 1 -maxdepth 1 -type d \

				              ! -name "$current" -mmin "+$quiet_minutes" -printf '%f\n' | sort)

Sweetbread commented

2026-09-23 13:34:10 +00:00

It's a bad idea to use mtime. Better keep last N builds

It's a bad idea to use `mtime`. Better keep last N builds

Burer commented

2026-09-25 12:32:14 +00:00

Fixed.

Sweetbread marked this conversation as resolved

CODEOWNERS rules requested review from devops 2026-09-25 10:52:44 +00:00

Sweetbread approved these changes

2026-09-26 14:44:33 +00:00

Doczom approved these changes

2026-09-26 18:23:09 +00:00

Burer added 5 commits 2026-09-28 07:26:00 +00:00

ci: compact older builds on the storage server 88d91c1cc5

Every push to main publishes four images per language plus the
unpacked build tree, and nothing ever shrinks: kolibri.raw alone is
a fixed 128 MiB apiece, and kolibri.img lands three times over.

Keep the newest build as it is - bare images and a browsable tree -
and reduce every older one to zipped images. Distribution kits,
checksums and build logs stay untouched.

Capped per run, since the first pass has the whole backlog to get
through, and skips anything touched in the last half hour so a
concurrent deploy is never compacted mid-upload.

Assisted-by: Claude Opus 5 <noreply@anthropic.com>

ci: pick builds to compact by commit count, not mtime 39e29bfbe3

- keep the newest 5 builds uncompacted, ordered by the commit count in the
  directory name; mtime says nothing about build order after a re-upload
- name the leftovers() parameter so it does not read as the script's own $1
- lower the per-run cap to 10 builds

ci: rehearse with DRY_RUN, zip in parallel, yield to the site 7be0d8cf07

- DRY_RUN=1 walks the same selection and changes nothing
- compress one image per core: zip is single threaded and a build holds nine,
  which took a build from 105s to 54s on the storage server
- run zip under nice/ionice so compaction does not disturb the NFS export

ci: archive a build before dropping its unpacked tree 1629406ba1

A zip that fails midway used to leave the build stripped of its tree and
still holding bare images. Archiving first makes a failure leave it merely
uncompacted, and the next run selects it again.

ci: compress old builds with zstd instead of zip

Test PR / Build (en_US) (pull_request) Successful in 3m22s

Details

Test PR / Build (es_ES) (pull_request) Successful in 2m27s

Details

Test PR / Build (ru_RU) (pull_request) Successful in 2m17s

Details

d863551e39

zstd -12 beats zip -9 on both counts on the storage server: a build goes to
290 MB instead of 314 MB, and takes 42s instead of 54s. These are archives,
not everyday downloads, so the less familiar extension costs little.

zstd appends .zst to the file it is given, so the subshell that zip needed
to keep paths out of the archive is gone. Each zstd is held to one thread
because the files are already compressed one per core.

Burer force-pushed builds-cleanup from ad797b5fe4 to d863551e39

2026-09-28 07:26:00 +00:00

Compare

CODEOWNERS rules requested review from devops 2026-09-28 07:26:01 +00:00

Burer merged commit 75cb80fd33 into main

2026-09-28 07:32:38 +00:00

Burer deleted branch builds-cleanup

2026-09-28 07:32:39 +00:00

Sign in to join this conversation.

No Reviewers

Sweetbread

Doczom

KolibriOS/devops

3 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: KolibriOS/kolibrios#697