Lots of bootstrap script clean ups - #2326
Open
mmlb wants to merge 8 commits into
Open
Conversation
mmlb
force-pushed
the
manny/psql-1450-minimize-apt-get-updatesupgrades-1-at-start-1-at-end
branch
2 times, most recently
from
July 30, 2026 23:08
7e7df68 to
445e111
Compare
PostgreSQL Extension Dependency Analysis: PR #2326
SummaryNo extensions had dependencies with MAJOR version updates. Full Analysis ResultsPostgreSQL 15 Extension DependenciesPostgreSQL 17 Extension DependenciesOrioleDB 17 Extension Dependencies |
PostgreSQL Package Dependency Analysis: PR #2326
SummaryNo packages had MAJOR version updates. Full Analysis ResultsPostgreSQL 15 Dependency ChangesExtracting PostgreSQL 15 dependencies...
Runtime Closure Size
Raw Dependency ClosurePostgreSQL 17 Dependency ChangesExtracting PostgreSQL 17 dependencies...
Runtime Closure Size
Raw Dependency Closure |
mmlb
force-pushed
the
manny/psql-1450-minimize-apt-get-updatesupgrades-1-at-start-1-at-end
branch
2 times, most recently
from
July 31, 2026 21:42
6246142 to
53dabbf
Compare
Most of these files aren't being used so no point having them in the tree. Its a little weird to have cleanup-qemu.sh in ebssurrogate subdir but there's already one file there so meh.
These are all pretty mechanical cleanups/simplifications + making
shellcheck happy. Using <<- flavored heredocs is nice to have because we
get better indentation based code scanning. sudo/su isn't needed any of
the invocations, the whole script is run as root in all cases. I used
[[ ]] and (( )) bashism because they are better than POSIX they replace.
I also dropped unnecessary `${}` and `var="..."` for both consistency
and because I like it better that way :D. Also because I think over-user
of `${}` can lead to thinking its same as "$" for safety but its not and
leads to inconsistent use and assignment doesn't need quotes because
expansion does not take place.
mmlb
force-pushed
the
manny/psql-1450-minimize-apt-get-updatesupgrades-1-at-start-1-at-end
branch
2 times, most recently
from
July 31, 2026 23:34
204eded to
30be7ff
Compare
We are wasting a bunch of time and brain power keeping track of package installs, so lets minimize them while keeping the grouping/messages intact. I did move the package installs from setup_grub into update_install_packages because there were already some there and having similar package sets in 2 locations makes no sense. I moved them out of setup_grub because we package installation should all be done early and ASAP IMO.
It turns out that most of the code deleted here was actually not doing anyting useful. For example, switch_mirror modifies /etc/apt/sources.list in place but the file doesn't have any mirrors configured there, so it did nothing! Here's the contents from an instance I just fired up (AMI=ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-amd64-server-20260604): ``` ubuntu@ip-172-31-26-227:~$ cat /etc/apt/sources.list # Ubuntu sources have moved to the /etc/apt/sources.list.d/ubuntu.sources # file, which uses the deb822 format. Use deb822-formatted .sources files # to manage package sources in the /etc/apt/sources.list.d/ directory. # See the sources.list(5) manual page for details. ``` And the actual mirror config is in /etc/apt/sources.list.d/ubuntu.sources: ``` ubuntu@ip-172-31-26-227:~$ cat /etc/apt/sources.list.d/ubuntu.sources ## Note, this file is written by cloud-init on first boot of an instance ## modifications made here will not survive a re-bundle. --- 8< --- Types: deb URIs: http://us-east-2.ec2.archive.ubuntu.com/ubuntu/ Suites: noble noble-updates noble-backports Components: main universe restricted multiverse Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg ## Ubuntu security updates. Aside from URIs and Suites, ## this should mirror your choices in the previous section. Types: deb URIs: http://security.ubuntu.com/ubuntu Suites: noble-security Components: main universe restricted multiverse Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg ``` Also we weren't being consistent with APT_OPTIONS usage, qemu-bootstrap-nix.sh wasn't using it all and chroot-bootstrap-nix.sh doesn't consitently use it. chroot doesn't use it to install clout-init which brings in software-properties-common which brings in add-apt-repository. Speaking of which add-apt-repository, its not needed anymore since we already have universe enabled by the default sources file. So this commit deletes a lot of confusing, unused and/or inconsistently used code, replacing it with simpler more consistent use. We get fallback handling by apt itself by adding multiple mirrors in URI, apt tries first (the regional) and falls back to global ubuntu repos if there's an issue. We also setup a temporary apt config that forces the APT_OPTIONS for all apt calls so we don't need to do it in every call site.
No yum handling necessary and commented out code should be deleted.
… scripts One is enough.
I added a second call to update_and_upgrade_apt in nix-provision so we match the overall flow done in the bootstrap scripts. Also ensures upgraded packages at the end to pick up any new ones.
mmlb
force-pushed
the
manny/psql-1450-minimize-apt-get-updatesupgrades-1-at-start-1-at-end
branch
from
August 1, 2026 00:47
30be7ff to
051bb1d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
Maintenance
What is the current behavior?
So much confusing code here when apt can do it better and its not used in all apt calls. Non nix'd scripts aren't shellcheck safe. Bunch of dead/unused code hanging around. We run apt-get upgrade all over the ami/qemu image build stages.
What is the new behavior?
We now let apt handle fallbacks itself. All the scripts use apt with the same settings now. I deleted unused files from scripts/ and moved those that were used into ebssurrogate/scripts. I enabled shellcheck in git-hooks and excluded currently failing files.
This is clean ups/refactors in preparation of some other changes in ansible that will come next.