Repository navigation
Conversation
As far as I can tell, practically all the workarounds in this script are no longer required. A scope unit (as opposed to a service unit) seems to do everything we want. Scope units automatically inherit the environment, avoiding the need to explicitly pass potentially sensitive values on the command line. Despite the 5 year old comment about using a system unit because a user unit may not work in CI, I have found that a user unit works just fine. Conversely, a system scope unit created with sudo breaks access to the Docker socket because it does not set up the supplementary groups. Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
We want environment variables passed through Docker with --env and --env-file to survive the sudo call, but -E cannot be combined with -i. However, we can still effectively get a login shell by calling bash -l. I considered runuser, which doesn't have the stdout/stderr issue we faced earlier, but that has the same limitation as sudo. I also considered setpriv, but that is quite low-level, requiring you to manually fix up variables like HOME. What the comment said about Docker's --user option only applying a single group doesn't appear to be true (anymore?), so we could probably start the container as the sdk user and do the initial privileged tasks with sudo, but that's a bigger change for later. While doing this, I realised that the passing commands through can be done much more simply by passing them as additional arguments to bash. Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
chewi
force-pushed
the
chewi/no-dot-env
branch
from
October 5, 2026 13:48
58b12fb to
9ed64f0
Compare
shadow's su is deprecated and will eventually be dropped. We used it to avoid PAM in the SDK, but we should have used util-linux's for the production image to begin with. We don't actually need su in the SDK though as sudo does the job, so just drop it from there entirely. Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
chewi
force-pushed
the
chewi/no-dot-env
branch
from
October 5, 2026 14:53
9ed64f0 to
8fde641
Compare
chewi
force-pushed
the
chewi/no-dot-env
branch
from
October 5, 2026 15:25
8fde641 to
f9c0c8a
Compare
Writing potentially sensitive environment variables to an on-disk shell snippet is not a good idea. It's also unnecessary. Passing the environment variables to `docker run` bakes the values in at container creation time, but passing them to `docker exec` instead allows them to stay current when starting run_sdk_container. The list of environment variables is now split into those we just want to keep when using sudo and those we also want to pass through to the SDK container. These lists are used to generate the sudo `env_keep` value when the SDK is built. This drops a lot of the Google SDK setup, but none of this works anyway, and the rest will be completely dropped soon. This also fixes the GPG agent pass-through. Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
These were used by the Equinix Metal testing script. Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Docker can pass these through by name without the values. Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
chewi
force-pushed
the
chewi/no-dot-env
branch
from
October 5, 2026 15:43
f9c0c8a to
9dc87e8
Compare
Writing potentially sensitive environment variables to an on-disk shell snippet is not a good idea. It's also unnecessary. A list of these Mantle-specific variables now live in a text file that can be passed to `docker run`, allowing all of them to be passed through directly. Flatcar's Jenkins scripts also used .env to set arbitrary variables and potentially run other commands. This is now supported via standard input to the test_run function. Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
This only supports passing through variables by name, not setting their
values. While it could easily do this, it would encourage the inclusion
of secrets on the command line, which is insecure.
Docker's -e option doesn't allow separating multiple names with
whitespace, but I thought it would be useful here so that you could do
things like -e "${!RCLONE_S3_*}".
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
chewi
force-pushed
the
chewi/no-dot-env
branch
from
October 5, 2026 15:55
9dc87e8 to
366cb26
Compare
This branch is waiting to be deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Writing potentially sensitive environment variables to on-disk shell snippets is not a good idea. It's also unnecessary. Passing the environment variables to
docker runbakes the values in at container creation time, but passing them todocker execinstead allows them to stay current when startingrun_sdk_container.The list of environment variables is now split into those we just want to keep when using sudo, those we also want to pass through to the SDK container, and those only used with Mantle. The first two lists are used to generate the sudo
env_keepvalue when the SDK is built.Flatcar's Jenkins scripts also used .env to set arbitrary variables and potentially run other commands. This is now supported via standard input to the
test_runfunction.This drops a lot of the Google SDK setup, but none of this works anyway, and the rest will be completely dropped soon.
This also fixes the GPG agent pass-through.
Being able to pass additional variables through to the SDK container is still generally useful, and I would like to do that for the distfiles mirroring later, so I have added an
-eoption torun_sdk_container.This only supports passing through variables by name, not setting their values. While it could easily do this, it would encourage the inclusion of secrets on the command line, which is insecure.
Docker's
-eoption doesn't allow separating multiple names with whitespace, but I thought it would be useful here so that you could do things like-e "${!RCLONE_S3_*}".While I was looking at this, I noticed that we ship the shadow's implementation of su rather than util-linux's. The former is deprecated and due to be dropped entirely. We used it to avoid PAM in the SDK, but we should have used util-linux's for the production image to begin with, so this makes that change. We don't actually need su in the SDK as sudo does the job, so this just drops it from there entirely.
See flatcar/jenkins-os#491 for the Jenkins half of this.
How to use
Fire up the SDK, using the new
-eoption, and also try a build in CI.Testing done
I've successfully performed a two-phase SDK build in Jenkins with tests on all the platforms. Sorry for the noise there. I have confirmed that
CUSTOM_CI_CONFIGworks for the test job.changelog/directory (user-facing change, bug fix, security fix, update)/bootand/usrsize, packages, list files for any missing binaries, kernel modules, config files, kernel modules, etc.