CI: plugin-docker-buildx cannot connect to Docker transient failure #793

Closed
opened 3 weeks ago by dachary · 16 comments

The following CI run fails with this error:

image

The corresponding pipeline can be found here and has been tested to run on a local woodpecker instance with the "trusted" setting checked.

image

Could the reason for the failure be that it needs to be trusted?

Thanks in advance for your help!


To be precise, you can find attached the woodpecker configuration that was used locally, with changes coping with ownca & private IPs. I think the changes are not relevant but... the devil is in the details.

The [following CI run fails](https://ci.codeberg.org/codename/codename/pipeline/7/4) with this error: ![image](/attachments/ebbf1bdc-648c-4565-a30f-81d2d5bac8d6) The corresponding pipeline can be [found here](https://codeberg.org/codename/codename/src/tag/v2.1.1/.woodpecker/docker-linux-amd64-release-version.yml) and has been tested to run on a local woodpecker instance with the "trusted" setting checked. ![image](/attachments/55a110c0-ef9e-42af-b962-852dfc5ed43d) Could the reason for the failure be that it needs to be trusted? Thanks in advance for your help! --- To be precise, you can find attached the woodpecker configuration that was used locally, with changes coping with ownca & private IPs. I think the changes are not relevant but... the devil is in the details.
fnetX added the
s/Woodpecker
label 3 weeks ago
Owner

I'm not very familiar with Woodpecker. What does "trusted" mean here?

I'm not very familiar with Woodpecker. What does "trusted" mean here?
Collaborator
https://github.com/woodpecker-ci/woodpecker/blob/f88c70b55ec465f400c4ebb486f71205aa2d6969/web/src/lib/api/types/repo.ts#L39
Poster

In the web interface, here is what I have on my local Woodpecker instance:

image

And on the Codeberg CI web interface I do not have the Trusted checkbox:

image

But you should see this check box since you have elevated privileges @fnetX

In the web interface, here is what I have on my local Woodpecker instance: ![image](/attachments/b6670f2f-c6f2-4af8-8c94-96308f9fead9) And on the Codeberg CI web interface I do not have the **Trusted** checkbox: ![image](/attachments/01e275f5-14cc-4e4c-9954-77ae8f9e5959) But you should [see this check box](https://ci.codeberg.org/codename/codename/settings#general) since you have elevated privileges @fnetX
Poster

I'm blocked moving forward with the CI until this is resolved. I'd be grateful if someone with the power to do so agrees to temporarily grant "Trusted" runs to this https://ci.codeberg.org/codename/codename/settings#general repository.

@fnetX @6543 is there someone you suggest I get in touch with about this?

I'm blocked moving forward with the CI until this is resolved. I'd be grateful if someone with the power to do so agrees to temporarily grant "Trusted" runs to this https://ci.codeberg.org/codename/codename/settings#general repository. @fnetX @6543 is there someone you suggest I get in touch with about this?
Owner

I don't have the power to toggle that switch (it's invisible for me, too), and I also don't understand the implications of this change.

It is desirable to allow every project to build docker images using CI IMHO.

I don't have the power to toggle that switch (it's invisible for me, too), and I also don't understand the implications of this change. It is desirable to allow every project to build docker images using CI IMHO.
6543 was assigned by fnetX 2 weeks ago
Collaborator

I did not had time to read the pipeline but it sound's wrong to depend on trusted, trusted means in worst case, allow admin access to agent runing systems by those who can alter the pipeline config - so I'm realy not confortable with that

I did not had time to read the pipeline but it sound's wrong to depend on trusted, trusted means in worst case, allow admin access to agent runing systems by those who can alter the pipeline config - so I'm realy not confortable with that
Poster

@6543 I'm asking for a temporary trusted run, just for today to figure things out. I'm not entirely sure the trusted permissions are required and that will clarify that. You can remove it tomorrow, regardless of what happens. Would that be ok?

@6543 I'm asking for a temporary trusted run, **just for today to figure things out**. I'm not entirely sure the trusted permissions are required and that will clarify that. You can remove it tomorrow, regardless of what happens. Would that be ok?
Poster

A little more debug information. There exists a Woodpecker CI instance with a job that uses the same plugin and that works although there is no trusted flag, see here for the definition of the job and here for a successfull run dated last week.

It is running on:

  • woodpeckerci/woodpecker-server:next@sha256:a2dca0198a81e916cd4b954b6253b264aac627b5b437739c04108e3d203d2a97@sha256:19933c9dd127877d1e5c7dc71cef01957758659d21886e738f04de131be3be51
  • woodpeckerci/woodpecker-agent:next@sha256:853f46776913dc60bd1829d0ede0ced429ad197efa5202206efceb5dc48ff40a@sha256:e4dbbb9efd3e9c754ad99fa8aa06b177e5f878767e1fedc3dc7f7351108ec5a3

and here is a screenshot that shows there is no "trusted" checked.

image

See also the chat log for the associated conversation.

Now I'm curious to understand what is the difference with Codeberg CI, if not the "trusted flag". 🤔

A little more debug information. There exists a Woodpecker CI instance with a job that uses the same plugin and that works although there is no trusted flag, see [here](https://git.rustybever.be/vieter-v/vieter/src/branch/dev/.woodpecker/docker.yml#L8) for the definition of the job and [here](https://ci.rustybever.be/vieter-v/vieter/build/60/26) for a successfull run dated last week. It is running on: * woodpeckerci/woodpecker-server:next@sha256:a2dca0198a81e916cd4b954b6253b264aac627b5b437739c04108e3d203d2a97@sha256:19933c9dd127877d1e5c7dc71cef01957758659d21886e738f04de131be3be51 * woodpeckerci/woodpecker-agent:next@sha256:853f46776913dc60bd1829d0ede0ced429ad197efa5202206efceb5dc48ff40a@sha256:e4dbbb9efd3e9c754ad99fa8aa06b177e5f878767e1fedc3dc7f7351108ec5a3 and here is a screenshot that shows there is no "trusted" checked. ![image](/attachments/ec5ca57b-7cf3-4c19-ba15-02ee01457b41) See also the [chat log](https://matrix.to/#/!vbfAJmueDTbflXuvif:obermui.de/$9pkwbwOliJeha5iS7VbnG-iaXydf5_NN4DL-Ow1NwFc?via=matrix.org&via=t2bot.io&via=obermui.de) for the associated conversation. Now I'm curious to understand what is the difference with Codeberg CI, if not the "trusted flag". :thinking:
Collaborator

@dachary changed - please test asap

@dachary changed - please test asap
Poster

Thanks for setting "trusted". The job works with this flag set, as shown in the log below. Knowing that, do you have any idea how this job can run without "trusted"? And why it work on this other Woodpecker instance?

Thanks for setting "trusted". The job works with this flag set, as shown in the log below. Knowing that, do you have any idea how this job can run without "trusted"? And why it work on this other Woodpecker instance?
Collaborator

this https://codeberg.org/woodpecker-plugins/plugin-docker-buildx for example also use the same plugin and is not trusted flaged

this https://codeberg.org/woodpecker-plugins/plugin-docker-buildx for example also use the same plugin and is not trusted flaged
Poster

image

I tried restarting the job with the exact same commit that failed four days ago and now it passes.

I'll close this and re-open if this shows up again. Let's assume it was a transient Woodpecker CI error and that it won't show up again!

Thanks for the help 👍

![image](/attachments/f3730bbd-fd3e-4915-9729-3ecbd547b363) I tried restarting the job with the exact same commit that [failed four days ago](https://codeberg.org/Codeberg/Community/issues/793#issue-209366) and now it passes. I'll close this and re-open if this shows up again. Let's assume it was a transient Woodpecker CI error and that it won't show up again! Thanks for the help 👍
147 KiB
dachary closed this issue 2 weeks ago
Poster

For the record the same error happened https://ci.codeberg.org/woodpecker-plugins/plugin-docker-buildx/pipeline/159/3

image

Whatever underlying problem is causing this is therefore not unique to the codename pipeline.

For the record the same error happened https://ci.codeberg.org/woodpecker-plugins/plugin-docker-buildx/pipeline/159/3 ![image](/attachments/bd1523b9-c115-4419-83a7-070d63416093) Whatever underlying problem is causing this is therefore not unique to the codename pipeline.
113 KiB
Poster
Another occurrence at https://ci.codeberg.org/forgejo/forgejo/pipeline/39/12
dachary reopened this issue 2 weeks ago
dachary changed title from CI and publishing docker images to CI: plugin-docker-buildx cannot connect to Docker transient failure 2 weeks ago
Poster
Another occurrence https://ci.codeberg.org/dachary/forgejo/pipeline/35/12 ![image](/attachments/17886426-1c81-47f8-88d5-5ce8e1ac9b9d)
158 KiB
Poster

This did not show up in a while, maybe it was a side effect of last week migration. Let's close this for now.

This did not show up in a while, maybe it was a side effect of last week migration. Let's close this for now.
dachary closed this issue 3 days ago
Sign in to join this conversation.
No Milestone
No Assignees
4 Participants
Notifications
Due Date

No due date set.

Reference: Codeberg/Community#793
Loading…
There is no content yet.