Base docker image for air gap enviroment work -not working

[S1d0]

Hi,

I searched but didn't find anything about this topic. In my project we have private docker repository and we work wit AIR GAP enviroment. I'm trying to create custom image from debian:stretch with all needed parts already installed on board to make sure the new base image has no-op's for apt and curl when they are invoked (and possibly other commands; not sure yet), so they don't require online access when applied when building the porter invocation image.

For local development we are using minikube as kubernetes cluster

Assumption is to use exec, helm3 and kubernetes mixin.
here is dockerfile that i came up.

`FROM debian:stretch

RUN apt-get update &&
apt-get install -y ca-certificates

RUN apt-get update &&
apt-get install -y apt-transport-https curl &&
curl -o kubectl https://storage.googleapis.com/kubernetes-release/release/v1.19.0/bin/linux/amd64/kubectl &&
mv kubectl /usr/local/bin &&
chmod a+x /usr/local/bin/kubectl

RUN curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 &&
chmod 700 get_helm.sh &&
./get_helm.sh

RUN apt-get update &&
apt-get -y install python-pip &&
pip install --upgrade pip`

but it's failing to porter build with error message:
"Error: unable to build CNAB invocation image: Error response from daemon: Client sent an HTTP request to an HTTPS server."

Steps to recreate:

1. Build custom debian image from my Dockerfile
2. Create simple bundle manifest with command: porter create
3. Edit Dockerfile.tmpl to use build custom debian image
4. Edit porter.yaml: add dockerfile: Dockerfile.tmpl
5. Configure minikube with command: minikube config set embed-certs true
6. Run command portet build

Error:
Starting Invocation Image Build =======>
time="2020-11-27T12:01:15+01:00" level=error msg="Can't add file \\?\C:\project\cloud\deployment\porter\bundle\.cnab\app\mixins\exec\exec.exe to tar: io: read/write on closed pipe"
time="2020-11-27T12:01:15+01:00" level=error msg="Can't close tar writer: io: read/write on closed pipe"
Error: unable to build CNAB invocation image: error during connect: Post http://192.168.108.51:2376/v1.24/build?buildargs=%7B%22BUNDLE_DIR%22%3A%22%2Fcnab%2Fapp%22%7D&cachefrom=null&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=null&memory=0&memswap=0&networkmode=&rm=1&shmsize=0&t=sitaware%2Fcloud-app-installer%3Av0.1.0&target=&ulimits=null&version=: write tcp 192.168.108.49:54390->192.168.108.51:2376: wsasend: An existing connection was forcibly closed by the remote host.

Can anyone could help me to understand what's happening?

[vdice]

Hi @S1d0 ,

I'm attempting to reproduce the build error. I've copied the Dockerfile.tmpl you've provided above, but also added the required ARG BUNDLE_DIR line and (not required, but needed to copy bundle apps/assets into the image) COPY . $BUNDLE_DIR lines:

FROM debian:stretch

ARG BUNDLE_DIR

RUN apt-get update && \
  apt-get install -y ca-certificates

RUN apt-get update && \
  apt-get install -y apt-transport-https curl && \
  curl -o kubectl https://storage.googleapis.com/kubernetes-release/release/v1.19.0/bin/linux/amd64/kubectl && \
  mv kubectl /usr/local/bin && \
  chmod a+x /usr/local/bin/kubectl

RUN curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 && \
  chmod 700 get_helm.sh && \
  ./get_helm.sh

RUN apt-get update && \
  apt-get -y install python-pip && \
  pip install --upgrade pip

COPY . $BUNDLE_DIR

On my setup (Mac OS X, Docker for Mac), it builds with no errors. I'm using Porter built from the most recent commit of the main branch (porter v0.30.1-19-g50ce74ec (50ce74ec)).

Can you supply similar details?

From the errors above, my first guess is Docker daemon issues (unable to build CNAB invocation image: Error response from daemon: Client sent an HTTP request to an HTTPS server.). Are you able to build other images using this same daemon? (For example, with a Dockerfile of just FROM alpine and run docker build ., etc.)

[S1d0]

Hi @vdice,

my porter version is v0.30.1 (2931a5c)
my system Windows 😒

thanks for your replay. I found out that problem is not in porter.yaml or Dockerfile.tmpl so you are probably right it is docker deamon.
I think porter build command tries to push image to docker repository and for some reason porter is blocked to push to minikube docker repository.
Before running porter build, I run command eval $(minikube docker-env)
My main goal is to work on my machine with bundles deployed on cluster on minikube.
Is there way to set up porter to work with minikube?

I follow https://porter.sh/best-practices/minikube/ but it still doesn't work.

[vdice]

Ah, thank you for the clarification. I see now the issue is when attempting to use Minikube's docker daemon when building bundles via Porter. Indeed, I reproduce the same/similar errors when I try this (eval $(minikube docker-env), porter build...)

I'll need to look through Porter's build code to find out what is going wrong. I am curious on why this doesn't work currently.

In the meantime, though perhaps not clear (as you've pointed out), the current minikube doc/usage notes for Porter are for using minikube as an underlying k8s cluster to install a bundle on to. In other words, to inject the minikube kubeconfig at runtime when installing/upgrading a bundle.

[S1d0]

Thanks for reply. Installing bundle on minikube is imposible for the same reason as building bundle.
I'm trying to deploy on minikube simple hello world generated by porter create - so nothing fancy 😉 In my porter.yaml I got this line:

credentials:
  - name: kubeconfig
    path: /root/.kube/config

And installation steps looks like that in terminal:

1. minikube start --drive=hyperv
2. eval $(minikube docker-env)
3. porter install -c kubecred
   installing porter-hello...
   Error: 1 error occurred:
   * cannot create container: read tcp 192.168.108.49:51077->192.168.108.59:2376: wsarecv: An existing connection was forcibly closed by the remote host.

Where kubecred.json is in my folder PORTER_HOME/credentials/ and looks like that:
{ "schemaVersion": "1.0.0-DRAFT+b6c701f", "name": "kubecred", "created": "2020-11-30T13:02:02.8587334+01:00", "modified": "2020-11-30T13:02:02.8587334+01:00", "credentials": [ { "name": "kubeconfig", "source": { "path": "C:/Users/my-user/.kube/config" } } ] }

[vdice]

Hmmm, indeed, the runtime container cannot be created for the same reason of incompatibility with the minikube Docker daemon. Here we can't actually get the container to run the installation steps (or attempt to use the provided kubeconfig, etc.)

My first hunch is that the env vars that are loaded into the current shell (via eval $(minikube docker-env)) aren't actually being used by Porter when it instantiates the Docker build client (looking at this section of the build code and how it uses stock/default client options). I think creating a ticket around investigating the issue(s) here and hopefully figuring out how to make it work would be prudent.

In the meantime, to unblock you, would it be possible to try using the Docker for Windows client to build and run Porter images? (The minikube cluster should still be viable for use for when the Porter container attempts to run installation onto the cluster via the kubeconfig.)

[ska-ibees]

Ah, thank you for the clarification. I see now the issue is when attempting to use Minikube's docker daemon when building bundles via Porter. Indeed, I reproduce the same/similar errors when I try this (eval $(minikube docker-env), porter build...)

I'll need to look through Porter's build code to find out what is going wrong. I am curious on why this doesn't work currently.

In the meantime, though perhaps not clear (as you've pointed out), the current minikube doc/usage notes for Porter are for using minikube as an underlying k8s cluster to install a bundle on to. In other words, to inject the minikube kubeconfig at runtime when installing/upgrading a bundle.

Yep, doc/usage needs to clarify this point. Otherwise, it is very hard to understand the difference. On the same setup kubectl itself is able to perform all tasks on minikube cluster but porter is failing. I was able to access minikube cluster using:

minikube ip
192.168.49.2

docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4398baa65f1f gcr.io/k8s-minikube/kicbase:v0.0.14 "/usr/local/bin/entr…" 4 hours ago Up About an hour 127.0.0.1:32775->22/tcp, 127.0.0.1:32774->2376/tcp, 127.0.0.1:32773->5000/tcp, 127.0.0.1:32772->8443/tcp minikube

ssh -i ~/.minikube/machines/minikube/id_rsa docker@192.168.49.2 -L \*:2376:0.0.0.0:2376

Once there, I was able to perform steps given in quickstart

However, this is not a desired setup. Hope for the early resolution of the #1383

[S1d0]

After all troubles with minikube, we decided to switch currently to k3d.
By the way what is this image kickbase doing?

[carolynvs]

I am just following up to identify follow-up issues, let me know if I am misunderstanding or missing anything:

• Clarify that the minikube best practice page that we have is for connecting to minikube from a bundle (Clarify Minkube doc is for connecting not building #1404)
• Troubleshoot why Porter isn't using Minikube's docker daemon (Add support for using the Minikube daemon for build/run #1383)

I'm not sure what you mean by this though

By the way what is this image kickbase doing?