Support deployment of Compose stacks inside a Swarm cluster

[jeysibel]

Bug description

Although registered in database, templates type 3 (compose stack) aren't showing up in the Ui either created via Ui or via API. The create process is ok, in both cases (i can see the template description via api), but after creation, the UI is showing only swarm stack and single container templates.

Expected behavior
The Ui must show every container template add by user.

I expect to see compose stack templates alongside the swarm and single container templates.

Steps to reproduce the issue:

1. Go to Web UI or via API and create a type 3 template
2. Click on submit or send the request
3. Click the refresh button and you won't see the template you've created
4. get templates list via API, you can see the template you've created.
5. See error

Technical details:

• Portainer version: 1.19.1
• Docker version (managed by Portainer): 18.06-ce
• Platform (windows/linux): linux
• Command used to start Portainer (docker run -p 9000:9000 portainer/portainer): Compose file
• Browser: firefox

Additional context
Add any other context about the problem here.

[chiptus]

confirmed

[chiptus]

The error seems to be here:
https://github.com/portainer/portainer/blob/develop/app/portainer/components/template-list/template-list.js#L34

it seems like we're filtering by type, but show-swarm-stack will always be true for a swarm endpoint.
I think that either we don't let create compose stacks in this situation, or otherwise instead of having a toggle between container and swarm, we will have a select box to choose the type of templates to show

[deviantony]

Ok so not really a bug here. Actually, our intention was to force the user to deploy Swarm stacks only (and do not allow Compose stack deployment) when connected to a Swarm cluster.

Few reasons are:

• When managing Swarm clusters, users should be using services instead of containers
• Compose support is actually far behind Swarm stack support

Thoughts @ncresswell ?

[ncresswell]

100% agree. If using swarm, we must force the use of swarm stacks. Rgds, Neil Cresswell On 27/09/2018, at 8:19 AM, Anthony Lapenna <notifications@github.com<mailto:notifications@github.com>> wrote: Ok so not really a bug here. Actually, our intention was to force the user to deploy Swarm stacks only (and do not allow Compose stack deployment) when connected to a Swarm cluster. Few reasons are: * When managing Swarm clusters, users should be using services instead of containers * Compose support is actually far behind Swarm stack support Thoughts @ncresswell<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_ncresswell&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=dx2Y6wrnNqLVrSudQtMEp2ex1azn9f5gAkIdSwHBlRk&s=5ldVm_2-nul-xorOG1T8D_YQjbQ2H6bAzlfL5tfBhWk&e=> ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_portainer_portainer_issues_2134-23issuecomment-2D424855724&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=dx2Y6wrnNqLVrSudQtMEp2ex1azn9f5gAkIdSwHBlRk&s=i9D1xJZu7erD2_Mu5hI09l5qqUlhdki8fBtmDvXv7w4&e=>, or mute the thread<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AWGrlbyq3Auo6mc50SyhiXpQIHVQMKvlks5ue-2DFcgaJpZM4Vw9Gi&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=dx2Y6wrnNqLVrSudQtMEp2ex1azn9f5gAkIdSwHBlRk&s=ri0K6wQihSsy_mQ07car3u8ndwH6nB-ksdDtTCDms9c&e=>.

[deviantony]

Ok, closing this issue as this is the actual behavior.

[jeysibel]

Hello, i opened this issue because, actually, we're migrating from legacy compose services to swarm services, they coexist in our approach, at this time.
How you must know, some services are more suitable to a stateless architecture than others, and, in this way we choosed migrate only part of services, thinking in security issues or avoid problems like when n+1 docker swarm nodes went off or with log replication stopped due network instability.
We're working on private cloud, as many others around the world, so I suggest you all to think in other scenarios that differ from the mainstrean. i'ts powerfull and fexible.
This bug appear to be a low cost to correct and will turn portainer more flexible, once docker allow to run: stacks, single containers, compose e swarm stacks in a same docker node.

The issue was perceived when we create a docker machine to allow developers to learn and practice deploying and working with docker, so we need to supply templates to different needs and knowledge levels(from novice to the power user).

[deviantony]

Following a discussion between @jeysibel @ncresswell and me, we decided to go for the following:

• Add a new switch in the settings allowing the usage of Compose files with Swarm clusters: "Enable support for legacy compose files with Swarm"
• When connected to an agent setup, the user will have the ability to select the node on which to deploy the stack
• If not connected to an agent but to a Swarm manager directly, the stack will be deployed on the manager directly
• Template labels should be updated (stack should be replaced with either "Swarm" or "Standalone")

[ncresswell]

When enabling support, user should be shown a message advising about the deployment constraints (agent vs no agent).

[flothemaker]

Is there any timeframe for this issue?

[deviantony]

@braaccckkk
No ETA yet.

[pdschandler]

Hello, just checking in a year later :) I'm basically facing the same issue as described above and it'd be so useful to keep deploying some containers this way, to keep things neat for some 'services' while retaining some features of docker run like --cap-add. As @jeysibel mentioned, some applications just won't play nice with swarm, but they're all apps I'd only have running on my manager anyway so it's not like they need that management level.

I defined pretty much our entire company infrastructure through compose v2 files and relied on compose stacks for a long time to manage things. Works fantastic and it's a nice and simple way to define our infra. I'd love to continue managing things in a similar way while opening up swarm capabilities - not a hard required switch. I'm facing a need to use Traefik's swarm capabilities to do some more complex proxying, so it seems like I'm stuck with switching to swarm but I'd hate to have to 'downgrade' some of my template entries to normal containers (type 1) instead of the stacks I already have.

[tgrimminger]

Hey, I just wanted to bump this issue and ask if this is still on the roadmap or not. I'm using swarm primarily, but there are still some containers I need to deploy using legacy compose files. This is due to swarm not supporting things like network_node or cap_add, which are crucial for some apps I deploy.

[abate]

A simple user case for me is the need of deploying a traefik proxy in swarm mode, but also be able to manage a container that need a specific hardware device present only on a specific node. At the moment, I sidestep this issue creating the container via cli on the host, but it would be great to be able to handle both in portainer. Ideally having a dropdown selecting the deployment method ( compose or stack ), or deducing the deployment mode from the shape of the compose specification ( like absence of deploy keyword, implies using docker compose ), but I can see that this can be a bit weak as it's based on a heuristic.