Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add container support to repo #43

Merged
merged 13 commits into from
Oct 11, 2023
18 changes: 18 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
.git
.gitattributes
.gitignore
.dockerignore

debug/
target/

# Allow configuration files
!target/debug/*.yaml

.devcontainer/
.github/
devops/
docs/
tools/

Cargo.lock
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,6 @@ Cargo.lock

# Stops pushes of local vscode files.
/.vscode/*

# Do not include .env files for Docker.
/*.env
70 changes: 70 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT license.
# SPDX-License-Identifier: MIT

# syntax=docker/dockerfile:1

# Comments are provided throughout this file to help you get started.
# If you need more help, visit the Dockerfile reference guide at
# https://docs.docker.com/engine/reference/builder/

################################################################################
# Create a stage for building the application.

ARG RUST_VERSION=1.72.1
ARG APP_NAME=pub-sub-service
FROM rust:${RUST_VERSION}-slim-bullseye AS build
ARG APP_NAME
devkelley marked this conversation as resolved.
Show resolved Hide resolved
WORKDIR /app

COPY ./ .
devkelley marked this conversation as resolved.
Show resolved Hide resolved
COPY ./target/debug/*.yaml /app/target/debug/

# Add Build dependencies.
RUN apt update && apt upgrade -y
RUN apt install -y cmake protobuf-compiler pkg-config libssl-dev

# Build the application with the 'containerize' feature/
RUN cargo build --features containerize --release -p pub-sub-service

RUN cp ./target/release/$APP_NAME /app/$APP_NAME

################################################################################
# Create a new stage for running the application that contains the minimal
# runtime dependencies for the application. This often uses a different base
# image from the build stage where the necessary files are copied from the build
# stage.
#
# The example below uses the debian bullseye image as the foundation for running the app.
# By specifying the "bullseye-slim" tag, it will also use whatever happens to be the
# most recent version of that tag when you build your Dockerfile. If
# reproducability is important, consider using a digest
# (e.g., debian@sha256:ac707220fbd7b67fc19b112cee8170b41a9e97f703f588b2cdbbcdcecdd8af57).
FROM debian:bullseye-slim AS final
ARG APP_NAME
ENV APP_NAME=$APP_NAME
devkelley marked this conversation as resolved.
Show resolved Hide resolved

# Create a non-privileged user that the app will run under.
# See https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#user
ARG UID=10001
RUN adduser \
--disabled-password \
--gecos "" \
--home "/nonexistent" \
--shell "/sbin/nologin" \
--no-create-home \
--uid "${UID}" \
appuser
USER appuser

WORKDIR /sdv

# Copy the executable from the "build" stage.
COPY --from=build /app/$APP_NAME /sdv/
COPY --from=build /app/target/debug/*.yaml /sdv/target/debug/

# Expose the port that the application listens on.
EXPOSE 50051

# What the container should run when it is started.
CMD ./$APP_NAME
98 changes: 97 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@
<p align="center">
<a href="#getting-started">Getting Started</a> •
<a href="#configuration-setup">Configuration Setup</a> •
<a href="#running-the-service">Running the Service</a>
<a href="#running-the-service">Running the Service</a> •
<a href="#running-in-a-container">Running in a Container</a>
</p>

</br>
Expand Down Expand Up @@ -264,6 +265,101 @@ These two methods are used by a publisher to dynamically manage a topic. Please
see more full featured examples in
[Running the Simple Samples](./samples/README.md#running-the-simple-samples).

## Running in a Container

See below for instructions on how to run the service in a container. Currently, there is support
devkelley marked this conversation as resolved.
Show resolved Hide resolved
for both Docker and Podman containers. Both variations expect that the other steps have been
followed above to configure the service and start the MQTT broker.

### Docker

#### Prequisites

Install Docker: [Docker Installation](https://docs.docker.com/engine/install/)
devkelley marked this conversation as resolved.
Show resolved Hide resolved

#### Running in Docker

To run the service in a Docker container:

1. Copy the [docker.env](./container/template/docker.env) template from the
devkelley marked this conversation as resolved.
Show resolved Hide resolved
[container](./container/) directory into the project root directory. This file should already be
devkelley marked this conversation as resolved.
Show resolved Hide resolved
set up with out any modification needed.
devkelley marked this conversation as resolved.
Show resolved Hide resolved

1. Run the following command in the project root directory to build the docker container from the
Dockerfile:

```shell
docker build -t pub_sub_service -f Dockerfile .
```

1. Once the container has been built, start the container in interactive mode with the following
command in the project root directory:

```shell
docker run --name pub_sub_service -p 50051:50051 --env-file=docker.env --add-host=host.docker.internal:host-gateway -it --rm pub_sub_service
```

1. To detach from the container, enter:

```shell
Ctrl-p Ctrl-q
devkelley marked this conversation as resolved.
Show resolved Hide resolved
```

1. To stop the container, enter:

```shell
docker stop pub_sub_service
```

### Podman

#### Prequisites

Install Podman: [Podman Installation](https://podman.io/docs/installation)
devkelley marked this conversation as resolved.
Show resolved Hide resolved

#### Running in Podman

To run the service in a Podman container:

1. Copy the [podman.env](./container/template/podman.env) template from the
devkelley marked this conversation as resolved.
Show resolved Hide resolved
[container](./container/) directory into the project root directory. This file should already be
set up with out any modification needed.

1. Run the following command in the project root directory to build the podman container from the
Dockerfile:

```shell
podman build -t pub_sub_service:latest -f Dockerfile .
```

1. Once the container has been built, start the container with the following command in the project
root directory:

```shell
podman run -p 50051:50051 --env-file=podman.env --network=slirp4netns:allow_host_loopback=true localhost/pub_sub_service
devkelley marked this conversation as resolved.
Show resolved Hide resolved
```

1. To stop the container, find the container with:

```shell
podman ps
```

Then run:

```shell
podman stop <container_name>
devkelley marked this conversation as resolved.
Show resolved Hide resolved
```

#### Notes

1. By default, podman does not recognize docker images for dockerfile. To fix this, one can add the
`docker.io` registry to `/etc/containers/registries.conf` by changing the following field:

```conf
unqualified-search-registries = ["docker.io"]
```

## Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft
Expand Down
9 changes: 9 additions & 0 deletions container/template/docker.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT license.
# SPDX-License-Identifier: MIT

# DNS name used by the container to communicate with host.
HOST_GATEWAY=host.docker.internal

# Alias for localhost to be replaced by HOST_GATEWAY if run in a container.
LOCALHOST_ALIAS=0.0.0.0
9 changes: 9 additions & 0 deletions container/template/podman.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT license.
# SPDX-License-Identifier: MIT

# DNS name used by the container to communicate with host.
HOST_GATEWAY=host.containers.internal

# Alias for localhost to be replaced by HOST_GATEWAY if run in a container.
LOCALHOST_ALIAS=0.0.0.0
3 changes: 2 additions & 1 deletion pub-sub-service/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,5 @@ yaml-rust = { workspace = true, optional = true }

[features]
default = ["yaml"]
yaml = ["yaml-rust"]
yaml = ["yaml-rust"]
containerize = []
4 changes: 3 additions & 1 deletion pub-sub-service/src/connectors/chariott_connector.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ use proto::{
service_registry::v1::{RegisterRequest, ServiceMetadata},
};

use crate::load_config::localhost;
Fixed Show fixed Hide fixed

type ChariottClient = ServiceRegistryClient<Channel>;

/// Object that contains the necessary information for identifying a specific service.
Expand All @@ -39,7 +41,7 @@ pub async fn connect_to_chariott_with_retry(
let mut reason = String::new();

while client_opt.is_none() {
client_opt = match ServiceRegistryClient::connect(chariott_uri.to_string()).await {
client_opt = match ServiceRegistryClient::connect(localhost(chariott_uri)).await {
Fixed Show fixed Hide fixed
Ok(client) => Some(client),
Err(e) => {
let status = Status::from_error(Box::new(e));
Expand Down
7 changes: 5 additions & 2 deletions pub-sub-service/src/connectors/mosquitto_connector.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,10 @@ use log::{error, info, warn};
use paho_mqtt::{self as mqtt, MQTT_VERSION_5};
use std::{process, sync::mpsc};

use crate::pubsub_connector::{self, MonitorMessage, PubSubAction, PubSubConnector};
use crate::{
load_config::localhost,
Fixed Show fixed Hide fixed
pubsub_connector::{self, MonitorMessage, PubSubAction, PubSubConnector},
};

/// Mosquitto broker's reserved topic for subscribe related notifications.
const SUBSCRIBE: &str = "$SYS/broker/log/M/subscribe";
Expand All @@ -36,7 +39,7 @@ impl MqttFiveBrokerConnector {
/// * `client_id` - Id used when creating a new mqtt client.
/// * `broker_uri` - The uri of the broker that the client is connecting to.
fn new(client_id: String, broker_uri: String) -> Self {
let host = broker_uri;
let host = localhost(&broker_uri);
Fixed Show fixed Hide fixed

let create_opts = mqtt::CreateOptionsBuilder::new()
.server_uri(host)
Expand Down
26 changes: 26 additions & 0 deletions pub-sub-service/src/load_config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,39 @@

#![cfg(feature = "yaml")]

use std::env;

use config::{Config, File, FileFormat};
use log::error;
use serde_derive::{Deserialize, Serialize};

const CONFIG_FILE: &str = "target/debug/pub_sub_service_settings";
const CONSTANTS_FILE: &str = "target/debug/constants_settings";

/// If feature 'containerize' is set, will modify a localhost uri to point to container's localhost
/// DNS alias. Otherwise, returns the uri as a String.
///
/// # Arguments
/// * `uri` - The uri to potentially modify.
pub fn localhost(uri: &str) -> String {
Fixed Show fixed Hide fixed
devkelley marked this conversation as resolved.
Show resolved Hide resolved
#[cfg(feature = "containerize")]
let uri = {
// Container env variable names.
const HOST_GATEWAY_ENV_VAR: &str = "HOST_GATEWAY";
const LOCALHOST_ALIAS_ENV_VAR: &str = "LOCALHOST_ALIAS";

// Exit the program if container env variables are not set.
let host_gateway = env::var(HOST_GATEWAY_ENV_VAR)
devkelley marked this conversation as resolved.
Show resolved Hide resolved
.unwrap_or_else(|_| panic!("{HOST_GATEWAY_ENV_VAR} is not set"));
let localhost_alias = env::var(LOCALHOST_ALIAS_ENV_VAR)
Fixed Show fixed Hide fixed
.unwrap_or_else(|_| panic!("{LOCALHOST_ALIAS_ENV_VAR} is not set"));

uri.replace(&localhost_alias, &host_gateway)
Fixed Show fixed Hide fixed
Fixed Show fixed Hide fixed
};

uri.to_string()
}

/// Object that contains constants used for establishing connection between services.
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct CommunicationConstants {
Expand Down
8 changes: 6 additions & 2 deletions pub-sub-service/src/topic_manager.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,10 @@ use proto::publisher::v1::{
};
use tonic::Request;

use crate::pubsub_connector::{MonitorMessage, PubSubAction};
use crate::{
load_config::localhost,
Fixed Show fixed Hide fixed
pubsub_connector::{MonitorMessage, PubSubAction},
};

/// Metadata relevant to a dynamic topic.
#[derive(Clone, Debug, PartialEq)]
Expand Down Expand Up @@ -294,7 +297,8 @@ impl TopicManager {
}

// Get information from publisher client
let mut pub_client = PublisherCallbackClient::connect(action_metadata.uri.clone()).await?;
let mut pub_client =
PublisherCallbackClient::connect(localhost(&action_metadata.uri)).await?;
Fixed Show fixed Hide fixed

let request = Request::new(ManageTopicRequest {
topic: action_metadata.topic.clone(),
Expand Down
Loading