Platform

• 1: Requirements
• 2: Deploy an application
• 3: Update configurations
• 4: Update secrets
• 5: FAQ
• 6: Aigonix Studio
• 6.1: Overview
• 6.2: Application
• 6.3: Environment
• 6.4: Microservice
• 6.5: Integrations
• 6.6: Container Registry

1 - Requirements

To be compatible with the environment of the Aigonix platform, there are certain requirements we impose on your microservices. If they are not met, your application might behave unexpectedly - or in the worst case - not work at all. The following list of requirements is subject to change, but we will always notify you when you have an application running in our platform before making any changes.

1. Your application must use the resource system

To ensure data privacy, security and proper segregation of your tenant’s data, our platform has a resource management system. This system controls access and connection settings for resources on a per request basis and will provide your microservice with the necessary information for accessing these resources programmatically. The connection information will not be the same as when developing locally, so you must not embed connection settings in your code.

This requirement applies to read and write data to databases or files, or while making API-calls to services, both to internal resources provided by the Aigonix platform and external 3rd party services.

2. All your applications external endpoints must be configured and exposed through the platform

For the resource management system to work, and to protect your application and users from data leakage, we encrypt and authenticate all interactions with your application through the platform. This means that your microservices will be completely isolated by default, and all endpoints that should be accessible outside our platform needs to be exposed explicitly and configured with appropriate encryption and authentication schemes.

To enable same-origin authentication flows and adhere to internet best practices, the platform will take control of a set of URIs for the hostnames you have allocated to your application. The following paths and any sub-path of these (in any form of capitalisation) are reserved for the platform:

• /.well-known
• /robots.txt
• /sitemap
• /api/Dolittle
• /Dolittle

3. Your microservices must be stateless, scalable and probeable

To allow for efficient hosting of your application, we have to able to upgrade, re-start, move and scale your microservices to handle the load and perform necessary security upgrades. This means that you must not rely on any in-memory state for anything apart from the per-transaction state, and you must not rely on there being a single instance of your microservices at any point in time.

To ensure that your microservices are healthy and ready to perform work, your microservices must expose both liveness and readiness probes. The microservice should respond to the liveness probe whenever it has successfully started and is in a functional state, and should respond to the readiness probe whenever it is free to handle incoming requests from users.

4. Your application must adhere to semantic versioning of your microservices

We rely on semantic versioning to properly track changes of your microservices (from an operational aspect) and to decide on the correct course of action when new versions of your microservices are built. Minor or patch increments will result in automatic upgrades of your running microservices without any human interaction, while major increments require manual approval and potential updates of configuration or data structures. This means that you must increment the major number when making changes to your microservices that require changes in the platform for your application to work properly.

5. Your frontend must be a static single-page application

To ensure that any user-facing frontend is served quickly and with minimal data-usage, we serve your frontend using separate servers with appropriate caching, compression and CDN strategies. This means that your frontend must be built as a single-page application to static HTML, CSS and js files. These files must be built and versioned alongside your backend microservices to ensure that the frontend and backend versions are aligned and function properly.

2 - Deploy an application

Prerequisites

Familiar with the following:

• Docker containers
• Kubernetes
• Microsoft Azure

Recommendation

For users on Windows OS, we recommend that you use WSL/Ubuntu as your shell/terminal instead of CMD/Powershell.

• Install WSL on Windows 10

Installation

Install the following software:

• Docker Desktop
• Kubectl
• Azure CLI

Configuration

After an environment has been provisioned for you in the Dolittle PaaS, you will receive these details to use with the deployment commands in the following sections:

Subscription ID
Resource Group
Cluster Name
Application Namespace
ACR Registry
Image Repository
Deployment Name
Application URL

Setup

All commands are meant to be run in a terminal (Shell)

AZURE

Login to Azure:

az login

AKS - Azure Container Service

Get credentials from Dolittle’s AKS cluster

az aks get-credentials -g <Resource Group> -n <Cluster Name> --subscription <Subscription ID>

ACR - Azure Container Registry

Get credentials to Azure Container Registry

az acr login -n <ACR Registry> --subscription <Subscription ID>

Deployment

To deploy a new version of your application, follow these steps. For use semantic versioning, e.g. “1.0.0”.

Docker

Build your image

docker build -t <Image Repository>:<Tag> .

Push the image to ACR

docker push <Image Repository>:<Tag>

Kubernetes

Patch the Kubernetes deployment to run your new version

kubectl patch --namespace <Application Namespace> deployment <Deployment Name> -p '{"spec": { "template": { "spec": { "containers": [{ "name":"head", "image": "<Image Repository>:<Tag>"}] }}}}'

Debugging

kubectl commands:

Show the status of your application pods

kubectl -n <Application Namespace> get pods

Show deployed version of your application

kubectl -n <Application Namespace> get deployment -o wide

Show the logs of the last deployed version of the application

kubectl -n <Application Namespace> logs deployments/<Deployment Name>

Logs for the application, last 100 lines

kubectl -n <Application Namespace> logs deployments/<Deployment Name> --tail=100

3 - Update configurations

Prerequisites

Familiar with the following:

• Kubernetes
• yaml

Recommendation

For users on Windows OS, we recommend that you use WSL/Ubuntu as your shell/terminal instead of CMD/Poweshell.

• Install WSL on Windows 10

Installation

Install the following software:

• Kubectl

Configuration

After an environment has been provisioned for you in the Dolittle PaaS, you will receive a yaml file per environment. The files will be similar to this:

---
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: application-namespace
  name: app-dev-ms-env-variables
  labels:
    tenant: Customer
    application: App-Dev
    microservice: MS-A
data:
  OPENID_AUTHORITY: "yourapp.auth0.com"
  OPENID_CLIENT: "client-id"
  OPENID_CLIENTSECRET: "client-secret"

---
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: application-namespace
  name: app-dev-ms-config-files
  labels:
    tenant: Customer
    application: App-Dev
    microservice: MS-A
data:
  myapp.json: |
    {
       "somekey": "somevalue"
    }    

The files represent configmap resources in Kubernetes. We recommend that you store the files in a version control system(VCS) of your choice.

Purpose

Each yaml file consists of 2 configmaps per micro-service:

• app-dev-ms-env-variables: This configmap is for your environmental variables that will be passed on to the container at start up.
• app-dev-ms-config-files: This configmap is for add/override files. The default mount point is app/data

---
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: application-namespace
  name: app-dev-ms-env-variables
  labels:
    tenant: Customer
    application: App-Dev
    microservice: MS-A
data:

You may alter the content under data:

  OPENID_AUTHORITY: "yourapp.auth0.com"
  OPENID_CLIENT: "client-id"
  OPENID_CLIENTSECRET: "client-secret"
  connectionstring__myconnection: "strings"

Alter existing or add new key/value pairs.

myapp.json: |
    {
       "somekey": "somevalue"
    }    

customSetting.json: |
  {
    "settings": {
       "connection":"connectionstring"
     }
  }  

Alter existing or add new JSON data that will be linked to a specific file that will be available at runtime under app/data/

Setup

You need to setup your AKS credentials.

Update configurations

To update the configurations:

kubectl apply -f <filename>

You must be in the directory of the yaml file before running the command.

To update/add a single key in the config:

kubectl patch -n <Application Namespace> configmap <Configmap Name>  -p '{"data":{"my-key":"value that i want"}}'

To remove a single key from the configuration:

kubectl patch -n <Application Namespace> configmap <Configmap Name>  -p '{"data":{"my-key":null}}'

See configurations

JSON output

kubectl get -n <Application Namespace> configmap <Configmap Name> -o json

YAML output:

kubectl get -n <Application Namespace> configmap <Configmap Name> -o yaml

For an advanced print out, you need a tool called jq for parsing

kubectl get -n configmap -o json | jq -j ‘.data | to_entries | .[] | “(.key): (.value)\n”’

4 - Update secrets

Prerequisites

Familiar with the following:

• Kubernetes
• yaml

Recommendation

For users on Windows OS, we recommend that you use WSL/Ubuntu as your shell/terminal instead of CMD/Poweshell.

• Install WSL on Windows 10

Installation

Install the following software:

• Kubectl

Secrets

After an environment has been provisioned for you in the Dolittle PaaS, you will receive a yaml file per environment. The files will be similar to this:

---
apiVersion: v1
kind: Secret
metadata:
  namespace: application-namespace
  name: apps-dev-ms-secret-env-variables
  labels:
    tenant: Customer
    application: App-Dev
    microservice: MS-A
type: Opaque
data:
   OPENID_SECRET: b3BlbiBpZCBzZWNyZXQ=

The files represent the Secrets -resource in Kubernetes. We recommend that you store the files in a version control system(VCS) of your choice.

Purpose

Each yaml file consists of a secret per micro-service:

• app-dev-ms-secret-env-variables: This secret is for your environmental variables that will be passed on to the container at start up. One important thing to remember is that the values have to be encoded using base64.

---
apiVersion: v1
kind: Secret
metadata:
  namespace: application-namespace
  name: apps-dev-ms-secret-env-variables
  labels:
    tenant: Customer
    application: App-Dev
    microservice: MS-A
type: Opaque
data:

You may alter existing or add new key/value pairs.

  OPENID_SECRET: b3BlbiBpZCBzZWNyZXQ=
  DB_PASSWORD: c29tZSBwYXNzd29yZA==

Setup

You need to setup your AKS credentials.

Encode secrets

To encode values:

echo -n "my super secret pwd" | base64 -w0

The above command will give you:

bXkgc3VwZXIgc2VjcmV0IHB3ZA==

The value can then be added to the secrets:

MY_SECRET: bXkgc3VwZXIgc2VjcmV0IHB3ZA==

Update secrets

To update the secrets:

kubectl apply -f <filename>

You must be in the directory of the yaml file before running the command.

To update/add a single key in the secrets:

kubectl patch -n <Application Namespace> secret <Secrets Name>  -p '{"data":{"my-key":"value that i want encoded using base64"}}'

To remove a single key from the configuration:

kubectl patch -n <Application Namespace> secret <Secrets Name>  -p '{"data":{"my-key":null}}'

See secrets

JSON output:

kubectl get -n <Application Namespace> secret <Secrets Name> -o json

YAML output:

kubectl get -n <Application Namespace> secret <Secrets Name> -o yaml

For an advanced print out, you need a tool called jq for parsing the JSON in you shell:

kubectl get -n <Application Namespace> secret <Secrets Name> -o json | jq -j '.data | to_entries | .[] | "\(.key): \(.value)\n"'

5 - FAQ

Can I login without allowing cookies?

If you’re getting strange results with logon through Sentry or another OIDC service - check that you’re allowing cookies for the domains!

Without cookies you cannot logon - at all. Sorry!

6 - Aigonix Studio

Aigonix Studio is your management-tool to interact with services and products you run in the Aigonix Platform. It is a web-based application that is available at dolittle.studio.

6.1 - Overview

Aigonix Studio is the web-based interface for managing your Applications and Microservices in the Aigonix Platform. It is the main interface for interacting with the platform and is where you will spend most of your time when interacting with the platform.

In Studio you can create and manage Applications, Environments, Microservices and other products and services.

Getting started

To access Studio you need to be a customer of Aigonix. If you are not a customer, you can contact us to learn more and hopefully become one.

Once you have access to Studio, you can log in using your credentials at dolittle.studio. You can now create your first Application and start deploying your Microservices into that application.

Components

When you run your Microservices in the Aigonix Platform you will have access to a number of components that will help you manage your services.

You will define each Application with its Environments, and add Microservices to them. You define which Docker image to use for each Microservice, and whether or not you want to use the Dolittle Runtime. You can use a publicly available image, or store your container image in the provided container-registry.

If you use the Dolittle Runtime one will be made available to your Head and through it you will have access to Tenanted resources like the Event Store and Read Cache. If you do not use the Dolittle Runtime the service will run the assigned Docker-image without permanent storage (stateless).

You can make your services available to the internet if you so wish. If you do not they will only be available within the platform.

6.2 - Application

An Application is a logical grouping of Microservices. It is the top-level container for your Microservices and is used to group them together. An Application can have multiple Environments, which are used to separate your Microservices into different stages of your development process.

When you create a new application you will define a name for it and which environments the application should have.

Microservices

Each application acts as a separate area wherein your microservices run. When you navigate in an application in Studio you will see the services that are deployed in that application.

Each microservice will be listed with their name, container-image, which version of the Dolittle Runtime (if any) they are set up with, their publich URL (if any) and their status. If your application has multiple environments you can switch between them using the dropdown in the top right corner.

You can navigate into each Microservice to see more details about it, such as the logs for the service, the configuration for the service and metrics like the CPU and Memory consumption of the service. You can change the configuration of the service and restart it from this view.

Functions

There are several functions available in an Application -view. You can create new Microservices, and you can access the backups of the databases that are used by the services in that application.

Each application has its own docker-image container registry, which you can access from the application view.

There is a log-viewer, which consolidates logs from all the services in the application into one view. Here you can filter the logs by service, time and search for specific text. You can also have a live-view of the logs, which will update the view as new logs are generated.

There is a documentation-view in the application that contains sections on how to access the container-registry, how to access the underlying Kubernetes cluster if needed and how to setup Azure Pipelines to deploy your containers directly to the application.

6.3 - Environment

An environment is a grouping of Microservices within an Application. Environments are commonly used to separate instances of your Microservices into different stages of your development process.

When you create an Application you define which environment(s) you want in that Application. The available options are Development, Test and Production. You can choose to have all three, or just one or two of them.

Microservices

Each environment acts as a separate area wherein your microservices run. When you navigate to an Application in Studio it will list the Microservices running in the current environment. The current Environment is displayed and can be changed in a drop-down box on the top right corner of the page.

Databases

Each environment has its own database. The database is used by the Microservices in that environment to store their data. The database is a MongoDB database and is managed by the Aigonix Platform. You can access the database backups through the Studio interface. If necessary you can also access the database through port-forwarding to the Kubernetes cluster. You will need to contact Aigonix Support to get the elevated permissions needed to access the database directly.

6.4 - Microservice

A Microservice is a single unit of deployment in the Aigonix Platform. It is a containerized application that runs in a Kubernetes cluster. Each Microservice is deployed as a separate container in the cluster.

A Microservice lives inside an Application and can be deployed to multiple Environments in that application.

A Microservice in the Aigonix Platform is a Head and optionally a Runtime and data-storage. If you use the Dolittle Runtime you will have access to the Event Store and Read Cache for that Microservice.

If you do not use the Dolittle Runtime the Microservice will be stateless and will not have access to any storage. Any state you need to store will have to be stored in an external service. You will have access to the local file-system, but that is not persisted between Microservice restarts, and we do not recommend relying upon it.

Docker image

Each Microservice is deployed using a Docker image. You can use a publicly available image, or you can use a Docker image that you have built yourself and stored in the container-registry that is available for each application.

Head

Head is the name of the pod where your Microservice Docker image is deployed. It is the entry-point for your Microservice and is the only part of your Microservice that may be exposed to the internet. A Microservice must have a Head and can have a Runtime.

Dolittle Runtime

Each Microservice can be deployed with the Dolittle Runtime. The Dolittle Runtime is available as a docker image, and will be deployed alongside your Head. Your code communicates with the Runtime through the Dolittle SDK. The Runtime is used to connect to the Event Store and will make the resource-system available to you, where you can get the Tenanted Read Cache and to publish events to the Event Horizon.

Configuration

Each Microservice can be configured with a set of environment variables and configuration-files. These variables are available to your code and can be used to configure your Microservice. The Aigonix Studio gives you an interface to manage these variables and files.

Your code can read the environment variables and configuration-files. The files will be available under the /app/data -folder in your container. The Aigonix Platform will make sure that the files are available in the container when it is deployed. Therefore any files your image contains in /app/data will not be available when running in the Aigonix Platform.

You can upload files through Aigonix Studio, and you can also download the files and environment variables from Studio.

Secrets

You can also use the environment variables to store secrets, like connection strings and API-keys. Mark these environment-variables as secret to ensure that they be stored encrypted in the platform. You may want to look into a proper secret-management system, like Azure Key Vault or Amazon Key Management System if you have a lot of secrets, and just store the access-keys in the environment variables.

6.5 - Integrations

Integrations in Aigonix lets you work with your existing business-data in an ERP-system. The integrations support an Event Driven Architecture and is designed to make translation from ERP-specific terms to domain terms easy.

By setting up integrations your business data from an ERP -system (Infor M3) synchronize to an message stream. This data can then be used by your Microservices to provide functionality to your users and customers.

This article will introduce you to the concepts of the Aigonix integrations and show you how to set up a connection to an ERP -system and how to map data from the ERP -system into messages. It also describes how to consume these messages in your services.

Conceptual overview

Central to this is the concept of a message-type. You define how to translate data from the ERP -system into messages. Message-types are defined in the Aigonix Studio -interface where you can select the tables and fields you want to translate. This mapping then translates the changes in business-data into messages whenever such a change is detected.

Connections

In order to translate data from the ERP -system into messages a connection must be specified. The connection defines how to connect to the ERP -system and the message-types how to translate the data into messages. Once you define the connection you can either run an instance of the ERP Integration service yourself (on-premises), or automatically host one in the Aigonix platform. This service is the conduit between the messages and the ERP -system. It discovers changes and translates those into messages. It also reacts to messages and calls programs in the ERP-system at your request.

Message-types

The message-types are defined in the Aigonix Studio -interface, and deployed to the ERP Integration service. They are defined in terms of the data in the ERP -system.

Setting up a connection

Step 0: Name and deploy the connection

Step 1: Connect to the Metadata Publisher

Step 2: Connect to ION

Mapping tables

At this point you have a connection to your M3 -instance, and are ready to start mapping data from it to messages with the mapping interface. This interface is available by clicking on the “message types”, next to “configuration”.

Step 0: Create a message-type

Step 1: Name the message-type

Step 2: Select the table

Step 3: Select the fields

Step 4: Name the fields

Step 5: Save the message-type

Step 6: Deploy the message-type

Step N: Refine the message-types

Consuming data

Having mapped the business-data from your ERP -instance into messages-types and deployed these message-type to the connection you can now consume messages in your services.

The connection will translate changes in the ERP -instance into messages that are available as a stream on a Kafka -topic.

If you do not want or need a full stream of changes, just the “latest state” of the message-type (in our example - the latest state of an item in the catalogue) you can activate a REST -interface. This is often what you need if you want to display the data in a user interface, or work with it in tools like Power-BI.

Option 1: Event stream (messages on a Kafka topic)

Option 2: REST API (latest state of a message-type)

Summary

In this article we have seen how to set up a connection to an ERP -system and how to map data from the ERP -system into messages. We have also shown how to consume these messages in your services. This is the core of the Aigonix integration, and it is how you make your business-data available to your services.

The Aigonix integration is designed to make it easy to work with your business-data and to make it easy to translate from the “language” of your ERP -system to the “language” of your services. This is done by mapping the data from the ERP -system into messages that are available as a stream. You can also use the REST -interface to get the latest-state of the message-types.

By setting up a connection to your organization’s ERP -system you make the business-data available to your services. This lets you build services that react to changes in the business-data expressed in your domain-language and that affect changes in the business-data, without needing a direct connection to the ERP -system.

Details (for the technically inclined)

The Aigonix integration works by connecting to the ERP -system (M3) and translating changes in the business-data into messages. This is done by the ERP Integration service, which knows how to poll and to set up webhooks for ION. It translates to messages, and sends that to the correct streams. The ERP integration service exposes webhooks that ION can use to notify of changes, and it polls at intervals to discover changes in data.

An interface to configure the ERP Integrations service exists in Aigonix studio, which is how you configure message-types and set up new connections. The ERP Integration service can run on-premises, or it can be hosted in the Aigonix platform. In either case it is controlled by your setup in Aigonix studio.

You decide whether to self-host or use the Aigonix platform to run the integration-service. This decision typically relies on your networking rules.

6.6 - Container Registry

All Microservices running in the Aigonix Platform are running in Docker containers. You can run public images, or you can run your own images. If you do not wish to publish the images of your Microservices to publicly available registries you can use one Aigonix provides for you.

This container registry is available at a unique host for your customer account. You can find the host in the documentation for your Application in Studio. It will look like {your-random-id}.azurecr.io.

To list what images are available in the registry you can navigate to the container-registry in Aigonix Studio, or you use the az command-line tool. You will need to log in to the Azure CLI and then you can run

az acr repository list --name {your-random-id} -o table

from your command-line. This will list all the images available in the registry.

To push a new image to the registry you tag the image with the registry host and then push it to the registry. You can do this with the docker command-line tool. Let us say you have an image called my-image that you want to push to the registry. You would then run

docker tag my-image {your-random-id}.azurecr.io/my-image:latest
docker push {your-random-id}.azurecr.io/my-image:latest

from your command-line. This will push the image to the registry and make it available for you to use in your Microservices. This image will not be available on public registries, it will only be available to you.