Guidelines

• 1: The Vision
• 2: Code of conduct
• 3: Core values
• 4: Core principles
• 5: Logging
• 6: C# coding styles
• 7: C# Specifications
• 8: Copyright header

1 - The Vision

Our vision at Dolittle is to build a platform to solve problems for line-of-business applications that is easy to use, increases developer productivity while remaining easy to maintain.

While our vision remains constant details around what needs to be implemented shifts over time as we learn more and gain experience on how the Dolittle framework is used in production. Dolittle will adapt as new techniques and technologies emerge.

Background

Dolittle targets the line of business type of application development. In this space there are very often requirements that are somewhat different than making other types of applications. Unlike creating a web site with content, line of business applications has more advanced business logic and rules associated with it. In addition, most line of business applications tend to live for a long time once they are being used by users. Big rewrites are often not an option, as it involves a lot of work to capture existing features and domain logic in a new implementation. This means that one needs to think more about the maintainability of the product. In addition to this, in a fast moving world, code needs to built in a way that allows for rapidly adapting to new requirements. It truly can be a life/death situation for a company if the company is not able to adapt to market changes, competitors or users wanting new features. Traditional techniques for building software have issues related to this. N-tier architecture tends to mix concerns and responsibilities and thus leading to software that is hard to maintain. According to Fred Brooks and “The Mythical Man-Month”, 90% of the cost related to a typical system arise in the maintenance phase. This means that we should aim towards building our systems in a way that makes the maintenance phase as easy as possible.

The goal of Dolittle is to help make this better by focusing on bringing together good software patterns and practices, and sticking to them without compromise. Dolittle embraces a set of practices described in this article and aims to adhere to them fully.

History

The project got started by Einar Ingebrigtsen in late 2008 with the first public commits going out to Codeplex in early 2009. It was originally called Bifrost. Source control History between 2009 and 2012 still sits there. The initial thoughts behind the project was to encapsulate commonly used building blocks. In 2009, Michael Smith and Einar took the project in a completely different direction after real world experience with traditional n-tier architecture and the discovery of commands. In 2012 it was moved to GitHub.

The original Bifrost repository can be found here.

From the beginning the project evolved through the needs we saw when consulting for different companies. Amongst these were Komplett. It has always had a high focus on delivering the building blocks to be able to deliver the true business value. This has been possible by engaging very close with domain experts and developers working on line of business solutions.

A presentation @ NDC 2011 showcases the work that was done, you can find it here. From 2012 to 2015 it got further developed @ Statoil and their needs for a critical LOB application; ProCoSys. In 2015, Børge Nordli became the primary Dolittle resource @ Statoil and late 2015 he started maintaining a fork that was used by the project. Pull Requests from the fork has been coming in steadily.

The effort of design and thoughtwork going into the project is a result of great collaboration over the years. Not only by the primary maintainers; Michael, Børge and Einar - but all colleagues and other contributors to the project.

2 - Code of conduct

Contributor Covenant Code of Conduct

Our Pledge

In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.

Our Standards

Examples of behavior that contributes to creating a positive environment include:

• Using welcoming and inclusive language
• Being respectful of differing viewpoints and experiences
• Gracefully accepting constructive criticism
• Focusing on what is best for the community
• Showing empathy towards other community members

Examples of unacceptable behavior by participants include:

• The use of sexualized language or imagery and unwelcome sexual attention or advances
• Trolling, insulting/derogatory comments, and personal or political attacks
• Public or private harassment
• Publishing others' private information, such as a physical or electronic address, without explicit permission
• Other conduct which could reasonably be considered inappropriate in a professional setting

Our Responsibilities

Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.

Scope

This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.

Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting hello@dolittle.com. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.

Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project’s leadership.

Attribution

This Code of Conduct is adapted from the Contributor Covenant, version 1.4, available at http://contributor-covenant.org/version/1/4

3 - Core values

At Dolittle we believe that good software stems from a set of core values. These values guides us towards our core principles and also manifested in our development principles that translates it into guidelines we use for our development. This page describes these core values to help put ourselves into the pit of success.

Privacy

We value privacy at all levels. Core to everything we do is rooted in this. This means we will always strive towards making the right technology choice that lets the owner of data have full control over where it is stored and the ownership is always very clear. These things should always be at the back every developers mind when making choices. It is easy to forget that even a little log statement could violate this.

Empowering developers

The Dolittle mission is to empower developers to create great, sustainable, maintainable software so that they can make their users feel like heroes. This is part of our DNA - representing how we think and how we approach every aspect of our product development. Our products range from libraries to frameworks to tooling, and every step of the way we try to make it as easy as possible for the developer consuming our technology.

Delivering business value

When empowering developers, this is because we want to make it easier to create great technical solutions without having to implement all the nitty and gritty details of doing so; so that our end-users - the developers we are building for, can focus on delivering the business value for their businesses. For Dolittle the developer using our technology is our end-users and represent our business value. Our promise in this is that we will build relevant technology and not technology for the technology sake. Obviously, this is balanced with innovation and we will try out things, but close the feedback loop as tight as possible so that we try things out and iterate on it to see if things are delivering business value.

User focused

At the end of the day, whatever we are building and at any level - we build things that affect an end user. There is always a person at the end of everything being done. This is critical to remember. We build software to help people build software that are more relevant and improves the lives of the actual end user using the software.

With this in hand, we work hard to understand the persona; who are we building for and what is the effect of building something.

Embracing change

The world is constantly changing, so should software. Adapting to new knowledge, new opportunities and new challenges is how the world has always moved on. It is therefor a vital piece of Dolittle to be able to embrace this change. This is a mindset and something we strongly believe in, and is also something we stribe towards in our codebase; making it possible to adapt and change to new requirements without having to recreate everything.

Being pragmatic

Pragmatism is important, keeping things real, relevant and practical is at the core of this. However, it should be treated as a trumph card for taking shortcuts and not adhering to our principles - “as that would be the pragmatic way”. It is related to the approach, which tool and in general the how. We also keep our focus on the outcome and never deviate from what outcome we are trying to achieve.

4 - Core principles

Security

From everything we do; security is at the heart. We want users to feel secure when using systems built on top of the Dolittle frameworks and platform. Zero trust is way of thinking that basically ensures that all data and resources are accessed in a secure manner.

Storage over compute

For everything we do at Dolittle and in the Dolittle frameworks, we always favor using more storage than compute. Compute-power is always the most expensive part of systems while storage is the cheapest. This means if one has the chance and it sustainable - duplicates in storage for different purposes is always preferred. Since the Dolittle architecture is built around events and the source of truth is sitting inside an event store, there is a great opportunity of leveraging the storage capabilities out there and really not be afraid of duplicates. This do however mean one needs to embrace the concept of eventual consistency.

Multi-tenancy

Since compute is the most expensive, the Dolittle frameworks and platform has been built from the ground up with multi-tenancy in mind. This basically means that a single process running the Dolittle runtime, can represent multiple tenants of the application the runtime represents. This makes for a more optimal use of resources. The way one then does things is based on the execution context with the tenant information in it, we can use the correct connection-string for a database for instance or other information to a resource.

Tenant segregation

With everything being multi-tenant we also focus on segregating the tenants. This principle means that we do not share a resource - unless it can cryptographically guarantee that data could not be shared between two tenants by accident. Everything in the Dolittle frameworks has been built from the ground up with this in mind and with the resource system at play, you’ll be able to transparently work as if it was a single tenant solution and the Dolittle frameworks in conjunction with the platform would then guarantee the correct resource.

Privacy

Data should in no way made available to arbitrary personnel. Only if the data owner has consented should one get access to data. Much like GDPR is saying for personal data and the consent framework defined, business to business should be treated in the same way. This means that developers trying to hunt down a bug, shouldn’t just be granted access to a production system and its data without the consent of the actual data owner. An application developer that builds a multi-tenant application might not even be the data owner, while its customers probably are. This should be governed in agreements between the application owner and the data owner.

Just enough software

A very core value we have at Dolittle is to not deliver more than just enough. We know the least at the beginning of a project and the only way we can know if anything works is to put it into the hands of others. Only then can we really see what worked and what didn’t. Therefor it is essential that we only do just enough. In the words of Sarah Lewis; “We thrive not when we’ve done it all, but when we still have more to do” (See her TED talk here. Others has said similar things with the same sentiments - like LinkedIns Reid Hoffman said; “If you’re not embarrassed by your first product release, you’ve released it too late.”.

In order to be able to do so and guarantee a consistent level og quality, you have to have some core values and guiding principles to help you along the way.

5 - Logging

Logs are an important tool for developers to both understand program flow, and trace down bugs and errors as they appear in their software. Comprehensive, cohesive and focused log messages are key to the efficacy of logs as a development tool. To ensure that we empower developers with our software, we have put in place five guiding principles for writing log messages.

Structured log messages

Traditionally log messages have been stored as strings with data embedded with string formatting. While being simple to store and transmit, these strings loose semantic and contextual information about data types and parameters. This in turn makes searching and displaying log messages labour intensive and require specialized tools.

Modern logging frameworks support structured or semantic log messages. These frameworks split the definition of the human readable log message from the data it contains. All popular logging frameworks support the message template format specification, or a subset thereof.

logger.Trace("Committing events for {AggregateRoot} on {EventSource}", aggregateRoot.Id, eventSourceId);

TRACE 2020/04/03 12:19:58 Committing events for 9eb48567-c3ac-434b-90f1-26660723103b on 2fd8866a-9a4b-492b-8e98-791118552426

{
    "level": "trace",
    "timestamp": "2020-04-03T12:19:58.060Z",
    "category": "Dolittle.Commands.Coordination.Runtime",
    "template": "Committing events for {AggregateRoot} on {EventSource}",
    "data": {
        "AggregateRoot": "9eb48567-c3ac-434b-90f1-26660723103b",
        "EventSource": "2fd8866a-9a4b-492b-8e98-791118552426"
    }
}

Log message categories

To allow filtering of log messages from different parts of the source code during execution flow, log messages must contain a category. In most languages this category is defined by the fully qualified name of the types that define the code executed, including the package or namespace in which the type resides. These categories are commonly used during debugging to selectively enable Debug or Trace messages for parts of the software by defining filters on the log message output.

Log message levels

We define five log message levels that represent the intent or severity of the log message. They are, in decreasing order of severity:

• Error - unrecoverable failure, resulting in end-user error.
• Warning - recoverable failure, performance or functionality is degraded.
• Information - information that is needed to use the software, and user activity traces.
• Debug - execution activity and sub-activity checkpoints.
• Trace - detailed execution trace with data that affects flow path.

Error

An error log message indicates that an unrecoverable failure has occurred, and that the current execution flow has stopped as a consequence of the failure. The current activity that the software was performing is not possible to complete, and will therefore in most cases lead to an end user error message being shown. For languages that have the concept of exceptions or errors, these must be included in an error log message. An error log message indicates that immediate action is required to recover full software functionality.

Warning

While an error message indicates an unrecoverable failure, warning log messages indicate a recoverable failure or abnormal or unexpected behavior. The current execution flow is able to continue to complete the current activity by recovering to a fail-safe state, albeit with possible degraded performance or functionality. Typical examples would be that an expected data structure that was not found but it is possible to continue with default values, or multiple data structures were found where there should only be one, but it is safe to continue. A warning log message indicates that cleanup or validation is required at a later point in time to recover or verify the intended software functionality.

Warning log messages are also used to warn developers about wrong usage of functionality, and deprecated functionality that will be removed in the future.

Information

Informational log messages tracks the general execution flow of the software, and provides the developer with required information to use the software correctly. These log messages have long term value, and typically include host startup information and user interactions with the application.

Information level log messages is typically the lowest severity messages that will be written by default, and must therefore not be used to log messages that are not useful for while the software is working as expected.

Debug

Debug log messages are used by developers to figure out where failures occur during execution flow while investigating interactively with the software. These log messages represents high-level checkpoints of activities and sub-activities during execution flow, to give hints for what log message categories and source code to investigate in more detail. Debug messages should not contain any data other than correlation and trace identifiers used to identify unique failing interactions.

Trace

Trace log messages are the most verbose of the log messages. They are used by developers to figure out what caused a failure or an unexpected behavior, and should therefore contain the data that affects the execution flow path. Typical uses of trace log messages are public methods on interface implementations, and contents of collections used for lookup.

Log output

The logs of an applications is its source of truth. It is important that log messages are consistent in where they are outputted and the format in which they are outputted. They should be outputted to a place where they can be easily retrieved by anyone who is supposed to read them. Log messages are normally outputted to the console, but they can also be appended to files. The log messages that are outputted should be readable and have a consistent style and format.

Configuring

We’re not necessarily interested in all of the logging levels or all of the categories each time we run an application. The logging should be easily configurable so that we can choose what we want to see in terms of categories and the levels of the logging. For instance software running in a production environment should consider only logging information, warning and error log messages. While we may want to show more log messages when running in development mode. It is also important to keep in mind that logging can possibly have a considerable performance cost. This is especially important to consider when deploying software with lots of logging to production environments.

Asp.Net Core

We’re using Microsoft’s logger in the Dolittle framework for .Net. We can use the ‘appsettings.json’ to configure the logging and we can provide different configurations for different environments like production and development. Look here for information on Microsoft’s logger.

Log message

Log messages should be written in a style that makes it easy to navigate and filter out irrelevant information so that we can find the cause of any error that has occurred by simply reviewing the them. Logs should be focused and comprehensive for both humans and machines. They should also be consistent in format and style across platforms, languages and frameworks.

Stick to English

There are arguably many reasons to stick to English-only log messages. One technical reason is that English ensures us that we stick to ASCII character set. This is important because we don’t necessarily know what happens to the log message. If the log messages uses specials character sets it might not render correctly or can become corrupt and thus unreadable.

Log context

Each log message should contain enough information so that the intended reader understands exactly what is going on without having to read any prior log messages. When we write log messages it is in the context of the code that we write, in the context of where the log statement is, and it is easy to forget that this context information is not implicit in the outputted log. And depending on the content of those log messages they might even not be comprehendible in the end.

There are possibly multiple aspects of ‘context’ in regards to logging. One might be the current environment or execution context of the application for when the logging is performed and the other might be domain specific context, meaning information regarding where the logging is taking place in the execution flow of an operation or values of interest like IDs and names. Log messages should have the appropriate information regarding the context relevant to the information that is intended to be communicated. For example for multi-threaded applications it would make sense to add information of the executing thread id and correlations between actions. And for multi-tenanted applications it would make sense to have information about the tenant the procedures are performed in.

It is important to consider the weight of the contextual information added to each log message. Adding lots of context information to every log message makes the log messages bloated and less human-readable. The amount of context information on a log message should be in proportion to the log message level. For instance an information log message should not contain lots of contextual information that is not strictly needed for the end-user to use the software while a trace or debug log message should contain the necessary information to deduce the cause of an error. For warning and error log messages that are produced as a result of an exception or error it is important to include the stacktrace of the exception/error as part of the log message. Usually the methods or procedures to create log messages at these levels has its own parameter for an exception/error that outputs a log with the stacktrace nicely formatted.

For statically typed languages the namespace of the code executing the logging statement is usually provided with the log message which is helpful information for the developers in the case of troubleshooting.

Keep in mind the reader of the logs

We add logs to software because someone most likely has to read them someday. Thus it makes sense that we should keep in mind the target audience when writing log messages. Which person is most likely going to read a log message affects all the aspects of that log message; The log message content, level and category is dependent on that. Information log messages is intended for the end-user while trace and debug messages are most likely only read in the case of troubleshooting, meaning that only developers will read them. The content of the log message be targeted towards the intended audience.

Don’t log sensitive information

Sensitive information like personal identifiable information, passwords and social security numbers has no place in log messages.

6 - C# coding styles

This is the to be considered the coding standard for Dolittle and is subject to automated verification during automated builds and also part of code-reviews such as those done for pull requests.

Values, principles and patterns & practices

It is assumed that all code written is adhering to our core values and core principles.

Compactness

In general, code should be compact in the sense that any “noise” of language artifacts or similar that aren’t really needed SHALL NOT be used. This to increase readability, not decrease it. Things that are implicit, SHALL be left implicit and not turned into explicits.

Keywords

Use of var

Types are implicitly provided by the compiler and considered noise during declaration. If one feel the need for explicitly declaring variables with their type, it is often a symptom of something else being wrong - such as large methods that you can’t get a feel for straight away. This is most likely breaking the Single Responsibility Principle. You MUST use var and let the compiler infer the type implicitly.

Private members

In C# the private modifier is not needed as this is the default modifier if nothing is specified. Private members SHALL NOT have a private modifier.

Example:

public class SomeClass
{
    string _someString;
}

this

Explicit use of this SHALL NOT be used. With the convention for prefixing private members, the differentiation is clear.

Prefixes and postfixes

A very common thing in naming is to include pre/post fixes that describes the technical implementation or even the pattern that is being used in the implementation. This does not serve as useful information. Examples of this is Manager, Helper, Repository, Controller and more (e.g. EmployeeRepository). You SHOULD NOT pre or postfix, but rather come up with a name that describes what it is. Take EmployeeRepositorysample, the postfix Repository is not useful for the consumer; a better name would be Employees.

Member variables

Member variables MUST be prefixed with an underscore.

Example:

public class SomeClass
{
    string _someInstanceMember;
    static string _someStaticMember;
}

One type per file

All files MUST contain only one type.

Class naming

Naming of classes SHALL be unambiguous and by name tell exactly what it is providing. Example:

// Coordinates uncommitted event streams
public class UncommittedEventStreamCoordinator {}

Interface naming

Its been a common naming strategy to include Iin front of any interface. Prefixing with Ican have other meaning as well, such as the actual word “I”. This can give better naming to interfaces and better meaning to names.

Examples:

// Implemented by types that can provide configuration
public interface ICanConfigure {}

// Implemented by a type that can provide a container instance
public interface ICanCreateContainer

You SHOULD try look for this way of naming, as it provides a whole new level of expressing intent in the code.

Private methods

Private methods MUST be placed at the end of a class.

Example:

public class SomeClass
{
    public void PublicMethod()
    {
        PrivateMethod();
    }


    void PrivateMethod()
    {

    }
}

Exceptions

flow

Exceptions are to be considered exceptional state. They MUST NOT be used to control program flow. Exceptional state is typically caused by infrastructure problems or other problems causing normal flow to be able to continue.

types

You MUST create explicit exception types and NOT use built in ones. The exception type can implement one of the standard ones.

Example:

public class SomethingIsNull : ArgumentException
{
    public SomethingIsNull() : base("Something was null") {}
}

Throwing

If there is a reason to throw an exception, your validation code and actual throwing MUST be in a separate private method.

Example:

public class SomeClass
{
    public void PublicMethod(string something)
    {
        ThrowIfSomethingIsNull(something);
    }

    void ThrowIfSomethingIsNull(string something)
    {
        if( something == null ) throw new SomethingIsNull();
    }
}

Async / Await

In C# the async / await keywords should be used with utmost care. It is a thing that without really thinking it through can bleed throughout your codebase without necessarily a good reason. Alongside async / await comes the Task type that needs to be there. The places where threading is necessary, it MUST be dealt with internally to the implementation and not bleed throughout its APIs. Dolittle has a very good handle on its entrypoints and from these entrypoints, the need for scaling out across multiple threads are rarely needed. With the underlying infrastructure being relied on, web requests are already threaded. Since we enter the system and returns back as soon possible, we have a good grip of when this is needed. Threads can easily get out of hand and actually slow down systems.

Exposing IList / ICollection

Public APIs SHALL NOT have mutable types as return types, such as IList, ICollection. The responsibility for maintaining state should be with the owner of it. By exposing the ability for changing state outside the owner, you lose control over who can change state and side-effects occur that aren’t clear. Instead you should always expose immutable types like IEnumerable instead.

Mutability

One of the biggest cause of side-effects in a system is the ability to mutate state and possibly state one does not necessarily own. The example is something creates an instance of an object and exposes public getters and setters for its properties and inviting anyone to change this state. This makes it hard to track which part of the system actually changed the state. Be very conscious about ownership of instances. Avoid mutability. Most of the time it is not needed. Instead, create new objects with the mutation in place.

7 - C# Specifications

All the C# code has been specified by using Machine Specifications with an adapted style. Since we’re using this for specifying units as well, we have a certain structure to reflect this. The structure is reflected in the folder structure and naming of files.

Folder structure

The basic folder structure we have is:

(project to specify).Specs
    (namespace)
        for_(unit to specify)
            given
                a_(context).cs
            when_(behavior to specify).cs

A concrete sample of this would be:

Dolittle.Specs
    Commands
        for_CommandContext
            given
                a_command_context_for_a_simple_command_with_one_tracked_object.cs
            when_committing.cs

The implementation SHOULD then look something like this :

    public class when_committing : given.a_command_context_for_a_simple_command_with_one_tracked_object_with_one_uncommitted_event
    {
        static UncommittedEventStream   event_stream;

        Establish context = () => event_store_mock.Setup(e=>e.Save(Moq.It.IsAny<UncommittedEventStream>())).Callback((UncommittedEventStream s) => event_stream = s);

        Because of = () => command_context.Commit();

        It should_call_save = () => event_stream.ShouldNotBeNull();
        It should_call_save_with_the_event_in_event_stream = () => event_stream.ShouldContainOnly(uncommitted_event);
        It should_commit_aggregated_root = () => aggregated_root.CommitCalled.ShouldBeTrue();
    }

The specifications should read out very clearly in plain English, which makes the code look very different from what we do for our units. For instance we use underscore (_) as space in type names, variable names and the specification delegates. We also want to keep things as one-liners, so your Establish, Because and It statements should preferably be on one line. There are some cases were this does not make any sense, when you need to verify more complex scenarios. This also means that an It statement should be one assert. Moq is used for for handling mocking / faking of objects.

8 - Copyright header

Code files

All code files MUST to have the following copyright header, this includes even automated test files for all languages. The format needs to adhere to the following.

// Copyright (c) Dolittle. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

For XML based languages, this would look like:

<!-- Copyright (c) Dolittle.  All Rights Reserved.  Licensed under the MIT License. See LICENSE file in the project root for full license information. -->

Other languages might have other ways to represents comments, for instance bash/shell scripts or similar:

# Copyright (c) Dolittle. All rights reserved.
# Licensed under the MIT license. See LICENSE file in the project root for full license information.