How Mongo DB can solve Event Sourcing versionning (Part 1 of 2)

First of all I invite you to read my previous post on How to support changes in your software. It will give you a good heads up on where I’m going with this post.

Having a strategy to support change is a good practice but sometimes it fails on the first change because, well you know, it’s not that easy. I’m a big fan of document database and NoSQL. Mongo DB is a good match for me because it’s easy to install on Windows and use from C# application. I won’t go in to detail on how to create a .NET Application that connect to Mongo. The are plenty of site out there (here, here, here and here) that can help you with that. Document databases are often referred to as schema less database but it’s not completely true. There is a schema, it’s just not as rigid and well defined as traditional databases. The schema used Mongo is based on Json format. For example look at the following Json document. (Only the event payload is shown here and in all other sample)

{
    "Name" : "Logitech wireless mouse",
    "Price" : "29.99$"
}

This document can be the definition of a event in an Event Store. The C# class to handle this event should be:

public class ProductAddedEvent : DomainEvent
{
    public string Name { get; set; }
    public string Price { get; set; }
}

The missing fields are manage by the DomainEventbase class. This is a really simple object to illustrate the concept.

Now we realize that we made an error in the definition of the Price field. We want to convert it into a double type to be able to make some calculation with it. In an event sourcing system the events are immutable, they must never be changed. The past is written in stone. So we need to work pretty much like functional programming to evolve our event to a new format. The first thing we need to do is to tag each evolution with a version number. Because all our event derived from the DomainEvent class that is easy. First add a base type to DomainEvent to hold the versioning concerns :

[Serializable]
public abstract class DomainEvent : Versionable
{
  // ...
}

Now take alook at the Versionable class :

[Serializable]
public abstract class Versionable
{
  private int? _version;

  public int Version
  {
    get { return _version.GetValueOrDefault(CurrentVersion); }
    set { _version = value; }
  }

  protected internal virtual int CurrentVersion
  {
    get { return 1; }
  }
}

This will set any new DomainEvent to version 1 by default and provide the ability to change it in the future. The Version field will be serialized and save in Mongo DB. Here is the C# class that define the event payload :

public class FooEvent : DomainEvent
{
    public string Name { get; set; }
    public string Value { get; set; }
}

The serialized version of that class will look like this :

{
    "Version" : 1,
    "Name" : "Foo",
    "Value" : "3.00$"
}

Now that we can know which version of the document we are processing we can start thinking about changing the C# class. The goal is to preserve all needed data from one version to the other. Depending on the type of modification different technique can be used. Our first change will be to change the Value field type from string to double. The only good way to do that is to use Mongo BSonSerializer attribute.

public class FooEvent : DomainEvent
{
  public string Name { get; set; }
  [BSonSerializer(typeof(AmountToDoubleSerializer)]
  public double Value { get; set; }
}

public class AmountToDoubleSerializer : BsonBaseSerializer
{
  public override object Deserialize(
    BsonReader bsonReader,
    Type nominalType,
    Type actualType,
    IBsonSerializationOptions options)
  {
    VerifyTypes(nominalType, actualType, typeof(Double));

    BsonType bsonType = bsonReader.GetCurrentBsonType();
    switch (bsonType)
    {
    case BsonType.String:
      string readString = bsonReader.ReadString();
      double value = Double.Parse(readString.Replace("$", ""), CultureInfo.InvariantCulture);
      return value;
    case BsonType.Double:
      return bsonReader.ReadDouble();
    default:
      string message = string.Format("Cannot deserialize BsonString from BsonType {0}.", bsonType);
      throw new FileFormatException(message);
    }
  }

  public override void Serialize(
    BsonWriter bsonWriter,
    Type nominalType,
    object value,
    IBsonSerializationOptions options)
  {
    if (value == null)
    {
      throw new ArgumentNullException("value");
    }

    bsonWriter.WriteDouble((Double)value);
  }
}

The AmountToDoubleSerializer will be able to convert string type to double but also to read and save properties that are already in double format. This will allow for our system to read past events as well as new ones.

In the next post I will show how can we do other transformation such as adding, removing and renaming a property.

How to support changes in your Software

Event Sourcing is a very powerful architectural concept. Pretty much like a bank account statement everything you write on it is considered immutable. There is no way you can change anything in the past. The only option is to add a new entry to fix a previously made error.

The goal of such concept is to never loose any information not even mistakes. In real life we make mistake all the time. In any good entry form the will be a lot of validation in place to limit those mistakes. But even then mistake can happen.

If the only mistake would be data it won’t be too much of a problem. Any bad data entry can be corrected by a compensating entry that does the exact opposite. But what happens when mistakes are structural.

In software development we always have to choose amongst many differents options. Every time we decide between one structure over another we have to live with the consequences of that choice for the rest of the life of our software. The ability to change the structure of our software decrease exponentially as the features are added. For every new feature we multiply the number of ways we can combine it with other existing features.

How can we build a software that will last longer than its competitor? How cae we embrace changes in our software? In a CQRS and Event Sourcing developed software it’s easy. There is only two main parts that are subject to big change: The Read Model and the Domain.

Changes in the Read Model are not so bad. Because all Read Models are built by replaying all historical events, it’s easy to flush them and rebuild them from scratch. The real challenge is to maintain domain events integrity across the life time of your application.

As I wrote before, event store is meant to be immutable therefore it shouldn’t change in any way. But what if you realize that you forgot an important information that you need to track in your domain? Worse, what if you need to remove some properties of your event or an event property need to change its type. Of course those changes should not be the norm but, you know, sh*t happens. In a CQRS and Event Sourcing application supported by Domain Driven Design (DDD) it should be possible to allow such changes. The domain itself may evolve over time. Some business rule may be changed or added and they may need more information to be applied.

The domain objects are good for hiding the internal process of business rules but, in order to be able to do their job, they need some external information. Those information will be persisted in the event store so they will be available later to rebuild the entire object state and be ready to accept any new state change.

Conceptually the domain only need to know the latest structure of each event in the event store. It should be able to apply them as is to its internal state. In fact the event store will contain earlier version of those events. The goal is to threat them as if they are like their latest counterparts. The best way to do this is exactly like how source control system work such as Git or Mercurial. In those SCM each change set is recorded as a delta from the previous state. They records any new or removed lines of code. So to make that work all we need to do is to have a piece of code that manage transition from any version to the next. Then we need to apply those transitions from the last saved version to the latest version.

How can we do that in our events? See my next post to know more about how to do just like that with a Mongo DB event store.

Imagine a world where the past is all and only truth

Your computer system must be full of structured and relational databases. You might take regular backup of them if you don’t want to loose any information. Despite all those precautions you loose all in-between state of your information.
If all you care about is the final state it’s not a big deal but there are good chances that you have to answer some questions like:

• How much time passed between the first item was put in the shopping cart and the completion of the transaction
• How many times an item was removed from a shopping cart
• What was the state the purchase order before the crash

If you didn’t implement some ways to trace those events you won’t have any answer to give. Even if you find a way to do it, you will only get information from the moment you put it in place.
I suggest you a new approach to never have to answer “it’s imposible” to your boss when he asks you that kind of question: CQRS and Event Sourcing. The power of this duo comes manly from Event Sourcing. You remember everything that you learned about side effects and why you should do everything possible to avoid them. Here those effects are the only important things. In that kind of system we do not keep data for say, we keep the effect of an action that we call passed events. For example, if we execute the following command:

var command = new AddItemToCart(cartId, itemId);
command.Execute();

The system will produce the following event:

var @event = new ItemAdded(cartId, itemId);
ApplyEvent(@event);

The strength of the system comes when we delete items. We are able to trace those delete as events too:

var command = new RemoveItemFromCart(cartId, cartItemIndex);
command.Execute();

Will produce:

var @event = new ItemRemoved(cartId, cartItemIndex);
ApplyEvent(@event);

In this system, all event derived from a base event:

public class EventBase
{
 public Guid Id { get; set; }
 public int Sequence { get; set; }
 public DateTime TimeStamp { get; set; }
}

A framework class ensure that every events get its date and sequence properties set.
In such system, only events are valuable. They reflect what really happened in the past. Those event will be used to build specific read models for each surrounding system the are interested. Each one will have its own small database to answer to its needs and any changes to that database will only affect this system.
Those read models are only transient representation of the system’s past.