Showing posts with label Architecture. Show all posts
Showing posts with label Architecture. Show all posts

Saturday, May 3, 2014

How Mongo DB can solve Event Sourcing versionning (Part 1 of 2)

First of all I invite you to read my previous post on How to support changes in your software. It will give you a good heads up on where I’m going with this post.

imageHaving a strategy to support change is a good practice but sometimes it fails on the first change because, well you know, it’s not that easy. I’m a big fan of document database and NoSQL. Mongo DB is a good match for me because it’s easy to install on Windows and use from C# application. I won’t go in to detail on how to create a .NET Application that connect to Mongo. The are plenty of site out there (here, here, here and here) that can help you with that. Document databases are often referred to as schema less database but it’s not completely true. There is a schema, it’s just not as rigid and well defined as traditional databases. The schema used Mongo is based on Json format. For example look at the following Json document. (Only the event payload is shown here and in all other sample)

{
    "Name" : "Logitech wireless mouse",
    "Price" : "29.99$"
}

This document can be the definition of a event in an Event Store. The C# class to handle this event should be:

public class ProductAddedEvent : DomainEvent
{
    public string Name { get; set; }
    public string Price { get; set; }
}

The missing fields are manage by the DomainEventbase class. This is a really simple object to illustrate the concept.

imageNow we realize that we made an error in the definition of the Price field. We want to convert it into a double type to be able to make some calculation with it. In an event sourcing system the events are immutable, they must never be changed. The past is written in stone. So we need to work pretty much like functional programming to evolve our event to a new format. The first thing we need to do is to tag each evolution with a version number. Because all our event derived from the DomainEvent class that is easy. First add a base type to DomainEvent to hold the versioning concerns :

[Serializable]
public abstract class DomainEvent : Versionable
{
  // ...
}

Now take alook at the Versionable class :

[Serializable]
public abstract class Versionable
{
  private int? _version;

  public int Version
  {
    get { return _version.GetValueOrDefault(CurrentVersion); }
    set { _version = value; }
  }

  protected internal virtual int CurrentVersion
  {
    get { return 1; }
  }
}

This will set any new DomainEvent to version 1 by default and provide the ability to change it in the future. The Version field will be serialized and save in Mongo DB. Here is the C# class that define the event payload :

public class FooEvent : DomainEvent
{
    public string Name { get; set; }
    public string Value { get; set; }
}

The serialized version of that class will look like this :

{
    "Version" : 1,
    "Name" : "Foo",
    "Value" : "3.00$"
}

imageNow that we can know which version of the document we are processing we can start thinking about changing the C# class. The goal is to preserve all needed data from one version to the other. Depending on the type of modification different technique can be used. Our first change will be to change the Value field type from string to double. The only good way to do that is to use Mongo BSonSerializer attribute.

public class FooEvent : DomainEvent
{
  public string Name { get; set; }
  [BSonSerializer(typeof(AmountToDoubleSerializer)]
  public double Value { get; set; }
}

public class AmountToDoubleSerializer : BsonBaseSerializer
{
  public override object Deserialize(
    BsonReader bsonReader,
    Type nominalType,
    Type actualType,
    IBsonSerializationOptions options)
  {
    VerifyTypes(nominalType, actualType, typeof(Double));

    BsonType bsonType = bsonReader.GetCurrentBsonType();
    switch (bsonType)
    {
    case BsonType.String:
      string readString = bsonReader.ReadString();
      double value = Double.Parse(readString.Replace("$", ""), CultureInfo.InvariantCulture);
      return value;
    case BsonType.Double:
      return bsonReader.ReadDouble();
    default:
      string message = string.Format("Cannot deserialize BsonString from BsonType {0}.", bsonType);
      throw new FileFormatException(message);
    }
  }

  public override void Serialize(
    BsonWriter bsonWriter,
    Type nominalType,
    object value,
    IBsonSerializationOptions options)
  {
    if (value == null)
    {
      throw new ArgumentNullException("value");
    }

    bsonWriter.WriteDouble((Double)value);
  }
}

The AmountToDoubleSerializer will be able to convert string type to double but also to read and save properties that are already in double format. This will allow for our system to read past events as well as new ones.

In the next post I will show how can we do other transformation such as adding, removing and renaming a property.

Tuesday, April 29, 2014

How to support changes in your Software

Event Sourcing is a very powerful architectural concept. Pretty much like a bank account statement everything you write on it is considered immutable. There is no way you can change anything in the past. The only option is to add a new entry to fix a previously made error.

imageThe goal of such concept is to never loose any information not even mistakes. In real life we make mistake all the time. In any good entry form the will be a lot of validation in place to limit those mistakes. But even then mistake can happen.

If the only mistake would be data it won’t be too much of a problem. Any bad data entry can be corrected by a compensating entry that does the exact opposite. But what happens when mistakes are structural.

In software development we always have to choose amongst many differents options. Every time we decide between one structure over another we have to live with the consequences of that choice for the rest of the life of our software. The ability to change the structure of our software decrease exponentially as the features are added. For every new feature we multiply the number of ways we can combine it with other existing features.

How can we build a software that will last longer than its competitor? How cae we embrace changes in our software? In a CQRS and Event Sourcing developed software it’s easy. There is only two main parts that are subject to big change: The Read Model and the Domain.

imageChanges in the Read Model are not so bad. Because all Read Models are built by replaying all historical events, it’s easy to flush them and rebuild them from scratch. The real challenge is to maintain domain events integrity across the life time of your application.

As I wrote before, event store is meant to be immutable therefore it shouldn’t change in any way. But what if you realize that you forgot an important information that you need to track in your domain? Worse, what if you need to remove some properties of your event or an event property need to change its type. Of course those changes should not be the norm but, you know, sh*t happens. In a CQRS and Event Sourcing application supported by Domain Driven Design (DDD) it should be possible to allow such changes. The domain itself may evolve over time. Some business rule may be changed or added and they may need more information to be applied.

The domain objects are good for hiding the internal process of business rules but, in order to be able to do their job, they need some external information. Those information will be persisted in the event store so they will be available later to rebuild the entire object state and be ready to accept any new state change.

imageConceptually the domain only need to know the latest structure of each event in the event store. It should be able to apply them as is to its internal state. In fact the event store will contain earlier version of those events. The goal is to threat them as if they are like their latest counterparts. The best way to do this is exactly like how source control system work such as Git or Mercurial. In those SCM each change set is recorded as a delta from the previous state. They records any new or removed lines of code. So to make that work all we need to do is to have a piece of code that manage transition from any version to the next. Then we need to apply those transitions from the last saved version to the latest version.

How can we do that in our events? See my next post to know more about how to do just like that with a Mongo DB event store.

Saturday, April 26, 2014

Imagine a world where the past is all and only truth

Palais_de_la_Decouverte_Tyrannosaurus_rex_p1050042 Your computer system must be full of structured and relational databases. You might take regular backup of them if you don’t want to loose any information. Despite all those precautions you loose all in-between state of your information.
If all you care about is the final state it’s not a big deal but there are good chances that you have to answer some questions like:
  • How much time passed between the first item was put in the shopping cart and the completion of the transaction
  • How many times an item was removed from a shopping cart
  • What was the state the purchase order before the crash
If you didn’t implement some ways to trace those events you won’t have any answer to give. Even if you find a way to do it, you will only get information from the moment you put it in place.
I suggest you a new approach to never have to answer “it’s imposible” to your boss when he asks you that kind of question: CQRS and Event Sourcing. The power of this duo comes manly from Event Sourcing. You remember everything that you learned about side effects and why you should do everything possible to avoid them. Here those effects are the only important things. In that kind of system we do not keep data for say, we keep the effect of an action that we call passed events. For example, if we execute the following command:
var command = new AddItemToCart(cartId, itemId);
command.Execute();
The system will produce the following event:
var @event = new ItemAdded(cartId, itemId);
ApplyEvent(@event);
The strength of the system comes when we delete items. We are able to trace those delete as events too:
var command = new RemoveItemFromCart(cartId, cartItemIndex);
command.Execute();
Will produce:
var @event = new ItemRemoved(cartId, cartItemIndex);
ApplyEvent(@event);
In this system, all event derived from a base event:
public class EventBase
{
 public Guid Id { get; set; }
 public int Sequence { get; set; }
 public DateTime TimeStamp { get; set; }
}
A framework class ensure that every events get its date and sequence properties set.
In such system, only events are valuable. They reflect what really happened in the past. Those event will be used to build specific read models for each surrounding system the are interested. Each one will have its own small database to answer to its needs and any changes to that database will only affect this system.
Those read models are only transient representation of the system’s past.

Wednesday, September 30, 2009

Using Decorator (or Wrapper) Design Patterns to add Validation to an object

Context

In most cases when someone write about the Decorator pattern it is usually related to UI stuff. The most common example is adding “decoration” to a control, for example a scroll bar. But in my humble opinion, this is not the most useful usage of Decorator. The purpose this pattern is:

“Attach additional responsibilities to an object dynamically. Decorators provide a flexible alternative to sub classing for extending functionality”

Gang of Four

If you think about it, Validation is a responsibility and so it can be added by this pattern. Of course we can add the validation to the class itself or in its base class, but how would you reuse this validation across many unrelated objects and what if you object must derive from another base class?

Solution

Decorator The solution is the Decorator pattern. With this pattern we can add new responsibility to an object without changing its internals.

For simplicity purpose we will take a simple sample that anyone can understand but the concept shown here can apply to much more complex object.

Four our sample we will take a bank account. On this account we should be able to to money deposit and withdraw. The class diagram on the right illustrate this design.

Let’s start by defining our IAccount interface

namespace Model
{
   public interface IAccount {
       string AccountNumber { get; }
       decimal Balance { get; }
       bool Active { get; }
       void Deposit(decimal amount);
       void Withdraw(decimal amount);
       void Close();
   }
}

This simple interface will be the central abstraction of the system. The goal is to never depends on concrete class and this interface in enough to add a lot of functionality around an account.

Now here the interface implementation as an Account:

using System.Diagnostics;

namespace Model
{
   [DebuggerDisplay("Account = {_accountNumber}, Balance = {_balance}")]
   public class Account : IAccount
   {
       public string AccountNumber { get; private set; }
       public decimal Balance { get; private set; }
       public bool Active { get; set; }

       internal Account(string accountNumber, decimal balance)
       {
           AccountNumber = accountNumber;
           Balance = balance;
           Active = true;
       }

       public void Deposit(decimal amount)
       {
           if (OnBeforeDeposit())
               Balance += amount;
           OnAfterDeposit();
       }

       protected virtual bool OnBeforeDeposit()
       {
           return true;
       }

       protected virtual void OnAfterDeposit() { }

       public void Withdraw(decimal amount)
       {
           if (OnBeforeWithdraw())
               Balance -= amount;
           OnAfterWithdraw();
       }

       protected virtual bool OnBeforeWithdraw()
       {
           return true;
       }

       protected virtual void OnAfterWithdraw() { }

       public void Close()
       {
           if (OnBeforeClose())
               Active = false;
           OnAfterClose();
       }

       protected virtual bool OnBeforeClose()
       {
           return true;
       }

       protected virtual void OnAfterClose() { }
   }
}

If you expand the previous block of code you will see that the implementation of IAccount is only doing business stuff. There is no other responsibility in this class than the one that is meant for. We can clearly see “Template Method” design pattern here. All “OnSomething()” method are protected and any derided class can add implementation around the process. All “Before” method can cancel the process if the return value is false. This allow extension classes to add some specific behaviour.

But one of the main thing missing in this class is “validation”. We will use the decorator pattern to do that. A decorator class is a class thst implement all the member of an abstraction and forward all the calls to an internal instance of a real implementation that abstraction. In our case the abstraction is “IAccount” so we have to make a decorator that implement that interface.

namespace Model.Decorator
{
   public abstract class AccountDecorator : IAccount
   {
       private readonly IAccount _account;

       protected IAccount Account
       {
           get { return _account; }
       }

       protected AccountDecorator(IAccount account)
       {
           _account = account;
       }

       public string AccountNumber
       {
           get { return Account.AccountNumber; }
       }

       public decimal Balance
       {
           get { return Account.Balance; }
       }

       public bool Active
       {
           get { return Account.Active; }
       }

       public virtual void Deposit(decimal amount)
       {
           Account.Deposit(amount);
       }

       public virtual void Withdraw(decimal amount)
       {
           Account.Withdraw(amount);
       }

       public virtual void Close()
       {
           Account.Close();
       }
   }
}

As you can see the constructor of the decorator takes an instance of “IAccount”. All method forward their call to that instance. This base class simplifies the process of creating concrete decorator by allowing other decorator to implement some but not all members of the interface.

Now is the time to start implementing our validation class structure. For that purpose we will create a base validation class that will be responsible of applying the validation the IAccount instance.

using System.Collections.Generic;
using Model.Decorator;

namespace Model.Validator
{
   public abstract class AccountValidatorBase : AccountDecorator
   {
       protected abstract IEnumerable<IValidation> GetValidations(decimal amount);

       protected AccountValidatorBase(IAccount account) : base(account) {}

       protected void Validate(decimal amount)
       {
           foreach (var validation in GetValidations(amount))
               if (!validation.IsValid)
                   throw validation.Exception;
       }
   }
}

This simple base class define a “GetValidations” method that all derived class must override to add a list of validation to perform on method call.

Now to implement the “Deposit” validation we have to create this class:

using System.Collections.Generic;
using Model.Validation;

namespace Model.Validator
{
   public class AccountDepositValidator : AccountValidatorBase
   {
       public AccountDepositValidator(IAccount account) : base(account) {}

       protected override IEnumerable<IValidation> GetValidations(decimal amount)
       {
           return new List<IValidation>
           {
               new AmountGreaterThanZeroValidation(amount),
               new AmountShouldHaveOnlyTwoDecimals(amount),
               new AccountMustBeActiveValidation(Account),
               new DepositAmountNotExceedMaxValidation(Account, amount)
           };
       }

       public override void Deposit(decimal amount)
       {
           Validate(amount);
           Account.Deposit(amount);
       }
   }
}

This class is responsible for creating all validation instances. The “Deposit” method is overridden to call the base “Validate” method before doing the actual deposit.

To implement those validation we need to define the “IValidation” interface.

using System;

namespace Model.Validator
{
   public interface IValidation
   {
       bool IsValid { get; }
       Exception Exception { get; }
   }
}

Here is a sample implementation:

using System;
using Model.Validator;

namespace Model.Validation
{
   public class AmountGreaterThanZeroValidation : IValidation
   {
       public AmountGreaterThanZeroValidation(decimal amount)
       {
           Amount = amount;
       }

       public decimal Amount { get; set; }

       public bool IsValid
       {
           get { return Amount > 0; }
       }

       public Exception Exception
       {
           get { return new ArgumentOutOfRangeException("amount", Amount, "Deposit amount should be greater than 0."); }
       }
   }
}

All other validations used in “AccountDepositValidator” can be described the same way.

The last step is to create an actual “IAccount” that will implement all this stuff. To do that we will need a Bank class. A bank is responsible of creating account. It will also be responsible of decorating it with all necessary decorators.

using System;
using System.Collections.Generic;
using Model.Logging;
using Model.Validator;

namespace Model
{
   public static class Bank
   {
       static readonly SortedDictionary<string, IAccount> _accounts = new SortedDictionary<string, IAccount>();
       private static int _accountSequence = 1;

       public static IAccount CreateAccount(decimal balance)
       {
           string accountNumber = String.Format("A{0:0000}", _accountSequence++);
           IAccount account = new Account(accountNumber, balance);
           account = new AccountDepositValidator(account);
           _accounts[account.AccountNumber] = account;
           return account;
       }
   }
}

To illustrate this process in action here is a sequence diagram:

Validation with Decorator

  1. Call Bank.CreateAccount.
  2. The Bank instantiate an Account class.
  3. The Bank create an AccountDepositValidator and wrap Account with it
  4. The Bank return an instance of IAccount.
  5. Deposit is called on IAccount which is an instance of AccountDepositValidator
  6. AccountDepositValidator call Validate
  7. AccountDepositValidator call GetValidations to retrieve the list of validation to evaluate
  8. An AmountGreaterThaZeroValidation is created
  9. IsValid returns true
  10. AccountDepositValidator call base Deposit method
  11. Balance value is updated

Next

With all this in place the only thing left to do is to implement all the other validators for all method of “IAccount”.

Monday, February 23, 2009

Extension methods series: Extension points

As mentioned earlier (the basics, managing the scope, use interfaces) it is important to manage the scope of your extension methods. One other way to do that is to use extension point concept. An extension point is itself an extension method which is only purpose is to transform your object into another type on which you have define plenty of extensions.

The following sample comes from an open source project called Umbrella.

Let’s first try to wrap this concept into an interface:

public interface IExtensionPoint
{
  object ExtendedValue { get; }
  Type ExtendedType { get; }
}

This interface define the basis of an extension point. “ExtendedValue” will hold the source object reference and “ExtendedType” the type of the extended object. Now here is it generic base implementation:

public class ExtensionPoint<T> : IExtensionPoint<T>
{
  private readonly Type type;
  private readonly T value;

  public ExtensionPoint(T value)
  {
      this.value = value;
  }

  public ExtensionPoint(Type type)
  {
      this.type = type;
  }

  #region IExtensionPoint<T> Members

  public T ExtendedValue
  {
      get { return value; }
  }

  object IExtensionPoint.ExtendedValue
  {
      get { return value; }
  }

  public Type ExtendedType
  {
      get { return type ?? (value == null ? typeof (T) : value.GetType()); }
  }

  #endregion
}

This generic class will be used as a base class for all extension points. It contains all the logic to store the value and some read-only properties to get information about it.

Let’s say we want to build some xml serialization extensions, we can start by creating our xml serialization extension point:

public class SerializationExtensionPoint<T> : ExtensionPoint<T>
{
  public SerializationExtensionPoint(T value)
      : base(value)
  {
  }

  public SerializationExtensionPoint(Type type)
      : base(type)
  {
  }
}

Like I said earlier, this class doesn’t do a lot. It’s purpose is only to convert an extension point of T into a serialization extension point of T. To use this we must have a converter extension method in scope:

public static class SerializationExtensions
{
  public static SerializationExtensionPoint<T> Serialize<T>(this T value)
  {
      return new SerializationExtensionPoint<T>(value);
  }
}

The “Serialization” extension method is called to get access to all other serialization extension methods.

Somewhere in you code you will have this method. This method can be applied to any type because it takes T as a source. The last step is to define an extension method on SerializationExtensionPoint:

public static string ToXml<T>(this SerializationExtensionPoint<T> extensionPoint)
{
  using (var stream = new MemoryStream())
  {
      Xml(extensionPoint, stream, extensionPoint.ExtendedValue);

      stream.Position = 0;
      StreamReader reader = new StreamReader(stream);

      return reader.ReadToEnd();
  }
}

This method will convert any object into XML. Look how easy it is to read this: “source serialize to xml”.

var source = new List<string>();
source.Add("Test1");
source.Add("Test2");
source.Add("Test3");
var xml = source.Serialize().ToXml();

You will get:

<?xml version="1.0" ?>
<ArrayOfString xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<string>Test1</string>
<string>Test2</string>
<string>Test3</string>
</ArrayOfString>

Like any API development, one good practice to follow is to start by writing how you will use it. For example, in the last sample we could have started by writing “source.Serialize().ToXml();” and then start implementing the whole thing to make it work. The result of this is improved readability and reusability. Two things that helps a lot when someone else (or even you) have to modify your code later.

Friday, September 12, 2008

Presenting @ DevTeach

If you needed one more reason to register for DevTeach here it is. I'm giving a talk about parallel extensions. Here is the teaser:
A glimpse into the parallel universe
There is no more free lunch! The Moore's law is over. If we want more power we need to cross the processor barrier and do work in parallel. In June 2008, Microsoft released its second CTP of Parallel Extensions library. Come with me to see how easy it will be to make that leap of faith into the world of parallel processing. We will see how Task, concurrent collections, lazy initialization, parallel Linq and other tools can help us in this endeavour.