Today we’ll get our hands dirty.

If you are following the CQRS design pattern you might run into trouble when it comes to set based validation. Consider the task of registering a new user. How would you in your domain figure out if that user name is unique? You can obviously not query your event store, because the event store is not built for that purpose.

Querying the reporting database
The solution is actually querying the reporting database on the read side to figure out if the user name is unique. There is (in my opinion) a little problem with this approach. By querying this way you are making the read side responsible for knowing domain concerns. You are effectively making the read-side a part of the domains bounded context and adding more responsibilities to it. Before we introduced set based validation the separation was clean and nice. It is not that clean any more.

You could keep the design clean by having a separate database with the only purpose of answering set based queries and keep this database within the bounded context of the domain. This would keep the design cleaner (perhaps), but it would not be a practical choice in most scenarios.

Another oops
In a scalable architecture you are publishing events asynchronously. They are picked up by the read-side and applied to the reporting database thru the denormalizer. Consider the situation where the UserCreatedEvent(UserName=”BjartN”) is in the denormalizer queue, and the domain is instructed to add yet another user named “BjartN”. The reporting database knows northing of the first event resulting in the domain publishing the same event twice. When the second event hits the reporting database the entire thing blows up, because the user already exists. What to do next ? The event has already occurred (twice) and has been published to many other systems. Basically the events have occurred and you need to deal with it. The only way to fix it is to issue an compensating action. This could be issued by the denormalizer to the domain as a command.

Be consistent if you can
Dealing with these kinds of concurrency issues is not something you want to do for fun. If you have the option to publish the event and save to the reporting database in a single transaction (e.g. synchronously) you should consider that.

Comments F Share

I got this great question in one of my comments:

Modeling the entire history of the domain seems to only be valuable if the domain requires it as part of your domain? I guess the biggest concern with a pattern like this would be how you would effectively deal with different versions of code that have existed at different points along the path. The report one would generate from 10 releases ago might not be the same if you are reapplying events as you go. This could certainly have business implications too, how do you suggest dealing with this?

If the the application of an event changes, do you simply create new events and never modify old ones?

I will try to answer some of it:

Storing the entire history
It’s not so much the domain that usually requires you to keep the entire history, but you keep it for auditing. If you only keep your current state in your database, how do you know that the data is correct ? Anything could have happened to keep it from being correct, like bugs and evil people :) With an add-only model you even have the possibility of writing the events to a write-once-storage. Now it is impossible to manipulate the data. If you write a bug that creates false events, you would later execute compensating actions to undo what you did. Remember that an event is something that  has happened (past tense) so it makes no sense to change it in any way.

There is another good reason to keep the history as well. When saving events you are not saving data to your current data model, but you are actually storing all the user behaviour. If you only store the current state of the domain, the reporting you can do on this data is limited. The information stored in an add-only model is much richer. You can do reports on things you didn’t event think of when you created the application.

Versioning
If your events changes you would create a new version of that event, and  keep the old ones. To keep your domain code form being bloated with handling of all versions of  events you would basically introduce a component that converts your events from previous  to newer versions, and then apply them on the domain. Remember that events are things that actually happened in your domain so in most cases the information in deprecated events are valuable.

That said, there are of course situations where you would choose different architectural patterns.

Comments F Share

The event storage is basically where you persist all the events your domain is publishing. The only thing that makes it a little bit tricky to implement is the fact that you need to worry about concurrency. Concurrency is tricky, but not that tricky.

Event storages can be created using various technologies.  According to Greg the best way is to write the events onto the disc, circumventing the file system. Of course then you’ll need to write your own indexing as well. That’s just crazy stuff and I’m not digging into those APIs any time soon :p In this example I’ll be showing an event storage for a sql database. We need to be able to store our serialized events and keep track of what aggregate root published the event. Also we need to maintain the concurrency version of the aggregate root.

Database

First let’s look at the database schema

EventProviders

EventProviderId (uniqueidentifier)
Type (nvarchar)
VersionNumber (int)

Events

FkEventProviderId (uniqueidentifier)
DateTime (datetime)
Data (varbinary)

The name “event provider” refers in this case to the aggregate root. Each event provider has an id, a type name and a version number. The version number is used to do optimistic locking, making sure we don’t update an aggregate root that has been changed in the meantime. The version number also serves another task; the version number is always equal to the number of events published by the aggregate root.

Now let’s look at the events table. Each event has a reference to the event provider that published the event. We also store a serialized version of the event and the time it was persisted. Notice we have no “eventId”. We don’t need one.

Event storage

We will access the event storage using the IEventStorage interface.

public interface IEventStorage
{
    IEnumerable GetAllEventsForEventProvider(Guid id);
    void Save(IEventProvider provider);
}

To save an aggregate root we pass the a reference to the IEventProvider interface (implemented by the aggregate root) to the event storage.

public interface IEventProvider
  {
    IEnumerable GetChanges();
    void ClearChanges();
    Guid Id { get; set; }
    int Version { get; set; }
  }

The Save method

To save the IEventProvider (i.e. Aggregate Root) we need to do the following

• Start transaction
  • Get the event provider from the database
  • If the event provider does not exist in the database, create it
  • If the event provider exists, check that the database version matches the current version of the aggregate root. If not throw a concurrency exception.
  • Save all events the event provider has published after previous version
  • Update the version number in the event provider. The version number is now the previous version number plus the number of events published since previous version.
• End transaction

The implementation is pretty straight forward. As you see all the complexity in this operation comes from the fact that we need to check for concurrency violations.

Included project

I am including a project with an implementation of the event store and some other things like the repository and the aggregate root base class. This was written quickly at the DDD Course by me and some other peeps, and is far from production ready and it has no UI. However I think it’s better to post this (crappy) code than to post nothing at all. The idea is that you can have some code to look at. There are some integration tests you can run if you set up the database. If you want to use an implementation like this, you can get some speed optimizations by using stored procedures. It’s evil, but still.

Some things to keep in mind

I have not talked about implementing snapshots to improve performance but it is really not that hard. I would use the Memento Pattern and not store the actual aggregate root. From there on it is just plankekjøring ;) Mark Nijhof is publishing an extensive example at some point in the near or far future.

Comments F Share

This post is related to the following ones

1. Introduction to CQRS
2. Testing the domain
3. Modeling behavior
4. Implementing an aggregate root

In one of my previous post I explained the testing benefits of the CQS architecture.

• I arrange my tests by replaying a set of events.
• I act by executing a set of commands.
• And I assert by comparing the events published by the aggregate root to the events I expected to be published.

I am doing Act-Arrange-Assert testing in the simplest most readable way and at the same time creating business value.

In this post I will look at how I would implement my aggregate root to enable this kind of testing. What we need to look at is how the aggregate root implements the replaying of events. This is the key to loading the aggregate root into it’s current state.

The base class

In addition to managing it’s domain logic each aggregate root is responsible for

• Keeping list of all the events that the aggregate root has published after it was loaded.
• Maintaining a mapping between events and the methods for applying the events to the aggregate root.

When saving an aggregate root to the event store you would ask for the events that has been published since it was loaded. That is why we need to store this information. In a testing scenario it is also useful to be able to assert against the events published by the aggregate. For each event, the aggregate root has a method for applying that event. A dictionary of these event-method pairs is maintained by the aggregate root. These responsibilities are common for all aggregate roots, so we will put it in a base class. Before I elaborate on this base class (called HandlesEvents) lets look at an example.

The aggregate root

The aggregate root has a behaviour (i.e. method) that corresponds to a command. Given a ChangeDescriptionCommand coming from the UI you would typically have a ChangeDescription method on the aggregate root. In the command handler you would map a command to a behaviour on the aggregate root.

The implementation of a given behaviour is the key here. In a behaviour you would

1. validate the input,
2. create an event
3. and call a method that applies the event to the aggregate root.

The ChangeDescription behaviour implementation would look something like this:

public class Foo:HandelsEvents {

  private string _description;
  
  public Foo(){
    RegisterHandler<ChangeDescriptionEvent>(applyChangeDescripionEvent);
  } 
      
  public void ChangeDescription(string description){
    if(description == null)
      throw new InvalidDescriptionException();

    ApplyEvent(new ChangeDescriptionEvent(description));
  }

  private applyChangeDescripionEvent(ChangeDescriptionEvent e) {
    _description = e.Description;
  }
}

When the ChangeDecription behaviour is called, the input is validated, the event is created and applied to the aggregate root. Notice I’m am calling the applyChangeDescriptionEvent method thru the base class in stead of calling it directly. This is to enable the base class to do some bookkeeping on what events are called and how they are applied to the aggregate root. Notice that we register the mapping between the event and the method in the constructor. This way the base class knows which method to call when applying the event, both when you are loading an aggregate root into it’s current state and when your applying the event thru a behaviour.

More on the base class

Now lets look at the base class HandlesEvents. The HandlesEvents base class is here for convenience. If I want to replay a set for events I will only need to call the method LoadFromHistory(IEnumerable<IEvent> history) on the HandlesEvent class. The HandlesEvent class will then get the aggregate root in the required state by applying the events to the appropriate methods.

public void LoadFromHistory(IEnumerable<IEvent> history)
{
  foreach(var e in history)
      ApplyEvent(e,false);
}

The ApplyEvent method finds the given handler method for the event and call this method with the event as an parameter. The boolean parameter false is an instruction to the ApplyEvent method not to keep a record of this event. We are only interested in the events that happend after the aggregate root was put in its current state. The ApplyEvent in the HandlesEvent base class will look something like this.

protected void ApplyEvent(IEvent e, bool add)
{
  AppliesEvent handler;

  if(!_lookUp.TryGetValue(e.GetType(), out handler))
    throw new HandlerNotFoundException();
  
  handler(e);

  if(add) _events.Add(e);
}

The HandelsEvents base class will also have a public method GetChanges to expose all the events that where published after the aggregate root was loaded into it’s initial state.

So lets sum up what we’ve got here.

• You can load a set of events into your aggregate root to get it to the required state
• You can invoke a behaviour on the aggregate root
• You can get a list of events that the aggregate root has published

You are ready to start testing.

Should I go deeper into this stuff ? Was this post just confusing ? Is this English ? Hit me with a comment :)

Comments F Share

I got this question in my comments section. I’ll try to answer it the best I can, but I might be leaving out good arguments. Who knows..

Why cannot a single model serve reporting, transactions and searching at the same time? I guess it has something to do with scalability and performance of the application to do. What I have read about CQS this far I have not seen any list of all the benefits of it and I have not yet found the real arguments that I can use to convince the other guys in the team.

The main reason for applying the CQS architecture and DDD is to model complex behaviours in your organization. If your only goal is to put a set of forms on top of your relational database, this approach is not for you.  In that case you would go with Active Record or Row Data gateway, or smack a MS Access application on top of your database. Also if you wan’t to utilize the CQS Architecture it’s best done in green field projects.

Behaviours and data

In object-oriented programming behaviours are the things the objects can do. If I wan’t to create a scalable object model of my (complex) domain I am interested in exposing the behaviour of my objects and encapsulating the internal state of the object (the data). I aim to  keep my object seams as tight as possible. A seam in this context is the exposed interface of my object.

I will now try to explain the difference between exposing data and exposing behaviour.

Let consider the entity “Employee” in a Human Resources Management application. At some point you would like to increase the salary of the employee. In a typical system you would update the “Salary” property on the employee to reflect the changes. Obviously there was a reason for the employee to get his salary changed. Maybe he got promoted, maybe it was just the yearly correction, or maybe he finally completed his master thesis. Let assume the reason was that he finally completed his master thesis. In many companies (at least in Norway) the salary is partly based on education. To capture the new education you would update the “YearsOfEducation” property on the Employee as well. (Simplified, but still.)

The person who is managing the employees need to update different properties in the HRM application to complete his task. Maybe he even has a written note of all the things he has to do in the system in order to update the education of the employee:

In a behaviour centric system this behavior would have been captured by exposing an AddEducation(..) behaviour on the employee. This behaviour (i.e. method) would be responsible for handling all the things that happens when an Employee adds on his education, like getting his salary increased. It would even publish an event so that the salary application could pick up the changes in the employees status.

We want to expose behaviours not data. As a consequence

Setters (i.e public settable properties) are a domain anti-pattern

Since I am interested in modelling behaviours in my domain and keeping a maintainable object model I do not wan’t to expose the data the behaviours is operating upon. This would lead to hard times when I need to change the implementation of my behaviors. As a consequence

Getters are a domain anti-pattern.

To summarize: Do not expose data on your domain model

…and that is the very short answer to why you would have one model for reporting and one for transactions on your domain.

This is the best resource on the  web for really getting why you would model behaviour in your application (and yes, I have checked the entire internet): http://herdingcode.com/?p=189

Performance:

I am hoping to go deeper into consistency, partitioning and availability in an upcoming post. Still, what is interesting about the CQS Architecture is that in many cases you can fire off your commands on the domain asynchronously. Fire and forget. You don’t need to wait for the entire processing to have occurred before returning control to your user.

I hope this answers at least some of your questions Anders.

Comments F Share

One of the most powerful features of the CQS Architecture is the ability to create highly expressive tests.

When a set of commands are executed on an aggregate root the expected result is one or more events.



Before you execute your commands you must get your aggregate root in the required state, in other words you need to set up the context of the test. Loading an aggregate root into the required state is done by replaying a set of events.

Do you see the power of this way of testing ? You can express your test only using events and commands ! Yes. It deserves an exclamation mark. A specification/test would look something like this:

Given
UserCreatedEvent {Name=”Greggy”, Age=12, Address=”Bergen”}

When
ChangeUserNameCommand{Name=”Greg”}

Then
UserNameChangeEvent{Name=”Greg”}

OK, I’ll admit to the fact that this wasn’t a very good example :) Ignoring that, I’m hoping you see the power of writing tests this way. Look at how the language of the events in the past tense and the commands in the imperative form goes together with the “Given-When-Then” paradigm. It’s beautiful how things just fit together using BDD and DDD this way. Of course you would expect that from two so similar abbreviations. In case your were wondering, it was Greg who pointed this out :)

Creating the specifications this way fits well in with DDD idea of the ubiquitous language. You are really getting closer to the BDD dream of having your business people writing your specifications. Sweet.

Should I give some code examples perhaps?

Edit:

In the post I omitted the fact that it’s the command handlers in front of the aggregate root that is actually getting the commands and mapping them to behaviors in the aggregate root. It won’t change the post that much, but it’s important information.

Comments F Share

I have been on a two day intensive DDD course with Greg Young. The course was great both from a social and educational perspective. To make sure I don’t forget all of what I’ve learned at the course I will try to write a couple of blog posts to remind myself, and perhaps give others some valuable information.

The majority of the course revolved around the Command Query Separation (CQS) architecture promoted by Greg in various presentations.

<EDIT>
We now call it  command and query responsibility segregation CQRS :)
</EDIT>

http://www.infoq.com/presentations/greg-young-unshackle-qcon08
http://www.infoq.com/interviews/greg-young-ddd
http://www.vimeo.com/3171910


The CQS architecture follows the same principle as Bertrand Meyer’s Query Command Separation principle ( http://en.wikipedia.org/wiki/Command-query_separation ), but they are not the same.

Overview of the CQS architecture
One of the main ideas of the CQS architecture is keeping a clear separation between changing the state of the domain and querying . The key is that:

“A single model cannot be appropriate for reporting, searching and transactional data”

In box style the architecture it will look something like this:



The write side
The UI will query the read model to get the DTOs it wishes to display. In the UI the user will build up commands and send them to the domain. The commands will be in the imperative form like “ChangeDescriptionCommand”. The domain will process the commands and events. The events will be in the past tense like “DescriptionChangedEvent”. The language is important. It does not need to be a one-to-one relationship between the commands and the events as in the example I just gave.

The read side
On the “read” side the denormalizer subscribes to the events from the domain, and changes the model according to the events published by the domain. The read model is usually optimized for querying, and is often kept in 1. normal form. Depending on your non-functional requirements, the read model can be fully consistent or eventually consistent with the domain.

Add only model
The domain events are stored in the event store. The events are serialized and stored in the order the are published. The event store is important. Notice we have a add-only model. As opposed to the common CRUD paradigm we are only adding information. We are not updating or deleting anything in the event store. That means that we don’t loose any information. Consider a shopping cart. A user adds an item to the shopping cart, and then he removes it again. In a common CRUD scenario we would insert the item and then delete it. When inspecting the data we would see an empty shopping cart. The fact that it was added and then removed is lost. In a add-only model we would still have this information. For “the business” these things can be very valuable.

Of course in a CRUD model you could start recording this information and in a couple of months you could start giving “the business” reports about users adding and removing items. In an add-only model your could create a report of user behavior from the beginning of time. Do you see the difference ? Using an add only model you don’t loose information. You are capturing user behavior as opposed to just updating the data. It’s sweet.

Loading an aggregate root
Loading an aggregate root is a little bit different. Since you are not storing the current state of the domain, but the just the events that have happened you need to load up the aggregate root by supplying all the events that have occurred since the beginning of time and replaying these. This will obviously give an performance problem if you need to load 500 000 events to get your aggregate root to the current state. To solve this problem you can store a snapshot of the aggregate root at certain intervals. When loading the aggregate you load up the last snapshot of the aggregate and replay only the events that happened after the last snapshot.

Implementing the event storage and the snapshots is straight forward, but as in all systems you need to handle concurrency issues which makes it a little bit tricky.

To be continued..
I’m planning to make a series of blog posts digging deeper into various topics and looking at how you would solve some of the obvious problems you will run into using this architecture. I am hoping to cover the following topics, but no promises :)

• Building up commands
• Handling commands
• Building up aggregate roots using the repository and event storage
• Creating the event storage
• Concurrency issues
• Set based validation
• Testing the domain in a BDD style
• Testing the denormalizer
• Disconnected clients
• Contract first development
• Enriching events and moving domain logic into independent components.

Comments F Share