There is a growing change in the software development world around considering the choice of a NoSQL database platform over the more typical SQL database. SQL databases are centered around the idea of relational tables. Data is normalized and separated, optimizing disk space and query speed, and then linked together with foreign keys to create relationships. In contrast, NoSQL deals primarily with documents. Documents consists of flattened data, that might typically be normalized. While this may result in duplication of data and increased disk space, NoSQL is able to optimize query speed, based upon a key/value query process. Of course, one of the most popular features of NoSQL, particularly for developers, is the lack of a database schema design. NoSQL databases allow automatic creation of the database schema purely from the C# .NET data types. Changes to the database design are driven completely from the software developer’s code (ie., type library). With a new database model, some slight changes to traditional software architecture is required.

In this tutorial, we’ll create a basic C# .NET application that creates and displays Dragons from a NoSQL MongoDb database. Our C# .NET architecture will utilize the repository pattern, combined with a global database context provider. We’ll create a 3-tier system for accessing the Dragons, creating, updating, and deleting.

Display a list of dragons with the MongoDb Repository Pattern NoSQL C# ASP .NET

Database Administrators - Do Not Read

One the more touted features of NoSQL, and MongoDb in particular, is the creation of database tables and automatic schemas purely from the software C# .NET classes. Rather than having a database administrator DBA create the initial database schema and then having the developer mirror the schema with a C# .NET class library, the chore can be done in a single location.

To begin a MongoDb implementation, the C# .NET software developer can create the C# classes for the entities to track in the database. Upon saving the entities to the NoSQL database, the data will automatically be stored and persisted, without a required schema.

MongoDb uses JSON to store documents. Any changes to the document format or the C# class will continue to operate as expected; ignoring new fields, and inserting new fields where applicable.

But, What About All That Wasted Disk Space?

A popular counter-argument to using a NoSQL MongoDb document database in place of an SQL relational database is the duplication of data, due to the flattened documents stored within. Where you would typically normalize the data to avoid replication and optimize every bit stored, NoSQL document databases take a different stance, often storing data within the cloud, where disk space and consumption is less of a restriction.

Database Storage in the Cloud

For our example C# .NET MongoDb application, we’ll be utilizing a free database, stored in the cloud, with MongoLab. Connection strings for MongoDb take the form of a url in the format mongodb://username:password@ds012345.mongolab.com:12345/yourdatabase Our web.config would define the connection string, as follows:

1	<add name="db" connectionString=" mongodb://username:password@ds012345.mongolab.com:12345/yourdatabase?strict=false"/>

Choosing a MongoDb Driver

As with any communication protocol and database platform, we’ll need to select a driver to use for connecting to our MongoDb database. Popular choices for C# .NET applications include the 10gen MongoDb driver and NoRM. Since we’ll be using LINQ exclusively for persisting data, we’ll be using the NoRM driver for accessing our data.

Dragon Breath

To begin our example C# .NET MongoDb implementation, we’ll create our C# .NET class library to consist of a Dragon type. To demonstrate the document format that NoSQL uses, we’ll include an additional embedded type of Breath, which will contain its own set of fields. A copy of Breath will be stored inside Dragon in the database, demonstrating the flattened data document. Also note, once these C# .NET types are persisted to the MongoDb database, the documents are automatically created - no schema required. We can define our types as follows:

public class Dragon
{
    public ObjectId Id { get; private set; }
    public string Name { get; set; }
    public int Age { get; set; }
    public string Description { get; set; }
    public int Gold { get; set; }
    public int MaxHP { get; set; }
    public int HP { get; set; }
    public Breath Weapon { get; set; }
    public DateTime DateBorn { get; set; }
    public DateTime? DateDied { get; set; }

    public Dragon()
    {
        DateBorn = DateTime.Now;
    }
}

public class Breath
{
    public enum BreathType
    {
        Fire,
        Ice,
        Lightning,
        PoisonGas,
        Darkness,
        Light
    };

    public string Name { get; set; }
    public string Description { get; set; }
    public BreathType Type { get; set; }
}

In the above code, we’ve simply defined two basic models for our Dragon’s data. This data will be persisted to the MongoDb database exactly as defined. Our unique identifier (defined as an ObjectId) can be considered as the primary key for our Dragon table. MongoDb will generate an Id upon saving.

To begin persisting the data to our MongoDb MongoLab database, we’ll need to setup a basic repository class.

Adding Some Enterprise to Your Repository

We could call the C# .NET MongoDb driver’s direct methods for reading and saving data. However, in this tutorial, we’ll create something a little more robust. We’ll implement a basic repository pattern for accessing the data, utilizing LINQ for queries. We’ll also provide a thread-safe global data context that can be used in a desktop application or web application for accessing the database.

We’ll implement the following interface for our repository:

public interface IRepository : IDisposable
{
    void Delete<T>(Expression<Func<T, bool>> expression) where T : class, new();
    void Delete<T>(T item) where T : class, new();
    void DeleteAll<T>() where T : class, new();
    T Single<T>(Expression<Func<T, bool>> expression) where T : class, new();
    System.Linq.IQueryable<T> All<T>() where T : class, new();
    System.Linq.IQueryable<T> All<T>(int page, int pageSize) where T : class, new();
    void Add<T>(T item) where T : class, new();
    void Add<T>(IEnumerable<T> items) where T : class, new();
}

The above interface contains basic methods for adding, deleting, and querying data. We can implement the repository interface for our MongoDb NoRM driver with the following concrete provider class:

public class MongoRepository : IRepository
{
    private IMongo _provider;
    private IMongoDatabase _db { get { return this._provider.Database; } }

    public MongoRepository()
    {
        _provider = Mongo.Create(ConfigurationManager.ConnectionStrings["db"].ConnectionString);
    }

    public void Delete<T>(System.Linq.Expressions.Expression<Func<T, bool>> expression) where T : class, new()
    {
        var items = All<T>().Where(expression);
        foreach (T item in items)
        {
            Delete(item);
        }
    }

    public void Delete<T>(T item) where T : class, new()
    {
        // Remove the object.
        _db.GetCollection<T>().Delete(item);
    }

    public void DeleteAll<T>() where T : class, new()
    {
        _db.DropCollection(typeof(T).Name);
    }

    public T Single<T>(System.Linq.Expressions.Expression<Func<T, bool>> expression) where T : class, new()
    {
        return All<T>().Where(expression).SingleOrDefault();
    }

    public IQueryable<T> All<T>() where T : class, new()
    {
        return _db.GetCollection<T>().AsQueryable();
    }

    public IQueryable<T> All<T>(int page, int pageSize) where T : class, new()
    {
        return PagingExtensions.Page(All<T>(), page, pageSize);
    }

    public void Add<T>(T item) where T : class, new()
    {
        _db.GetCollection<T>().Save(item);
    }

    public void Add<T>(IEnumerable<T> items) where T : class, new()
    {
        foreach (T item in items)
        {
            Add(item);
        }
    }

    public void Dispose()
    {
        _provider.Dispose();
    }
}

In the above concrete provider code for the MongoDb NoRM driver, we implement each interface method by calling LINQ compatible methods. Querying the database returns IQueryable interfaces, allowing us to refine the query before actually sending it to the database. MongoDb will take care of optimizing the call and executing the query, before returning the data back to our C# .NET application.

Not All Global Variables Are Evil

We can enhance our repository pattern with a global database context provider. This will allow us to access the database provider from any point within our C# .NET application and optimize the usage of the same application pool thread and database provider call. For web applications, we’ll store the provider context within the HttpContext model. For desktop and console applications (.exe), we’ll store the provider in a local HashTable, per thread. This will allow us to call the MongoDb database by using a call such as:

1	var dragons = DbContext.Current.All<Dragon>().ToList();

Our DbContext database context provider appears, as follows:

public static class DbContext
{
    private const string HTTPCONTEXTKEY = "Session.Base.HttpContext.Key";
    private static readonly Hashtable _threads = new Hashtable();

    /// <summary>
    /// Returns a database context or creates one if it doesn't exist.
    /// </summary>
    public static IRepository Current
    {
        get
        {
            return GetOrCreateSession();
        }
    }

    /// <summary>
    /// Returns true if a database context is open.
    /// </summary>
    public static bool IsOpen
    {
        get
        {
            IRepository session = GetSession();
            return (session != null);
        }
    }

    #region Private Helpers

    private static IRepository GetOrCreateSession()
    {
        IRepository session = GetSession();
        if (session == null)
        {
            session = ObjectFactory.GetInstance<IRepository>();

            SaveSession(session);
        }

        return session;
    }

    private static IRepository GetSession()
    {
        if (HttpContext.Current != null)
        {
            if (HttpContext.Current.Items.Contains(HTTPCONTEXTKEY))
            {
                return (IRepository)HttpContext.Current.Items[HTTPCONTEXTKEY];
            }

            return null;
        }
        else
        {
            Thread thread = Thread.CurrentThread;
            if (string.IsNullOrEmpty(thread.Name))
            {
                thread.Name = Guid.NewGuid().ToString();
                return null;
            }
            else
            {
                lock (_threads.SyncRoot)
                {
                    return (IRepository)_threads[Thread.CurrentThread.Name];
                }
            }
        }
    }

    private static void SaveSession(IRepository session)
    {
        if (HttpContext.Current != null)
        {
            HttpContext.Current.Items[HTTPCONTEXTKEY] = session;
        }
        else
        {
            lock (_threads.SyncRoot)
            {
                _threads[Thread.CurrentThread.Name] = session;
            }
        }
    }

    #endregion
}

Note in the above code, the property “Current” allows us to access our database provider to load or save data to MongoDb. This property, in turn, calls GetOrCreateSession() which will retrieve the currently active database connection or create a new one, if one does not exist.

You may notice our DbContext provider contains no reference to a concrete provider type. This allows us to easily swap in and out different MongoDb providers. To change providers, simply reference the associated DLLs and implement a new IRepository class for your concrete provider. Then specify which concrete provider to load.

Spice it Up With Some Dependency Injection

The choice of concrete provider in our database context provider is performed in the call to ObjectFactory.GetInstance(). We’re actually using StructureMap as our dependency injection tool. This lets us specify the concrete class to load, for our repository, in our main program. Creating new drivers becomes an easy task.

Our main program will include a simple Setup class, which initializes StructureMap to tell it which concrete provider class to load upon startup. In our case, this will be our MongoRepository class, which uses the NoRM MongoDb driver.

internal static class Setup
{
    /// <summary>
    /// Initializes StructureMap (dependency injector) to setup our database provider.
    /// </summary>
    public static void Initialize()
    {
        // Initialize our concrete database provider type.
        ObjectFactory.Initialize(x => { x.For<IRepository>().Use<MongoRepository>(); });
    }

    /// <summary>
    /// Disposes the database provider context.
    /// </summary>
    public static void Close()
    {
        if (DbContext.IsOpen)
        {
            DbContext.Current.Dispose();
        }
    }
}

When using in a C# ASP .NET web application, you’ll want to dispose of the MongoDb database connection at the end of each request. You can do this by editing your Global.asax.cs and overriding protected void Application_EndRequest(object sender, EventArgs e) to call the Setup.Close() method, as shown above.

Creating our Business Logic Layer

We’ve finished implementing our database tier layer, which consisted of our MongoDb database provider, a repository pattern, and our db context. We can now implement the mid-tier (2nd-tier) which will contain our business logic for accessing the database. All calls to the MongoDb database will go through our business logic layer first.

We can create a simple DragonManager class for accessing the Dragon data, as follows:

public static List<Dragon> GetAll()
{
    return DbContext.Current.All<Dragon>().OrderBy(d => d.Name).ToList();
}

public static List<Dragon> Find(string keyword)
{
    List<Dragon> dragons = null;

    if (keyword.Length > 0)
    {
        dragons = DbContext.Current.All<Dragon>().Where(d =>
        d.Name.ToLower().Contains(keyword.ToLower())).OrderBy(d => d.Name).ToList();
    }
    else
    {
        dragons = GetAll();
    }

    return dragons;
}

public static void Save(Dragon dragon)
{
    DbContext.Current.Add(dragon);
}

public static void Delete(Dragon dragon)
{
    DbContext.Current.Delete<Dragon>(d => d.Id == dragon.Id);
}

Note, in the above code, we’ve wrapped calls to our C# .NET MongoDb database provider to encapsulate business logic. We’ve provided a method for retrieving all Dragons from the database. We’ve also provided a method for searching the dragons by keyword. The Save and Delete methods are also included.

Here Come The Dragons

With our C# .NET MongoDb repository complete, we can now access our database. First, we’ll create some Dragons with the following code:

Dragon dragon = new Dragon();
dragon.Name = "Scary Green Guy";
dragon.Age = 99;
dragon.Description = "A big dragon.";
dragon.Gold = 1000000;
dragon.Weapon = new Breath { Name = "Breath", Description = "A breath attack.",
 Type = Breath.BreathType.Fire };
dragon.MaxHP = 60;
dragon.HP = 60;

We can then save the database directly to our MongoDb database, without even creating a schema, with the following code:

1	DragonManager.Save(dragon);

We can query the dragons with code similar to the following simple C# .NET MongoDb console program:

static void Main(string[] args)
{
    string keyword = "";

    // Initialize our database provider.
    Setup.Initialize();

    while (true)
    {
        // Search for dragons.
        List<Dragon> dragons = DragonManager.Find(keyword);

        // Display the dragons.
        DisplayDragons(dragons);

        // Get input from the user.
        Console.Write("Enter text to search by or Q to quit:>");
        keyword = Console.ReadLine();

        // Check the input.
        if (keyword.ToUpper() == "Q")
            break;
    }

    Setup.Close();
}

Note, the above code first initializes our dependency injector to specify the concrete MongoDb driver to use, which in our case is our MongoRepository class (using NoRM). We then call our business logic tier’s DragonManager.Find() method to retrieve a list of dragons.

Peeking Under the Covers of MongoDb

For a sneak peak at what happens when you issue a query to MongoDb, we can look in the NoSQL profiler and record a query command. We can see the following JSON command passed to the MongoDb server:

{
  "ts" : new Date("Thu, 01 Jan 2012 01:01:01 GMT -01:00"),
  "op" : "query",
  "ns" : "dragons.Dragon",
  "query" : {
    "_id" : ObjectId("9ea7c100d38281e40a000000")
  },
  "ntoreturn" : 1,
  "idhack" : true,
  "responseLength" : 259,
  "millis" : 0,
  "client" : "1.1.1.1",
  "user" : ""
}

Note in the above JSON packet, the query includes an ObjectId, representing the primary key (unique identifier) for our Dragon. Since unique id’s are generated automatically, we can ensure compatibility across database servers in the cloud if sharding is used to scale the database implementation. This is in contrast to using auto-incremented identifiers, which could result in conflicts when sharding across multiple database servers.

LINQ queries to manipulate the data are also executed on the MongoDb database server, in a similar fashion to the above JSON query packet. The data is then returned to the C# .NET application layer.

Taking a look in our MongoLab administrator panel, we can view the contents of a Dragon record. Our Dragon is also stored in JSON format, as the following record demonstrates:

{
    "_id": {
        "$oid": "3d0b95030ad4cdc00e000000"
    },
    "Name": "Black Serpent",
    "Age": 99,
    "Description": "A big dragon.",
    "Gold": 781,
    "MaxHP": 11,
    "HP": 11,
    "Weapon": {
        "Name": "Breath",
        "Description": "A breath attack.",
        "Type": 2
    },
    "DateBorn": {
        "$date": "2012-01-09T16:41:39.376Z"
    },
    "DateDied": null
}

Notice how the Weapon (our Breath class) is embedded in the record. A single call for a Dragon can return its Breath type as well. Of course, you can filter the results and select individual columns to return. This can allow you to limit the bandwidth consumed per request.

Here is another example MongoDb record, this time filtering results by searching for records containing the Name “Dark” within it:

{
  "ts" : new Date("Thu, 01 Jan 2012 01:01:01 GMT -01:00"),
  "op" : "query",
  "ns" : "dragons.Dragon",
  "query" : {
    "query" : {
      "Name" : /Dark/
    },
    "orderby" : { }
  },
  "ntoreturn" : 2147483647,
  "nscanned" : 15,
  "nreturned" : 2,
  "responseLength" : 494,
  "millis" : 0,
  "client" : "1.1.1.1",
  "user" : "dragon"
}

Running Our Application

Our application produces the following results after calling our C# .NET MongoDb repository:

 Id | Name              | Age | Gold |  HP |     Breath |     Born | Realm
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
  1 | Black Cheating    |  97 |   19 |  18 | Light      | 1/9/2012 | Hell
  2 | Black Chimaera    |  18 |  407 |  15 | Lightning  | 1/9/2012 | Hell
  3 | Black Serpent     |  99 |  781 |  11 | Lightning  | 1/9/2012 | Love
  4 | Black Sneaky      |  13 |  677 |  13 | Fire       | 1/9/2012 | Love
  5 | Cunning Chimaera  |  93 |  271 |  18 | Lightning  | 1/9/2012 | Hatred
  6 | Dark Cheating     |  12 |  150 |  11 | Ice        | 1/9/2012 | Love
  7 | Evil Chimaera     |  62 |  833 |  20 | Lightning  | 1/9/2012 | Love
  8 | Evil Stealth      |  76 |  299 |  10 | Darkness   | 1/9/2012 | Love
  9 | Golden Hippogryph |  80 |  693 |  10 | PoisonGas  | 1/9/2012 | Love
 10 | Golden Spirit     |  84 |  621 |  10 | PoisonGas  | 1/9/2012 | Abyss
 11 | Light Spirit      |  66 | 1000 |  20 | Lightning  | 1/9/2012 | Abyss
 12 | Magic Hippogryph  |  95 |   62 |  19 | Lightning  | 1/9/2012 | Love
 13 | Magic Stealth     |  47 |  123 |  18 | Light      | 1/9/2012 | Hell
 14 | Magic Stealth     |  84 |   50 |  19 | PoisonGas  | 1/9/2012 | Abyss
 15 | Silver Legendary  |  42 |  745 |  14 | Fire       | 1/9/2012 | Hell
 16 | Slimy Chimaera    |  92 |  285 |  14 | Fire       | 1/9/2012 | Abyss
 17 | Slimy Sneaky      |  99 |  240 |  14 | Ice        | 1/9/2012 | Love
 18 | White Hippogryph  |  19 |  456 |  11 | Light      | 1/9/2012 | Hell
 19 | White Skeleton    |  24 |    1 |  12 | Ice        | 1/9/2012 | Hell
 20 | White Stealth     |  80 |  197 |  15 | Lightning  | 1/9/2012 | Hell

Enter text to search by or Q to quit:>magic

 Id | Name              | Age | Gold |  HP |     Breath |     Born | Realm
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
  1 | Magic Hippogryph  |  95 |   62 |  19 | Lightning  | 1/9/2012 | Love
  2 | Magic Stealth     |  47 |  123 |  18 | Light      | 1/9/2012 | Hell
  3 | Magic Stealth     |  84 |   50 |  19 | PoisonGas  | 1/9/2012 | Abyss

Enter text to search by or Q to quit:>

Relational Data in a Non-Relational World

Up until now, we’ve implemented a strict NoSQL document-style flat record for a Dragon and its Breath. However, you can also implement relational data in the non-relation MongoDb database. We can do this by including foreign key references to serve as links, connecting tables. For example, to add a Realm for our dragons to inherit from, we could create a new Realm data type, as follows:

public class Realm
{
    public enum RegionType
    {
        Mist,
        Love,
        Abyss,
        Hatred,
        Hell
    };

    public ObjectId Id { get; private set; }
    public string Name { get; set; }
    public RegionType Region { get; set; }

    public Realm()
    {
    }

    public Realm(RegionType region)
    {
        Name = region.ToString();
        Region = region;
    }
}

So far, nothing has changed from our Dragon implementation. We’ve created a simple type, Realm. It contains a primary key of type ObjectId (a unique generated id from MongoDb).

We can add the Realm to our Dragon in a one-to-many relationship by including a reference to the Realm foreign key, as follows:

public class Dragon
{
    public ObjectId Id { get; private set; }
    public ObjectId RealmId { get; set; }

    public string Name { get; set; }
    public int Age { get; set; }
    public string Description { get; set; }

    ...
}

Our foreign key is simple a copy of the Realm table’s primary identifier. Of course, we’ll need to track the Realm object and Id property, which we can do with the help of a lazy-load property.

Lazy Loading a Foreign Key in MongoDb

We can add the following property to the Dragon class to include a lazy-load property as a foreign key reference:

private Realm _realm;
[MongoIgnore]
public Realm Realm
{
    get
    {
        // Lazy-load.
        if (_realm == null)
        {
            _realm = DbContext.Current.Single<Realm>(r => r.Id == RealmId);
        }

        return _realm;
    }
    set
    {
        RealmId = value.Id;
        _realm = value;
    }
}

Note, since we’ll be maintaining this property ourselves, within the code, we include a MongoIgnore attribute. This tells MongoDb to ignore persisting the Realm property within the Dragon document. Instead, we’ll simply link to our Realm record, rather than including a copy of it within. When we’re reading to access the Realm property, we’ll automatically fetch the Realm data, using its primary key (which is stored on the Dragon document instead of the Realm data). If we’ve already fetched the Realm, we just return it.

A Simple Manager Class for a Static List

Since our dragon’s Realm object is just a static list of data, similar to the contents of a drop-down selector, we can create a business logic manager class to retrieve the data, as follows:

public static class RealmManager
{
    public static Realm GetByRegion(Realm.RegionType region)
    {
        return DbContext.Current.Single<Realm>(r => r.Region == region);
    }

    public static void Save(Realm realm)
    {
        DbContext.Current.Add(realm);
    }

    #region Helpers

    public static Realm CreateRandom()
    {            
        Realm.RegionType region = (Realm.RegionType)HelperManager.RandomGenerator.Next(1, 5);

        // Load the realm.
        Realm realm = GetByRegion(region);
        if (realm == null)
        {
            // Create the realm if it doesn't exist.
            realm = new Realm(region);
            Save(realm);
        }

        return realm;
    }

    #endregion
}

Note, in our CreateRandom() method, when creating a new Realm (or any item from a static list), we first check if it exists by trying to load it. If it does not yet exist, we create the new entry and return it.

Under the Covers of a Relational Record in MongoDb

So what does our new Dragon record look like, with its new relational link to Realm?

{
    "_id": {
        "$oid": "be0b95030ad4cdc00e090000"
    },
    "RealmId": {
        "$oid": "ac1b94030ad4cd0c1b060000"
    },
    "Name": "Golden Spirit",
    "Age": 84,
    "Description": "A big dragon.",
    "Gold": 621,
    "MaxHP": 10,
    "HP": 10,
    "Weapon": {
        "Name": "Breath",
        "Description": "A breath attack.",
        "Type": 3
    },
    "DateBorn": {
        "$date": "2012-01-01T01:01:01.001Z"
    },
    "DateDied": null
}

Note the new property “RealmId” included in our document. This is a relational reference to another MongoDb document. Looking in our MongoLab administrative area, we can find the associated Realm document, defined as follows:

{
    "_id": {
        "$oid": "ac1b94030ad4cd0c1b060000"
    },
    "Name": "Abyss",
    "Region": 2
}

See It In Action

Download @ GitHub

You can download the project source code on GitHub by visiting the project home page.

Perfect MVC code

Friday, 20 May 2016

MongoDb Enterprise Design with C# .NET, the Repository Pattern, and NoSQL