Introduction

This article is an overview about my database project LiteDB - a small, fast and free embedded .NET NoSQL Document Store for .NET in a single datafile - and now it's in new version 4.0 .

More examples and online shell at LiteDB.org

Source code hosted on GitHub or Download Binary

Full documentation in GitHub Wiki Pages

Features

Serverless NoSQL Document Store

Simple API very similar MongoDB Official Driver

100% C# code for .NET 3.5/4.0 in a single DLL (less than 300KB) - No dependencies

Support for portable platforms: Full .NET and NET Standard 1.3+2.0 (for UWP 10, Xamarin iOS and Android)

Thread-safe / Multi-processes

Recovery in writing failure (journal mode)

Support for POCO classes or BsonDocument

Encryption datafile using DES cryptography

FileStorage for files and stream data (like GridFS in MongoDB)

Single data file storage (like SQLite)

Indexed document fields for fast search (up to 16 indexes per collection)

LINQ support for queries with Include support

Shell command line ( try this online version )

Open source and free for everyone - including commercial use

Install from NuGet: Install-Package LiteDB

What's New

New JSON Path support

New expression support on index definition. You can index any field of your document

Map your entity class using Attributes or Fluent API

LiteRepository class for simple and easy access

DbRef for cross document reference

LiteDB has a big inspiration on MongoDB. I tried to create a database that works like MongoDB, but on a very small scale, using only most important features for small applications. If you know MongoDB, you already know LiteDB.

How to Use

A quick example for store and search documents:

// Open data file (or create if not exits) using ( var db = new LiteDatabase( @" C:\Temp\MyData.db" )) // Get customer collection var col = db.GetCollection<Customer>( " customers" ); var customer = new Customer { Id = 1 , Name = " John Doe" }; // Insert new customer document col.Insert(customer); // Update a document inside a collection customer.Name = " Joana Doe" ; col.Update(customer); // Index document using a document property col.EnsureIndex(x => x.Name); // Simple Linq support var result = col.Find(x => x.Name.StartsWith( " Jo" ));

Where to Use?

Small web applications, sites blogs or forums

One datafile per account/user data store

Desktop/local small applications

Application file format

Few concurrency write users operations

Quick Starts

Let´s see some concepts about LiteDB database and data structure. If you need more documents, you can read Github Wiki documentation .

Documents

LiteDB works with documents to store and retrieve data inside data file. Your document definition can be a POCO class or BsonDocument class. In both case, LiteDB will convert your document in a BSON format to store inside disk.

BSON is a Binary JSON, a serialization for store data objects as binary array. In BSON, we have more data types than JSON, like DateTime , Guid and ObjectId .

Documents using POCO Class

POCO class are simple C# classes using only get / set properties. It's the best way to create a strong typed documents. Your class must have an identifier property. You can use Id named property, <ClassName>Id or decorate any property with [BsonId] attribute.

// A poco entity, must have Id public class Customer public int Id { get ; set ; } public string Name { get ; set ; } public List<Phone> Phones { get ; set ; } // It's not a entity, don't need Id public class Phone public int Code { get ; set ; } public string Number { get ; set ; } public PhoneType Type { get ; set ; } public enum PhoneType { Mobile, Landline }

Internal, document id is represent as _id property

Do not use complex data types (like DataSet , DataTable )

Do not use disposable objects (like Stream , Graphics )

Enum s will be converted in string s when serialized

Your Id value must be a unique and not null

If you leave Id empty, LiteDB will auto generate on insert .

Mapper Conventions

BsonMapper.ToDocument() auto convert each property class in a document field following this convention:

Classes must be public with a public non-parameter constructor

Properties must be public

Properties can be read-only or read/write

Class must have an Id property, <ClassName>Id property or any property with [BsonId] attribute to store Id information

A property can be decorated with [BsonIgnore] to not be mapped to document

A property can be decorated with [BsonField] to have a custom name field

No circular references allowed

Max of 20 depth inner classes

Class fields are not converted to document

BsonMapper use a global instance that cache mapping information for a better performance. This instance is on LiteDatabase.Mapper property too.

Fluent API for Document Mapping

If you want to change default conventions mapping, you can use EntityBuilder to map your entity class to BsonDocument .

var mapper = BsonMapper.Global; mapper.Entity<Customer>() .Id(x => x.CustomerKey) .Field(x => x.Name, " customer_name" ) .Ignore(x => x.Age) .Index(x => x.Name, unique);

DbRef for Cross Document Collection Reference

LiteDB can use cross document references to map an inner entity using only _id value (not embedding whole sub document).

// Your entity order class public class Order public int OrderId { get ; set ; } public Customer Customer { get ; set ; } // Map Customer property to "customers" collections BsonMapper.Global.Entity<Order>() .DbRef(x => x.Customer, " customers" ); // When insert an order, only customer _id will be included as reference // Get orders collection var orders = db.GetCollection<Order>( " orders" ); // Find an order including Customer reference var order = orders .Include(x => x.Customer) .FindById( 123 );

LiteDB will serialize Order document as ` { _id: 123, Customer: { $id:1, $ref: "customers" } }` . When you need query Order including Customer instance, just add before run Find method.

Documents using BsonDocument

BsonDocument is a special class that maps any document with a internal Dictionary<string, object> . It is very useful to read a unknown document type or use as a generic document.

// Create a BsonDocument for Customer with phones var doc = new BsonDocument(); doc[ " _id" ] = ObjectId.NewObjectId(); doc[ " Name" ] = " John Doe" ; doc[ " Phones" ] = new BsonArray(); doc[ " Phones" ].Add( new BsonObject()); doc[ " Phones" ][0][ " Code" ] = 55 ; doc[ " Phones" ][0][ " Number" ] = " (51) 8000-1234" ; doc[ " Phones" ][0][ " Type" ] = " Mobile" ;

With BsonDocument , you can create any complex document schema.

Collections - The Store

LiteDB organize documents in stores (called in LiteDB as collections). Each collection has a unique name and contains documents with same schema/type. You can get a strong typed collection or a generic BsonDocument collections, using GetCollection from LiteDatabase instance.

var db = new LiteDatabase(stringConnection); // Get a strong typed collection var customers = db.GetCollection<Customer>( " Customers" ); // Get a BsonDocument collection var customers = db.GetCollection( " Customers" );

Collection contains all methods to manipulate documents:

Insert - Insert a new document

FindById , FindOne or Find - Find a document using Query object or LINQ expression. At this point, only simple LINQ are supported - attribute on left, value on right side.

Update - Update a document

Delete - Delete a document id or using a query

Include - Use include to populate properties based on others collections

EnsureIndex - Create a index if not exists. All queries must have a index.

Query

In LiteDB , queries use indexes to find documents. You can use Query helper or Linq expressions.

var customers = db.GetCollection<Customer>( " customers" ); // Create a new index (if not exists) customers.EnsureIndex( " Name" ); // Query documents using 'Name' index var results = customers.Find(Query.StartsWith( " Name" , " John" )); // Or using Linq var results = customers.Find(x => x.Name.StartsWith( " John" )); // Return document by ID (PK index) var customer = customers.FindById( 1 ); // Count documents using Query helper var count = customers.Count(Query.GTE( " Age" , 22 )); // All query results returns an IEnumerable<T>, so you can use Linq in results var results = customers .Find(x => x.Name.StartsWith( " John" ) && x.Salary > 500 ) // two indexed queries .Where(x => x.LastName.Length > 5 ) // in memory query .Select(x => new { x.Name, x.Salary }) .OrderBy(x => x.Name);

Query class supports All , Equals , Not , GreaterThan , LessThan, Between , In , StartsWtih, Contains, AND and OR .

Fail Tolerance - Journaling/Recovery

LiteDB uses on-disk journal file to guarantee write operation durability. This mean that before a write operation, all dirty pages will be first written in same file before write on main datafile (use end of file to store clear pages).

This feature can be disabled on connection string - runs faster but if has a crash on write operation your datafile can be inconsistent.

Working with Files - FileStorage

At the same time, we need store files in database. For this, LiteDB has a special FileStorage collection to store files without document size limit (file limit is 2Gb per file). It works like MongoDB GridFS .

// Storing a file stream inside database db.FileStorage.Upload( " my_key.png" , stream); // Get file reference using file id var file = db.FileStorage.FindById( " my_key.png" ); // Find all files using StartsWith var files = db.Files.Find( " my_" ); // Get file stream var stream = file.OpenRead(); // Write file stream in a external stream db.FileStorage.Download( " my_key.png" , stream);

LiteDB creates two collections to handle files: _files and _chunks. The collection _files contains file information (file id, filename, upload date and metadata). File data content is split in _chunks collection.

LiteDB.Shell

LiteDB contains a shell console application included. This shell can be used to run commands in datafile without any other application. See the most useful commands below. To see all commands, use " help " command.

Shell commands ============== > open <filename|connectionString> : Open a new database > pretty on|off : Turns on/off pretty json format > timer : Show timer before prompt > dump > <dumpfile.dmp> : Export database as insert commands > dump < <dumpfile.dmp> : Import database Collections commands ==================== > db.<collection>.insert <jsonDoc> : Insert a new document into collection > db.<collection>.update <jsonDoc> : Update a document inside collection > db.<collection>.delete <filter> : Delete documents using a filter clausule (see find) > db.<collection>.find [top N] <filter> : Show filtered documents based on index search > db.<collection>.count <filter> : Show count rows according query filter > db.<collection>.ensureIndex <field> [unique] : Create a new index document field > db.<collection>.drop : Drop collection and destroy all documents inside <filter> = <field> [=|>|>=|<|<=|!=|like|contains|between] <jsonValue> : Filter query syntax <filter> = (<filter> [and|or] <filter> [and|or] ...) : Multi queries syntax <jsonDoc> = {_id: ... , key: value, key1: value1 } : Represent an extended json for a BsonDocument. File storage commands ===================== > fs.find : List all files on datafile > fs.find <fileId> : List file info from a key. Supports * for starts with key > fs.upload <fileId> <filename> : Insert a new file inside database > fs.download <fileId> <filename> : Save a file to disk passing a file key and filename > fs.update <fileId> {key:value} : Update metadata file > fs.delete <fileId> : Remove a file inside database

History

Update v4.0.0 - 18 ^th October, 2017

.NET4 only

New Expression support

Fix concurrency problems from v3

Portable support using NETStandard 1.3 + 2.0

Update v2.0.0 - 1 ^st August, 2016

Support for Portable - UWP, Xamarin - iOS and Android

.NET 3.5

New disk access avoiding lock functions: use only read/write exclusive mode

Add some v1 features back: user transaction control, register autoId and global mapper instance

Update v.2.0.0-rc - 24 ^th December, 2015

Encryption datafile

Datafile migration from v0.9.0 / v1.0.x

Dump datafile as insert

Better cache/concurrency support

Merry Xmas :)

Update v.2.0.0-beta - 27 ^th November, 2015

Abstract persist layer

Fluent mapper API

new DbRef

Virtual index field

New cleanup cache system

Support for initial database size and max database size

Lazy load

ThreadSafe and ProcessSafe

Shrink datafile

Database log information

Update v1.0.2 - 17 ^th May, 2015

Better BsonMapper serialize/deserialize for interfaces and base classes using _type field in document (as used in MongoDB)

BsonMapper for NameValueCollection

Added support to boolean Linq operations, like x => x.IsActive

Bugfix in string index update / delete operations

Removed full scan document find operations - works now only with index

Autocreate index if not exists

Update v1.0 - 28 ^th March, 2015

New BsonSerializer removing fastBinaryJson and implement real BSON specification

New BsonMapper to get more configurable POCO from/to BsonDocument

New JsonReader implementation: no regex and 4 times faster

New ObjectId to be used in Id documents

Index creation options - remove whitespaces, remove accents, ignore case

[BsonIndex] attribute to mark your entity property to auto create index when query

Autogeneration Id for entity Id property

Find() can be executed without an index (will execute a full document scan)

FindAll() supports ascending/descending results

New Query.Contains

Min() / Max() value from an index

DbRef<> - a simple class for reference document

Drop collection and drop index improved

Removed _master collection - avoid 1 page read

Update v0.9 - 7 ^th February, 2015

Add MongoDB benchmark

Add shell quick tour

Initial version - 25 ^th January, 2015

I have a problem while saving List of Objects in Lite Db. How to save Orders, PaymentDetails and then Sale. public class Sale [BsonId] public int SaleId { get ; set ; } public decimal SubTotal { get ; set ; } public decimal Tax { get ; set ; } public decimal Handling { get ; set ; } public decimal Discount { get ; set ; } public decimal Total { get ; set ; } public decimal Paid { get ; set ; } public decimal Balance { get ; set ; } public string PaymentStatus { get ; set ; } public DateTime SaleDate { get ; set ; } [BsonRef( " Customer" )] public Customer Customer { get ; set ; } [BsonRef( " PaymentDetail" )] public List<paymentdetail> PaymentDetails { get ; set ; } [BsonRef( " Order" )] public List<order> Orders { get ; set ; } public class Order [BsonId] public int OrderId { get ; set ; } [BsonRef( " Product" )] public Product Product { get ; set ; } public string Item_Qty { get ; set ; } public string Item_Description { get ; set ; } public string Item_Price { get ; set ; } public string Item_Discount { get ; set ; } public string Item_Extend { get ; set ; } public string Item_Tax { get ; set ; } public string Item_Discount { get ; set ; } public string Item_Discount_Type { get ; set ; } public class Product [BsonId] public int ProductId { get ; set ; } public string ItemNumber { get ; set ; } public string UPC { get ; set ; } public string Description { get ; set ; } public string LongDescription { get ; set ; } public string Price1 { get ; set ; } public string Price2 { get ; set ; } public string Price3 { get ; set ; } public string Price4 { get ; set ; } public string Price5 { get ; set ; } public string KitchenLable { get ; set ; } public string Weight { get ; set ; } public string Stock { get ; set ; } public string Menufacture { get ; set ; } public string TaxPer { get ; set ; } public bool BarItem { get ; set ; } public string ImageFileName { get ; set ; } public byte [] ProductImage { get ; set ; } public bool IsActive { get ; set ; } public string BackColor { get ; set ; } public string FontColor { get ; set ; } public int FontSize { get ; set ; } public class Customer [BsonId] public int CustomerId { get ; set ; } public string Name { get ; set ; } public string Company { get ; set ; } public string Address { get ; set ; } public string City { get ; set ; } public string Street { get ; set ; } public string Code { get ; set ; } public string Phone { get ; set ; } public string Email { get ; set ; } public string TaxNumber { get ; set ; } public DateTime LicDate { get ; set ; } public string Contact { get ; set ; } public string BillingAddress { get ; set ; } public int UserStamp { get ; set ; } public DateTime DateTimeStamp { get ; set ; } }


Sign In · View Thread 5 stars for your great effort. But how to use it in MVC .NET core web application?

Member 14157083 5-Apr-19 5:21

Member 14157083 5-Apr-19 5:21 I have a problem. I have used MongoDB seamlessly in .NET core 2.2 (as microsoft documents provide an example for MongoDB). But I don't know how to use LiteDB in .NET core MVC web application seamlessly.
I am new to .NET and visual studio, and entity framework provider or connection string in configuration files is not clear for LiteDB.
Have you plans for such seamless support?
Sign In · View Thread As Unity's main programming language is C# I was wondering if I could you use LiteDB with my game. For what I've seen it is supposed to work as an ORM. So can it do it?
Greetings and great job. Can you publish your twitter account please?
Sign In · View Thread I have built your source code out of the box using .Net 4.6 and run the LiteDB.Demo in Debug mode. It generates a steady stream of System.IO.IOException errors and finally gives an unhandled exception of type System.AggregateException in mscorlib.dll. This is using Visual Studio Community 2017 Version 15.4.1. Any suggestions would be welcome. Thanks

Nigel Tucker

Sign In · View Thread

Hi Nigel, can open an issues in github about this? Current LiteDB.Demo are compiled in .net 4.0, but i changed to .net46 and works fine here. I'm using visual studio 2017 15.3.0

Mauricio David - http://www.numeria.com.br

Sign In · View Thread Hi i want store about 10 millions records and perform frequently database read and write operations, is LiteDB best suite for this?
Sign In · View Thread var customer = new Customer { Id = 1 , Name = " John Doe" };
For some reason, Visual Studio won't recognize "Customer". Am I missing some code?
Sign In · View Thread Have you introduced Customer class?
You have to introduce that class first. I am quite sure that you have already figured this out but here:
public class Customer{
public int Id { get;set; }
public string Name { get; set; }
Sign In · View Thread var results = liteCollection.Find(Query.And(Query.All( " m_strTax" , Query.Ascending), Query.All( " m_strDate" , Query.Ascending))); Sign In · View Thread I have a requirement to allow end user to perform dynamic query at run time. So this means I can't use Linq for that particular function. How would you pass in a string where statement?
Sign In · View Thread

I'm not sure how much experience you have, but I've been looking at this (sort of thing) myself recently - there seems to be a number of thoughts along the lines of 'dynamic LINQ expression from string'
a) c# - How to convert a String to its equivalent LINQ Expression Tree? - Stack Overflow
b) Building LINQ Queries at Runtime in C# - Tomas Petricek
c) Building Linq Expressions Dynamically
I havnt tried any of these - for the Article/Blog I'm playing with, I found it suitable for a Func<string, bool=""> Lambda expression to be passed into a 'function', with a default of 'null', and using that 'Lambda' in the .Where() clause.
So what you want IS possible, it may take a bit of work though
Sign In · View Thread An exception of type ' LiteDB.LiteException' occurred in Crawler.dll but was not handled in user code Additional information: Timeout. Database is locked for more than 00 : 00 : 00 .
The offending code is:
Does LiteDB not allow parallel inserts? If it does, how do I accomplish this? I'm using the latest version.
Sign In · View Thread While 2GB may seem like a lot, it really isn't, not when you're talking about storing some data from an Enterprise CRM. Some files can be as big as 10 or 30GB. It would be really nice if there wasn't a hard 2GB limit. Please let me know if this can be achieved.
I realize I can just save a file to disk and reference it, but I'd like to make everything self contained in a single database file for portability.
Sign In · View Thread I am working on a project that will require a small database, say 50 tables with a few thousand rows each. I started in 2014 and all I could find was Sqlite. It is ok, I have plenty of experience with SQL, but it really was massive overkill and overhead.
I shelved the project for a time and when I came back to it, unsatisfied with the amount of extra work SQL was engendering, I took another Google and found LiteDb.
This is EXACTLY what I was looking for. I bit the bullet and converted all my code to use LiteDb and I have not only not regretted it, it has been a godsend.
As yet, I have not encountered a single issue using it. Granted, I'm doing some pretty basic stuff, CRUD to a small db. Documentation is a bit scarce, but I've never taken very long to figure out how to do something. I supposed those who are already familiar with Mongo have an easier time but I am not complaining about that, the usage is mostly so straightforward not a lot of doc is required.
Thanks to the author for putting something so useful and appropriate out there!
Sign In · View Thread And 5 star is goes to....LiteDB. You are the one Mauricio David. I explore 50-100 nosql solutions and select yours. Because your solution/library is so simple to understand your code very clear also robust enough to achieve many situations about storing data to a single, nosql and serverless file.
Praise is enough and time to criticism Documentation not enough but also not bad too. But samples absolutely not enough. My opinion at least you have to write one sample for one complex scenario.
And i'have five questions for you.
1. I have a simple entity that have int id and string name. I want to use fluent api and point id is primary index and name is unique index. Because of that i write below code;
var connectionString = Path.Combine(directoryService.DataDirectory, $ " Data.{WellKnownExtensionNames.DataExtensionName}" ); _unityContainer.RegisterType<IDatabase, Database>( new ContainerControlledLifetimeManager(), new InjectionConstructor(connectionString, bsonMapper));
After registration at some point i write/read my currencies collection (i didn't share these codes because i'm using many patterns like entity, repository, unitofwork etc. and post will be so dirty) and don't receiving any error and see my data.db is created. I used shell to check every currency is write correctly. I see currencies collection is created, inserted data is there also but if i check indexes via db.currencies.indexes i see there is Name index but not unique index. So maybe not today but sooner or later i'll receive an exception. Please help me about that why my index not creating unique like i indicate my mapper.
PS: Also i try to use BsonMapper.Global too but nothing changed.
2. What are you thinking about time series databases. Are you have any plan to write time series version of LiteDB (LiteTSDB)? What is your opinion about that this type databases really useful for especially for financial applications?
3. If my entities have special properties like TimeZonInfo, how can i read/write this properties from/to storage. I have to write converters for this type properties? If so how, please write simple example about it.
4. When subject is data, relations are Inevitable. So could you write example about all relation types, one to one, one to many, many to many.
5. I don't understand usage of EnsureIndex. If i use mapper and specify indexes why i have to call it? Why i have to call it after every insert a record? Am i have to call it after every update & delete operations too? Sometimes you call it after call GetCollection<customer>("Customers"). I'm so confused about usage of it. Please give me a detailed info about it.
Sorry for too long post. Again many thanks for all your effort for develop this incredible library. God bless you!

modified 30-Jan-17 13:21pm.

Sign In · View Thread I think that you have to call EnsureIndex only when database is opened. I think it is not needed for every insert.
It just ensures that index exists and if not then it creates it. Nothing else.
But then again i am not 100% sure
Sign In · View Thread Hello Mauricio
Hope you are doing well. Is there an Admin tool to open a database file and view what collections there are what files are stored?
Regards,
Simon
Sign In · View Thread

Looks like there is: GitHub - falahati/LiteDBViewer: LiteDB Viewer is a simple tool that let you open a LiteDB database file and view its contents [ ^ ]
Not sure if it's everything you're looking for, but it may be a good start.
Sign In · View Thread Mauricio, atuamente temos um ERP para pequenos e medios varejistas. atualmente utilizamos MYSQL. porem estamos analisando algumas solucoes para banco de dados embarcado/servless. Temos algumas tabelas de movimentacao com 500.000 a 2.000.000 de registros que vao se acumulando durante alguns anos...gostaria de saber se o LiteDB funciona bem com relacao a essa quantidade de dados.

==============================
Sign In · View Thread

Olá Alexandre, em relação a quantidade de dados não vejo problema pois o LiteDB tem suporte a vários milhões de registros. Não vejo tamanho como problema. A questão maior teria a ver com concorrência de usuários, se a aplicação é desktop ou web, enfim, fatores que podem definir se utilizar um banco de dados embedded valem a pena. Em soluções grandes geralmente se opta por um banco de dados centralizado, que funcione como serviço.
Se for te fazer alguma recomendação seria para: modele uma estrutura de dados que não seja fixa em um banco de dados. Se a aplicação necessitar uma demanda maior teria como, com certa facilidade, migrar para o MongoDB, por exemplo, pois o LiteDB tem uma API/forma de trabalhar muito proxima do MongoDB.

Mauricio David - http://www.numeria.com.br

Sign In · View Thread Web02 2.8:2023-09-29:1