Try fast search NHibernate

24 August 2009

NHibernate battle

Sorry but I can’t write this in english.

Bueno el guerrero, como se dice en la ciudad porteña, “arrugó”.

Miren este lindo post y las dos páginas de comentarios. A parte el ataco personal a Oren (aka Ayende) está muy claro que el sitio nació con el intento de publicizar un producto comercial.

Mis preguntas fueron:

You said "We've shown our product wins almost all the competitors on CUD tests", which is "your product" ?
Can you confirm that the code of the test of "your product" represent the best optimization you can do ?

A nuestro aspirante guerrero, y comercial promoter, le costó un poco mandarme el link del producto comercial y a la segunda pregunta contestó de la siguiente manera:

No, obviously, I can't confirm this. It will be much better this month. On the other hand, all we did was enough to show there is a lot of room for performance optimizations.

En Buenos Aires eso se nombra : “arrugar”.

En otro blog el chico pide que quien le diga que su test de performance está muy cercano a un MDTD (Monkey Driven Test Development), y por lo tanto no significativo, me pide que nosotros seamos quien provea un test que represente la realidad, lo cual es muy grave. ¿Estamos hablando de alguien que pretende publicitar un producto comercial sin tener idea de como crear un test de performance usando casos reales? ¿Que tendría que esperarme de ese producto?

Mirando parte del test me quedó claro que, deliberadamente o no, nisiquiera aplicaron la best practices que están claramente descriptas en las reference de NHibernate y no solo no aplicaron cosas como la descriptas en esta wiki si no que directamente la pasaron como algo no suportado por NHibernate. Yo no se como son las legislaciones internacionales pero eso, en Italia, estaría penalizado como “publicidad engañosa”.

Igual hay que felicitar al chico porque : No importa que se hable bien o mal lo importante es que se hable del producto… alguien va a caer seguro.

23 August 2009

NHibernate Perfomance: Analisys

In this post I will show the results of the previous post from another point of view hoping it would be useful to Gergely Orosz to understand how work with NHibernate to have better performance for his test (in that post you can download the original code).

The environment

Pentium D840, 3GB ram, winXP, MsSQL2008Express, .NET3.5, NHibernate2.1.0GA.

The domain is the same:

domain

where each Company has 24 Employees and each employee has 24 Reports (save a company mean save 601 entities).

Storing data

The StoreData test is basically a bulk insertion, of the domain, importing data from XML. To have better performance, in bulk inserts, there are basically two “tricks” as described in NHibernate reference (see Chapter 12): the adonet.batch_size and the StatelessSession. If you know how work the batcher, studying the code or watching the SQL log or reading this post, you can even change your code to improve performances having less round trips.

I haven’t used an “extreme” configuration (batch_size=500) but the result is:

To save 24 Companies, that mean 14424 entities, my code toke 39 roundtrips (in 5 transactions).

This result mean ~3050 entities per second, with normal behavior, and ~5300 entities per second using compiled queries (note the domain is something with parent-child relationship and a big NVARCHAR and not the same used by ORMBattle.NET).

If I play a little bit more, with the code, I may have even better performance.

Read All

Here I can show the code:

IList<NHCompany> result;
using (var session = Sf.OpenSession())
{
result = session.GetNamedQuery("NHCompany.All").List<NHCompany>();
}
foreach (NHCompany company in result)
{
writer.WriteLine(company);
foreach (NHEmployee employee in company.Employees)
{
writer.WriteLine(employee);
foreach (NHReport report in employee.Reports)
{
writer.WriteLine(report);
}
}
}

As you can see “read all” mean literally read-all and then show results in a log file. The difference here is that I’m loading 24 companies with all his employees and all reports (again 14424 entities) in only one roundtrip and only one SQL.

Read by ID

Again the code:

foreach (Company company in companies)
{
NHCompany foundCompany;
using (ISession session = Sf.OpenSession())
{
foundCompany = session.GetNamedQuery("NHCompany.One")
.SetInt32("pId", company.Id).UniqueResult<NHCompany>();
}

writer.WriteLine(foundCompany);
foreach (var foundEmployee in foundCompany.Employees)
{
writer.WriteLine(foundEmployee);
foreach (var foundReport in foundEmployee.Reports)
{
writer.WriteLine(foundReport);
}
}
}

Which sense has this test I don’t know, by the way it is loading again each company (24) with all his employees and all reports (again 14424 entities) but, in my implementation, in only 24 roundtrips.

Update

foreach (NHCompany company in companies)
{
company.Name += valueToAdd;
foreach (NHEmployee employee in company.Employees)
{
employee.Name += valueToAdd;
foreach (NHReport report in employee.Reports)
{
report.Text = valueToAdd + report.Text;
}
}
session.Update(company);
}

This code is updating each entity (again 14424) and I’m doing it in 5 transactions. Thanks to this test I found a missed feature in NHibernate (the code is there but the configuration does not allow its usage) because, so far, this code is working using 49 roundtrips per each company when we can obtain the same final result with 39 roundtrips in total (for all 14424). As you can see I’m using a normal ISession (you can see it because there is only a session.Update and StateLessSession does not work with cascade).

Even if NH2.1.0GA is performing this test in 49 roundtrips per each company, the final result is ~1550 updates per second, with normal behavior, and ~2450 updates per second using compiled queries.

Delete

public override void DeleteData()
{
using (var session = Sf.OpenStatelessSession())
using (var tx = session.BeginTransaction())
{
session.CreateQuery("delete from NHCompany").ExecuteUpdate();
tx.Commit();
}
}

Well… is there something to say here ?

It is an entirely database clean in only one SQL and only one roundtrip in only one transaction.

Conclusions

I’m sorry with Gergely because his fault was “only” to publish the wrong post in the wrong moment; as I said before, in other moment my reaction would be only close the browser.

Anyway my opinion about MDTD continuing being the same (MDTD = Monkey Driven Test Development).

21 August 2009

NHibernate Perfomance: Cramer vs Cramer

MonkeyDriven

In this last weeks I saw the birth of a new Sport: Kick NHibernate.

So far I still waiting for a real and serious "test" checking difference in performance between NHibernate and something else using a real world environment instead bulk operations… by the way these players had choose the place and the game and we can play their game in their place.

For real this is a really boring task… especially the first time when I ran tests and wait and wait and wait to see results without touch the original code.

Quick check

After a quick watch to a blog post, comparing NH’s performance with something else, before continue reading the post, I’m going directly to the code. First of all I’m checking the NHibernate configuration and the mappings. If the configuration look like the one I’m using for a NH’s course for beginners, in general, the following step is close the browser and the story end there. This time the story is different only because I’m something tired to hear beginners (at least to hear those saying they know what mean persistent-layer and how a good persistent-layer should work).

The competition

The competition is about progressive bulk operation over this domain (note: the Text property has length="8000")

domain

The “test” is loading data from various XMLs and importing/fetching/updating/deleting data on DB. For each operation the “test” is writing imported/fetched/updated/deleted data to a text file so my target is: have the same output with better performance using NHibernate.

This are results of the first long run :

Total Entities Store Read all Read by ID Find each by text Update Delete
1110 27.69’’ 2.12’’ 2.41’’ 29.7’’ 3.81’’ 4.06’’
2954 77.05’’ 5.11’’ 6.51’’ 182.87’’ 9.05’’ 9.48’’
4368 139.02’’ 6.49’’ 12.9’’ 209.84’’ 13.38’’ 17.96’’
8420 292.39’’ 13.4’’ 29.97’’ 471.34’’ 22.81’’ 33.16’’

Note: for all tests the time reported include the time to write log files.

I’ll try to avoid comments about others parts of the code by the way I have recreated the same behavior.

After the long run the step was: check the schema. The schema proposed does not match with the schema generated by NHibernate; you know that is one of my bets practice : check your schema with the one defined in the mapping.

To change performances I have applied ORM techniques and NHibernate’s best practices:

  • Modified the code to create/drop the schema
  • cleaned a little bit more the implementation of NH’s POCO entities.
  • changed mappings (the competition is about persistent layers based on Object-Relation-Mapping; know what ORM mean is not an option)
  • optimize the code

The results after the cure

Total Entities Store Read all Read by ID Find each by text Update Delete
1110 1.05’’ 0.22’’ 0.28’’ 11.05’’ 2.48’’ 0.1’’
2954 2.02’’ 0.59’’ 0.67’’ 46.74’’ 5.06’’ 0.07’’
4368 2.51’’ 0.87’’ 0.98’’ 89.93’’ 6.63’’ 0.12’’
8420 3.5’’ 1.6’’ 1.78’’ 225.42’’ 10.88’’ 0.33’’

After the cure using compiled queries

Total Entities Store Read all Read by ID Find each by text Update Delete
1110 0.26’’ 0.2’’ 0.27’’ 7.39’’ 1.1’’ 0.57’’
2954 0.59’’ 0.56’’ 0.69’’ 22.75’’ 2.84’’ 0.72’’
4368 0.85’’ 0.79’’ 0.97’’ 37.73’’ 4.63’’ 1.2’’
8420 1.76’’ 1.6’’ 1.67’’ 96.02’’ 8.09’’ 1.81’’

There are some details here about how each test is done but I said I would avoid comments about the code… well only a note: the “Find each by text” mean find each entity by a text and even show the count of each collection.

Now try to take each result and compare it with the worst after the cure.

Perhaps you would see the code modified… I’m working as consulter, if you want see the code I will happy to send you the price.

Conclusions

Owed to the fact that the speed of light is greater than the sound speed, some people may seem brilliant before one hears the stupidities they say.

The next time you will see a competitions, before look at results, have a look to the driver.