Try fast search NHibernate

21 August 2009

NHibernate Perfomance: Cramer vs Cramer

MonkeyDriven

In this last weeks I saw the birth of a new Sport: Kick NHibernate.

So far I still waiting for a real and serious "test" checking difference in performance between NHibernate and something else using a real world environment instead bulk operations… by the way these players had choose the place and the game and we can play their game in their place.

For real this is a really boring task… especially the first time when I ran tests and wait and wait and wait to see results without touch the original code.

Quick check

After a quick watch to a blog post, comparing NH’s performance with something else, before continue reading the post, I’m going directly to the code. First of all I’m checking the NHibernate configuration and the mappings. If the configuration look like the one I’m using for a NH’s course for beginners, in general, the following step is close the browser and the story end there. This time the story is different only because I’m something tired to hear beginners (at least to hear those saying they know what mean persistent-layer and how a good persistent-layer should work).

The competition

The competition is about progressive bulk operation over this domain (note: the Text property has length="8000")

domain

The “test” is loading data from various XMLs and importing/fetching/updating/deleting data on DB. For each operation the “test” is writing imported/fetched/updated/deleted data to a text file so my target is: have the same output with better performance using NHibernate.

This are results of the first long run :

Total Entities Store Read all Read by ID Find each by text Update Delete
1110 27.69’’ 2.12’’ 2.41’’ 29.7’’ 3.81’’ 4.06’’
2954 77.05’’ 5.11’’ 6.51’’ 182.87’’ 9.05’’ 9.48’’
4368 139.02’’ 6.49’’ 12.9’’ 209.84’’ 13.38’’ 17.96’’
8420 292.39’’ 13.4’’ 29.97’’ 471.34’’ 22.81’’ 33.16’’

Note: for all tests the time reported include the time to write log files.

I’ll try to avoid comments about others parts of the code by the way I have recreated the same behavior.

After the long run the step was: check the schema. The schema proposed does not match with the schema generated by NHibernate; you know that is one of my bets practice : check your schema with the one defined in the mapping.

To change performances I have applied ORM techniques and NHibernate’s best practices:

  • Modified the code to create/drop the schema
  • cleaned a little bit more the implementation of NH’s POCO entities.
  • changed mappings (the competition is about persistent layers based on Object-Relation-Mapping; know what ORM mean is not an option)
  • optimize the code

The results after the cure

Total Entities Store Read all Read by ID Find each by text Update Delete
1110 1.05’’ 0.22’’ 0.28’’ 11.05’’ 2.48’’ 0.1’’
2954 2.02’’ 0.59’’ 0.67’’ 46.74’’ 5.06’’ 0.07’’
4368 2.51’’ 0.87’’ 0.98’’ 89.93’’ 6.63’’ 0.12’’
8420 3.5’’ 1.6’’ 1.78’’ 225.42’’ 10.88’’ 0.33’’

After the cure using compiled queries

Total Entities Store Read all Read by ID Find each by text Update Delete
1110 0.26’’ 0.2’’ 0.27’’ 7.39’’ 1.1’’ 0.57’’
2954 0.59’’ 0.56’’ 0.69’’ 22.75’’ 2.84’’ 0.72’’
4368 0.85’’ 0.79’’ 0.97’’ 37.73’’ 4.63’’ 1.2’’
8420 1.76’’ 1.6’’ 1.67’’ 96.02’’ 8.09’’ 1.81’’

There are some details here about how each test is done but I said I would avoid comments about the code… well only a note: the “Find each by text” mean find each entity by a text and even show the count of each collection.

Now try to take each result and compare it with the worst after the cure.

Perhaps you would see the code modified… I’m working as consulter, if you want see the code I will happy to send you the price.

Conclusions

Owed to the fact that the speed of light is greater than the sound speed, some people may seem brilliant before one hears the stupidities they say.

The next time you will see a competitions, before look at results, have a look to the driver.

17 comments:

  1. Nice post! Could you please post the source code as well as the DB scripts? It would be interesting to see what you optimized.

    ReplyDelete
  2. Please do post source code, would help optimize my code.

    ReplyDelete
  3. @Dancemonkey @rjperes
    As I said in the post:
    "I’m working as consulter, if you want see the code I will happy to send you the price."

    ReplyDelete
  4. I really like your speed of light vs speed of sound.

    ReplyDelete
  5. Hey Fabio, very nice Post!
    Send me the price, i'm very curious for see the optimizations...

    ReplyDelete
  6. @KeitaroSan
    contact me in my private mail.

    ReplyDelete
  7. Hello,

    I am the driver in this case and it seems to me that I am not quite the monkey in this case starting with the fact that I can talk. And I do disagree with some parts of your post.

    1. I did not join the sport "Kicking NHibernate". I published some results, the source and suggested that I didn't find real performance differences between NH and EF and suggested you use which ever you are more productive with. Where's the kick there?

    2. I stated that NH is slower with bulk storing which turned out to be due for not enabling bulk operations. Luckily some _nice_ members of the NH community took the time to revise and share the new code with me (thx Tuna Toksoz).

    3. I published my source for the public to be able to reproduce results, you didn't. I would challenge you to do so otherwise this comparison wouldn't be fair.

    4. Search by text was not meant to be part of the comparison, I never published anything of that on the original blog post (http://gregdoesit.com/2009/08/nhibernate-vs-entity-framework-a-performance-test)

    5. If you've been waiting for a good comparison with ORM tools for so long than why don't you create one yourself? Criticizing seems an ease for you, I'm sure you would also be good at creating (and sharing!) value.

    I have nothing against criticism as long as it's constructive. I haven't read a single constructive thought in your article apart from the ones that you're offering for money.

    ReplyDelete
  8. @Gergely
    I'm not interested in performance comparison between NHibernate and something else... when the moment become I'm open to change NH with another persistent-layer.
    We can come to you and explain you how change your code to have better performance but in your post you should say "I'm not an expert using NH, so be careful with my results" (be honest before and not after). When you write an article comparing an OSS FX with something else you must be prepared to the reaction. I'm hoping, the next time, you, or somebody else, may ask for help before write such kind of articles (the NH's community is here to help you).

    About money: my friend... actually, only in NH-Core, I have 970 commits with 281731 C# lines new/changed; I can give you a lot for free but performance optimizations is part of my professional work and 1.83m x 120Kg is really heavy to maintain.

    ReplyDelete
  9. Sorry, only now did I realize that you're post is only about marketing you're professional services. Until now I took it personally (thanks to the monkey picture).

    Bottom line: you're an NH pro. I'm not. You contribute to the NH project for free but share your experience for money. I don't contribute to the NH project but share my NH experience for free. I guess this controversy shows this conversation won't be going anywhere.

    I'll share though how the NH community helped optimize my code to almost the extents you've done so.

    ReplyDelete
  10. @Gergely
    You don't know what you are talking about.
    How I'm sharing, for free, my experience with NH is public. You can check it by yourself in this blog, in NH-Forge, in nhusers, in others NH forums in others languages, in uNhAddIns and so on.

    If you want publish another post about NH performance (compared or not) come to me/us before do it.

    ReplyDelete
  11. It's not new but people want all for free and less they pay more they want, it's like that.

    I don't use NH from a long time but Fabio, Oren and some others help me via blog, mail, IM sometimes and they didn't ask anything in return.

    ReplyDelete
  12. Fabio, so if you have modified NH source code to make it work better, we'd be glad to test the new version.

    Kind regards,
    Alex Yakunin, ORMBattle.NET.

    ReplyDelete
  13. @Alex
    What?
    I have changed the test not NH sources.
    I have used NH2.1.0GA (last released).

    ReplyDelete
  14. @Fabio Now thats something constructive. Thanks, will do.

    ReplyDelete
  15. @Gergely Orosz:
    I'm with you. The sad thing about NHibernate (Fabio + Oren) "community" is their attitude.

    There's many open-source projects helping their work (i.e. FluentNhibernate or LINQ from Steve Strong) yet they act like they are the only ones...but they are just using the train (taken from JAVA) but charging for the tickets.
    (with an attitude they are holy ones)

    All the time their answer is DONATE or PAY and I will implement this feature if you want it, but you will donate, you will share knowledge, and they still be behind the wheel of a locomotive.


    Just to give you an example, I earned in a YEAR the same ammount of money Oren earned in a DAY giving his presentations (to complex NH framework, it has a reason there's no good docs/tutorials), but is still acting like a poor kid working on NH for FREE (what a dirty lie! when you're making living out of it).

    Now, when I will be earning 30 thousands a day, I will fear any competition stealing the consultancy work from me...

    NH is OPEN SOURCE only on first glance, till you get into production and you will see the whole machinery behind it.

    ReplyDelete
  16. @friend
    which was your support ?
    which is your name? your profile is private.

    ReplyDelete
  17. friend i'm using nhibernate in different production projects and i didn't paid nothing, for what i need it works greatly without the need of any external consultant.
    fabio helps a lot for free in the nhibernate mainling list, the next time before to speak you must know. ;)
    clearly if you asks to have custom personalized code he asks money

    ReplyDelete