Wednesday, October 28, 2009
NHibernate Optimization - Part 2 - Data Access Object (DAO)
In the first post on NHibernate Optimization, mapping file configuration was discussed. This post will focus on code optimization in data access objects (DAOs).
Load objects by ID.
Object should be loaded by ID whenever possible to benefit from first-level cache (session cache). Whenever an object is saved or loaded it is added to NHibernate's first-level cache with the ID as the key. As a reminder when NHibernate loads an object, it first looks in the first-level cache, then query cache, followed by second-level cache and finally SQL is executed. Therefore, loading an object by ID will be very quick if it is already in first-level cache.
Sometimes the UI layer needs to request an object with a business key rather than a surrogate key. In this case, a thread-safe Dictionary with key=business key and value=surrogate key should be considered. The UI layer requests the object by business key from the service layer. The service layer finds the surrogate key in the Dictionary and uses it to query the DAO. This is a little extra work, but can reap appreciable performance benefits.
Use HQL, DTOs, and/or hand-coded ADO.NET in bottlenecks.
Oftentimes the UI layer presents a screen that summarizes the state of many different objects, which can force many separate and expensive database calls that return much more data that is actually necessary. A good way to optimize this is to use HQL or hand-coded ADO.NET to SELECT only the required data, using aggregate functions if necessary, and return the data in a non-hibernate mapped data transfer object (DTO).
This post scratched the surface of code optimizations in the data access layer. Please stand by for upcoming code optimization recommendations.
Load objects by ID.
Object should be loaded by ID whenever possible to benefit from first-level cache (session cache). Whenever an object is saved or loaded it is added to NHibernate's first-level cache with the ID as the key. As a reminder when NHibernate loads an object, it first looks in the first-level cache, then query cache, followed by second-level cache and finally SQL is executed. Therefore, loading an object by ID will be very quick if it is already in first-level cache.
Sometimes the UI layer needs to request an object with a business key rather than a surrogate key. In this case, a thread-safe Dictionary with key=business key and value=surrogate key should be considered. The UI layer requests the object by business key from the service layer. The service layer finds the surrogate key in the Dictionary and uses it to query the DAO. This is a little extra work, but can reap appreciable performance benefits.
Use HQL, DTOs, and/or hand-coded ADO.NET in bottlenecks.
Oftentimes the UI layer presents a screen that summarizes the state of many different objects, which can force many separate and expensive database calls that return much more data that is actually necessary. A good way to optimize this is to use HQL or hand-coded ADO.NET to SELECT only the required data, using aggregate functions if necessary, and return the data in a non-hibernate mapped data transfer object (DTO).
This post scratched the surface of code optimizations in the data access layer. Please stand by for upcoming code optimization recommendations.
Labels: .Net, NHibernate
Monday, October 26, 2009
Interval Rails Caching
Feedpoint (source) is a blog aggregator written with Ruby on Rails deployed on a linux server. It retrieves RSS content from blogs and displays them sorted by date at http://feedpoint.hertler.org. Retrieving the RSS content is an expensive step and caching is needed to speed up HTTP requests. Google searching for a way to expire page caching on a timed interval did not yield promising results so I developed a very easy custom strategy through the use of Linux cron jobs.
The Rails code for the page caching is in home_controller.rb.
caches_page and expire_page are ActionController methods. caches_page caches the the :index and :planet_redpoint actions and expire_page expires them.
With this setup, http://feedpoint.hertler.org/home/ will load the :index into page cache and http://feedpoint.hertler.org/home/expire_cache will expire the cache. At this point, monkeys could be trained to navigate to the expire_cache URL every ten minutes or a cron job could be set up. This post will focus on setting up the cron job.
The cron job was configured by entering crontab -e from a terminal and then adding the following line:
This job uses cURL to call the expire_page action every ten minutes. Good cron tutorials are all over the web and beyond the scope of this post.
One added benefit of this approach is that if a blogger is anxious for a recently posted blog to show up on the world famous http://feedpoint.hertler.org site, it can manually be refreshed right away!
The Rails code for the page caching is in home_controller.rb.
caches_page and expire_page are ActionController methods. caches_page caches the the :index and :planet_redpoint actions and expire_page expires them.
With this setup, http://feedpoint.hertler.org/home/ will load the :index into page cache and http://feedpoint.hertler.org/home/expire_cache will expire the cache. At this point, monkeys could be trained to navigate to the expire_cache URL every ten minutes or a cron job could be set up. This post will focus on setting up the cron job.
The cron job was configured by entering crontab -e from a terminal and then adding the following line:
This job uses cURL to call the expire_page action every ten minutes. Good cron tutorials are all over the web and beyond the scope of this post.
One added benefit of this approach is that if a blogger is anxious for a recently posted blog to show up on the world famous http://feedpoint.hertler.org site, it can manually be refreshed right away!
NHibernate Optimization - Part 1 - Mapping Files
NHibernate is a powerful object/relational persistence and query service that has deep market penetration in the Java space and is rapidly gaining acceptance in the .Net community. NHibernate is particular beneficial to domain-driven projects as developers can focus on good OO design and have the database automatically generated from the domain model. Since developers are focused on classes and not tables, performance can suffer because little thought is given to how SQL is generated and executed. This is the first post in a series that will focus on strategies to monitor performance, best practices, and optimizing N/Hibernate code.
The Problem
Improper data access code and/or hibernate mapping files can cause NHibernate to execute inefficient queries, execute the same queries over and over, and/or retrieve much more data than needed. The three main areas of concern when dealing with Nhibernate performance are the mapping files, data access objects (DAOs), and monitoring/profiling tools.
Mapping file optimization
Improper fetch strategies are the most common mistakes in Nhibernate mapping files. Many times developers simply copy and paste an existing mapping file and just change the table and column names without giving any thought to fetching strategies used. As a general rule, lazy associations should be used by default because no matter what fetching strategy is used, non-lazy relationships will be fully loaded into memory executing SQL queries if necessary.
The Problem
Improper data access code and/or hibernate mapping files can cause NHibernate to execute inefficient queries, execute the same queries over and over, and/or retrieve much more data than needed. The three main areas of concern when dealing with Nhibernate performance are the mapping files, data access objects (DAOs), and monitoring/profiling tools.
Mapping file optimization
Improper fetch strategies are the most common mistakes in Nhibernate mapping files. Many times developers simply copy and paste an existing mapping file and just change the table and column names without giving any thought to fetching strategies used. As a general rule, lazy associations should be used by default because no matter what fetching strategy is used, non-lazy relationships will be fully loaded into memory executing SQL queries if necessary.
The Nhibernate developer should read and understand the fetching strategy section of the reference documentation. Batch fetching and subselect fetching can significantly reduce the number of SQL queries executed.
Another mapping element that can speed up the NHibernate application is second-level caching. A good idea for all applications is to enable read-only cache on "look-up" data that never changes. Nonstrict-read-write works well for objects that are mostly read and seldom written. More on caching will follow in upcoming posts.
In summary, use lazy associations by default, choose the write fetching strategy, and include second-level caching when appropriate.
Another mapping element that can speed up the NHibernate application is second-level caching. A good idea for all applications is to enable read-only cache on "look-up" data that never changes. Nonstrict-read-write works well for objects that are mostly read and seldom written. More on caching will follow in upcoming posts.
In summary, use lazy associations by default, choose the write fetching strategy, and include second-level caching when appropriate.
Labels: .Net, NHibernate, Performance
Friday, October 23, 2009
chroot sftp for blogger.com Tutorial
I use blogger.com to author and publish this blog. After clicking the PUBLISH POST button, blogger.com moves the post to hertler.org, which means that I need to give blogger.com access to my server.
To do this securely, I only allow blogger.com access to hertler.org through SFTP and I chroot the blogger.com user to /home/blogger so that only blog files are permitted to be updated. I set this up on my ubuntu virtual machine through hostingrails.com, which by the way is a fantastic hosting provider.
Here are the steps to chroot blogger.com:
1) Verify that you have an OpenSSH >= 4.9. Update OpenSSH if needed.
2) Create the blogger.com user and change the owner and group of its home directory to root.
3) Edit /etc/ssh/sshd_config to set up the chroot.
4) Restart ssh.
5) Verify that you can sftp in with the blogger user and that the root directory for the user is /home/blogger.
OK, now we have the chrooted sftp access for blogger set up, but we still need to server those files on the web site. We can simply set up a symbolic link to /home/blogger from our virtual host directory.
Now we are securely giving blogger.com sftp access to our web server.
To do this securely, I only allow blogger.com access to hertler.org through SFTP and I chroot the blogger.com user to /home/blogger so that only blog files are permitted to be updated. I set this up on my ubuntu virtual machine through hostingrails.com, which by the way is a fantastic hosting provider.
Here are the steps to chroot blogger.com:
1) Verify that you have an OpenSSH >= 4.9. Update OpenSSH if needed.
root@web1:/# ssh -V
OpenSSH_5.1p1 Debian-3ubuntu1, OpenSSL 0.9.8g 19 Oct 2007
2) Create the blogger.com user and change the owner and group of its home directory to root.
root@web1:/# /usr/sbin/useradd blogger
root@web1:/# passwd blogger
root@web1:/# chown -R root.root /home/blogger
3) Edit /etc/ssh/sshd_config to set up the chroot.
Subsystem sftp internal-sftp
Match user blogger
ChrootDirectory /home/blogger
AllowTcpForwarding no
ForceCommand internal-sftp
4) Restart ssh.
/etc/init.d/ssh restart
5) Verify that you can sftp in with the blogger user and that the root directory for the user is /home/blogger.
perry-hertlers-mac-pro:~ perry$ sftp blogger@hertler.org
Connecting to hertler.org...
blogger@hertler.org's password:
sftp> pwd
Remote working directory: /
sftp> ls
testfileinbloggerhomedir.txt
sftp>
OK, now we have the chrooted sftp access for blogger set up, but we still need to server those files on the web site. We can simply set up a symbolic link to /home/blogger from our virtual host directory.
root@web1:/# ln -s /home/blogger /var/www/blog.hertler.org/current
Now we are securely giving blogger.com sftp access to our web server.
Labels: linux
Subscribe to Posts [Atom]