Empirical Testing of Custom Tag vs. CFC for Output
This is one of the drier posts I've ever written. And I've written some dullards. If you start to fall asleep, look at the pretty tables.
It is recommended to use CFC's for "business logic" encaspulation, and leave UI encapsulation to custom tags. This is based on what we've read out on the Adobe Blogs about the subject. There's one problem with that I dislike custom tags. I have no reason for it. I could argue that I prefer a few large files as opposed to many small files. This is probably why I jumped onto CFC's so quickly.
Consequently, I've been doing UI encapsulation in CFC's. Specifically, I put all of my reused UI components (headers, footer, navigation menus) in a single CFC named "interface.cfc." I then push that to the application scope, and call it from there. I will totally own up to the fact that this is my personal preference.
However at a code review, my coworkers and I got into a discussion of the pro's and con's of using such a memory resident 'output' CFC . The discussion of to Custom Tag or CFC came up again. At the end of the discussion, I was willing to leave it at "It is a stylistic concern and not a performance based concern. " But frankly, that was based on what I knew about CFC execution and gut feelings, but certainly not hard facts. So I decided to do some performance tests.
I created a custom tag that outputted the letter 'H' with no markup. I created a CFC with a method named test, that outputted the letter 'H' with no markup. I then created an index page that would loop through both calls the same number of iterations, and record the length of time it took to loop through the calling method at that iteration count. I tried three different methods:
- Straight Custom Tag Call
- Invoke on an CFC directly
- Invoke on an CFC that had been placed into memory as an object.
My hypothesis was that method number 3 would be the fastest performer, but that it wouldn't be an order of magnitude thing, just a percentage thing.
| Methodology | Average Time (ms) |
|---|---|
| CFC Invoke from Memory | 23 |
| Custom Tag | 33 |
| CFC Direct Invoke | 68 |
My first test upheld the hypothesis, but the differences there were pretty small so I fooled around with the number of iterations until I found one that ran quickly, but still showed clear patterns. The number I came up with was 1000. Below that it all method were too fast to see any difference.
| Methodology | Average Time (ms) |
|---|---|
| CFC Invoke from Memory | 2313 |
| Custom Tag | 4272 |
| CFC Direct Invoke | 5542 |
That continued to uphold my hypothesis. However, I was running it the code on a development server. There are differences between the performance of development and production servers. Debugging information gathered on our development servers tends to slow down CFC method execution. So I ran that same iteration test on a production-configured instance of ColdFusion that sits on our development server. It has debugging turned off. Other than that there is no difference, same hardware, same memory allocation. Here are those results.
| Methodology | Average Time (ms) |
|---|---|
| CFC Invoke from Memory | 39 |
| Custom Tag | 1262 |
| CFC Direct Invoke | 2649 |
That's not a typo. On a production-configured machine CFC invocation from memory will run in about 3% of the time it will take Custom Tag execution to occur even for output components.
I figured I'd share these results and see if I'm on to something, or did I miss something major in my methodology. Does this change anyone's mind on using CFC for output?
Source Files: If you would like to reproduce my experiment, please feel free to give it a try: Source Files
Comments
That being said - it does make sense in that calling a custom tag will typically result in overhead from the scope creation (caller, variables, this) that a created CFC would be able to skip (it just creates arguments and the local scope for the CFC method execution).
I'm probably rambing a bit - but I'd probably say use the method that works best for you and your team.
What's the trade-offs/difference between the CFC/custom tag method a cfinclude or reusable interface content?
Mensch: You were warned.
Mark: I'll try and give that a shot and get back to you.
Ray: Obviously more goes into coding than performance. I wouldn't say otherwise. But, my curiosity was mainly about performance. Do you have a method for testing performance that you rely on?
Joe: Encapsulation technique like CFC functions and custom tags are in general better for this type of operation because they allow for variable passing between the calling document and the encapsulated code while maintaining a separate set of scopes for the encapsulated code. Cfincluding a template will encapsulate it, but you can't pass in variables. (However you can set them before you call the included file.) For example, let say I had an encapsulated navigation bar, and I wanted it to behave differently on different pages. With cfinclude, I would have to set a variable somewhere before I called the included code to indicate which page it was on, or use the cgi.scope to determine it. With other encapulation techniques like custom tags and CFC's you explicitly pass them. Does that make all make sense?
FWIW, the direct CFC invocation was way slower on my system than on yours, both with and without trusted cache.
Definitely an interesting observation tho'!
My main argument for not using CFCs for output (and preferring either custom tags or simple CFML pages) is that you can't really have a graphic designer / web production person editing your CFCs, whereas they can easily edit custom tags and CFML pages in their regular WYSIWYG HTML editor. That may not be important in your environment but it's important in mine :)
... anything from 10-50% faster ... [than the custom tag invocation]
And then an error:
My custom tag invocation tests typically ran in around 70 (dev [I meant prod of course]) to 200ms (dev).
Sean: First off, thanks for giving me the first practical explanation as to why CFC's are less preferable than custom tags for output. I've never heard that reason before, and while it doesn't apply to my environment, I can see why it would have an impact on yours. As for your results. I can see I left the trusted cache off on my production instance. I will re-run after tweaking that, and post my results in a day or so.
Posted by: mensch at May 19, 2006 12:47 PM