Aarrgghh!!

Using ColdFusion and SVN to Create Release Notes

I was talking last week about using XML to act as an intermediate step in building your documentation (Automating Documentation and Automating Documentation Part 2). It dawned on me that I could also share my technique for building release notes.

I use Subversion. I've gotten positively anal about commenting when I make changes. So if you were to take the history of my SVN commits, they make pretty decent release notes. The trick is to grab them and automatically turn them into documentation.

SVN makes this pretty easy. It takes one command to grab all of the changes. It takes one switch to make the change export as XML. Once that is done, manipulating the content in ColdFusion is a breeze. I do it using <cfexecute> and svn.exe, below is my code:

<cfset svn.wc = "[path to svn working copy]" />

<cfset svn.exe = "[path to svn executable]" />

<!--- -v = verbose --xml outputs it as xml --->

<cfset svn.arg = "log #svn.wc# -v --xml" />

<!--- get contents of SVN history --->

<cfexecute name="#svn.exe#"

arguments="#svn.arg#"

timeout="30"

variable="changes" />

<cfset changes = XMLParse(changes) />

<cfoutput>

<cfloop index="i" from="1"

to="#arrayLen(changes.log.logEntry)#">

<!--- lxml = LoopLocalXML (shorted for display) --->

<cfset lXML = changes.log.logEntry[i] />

<dl>

<dt>#lXML.XmlAttributes.revision#</st>

<dd>

<ul>

<li>#lXML.author.XMLText#</li>

<li>#lXML.date.XMLText#</li>

<li>#ParagraphFormat(lXML.msg.XMLText)#</li>

</ul>

<p>Files Effected</p>

<ul>

<cfloop index="j" from="1"

to="#arraylen(lXML.paths.path)#">

<!--- lPaths = LoopLocalpaths --->

<cfset lPaths = lXML.paths.path[j] />

<li>

#lPaths.xmltext#

(#lPaths.xmlattributes.action#)

</li>

</cfloop>

</ul>

</dd>

</dl>

</cfloop>

</cfoutput>

In my build process, I use <cfsavecontent> and <cffile> to write the content to a file, and then use ANT to call the CFML page that creates the release notes - voila, release notes are now part of every build, with no extra work on my part.

February 19, 2008 Posted by Terrence Ryan at 10:53 AM

Comments (3) | TrackBack (0)

ColdFusion, Running a ColdFusion Shop, Web Development,

ColdFusion and ODBC Agent

This sent a co-worker down a wrong path yesterday, so I thought I would blog it in case it tripped anyone else up.

There was a problem with one of our SQL servers yesterday. (It was down a long time during a routine patch and reboot due to extra stuff in a Microsoft Tuesday patch.) In trying to troubleshoot the issue, one of the administrators saw errors in the event logs coming from the ColdFusion ODBC Agent.

We are using default driver for Microsoft SQL Server in the drop down on the Data Sources page. Therefore, we were using JDBC driver and weren't impacted by the ODBC error. The error message was a red herring. It took focus away from the real problem, namely that the database server hadn't come back online yet.

Going further, as far as I can tell all of the defaults on that page are JDBC drivers, with the exceptions of "ODBC Socket" and "Microsoft Access". According to this Damon's comment on this blog post, the "Microsoft Access with Unicode" doesn't even need the ODBC driver. So I would say, unless you are doing coding against Access databases, or a known ODBC only product, you probably don't need to install the ODBC services; especially since you can always install them if you need them.

I'd be interested to know if anyone violently disagrees, or if this has been said authoritatively somewhere, and I just not up on my Google-Fu.

February 14, 2008 Posted by Terrence Ryan at 10:16 AM

Comments (1) | TrackBack (0)

ColdFusion, Running a ColdFusion Shop, Web Development,

On Code Reviews... again

I gave a presentation on code reviews to the Philly CFUG a few weeks back. Then my host died, and I never posted this. I'll try and get the presentation up on the site in the next few days. But two questions arose that I figured I would discuss here:

1. Do you check the code after the review to make sure developers have made the changes for which the team has asked?
2. Can developers request a code review?

Both of these speak to a subtler issue. Does your team want to be adequate or excellent? What about each member, what do they want? The answer to this question will inform the answer to the previous ones.

Now this isn't want of these things where I'm using reverse psychology to encourage what I want others to do --- "Well if you want to be only adequate..." Look, not everyone can be excellent. I don't want to get into a whole discussion on where someone brings up "Free Markets" and someone brings up Harrison Bergeron and then we all get angry at each other. My brief statement on this is that there is plenty of value contributed by people who are only adequate.

But it's going to make a difference with a whole lot of things, including code reviews. Those that want to be excellent welcome criticism, because the idea of their product being flawed in any way is appalling to them. It's more appalling than letting other people find those flaws. But really, they like to show off what they've done, and want to be acknowledged by their peers for it too.

On the other hand, those that only want to be adequate don't care if something has flaws, as long as it works. They may try and avoid reviews, but mostly they'll tend to be indifferent to the whole process, and will only do it if they are mandated. Unfortunately once it's mandated it becomes a perfunctory step for them, a hoop to jump through. Some of the more militantly adequate will say things like "Don't bother fixing it; let's see if we can sneak it past the code review."

So, if you have group of developers striving to be excellent, they absolutely will ask for their own reviews (but you might need to let them know they are available) and you won't have to check them afterwards. On the other hand, if your team is on the other end of the spectrum, you might just have to mandate them. Perfunctory code reviews while painful are better than none at all.

As for double checking the code, it comes down to trust. Do you trust your teammates? I say err on the side of trust. Code reviews tend to ruffle a few emotional feathers to begin with. To big brother people afterwards would just add insult to injury. Trust people until they give a reason not to trust them. Obviously if you are doing work with sensitive data you may want to evaluate this on your terms.

But if people have violated your trust, you're going to be glad you followed the rest of my advice on code reviews. That way you'll have page names and line numbers by which to judge whether or not they kept up their end of the bargain.

This tension between adequate and excellent effects many other aspects of your team. I'm thinking you may want to look at this for more than just code reviews. I figured this would be a good place to sneak it in.

July 11, 2007 Posted by Terrence Ryan at 11:11 PM

Comments (2) | TrackBack (0)

ColdFusion, Running a ColdFusion Shop, Web Development,

Top Code Review Issues

I figured I would blog about what I've been seeing during code reviews lately. Maybe someone else has been seeing too, or seeing other issues. I've omitted problems that deal with our shop's internal best practices, and tried to stick with things that would apply more globally. If I over applied something let me know. It's a long one, so more after the jump.

Continue reading "Top Code Review Issues"

April 19, 2007 Posted by Terrence Ryan at 5:50 PM

Comments (3) | TrackBack (0)

ColdFusion, Running a ColdFusion Shop, Web Development,

Running a ColdFusion Shop Part 4

I received an email today, that reminded me of a topic I wanted to throw up on my Running A ColdFusion Shop series.

The email:

Subject: Mail Queues Need Attention

Body: Hoth Spool could be backed up.

To unblock spool:
Drain stop node(wlbs drainstop)
Restart Cold Fusion Service.
Restore node (wlbs start)

Make sure before doing this that the node is actually backed up.
One or two messages in the queue does not a blockage make.
But depending on the number of items and length of their stay, you can make the determination.

What this email reminded me to say was: DRY isn't just for code.

We had this problem a while back. Every once in awhile, the mail queues on a random one of our ColdFusion servers would back up. The mail would remain stuck until the server was restarted. Developers and users started getting pissed about their application email being delayed.

The short term solution was to check the queues every once in awhile. Once the problem stopped occurring, we stopped checking... until it happened again. We looked for a hotfix, to no avail. We did this a couple times, and each time we got burned when we stopped being vigilant.

Finally I said "Screw it; I'm scripting a solution to it." I started checking to see if files were in the spool directory of all of our servers. Then I had to make sure that the files had sat there for a little bit instead of just having been written there. Finally I had to send an alert with the needed fix.

No big deal, it's not advanced programming, it's not even particularly good code.

The point here is that it was running in a scheduled task... since I wrote it in 2005 (and restarted it earlier this year.) We had this problem again today, and nobody outside of my team noticed.

So what's the point of this besides a little bragging? Don't do things by hand that you can automate and forget about, or DRY isn't just for code.

April 17, 2007 Posted by Terrence Ryan at 7:23 PM

Comments (4) | TrackBack (0)

ColdFusion, Running a ColdFusion Shop, Web Development,

Running a ColdFusion Shop 3

For Reference: Part 1 and Part2.

It's been awhile, but I've finally got more to say about all this.

Backup

In the last one of these, I told you to back up your system. Maybe not directly, but it was there. Okay, so I didn't say it. I'm saying it now. Backup your machines. Do a tape backup, export to an external drive, do anything. Just backup your machines.

Backup Configuration Files Separately

Independent of a backup solution that takes care of your whole system, I would also recommend backing up configurations to separate location, in their current format. Meaning, don't put them on a tape, don't ship them off site. Just keep them somewhere you can access them. Why? Coldfusion is pretty light. CFML sites are comprised of pretty small files. Odds are, if you have a systems failure, it very well may be faster to rebuild a system from scratch then it will be to get a system backup restored and working. Rebuild your server, reinstall your webserver, and reinstall ColdFusion, all while getting the CFML files from backup. Then use configuration files you've stored elsewhere to restore both your webserver and Coldfusion to its old state. By this point, the CFML files should be out from a restore.

There are other uses for these files. You can use this method to clone your boxes, for clustering for example. Additionally, the ways they are broken up make it easy to share certain areas of configuration between machines. The most useful of these is neo-query.xml, which has all of your datasources on it.

Anyway, you probably want to know where all of these are:

• ColdFusion configuration information is stored in xml files, in [CfusionRoot]\lib.
• IIS XML configurations can be exported from the IIS Manager
  • Right Click website
  • Choose "All Tasks"
  • Choose "Save Configuration to file."

I'm sure that Apache has something to handle this too. But I don't have any experience with it.

Backup Certificates

If you are using SSL certificates, back them up. They are a pain in the ass to manage. If you lose one without a backup you have to depend on the Certificate Authority to get your SSL service back up. Better to just have them ready to go in case of a problem.

Again I can only speak authoritatively about IIS, but:

• Go to IIS Manager
• Right click your SSL secured site
• Choose "Properties"
• Go to "Directory Security"
• Under "Secure Communication" hit the "View Certificate" button
• Choose "Details" tab
• At the bottom "Copy To File"

That will bring up the "Certificate Export Wizard." I usually choose the default options, with one exception. I choose "Yes export the private key." It's less secure, so you want to make sure you store them somewhere safe, but you'll be able to do this again with a certification created from a backup with the private key.

That's all I got this time around. Sorry it's all backup related. Nothing happened; I was just collating tips for building a new Coldfusion box in our environment, and realized that these were all pretty important to doing that

April 12, 2007 Posted by Terrence Ryan at 1:42 AM

Comments (4) | TrackBack (0)

ColdFusion, Running a ColdFusion Shop, Web Development,

On Code Reviews

I know I said in a previous post, that enough has been said about code reviews, but I'm writing about them. I may have to do more code reviews in the near future, so it was a good time for me to gather myself and think of all the principles I should keep in mind. Also, if anyone else has any thoughts on the matter, I would love to hear them.

Continue reading "On Code Reviews"

February 6, 2007 Posted by Terrence Ryan at 1:06 PM

Comments (4) | TrackBack (0)

ColdFusion, Running a ColdFusion Shop, Web Development,

Running a ColdFusion Shop 2

I had more to say about running a ColdFusion shop. Hopefully I'm not overstaying my welcome on the topic.

Developers are your Users

This might be an odd bit of information, and you shouldn't forget about the users of the applications, but developers are your users. This has a few different implications:

Development servers are production servers as far as your developers are concerned. I learned this the hard way a few years back when we lost our production environment due to a compromise. (Yeah, I had a server or two hacked, don't judge me.) We only had a single node production environment. Part of our recovery plan was, in case of emergency, use the staging environment as the production environment. This could work for a day or two, but day three came around and the complaining started. Perhaps justified, but when you are trying to get a new production environment up, you don't want to hear "we need to work." In the end, you want to make sure that the development servers stay up during regular working hours.

Users have their own definition of what is and is not your responsibility. "What's that you say? You deleted your entire directory, and want me to run a backup?" Developers will eventually either accidently wipe out files, or go down a bad path with their code and want a restore. We've all done it, but as an administrator you're going to want to ensure that their problems don't become yours. Make sure they have access to a source control product. Encourage them to use it. If you're running on Windows enable Shadow Copy. If you do regular tape backups try and have a local backup that you can easily restore from. The idea here is that at some point you will be asked to run a restore, you want to make sure that you never really have to.

Don't be an asshole about security. I know I just said something about being hacked, but bear with me. The first of Microsoft's Immutable Laws of Security is:

If a bad guy can persuade you to run his program on your computer, it's not your computer anymore.

Your developers can run code on your server; by definition you are already compromised. Don't lock down tags. Don't prevent object creation. A smart annoyed developer will get around you. The administrator is pretty easy to get around if you can do any sort of file manipulation. Even if this doesn't work, there are many ways a developer can outsmart you if you are unnecessarily restrictive.

Obviously this is for groups of developers that work for the same company. I hope hosting companies don't have this attitude. Additionally, don't ignore external security.

Be open about your standards. A few years ago people left every code review we had saying, "Why didn't anyone tell us about these new standards?" The reason was that the central group of administrators trolled the blogs, read articles and paid attention to what was the newest recommendations. We would talk about them, and come up with new guidelines. Problem was that we didn't talk to the rest of the group until code reviews came up. My group started posting our standards on our Developer's blog, and now people don't leave code reviews feeling like they've been ambushed.

That's it for now, I'm sure I remember something afterwards which can wait for an eventual part 3.

January 25, 2007 Posted by Terrence Ryan at 9:30 PM

Comments (2) | TrackBack (0)

ColdFusion, Running a ColdFusion Shop, Web Development,

Running A ColdFusion Shop

This is a post I've been mulling for awhile and since my office is so cold that I can't concentrate, I figured I would give it a shot.

In addition to writing applications in ColdFusion I'm also responsible for maintaining our environment. Our environment consists of 20 or so full time developers, with another 10 to 20 or so part time developers. Our code lives on around 20 servers, which are split up according to different versions and purposes. We run sub-environments for ColdFusion 6, ColdFusion 7 and Flex 1.5 (which is a much different animal then Flex 2.0.) The developer's group as a whole maintains about 100 or so applications. These applications range in size from a simple one form troubleshooting reporting tool to our student portal that serves our entire student body, and won a Max Award for Education Experiences back in 2004.

I figured I would share some of my observations about what has made our environment more stable over the past three years. Maybe it can help; maybe I'm wrong and need correcting.

Continue reading "Running A ColdFusion Shop"

January 18, 2007 Posted by Terrence Ryan at 4:53 PM

Comments (2) | TrackBack (0)

ColdFusion, Running a ColdFusion Shop, Web Development,