Havagan’s Mind Spew

About six months ago, after dealing with the unique experience of being involved in the instability of seeing a government contract transition, I left employment as a full-time developer and became more of a data analyst. I switched companies and have physically moved my office all of a mile down the road. I am thoroughly enjoying getting back in the trenches of processing large amounts of data, seeking patterns and anti-patterns, presenting a hypothesis and arguing in support of my position.

I’ve also had the chance to use my own software on a daily basis. (My software serves as the primary repository for the data that I analyze.) This use has shown me that there is a solid core of functionality that does very well (or the software wouldn’t have surivived six years in production) but there are some items that could be more streamlined. Interestingly, I see the same problem in other, competing software so I’m in good company.

The final facet to my switching positions, is that I get to see the effect that my leaving has had on my former development team. As an outsider looking in, I’m also getting to learn from other people’s mistakes as the team disintegrates and the application’s forward progress (development) grinds to a stand-still. Some of this degradation is attributable to my leaving — since I was the lead developer for two years and the acting team lead for the final six months I was there — and it leaves me feeling slightly guilty, but much of it has nothing to do with me.

I may post some thoughts on the matter for my own edification to try to keep from making the same mistakes that current management has made. Most of these mistakes stem from management: hiring unqualified personnel and then providing them no training and no direction, lack of decision making (or as bad, totally random decision making based on what is easiest versus any real technical or logical reason), and lack of addressing personality conflicts head-on (again, because it is easier to ignore the problem).

These problems are exacerbated by the difficulty of the development team’s mission (and undesirable physical location) but I don’t believe that the problems are insurmountable. It may be naive of me to think that these problems can be solved — and sometimes the cynic comes out and believes that management doesn’t want to actually solve these problems. Is it better financially to take the money and run now (and not do things as well as you can) or take less profit now, do things as well as you can, so that your performance wins you more contracts, positions and income down the road. That’s where I may be naive: are contracts really awarded by merit?

I was prompted by Serp to post something to prove I’m still alive and that the new mountain bike didn’t kill me — I haven’t even crashed… yet.

It’s been a busy and frustrating couple of months work-wise. Our contract is due to expire on April 30th and as the date nears people are jumping ship and manning is becoming critical. It doesn’t help that we were informed that we had received a six month contract extension then two days later we were informed that we did not have a contract extension. It doesn’t make you feel well informed and your faith in leadership takes a hit.

So as of right now, I’ll be out of a job and leaving here on or around April 30th. Maybe. To cover my behind, I’ve been applying for various jobs overseas, in Colorado, California, Arizona and North Carolina. I’ve got some hits from recruiters and hiring managers so there’s been immediate interest but no solid forward progress yet. GD is working to place us/retain us in the event we lose the contract.

Scenarios:

We lose the contract

- I stay with my current employer and move elsewhere as they place me.
- I leave my current employer and try to get a position with the company that wins the new contract.

We get the contract

- I stay with my current employer and stay here for a while longer.
- I stay with my current employer but have already committed to another location and head for greener pastures.
- I leave my current employer because I already committed to another company in another location and head on out.

As always there’s a ton of variables to consider and it all hinges on the contract outcome. There’s currently no word on when there will be a decision made on the contract. Even then, with their past history of flip-flopping, do you believe it?

I’m taking solace in knowing that I’ll have a concrete answer in 41 days.

I’ve informed my chain of command, both corporate and military, that it’s a race. The first one to the door with an acceptable offer for an interesting position wins; I can’t turn down a concrete job offer for a “maybe” on the contract here. I don’t like being out of work.

I’m not involved in contract discussions so I’m not sure who is at fault for letting it get to this stage. In the long run, the biggest loser will be the mission as they wait for the incoming personnel to get up to speed. Even then, you’re losing years of institutional knowledge.

So there you go Serp, I’m alive and kicking and starting to lose patience with bureaucracy.

I’ve gotten back into cycling lately to keep on the weight loss track. Unfortunately, my Specialized road bike simply doesn’t cut it for the rough roads around the base. So after breaking a spoke and discussing options with the wife (self-repair, sending the wheel out, buying a used or new wheel) she thought I should break down and buy a mountain bike. It will handle the rough roads and also allow me to take advantage of the riding trails that we’ve been hiking.

I bit the bullet and purchased a 2008 Specialized Stumpjumper HT Comp.

It should be here in a couple of weeks.

I very rarely post any of my own code online (with the exception of helping folks over at Experts Exchange) because most of my coding is done at work and there’s so little about work I can post. So here’s a rare one inspired by Jon Galloway’s post on simple trigger-based auditing.

Oh, and since my home development computer is dead, no pretty screen shots for you!

My work demands that we have 100% accountability in terms of data creation and modification. With our recent upgrade to SQL2005 I took the opportunity to work with the xml datatype to implement trigger based auditing that stores the data as XML. I get all the benefits of XML in SQL2005 (like xquery support), I can easily display the data in the web application with XSLT, and I still have some structure to the audited data without having to worry about the source table schema changing.

In order to make some standard querying easier, the trigger extracts out some data that we are often requested to query on (primary key ID, created on/created by, updated on/updated by). This allows for answers to simple questions like “when was this record created”, “how often has it been modified”, “when was the last time it was modified”, and “who modified it.”

The trigger itself is for INSERT, UPDATE and DELETE so that I can have a record of the data’s changes from insertion through deletion. The end result is one row in the audit table for each modification to the source row with the full contents of the source row stored as xml in the AuditedData column.

The structure is simple: a table to hold the audit data and one trigger per table being audited.

1. The Audit Table.

CREATE TABLE [tblAudit]
[AuditID] [uniqueidentifier] ROWGUIDCOL NOT NULL CONSTRAINT [DF_tblAudit_AuditID] DEFAULT (newid()),
[AuditedOn] [datetime] NOT NULL CONSTRAINT [DF_tblAudit_AuditedOn] DEFAULT (getutcdate()),
[AuditedData] [xml] NOT NULL,
[PrimaryKeyID] [uniqueidentifier] NOT NULL,
[CreatedBy] [uniqueidentifier] NOT NULL,
[CreatedOn] [datetime] NOT NULL,
[UpdatedBy] [uniqueidentifier] NULL,
[UpdatedOn] [datetime] NULL,
CONSTRAINT (PK_tblAudit] PRIMARY KEY CLUSTERED
(
[AuditID] ASC
) ON PRIMARY


2. The Trigger

CREATE TRIGGER [trgAudit_tblMyTable]
ON [tblMyTable]
FOR INSERT, UPDATE, DELETE
AS
BEGIN
SET NOCOUNT ON;
DECLARE @Xml XML
IF EXISTS(SELECT TOP 1 * FROM Inserted) BEGIN
SET @Xml = (SELECT * FROM Inserted AS tblMyTable FOR XML AUTO)
INSERT INTO tblAudit (AuditedData, PrimaryKeyID, CreatedOn, CreatedBy, UpdatedOn, UpdatedBy)
SELECT @Xml.query('tblMyTable[@MyTableID=sql:column("MyTableID")]'), MyTableID, CreatedOn, CreatedBy, UpdatedOn, UpdatedBy FROM Inserted
END
ELSE IF EXISTS(SELECT TOP 1 * FROM Deleted) BEGIN
SET @Xml = (SELECT * FROM Deleted AS tblMyTable FOR XML AUTO)
INSERT INTO tblAudit (AuditedData, PrimaryKeyID, CreatedOn, CreatedBy, UpdatedOn, UpdatedBy)
SELECT @Xml.query('tblMyTable[@MyTableID=sql:column("MyTableID")]'), MyTableID, CreatedOn, CreatedBy, UpdatedOn, UpdatedBy FROM Deleted
END
SET NOCOUNT OFF;
END


Querying values inside of the XML data is done using SQL 2005’s XQuery support.

SELECT * FROM tblAudit
WHERE AuditedData.exist('tblMyTable[@SomeField="SomeValue"]') = 1


Besides data auditing, you can also use XML and triggers to audit database object schema changes.

When I wrote my post about CA’s code issues, I emailed CA’s customer support and senior editor/webmaster to let them know that I had blogged about them. This was after I had emailed them previously to let them know I was going to write up my experiences on my blog. I like to let people know when I’m talking about/making fun of them.

After waiting a few days without a response, I posted a short blurb to their community forums. A few days after that, I received notice that my account had been flagged as requiring moderation for all posts and my forum post that linked to my blog deleted. They didn’t like that by posting a URL, I was exposing paid content to the masses.

I respect their right to moderate their forums as they see fit, but I continue to be amused by their actions. Removing the forum post doesn’t stop the information from getting out there. It’s on blogs, it’s on Twitter, it’s on social bookmarking sites. Stop trying to contain the spillage; plug the leak instead.

It’s your site guys, the buck stops with you. Your site allows unrestricted access to content. Crossing your fingers and hoping no one shares a URL doesn’t work.

This is where my curiosity can get the better of me. My forum comments are moderated, but are my blog comments? Is their forum code as shaky as their website code? Is it possible to bypass their moderation mechanism? It’s a good thing my computer’s broken or I might have dug around.

One of the things a developer runs into frequently is the “done well enough” versus “done right” scenario. This is a follow up to an earlier post on dealing with Cigar Aficionado’s Customer Support about a log in problem on the Cigar Aficionado website.

On the one hand you have Cigar Aficionado, part of M. Shanken Communications, which is a successful magazine and accompanying web property. Their website has functionality that people are willing to pay for (Cigar Insider Newsletter and Cigar Ratings Search) and they have an active community in their forums. The majority of their users are happy with the site as-is. So obviously their code is done “well enough” for their daily needs.

On the other hand, their code is shaky. Really shaky. Their website authorization is so flimsy as to be useless. You can fool their authentication and get access to paid content by using Greasemonkey and a few lines of javascript. I hope they’re protecting my sensitive personal data better than they’re protecting their paid content. (Disclaimer, I’m a paid subscriber and being able to see the results of their authenticiation allowed me to save some time on bypassing their authentication. That said, it’s not exactly hard to figure out and guess what is going on.)

There’s no excuse for bad code. I’m not saying you should take abstraction to the Nth degree, but you should at least have a site that is somewhat professional.

Their editors fly all around the world and focus on “the good life” and they claim 1.8 million “affluent readers”, but they don’t appear to spend money on hiring solid developers. Sure, they have an expensive CMS in Vignette which is listed as costing over $250,000, but it’s probably not a poster child implementation. I hope it’s not a poster child implementation.

So here’s me playing Monday Morning Quarterback and a nice “try to avoid” list.

• “Make changes only in a development environment” and it’s corollary “don’t make changes directly to your production environment.” The developer sat on the phone with me and made changes to the production site so I could then test his changes. Whoa.
• Use source control: If you’re making changes directly to production code, I guess you’re not using source control. Maybe Vignette does automatic versioning.
• Know what the code does before you change it: Guessing and crossing your fingers isn’t a very good coding paradigm. Telling the customer “it’s not my code, I don’t know what it does, but I changed some date checks in the login code” isn’t normally a good idea although I appreciate the honesty.
• Don’t use obscurity as a security measure. With the rise of social bookmarking sites like del.icio.us, someone clipping your latest version of Cigar Insider could expose paid content to everyone on the web simply by pasting the URL that you think is well hidden.
• Multiple methods duplicating functionality: getCookie(), getCookie2() and readCookie().
• Non-specific method names: popUp1, popUp2, popUp3, popUp4, popUp5, popUp6, popUp7, popUp8.
• Obsolete code in production: popUp1 is from 2006. popUp2 is from “Holiday05″ which I guess is from 2005. Either way they’re not in use so why is it still there?
• Poor accessablity: ALT tags say one thing while the URL says another. (For example a link to the “Blogs” section has an ALT that says “Cigar Ratings.”)
• Avoid broken links: The site throws 404’s like I throw expletives.
• Analyze your server logs: Telling a customer “our server logs rollover too fast for us to look at them” is bogus even if you “have more than one site running on our Apache server.” Without the ability to analyze your server logs you don’t know when you have a bunch of broken links on your site. Or if your paid content is being seen by non-paid users. Or knowing who is reading what for marketing purposes. Ad nauseum.
• Inconsistent use of http response codes: If you’re going to redirect using http response codes, always use 302. Don’t use 301 one one page and 302 on another.
• Improper use of http response codes: Don’t throw a 404 (not found) when the page exists but I’m just not authorized to see it. All that does is make me think that I have a bad bookmark. Oh, and if you were able to analyze your server logs, you couldn’t tell the difference between a real 404 from a bad link and a fake 404 from a non-authorized page access.
• Consistent behavior across pages: If you have four pages that are part of your paid content package have them behave in the same manner for a non-authenticated user. Having one issue a 301, another issue a 404, the third issue a 403 and the fourth just work but show no useable content isn’t very user friendly.
• Meta tag keyword loading: cigars, cigars, cigars, cigars, cigars. Nit-picky I’ll admit, but I don’t think google even looks at this over-abused meta tag.

I think you get my point, but there’s more than just the items above. Time to wake up a bit Cigar Aficionado. As tech savvy users age into your core demographic, a more pleasant web experience will be demanded.

After three years without an upgrade, I’m putting some money into the old desktop.

Out with the old (P4 3.4GHz 775 single core, 2GB DDR2-533, Asus P5AD2) and in with the new (Pro Quad Core 2.4GHz, 4GB DDR2-1066, Asus P5K Black Pearl). The video card isn’t being touched since I’m not gaming and I’m already running an Antec 550W power supply and a pair of Raptor 10K SATA drives in RAID-0. I did order an aftermarket CPU cooler so that I can overlock the new quad core. This should add some much un-needed processing power to my development machine.

CORSAIR XMS2 Dominator 4GB PC2-8500 Desktop Memory Kit QUAD2X4096-8500C5DF Memory: $337
ASUS P35 Intel DDR2 1333MHz Black Pearl Edition P5K Premium / WiFi-AP Motherboard: $199.98
Intel Core 2 Quad Q6600 Quad-Core Processor 2.4GHz, 1066FSB, LGA775, 8MB cache: $278.90
THERMALTAKE V1 CL-P0401 CPU Cooler: $53.99

Time to work some over time to pay for this…

[UPDATE 14-JAN-2008]

I never could get my PC completely stable and I’m now placing the blame on the motherboard. This past weekend, the computer (meaning the new motherboard) gave up the ghost. It refuses to POST and simply powers up (LEDs and case fans) and then powers down. Then powers up and then powers down. Only unplugging it stops the cycle. I decided to not bet that the instability was due to a failing component, so I switched motherboards (from the Asus board to a Gigabyte GA-P35-DS4 rev. 2).

I’m getting lazy with regards to online applications/services. If it’s not immediate and 100% online, I give up and move on. Recent examples:

IEInspector’s HTTP Analyzer

I wanted to buy it. It was useful software and I liked that I could attach to any process. Their site (which uses Plimus for payments) demands a phone number so that they can call you to confirm your order. If I wanted to deal with the company by phone, I would have called and placed an order. I wanted to click a few buttons and be done. I don’t have a US phone number and they see my international number as being in Cuba (Cuba proper, not the Navy base Cuba). Their answer was to have me jump through additional hoops to link my personal Pay Pal account to a military or corporate email address. No, it’s too much effort to deal with all of that crap, so no sale.

FAU’s Job Posting Board

I can’t just enter my contact information and a job posting. I have to register the company I work for, wait 3 business days for approval, and then continue with additional hoop jumping. No, it’s too much effort. Craigslist is easier.

Isn’t the internet about clicking buttons and making it easier (and faster) than the traditional ways of interfacing with companies?

[Update 24-NOV-2007]
I also forgot to mention, the other solution Plimus offered to my not having a US phone number was to have someone in the US purchase the software for me, field their annoying phone call, and then my stateside buddy could forward the licensing information to me. All this so they can make a phone call that doesn’t prove anything related to my physical location. A Skype “305″ number that I answer on my computer doesn’t mean I’m in Miami any more than a corporate/military email address proves that I’m a US citizen or in the US physically.

[Updated 25-NOV-2007]
IEInspector’s website had a feedback email, so I popped off the rant from above. In a short time I had a response from IEInspector providing me a direct link allowing me to purchase via PayPal. Follow the link, confirm the payment on PayPal and a few minutes later I had the registration code in my inbox. Why couldn’t they have done that from the start?

Today I tried to install the Test Exam from the Microsoft Press Training Kit on Vista. It installed but it wouldn’t run.

Run-time error ‘-2147319779 (8002801d)’:
Automation error
Library not registered.

That’s it. No errors in the event logs, nada. I upgraded from 7.9.0.2 to 7.9.0.9 with the same result. Gah.

Cigar Aficionado is a magazine for cigar lovers and aims for the upscale market. The website (www.cigaraficionado.com) comes off a little shaky, IMHO. Not sure if their CMS software is to blame (Vignette 6) or their developers (the js seems pretty fugly with three functions to read a cookie — getCookie, getCookie2, readCookie — checks for cookie names which don’t exist, js code for 2005 christmas popups still in place, etc.).

The issue I’m having is that a single page will not load, it sends me to the “Access Denied” page. So, I did what I rarely do, and contacted them about their site issues. I received an email back telling me it was probably a cookie issue and to clear my browser’s cookies. Being the anal retentive person that I am, I had already cleared my cookies, my cache, manually deleted cookies, attempted to login on different browsers, under different profiles on one machine and on three different machines. I like to check things like that before I raise a red flag and then look like an idiot when I just needed to clear my cookies. Then they sent me links to pages on their site to clear my cookies and perform a cookie writing/reading test. I passed! Woo! That was the last I heard from their customer support.

A few days later (a week?), assuming my issue had been ignored, I contacted them again via their forums. Surprisingly, I immediately received a reply from an editor. Soon after that I received an email from the same person in customer support who I had interfaced with previously. The developer geek in me came out. I ensured the problem existed on multiple computers, profiles and browsers, created a quick script to do some cookie read/write tests, logged cookie additions across requests, took screen shots, and sent them off the information along with details on how to recreate my issue. I hate javascript, but I take pleasure in getting it to work.

One of their developers contacts me regarding doing a Copilot session. I wanted to check out Fog Creek’s Copilot anyways, so off we went. Turns out Copilot is useless over a high-latency, low-bandwidth satellite connection. Said developer was disconnected (disconnects?) and informs me that he really couldn’t do much because Copilot was unresponsive. That was it. No next steps, no follow up. I have no idea if anything else is being done on my issue, my follow up emails/questions haven’t yet been answered.

So I dug some more and analyzed the request/response information and it turns out the single page that I’m having issues with is returning a 301 status code. I’m not sure why their server would be returning a 301, but it would explain my issue. I’ve asked them to analyze their Apache logs.

From a customer support perspective (which I get stuck doing on occasions remotely supporting my web application) I guess the takeaways (the things that annoyed me) are:

1. Be responsive. Let the user know you’re working on their issue. Don’t leave them in limbo for days or weeks as to the status of their support request.
2. Ask the user to do some troubleshooting for you. While most users are idiots, the ones who aren’t idiots get annoyed when you assume they are idiots and treat them like idiots. In other words, assume the best about the end user and not the worst.
3. For tricky issues, have a customer support representative coordinate the communications, but get a developer on board immediately. It will probably save time and frustration in the end.

My wife thinks I’m an idiot for spending my free hours diagnosing this stuff (”let them fix their own website”), but I’m a geek and I like it. It’s what made me an Experts-Exchange Master. (Even I draw the line at becoming a Guru.)

UPDATE: 11-NOV-2007

They asked if I could get a better network connection so they could see about checking for an error from me. If only. If only.

I replied that I had sent cookies and HTTP header information and asked what other information they were looking for.

Their response was that no one else was having these problems, so the thing to do would be to test my machine. There is nothing alarming on their servers.

I don’t know what other machine testing I can do, but some specific “we’re looking for x,y,z” would be helpful. My non-specific requests for “what information are you looking for” were never answered so I asked them to specifically tell me what data they’re trying to gather and what tests they want to run to gather it, and I’ll do my best to do it.

Personally I think their server is issuing a 301 (perm redirect) when the restricted page is accessed without being logged in. Really a 307 (temp redirect) would be more appropriate. Somewhere along the line that “access denied” URL is being cached and returned (as expected according to the HTTP 1.1 specs) instead of the page that I want. This is supported by being able to access the page via a secured connection (SSL requests are not cached IIRC) and being able to access the page if I append a useless query string
to the URL (new non-cached request). We’ll see where this leads.

UPDATE: 13-DEC-2007

I had a phone troubleshooting session with one of their developers. It was more disturbing than helpful, but I appreciate the effort they went through. The end result was nothing has changed although I got some additional insight into their coding practices — which I emailed the editor, David Savona, about and will follow-up with another blog post when I get some time.