Thursday, September 24, 2009

Feeling dejected

Looking for work. Just feeling bummed. How to find a position that will pay enough and something with a smidgen of meaning? Never mind that -- how to make meaningful connections with people?

I truly wonder. Networking in the area of programming too, seems to have been a complete dead end for me. Somehow after all these years, despite a bruising effort in school, working on projects part time, keeping up with the Web Design meetup, I keep ending up in the margins, being forgotten, and outsider. It isn't surprising -- it is pretty much a pattern in my life since I was a kid -- it just seems inevitable when it really shouldn't be.

A recruiter commented this morning that she was looking for someone who lived and breathed technology, who had a passion for the work. Mmm-hmmm. I've heard that expression before from peers who encouraged me to do open source projects. I get being passionate about what you do, really! And if you looked at my stacks of books and magazines, loads of notebooks with scribbled ideas and designs, and laptop folders filled with libraries and project work, you might get the idea that I have some enduring interest in the field. But I got the impression that was not precisely what the recruiter meant.

The recruiter seemed to be using a euphemism. That is, they wanted a person who would work on projects night and day, long hours and without adequate pay. Someone who is entrenched in one technology and completely two-dimensional in everything else.

Sigh. That isn't me. I tried to broaden my search recently, but most of the opportunities have been shot down for basically that reason. The more that happens, the more it makes me wonder if I should be dropping out of the field, and doing something completely different. That's why I feel a lack of passion. I need something objective to motivate me, something with meaning.

It is just getting a lot harder to envision what kind of opportunities I should go after.

Wednesday, September 23, 2009

Being a _____ Programmer

A recruiter called today, asking if I was a PHP programmer. Sure, let's go with that. Truth is, when people ask questions like that, they are asking if that's all I've done for the past several years. But for the past couple of years I've been studying mathematics. So the general answer to most of these questions is no, I'm not a _____ Programmer. I'm just a programmer who has done some work in _____.

And it hasn't been all continuous. I spend weeks here and there working in two or three programming languages at a time, on completely different types of projects. Then I've moved on. "Why," people ask, "don't you stay on one project longer"? Well, when the project is done, you move on. I don't believe in doing projects just for the sake of building IT cruft to circle the customer in, make them pay for junk they don't need. To my way of thinking, people could get away with about 20 percent of the technology they have, and would be better off without the other 80 percent. So I spend some of my time destroying technology. That's as much a description of my past work as anything else.

Take ETL for instance. I've done jobs for Nortel, converting telephone listings for exchange between their directory services products. These were one-of, batch-mode projects, with big buffers, logging, audit trails, and programming to maintain transactional integrity. Completely tested. Stuff you'd use on a 911 system, and we did. The whole point of the system was to be able to say with confidence, that the old system could be turned down, shut off, and eventually discarded. That made the sale possible.

Or PHP. I hate it when people ask me, in not so many words, if I'm a web monkey. No, I have not spent years pretending to do serious Object-Oriented programming with psuedo-object-based template language evolved from a hacked version of Perl. I have coded with PHP in MVC frameworks though, mostly the code in Mambo/Joomla components, modules and templates, but some other stuff as well.

Let's see... the first I ever built was a personalized print-ordering preview component, using a Flash movie to show the look of a client's logo and address on a piece of printed material they could order. I also build a bridge component in PHP/MySQL code to migrate customers from a Bosch Security corporate feed into that system's user table. That interface had to be transactional: we had to update and disable client records as well. So I wrote a differencing algorithm, polling a REST XML URL they provided to source the data twice a day, and applying the transactions in SQL. I really hacked the Mambo core components up for that project, so we could integrate in an existing shopping cart's PHP code. It took a while, but I was able to work the Mambo template to conform very closely to the Bosch Corporate Web site style guide. In the end the transitions back and forth were seamless (we used no disclaimer because the users were restricted dealerships signed in under a special program login).

For the Bosch site we needed to tie back specialized marketing print collateral articles to back-end templates, front-end previews, and a pricing matrix implemented using shopping cart product catalog codes. The way this was done was by introducing semantic tags inside the normal Mambo product articles. The semantic tags carried enough information to do the linking, and were stripped out before they got to the browser. That way we avoided having to fork the stock Mambo com_content table structure. We still needed specialized functions for the previews -- it would be a lot easier with Joomla 1.5's template overrides -- but it worked pretty flexibly.

For production data feeds, instead of using REST I deployed a GIT-like tool as a means of archiving and transferring orders from the front-end site to the production server. This was nice because we could recall the orders for a given time period and rebuild production queues on demand, using the scm's commands, anywhere we wanted.

When I started consulting I contributed a Joomla site to a local Chamber, and built a PHP/XSLT/SQL component for that site. They needed a way to maintain their customers on their Web site. I put together a Joomla Web site, with lots of components specially found to meet their needs for advertising, searching member's profiles, posting special feature articles on the site, and so forth. It was in the bag, DONE. So for icing on the cake I wrote a bridge component to let them dump their QuickBooks Customer Database to the Joomla user's database. Been there, done that, did it differently. This time, I took an XML file and wrote an admin interface to let the file be uploaded, transformed via XSLT into MySQL code, and applied it directly to the database. Batch mode stuff. It wasn't the prettiest, but it worked swell. An advantage here was that the Chamber could have maintained all their membership info within QuickBooks, including categories, and we could robustly populate the Web site with new features in a snap. They had that all, then out of left field asked for an ISV to bid on a new site. The ISV got 6 grand to build a new site, but provided no means of updating the most important advertising content. And the Chamber now has to update the membership list manually. Too bad for them. That's what I get for volunteering. (See my swimming with sharks post elsewhere on the blog).

I've been somewhat conservative with the NC State Mindset Project site. The three most critical issues there were (1) running Joomla in a restricted hosting environment (which was incredibly difficult), (2) making sure the site passed accessibility checks and (3) not introducing unmaintainable code for a research project with limited resources. I did copy and adapted a few Joomla components to meet the unique needs of the project -- a help request submission and tracking component, a personal journal entry component, a simple forums component, and a plug-in to go along. Those adaptations were straightforward and did not impact the core Joomla codebase. What was more of an issue was that certain PHP functions banned in the hosting environment were buried in core code. Some changes to a few components' MVC models and controllers were necessary, and to the Joomla core library, leading to a need to migrate patches forward for every security upgrade. Joomla 1.5's template overrides help here but do not eliminate the issues.

What else, what else... ? I think that's it. I piddled with PHP sometimes at Alcatel, but it was small time stuff. I had more serious Oracle database ETL tasks and 4GL application UI work to concern myself with there. In an enterprise situation often the best strategy is to rely directly only upon the most stable, high-performance systems, and isolate, insulate, and marginalize the rest of the systems through batch-mode or REST-style interfaces. Doing so is akin to normalization with respect to a relational database: you ensure that the solution is structured with the least volatility.

Tired of Fracking around because of Internet Explorer

So, I was just looking at a form that worked well under Firefox. It looks like crap under IE 6/7/8 -- all the select boxes are mutilated and there's no accessible way to fix it.

Yeah, yeah, yeah, you tell me IE is used by everyone and his brother, so I've got to make my Web pages look good in IE. But I'm not going to. See my previous post. This qualifies as completely pointless effort. I'm not going to change my content just to accommodate the gratuitously introduced defects of an antiquated product.

If a Web page looks like crap because I didn't design it well, I'll fix it. Otherwise, if it looks like crap because the user insists on using the crappiest browser on the Web to view it, I don't have the time to waste. I'm not going to keep trying to teach this pig to sing.

I refuse.

Tired of burning away time

As a techie, I've learned and played with countless technologies in my career, and worked on a lot of projects, some important but not forgotten and some completely pointless. All without substance: without form and void.

Remember IRTS, SMSR, TAXPAY, ICSS, GATEWAY, PRTS? No? You wouldn't. They're gone, all faded away. To what body of knowledge have these corporate business projects added? None. What understanding of professional practice have these projects advanced? Well, some. Perhaps like the beating of the butterfly's wings a few like ICSS made some subtle impact, but the effects have long since been obscured and obliterated by non-competes and proprietary information agreements.

I don't think I want to sign another "proprietary information" agreement or "non-compete" again. You'd have to put me in the next pay grade for sure.

Then people suggest I work for free. Open Source is fine and dandy, but I cannot eat, pay my mortgage, cover myself with insurance, educate my child, or in general, live, for free. And I have considered open source. Yet I haven't found a project that is worth my time, that isn't almost completely pointless.

And why is it that, as a developer, I am the one that needs to get excited about other's projects? People who are altruistically committed to what it is they do, frequently do not get compensated adequately. Companies like IBM and Red Hat benefit enormously from the donations of developers. That's OK, they can benefit, but many of the developers got nothing substantive in return.

If you've got the money, the need, and the interest, then it is up to YOU to excite ME. Convince ME that your minuscule, self-important activity is worth my time and attention. How does your organization benefit society, the body of knowledge, and my interests? Why should I care?

I don't want to spend another iota of my time or attention on pointless projects. It destroys my very soul.

Tuesday, September 22, 2009

Drupal Considered Harmful

Boy, you really have to wonder. I have been evaluating Drupal for my new company site. Logging in recently to see how easy it would be to put up a nice FTP access form for my client's deliverables, I am told by Drupal that a site update is needed. Apparently there's a security hole that needs to be patched. "OK, it happens," I thought, and went to click the update link.

No update link. That would be way too easy. You get a download link instead. OK, normal for open source stuff. Like Joomla.

NOT.

With Joomla, security patches are self-contained packages. You download, check that obvious prerequisites are met (the version you have is the one the patch upgrades from), take backups, take the site off-line, and apply the patch. Take the site back on-line and you're DONE.

Look, I develop software for remote installation and upgrading. The Drupal developers don't have a clue about packaging. As I read through the update instructions, you find statements like:

  • log on as the user with user ID 1
  • switch to a core theme, such as Garland
  • Disable all custom and contributed modules
  • Remove all old files and directories
  • Copy your backed up "files" and "sites" directories


You've got to be kidding me. Basically, you have to roll back the changes you've made to the site, apply the update, then roll forward the changes again. But the update process is entirely manual, several steps are exceptionally risky to take in and of themselves, and the descriptions of the steps leave out lots of important detail... to be filled in by the (in)experience of the administrator. That's asking for a heck of a lot of trouble.

Not to mention that all my custom modules disappeared after the upgrade. The upgrade readme says to check Web sites for versions of modules compatible with the upgraded Drupal, but doesn't mention that, oh by the way, the process wipes out the modules you had installed.

So really, Drupal doesn't allow upgrades so much as doing complete re-installs of the site. This is one of the inane update processes I've ever seen in a Content Management System promoted as a professional-quality system.

If Joomla's upgrade path is a dirt trail, then Drupal's is a dirt trail covered with brambles. To my mind security patches should be reliable, simple, and quick to apply, so that sites are not left exposed longer than necessary. That rules out Drupal for me. I already have to spend hours on sites I maintain, every time new security holes are found in the CMS -- Drupal's process would at least double that time.

Tuesday, September 15, 2009

Lack of Discipline or Inappropriate Deployment ?

Speaking with a fellow consultant at the Real World DITA Conference, there was some talk that the content management profession was unfocused, that the industry was expending too much effort doing the same things in so many different ways that reuse of strategies is hindered. One thought posited was that people needed to focus effort on standardized methods, which seemed to be premised on the idea that the diversity of practice was a bad thing in and of itself.

It may be. As a programmer dealing with ever shifting platforms and ever expanding universe of "standards", I sympathize with the sense of frustration. We do spend way too much time on retooling, and too little time getting use out of the tools. Perhaps open source is to partially to blame here, as it has cheapened the perceived value of the tool chains. That cannot explain why professionals don't practice their trade uniformly.

But it isn't just a matter of diversity of practice, it is also, more fundamentally, a matter of deploying layers of IT infrastructure with inappropriate technology. Object-oriented MVC Java and .NET technologies are the main culprits here, in so far as these technologies have served to mask and increase complexity, rather than actively reduce it.

The insistence on compartmentalizing the handling of data through physical call-return control flows contributes to complexity. Frameworks like Spring introduce inversion of control in an attempt to mitigate issues in software construction and platform integration processes. But doesn't the need for such dynamic API mappings suggest that the underlying architectural layering assumptions were wrong to begin with? I'm not arguing that Spring, MVC, or inversion control are bad, but that other approaches to integrating applications may bypass the need for direct interactions between components, and thus eliminate some of the complexities introduced by over reliance on these low-level mechanism.

Scripted extensibility interfaces, IMHO, are too little appreciated in the industry in comparison with the twin bastard "platform" technologies Java and .NET . I have worked with proprietary client-server application platforms which provided far greater levels of productivity. One way they did this was by limiting their scope of application to database client-server GUIs -- a fair criticism.

Another observation though, is that many of these 4GL environments did a good job of closing the loop for building an application, and of providing straightforward, concrete patterns of practice to achieve useful application behavior. And the applications were maintainable. We could make them work and put them into production for years. Today's browser/server environments are certainly richer and provide a more robust vocabulary, but they are also much less stable and don't last for what -- months -- in practice?

I leave this post with a phrase, with little further explanation: Invasive Programming by Side Effects. We need to build pipelines.

Wednesday, September 9, 2009

Improvisational Content

While attending the Raleigh-Durham Web Design Meetup last night, I had a flash of inspiration. Not insight, mind you, but inspiration: we have a habit as Web designers of viewing pages as static entities. Even when we build-in things like AJAX-based navigation, data feeds and animations, the end result is really a snapshot of some static model stored between a database and file system. So what it that were not so?

OK, let me illustrate with a simile: we go to every once in a while for entertainment. They are an improv group, and play silly little skits to make us -- the paying audience -- laugh. One of the skits involves the rehearsal of a sort of made-up theatrical play using ideas obtained by a quick survey of the audience. As the players act out their parts, the ringing of a whistle signifies an instant-do-over: the speaking player must stop, and provide some new piece of improvised dialog instead of what they had just said. Improvisational Content takes that idea, and applies it to the paragraphs, sentences, and search terms used within a page.

Improvisational Content is not new. We already see this done at the level of images, particularly advertising content. What I'm suggesting that could be different, is that we could treat the main content in the same way as other dynamic page elements. The control triggers for introducing alternative content could be the time of day a page is loaded, the season, the locale of the viewer, etc.

Now, if you think the whole idea is crazy and should never be done, you're already too late: marketers are already finding ways to "personalize" and provide situational, contextualized responses. So it isn't a question of whether it should be done, but how it impacts the expectations and implied social contracts made by a Web site.

As a simple example, reload this page a few times and look at the link above. It should flip between the name of the venue and a generic description of the type of venue, with a 50% probability. A robot searching the page will either see a script or one of those two possible strings. People and robots browsing at different times will see slightly different content, which, however, means basically the same thing.

I suppose improvisational content would probably be considered a black-hat kind of trick by Google and other search engine operators. On the other hand, if it were implemented by server-side technology and not executed at a time granularity finer than, say, a few hours, it might take quite a while to detect. Indeed, if done right it would look much like any other minor content update, and serve much the same purpose from the user's perspective -- removing contextually-irrelevant data and varying the reading so as to keep it more current.

One can also envision entire pages made this way, constructed as fragments of machine-improvised content. Given enough context from a human author, such fragments might actually appear cohesive. This is topical minimalism taken to an extreme.

Wednesday, September 2, 2009

What happened to the software profession?

There is a tendency of hiring managers to overstate the importance of their own domain, and severely diminish the importance of the software. If professions were meals, software would not be considered the main course but a condiment, the ketchup on the side of the plate. But think about what software does, the extent to which it forms the very tissue and sinew of the corporate entity. Where's the beef? It's in the software.

CDISC interchange standards in clinical research

Attending a user group for XML-based clinical studies interchange standards (CDSIC) , it gave me pause to think about how casual the presenters were about the effort required and their level of knowledge of the technology. There was no sense of urgency or an objective need -- the kind of urgency that one would expect to arise from competition, and the kind of urgency that would enhance the value of an IT person specializing in the markup technology.

Although there was plenty of talk about what could be done, there was no clear value proposition behind the discussion. Ostensibly, the goal is to satisfy FDA submission requirements and lower the costs of transferring data between Pharmas, CROs, and third parties. There is no good editing interface for the collected data, no way to ease the collection of the metadata comprising the taxonomic portions, and the stylesheet outputs are usable but they look like crap. But there seems no channel for someone like me to get paid to work these issues.

One presenter remarked that it took him two years to reach a basic level of understanding of the use of markup, and he still wasn't quite confident in his abilities. How much was really due to the learning curve of the CDISC standard and how much due to the XML technologies, was unclear, but he also mentioned that XSLT was a major stumbling block, being "unlike any other programming languages".

I've seen this pattern many times before, in which Subject Matter Experts will simply take their time with something that could be accomplished much more rapidly with the collaboration of a technology specialist. The reasoning for having an SME work on the content is obvious, since they are trained in field protocols and the vocabulary of the domain. Yet bringing in one specialist, someone who through training or experience is well versed in the technology side of the equation, could have reduced the lag time by many months. From what I could tell, what has been accomplished technologically (excluding the standards production process itself) could have been done for well under six man months of effort, and without a lot of the proprietary SAS and Java infrastructure.

How can an industry survive by taking a multi-year cycle to respond to a need? I suppose they've got cash to burn.