Wednesday, August 31, 2011

Configuring a GIT controlled Web site

This is assuming you're using a *nix setup for the server, and Mac OSX for the workstation.

First, register your public key with the server's SSH. On a shared host, this would typically be under a control panel icon for Shell Access or SFTP/SSH . Consult your hosting service for details.

Suppose your site is "" and your user name is "jonathan".
You should be able to log in to your site. Open a command terminal window, and:


If you can't do that and get a shell prompt, you're hosed. There are ways around it, but that's another post. Knowing how to navigate this level of complexity is a sort of prerequisite... you don't need to know, just know how to find out how to do this kind of stuff.

So, you are logged in to your server, and you create a subdirectory for your repositories, and a directory for your web site or app:

mkdir -p ~/repositories/

You initialize that directory as a "bare" git repo. (A bare git repo has no modifiable source files present, just the repository metadata and data files). This will be the repo you will stage changes to immediately prior to deploying:

cd ~/repositories/
git init --bare

Now, you switch to another terminal, either another window or a tab. Create and/or change your to the working directory for your lili.pops site:

mkdir -p ~/workspace/
cd ~/workspace/

Now, at this point, we're assuming that there are oodles and oodles of cool source files with names like "index.html" and "flavor_picker.js" just sitting around in ~/workspace/

We can (we should, nay, we must!) turn the directory into a git repo (a non-bare repo with source files, that is):

git init
git add .
git commit -m "initial commit of content"

This is the really weird part:Git links up repositories very loosely. Git keeps data about the sets of files that are changed, which it calls a "commit". These commits can be pushed by you to bare repositories, or pulled by people working with you into their own repositories.

So people contribute independently. Note also, that it isn't necessary for everyone to have complete and utter exposure to all of your files.

Let's link our repository to the bare repository on the server:

git remote add staging ssh://

# (Uhhhh.... long line folding is making this look wrong. It should all be one line.)

What we are doing here is simply making an association between a local label "staging", and a url that points to the server's repository. In this case, we're using the SSH protocol to make the link. Since we provided a public key, we aren't going to be asked for a password in the next step.

The next step is to set up a branch on the remote that will track our local main line branch, and copy our changes to it:

git push staging +master:refs/heads/master

The "+master:" part means "create a master branch on the remote repository".
The "refs/heads/master" part means "compare the remote branch's references to references from the local head master branch".

Once that association has been made between the local and remote repositories, the command to push our changes is much simpler: we just refer to the label we used ("staging"):

git push staging

Go ahead, repeat that command. You should see:

Everything up-to-date

At this point, all we've done is pushed changes to a remote repository. No one can see our files because that remote is "bare" -- there is no working source tree. The most obvious way to deal with this is to go to the server and ask git to pull out a copy of the working files into some web root location.

Switch back to your SSH command window, and checkout the changes on the server:

GIT_DIR=~/repositories/ GIT_WORK_TREE=~/public_html/staging/latest git checkout -f

Here, I'm assuming that you use public_html/staging/latest as a location to push your changes to before they've been finally released.

Now, that's all that is necessary, but we can streamline the process by automating that last step. There are two ways to do this. One is by way of a shell script, which I'll call "deploy":
user = ${1%%@*}
server = ${1##*@}
branch = $2
git push $server $branch
ssh ${user}@$server "GIT_DIR=~/repositories/ GIT_WORK_TREE=~/public_html/staging/latest git checkout -f"

Well, that's a rough guesscript. It looks like it would work, but I haven't tested it.

Another way is by a Git "hook". Go to the SSH window, and run the following:

cd ~/repositories/
cat > "hooks/post-update" <<EOT
GIT_DIR=~/repositories/ GIT_WORK_TREE=~/public_html/staging/latest git checkout -f
chmod u+x "hooks/post-update"

Now, whenever you do a git push staging, the post-update hook on the server will do the checkout for you.

Note: if you are deploying applications, you probably already use a .htaccess file or alternatives to hide or block access to resources that aught not to be exposed on your site.

Sunday, August 21, 2011

A case for web site Minimalism

It has been several months since my old Joomla based site was hacked. At the time the workaround was to put up a static placeholder page, with minimal contact information on it. It seems as if the time has come to revisit the site, now that I'm again on the prowl for more work.

While I endeavor not to lose focus, too many questions arise. "Why?" for instance. My Web site has never been a source of clients. For technical communication purposes I blog here, and for project work I use external services like github. There is just a little bit of lie to the oft told tale that a business "needs a Website".

What a business needs is a reputation. A Website is just one location people expect to help them inform their own opinions. Most people don't bother to read, let alone dig around for information, so unless a bit provides some meaningful novelty that contributes to your reputation that bit should be discarded.

And that's just information on a static site. Bits in database driven CMS sites come with much higher initial costs, a greater ongoing maintenance burden, and unmeasured risk exposures. Is a PHP content management system a worthwhile expense? Maybe. Is it an investment in an asset? It could be, if your brand is worth something. But from a technology perspective most CMS systems present more of a liability than asset.

Thursday, August 18, 2011

Debugging Consoles with Ruby

In ruby the debugger is not nearly as nifty as the console. Fortunately, there is, a gem that provides an embeddable console.  In the simplest terms, Pry provides real-time access to your program's state through a Read-Eval-Print-Loop, much like Rails' console.

I've been a shell programmer for most of my working life, but I've never, ever, ever, been a fan of REPL user interfaces. They suck.  Readline interfaces are fine for shell tools, but there's no particular reason the user interface has to be line-oriented.

One of my first computers, an Atari 130XE, had an interesting editing interface for prototyping code.  It kept lines in a buffer, and supported both immediate execution (of un-numberd lines) and editing (of numbered lines).  Buffering, baby, buffering!  That's the difference between a 30 year old 8k BASIC console and a modern Ruby console: in-place editing.

But wait -- we can get a first approximation of in-place editing by cleverly hooking in an editor component. A quick search turned up two possibilities:  and . I gave the interactive_editor gem a try.

To summarize, in a Rails 3.x app, you would just add the two gems to your Gemfile, run bundle, and then embed the expression


in your code where you want the console to open.

There are a few other details. The vimcast has some useful tips on setting up the interactive_editor, for instance, to make sure you get nice colorization turned on. But after that, using it is a breeze. Just use


to open a vim editor into a temporary buffer. When you save and close, the console will run your code. Running vi again will reopen the last temporary file. It's a VEPL,  a Vim-Eval-Print-Loop.

But even more interesting for debugging, you can use the editor to work with objects through YAML representations.  Just add ".vi" to the end of an object, and your vim editor will open with the YAML. For instance, in UserController#update, I might do:

You get something like this:

  1 --- !ruby/object:User                                                                                                                                                             
  2 attributes:
  3   id: 2
  4   email:
  5   encrypted_password: $2a$10$s/NKiMEC1UfHzgG5JlCCJuHpZbAZ77dz623rq6gt12YUqW7RpvoWW
  6   reset_password_token:
  7   remember_token:

and so on (except, nicely colorized if you followed the vimcast). 

Not only is it much easier to look at and inspect than a puts or the debugger's "p" output, but the YAML is a mutable representation. Editing it, saving and closing will modify the in-memory object. You could do this sort of hot debugging already, but the editor makes it a lot easier to visualize, and lets you tweak multiple properties in one step without having to type any code.

Tuesday, August 16, 2011

Is DRY a little wet? Part 2

A more fundamental fallacy behind DRY is the illusion of orthogonality. That is, hierarchical categorization is by and large achieved through a self-imposed discipline. Even where structural relationships are apparently observable in physical, neither the naming or the affiliations of parts are essential, intrinsic properties of the system.

Like Feynman's birds, one can know all the names for a creature in every human language, and yet know nothing at all about the animal itself.

I know, I know, it is a fine distinction, possible to the point of splitting hairs. Software systems are systems of logic, and as such they are synthetic human contrivances, are they not? So why can't we just assert that one perspective take precedence, and rationalize that the collection of all properties forms a space with an orthogonal basis?

Well, we can deliberately live in such denial. That's just the kind of linear thinking that has served the scientific and engineering communities well over the past few centuries. It is quite pragmatic in fact.

Yet it is also self-limiting. Eventually, the addition of features causes the system to undergo a phase change, as thresholds are reached in the expressiveness of the supposed authoritative source, and the constructions of the relations from the source to the dependent sinks.

More pragmatically, the fallacy can take root in the form of overt and aggressive over-simplification, in which the developer mistakes a source for a sink, and summarily discards it.

Algebra is the branch of mathematics that speaks to orthogonality, giving us the axioms and theorems by which to formally comprehend how such systems are structured. If we look at a software system as if it were some abstract vector space, this would tell us that there must exist some set of primitive vectors that form a basis for the space. Such a basis need not be finite, but ideally it is free of interdependencies... any one given basis vector cannot be generated by any combination of the remaining basis vectors. Approaching that kind of behavior is an objective of DRY. Improperly identifying the supposed basis, the "single source of information," is the underlying cause of the fallacy described here.

What Algebra cannot tell us, is how will one representation of information varies with respect to another, between stakeholders or over time. It is dead on exact regarding the dependent representations, but an Algebraic system cannot determine its own axioms. The real world is the primary source. In practical terms, this is why normalization, founded though it is on mathematical theory, is yet still a very subjective practice.

DRY must be just as subjective, if not more so, for it is a(n informal) form of normalization. Coincidence in space does not imply connection, and precedence in time does not imply causality, yet it seems to be common practice for some programmers to rely heavily upon such contingent phenomena to "DRY up" their code.

Programmers may even assert that as professionals, they know better than the customer, swapping out explicit distinctive representations in exchange for implicit dependencies upon co-incident data. Doing so may be a useful contingency while prototyping, but it is still a rationalization based upon assumptions. Not acknowledging that it compromises the model, if just a little bit at a time, is personal myopia but more critically this mode of thinking encourages retroactive imbedding of structural flaws.

Dealing with customers over how software should behave is a very probabilistic endeavor, and a world governed by rapidly changing probabilistic entities will only exhibit linear structure as an average over all the events and elements in its history. This indicates that doing "just enough" with respect to comprehending the source representation's dimensions will be a persistent cause of of rework, and a major injector of latent flaws. Yet we don't want Big Design Up Front- the shifting of the problem space makes that an even bigger risk factor. What to do?

I would suggest adopting confrontational forced choice testing whenever a supposed functional dependency is about to be removed and a source eliminated, and it seems the least bit questionable. In an optical exam, a confrontation test forces each eye to inspect a target independently, and a forced choice test is given when corrective lenses are progressively passed through, and beyond, optimal ranges. The idea is to force the subject just a little beyond their capabilities, to better assess the best overall focal corrections to make along multiple dimensions. The candidate bit - the "information source" for removal - is potentially a critical dimension of the problem; what is the strength of the concept in the customer's language? Even if it doesn't have a name, how established is it in the domain? Things the programmer thinks he sees may be his own figments and should be subjected to increased scrutiny as information sources. Ask what would happen if some other information sources were not present or altered after factoring out the candidate bit - if the other source has any impact at all on any one of the dependencies of the candidate it should determine them all, and only together. Functional dependencies cannot be partial. Ask yourself, after removing a candidate bit, have you introduced multiple conditionals where previously there was one or none? The more conditionals introduced, the further the system has become from being based on well-factored information sources.

In general, I lean toward explicit representations of feature information. Code should say what it means, and mean what it says, within the constraints of the expressiveness of the technology. DRY is not a reasonable justification for relying upon coincidence in your code, it is telling us not to multiply factual data or relations unnecessarily in our constructions. We used to call that Occam's Razor.

Is DRY a little wet? Part 1

DRY is a principle articulated by Agile practitioners (IIRC pragmatic programmers Dave Thomas and Andy Hunt are credited with coining and popularizing the term), which is supposedly often misapplied. Don't Repeat Yourself is the acronym, and the principle is stated as:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

The supposed fallacy that gives rise to the misapplication of DRY is being too narrowly focused upon one particular aspect of the system, commonly a project's source code base. Single-sourcing is the objective, not code reduction.

Now, kudos to the pragmatic programmers, but even more to all those technical writers who worked on single sourcing production processes, and the set-theoreticians who gave us relational normal form practices. If these other practitioners explored single sourcing and orthogonality for decades prior, the development community should be wary about just why it is they themselves as a profession are so late to the game, and what plays they might have missed along the way.

Friday, August 12, 2011

When acting "Professional" is anything but

There is an interesting antipattern of communication exhibited by many diverse professionals, when dealing with clients, namely, of asserting a direction that one should take even when it works across the purposes or completely against the client's apparent interests.

Two cases come to mind. There is the consulting software developer who does just enough to get by while carefully eliding any discussion of alternatives, consequences, or expectations of future outlays. The programmer exercises creative license when he chooses to build out crude approximations rather than represent the most likely costs and risks of minimally viable features; he walks the line when he continues to avoid generalizing solutions no matter what life cycle stage the system is at; but he crosses over when the customer asks for and expects a certain level of architectural repeatability and well-factored, reusable patterns, but gets point-to-point wired-up, one-off code instead.

There is also the all-knowing expert doctor. Recently, an eye doctor gave a prescription for glasses. I explained to him my difficulties in dealing with a previous doctor -- of being given false diagnoses, and the doctor's refusal to discuss options. Naturally, as an expert, the doctor nodded his head and did an exam, explaining to me the condition of my eyes and recommending progressive lenses. Well, the progressives were weird and unfortunately the optical shop could not get the prescription right. Everything looked trapezoidal with them, and half my peripheral world was always fuzzy. I never purchased the glasses. "That's stupid," I thought, "why would anyone want to live life with such an irritating gimmick making their eyes go buggy?" Explaining this to the doctor, I asked for contact lenses. By now he has put me in a pigeon hole, and instead of giving me a prescription that works, he gives me one fuzzy left contact and one overcorrected right contact. And I can't see anything as clearly as with my old glasses. "Give it two weeks," he says, "I've been doing this for 25 years". Well, I've had my eyes for over 47 years, and I know what I wanted, and it wasn't monocular vision with my eyes going all googley and being out of focus for two weeks. The contacts lasted for two hours before they got shelved.

In general, consultants are trained to think assertively, and the more successful consultants learn to steer clients in directions favorable to their pocketbook. Yet it is easy for that assertiveness and persuasiveness to push one down a slope of ethical blindness. The practice speaks to a lack of trust in the client's ability to look after their own interest, a parasitic paternalism emerging spontaneously from the professional relationship. At best, it leads to willful ignorance of the client's intentions. At worst, the process can bilk the client out of countless hours and lead down a path too costly to maintain.