tristan is a code monkey

Good Automated Testing

2009-02-01T02:23:00.000-08:00

Funny, how after I rant on unit testing a bit, the Stack Overflow podcast also has a rant about unit testing. But I want to make this clear: I like automated tests. A lot.

I work at a company now that has a whole lot of software that is only manually validated. This has caused nothing but regression after regression. Without calculating anything, I would easily bet anyone, even to odds like 10000:1, that if we get a majority of the code called, which means > 50%, through automated tests, it will save the company tremendous money. And I mean, the amount of time getting the tests written will save the company money over the long term. If you have to see the neverending story of our bug list, you'll know what I'm talking about.

Making tests count is the problem. These "unit-oriented TDD mocks" that want small little tests on everything seem to be working on a problem I've never had to write. That is: a set of services where most of the code manipulates a database. This can be seen like a gigantic global side-effect system. You write code, and you tweak this other thing, outside of the confines of your process' RAM. What this means is each unit test really has to integrate with a database-oriented test. Which doesn't really fit the definition of "unit" test, because sometimes you have:

Code and schemas.
Often more than one DB element.
Often, code that mirrors the schema, and code that stores it.

So how do you define unit, as in one, isolated, individual, thingy? I throw this definition away. What I really want is automated testing", because I just don't want people in the process. I'm thinking about saying automated component testing, because I also want the software to be described at the level where you might redistribute or reuse it - that should be your component.

My personal testing-oriented list of principles has become:

Define your components, and test the interaction at this level.
Cover as much as you can automatically easily.
If you have code that can not be called via your component interface, you probably don't need it.
Remove brittle tests.
Avoid dependencies on specialized configuration.
Make configuration painless.

I'm sure this list will be refined over time.

Read, and I mean read, your documentation

2009-01-29T11:47:00.000-08:00

It must suck being a technical documentation writer. Nobody reads your crap. Don't try to convince me of this otherwise. I've written enough planning and specification and API documentation to learn the Cardinal Rule of Technical Documentation: it will not be read. Skimmed, maybe, but not read.

At fault is the culture of technology. We live in a world of instant connection many thousands of instant messages, microblogs, wall notices, etc. But the display of the computer is still not good enough for general reading. And animation is that wonderful brain candy that everyone loves to hide everywhere in your display, distracting you from your train of thought at a moment's notice.

For code monkeys, this is a sin. Why? Because our documentation tools are usually paltry, clunky affairs to begin with. Take javadoc, for example. Creating javaDoc is a horrible writing experience.

It uses HTML, which really was never intended to be read, so your documentation is actually not really readable in it's natural state, the text file itself.
Almost every text editor I know does not allow you to create a readable measure in the source code file itself. You end up having to redistribute newlines, etc, yourself.
The default stylesheets of javadoc are, well, horrendous by any kind of typographic standard.

You add on top of this the technology tools of distraction, and sprinkle in code completion, and voila - nobody will read your API documentation. At least never in detail. And most of the time, not even to get things compiling.

Do yourself a favor. Read documentation you get thoroughly. And I mean truly read it. Explore it, find links, make a meta map. When you know a library well, you will not only start to use it, you will actually think instead of waiting for the compiler to kick in. You will memorize details you never thought possible. You will become an expert, rather than just another dude with a keyboard cranking out the lines like an IDE addict.

Tristan Can Rant About Testing Like Everyone

2009-01-18T14:56:00.000-08:00

Once upon a time, I tried to develop unit tests for some code running on a JBoss server. There was a mishmash of crap dependencies, with various calls to Hibernate (a framework I now loathe), lots of wacky static method calls, the list goes on. I barely got the thing running, and it took forever to initialize (well, like 15 seconds, which is insane for a unit test). And then it broke because someone tweaked the ant build file. And because I'm already bald, well, it's ok, I had no hair to pull out.

The development of this unit test gave me two observations:

1. Mocking DB requests are useless, especially with a persistence framework like Hibernate.
2. Our internal design sucked balls, and that was 99% of the reason developing a unit test was painful.

In the end, if you make testing easy, it will be fun, and you will write less code, and all will be well in the universe. The trick is making it easy.

Here's a few things I've done to start liking testing a bit more:

1. Get the testing pain out of the way. Use a CI server (Hudson) to drive the slow-ass tests. Keep them running, but don't piss your fellow developers off by having these kinds of tests running at any point during a standard build procedure. I once knew this guy who forced a maven build to run an insane amount of nitpicky testing whenever you tried to *build the main component*. In the end, I now spit upon his name whenever I see his handiwork on the screen.

2. Start writing things in Scala. (We're a Java shop, please don't hold it against me.) You could use Groovy, or Ruby here. I just like Scala, because I just understand it at a performance level. But the big win is a massive SLOC reduction. Less code wins.

3. Get as much stuff covered with as few lines of test code as possible. Often, this runs counter to the whole "unit testing" philosophy, but I don't care. Often, this drives me to re-architect code: change how exception handling happens, for example, because you realize that all this error handling happens in the middle of a call stack. It's a way of code spelunking. Over time, I start to refine the conventions I keep, and then break the tests up a bit more.

In my experience, testing often just gets the same short stick that documentation gets. I'm not simply talking about code comments, but just communication of "what got done". More code monkeys go the route of writing something like "crap happened" on a trouble ticket, and then making a ton of changes with no described rationale. Or they go into useless rambles where you end up skipping over the really important 5th sentence in the 4th paragraph of their diatribe.

Testing well is in some way like writing well. Elegant testing is fantastic, but it often takes several revisions for you to get there. The whole "unit tests are always great" nonsense reminds me of coders that would say crap like "for every line of code you should have a comment line". ... huh? what?

</rant>

Documented?

2008-12-13T06:34:00.000-08:00

So, I'm taking over the technical management of a web service. One of the first things I did was talk to everyone, mostly to check the temperature. The kinds of questions I asked were basic. If you ask a basic question, you get a better answer. And one of those questions was What can we do better? The one unanimous issue was that we needed better documentation.

Great. Only two problems with this.

Most of the team doesn't write well.
In the past, nobody has gone out of their way to really make documentation happen.

Why I point this out: most teams probably think that this is just a scheduling problem. As if you just added some time for Documentation, the problem is solved. I think if you just added more time for documentation, people will just half-ass some crap on the wall and probably spend the rest of the time you slated for documentation surfing the intertubes.

I need a strategy. I need a system that will help bad writers get better incrementally. Which means that everything must be painfully obvious. The first pieces of the puzzle are the stuff that actually exists, which are the most important parts, anyway.

Code/API Documentation — What does your code do? Especially, what side effects are there?
Adminsitration Guide — How do we run the system?
Users Guide — How do users run the system?

The other part is planning. Essentially, you need to identify groups of features, and then answer two big questions for each feature set. One, what's the business value here? Or, how do we get paid? Two, how is this thing going to work?

I'm not sure I've not forgotten about something completely hugemongous.

Women Are Out of The IT Compartment

2008-12-13T00:40:00.000-08:00

Why do we create stupid compartments for everything? At first, I thought it was just Americans with their shared cultural stupidity:

You can be black, or white. Or maybe brown or yellow, but those aren't really used. But I've actually checked White on forms in school.
Red state means religious conservative Republican, blue state means agnostic liberal Democrat.

But you know what? When it comes to programming, the whole world shares another category:

Programmer - a guy who likes tech. Pretty geeky, usually.

It's a shame, really. And it's everyone's fault. Another blog flipped a switch about the little ways we help permeate the chicks don't do IT thing. In the article, it was shopping, where a salesman kept ansering the brother, instead of the sister who asked the questions.

The entire time we were in the store, despite the fact that it was my sister asking the questions, despite the fact that I only answered questions that she asked of me directly (in other words, I was there to help her, not to help the sales guy sell to her), almost the entire conversation was spent with the sales guy talking to me, even if he was answering her question. His body language was unquestionably that of, "She's clearly not capable of making this decision herself", and addressed everything to me, despite her repeated attempts to catch his eye and have him talk to her, the actual purchaser with the question.

My concern is that this is a kind of "shared cultural discrimination". You stick a saleswoman in there answering questions, and most of the time, I think you'd get the same result. I don't think this is a "boy's club" sort of situation, but a general acceptance that women just aren't interested and don't really know about this sort of stuff.

Build some scala-swing documentation for 0.1

2008-09-24T13:39:00.000-07:00

Beta? I don't need no stinkin' beta!

One thing about HTML documentation, I love them links. Especially the ability to navigate up and down the inheritence heirarchy. Mmm... hierarchy.

But the new scala-swing project, which looks interesting, and is included in the latest release candidate for scala, has no documentation. Ergo, I went about tweaking the src jar just to get things a goin.

First, I unjarred the scala-swing-src.jar file into the directory scala-swing-src.

Second, I changed the scala.home property to my scala distribution, which I had downloaded.

Third I added this code to the build.xml file.


      <taskdef name="scaladoc"
        classname="scala.tools.ant.Scaladoc">
        <classpath>
          <pathelement location="${scala.home}/lib/scala-compiler.jar"/>
          <pathelement location="${scala.home}/lib/scala-library.jar"/>
        </classpath>
      </taskdef>

      <property name="docs.dir" value="api" />
      <property name="sources.dir" value=".." />

      <target name="docs">
        <mkdir dir="${docs.dir}"  />
        <scaladoc
          srcdir="${sources.dir}"
          destdir="${docs.dir}"
          deprecation="yes" unchecked="yes">
          <classpath>
            <pathelement location="${scala.home}/lib/scala-compiler.jar"/>
            <pathelement location="${scala.home}/lib/scala-library.jar"/>
          </classpath>
            <include name="**/*.scala"  />
        </scaladoc>

Third, I ran the command ant docs from the scala-swing-src/doc directory, and blammo, I've got scaladoc API.

Note: this is not a true step-by-step howto, because it's getting late. You will have to engage your brain to fill in 1-2 blank spots in the steps above.

tickets != tasks

2008-03-17T15:29:00.000-07:00

My company uses trac, which so far, is my favorite project management tool that I've used. It gets out of the way for the most part. Really it is perfect for a group of code monkeys. But it is missing something very important - scheduling.

The basic unit of Trac are tickets, assigned to milestones. This works well when it comes to reporting problems, but also pretty nice when you need to see the issues you are solving. That's the core of a ticketing system. Issues happen, you write a ticket. A series of issues tracked by milestone do a good job of saying this is what happened, what will happen, what we want. It does not, however, say when anything will happen.

To say when, it's better to start with an issue, and then try to define small steps that get you going. This seems like a natural one-to-many ratio of issues to tasks, and that's true. But the real problem lies with the fact you rarely can take the one issue and accurately define the many tasks. As time flies, your many will grow, and change, as you learn things, specs change, whatever. Ultimately it is very hard to embed task analysis inside of an issue-focused system.

While tasks start their lives once someone is taking on an issue, the culmination of tasks needs to be seen in the perspective of time. In fact, I would also like to see things like milestones in such a calendar as well. A project calendar like this would spot the when of a project. And leave the tickets and milestones to simply define the what happened.

Time to market isn't everything

2008-03-15T05:46:00.000-07:00

Tim Bray brings up an argument I have heard too often - time to market kills your competition. I think it is important, yes, but it does not kill competition. There are too many important examples of time to market not killing your competition.

Let's look at some of the most important high-tech goodies out there. Internet Explorer for web browsers, Google in search, iPods in music players. All of these did not exist when competition "had the market". They came in late and took the market by being better in different ways. IE was easier to set up in the OS - you might say illegally easy to set up, but there you go. Google was simply easier to find what you really wanted. iPods were all around superior devices to any other mp3 player I used - much easier to navigate.

But the key aspect here is that these players were different and significantly better than their competition. If you have the market, and you capitalize on that by learning and adjusting quickly to what people need, it's yours. But if you stop innovating, it's just a matter of time before folks flock to something else.

Monkeys don't make good admins

2008-03-14T10:07:00.001-07:00

Code monkeys are pretty damn amazing at some complex tasks, like climbing trees, or peeling bananas. We do not, however, like computer recipes. An example computer recipe: to upgrade Foo, check script X on machine B and then restart application Y on machine C. We don't become programmers to follow recipes, we want to make them and have the computer do them for us.

You better be careful if you make your code monkeys follow recipes. Many of them, like me, get irritated. Enough irritation, and we get angry. When we get angry, we start throwing poo. And then you just end up with shit everywhere.

What? No delete?

2008-03-13T07:01:00.000-07:00

Today I was shocked to learn I could not delete a simple web page connected to my domain on Yahoo's domain registration site.

Recently, I moved my blog from a shared hosting site to blogger. I had been using it to learn some web design, and now I've decided it was more important to spend the extra time learning how to write. Leave the graphic stuff behind for now, and just point the ol' domain at blogger, which is easy to do.

Except the yahoo CNAME forwarding will forward www.tristanhunt.com but not tristanhunt.com. Which is lame, because I don't like www. because it makes no sense.

While experimenting with just having a simple page do a manual redirect as a kludge, I learned two things about the Yahoo system:

They do not allow you to edit your simple landing page for your domain
Once generated they do not allow you to delete it.

Check for yourself.

I'm sorry, but I do not care who you are. You can make people jump hoops to delete things that might mess up functionality. But not allowing deletion? I'm sorry, this is just stupid.

Java one-beaners

2008-03-12T10:25:00.000-07:00

The get and set Java conventions sure seem out of date. They are the coding equivalent of pegging your Jeans. They repeat rather a rather boring statement, and feel like a Dick and Jane book. "See class. Class has Thing. Set Thing. Get Thing. Good Thing!"

To put this drudgery in its place, I don't give the dumb get and set methods the same posh multi-line status I give other methods. Instead, I relegate them to one-liners, which I henceforth dub "the one-beaner". When the one-beaner needs explaining, I stick the first explaination on the get method.

private Thing thing;

/** You really want to play with this class' thing. */
public Thing getThing() { return thing; }
public void setThing(Thing thing) { this.thing = thing; }

If my one-beaner needs extra logic, then it gets the full treatment. Extra whitespace like newlines to breathe. A comment describing perhaps why. Perhaps a shiatsu.

private Thing thing;

/** This thing ain't null. */
public Thing getThing() { return thing; }

/** Slap a IllegalArgumentException at the fool who tries to nullify this thing. */
public void setThing(Thing thing)
{
 if (thing == null) throw new IllegalArgumentException("thing != null, fool!");
 this.thing = thing;
}

Now, when I read my code, I can tell the one-beaners from the code that actually does something.

Work hard like Microsoft, and people will still say you suck

2008-03-12T03:06:00.002-07:00

Has anyone in your company ever said "let's add a Word converter for this". And then you added a word converter, probably from an open source project. And then from that point people sent end endless variations of files that broke or otherwise had a lot of problems.

It's not your fault. And Microsoft is not out to get you.

Joel Sposky had an interesting and informative article about the details of the Office File formats. My 5 second summary — the formats were designed for speed back when it was important. And Microsoft has since incrementally developed rather than redesign. And now, the best way to convert an Office document is to run office.

You can automate Office pretty easily - in Windows, at least. But then you need:

Office
Windows

Hard? No. However, if you happen to be a Linux shop, and we are, the idea of having a lone Windows server in your network feels just like throwing salt in your eyes or jumping onto a bed of nails repeatedly. For many whiny trivial reasons. And one big good reason.

You have to administer a new operating system. Yes, Windows sounds easy, but it's not. Probably easier than Linux to learn yes, but it's a ton of complexity to throw into a system where your typical hires might only use a Windows box for playing a few games.Thus, dealing Microsoft Office forces you into a really high-cost of training, and doing something very complicated (creating a heterogeneous network), just to convert content. For a non-Microsoft shop, this is the sort of trivial-turned-oh-my-god experience that makes you say "Microsoft Sucks".

What's interesting to me here is that Microsoft doesn't suck. They took on a nasty problem at the beginning, to make interactivity fast despite the hardware. They solved it, and then kept that part around, extending the software when the original problem went away — hardware got faster.

There is a lesson here. Pay attention to the nasty problems you solve today. Adding complexity to handle those nasty problems could well end up with the Office experience. Work very hard, only to have people say, "You suck".

Get that stupid JList to stop wrapping

2008-03-12T03:06:00.001-07:00

Here's the deal. I've got a list of "recipes", each is a node in a javax.swing.JList. Each recipe is a panel of many things: image, title, etc. The JList didn't do too much until I figured out this:

recipesList.setVisibleRowCount(1);

A-ha, now the list no longer automatically wraps when the surrounding window (which is a JScrollPane, of course.) Maybe I'm just hung over, but this wasn't obvious to me.

The NetBeans workaround effect

2008-03-12T03:05:00.003-07:00

NetBeans continues to toy with my fantasies. It is simpler than Eclipse, and far more humane. There are no Eclipse quirks, like having to reconfigure your environment whenever you want a different workspace. And the visual editor, which builds off of the Swing Application Framework, allows you to do slam together a few components together to try things out.

But the quality! Ugh! There were 2-3 moments today alone where the editor simply froze when doing editing. I am not sure who is making the QA decisions for Sun, but these kinds of experiences leave me asking do they care? Which is sad, because I'm sure they do — but someone won the argument to ship it.

In the end, NetBeans is now my Prototyper. I use it to create prototypes, particularly for GUIs. Once I get familiar with what I want, however, everything gets transferred to Eclipse.

What I find interesting is that I haven't really bitched about this, other than this blog.

There is a lesson here for other products. If people start having problems with your products, they will probably start finding a workaround. And not tell you.

For Java editors, where you have the Eclipse beast — lots of crap, but free, and generally works — I seriously doubt there's a market for competition. And so we are left with this: feel like you are overpaying for a product, or getting used to workarounds.

The programmer's portfolio

2008-03-12T03:05:00.001-07:00

OK, my company is starting to hire people. What is one of the first things
the boss says? "It's so hard to find J2EE people." And then we talk a bit.
Sure enough, the job advertisement is rife full of toolkits. You've seen
them, these job announcements that look like a book an
O'Reilly book catalog listing? Does anyone think that sounds like a
fun company to work for?

I want to see portfolios. If programmers put together highlights of their
work, displaying what they are most proud of in technical detail, I'd
probably hire them in a second. Especially if they could write well.

Now the interesting question becomes, what is a programmer's portfolio?
A beautiful leather-bound article on your favorite API. A website displaying
your interface development work. Articles on decisions you've made.

Something tells me the creation of a portfolio would be a better learning
experience in and of itself.

Interesting unit tests in EJB code are hard to find

2008-03-12T03:04:00.001-07:00

You write Java. You are using your wonderful coverage tools like EMMA to make sure your unit tests are great. You know what you can spend a lot of time keeping track of? Methods like this.

public int getFoo()
{
  return foo;
}

public void setFoo(int foo)
{
  this.foo = foo;
}

So f-in what? You write tests that check code that Eclipse will generate a warning for. I guess this is just a statement that coverage isn't everything. And I am not the first. But I would go further, and say that coverage is really, really misleading. Particularly when you are writing server-side Java code.

Not so obvious, but still meaningless, unit testing

EJB code is notoriously hard to test. Even these days, in the era of automatic object mocking. Take a look at the kind of bean code I see a lot of these days.

public void addFooToBar(long barId, Foo foo) throws SomeStuff
{
  // em is an EntityManager instance
  Bar bar = (Bar)em.createQuery("from Bar where id=:barId and other='constraint'")
      .setParameter("barId", barId)
      .getSingleResult();

  bar.addFoo(foo);

  em.merge(bar);
}

Initially, I was like, wow, with stuff like EasyMock I can actually get this covered! But then you realize that all interaction with the EntityManager is meaningless when you mock things out. What, are you going to do a string comparison that the query will work? Are you really interested in the call to .addFoo?

This kind of code is just as meaningless in a Java-based unit test as the standard bean property accessors. The reason of course is that all the real logic is in the string data — the database query — and interaction with the database.

Two interesting problems about uninteresting code

First, I want a way to measure interesting code. Number of statements? No, not really. If you could figure out anything that really interacts with certain interfaces and classes, like the EntityManager, well, maybe that might be interesting.

Second, I want a different framework that acts like a unit test for database interaction, both with toolkits like Hibernate, as well as straight JDBC. This, I think I could come up with. But I wouldn't use a Java-based unit test framework. That tests Java code, and here, I want to test database interaction. Different problems, and how well you write a DB test depends upon how far you push your schema, not on how many lines of Java code got executed. So the second problem is, with a DB-oriented test framework, how do you measure your schema variations?

Why spam wins

2008-03-12T03:03:00.002-07:00

Spammers have the best QA job in the universe: unlimited variations until you hit a win. It's not about being perfect, it's about not being blocked. And it's not like you need fantastic results. Just look at the latest spam that beat GMail today. And GMail is probably the best spam-blocker there is.


www SuggestionDeal com It is [Unknown Tag *$$thetime* Please Fix] [Unknown Tag *$$rest* Please Fix]
Just [Unknown Tag *$$come* Please Fix] And [Unknown Tag *$$most* Please Fix] [Unknown Tag *$$girl* Please Fix] can [Unknown Tag *$$give* Please Fix]
When you see [Unknown Tag *$$magic* Please Fix] Show Without paying any penny

marveling ophthalmia namability
oilway folklore kebabs ephyra amaretto
vitellose expressively alkalizate synodontoid juniorship
bopyridian horsedom fervor drips

Yes, there's nothing to really click on, and it really is mostly nonsense, notice how the spammer just has to hit a suggestion of a real link. Otherwise, it's just nonsense. There's something beautiful in this. I think it's the fact that they are using machines for brute force duplication, but, the strength is still in our power to bridge gaps automatically.

To me, there is a design lesson here: let the machine do what it does best, and let people bridge the right gaps. Things will happen — you don't always have to force people to see everything.

Only your unit needs this unit test

2008-03-12T03:03:00.001-07:00

Unit testing is great – test your code, but without your system!

It shows it's promise when you're out a fix to a problem, and that problem happens to be buried. So buried that reproduction only happens when the stars align and your operating system has a totally different setup and time zone or your application server decides it's going to leak or something equally nefarious.

The real difficulty is being able to quickly send wacky input to your code to test it, no matter where it runs. Because you can't always easily control these inputs.

And the first mistake is that people start building up a test for the unit and everything it references. This is bad, because then you're not just testing your unit, your testing the entire subsystem. Which often has it's own runtime requirements.

This leads me to my rule...

Only your unit needs your unit test

When you need to run everything underneath the unit, what happens if the unit happens to be at the top of the pile? Yep, you end up running the entire damn thing. And chances are, that isn't easy.

Plus...

Java has made this easy for you

When your code references other code through interfaces, you have an automatic win. Your unit test just has to somehow mimic how the interface is attached to reality. And lo and behold, there are alot of ways we achieve dependency injection in Java. Spring, Guice, EJBs.

We use a lot of EJBs, so I'm going to mention how that can work.

Cheat on your EJBs

Generally, our system uses a lot of stateless session beans, so we actually just use the whole injection framework purely for connecting up code. So all you need to do is fake the injection.


public static void inject(Object target, String fieldName, Object value) throws ...
{
  Field field = target.getClass().getDeclaredField(fieldName);
  field.setAccessible(true);
  field.set(target, value);
}

Yeah, it seems dirty, but that works on each instance I've actually needed, because we don't happen to have much else in the way of dependencies.

So now, to say, test my EJB, I might write the following code:


MyEJB myEJB = new MyEJB();
MyDependency myDependency = new TestDependency(); // TestDependency derives from the MyDependency interface
inject(myEJB, "myDependency", myDependency);

// do test stuff

Here is what MyEJB probably looks like:


@Stateless
public class MyEJB implements Whatever
{
 @EJB private MyDependency myDependency;
}

Of course, if you have more initialization and whatever going on, then you might want to take a look at EasyGloss for your needs.

Another issue is that TestDependency really doesn't need to do much, other than return some stub stuff. This leads to...

Use EasyMock to fake it

The EasyMock project was born to handle generating things like TestDependency. You have a lot of control on what you want to guarantee happens to things like MyDependency when MyEJB is running. The above example becomes


MyEJB myEJB = new MyEJB();
MyDependency myDependency = createMock(MyDependency.class);
inject(myEJB, "myDependency", myDependency);

expect(myDependency.callMe(5678309))
  .andReturn("No thanks, I'm doing my hair");

replay(myDependency);
// do test stuff
verify(myDependency);

Now we have verified that MyDependency successfully declined MyEJB's advances.

Mocking your code is great, but...

A final note is pointing out the major weaknesses of this approach. First, because real system isn't testing, mocking guarantees consistency, not success. Second, if your code isn't described in wonderful Java, you are probably not going to test it using this approach.

And what is used in Java but isn't Java? Database queries, of course... ew...

Project management shouldn't involve crystal balls

2008-03-12T03:02:00.002-07:00

One of the hardest things to do is to predict the future. The best practice is not to do it. But, sure enough, software companies often become fortune tellers for their customers. They need to tell them that if the customer gives us this money today, they will receive great fortune in only 12 calendar weeks!

So, it's time to make a plan. And a plan begins by writing a specification. Often called a spec – but only because the actual thought put into the document amounts to a speck of detail. Anyhow, a plan arrives at a promise to fulfill. Great people actually fulfill their promises. Most people miss a few things unintentionally. But no matter how well we fulfill our promises, we take the specification and turn it into an estimate of hours, as if we were perfect machines that input a task and output productivity, 8 perfect hours a day.

Or, a perfect 7.5 hours in Austria, which really means 8 hours a day except for 6.5 hours on Friday.

So, most companies assign such plans to a group of people to pull off. The first analysis tends to be something like this: our estimate says n hours, we have m people, and they tend to work at l hours per week. So, (n ÷ l) ÷ m calendar weeks!

I have never seen such a system work where you have multiple people, or for those math-inclined, m > 1.

So the next step turns into a process I like to call task deification. People turn their project into a series of interconnected tasks. One leads into another. And all roads lead towards a Microsoft Project Gantt chart, or some other surrogate. The reason behind all of this work is that you can easily visualize a way to parallelize your tasks.

Task deification isn't a horrible idea, it just doesn't work. The first mistake is assuming your task breakdown is unambiguous, and everyone will do it at about the same pace. If you have done this sort of thing before, it becomes easier, because a group of people will eventually all learn the same things. We work in software, however, and software has a underlying feature that the cost of manufacturing multiple units is practically nothing. So why would you ever solve the same problems twice? Sure, it happens, but chances are, the more you do of the same thing in software, the less significant that thing is.

The other reason task deification sucks is that it inverts the focus to tasks from where it should be – the people doing the work. On the website of Edward Tufte I found a wonderful approach to changing your overall scheduling view. Basically, you exchange axes of the Gantt chart from tasks over time to people over time. I like this approach, because you think first about the people.

People do strange things, especially in software. One programmer might need quiet time, away from disruption, to read about a platform they haven't used before. One QA tester may only understand your product as a black box, and not understand every place it can be broken. One programmer might develop higher-quality code, and another might need someone else to write the documentation. What you need to see are these people and how they operate together. Your tasks will then get great detail when written from the perspective of the person that must do them, instead of the vague specification written by one person.

If the person writing the specification is the person doing all the work, you will have very few problems. Otherwise, you need to step back and look at everything from your people. Projects are made by people. Multiple people do not become one machine. A team is just an organization of people, typically unorganized.

Towards better software projects

Over time, I've discovered a few principles to planning software that actually work.

Design until you can break the problem into chunks, each that will be less than 16 hours of work – the infamous "couple of days".
You only need to be definite about the next month or two, but you need to think, and design, to the whole.
Any other person that might do the work better understand all of the details of this chunk.
Plan to adjust; but don't let that be an excuse to ignore details.
The person that does the work should define the task, or at least complete it.
A volunteered idea is always better than an assigned one.

The thing most companies completely mess up is getting their workers to volunteer. They put themselves in a situation where they have to plan everything up front. And thus, their employees lose all the potential energy and excitement of seeing their one of their ideas come to fruition.

In Struts, one page means one action

2008-03-12T03:02:00.001-07:00

In our application, we have this one page where people can edit content. This page has links to edit other aspects as well. When you navigate to another page, we save the content before navigating. It makes sense – forcing people to save just to edit another aspect of the content sucks.

Where this comes tricky is error handling. We use Struts. And for this editing page, we have several simple Actions that back all this interaction. One action to save, one to load the content editing page, others to handle the other operations. We connect these actions through redirects, which seems like a freaking bad idea to me. Because when an error happens, you can no longer redirect, because we want to add all kinds of error messages to response object. And if you redirect, poof, no more errors!

So I have a rule, each page is backed by a single action. Any operation rolls through that action. This allows you to simply call one of the other functions rather than configure a redirect mechanism. Your API is cleaner, the logic flow can be read in a single file, and hey, if you have errors, you can report them to the same location!

Learning generics by collecting arrays

2008-03-12T03:01:00.001-07:00

So you have an array of objects. These objects are more than likely things that have properties. And life would be great if you could just get an array of all the values of just one of the properties of those objects.

Why would this be great? If these objects happen to map to a database instance (like entity beans), chances are each one of those objects has a long identifier, with a field probably named id, with an accessor method getId. My goal would be to do something like this:

List objectList = //... Collection ids = collect(objectList, "id");

As it turns out, this was not very difficult to pull off. And behold, there is something very similar in a project called the Java Generic Algorithms library. You can use the transform method with the GetProperty functor to achieve something very similar. The problem is that the GetProperty doesn't work quite right. So I rolled my own.

One first part of the API is answering the question, "what should I call"? I am assuming the following things:

You are thinking in terms of a field name, not the accessor function format, so I would always think "I'm getting the id" not "I'm getting the Id", and the case sensitivity matters.
There is likely a public accessor called getFoo, and if foo is a boolean, it might be called getFoo, or it might be called isFoo. Depends.
If you didn't write an accessor, you're probably thinking this is some kind of C-style struct, so assume the field itself is accessable.

Here's my breakdown of this in a function I called reflectSimpleField

static private  T reflectSimpleField (Object object, String field, Class returnClass) throws NoSuchFieldException
{
Object reference = null;
Class klass = object.getClass();

// check for accessor method in the form getField or isField.
try
{
  reference = reflectAccessorMethod(object, klass, field, "get");
}
catch (NoSuchMethodException ex)
{
  try
  {
    reference = reflectAccessorMethod(object, klass, field, "is");
  }
  catch (NoSuchMethodException ex2)
  {
    reference = reflectField(object, klass, field);
  }
}

return (T)reference;
}

Reflecting the individual methods and fields were pretty simple, you just have to figure out the name you're going to use, and if it isn't there, I decided I would throw NoSuchFieldException. This isn't a RuntimeException, which is a little weak, IMO. (In a production system, I might change that, but this is one of those coding excercises.) In general, there were too many damn exceptions being thrown anyway, which makes the API really messy:

static private Object reflectAccessorMethod (Object object, Class klass, String field, String accessor) throws NoSuchMethodException
{
Object reference = null;

try
{
  String methodName = accessor + capitalize(field);
  Method method = klass.getMethod(methodName, (Class[])null);
  reference = method.invoke(object, (Object[])null);
}
catch (IllegalAccessException ex)
{
  throw new NoSuchMethodException("rethrowing IllegalAccessException as NoSuchMethodException :" + ex.getMessage());
}
catch (InvocationTargetException ex)
{
  throw new NoSuchMethodException("rethrowing InvocationTargetException as NoSuchMethodException :" + ex.getMessage());
}

return reference;
}

static private Object reflectField(Object object, Class klass, String fieldName) throws NoSuchFieldException
{
Object reference = null;

try
{
  Field field = klass.getDeclaredField(fieldName);
  reference = field.get(object);
}
catch (IllegalAccessException ex)
{
  throw new NoSuchFieldException("rethrowing IllegalAccessException as NoSuchFieldException :" + ex.getMessage());
}

return reference;
}

private static String capitalize(String string)
{
return string.substring(0, 1).toUpperCase() + string.substring(1);
}

Finally, I came up with two variants of my original idea for collect. The first, collect was intended to also be usable in a for loop, like so:

for (Long id : collect(objects, "id", Long.class))
{
 // do something with id
}

This required adding the return type as an argument.

Just for kicks, I also came up with a specialized function for returning a List instead of something Iterable. This saves you from actually having to copy values just in case you don't want them.

My wack at these two functions:

public static  Iterable collect (Collection collection, String field, Class returnClass) throws NoSuchFieldException
{
ArrayList list = new ArrayList();
for(T item : collection)
  list.add(reflectSimpleField(item, field, returnClass));
return list;
}

public static  List collectList (Collection collection, String field) throws NoSuchFieldException
{
Class returnClass = null;
ArrayList list = new ArrayList();
for(T item : collection)
  list.add(reflectSimpleField(item, field, returnClass));
return list;
}

If you find this stuff interesting or useful, feel free to cargo cult all you want. I'm curious to see how it compares with other things out there in the wild, and mostly, it was a great way to get used to some Java generics.

Eclipse is dead! Long live Eclipse!

2008-03-12T03:00:00.001-07:00

I love Eclipse! I hate Eclipse! I some ways it's a left-handed tool, but others it's completely right.

What I love about Eclipse.

It builds and tells me my errors as I write. Used to be, you'd stare at your code, and then pray it would compile when you'd type make. And if it compiled, hey, you were almost ready for testing!

Now, as soon as I hit save, it's ready for testing.

It writes the stupid stuff for me. Let's face it, the Java language is a bit wonky. One retarded aspect of Java are accessor methods. We need them, but they are usually very simple methods. This means it's very easy to make a typo if you're not careful. With Eclipse, there are all kinds of methods that autogenerate the stupid stuff, and let's me focus on my logic.

It has a debugger. I can't believe this is still an issue, but after years of working with debuggers in C+, I couldn't find a satisfactory non-IDE Java debugger. Holy crappy command lines, Batman!

What I hate about Eclipse

Grep is better at finding things. I've given up on the Eclipse search functions. I keep a terminal open and use grep. It's far more faster. (Of course, I still avoid using find, more out of sheer anger over it's interface than anything else.)

I can pick APIs too quickly without reading about them. Eventually, I will code on the monitor I have at home, which is a lovely 30" cinema display. With this size of display, I can select a method, and have it's java doc display automatically in a separate window. Without this size of monitor, however, it is hard to manage the screen real estate.

So you end up doing the following: type, use autocomplete, and pray. Yeah, you could say stop using autocomplete, bitch. But it saves way too much typing. Go ahead, try to do it, you can't stop. Autocompletion is seriously the crack cocaine of programming.

It's too F-in' big to be my one-and-only. I still use vim. Why? Because when I'm poking around on one of our systems that's misbehavin', I often have to edit something minor. Or hell, just edit an XML file. Opening that file in Eclipse requires havin' it local, and havin' the right plugin, and havin' enough RAM free that most citizens will only acquire in their latest and greatest machines.

Opening that file in VIM means I don't jack the server performance, and I can do it remotely. Sorry, but I can't live my entire (electronic) life inside an IDE. Sometimes you have to wander into the wild wooly wilderness of a terminal session.

Configuration still sucks So we have development going on in multiple branches. I want different workspaces for each branch - that's all my hardware can handle at once. This means I have to reconfigure Eclipse with my settings for each damn workspace. What... the hell? Why can't I tell it my favorite font once, and then for each new workspace, it just uses the one I love?

The image title is Love and Hate.

Writings should live

2008-03-12T02:58:00.000-07:00

Why are most things I read on the job undead? Why are documents made in software companies like zombies?

Too often, they are born premature, they live quickly, then are dead before you know it. And not given a proper burial, they wander around, in our electronic halls of e-mail inboxes and obsolete web pages, looking for brains.

How else can I describe reading one of these things? You stare at the document, and usually

Some things are detailed, but don't really seem part of anything
Some things are vague, but stated as if they are obvious to everybody
Some things are obsolete

As you read it, half of your attention span turns to answering questions rather than learning material. These questions are often important, like

What does this mean?
Wait, didn't we agree to remove this feature 3 weeks ago?
How important is it to have our widget be resistant to power failures if the user can simply break it by uploading the wrong file?

There are many problems here. Bad writing. Bad publication. Bad. Bad. Bad.

Easy things to help avoid document zombiefication

Send links, not attachments. Host the latest version of a document somewhere. Then people should just always just jump to the latest rather than having to search their inbox.

Yeah, you can keep old ones around for posterity. But 9 times out of 10 you just need the latest version of whatever.

Write with meaning. When a concept is vague, make sure it sounds vague in writing. When it is not, use strong terms. A vague phrase: "it probably provides 10 feeds". An exact phrase: "it will provide 10 feeds".

This helps scheduling in a planning document. Vague things should be investigated before being done. Exact things can be put off.

Version with a date. There is absolutely no reason why your document shouldn't come with a date. You can use revision control systems, but I tend to find dates more universal, and easy to understand. Fancy numbering and variation schemes tend to muddy the water.

I have never, ever, solved a documentation problem by saying "what was the difference between version 3 and 4 of this document"? Instead, I think, "what changed the last 2 weeks"?

Think like a journalist. Most technical documents would be far better if they started with the big important stuff and then weaved in details later.

It's about context with your details. You need to know the why, not just the what. The who, the when, and the where are often not as interesting in software, but it can be.