Requirements

it’s not just software requirements that suffer the problem of being vague, open-ended and non-descript. anytime we want to express anything we require, we speak a kind of short-hand which is just something we’re terribly used to, and rely on, in everyday communication.

Starting with birth, we express our requirements in one word “Mommy” meaning anything from food to nurturing to play. We grow up expressing one word “food” to mean anything from a sandwich to an ice cream.

What do you feel like eating? Mmm… Food 🙂

So unless we’re particularly motivated, and capable, we express our requirement in sufficient detail so as not to leave any room for interpretation. Like a food craving.

What do you feel like eating? Tuna sandwich on wholewheat with lettuce and onion.

Software requirements fall into the same communication trap. When the product manager has “an” idea of what is required, but is not particularly motivated, or capable, of expressing that requirement, it comes off in short-hand, leaving many gaps in the interpretation. On the other hand, when the same is particulalry motivated, and/or capable, the requirement usually comes through in one detailed description requiring very little formal “Requirements Gathering” interaction. It’s understood with accuracy. Job requirements are much the same.

Everyone wants a developer who:
has at least n years experience, is motivated, can work independantly, can work under pressure, can communicate both technically and commercially, is a team player, delivers on time, understands patterns, knowledge of databases, fill-in-the-technology [related to the position]. thing is, these specs don’t describe what the company actually wants. All that it communicates is that the company [or recruiters] have “an” idea of what they’re looking for and are prepared to shop around and see what they can find. And then very rarely, there are companies who post up their requirements in such a way that it communicates what you really need to know about the going position:
company culture and problem domain. Knowing that up front saves a lot of communicating. i guess that’s why you need to check out the website. but even that just reflects the marketing department’s image of the corporate and not what’s really going on :p

Perception of Control

one of the biggest challenges facing any lead position [particularly in an agile domain, but not solely within that domain] is wrestling for control. the processes, frameworks, tools and values are established in order that some kind of control may be exerted over the chaos. and in a position where you are required to lead, you are perceived as controlling, to a greater or lesser degree, those systems. a kind of lion tamer, if you will. the tricky bit though is not to buy into the perspective that you are controlling anything. once you do bite, settle in for a fight. how so?

you have a great architecture all planned [or semi-planned]. you’ve been collaborating on it for some time and you [your team visionaries] release it for implementation. inevitably, it doesn’t go strictly according to plan. and that’s ok, as long as what does get implemented is not a deadly wound, right? likewise, you have an iteration all neatly planned, but once it gets started Mr. Murphy makes his appearance. and suddenly you need to deal with unplanned, _urgent_ stories. from requirements gathering to release, when you are tasked with leading any of the above, it’s _hard_ to control it. and depending on your personality, ethics, team-size, professionalism, culture, your reaction will fall somewhere between the extremes of HeartAttack and Whatever. Or for the more detailed, see The Stress Continuum.btw, this also applies to any lead position in commerce. any business manager, CEO, financial director- they can all fall into the same trap. my disclaimer here is that i am nowhere near close to any sort of expert in commerce, but i have seen remarkably similar behaviour where the issue of control is concerned.

now control is very difficult to get right because of the linear mindset we have engrained. inputs and outputs, objectives and outcomes, incentives and goals; all work well. defining them is cognitively sound, striving for them is rational, even good. getting to them is difficult. and it almost seems that the moment we think have control of the system, we lose it. we use terms like “hit a bump, snag or hitch”. we may even resort to phrases like “sabotaged” or “hijacked”. but that perception of losing control is more likely a result of something completely ordinary. ie, not always an intentful diabolical plot to thwart your efforts. even if paranoia is a virtue 🙂

and why the linear midset does not work is because we are not strictly dealing with a linear system in anything that we collaborate on. life is not linear. yes, we’d love to ignore the impact of home on work and vice-versa. it’s just not realistic. yes, we’d love to tell people to leave their problems at home, and vice-versa. you’re blinded. and don’t let something like a terrorism impact on this delivery. when you come to work, you work. you leave all that behind, right? mmm…. of course, we do recognise major events. particulalry, the ones close to home. and there’s also good reason why we simply can’t factor all those things in to our planning. it’s just too much. we’d never get started with anything. and we also have to draw lines about what we can _allow_ to affect us, for the sake of moving along. but truth be told, things completely unbeknown to us wrestle against our perceived control of any system.

so how to combat this? abandon your perceived control. recognise you have no real control and focus on influencing the system. a leader, and we can look to world leaders here for examples, doesn’t control a nation, or even group. the ones that do we recognise as cults. great leaders influence. and since this is not about morals, but about successfully achieving, we can ignore the good and bad when it comes to using the word “great”. if you can but exert the right type and amount of influence, you will probably achieve far more than when you try control.

architecture. don’t try and control it. influence your team to get it right. that involves education, imparting values and a lot of rhetoric. process, agile or not. don’t control it, again, influence it. gathering requirements and planning the iteration likewise requires a lot of influence to get it right. how do you stop unplanned, _urgent_ stories from distracting your attention 2 days into an interation? you can not control it. but if you have influenced your sales team and product managers correctly, they’re less likely to interrupt. but that’s not to say they won’t ever do it 🙂 how do control a junior developer from not writing tests? you can’t. but you can influence them to. and so every situation you’re faced with when leading something is never about control. if you think your job is to control it, you’re more likely to be frustrated than not.

so leading any position where control is perceived, your first task in accepting the role is to recognise that you have no control. even if others expect you to. there too, you can influence expectations to a greater understanding that life is non-linear. we don’t live in a box. we don’t achieve outputs based on linear inputs. there’s too much beyond our control [and influence] to even try cater for, so we don’t. yet, in some way, we do try cater. just differently to what we might expect would work.

Institutionalisation

it’s been a looong break {and good}, but getting back into the stream, one toe at a time. And as i venture forth into what is commonly known as the “new?” year, i keep coming up against this theme of “institutionalisation”. It’s almost tangible in every domain, which is no surprise, since it takes place at our begging. In fact, more than begging for it, we expend a large amount of effort to reach an acceptable level of “institutionalisation”.

wikipedia.org/wiki/Institutionalization labours more on the concept than i wish to for the purposes of this post, but does provide some interesting insight. In particular, “individuals who work within large established organisations can become socialised into organizational values and norms, and values and norms may become institutionalized”. Many companies actively strive to achieve this kind of congruence throughout their organisation at some level, since it will apparently promotes productivity through shared ideals, like-mindedness and co-operativity. Not to mention all the fluffy feel good factors which enable people just to work together in harmony. But does it reach a point where it becomes bad?

Neutrally, institutionalisation is just the embedding of those values. Negatively, it is the impact which that embedding has on the individual’s ability to function outside the structured environment. Positively, it is the culmination of effort in bringing people of varied personalities together such that they might work happily side-by-side.

Religion, politics, sportsfans- any social setting actually- all display some degree of institutionalisation. The extremes within each social environment tend to display the highest degree of institutionalisation, but only as a trend. Even groups which try and balance the extremes, if they play that game for too long and hard, achieve a level of institutionalisation in which they cannot operate within any extreme environment- if only for a season. Is that really so bad?

Then we look at the values embedded into our processes, our commercial ambitions, our societal goals and ideologies and how we champion them. We look at who champions them. Why do we support them? At what point do we stop supporting them? Do we champion something ourselves and why? And when we start scratching, we realise we want to achieve some sort of institutionalisation somewhere, but ironically, with really good intentions. [Ethics and morals aside, even if you’re downright evil, you’ll believe your intentions to be good] So, if we start something out with “good” intentions, can it really become bad?

The answer to that depends on many things i guess- again, more than i need to delve into. Let’s assume then that some people are happy to say “yes” and some people, happy with “no” and that some of their reasons and assumptions are valid. And some of them displaying clear characteristics as a result of institutionalisation. And some are happy to agree to disagree while others not. Sounds rather innocuous, doesn’t it?

But it’s exactly that boring mix of responses which we need to embed ourselves in. We need to be prepared from time to time to say both “yes” and “no”. We need to keep challenging the kind of institutionalisation we’re headed toward, without buying into it while at the same time believing it with all your heart and pushing into it 🙂 Shucks, half my brain just went on strike!

So, how exactly do we achieve that… ?

::shrug:: i think we just do, at least some of us do. From marriage, to social group, to local church, to sports club, to company, to development process- where ever you got people doing something together, you need to radically push to entrench the values and norms you believe in, all the while challenging them every step of the way. I don’t know of an easier way. But then again, it’s not supposed ot be “easy”, yet .. strangely .. it can be.

WT? #3

So, just when i thought i had seen it all… this is the REQUEST, as captured by Fiddler.
Yes, you read that right… the SWF builds the request and sends it through to the web server; in plain text.
(i’ve modified the actual contents of the SQL, naturally)

POST /flashsql.php?id=106 HTTP/1.1

= QUERYSTRING ====
 id=106

= BODY ====
 host=NNN.NNN.NN.NN
 sql_=SELECT DISTINCT( id ), name, filename FROM table LEFT
JOIN table2 ON ( id = id ) LEFT JOIN
table3 ON ( id = id ) LEFT JOIN table4 ON
( id = id ) LEFT JOIN table5 ON ( id = id ) WHERE id IN(155,150,52,149,134,133,76) AND
typeId=9 ORDER BY id
 dat=databasename
5 tables, no less and a database name. And a file on the server that happily accepts any SQL for execution. Oh, and this was an e-commerce website.

They were notified and they have subsequently made things a lot more secure.

WT? #2

This was a commercial online system- tried to make a reservation and…

 webexception.PNG

WT? #1

This last year, we have uncovered numerous websites with some severe flaws. And sometimes there are fairly _large_ websites where you’d think they’d have more professional skill available to avoid the kind of things we end up seeing…

And i don’t think ca$h (lack of) is the issue- in fact, i supect that sometimes there is more than sufficient being invested. The resultant product and runtime errors that crop up are, well… for the reader to discern. The names have been hidden to protect the embarassed.

First up: TheBrowserBackButton
This is a large university’s website. I was browsing through the Maths and Computer Science areas when i clicked on my browser’s back button:

Back Button

Maybe this was intentional? It sure does help getting a call from the help desk and they can tell you exactly which line of code is causing the problem 🙂

Duty of a Programmer

While reading through The Problem with Programming, an interview with Bjarne Stroustrup, i found myself nodding enthusiastically with punctuated Amen’s and Uhhuh’s. The nub of the problem:

I think the real problem is that “we” (that is, we software developers) are in a permanent state of emergency, grasping at straws to get our work done. 

which is exacerbated by

People reward developers who deliver software that is cheap, buggy, and first. 

which brings me back to an earlier post: Don’t Panic. Seriously, that’s my first rule of programming. The only time you panic is when the word “Don’t” precedes it.
Panic implies all sorts of rush and confusion; which leads to hasty decisions [both technical and commercial] and promises; which ultimately leads to sub-standard software; which becomes the norm 🙁 Or worse yet, you can only deliver software in a state of panic…

But ’tis a mindset thing.

And of course, the often neglected side to the debate is from the end-user:

My super high-tech cell phone crashes fairly often, and it takes 2 minutes to reboot. Sometimes I wonder, what if this ever happens in an emergency situation?…in an emergency, 2 minutes is an eternity, and it can easily mean the difference between life and death. The list of our everyday software dependence could go on and on, yet whenever you buy a software package, you’re always forced to accept the “…as is…” terms. In other words, the software makers never take responsibility if their crappy creations are causing you damage. I don’t know of any other product category, which is sold on an “as is” basis. Only used cars perhaps…but even there you can find lemon laws, and dealers who are willing to sign a warranty. 

I wonder if we, as developers, truly grok the impact of what we do [or don’t do] on society, aka enduser? More than mindset, it’s also an attitude of heart with which we enter into our profession, dare i say, calling. That same attitude which motivates us to work through every debate and every challenge we face [both internal and external] to deliver something reliable…
But don’t think i am getting it right *all the time* 🙂 I’m just learning, over time, what the goal should be and working towards that, slowly- but with intent.

NUnit 2.2.8 with NAnt 0.85 rc4

I recently attributed some blame to NUnit 2.2.8 for not maintaining backwards compatibility with the test case names. It turns out that it is half the problem with the new software releases of the open source agile tools.
If you’re executing tests with NAnt 0.85Rc4 and using the task, you’re going to skip a whole bunch of tests, unintentionally.
Using , only 44 of +1800 test cases get recognized! :0
Using exec and the nunit-console, i get all my test cases recognized [but that’s including the fix i made to nunit 2.2.8].

out with the old:

<target name="executeTest" verbose="true" failonerror="true" >
<nunit2 failonerror="false" verbose="true" >
<formatter type="Xml" usefile="true" outputdir=".\TestResults" extension=".xml" />
<test assemblyname="${test.assemblyPath}"
appconfig="${test.assemblyPath}.config" />
</nunit2>
<property name="testdllname" value="${test.assemblyName}"/>
</target>

in with the new:

<target name="executeTestEx" verbose="true" failonerror="false">
<exec program="${NUnit}\nunit-console.exe" failonerror="false">
<arg value="${test.assemblyPath}" />
<arg value="/xml=.\TestResults\${test.assemblyName}.xml" />
<arg value="/config='${test.assemblyPath}.config'" />
</exec>
</target>

.NET Migration v1.1 – v2.0

So the code migration was fairly straightforward. There’s enough documentation and learned lessons available to get you through that relatively pain-free. Also, striving to keep the code base as straightforward as possible helps tremendously. The deploy was not *as* obvious.

Dev boxes and staging environments deployed without fuss, but as it will be, the LIVE production box will always have something different about it 🙂

If you’re suddenly getting “404 Not Found” errors, check that v2 has actually been allowed to run. Verify with the “Web Service Extensions” section in managing IIS6.

These two urls were particularly useful:
“HTTP Error 404 – File or Directory not found” error message when you request dynamic content with IIS 6.0
IIS 6.0: ASP.NET Is Not Automatically Installed on Windows Server 2003

as part of a response to the question:
ASP.NET doesn’t work Error Code 404 2 1260

hopefully this saves you 30minutes somewhere 🙂

Open Source Advantages

There are just some things that *must* be open source. I usually find it’s the automated utility type software. Compilers, browsers, email clients, communication tools: they must just work and if they don’t. Uninstall, install something else. 🙂

But when it comes to software that automates tasks, and in my environment that includes mostly Nant, you can NOT have a black box. And no surprise, there really isn’t a black box alternative worth paying for. NUnit is my another automation machine.

These automation frameworks can be used in so many different ways that to NOT have the source code is just limiting yourself to all sorts of possibilities, the least of which is:
i..quick bug fixes not in main branch
ii.added functionality due to “custom” requirements [circumstances]

Like today for example. is NUnit 2.2.8 backwards compatible with test case method names [first four letters == ‘test’]? Documentation says yes, but i beg to differ…


modified Oct 05: as of 2.2.1 this compatibility was dropped but the documentation remains outdated.. now where has this debate surfaced before 🙂

Now to have your entire migration process bottlenecked by something silly like that… 🙂