Agile: dropping the Scrum training wheels

Many of us have been using parts of the Agile Manifesto for many years.  What you find in the agile movement,  is a rollup of good practices that many of us long time software development practitioners have found work.  I am a big fan of agile principles, specifically short development iterations, the focus on working code, and the focus on human interactions over documentation.

A couple of years ago we introduced scrum at my current company. Scrum enforces good practices that agile talks about, by introducing daily stand ups, sprints, demos, etc... It also introduces a Scrum Master - a facilitator that enables communications and makes sure that decisions are made.  A good Scrum Master also keeps the process wheels well lubricated to make sure everything moves along well.

We went all "in" on scrum - Story boards, story pointing, estimates - everything pretty much by the book.  We bought Jira + green hopper to help manage everything.  All went pretty well first year.  Every sprint and release cycle we got better at the process. People acted like adults and owned things.  Estimates were made, product managers were brought into the loop.  It also helped tremendously that I hired a Scrum Master who had lots of deep experience in Scrum, and who is a fantastic communicator and facilitator.

Then, we started dropping parts of scrum. Scrum helped us stick to agile principles, but over time, we started dropping some scrum things that we grew out of.  Scrum purists would scream foul, but I have always believed that process should adapt to your people and circumstances, not the other way around.  I believe we do agile, but not scrum anymore.

Here is some things we dropped:

• A full time Scrum Master.  Our Scrum Master is now our director of Software Development.  He still helps make sure that the process works well, but by and large the role of scrum master has been taken over by the software development teams themselves.  We have four development teams of about two developers and one test engineer.  They know enough about the process to move things along themselves.  If they need help, they ask and we get it for them.
• Story Point estimating. Story point estimating blows.  It's hard to get estimates right. You have to keep a history, but when people move around teams, the estimates get all messed up.  Now, we still develop stories and talk about them upfront. We still do exit criteria on stories, we prioritize stories, but we just don't story point them.  How do we estimate how much can get done in a release cycle or sprint? We do rough estimates with everyone involved ("I think we will get this far").  You know what? those work as well as all the time we put in story pointing.  Sometimes we are off, more often then not we pretty much on.
• Sprint retrospectives.  We do retrospectives, but not that often. If the teams are always talking (and they should be), there should be no surprises.

We have kept stand ups, demos, sprints (two-weeks) and we like Jira.  The key is that we are free to evolve the process as we need to, and don't necessarily feel we need to be held up by the scrum model.

There are plenty of things I wish we could do better.  We struggle communicating business workflows well, some developers try to be too heroic and do too much,  and have a tendency to leave the test team behind.  But, here is the key: we talk about these things and we try to do better every sprint.  

I know this is heretical to many of my scrum-devoted friends, but I say: make the process work for you and drop the scrum training wheels if you feel you need to.

Software Development: You don't have a technology problem, you have a people problem

"There is nothing new under the sun"  - Ecclesiastes 1:9

We rarely invent anything new in commercial software development.  Most of what we do is to apply well known patterns and technologies to the domain we are working in.  Now, this does not mean what we do is trivial or easy, but generally, we are not inventing anything new.

So then, why are so many software projects, late, bloated, over budget or failures?  According to Tom Demarco, in his classic book, Peopleware, it's not because of technology issues, its because of people issues.

That's right, you don't have a technology problem, you have a people problem.

Bad communication, egos, ignorance, office politics, inexperience, weird personalities all are bigger problems than technology issues. These things all get in the way of success. Unfortunately, the vast majority of technology leadership struggles with dealing with people issues.

It's easier to deal with a build problem, or to fix a bug,  then it is to figure out why Sally and Mike are not getting along on their project. Technology problems are discrete and deterministic, people are messy and emotional.

I have found that finding leadership that is both technical and people-oriented is very hard, and when you have someone who has both, hold on to them, but have to build these soft skills in your existing staff.  I do this following some loose principles:

1. Technology leadership needs to be technical.  This is critical - you can't lead people unless you know what they do so that  you can participate, lead and provide input.  You have to know how to code.
2. Focus on "doing the right thing", not pleasing some internal stake holder.  When you focus on doing the right thing, politics and bickering takes a back seat.
3. Don't worry about who get the credit.  This is easy to say, hard to do, but when you have a team that really believes this, great things can happen.
4. Remember that software development is a team sport.  Empower the team to make decisions and allow them to be responsible for their actions.  Treat them like adults.
5. Take time to talk and listen.  Spending an hour listening to a person who is struggling with a decision, giving input (not telling them what to do), and helping them make a decision is a critical management responsibility.  I make sure I have time every day to talk to people and listen.
6. Treat people like adults, not like children.  Management is not like parenting.  The people that work for you are adults, treat them as such and respect what they do.

It is true, there is a lot malpractice in this industry related to technology, but the issues all start with people.  People design, implement and test software.  Focus on your people problems first and your technology problems second, and you will have a lot more success.

Cloud-based Infrastructure: If you want to be available, be transient

So, there was outage a couple of weeks ago in the Amazon Web Services east coast region - a power problem affecting one of their zones. They have four (or now five) zones in the east coast, and apparently the others were unaffected.

What was amazing to me was not that there was an outage, but that so many users on the AWS forums had messages which essentially said "Help! I can't connect to my instance!".  Well, yes, the zone was offline for a few hours so instances running in that zone were offline.  But, if you rely so much on a single instance being available,  trouble is coming your way.

Cloud-based infrastructure needs to be thought of as transient - that is, like saying, "I like you but I am not committed to you".  If an instance or service is down, you should be able to shrug your shoulders and move on.

At BuildLinks, we are very heavy users of AWS.  We don't even own a single piece of hardware (except for laptops, a couple of hubs and wireless routers).  We are all "in" on AWS, but we continue to work hard and design our systems such that we are not committed to a particular zone or instance (heck, we even back up our databases to RackSpace outside of AWS, just to be paranoid).  If an instance goes offline, there is another to take its place.  AWS provides some services for doing this (like ELB), and some you have to build yourself.  For example, we use multi-zone web server instances fronted by ELB,  but we had to string together database replication ourselves -  our database is actively replicated to another zone in the east coast and also to the AWS west coast region. We back up our database to S3 and RackSpace every few minutes too.

Yes, we could experience an outage if an entire region goes offline, ELB fails or something of the like, but we would not be completely wiped out, given the replication and redundancy we took time to design and build, and continue to build and add with every product release. Cloud-based infrastructure reliability needs to be an on-going issue.

If you want to use cloud-based infrastructure, you need to take time to design and iterate your services for robustness - not just throw instances up with the assumption that they will be there forever.  It will also keep your blood pressure down and help you sleep. :)

If you have questions, drop me a line.  I am glad to share.

Peopleware

I believe that people who are learning and growing at work, are going to be more productive and positive.  To that end, the BuildLinks technology leadership team is reading Peopleware, by Tom DeMarco and Tim Lister.  It's a classic book on technology team management. Tom DeMarco has written several great books and articles and I am a big fan.

I first read Peopleware when it first came out years ago.  Re-reading it now has reminded me how deeply it has influenced my thinking on technology management.  I don't agree with everything in the book, but over my career I have found that most of their insights are spot on.

How do you respond to surprises?

I've been involved in several start-ups during my career and the one thing I have learned is that things don't go as expected.  A feature doesn't work well, the software doesn't scale, unexpected bugs, or customers who are not happy with some aspect of the product.

What have I learned?  Keep cool.  Don't start blaming people or look for someone to shoot.  It happens to the best of organizations - mistakes happen and things don't go as expected.

My approach for handling unexpected urgent events is something like this:
1) handle the situation in front of you - don't worry about how you got there - just fix it and get past the issue
2) do a blameless post-mortum.  Organizations make mistakes,  but if you are so bent on blaming someone, loosing your cool and throwing a tantrum - the organization becomes risk averse, non-cooperative and plays CYA like a professional sport.
3) don't feel like you need add a layer to the process to prevent whatever from happening again.  A process artifact is usual in place because someone made a mistake years before.  It drives me crazy.  Fix the problem, learn and move on.  You don't have to modify the process every time.

Loosing your cool, throwing tantrums or becoming moody is the sign of immaturity in an organization or individuals.  Amateurs.  Anyone who has been doing technology long enough knows that things happen, and knows enough to learn and move on.

On "Working 40 hours, and mailing it in"

A good friend of mine, who I have a lot of respect for, asked me a rhetorical question on Friday that has bothered me all weekend.  Referring to someone we both know, he asked "What's wrong with working 40 hours a week and just mailing it in?"  The funny thing is that my friend is one of the most passionate and involved guys in technology and technology processes I know.  A family man, who balances his work and life well, nevertheless he does not "just mail it in" when it comes to his career.

Once in my career a few years ago, I was in a "just mail it in" position.  I had a hard time filling 40 hours a week.  I was not challenged.  I was bored. Oh, I had a nice office, was well treated and respected, but not being challenged led me to spend a time finding more interesting things to do - and initiate quiet a few office pranks.  I left for something more challenging, that certainly required more hours - but, and here is the important point: it was a lot more fun!


As a technologist and software developer I have always looked for people in my organization who have a passion and are deeply interested in what they do.  People who write code on the side for fun, are the kind of people I want to work with.  Don't get me wrong, I am a deeply committed family man and have tried hard to spend all the time I could with my children when they were small, and now have a very good relationship with them now that they are college age.  I love spending time with my wife too.  But, my family knows how enthusiastic and interested I am in technology.  I love to build it and use.  I can't imagine having a job and career that I just clocked in for.  That would really blow.

Henry B. Eyring, the son of famed theoretical chemist Henry Eyring tells a story which I can relate to:

'Henry Eyring encouraged his sons to study physics and to prepare for a career in the sciences. Hal dutifully majored in physics at the University of Utah, but one day when he asked his father for help with a complex mathematical problem, it became apparent to Henry that Hal did not share his passion.

“My father was at a blackboard we kept in the basement,” Henry B. Eyring recalls. “Suddenly he stopped. ‘Hal,’ he said, ‘we were working at this same kind of problem a week ago. You don’t seem to understand it any better now than you did then. Haven’t you been working on it?’ ”

Hal said he had not. He then admitted to his father that physics was not something he constantly thought about. His father paused a moment and then, in tender words that released his son to pursue his own professional passion, he said, “You ought to find something that you love so much that when you don’t have to think about anything, that’s what you think about" '  (source)

That is great career advice I share often.  Technology for me is something I think about when I don't have anything else to think about.  It's cool and fun, and that's why I don't just work 40 hours a week and "mail it in".

Hiring Good Coders

"Why the new guy can't code?", a very good article from TechCrunch (no less), got me to thinking about my hiring practices when looking for new programmers.  My favorite and best programmers are those that do it for fun, not as a way to get the bills paid.  The best ones always have a project on the side, because they can't get away from it.  They have cool projects that they can brag about and are eager to share specifics.  During interviews, I look for that.  Those are the guys (and gals) who have the passion to do something right and figure out complex problems.  

As an example, a while ago I hired a new grad out of NC State, here local to RTP.  NC State has a terrific CompSci program, but what convinced me about this fellow was his personal web page and links to applications and algorithmic problems he was working on for fun, on his own.  I hired him, and never regretted him.  That was a job ago, but I still keep in touch for him and would love to have him on my current team, if the opportunity ever presents itself.