You know the saying "Opinions are like assholes - everyone has one"?
Well, project management processes and tools are similar - everyone has their own special setup. So this article isn't intended to give you The Answer™; rather give you some insight into methods that I've seen succeed.. and fail.
The elephant in the room - Agile
This term is so often misused that I ask candidate Project Managers to describe what Agile means. It's interesting hearing the results.
Most describe it as I would - iterative and incremental - but the devil is in the details. I've witnessed one particular bastardized version of this more than once:
Doesn't Agile mean that we can make up requirements as we go along?
You certainly can, but this comes at a cost. It's impossible to know when you'll be complete. You're trading flexibility (and arguably a better product) for a defined timeline and cost.
If dates are critical to you - say to launch at a particular event, or meet your shareholders quarterly revenue target - you're going to need more definition and some estimation upfront.
Simply dropping scope as a deadline approaches may not be acceptable.
Kanban is awesome - sometimes
There are circumstances where continuous iteration works really well - one example is on long-running "forever" projects where there are no deadlines or external pressures; just a customer who wants to build or improve their product.
In this case, add all the tasks you know about to your backlog and pull the highest priority ones into your kanban. Every day developers can pull from the top, build the highest priority that day and report on progress.
The customer gets exactly what they want, when they want it. Do they need to interrupt the developers for a big new sales engagement or demo? Pivot on a dime for a new feature you just dreamt up?
You'll need a hyper-engaged customer or product owner though, and they better be sitting with the team, or at a minimum attending every standup.
It's a great environment to work - customers have exact control over scope and schedule and budget. They get almost instant feedback on their ideas, and see results quickly.
There are downsides though - you have no idea in advance what the costs will be, and the customer truly has to be part of the team. Even a few days without eyes on and the team will start drifting off course.
I have a rule that if a customer misses two standups or one weekly demo the flag goes up to management - and it's raised as a red blocking issue on status reports.
And when I say you have no idea what the costs will be - without estimation other than a WAG each morning you can easily find yourself doing the highest priority task every day for your customer.. doing the right thing.. and at the same time burning through their budget like a forest fire.
I've seen a large software project that pivoted wildly every week - and spent millions with nothing to show for it. But, hey, it was a tax write-off for a billion-dollar company; NBD.
Scrum is just short bursts of Waterfall
This usually prompts comments from the theoretical PMP crowd, and I'll leave you to ponder articles like A Sprint is Not a Mini Waterfall if you really care about semantics. I'm no accredited PMP, I've just seen a lot of software being made.
To me, and most developers, waterfall simply means you have a complete definition up front, and you can build it without distraction from requirements changing.
So, just like a Scrum sprint then.
The obvious advantage is that with more definition you have more estimation and are more likely to hit your date. Not 100% likely of course, but we're in the same ballpark at least.
With sprints - or short waterfall projects - the customer needs to be highly responsive, but only during the initial planning phase. Less so during implementation, because the requirements are frozen.
Which you choose depends on your priorities and your culture. I've met developers who prefer a customer who pivots regularly, and others who want stability and change freaks them out. Hire them into the wrong environment and you'll soon know.
Even with sprints, shit happens
No changes during sprints is a great policy - until you have to make one.
Even product companies have to respond to high-level-interrupts. Everyone had better take it seriously when the CTO of your biggest client calls your CEO. Or your top salesman is sitting on a potential seven figure deal that needs an answer.
We're all in this to make money.
(ASIDE: Make no mistake, we're there to make the company more money than we cost. I'm all for perfect process and rules, but pragmatism is #1.
I've said this to many developers - until you've built and run your own company and paid a mortgage with the profit, you don't really understand why sometimes you need to write second-class code or take a shortcut with process)
Anyway, to get back on track - when interrupts come, how do you manage them? Communication of course - Tell Them Everything.
Do you have a Change Control Board? Yes you do! If not a formal one, your VP/CTO/COO/CEO is your CCB.
So, tell them (all of them) immediately what the effect will be. What's the cost and other consequences such as timeline?
Do this every time, and you'll soon stop interrupts for trivial matters. If requirements change during a sprint someone important and scary is going to know about it.
Communication is the most important issue no matter which SDLC process you follow.