Agile change management, governance processes and SOA

Wednesday, February 4th, 2009 | Articles

There has been lots written about how to rationalise SOA based systems development against agile methodologies but i thought i get in my two penneth so here goes, I’ve borrowed some of the concepts from agile methodologies published on the web, so if you want more information about any of the below then i’m sure you can fid it with a quick search:

A change to a SOA design philosophy has to have an inextricable impact on the way new services are constructed and maintained. At present it would seem that the current change management and governance processes in medium-sized and large corportations that i have dealt with are struggling to keep up, however with an ever increasing service orentated workload the current systems will eventually hit a crisis point where quality and throughput of the respective IT departments will be compromised.

With this in mind I have tried to put together this brief overview of agile methodologies and their use in the change management process, with particular reference to the SOA design philosophy. As businesses move up the SOA adoption curve and start to engage in broader enterprise-wide and more strategic engagements with a compounding level of business complexity, managing change with focus on the people and organizational factors becomes increasingly critical.

Successful SOA governance implementations need to focus on organisational change and change management, this is where a great deal of companies are getting it wrong. Adoption of SOA and Agile methodologies isn’t an IT issue, it’s company wide and requires substantial investment in people and time to get it right.

SOA isn’t just about wrapping the existing IT investments into modular services, however this is where most companies start. This will help in plugging the services into new business processes, and enable the services to provide the ability to incrementally replace these legacy applications with new applications with minimal integration challenges and the least disruption to the business operations when such replacement becomes economically feasible. There is more to it that that though, there will need to be a major focus on the impacts on Moving to and SOA based business infrastructure, it will create a significantly new environment for both business and IT. So what happens when the business require new developments that need to plug into the new SOA infrastructure and haven’t changed their internal process to reflect this new way of working? It goes bad and wrong.

An SOA governance group needs to be appointed, there job should be to create an overall organization change plan. The primary focus should be the formation of guidelines which should include details on subjects like the ones below:

  • Change and Release Management – when, what and by whom can services be changed, what versioning strategies should be employed
  • Requirements and Quality Management – creation of a process that ensures that services are developed in alignment with business requirements
  • Design and Construction – ensure sound design and development principles are adhered to for maximum service reuse and reliability
  • Process management – ensure each project follows the established governance policies through project clearly defined frameworks
  • Service management - The creation of SLA’s and the monitoring of the performance of each service

The transition to SOA is not easy. I think that most companies that are trying will realise soon that implementing SOA means far more than addressing such tangible and relatively straightforward problems as integration and consolidation of applications. The burden of SOA implementation will fall squarely on the management and organisational side of the business, where new skills, processes and responsibilities have to be introduced along with attention to the business requirements, and to the much tighter relationship between IT and business that SOA brings.

Within this push for SOA, There are several fundamental questions that will need to be asked before a full transition can occur:

  • Effectiveness of methods - what modeling, testing, version control, and design methods are going to be used, and how effective, efficient, and proven are they
  • Work domain expertise - are skilled professionals available in the various domains, including all aspects of business and technology that could be affected by changes to process and workload increases.
  • Availability of skilled professionals - the newer the technology, tools, methods, and domain, the smaller the pool of skilled professionals.
  • Stability of implementation technology - the newer the technology, the lower the stability and the greater the need to balance the technology with other technologies and manual procedures
  • Stability and power of tools - the newer and more powerful the development tool, the smaller the pool of skilled professionals and the more unstable the tool functionality

These aren’t problems themselves but do highlight that the company must do a great deal of groundwork before new services are implemented. The process which manages change must be adaptable enough to cope with a potential workload increase, changes in development practices, and everything else that comes with producing and supporting significant business-critical, enterprise systems in an SOA environment. The process also has to enable the dissemination of information by focusing more on people rather than process.

From the view i’ve had of companies that are going through this process, I have identified some of the common issues that stand in the way of progress, apologies if these points are unjustified or ill informed, it’s just my experience:

  • Management is ill informed about new procedures and practices
  • Ad hoc releases of individual fixes are expected by the business
  • Loose project management, or no project management for most changes.
  • Labour intensive release processes for business critical applications
  • Testing not unified or standardised
  • Changes to requirements can be informal and not documented or tested against.
  • Priorities are not set or communicated
  • Time wasted building multiple releases.
  • Lots of manual documentation for each release.
  • Duplication of documentation required.
  • There doesn’t seem to be an ‘owner’ for every change, or they seem to be invisible whilst
  • implementation is in progress.
  • Misunderstandings in requirements and ambiguity lead to unnecessary work being carried out.
  • There seems to be several ‘fire and forget’ changes raised that stay on the system well past their sell-by date.
  • Many of the development processes are uncontrolled or undocumented
  • The inputs and outputs are either unknown or loosely defined
  • Business owners are disengaged from the process
  • Quality control is not clearly defined or empowered.
  • The focus is on process, not on service or people

Does that sound like your business? If so, an outline of an alternative change management process
is below so don’t wory.

A change management process is there to help us do three things:

  1. Be more responsive to requests for change
  2. More quickly and easily implement those changes
  3. Be able to support and manage a greater workload

At the beginning of each iteration (which lasts typically between 1-3 weeks), Conduct a planning meeting; the project manager, developers, testers, DBAs and customers (business owners) get together in a room and look at the existing back-log of to be implemented requests (Change requests, sometimes called ‘stories’).

  1. Developer’s converse with the customers and attach estimates to each request.
  2. The customer decides which requests they want implemented during the next iteration. They do this based upon the iteration length, the estimates for each request, and their stated priorities about which requests are the most important.
  3. The customer may completely change priorities from one iteration to the next. This is okay as long as development and project management agree the result can be successfully implemented within the scheduled period.
  4. Once customers decide on the change requests to implement for an iteration, developers sign up for change requests to implement and may work only on CR’s for that iteration.
  5. Developers talk with customers to elaborate the details of a CR, if required. To make sure thes requirements are captured fully, customers write acceptance tests for each CR (basically a use case); this becomes part of the requirement doc.
  6. When a request has been implemented, integrated and tested, the corresponding change request is placed upon a chart that shows it as completed. The chart also conveys the number of passed versus still-to-be-passed test cases.
  7. During iteration, customers may reprioritise unimplemented use cases, and remove or replace use cases provided that the overall estimated effort remains within the planned iteration end-date.
  8. There is a brief daily stand-up meeting where each team member quickly gives the status of what they have been working on since the previous day’s meeting. If an issue threatens on-time completion of their task, the issue is raised to the customer or project manager, either with the onsite customer or in the daily stand-up meeting with the project manager.
  9. The customer then decides the course of action to take, based on the verbally communicated risk and impact, and the possible set of alternatives.
  10. At the end of the iteration (and for each build during the iteration, when feasible) all the completed (and preferably automated) customer acceptance tests are executed to ensure that the built result conforms to what the customers specified in the tests.

The premise behind this methodology is that instead of saying no to a change request, the answer should simply inform the customer of the associated risk and impact upon the currently agreed upon cost, schedule, scope and quality, and then let the customer make an informed decision.

The above outlined method enables projects to be more responsive to requests for change by:

  1. Using short and frequent iterations to minimise the time between specifying a requirement and seeing it implemented so that adjustments to functionality and priorities can be recognized sooner rather than later
  2. Requiring developers and customers to communicate and work together so that change-control decisions can be made and communicated quickly and communicated face-to-face with minimal waiting and documentation
  3. Allowing and accommodating changes in scope and priorities via a highly collaborative agreement process that informs the customer of impact and risk, and puts customers in control of the scope and priorities
  4. Authorising and empowering team members to correct problems with the code’s behavior and structure without having to suffer delays waiting for formal change proposal, review, and authorization of such corrective changes

These four things together are what help transform traditional unidirectional change flow into a more active customer collaboration environment.

More detail on the mechanics of the process:

  • Before the planning meeting, all customers need to submit any business requirements that they would like to be considered for release within the next iteration. If that can’t be done then a placeholder would be used instead. Each requirement needs a priority set by the customer. A Template requirement doc will need to be produced as a guide.
  • If multiple customers exist then arbitration will be needed by the project or change manager to find a priority order that is acceptable to each customer.
  • Each requirement is reviewed in turn and allocated a approximate time in man days for completion of the work. This is to include design, implementation, testing and any documentation/training needed.
  • The customers may add or remove requests from the stack at any time.
  • Once customers decide on the change requests to implement for this iteration then an approximate release date is set. This should be not too far in the future and should not contain too many large changes to the codebase.
  • The release doc should try to detail the acceptance tests that will be used to test the finished piece of work, these should be further refined during the meeting but can be changed at any time during the implementation as long as development and project management agree the result can be successfully implemented within the scheduled period and that if other work is affected then the customer must agree this change in the scope of deliverables.
  • Developers are permitted and trusted to make refactoring changes, or code clarity improvements (or coding standard violation corrections), and to fix any problems encountered, provided it doesn’t impact the schedule of their current use case.
  • A Change manager must ‘own’ this process and the onus is on them to make sure that the customers are informed of any changes to the schedule, scope or quality of the requested change. Conversely any decisions made unilaterally by any of the implementation team must be fed back to the change manager and the customers, this should be done as soon as an issue is identified.
  • Each application has a product manager, their role within the change management process is to facilitate, nudge, and mentor the customers in carrying out their partnership responsibilities. The Product Manager is also responsible for the participatory decision-making process on the customer side, just like the project manager is on the development side, it could even be the same person. Neither one’s primary remit is to make all the decisions, but they have the power if need be.

One issue could be the allocation of resources to individual products. You need to come up with a strategy that properly reflects the needs of the business. One solution could be that we try to calculate the relative value to the business of each change within each iteration, this is very hard in practice purely because it is very difficult to put a short term value on a long term perceived business gain.

The key success factors of agile change management are the use of iterative and incremental development with short feedback cycles, and close collaboration with frequent face-to-face interaction between developers and customers. This, coupled with the new SOA centric view of services can provide an excellent opportunity for companies to leverage their systems and resources in new and interesing ways and to provide a stable and dependable IT foundation to build upon.

No comments yet.

Leave a comment

Latest Article

Do I need a CMS?

A great deal of our clients come to us wanting to be able to update their own website. Some clients are sick of waiting for days or even weeks on their "web guy" .. Read more...

Get the newsletter

Search