Self-assessment of IT Governance Performance

A broad spectrum of results from a diverse and open audience

On October 25^th, 2005, twenty seven customers and guests of Corporate Information Systems Pty Ltd (CIS) attended a breakfast briefing, at which Infonomics Principal Mark Toomey discussed the intent of the Australian Standard for Corporate Governance of ICT (AS8015).

During the session, Mark asked participants to briefly assess the IT Governance performance of their own organisations, using 12 high level indicators of good governance performance.  The indicators reflect behaviour and performance.  The test assertions are:

G1 Governance system:         You have a system for governance of ICT.

G2 Management compliance:   Everybody understands and complies.

G3 Effective protection:          It protects you from ICT failures in operations and projects.

G4 Inform & engage:             It keeps management and directors properly informed of ICT status.

G5 Dependence understood:    Ongoing business dependence on ICT is well understood.

G6 Continuity & sustainability:  ICT adequately protects business continuity and sustainability.

G7 Business alignment:           ICT capability matches business needs and forward plans.

G8 Resource allocation:          ICT resource allocation matches the needs of the organisation.

G9 Business innovation:          Use of ICT balances business innovation against risk.

G10 Investment value:           ICT investments deliver results as per a formal business case.

G11 Deployment capability:     Demonstrated capability to deploy ICT initiatives matches aspiration.

G12 Acceptable risk:              The business risk of serious ICT failure is understood and managed.

The indicators were ranked on a simple scale that translates into colours on the charts:

4 - n “Absolutely!”  (Very well).   3 - n “Yes…” (Reasonably well)  2 - n “Sort of” (A little)   

1 - n “No” (Definitely not)         0 - n  “huh?” (The organisation generally does not understand this concept)

The 17 complete responses give some insight into IT Governance effectiveness in 11 diverse organisations.  Organisations providing input included local government (LG), a state government department (Gov), a construction firm (Const), health care (HC) providers, wholesale/distribution companies and an infrastructure utility company.  Two responses did not identify their sources.

A Key Performance Indicator shows persistent weakness!

Recent research by world-renowned Australian Academic Peter Weill confirms a link between the extent of management awareness of and compliance with the organisation’s system of IT Governance, the organisation’s success with the use of ICT, and bottom line business performance. History shows that many IT initiatives are technically successful, but fail to deliver business outcomes.  A majority of clearly identified ICT project failures are attributed to problems with business engagement and strategic alignment.

The responses map at left ranks performance from best (left) to worst (right).  Its design provides a 3 dimensional profile where lower scores appear as peaks and ridges, while strong scores appear as valleys.  The common themes of ICT project failure clearly stand out.  As in a recent similar survey conducted among 60 IT Auditors, most attendees at this session ranked management compliance (G2) as being weak.  It is quite noteworthy that this weakness also corresponds strongly with poor scores allocated on other topics, such as deployment capability (G11), business alignment (G7) and resource allocation (G8).  Other areas which indicate broad issues include lack of confidence in protection of business continuity and sustainability (G6), and a lack of confidence that IT investments always deliver value – a result that matches findings in KPMG’s biennial Global IT Project Management Survey, released in September 2005.

Size makes little difference

Small organisations have few managers, and the managers are generally across most of what’s happening, and informal governance should be successful.  But, small organisations often experience limitations in areas other than day to day operations.  Note the weak scores around dependence (G5), continuity (G6) and deployment (G11) for small and medium organisations.

Diverse scores across the rest of the sample, indicate that organisation size probably has little to do with IT Governance performance.  Notice how some organisations appear to have uneven performance – good in some areas, not so good in others – while others exhibit uniformly poor performance.  See also that weakness in informing and engaging of managers in the IT Governance process (G4) tends to be most closely associated with organisations that have generally poor ratings across all the indicators.

Neither Industry, reporting lines, package use nor method of sourcing IT show themes.

In this chart the data is clustered according to the industry from which the response emanated.  Each individual response is plotted, and there is no levelling to reflect that several people may have responded in respect of a single organisation.  Similar plots were developed (but are not presented due to space restrictions) for IT reporting lines, the use of packages vs custom solutions for the main IT systems, and insourcing vs outsourcing.

Across all of these views, the patterns remained quite scattered – with stronger and weaker performers in all categories.  There is room for improvement in all situations.  Lessons may be learned from better performers.

What is your situation?

The results of this mini-survey are consistent with prior surveys and broad industry understanding of IT Governance performance.  Relatively few organisations can claim persistent long term success with ICT – and poor governance is a hallmark of failures.  If you honestly assess your organisation with the 12 indicators, where will you rank?  If you can’t score a solid 36 of the possible 48 points, you may have an unacceptable risk, and you should consider a more formal assessment of your IT Governance performance.