Audit of the Halifax-Class Modernization/Frigate Equipment Life Extension (HCM/FELEX) Project

March 2011
7050-50 (CRS)

Reviewed by CRS in accordance with the Access to Information Act (AIA).
Information UNCLASSIFIED.

Table of Contents

If you would prefer to view this page in its original form, please download the PDF version: Download Report (PDF 217kB)

Caveat

This audit is not intended to assess the performance of contractors; rather, it is an internal assessment of processes and practices within the Assistant Deputy Minister (Materiel) (ADM(Mat)) organization.

Acronyms and Abbreviations

ADM(Fin CS)
Assistant Deputy Minister (Finance and Corporate Services)
ADM(HR-Civ)
Assistant Deputy Minister (Human Resources–Civilian)
ADM(IM)
Assistant Deputy Minister (Information Management)
ADM(Mat)
Assistant Deputy Minister (Materiel)
CANOSCOM
Canadian Operational Support Command
CFSS
Canadian Forces Supply System
C Mgt O
Community Management Office
CMS
Chief of the Maritime Staff
COS Mat
Chief of Staff – Materiel
CRS
Chief Review Services
CS
Combat System
CSC
Canadian Surface Combatant
CSI DAB
Combat System Integration Design and Build
DFA
Director Financial Accounting
DGMEPM
Director General Maritime Equipment Program Management
DGMSSC
Director General Materiel Systems and Supply Chain
DMG Compt
Director Materiel Group Comptrollership
DMGHR
Director Material Group Human Resources
DMPOR
Director Maritime Policy Operations and Readiness
DMPP
Director Materiel Policy and Procedures
DND
Department of National Defence
DOORS
Dynamic Object Oriented Requirements Suite
DRMIS
Defence Resource Management Information System
DWP
Docking Work Period
ERP
Enterprise Resource Planning
FAA
Financial Administration Act
FELEX
Frigate Equipment Life Extension
FY
Fiscal Year
HCM
Halifax–Class Modernization
HMCS
Her Majesty’s Canadian Ship
HR
Human Resources
IOC
Initial Operating Capability
IRM
Integrated Risk Management
IS
Integrated Schedule
ISLA
Integrated Staffing Log Application
ISSC
In–Service Support Contract
MARLANT
Maritime Forces Atlantic
MARPAC
Maritime Forces Pacific
MLR
Mid–Life Refit
MSC
Multi–Ship Contract
O&M
Operations and Maintenance
OPI
Office of Primary Interest
P&A
Price and Availability
PAG
Project Approval Guide
PMM
Project Management Manual
PMO
Project Management Office
PMPR
Project Management Personnel Resources
P, O&M
Personnel, Operations and Maintenance
PWGSC
Public Works and Government Services Canada
PS
Performance Specification
RFP
Request for Proposal
RMP
Risk Management Plan
SOR
Statement of Requirements
SRB
Senior Review Board
TRP
Tiered Readiness Program
VCDS
Vice Chief of the Defence Staff
WBS
Work Breakdown Structure

Results in Brief

Overall Assessment

To mitigate the potential loss of operational capability and ensure the timely delivery of all ships, the project office needs to implement better risk management and control processes.

In a 2007 risk analysis [1] Chief Review Services (CRS) identified the HCM/FELEX project as warranting audit due to the high value of the project and the large number of inter–related projects.

To complete the navy’s planned mid–life upgrades of the 12 Halifax–class ships first delivered between 1991 and 1996, the HCM/FELEX project budget of $2.8 billion was approved in September 2008. As the most critical component of the $4.2–billion [2] HCM program, the HCM/FELEX project is the ship-level design integration and risk mitigation authority across the HCM program with emphasis on schedule coordination.

The three principal contracts include a $1.3–billion Combat System Integration Design and Build (CSI DAB) contract and two Multi–Ship Contracts (MSC)—one on each coast—to overhaul the frigate mechanical systems.

Findings and Recommendations

Project Schedule. Planned delivery of 12 modernized frigates within 75 months could be at risk. Key indicators of the potential slippage are an arbitrary18-month mid-life refit (MLR) period for each frigate, | | | | | | | | | | | | | | | | | | | | | historical delays with the Tiered Readiness Program (TRP), and benchmarks against other ship upgrades. Given the developmental nature of the MLR and the aggressive schedule for the lead ships on each coast, the current delays are not unexpected. However, it is the view of ADM(Mat) that learning curve efficiencies in subsequent MLRs could recoup the schedule slippage. Should schedule slippage occur in the future, the sponsor’s five scheduling criteria need to be prioritized, with consideration of industry capacity, navy resources and the impact on project elements. The current contract clauses provide flexibility to revise the project schedule.

It is recommended that a priority be established for the schedule criteria and that sensitivity analysis be performed to address potential slippage.

Contract Basis of Payment. The CSI contract basis of payment does not provide optimal value for the Crown. The Department of National Defence (DND) allowed a reduction of $48 million in holdbacks during a Request for Proposal (RFP) amendment process. As well, based on the current terms of payment, | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Even though a competitive process was followed, only one bid was received. However, DND chose not to exercise clauses in the CSI DAB RFP allowing price justification, due to concern for project delay and to maintain the integrity of the contract award process.

It is recommended that the Project Approval Guide (PAG) be revised to ensure that key decisions prior to contract award affecting cash flow or holdback changes be approved by the Senior Review Board (SRB).

Cost and Schedule Control. The Defence Resource Management Information System (DRMIS), the departmental Enterprise Resource Planning (ERP) system for materiel acquisition and support, has not been fully utilized due to some technical shortfalls. For example, DRMIS does not allow contract commitments greater than five years. [3] Although mandated by navy business rules as the record for project–related cost and schedule, the information maintained in DRMIS was incomplete.

It is recommended that DRMIS be modified to accept long-term contract commitments and that DRMIS be utilized for HCM/FELEX project cost control once technical issues are resolved.

Requirements. Not all requirements can be traced in the Dynamic Object Oriented Requirements Suite (DOORS) tool used by the HCM/FELEX project. In an audit sample, at least 33 percent of the Statement of Requirements (SOR) could not be linked to the CSI contract performance specification (PS), making it difficult to ensure all navy requirements are met by the contractor. There is no policy on the use of DOORS in DND.

It is recommended that a policy be established for the use of DOORS in order to improve the traceability of operational requirements.

Navy Warehouse Space. Warehouse construction and lease costs of up to $25 million could be avoided if supply managers review and dispose of dormant inventories currently consuming warehouse space.[4]

It is recommended that navy supply managers be directed to examine dormant stock in Esquimalt and Halifax to optimize the use of warehouse space.

Project Estimates/Validation. In the case of the HCM/FELEX project, a capped RFP was released to industry before project approval. The PAG does not require capped RFPs to be cost-validated by the Assistant Deputy Minister (Finance and Corporate Services) (ADM(Fin CS)) until the project expenditure approval stage. Increased rigour in the estimation of the total equipment life cycle costs is also needed. For example, interim Halifax–class capabilities will offset the operations and maintenance (O&M) cost of another decommissioned fleet by as much as $615 million but had not been included in the O&M estimates.

It is recommended that the PAG be amended to include cost validation for capped projects prior to RFPs and consider offsets from other fleets as part of approval documents.

Note: For a more detailed list of CRS recommendations and management response, please refer to Annex A—Management Action Plan.

Introduction

Background

The HCM/FELEX project is responsible for ship–level design integration and risk mitigation of all elements of the HCM.

In June 2002, prior to the definition phase, the MLR of the Halifax–class frigates, known as the FELEX project, was estimated to be $1.1 billion.

To reduce integration risks, the October 2005 FELEX SRB endorsed an increase to the project scope to include several existing stand–alone combat system enhancement projects such as Halifax–class Radar Upgrade, Multi–Link, Halifax Modernized Command and Control System, and Identification Friend or Foe Mode S/5.

A budget of $2.8 billion for the HCM/FELEX project was approved in September 2008. This included a $1.3–billion CSI DAB contract that was subsequently awarded in November 2008. To date, the CSI design has been completed and several reviews have taken place to assess if the designed items are ready to proceed into production and testing. The final critical design review took place in July 2010 and the first ship MLR commenced in October 2010.

The aim of the HCM/FELEX project is to complete the Halifax–class MLR by December 2016 and return fully operational ships to the Operational Authority by the end of January 2018.

Objective

The objective of the audit is to provide assurance that the HCM/FELEX project has effective governance, risk management and control frameworks in place to ensure a cost–effective and timely operational capability.

Scope

The audit scope included:

  • Activities from project inception to future planned activities, including the CSI In–service Support Contract (ISSC) estimates in the funding approval document; and
  • HCM/FELEX project’s inter–relations with other HCM stand–alone projects.

The scope does not include the contractors’performance, as it is the responsibility of Public Works and Government Services Canada (PWGSC).

Methodology

  • Interviews with staff at Chief of the Maritime Staff (CMS), Vice Chief of the Defence Staff (VCDS), ADM(Mat), and the Maritime Forces Atlantic (MARLANT);
  • Examination of project documents—SOR, Project Profile and Risk Assessment, Major Crown Project Inter–departmental Oversight Committee report and project charter;
  • Review of ADM(Mat) and VCDS policies and procedures;
  • Review of contract management practices that relate to Financial Administration Act (FAA) Section 34 Payment Certification;
  • Benchmarks on three ship upgrade projects, including two allies and one previous DND project;
  • Data Analysis—Financial Managerial Accounting System, DRMIS, DOORS, Resource Data Information Management System, Integrated Staffing Log Application (ISLA) and Canadian Forces Supply System (CFSS); and
  • Site visit with end users at MARLANT.

Audit Criteria

Please refer to Annex B for audit criteria.

Findings and Recommendations

Project Schedule

Project delivery within 75 months could be at risk due to an aggressive schedule without prioritized criteria.

Good Practice

  • The HCM is governed by a Committee of Sponsors that includes ADM(Mat), CMS, ADM Acquisitions from PWGSC and the Chief Executive Officer from each of the principal contractors.
  • Chaired by the project manager of the HCM/FELEX project, the Canadian Industry Integrated Project Team meets every three months with a current focus on an integrated schedule.

Compressed Schedule

The navy intends to complete the refit of the 12 Halifax–class frigates within 75 months from October 2010 to December 2016.[5] There are indicators that suggest schedule slippage within the planned 75–month time frame will likely occur.

  • Schedule Rationale. The 18-month MLR period for each ship was directed by the project sponsor, CMS, as the maximum time for ship availability. Rather than a bottom up estimate of all activities within a MLR, the current schedule includes 12 equal 18–month periods for all frigates undergoing MLRs. Although the project is maintaining the current planned schedule for the first ship, additional time to address lead ship issues may have been beneficial. Schedule efficiencies for follow–on ships should be planned as best practices and lessons learned are transferred from one ship to the next. Currently, an integrated schedule (IS) is being developed for each frigate which captures the work breakdown structures (WBS) for industry and the two navy formations. Once in place, the IS should provide a better estimate for the timeline of the MLR. However, the IS will not provide DND visibility into the contractors’ resource loading or productivity settings. Even though the MSC contractor is accountable for scheduling activities during the 18–month MLR period, over-reliance on the contractor, and a lack of visibility of the detailed schedule parameters, could increase risk of schedule slippage.
  • | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The CSI DAB contractor is responsible for procuring, designing and integrating new sub-systems on the 12 frigates. | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |[6] | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |[7] | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Although delays should be expected on the lead ship MLR, it is the view of ADM(Mat) that, as experienced with other ship refit programs, there will be opportunities to recover schedule slippage when applying subsequent MLR learning curve efficiencies.
  • Tiered Readiness Program. The Halifax–class TRP, a Maritime Command responsibility, has consistently experienced delays on the East Coast. The average time required for completion of five TRPs following the frigates’ docking work periods (DWP)[8] from 2006 to 2008 was 11.4 months—28 percent longer than the planned nine months.[9] Although the level of effort for a MLR is fivefold that of a DWP, the HCM/FELEX project schedule first assigned nine–month TRPs to all 12 frigates but later revised the first three frigates to 12–month TRPs. Historical TRP schedule slippage indicated that the nine month TRPs could be under–estimated. The project has planned some mitigating measures to reduce duplication of effort and limit schedule impact. For example, the contractor will conduct training prior to the TRP and ship acceptance activities will offset some of the force generation readiness work.[10]
  • Benchmarks. Other ship upgrade projects with 18–month planned MLR periods per ship have experienced 36 percent[11] schedule slippage.
    • The Australian Guided Missile Frigate fleet capability upgrade project was delayed 27 months from the original 68 months;
    • Tribal–class Upgrade and Modernization Project fleet was delayed 32 months from the original 55 months; and
    • The United States Aegis Cruiser upgrade project was on time and on schedule for its first ship delivery.

Sponsor Schedule Criteria

As the sponsor of the HCM/FELEX project, CMS has established five criteria to provide direction on the frigates’ refit schedule. However, the five criteria have not been prioritized. Instead, the HCM/FELEX project charter stated that the refit schedule will be a compromise of the five criteria:[12]

  1. Align with the 48-month maintenance cycle for each frigate;
  2. As closely as practical, the oldest of the class will be refitted first and newest last;
  3. All 12 MLRs will be completed within an 81–month period[13];
  4. | | | | | | | | Halifax–class ships will always be available per coast; and
  5. TRP overlap on the same coast is to be avoided if possible.

No recent sensitivity analysis has been performed by Director Maritime Policy Operations and Readiness (DMPOR) to assess the impact on the refit schedule if each criterion was to be met. The first three criteria have been incorporated in the current schedule, but not the 4th and 5th criteria. CRS conducted sensitivity analysis on the fourth and fifth criteria. Results showed that the current refit schedule could be delayed by as much as | | | | | | | | | | | | | | | | | | | | | | in order to fully satisfy both the fourth and fifth criteria. In conjunction with industry capacity and navy resources, prioritized schedule criteria would assist in the timely revision of the schedule should a need arise.

Halifax–class Life Expectancy

Although the project is under pressure to compress the 12 refits into a 75-month time frame, there may be flexibility to extend the MLR schedule beyond 75 months. The MLR is intended to extend the Halifax-class life expectancy by 15 years. As shown in Figure 1, for | | | | | | of the 12 ships, the replacement frigates are not planned to be delivered until 16 to 18 years after the refit. | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Both MSC and CSI DAB contracts provide some flexibility to the Crown to revise refit starts due to operational requirements.

Bar graph of the Halifax-class life expectancy

Figure 1. Halifax–class Life Expectancy. The Halifax–class life cycle will range from 30 to 42 years with an average of 35 years, longer than the original expected 30 years. The Halifax mid–life (yellow bar) represents the number of years between the original commission and refit completion; the orange bar represents the lesser of 15 years (the expected life extension) or the number of years from mid–life to replacement; and | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The data is summarized in Table 1:

Table Summary: The names of the Halifax-class frigates are listed in the left-hand column. Read across the row for each frigate to determine the number of years required for the Halifax mid-life, years of Halifax combat system life extension and the gap between the Halifax life extension and the new frigate replacement.

Table 1. Halifax-class Life Expectancy.
Halifax–class Frigates Halifax Mid–Life Years of Halifax Combat System Life Extension | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
HALIFAX 20 12 | |
CALGARY 18 12 | |
MONTREAL 19 13 | |
WINNIPEG 18 14 | |
FREDERICTON 20 14 | |
VANCOUVER 21 15 | |
CHARLOTTETOWN 19 15 | |
ST–JOHN’S 19 15 | |
OTTAWA 21 15 | |
VILLE DE QUÉBEC 22 15 | |
REGINA 22 15 | |
TORONTO 24 15 | |

Recommendation

It is recommended that CMS establish schedule criteria priority and that the HCM/FELEX Project Management Office (PMO) perform sensitivity analysis to address slippage potential.

OPI: CMS and HCM/FELEX PMO

Contract Basis of Payment

CSI contract basis of payment does not result in optimal value for the Crown.

Prototype Cost

Based on the current terms of payment, | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The $1.3–billion CSI DAB contract is broken down into a series of | | | milestone payments.[14]

  • | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
  • | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |[15]

This could reduce DND leverage to ensure that ongoing work is satisfactory for all ships and cost overruns are mitigated.

Bid Evaluation

Even though a competitive process was followed that involved three vendors, a single bid was received for the CSI contract. The bid closure date was extended, but no other bids were received. Although specified in the RFP, price justification[16] and the negotiation of the contract cash flow did not take place. The PMO decided to

  • avoid further project delays associated with price justification; and
  • maintain the integrity of the contract award process by not negotiating cash flow | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |

Good Practice

The final holdback to be released after the warranty period provides an incentive to the contractor to fulfill warranty obligations.

CSI Contract Holdbacks

The decision to amend the CSI RFP to reduce total contract milestone holdback by $48 million and the final milestone holdback by $41 million was not discussed at the SRB meetings. Currently, the PAG does not require that cash flow changes be discussed at the SRB. The original RFP called for initial holdbacks of $176 million. Decreasing the holdback reduces the incentive for the vendor to complete important milestones and carry out warranty commitments. | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On average, holdbacks were 14 percent in five other DND milestone-based contracts[17] ranging from $13 million to $380 million.

Recommendation

It is recommended that VCDS revise the PAG to ensure that key decisions prior to contract award affecting cash flow or holdback changes be approved by the SRB.

OPI: VCDS

Cost and Schedule Control

DRMIS is under–utilized for the project’s cost and schedule control.

Director General Maritime Equipment Program Management (DGMEPM) business rules[18] stipulate that DRMIS is the system to be utilized to monitor project schedule and cost. DRMIS project–related reports are to be standardized to monitor progress. A standard WBS template has been developed for all DGMEPM projects within DRMIS, which can be configured to the particular requirements of individual projects. Since the DRMIS roll-out in April 2010, there have been a number of technical issues that have prevented the full utilization of this tool.

Cost Planning

The PMO has entered insufficient information in DRMIS to create earned–value reports such as Cost Performance Index and Schedule Performance Index to monitor cost and schedule. The approved project budget in September 2008 was $2.8 billion; however, only $1.8 billion was entered in the DRMIS cost plan at the time of the audit. Instead, the project has been relying on a separate spreadsheet for cost planning which includes complete cost elements totalling $2.8 billion. As is done in other major navy projects, the full project budget should be entered in DRMIS.

Although the Department is moving towards implementing long–term support contracts, ADM(Fin CS) requires a formal request to allow the entry of contract commitments that are longer than five years in DRMIS. Without this request, the HCM/FELEX project is unable to create a commitment for the 10–year $1.3–billion CSI DAB contract. To enable all project and contract managers to enter long–term commitments in DRMIS, ADM(Fin CS) policy staff is willing to lift the five–year constraint on commitment durations for some general ledger accounts. As ADM(Mat) is the DND lead for procurement, it would be appropriate that this branch identify those general ledger accounts that would require long–term commitments.

Schedule Control

The DRMIS project systems module allows projects to enter durations of activities from project inception to implementation. The project or external organizations are not able to generate accurate DRMIS reports to monitor schedule performance because the information is incomplete.

  • Of the 2,195 HCM/FELEX WBS elements and activities in the DRMIS project systems module, only 63 percent contained forecast data; and
  • Of the 345 HCM/FELEX WBS elements and activities completed at the time of the audit, 5 percent contained actual duration data.

Some monitoring is occurring outside of DRMIS as the PMO used Microsoft Project for higher–level schedule planning. However, only approximately 100 WBS elements had been established—much fewer elements than in DRMIS—and none was updated with actual duration data. The IS is planned to provide schedule control of the contractors’ and the formations’ work during the refit when it is put in place for each ship.

Recommendation

It is recommended that ADM(Mat) identify those general ledger accounts in DRMIS that could require the entry of long–term commitments and the HCM/FELEX PMO maximize the use of DRMIS for cost control once the technical issues are resolved.

OPI: ADM(Mat) and HCM/FELEX PMO

Requirements Traceability

Requirements traceability between the SOR and contract PS needs improvement.

Traceability

Currently, the Department has not established a policy or guidance on the use of DOORS, a requirements traceability tool used by DND. As a result, project staff were not given instructions on when and how to use DOORS in a consistent manner. In the HCM/FELEX project, there was poor traceability from the source of the operational requirement to the SOR and from the SOR to contract PS. As well, the MSC component that includes engineering change installation work worth $258 million was not found in DOORS.

A sample of 221 line items representing 32 percent of the complete SOR showed incomplete links from the SOR to the CSI contract PS.

  • At least 33 percent of the system requirements contain no linkages to the contract PS; and
  • None of the SOR items in the system-level measures and training contain linkages to the contract test and trial plan and training plan.

Performance Specification Interpretation

During the CSI contract RFP stage, the meaning of the terms | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |[19] | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Clear definition of requirement terminology is needed in the future.

| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To date, there has been $47.4 million released from contingency funds to address derived requirements from the PS interpretation. To ensure requirements are adequately transposed to the PS, the Project Director staff deems it important for CMS to sign off on the PSs as currently only ADM(Mat) is required to do so.

Due to the nature of PS, particularly in a mid–life upgrade during which new requirements can be derived during implementation, a requirements trade–off analysis may be needed to stay within a constrained budget. However, this has not been done by the HCM/FELEX project. The PAG only suggests requirements trade–off analysis in the definition stage; however, PS interpretation issues may lead to trade–off requirements in the implementation phase. It is suggested that the PAG be amended accordingly.

Recommendation

It is recommended that ADM(Mat) establish a DOORS policy and guidance for DND.

OPI: ADM(Mat)

Navy Warehouse Space

Additional warehousing requirements could cost as much as $25 million to store HCM/FELEX materiel.

Warehousing costs of up to $25 million could be avoided if navy supply managers review and dispose of dormant inventories currently consuming warehouse space. In spite of the efforts of the HCM Disposal Focus Group in 2009 and 2010 to dispose of dormant repairable reserves,[20] the navy bases at Esquimalt and Halifax will not be able to meet the warehouse space requirements for stripped–out equipment and new materiel at the peak period of the project MLR. The East Coast base needs to provide warehouse space for up to five ships and the West Coast base four ships starting 2015. Currently, the East Coast can support three ships and the West Coast two ships. It is projected that additional leasing and construction of more space could cost the navy up to $25 million.

The CRS Audit of Inventory Management: Surplus & Disposals approved in February 2010 found that 57 percent of stock in the navy’s Halifax warehouse has not moved in six years. The ADM(Mat) management action plan stated that the supply managers would be trained to review excess materiel using the new Defence Resource Planning application starting in March 2010.

Currently, the ADM(Mat) inventory management priority is the transfer of centrally managed spare parts from the CFSS to DRMIS. Removal of surplus stock is a lower priority than ensuring there is sufficient stock to maintain operations. The audit of HCM/FELEX project confirmed that although some effort was directed at reviewing repairable reserves, no action has been taken to review dormant stock to date.

Recommendation

It is recommended that ADM(Mat) direct supply managers to examine dormant stock in Esquimalt and Halifax to optimize the use of warehouse space.

OPI: ADM(Mat)

Project Cost Estimates/Validation

Total equipment life cycle cost estimates for the HCM/FELEX project require more rigour.

Personnel, Operations and Maintenance Cost Estimates

Capital projects are required to include Personnel, Operations and Maintenance (P, O&M) costs as part of the total cost of ownership of the fleet. Accordingly, the 24–year $19–billion P, O&M estimate should have included:

  • Offsets[21] of up to $615 million to the navy’s future operating costs as a result of the Tribal-class decommission. | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |[22] | | | | | | | | | | | | | | |[23] | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
  • Additional personnel costs of up to $224 million over the eight–year refit period[24] because the navy will continue to employ the same number of sailors during the MLR.

In–Service Support Contract Cash Flow Over-Estimation

The 17–year $548–million ISSC[25] from FY 2008/09 to FY 2024/25 to provide maintenance support to the Halifax–and Tribal–class vessels’ command and control systems included maintenance estimates for the years that the Tribal–class vessels were expected to be decommissioned. Had the ISSC project office taken the Tribal–class decommissioning into account, the ISSC estimate approval could have been reduced by $18 to $31 million.[26]

Spare Parts Estimates

| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To calculate spare parts requirements, the project used an “Annual Estimated O&M” formula, which includes not only spare parts, but also materiel, repair and overhaul, engineering services and in–service maintenance. Spares are only | | | | | | | | | | | | | of the Halifax–class annual O&M.[27]

This estimated amount for spares became part of the $1.1–billion (current year) capped RFP to industry. Given the shortcomings in the estimate formula, | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is suggested that ADM(Fin CS) provide a better funding model for use in the Director Strategic Finance and Costing costing handbook.

Cost Validation Process

Currently the PAG only requires a formal cost validation by ADM(Fin CS) in preparation for expenditure approvals. Therefore, a project can issue a capped amount in a price and availability (P&A) request or an RFP to industry without going through this formal challenge function. In scenarios where significant contractual cost estimates are provided to the vendors via the Crown, sound estimates are needed so as not to inflate the initial baseline in the early stages of the contracting process.

Recommendation

It is recommended that VCDS amend the PAG to include SRB endorsement of cost for capped RFPs.

OPI: VCDS

Human Resource Staffing

Longer–term civilian project management personnel resources (PMPR) forecasting is needed to account for future projects, while streamlined classification and staffing processes are needed to address the HCM/FELEX Civilian PMPR vacancy of 36 percent.

Vacancy Forecasting

Civilian PMPR forecasting does not take into account the long–term growth requirements of the capital equipment programs needed to implement the Canada First Defence Strategy.[28] In the five–year forecast of civilian PMPR demand within ADM(Mat), an overall reduction was estimated due to project closures without accounting for the introduction of new projects. For example:

  • Civilian PMPR demand was forecasted to reduce from 404 in FY 2009/10 to 347 personnel in FY 2013/14; and
  • DGMEPM projects over the same time period were forecasted to reduce from 17 to four. The forecast did not take into account 18 navy equipment projects that may require PMPRs during this period.

Statement of Merit

For some complex projects, having generic experience levels in a statement of merit can cause excessive applicants. In situations such as naval engineer staffing, specific experience levels are of utmost importance. As there is no human resources (HR) policy that limits the experience–level criteria, flexibility should be considered to allow the project to create non–generic staffing requirements, while still utilizing the Community Management Office and Assistant Deputy Minister (Human Resources—Civilian) (ADM(HR–Civ)) process.[29]

Staffing and Classification Targets

If PMPR requirements are not forecasted, a project office will not be fully staffed with adequate PMPRs for seven months—29 percent of a two–year project definition phase.

The civilian PMPR classification and staffing process does not provide the project office with HR resources on a timely basis. ADM(HR–Civ) has set up performance service standards for classification and staffing processes. Classifying a position should take no longer than 5.5 months.

ADM(HR–Civ) has established a staffing target of 65 working days (three months) whereas ADM(Mat) has set up a longer seven–month target[30]. Although classification and staffing may be done concurrently, consistent target setting should be considered.

Human Resource Performance Measurement

Due to the number of organizations in the civilian PMPR classification and staffing, there is no single set of performance measures established to capture the entire process. A CRS audit sample of 30 HCM/FELEX civilian PMPR position files from FY 2000/01 to FY 2009/10 determined that the entire classification and staffing processes cannot be easily measured from start to finish:

  • The ADM(HR–Civ) ISLA system[31] does not track any of the classification process, while supporting documentation was only available to track the latter half of this 5.5–month process.
  • The staffing process meanwhile had no electronic data or supporting documentation to track the individual position files through approximately the first half of the seven–month staffing process.

To monitor improvement to the classification and staffing processes, end-to-end performance measures are required.

Recommendation

For projects in the options analysis phase, it is recommended that ADM(Mat) include longer–range forecasts of civilian PMPRs. Consistent staffing time targets should be adopted and non–generic experience requirements included in the statement of merit.

OPI: ADM(Mat)

Risk Management

Once improvements in the HCM/FELEX Risk Management Plan (RMP) and practices are in place, a common risk management framework is needed for all HCM projects.

Risk Management Plan

In order for DND to use a common baseline to assess risk, capital equipment projects must comply with a standard risk management framework. In a comparison of the HCM/FELEX RMP to the DND risk management policies, the following observations were made:

Good Practice

Sub–section Combat System Office RMP complied with guidance and is more advanced than project RMP.

  • The project RMP used three–level and five level risk probability scales, whereas DND Integrated Risk Management (IRM) policy[32] suggests a five-level scale.
  • The project RMP ranked risk severity based on three levels, whereas DND IRM policy and ADM(Mat) project management guidance[33] both require five levels.
  • Risk severity assessments were not designed with a risk ranking scoring system. A mixture of qualitative and quantitative factors could have been used to rank risks, but such criteria were not documented in the RMP.
  • The project’s risk management software only considers probability and impact. Factors deemed important by the project office such as time horizon of the risk were not included.

Risk Ranking Process

The risk ranking process does not ensure that all significant project risks are reported or mitigated.

  • The single repository of all risks tracked by the project office was not current;
  • The risk register did not include all high–ranking risks;
  • Standardized risk ranking criteria was not used to rank risks—the project and its sections could have ranked risks differently; and
  • Some medium– to high–severity risks lacked detailed mitigation and contingency plans.

HCM Integrated Risk Management Framework

The East and West Coast navy formations and 12 inter–related stand–alone HCM projects were not a part of the HCM/FELEX project risk management framework. However, the project charter[34] states that the project is responsible for the risk mitigation of all HCM program activities. To ensure that all risks are tracked and managed in a consistent manner, it is suggested that the formations and key inter–related stand–alone projects adopt the revised HCM/FELEX project RMP.

Recommendation

It is recommended that HCM/FELEX PMO update the RMP to comply with departmental guidance and ensure risk management practices are consistent with the plan.

OPI: HCM/FELEX PMO

Annex A—Management Action Plan

Project Schedule

CRS Recommendation

1. It is recommended that CMS establish schedule criteria priority and that the HCM/FELEX PMO perform sensitivity analysis to address slippage potential.

Management Action

CMS has reviewed the schedule criteria and has set priorities as follows:

  1. | | | | | | | | | Halifax–class ships will always be available per coast;
  2. Align with the 48 monthly maintenance cycle for each frigate;
  3. TRP overlap on the same coast is to be avoided if possible;
  4. As closely as practical, the oldest of the class will be refitted first and the newest last; and
  5. All 12 MLRs will be completed within an 81–month period.

Recognizing that priorities are not static in time, should schedule slippage occur in the future these priorities will be balanced with operational requirements of the fleet and institutional capacity in terms of size and composition of the fleet with the industry capacity as well as the scope and funding constraints placed on the HCM/FELEX PMO.

Through the HCM governance structure, the PMO is conducting ongoing oversight of these criteria during the refits utilizing the IS tool jointly developed and managed by CMS formations, and the three principal contractors. When major scheduling issues arise which require the current refit schedule to be significantly altered, the PMO will revisit all relevant criteria and conduct the necessary sensitivity analyses. These five criteria that focus only on the operational concerns will be balanced with other cost and schedule impacts as well as the impact to other key stakeholders and their activities related to the refits.

OPI: CMS and HCM/FELEX PMO
Target Date: Ongoing

Contract Basis of Payment

CRS Recommendation

2. It is recommended that VCDS revise the PAG to ensure that key decisions prior to contract award affecting cash flow or holdback changes be approved by the SRB.

Management Action

This recommendation will be instituted in the PAG via the current re–write which is already under way.

OPI: VCDS
Target Date: October 2011

Cost and Schedule Control

CRS Recommendation

3. It is recommended that ADM(Mat) identify those general ledger accounts in DRMIS that could require the entry of long–term commitments and the HCM/FELEX PMO maximize the use of DRMIS for cost control once the technical issues are resolved.

Management Action

General ledger accounts for major acquisition and in–service support contracts that exceed five years in duration will be brought to the attention of DFA.

OPI: ADM(Mat)/COS Mat/DMG Compt
Target Date: April 2011

The DRMIS tool will be more fully utilized once the multi–year (>5 years) commitments are included in DRMIS and there is more flexibility in adjusting numbers between years, budget elements and commitments.

OPI: HCM/FELEX PMO
Target Date: Resolution of DRMIS shortfalls

Requirements Traceability

CRS Recommendation

4. It is recommended that ADM(Mat) establish a DOORS policy and guidance for DND.

Management Action

Update to the Project Management Manual (PMM)
ADM(Mat) is in the process of a major revision to the PMM governing Materiel Acquisition and Support projects for which ADM(Mat) is the functional authority. We will ensure appropriate policy direction and guidance concerning requirements traceability is included in the manual.

OPI: ADM(Mat)/DGMSSC/DMPP
Target Date: December 2011

Navy Warehouse Space

CRS Recommendation

5. It is recommended that ADM(Mat) direct supply managers to examine dormant stock in Esquimalt and Halifax to optimize the use of warehouse space.

Management Action

ADM(Mat) and CANOSCOM have recently launched a Canadian Forces–wide Inventory Rationalization Programme to immediately reduce national inventory and to increase awareness of materiel flow, while building a long–term systematic approach to rationalize inventory holdings. The programme is comprised of four phases starting with “Definition and Preparation” from November 2010 to January 2011 and will conclude with the systematic rationalization of inventory from March 2012 and onward. As a mandated corporate programme, DGMEPM will participate fully in this initiative.

OPI: ADM(Mat)/DGMEPM
Target Date: Definition—31 January 2011; Implementation—Post–January 2011 onward

Project Cost Estimates/Validation

CRS Recommendation

6. It is recommended that VCDS amend the PAG to include SRB endorsement of cost for capped RFPs.

Management Action

This recommendation will be instituted in the PAG via the current re–write which is already under way.

OPI: VCDS
Target Date: October 2011

Human Resource Staffing

CRS Recommendation

7. For projects in the options analysis phase, it is recommended that ADM(Mat) include longer–range forecasts of civilian PMPRs. Consistent staffing time targets should be adopted and non–generic experience requirements included in the statement of merit.

Management Action

7.1—Longer–range forecasts of civilian PMPRs
DMGHR, in its annual call for military PMPR requirements, will include direction for forecasting civilian PMPR requirements. This process is launched each year in May and is completed at 30 September.

OPI: ADM(Mat)/DMGHR
Target Date: 31 October 2011

7.2—Consistent staffing time targets should be adopted
DMGHR developed its own staffing target timelines for collective staffing processes as the timelines set by ADM(HR–Civ) are currently not achievable. DMGHR will work with our Service Centre Manager from ADM(HR–Civ) to determine if there is flexibility to set more realistic staffing timeline targets. If this is not achievable, it will be necessary to maintain the status quo or risk not starting staffing processes in time to fill vacancies in a timely manner.

OPI: ADM(Mat)/DMGHR
Target Date: 28 February 2011

7.3—Non–generic experience requirements should be included in the statement of merit
As of 11 August 2010, non-generic experience criteria are being included as mandatory criteria for the recent competitions for the Ship Refit manager in PMO FELEX, as well as for engineering specialists in the Aerospace Equipment Program Management division.

OPI: ADM(Mat)/C Mgt O
Target Date: Complete

Risk Management

CRS Recommendation

8. It is recommended that HCM/FELEX PMO update the RMP to comply with departmental guidance and ensure risk management practices are consistent with the plan.

Management Action

PMO HCM/FELEX updated the project RMP on 13 August 2010 to comply with departmental guidance and ensure risk management practices are consistent with the plan.

OPI: HCM/FELEX PMO
Target Date: Complete

Annex B—Audit Criteria

Objective

To provide assurance that the HCM/FELEX project has effective governance, risk management and control frameworks in place to ensure a cost–effective and timely operational capability.

Criteria Assessment

Level 1 (Satisfactory); Level 2 (Needs Minor Improvement); Level 3 (Needs Moderate Improvement); Level 4 (Needs Significant Improvement); Level 5 (Unsatisfactory)

Governance

1.Criteria. Roles and responsibilities are defined and necessary skills, staff and resources are available to govern the projects.

Assessment. Level 2 (Needs Minor Improvement). Deficiencies in HR classification and staffing process; no long–term civilian PMPR planning to account for future program growth; no service–level agreement between the project and PWGSC (briefed); | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |

2. Criteria. An adequate monitoring process is in place that uses high–quality, up–to–date and accurate information as the basis for decision making.

Assessment. Level 3 (Needs Moderate Improvement). DRMIS is under–utilized for project cost and schedule control; tracking system for project detailed report issues deficient (briefed); key project approval documents non–compliant (briefed); MSC scorecard measures require improvement (briefed); inter–related stand–alone projects not covered in draft project communication plan (briefed); 26 months between project identification and charter sign–off—$1.1 million already spent (briefed).

Risk

3. Criteria. Risks are identified, assessed, ranked, mitigated, quantified cost impact and reported in accordance with relevant policy and best practices.

Assessment. Level 3 (Needs Moderate Improvement). Project RMP requires improvement; single repository for all project risks not updated and missing some sub-section risks; no standardized criteria to rank risks; some high risks lack contingency and mitigation plans.

Control

4. Criteria. Project schedule is achievable and is managed to avoid impact on operational requirements.

Assessment. Level 4 (Needs Significant Improvement). Indicators of project schedule risk; sponsor schedule criteria not prioritized; and no documented sensitivity analysis.

5. Criteria. Operational requirements are in accordance with defence policy, clearly defined, complete, prioritized, consistent and traceable throughout the project activities from SOR development to test, evaluation and training plans.

Assessment. Level 3 (Needs Moderate Improvement). Missing linkages in DOORS from operational research to SOR and from SOR to contract PS; no DOORS policy; contract PS interpretation issue.

6. Criteria. Financial management and materiel asset accountability is in accordance with FAA, DND and Treasury Board regulations while ensuring lowest total cost of ownership and facilities with reliable and relevant cost estimates.

Assessment. Level 4 (Needs Significant Improvement). More rigor needed to estimate total cost of ownership including P, O&M, ISSC and project spares costs; capped P&A released before cost validation completion; no effort yet to clean up dormant stock in order to free up navy warehouse space for project materiel—potential cost saving up to $25 million; | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | should not be capitalized (briefed).

7. Criteria. Contract terms and conditions optimize value for money.

Assessment. Level 4 (Needs Significant Improvement). Oversight of the CSI contract award process did not optimize best value; contract government–furnished materiel delivery not just-in-time and warranty expiration issue due to early delivery (briefed); | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | should not be capitalized (briefed).

Footnote 1 CRS Risk Analysis of Capital Projects, April 2007. (Return to footnote 1 source paragraph)

Footnote 2 HCM Master Implementation Plan V.2; $4.2-billion HCM program includes HCM/FELEX Project worth $2.8 billion, many stand–alone projects worth $900 million and numerous maintenance and sustainment national procurement activities worth $500 million. (Return to footnote 2 source paragraph)

Footnote 3 A commitment greater than five years can only be entered in DRMIS with authorization by exception. (Return to footnote 3 source paragraph)

Footnote 4 A CRS audit in February 2010 found 57 percent of line items in the navy’s Halifax warehouse had not been issued in six years. (Return to footnote 4 source paragraph)

Footnote 5 Draft navy 10–year fleet plan version 8.2, 10 April 2010. The last ship MLR period for HMCS TORONTO is scheduled to finish in December 2016, followed by a four–month engineering change work period and a nine–month TRP to achieve full operational capability by January 2018. (Return to footnote 5 source paragraph)

Footnote 6] | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (Return to footnote 6 source paragraph)

Footnote 7 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (Return to footnote 7 source paragraph)

Footnote 8 A DWP is a contractor’s work period to conduct corrective and preventive maintenance. (Return to footnote 8 source paragraph)

Footnote 9 A briefing memo prepared by N37 which identified factors responsible for differentials between actual and planned TRP timelines, 21 October 2008. (Return to footnote 9 source paragraph)

Footnote 10 MARLANT Halifax–class Modernization Master Implementation Plan V1.1, 29 June 2009. (Return to footnote 10 source paragraph)

Footnote 11 Program Evaluation and Review Technique: Estimated Duration = (Optimistic + Pessimistic + (4*Expected))/6. (Return to footnote 11 source paragraph)

Footnote 12 HCM/FELEX Project Charter, 20 March 2008, page 12. (Return to footnote 12 source paragraph)

Footnote 13 Sea acceptance trials and engineering change work could take six months after the MLR period. (Return to footnote 13 source paragraph)

Footnote 14 The CSI DAB contract RFP breaks the $1.3-billion payment milestones into four categories: | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (Return to footnote 14 source paragraph)

Footnote 15 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (Return to footnote 15 source paragraph)

Footnote 16 CSI DAB and ISSC contract RFP amendment 7, page 3/63 states "Milestone does not exceed the sum of incurred costs and the prorated portion of profit or fee for the associated work." (Return to footnote 16 source paragraph)

Footnote 17 Two contracts required holdbacks of 10 percent, one required holdbacks of between 5 to 10 percent, one required holdbacks of 15 percent, and one required holdbacks of 25 percent. (Return to footnote 17 source paragraph)

Footnote 18 DGMEPM, DGMEPM Business Management Process, 2009, v 2. (Return to footnote 18 source paragraph)

Footnote 19 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (Return to footnote 19 source paragraph)

Footnote 20 To address a navy warehousing shortfall, a HCM Disposal Focus Group conducted a full review in late 2009 and most of 2010 of all Repairable Reserve stock at the two bases with the intent to identify surplus/dormant stock. While the review is concluded, disposal action continues and the net result is expected to be significant warehouse space (918 cubic meters) being made available for the HCM/FELEX refit activities. (Return to footnote 20 source paragraph)

Footnote 21 Offsets "refers to the funds available for acquiring new P, O&M as a result of taking existing P, O&M out of service." PAG, 2.4.29. (Return to footnote 21 source paragraph)

Footnote 22 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (Return to footnote 22 source paragraph)

Footnote 23 | | | | | | | | | | | | | | | (Return to footnote 23 source paragraph)

Footnote 24 $224 million is the difference between a straight–line escalation (used by the project from FY 2017/18 onward) and a fluctuating estimate with peaks and valleys from FY 2009/10 to FY 2016/17. (Return to footnote 24 source paragraph)

Footnote 25 Original contract term is 11 years with two three–year options. (Return to footnote 25 source paragraph)

Footnote 26 The labour and materiel costs of the contract are at a negotiated level of effort and, therefore, the Crown is not required to pay the vendor for unused hours as a result of this estimate. Range of overestimation was determined by considering a 20–percent reduction in effort (3 of 15 ships) and a 35–percent reduction in effort (proportion of total effort that is attributable to the Tribal class). (Return to footnote 26 source paragraph)

Footnote 27 Cost Factors Manual for ship FY 2010/11. It is acknowledged that this amount would likely entail a majority of consumables as the Cost Factors Manual is a rolling average of the most recent four years, while the Crown likely needs to procure a number of repairables as part of its two–year spares requirement. (Return to footnote 27 source paragraph)

Footnote 28 Shorter planning horizons such as the annual "call letter for project PMPRs" and the three–year Strategic Intake Plan do provide for some visibility of demand for both funded and unfunded projects, and is the more relied upon information. (Return to footnote 28 source paragraph)

Footnote 29 Currently PMOs need to perform the additional work themselves if a non–generic process is to be used. (Return to footnote 29 source paragraph)

Footnote 30 Assuming concurrent activity, it could take up to seven months to classify and staff civilian positions for a project. (Return to footnote 30 source paragraph)

Footnote 31 The ISLA system was put into place by ADM(HR–Civ) to track staffing processes as a result of a CRS review of Human Resource Service Delivery (http://www.crs-csex.forces.gc.ca/reports-rapports/2009/137P0797-eng.aspx). (Return to footnote 31 source paragraph)

Footnote 32 VCDS, DND/CF Integrated Risk Management Policy, January 2007. (Return to footnote 32 source paragraph)

Footnote 33 Director Military Program Planning, ADM(Mat) Knowledge Network, Project Risk Management Guidance (Draft), Chapter 6. (Return to footnote 33 source paragraph)

Footnote 34 HCM/FELEX Project Charter, page 2, Definition, March 2008. (Return to footnote 34 source paragraph)