Evaluation of the Contribution Agreement of the Capital Assistance Program (CAP)

November 2009
1258-178 (CRS)

Reviewed by CRS in accordance with the Access to Information Act (AIA).
Information UNCLASSIFIED.

Table of Contents

1. Acronyms and Abbreviations
2. Results in Brief
3. Introduction
   • Background
   • Objectives
   • Scope
   • Methodology
   • Limitations
   • Description of Program
4. Findings
   • Relevance
   • Success
   • Cost Effectiveness
5. Conclusion and Recommendation
   • Conclusion
   • Recommendation
6. Annex A—Management Action Plan
7. Annex B—CAP Logical Model (from the 2006 RMAF)
8. Annex C—Levels of Expenditure Authority
9. Annex D—List of Proposed CAP Projects
10. Annex E—Description of the Three Successful Projects

Acronyms and Abbreviations

Results in Brief

The Terms and Conditions for Contributions in Support of the Capital Assistance Program (CAP)^[1] , approved by Treasury Board (TB), came into effect on 1 April 2006 and expires on 31 March 2011. According to the TB Policy on Transfer Payments, an evaluation is required to support the renewal of the Terms and Conditions of the Program prior the end of the five-year period.

The evaluation was to determine whether or not:

• The CAP remains relevant;
• CAP demonstrates efficiency and economy while achieving its expected outcomes;
• CAP was in compliance with the Terms and Conditions approved by TB; and
• The Assistant Deputy Minister (Infrastructure and Environment) (ADM(IE)) organization achieved the Management Action Plan (MAP) of the 2003 review.

Key Findings

Relevance. CAP is relevant and is aligned with federal government priorities, programs and objectives, particularly those of the Department of National Defence (DND) and the Canadian Forces (CF).

Performance (effectiveness, efficiency and economy). The Program is successful when it is effectively executed and is a cost-effective alternative to ownership and maintenance of infrastructure by DND. However, the lack of strategic direction and priority placed on transferring infrastructure to local municipalities has led to a severe underutilization of the Program and considerable wasted effort. Since 2001, only three of a potential eighteen projects have been completed for a total expenditure of $2.88 million. Although each completed project can demonstrate its worth, the limited number of completed projects and the low usage of available funds indicate that the Program, as a whole, is providing limited value for the level of effort and available resources.

It must also be noted that this Program has tremendous potential if the DND/CF were to make the divestment of infrastructure a priority and take the appropriate actions.

Recommendation

It is recommended that ADM(IE) consider the priority and potential benefit of this Program against the level of effort required to ensure effective direction and management. The outcome of this consideration should result in a clearly substantiated case to either terminate the Program or renew it in association with improved strategic direction and a governance model that will enable future success.

Introduction

Background

An evaluation of the CAP was included in the Chief Review Services (CRS) Evaluation Work Plan for fiscal years (FY) 2008/09-2009/10.

The Terms and Conditions for Contributions in Support of CAP approved by TB came into effect on 1 April 2006 and will expire on 31 March 2011. In accordance with TB’s Policy on Transfer Payments, an evaluation of a Contribution Agreement is required to support the renewal of the terms and conditions of the Program prior to the end of its five-year period.

A Results-based Management and Accountability Framework (RMAF), including a Logic Model (Annex B), was approved in 2006. In addition to describing the Program, the RMAF established the framework within which the Program was to be evaluated, along with the performance measures, outputs and outcomes of the Program.

Objectives

The evaluation objectives, as specified in the RMAF, were to assess the following:

Relevance. The extent to which CAP was relevant to the mandate and strategic objectives of the DND/CF in achieving its ultimate outcomes of:

• Reducing DND’s infrastructure by transferring it to eligible recipients;
• Developing and maintaining the infrastructure cost-effectively; and
• Enhancing the quality of life of members of the CF and their dependants.

CAP’s relevance measured the:

• Continued need for the Program;
• Alignment with government priorities; and
• Alignment with federal roles and responsibilities.

Performance. The extent to which effectiveness, efficiency and economy were met.

• Achievement of Expected Outcomes. The extent to which the Program was successful in achieving its expected outcomes of:
  • Number of established collaborations with local authorities;^[2]
  • Reduction of operating and capital costs;
  • Improvement/maintenance of transportation and utility infrastructure; and
  • Improvement/maintenance of community service and leisure infrastructure.
• Demonstration of Efficiency and Economy. The extent to which CAP is cost-effective in assessment of :
  • Long-term savings in upgrading and maintaining DND/CF infrastructure;
  • Administration cost of running the Program; and
  • Governance of the Program.

In addition, this review assessed the extent to which CAP complied with the Terms and Conditions for Contributions in Support of the Capital Assistance Program approved by TB in 2006.

This review also examined the MAP from the 2003 review and determined to what extent its recommendations were implemented.

Scope

This evaluation assessed program performance from FY 2006/07 to FY 2008/09. However, information since FY 2000/01 was employed to further illustrate results. The scope of this evaluation was mainly internal and was based on the performance measures stipulated in the RMAF. This evaluation also determined program relevance and performance in accordance with its objectives and the new TB Evaluation Policy.

Methodology

The following methods were used during the evaluation:

• Interviews with principals involved in infrastructure management within DND/CF;
• Review of file documentation, such as the business case and contribution agreement;
• Review of the CAP Procedure Manual;
• Review of the previous CAP evaluation report conducted in 2003;
• Review of the RMAF and the Terms and Conditions for Contributions in Support of CAP; and
• Analysis of other data and information obtained during the evaluation.

Limitations

Ten projects were initiated during the past three years, FY 2006/07 to FY 2008/09. Three projects successfully established a contribution agreement in that time period, two of which were initiated prior to FY 2006/07. Two of the three successful projects were completed and the third is in progress. The administration cost of running the Program was not tracked and monitored. As a result of insufficient available data, this evaluation does not include an in-depth cost-benefit analysis of administering CAP.

Due to the transfers of personnel, not all past decision makers could be interviewed.

Description of Program

The CAP Contribution Program started in FY 1983/84. It gives authority to the Minister of National Defence (MND) to provide funding under certain terms and conditions approved by TB in the form of a contribution to eligible recipients such as Provinces; Territories; Municipalities; Provincial and Territorial Crown Corporations and agencies; other corporations and/or organizations owned or controlled by Provinces, Municipalities and Territories; and other organizations that in the normal course of business provide municipal services. The funding is to be used for the upgrade and maintenance of DND’s infrastructure^[3] related to the operations of military bases and wings with the intention of transferring this infrastructure to the eligible recipients. The infrastructure provides community-based services to DND bases and wings as well as residential accommodation areas across Canada.

The rationale for the contributions is that municipalities are in a better position than DND to deliver more cost-effectively community-type (and municipal) services and maintain infrastructure in support of residential accommodation. This includes residential accommodation at bases and wings.

CAP objectives are to:

• Support DND's strategic direction and priorities, including optimization of DND's infrastructure to support the evolving and expanding force structure;
• Develop collaborative arrangements with Provinces, Territories and Municipalities and/or their agencies to transfer and upgrade or develop infrastructure;
• Ensure a cost-effective provision and maintenance of public utilities and community infrastructure to DND bases and wings in Canada;
• Enhance the quality of life of CF members and their dependants located at bases and wings through improved infrastructure;
• Maintain/improve infrastructure while correcting identified deficiencies; and
• Reduce operating and capital costs.

Typical CAP projects include:

• Transfer of infrastructure, including utility systems, roads, sidewalks, etc., to municipalities and other levels of government;
• Upgrades to and expansion of municipal utility plants to maximize usage and efficiency; and
• Contribution to the construction of facilities for shared use by the bases/wings and local municipalities, e.g., arenas, community centers, bus shelters, etc.

Projects are normally initiated under the authority of Base Commanders (BComd) and approved by either the Director General Realty Policy and Plans (DGRPP), ADM(IE), or MND, depending on the value of the project.^[4] The CAP contributions are awarded following a formal application and selection process. Each activity in the CAP process is funded differently.^[5] The application and selection processes are funded under Vote 1 where applicants need to provide a business case containing different scenarios/options. The cost implications of proposed projects are quantified during these processes in order to make informed decisions. Applications are assessed on how well they support the strategic directions of the Department and how they contribute toward enhancing the quality of life of CF members and operations. To be eligible for CAP, a project must meet a number of conditions, as specified in the CAP procedures manual. The contribution amounts for the approved CAP projects are provided for with Vote 10 funding.

Cash-phasing under Vote 10 for contributions in support of the CAP was set by TB at a maximum of $30 million per year. However, DND’s internal funding limit, established by the Vice Chief of the Defence Staff (VCDS), is at $5.45 million per year, which may be adjusted annually depending on program requirements but cannot exceed the $30 million ceiling. However, CAP has not initiated and completed the projects needed to expend the internally available $5.45 million annually and has fallen considerably short of generating new initiatives that could be accommodated within the $30 million annual expenditure ceiling. This significant underutilization of the Program calls into question its value to the Department.

Since 2001, 18 projects were proposed for CAP, 10 of which have been initiated over the past three years, FY 2006/07 to FY 2008/09.^[6] However, as shown in Table 1, there were only three successful CAP projects that established a contribution agreement:^[7]  As a result, total CAP spending to date within the current contribution agreement between TB and DND is only $2.88 million.

Findings

Relevance

CAP is relevant to federal government priorities, programs and objectives, and specifically those of DND and the CF.

CAP reduces the operating costs of bases and wings by transferring infrastructure to other levels of government or other eligible recipients that have the capacity to be the service providers and are in a better position to provide these services cost-effectively and efficiently. When used as intended, CAP encourages non-operational activities to be transferred to other organizations, thereby enabling DND/CF to keep its focus on maintaining and strengthening its operational capacity in order to meet defence and security commitments. As well, the upgraded infrastructure enhances the quality of life of military members and their families while at the same time limiting DND’s future liabilities.

CAP is a Government of Canada priority under DND’s Program Activity Architecture (PAA) since it is in alignment with the following strategic direction:

“Resources are acquired to meet Government Defence Expectations.” That is, “Capital Assistance Program projects support real property goals and objectives by encouraging cost-effective solutions for the provision of infrastructure on bases and wings across Canada.”

Success

CAP is successful only when projects are fully implemented. Despite the high potential for success, the limited expenditures and rate of projects completed versus the projects initiated suggest the Program as a whole provides low value for money.

Over the past eight years, eighteen potential projects have been identified and/or initiated to varying levels. Of these, only three have been completed and two more remain in consideration. Although the potential funds available to support this Program amount to $16.35 million over the last three years, only $2.88 million has been spent. CAP is therefore underutilized by DND/CF. Details of specific projects are included in annexes D and E. There are four main reasons why the Program has under-performed:

1. Lack of Priority and Strategic Direction

Although one of the main objectives of CAP is to transfer infrastructure from DND to local municipalities, there is no strategic master plan with a priority list of holdings to be transferred. Instead, individual projects are proposed at the local level and then prioritized. Based on the recent submissions, projects are focused on short-term improvements to local infrastructure rather than a master plan for the Department to reduce its infrastructure.

2. Lack of Management

The CAP program has had limited promotion within the DND/CF which has contributed to a lack of knowledge and understanding of the Program among internal stakeholders and potential users. As there is limited overall program monitoring, and therefore limited lessons learned system in place, issues that caused the cancellation of projects in 2001 are still recurring in the screening of projects in FY 2006/07. For example the issue of police service jurisdiction between the military and municipal police forces has resulted in three project cancellations between 2001 and 2006.

3. Accountability, Responsibility and Approval

ADM(IE) is the functional authority for the management of DND/CF infrastructure, including accountability for acquisition and disposal of real property. Approval of particular CAP projects is assigned to the MND, ADM(IE) or DGRPP depending on the value of the project. However, as there is no strategic direction on the use of CAP for disposal of infrastructure, the responsibility to initiate projects and subsequently the project management after approval rests with BComds.

Given their significant authority and lack of strategic direction, BComds have limited accountability for results until after project approval. Their accountability exists through the chain of command rather than to the functional authority. This has created a situation that provides little incentive for BComds to initiate projects and the opportunity to cancel them easily at any point short of approval.

4. CAP Process

DND uses a construction project process for CAP, which has proved to be inefficient and ineffective. This process assumes that DND is managing the CAP project as an internal construction project, whereas the real purpose of the Program is to shift the project management and implementation to the recipient. Consequently, the existing CAP process does not accomplish the original intent of the Program. On average the internal approval process requires approximately one year to complete a cycle. As a result, it is sometimes viewed as lengthy and time consuming. In one case,^[8] the Base—on the recommendation of the Director of Land Infrastructure (DLI)—opted out of the process as it was perceived to be too demanding.

Working on a CAP project with smaller communities has its additional challenges. One of the rules of the CAP process is that there will be no transfer of funds to the recipients before the contribution agreement is signed. A contribution agreement cannot be drafted and signed unless a project starts. Since, the smaller communities do not always have the financial capacity to start up a project, it is challenging to obtain their commitment on potential CAP projects, even though all their project-related expenses will eventually be reimbursed.

Time spent on CAP projects prior to a contribution agreement being signed is quite variable as these projects are horizontal initiatives between DND and the eligible recipients–mainly municipalities. Each party is dependent on the other’s collaboration and commitment to the project as well as their approval process. The longer the time from project initiation to signing an agreement, the greater is the opportunity for a project to become delayed or stopped altogether. Factors such as a change in municipal governments, turnover of base commanders with differing priorities and other circumstances can come into play. For example, the Valcartier Water Connection Project took four years to come close to an agreement point, but it was halted at that stage.

CAP has not received the required attention, commitment and priority to enable program success. The consequent impact of program underutilization on program funding is reflected in Table 2:

A review of Terms and Conditions for Contribution in Support of CAP was also done. The review shows that CAP has adhered to all of the Terms and Conditions of the Contribution Agreement between TB and DND except for maintaining a working group to prioritize and monitor CAP projects.

An assessment of the MAP of the 2003 review was also completed. Steps were taken to ensure that the recommendations included in the MAP of 2003 are reflected in the CAP procedures manual. As a consequence, the CAP mission and objectives are now clearer and a comprehensive RMAF has been developed. The CAP procedure manual was revised, but it still needs to be redesigned as it is lengthy and complex in its current form. There is a comprehensive template to be used for contribution agreements in order to ensure consistent and appropriate terms and conditions for each initiative. Agreements clearly stipulate that after completion of the project and a satisfactory assessment of the quality of work by DND, funds will be allocated. This ensures adequate monitoring of each project before it is considered complete. However, CAP is still lacking a departmental master plan for long-term planning purposes. Also, there is no communication package for CAP in order to promote it consistently to the bases and wings across Canada. Hence, there is still significant misunderstanding of appropriate program usage among internal stakeholders.

Cost Effectiveness

CAP is cost-effective when projects are successfully completed. It is not cost-effective when projects are halted part way through as the intended outcomes are not achieved.

In the past three years, FY 2006/07 to FY 2008/09, CAP was able to establish three initiatives for a total expenditure of $2.88 million. When a CAP initiative is completed, the project itself is cost-effective as the transfer of infrastructure relieves DND of any future financial liabilities. However, when projects are halted or stopped internally part way through the process, CAP is not cost-effective.

The main objective of CAP is to reduce future operation and maintenance costs. However, some CAP initiatives are intended to free DND and eventually the Crown from future legal liabilities. In these cases, CAP manages risk by mitigating liabilities against the Crown.

Conclusion and Recommendation

Conclusion

CAP can demonstrate its relevance; however, it is suffering from a lack of strategic priority and management with DND/CF. Without this, and an associated master plan to transfer infrastructure to municipal authorities, the Program is simply an option that can be utilized to support local initiatives. As these tend to be small in scale and risk intolerant, only three projects have been completed between FY 2006/07 and FY 2008/09 for a total expenditure of $2.88 million. Based on this performance and in consideration of the level of effort expended to achieve small results, the Program offers little value for money.

There remains, however, a significant potential to make this Program effective given a strategic intent to divest property and the establishment of an effective implementation plan to achieve that intent.

Recommendation

It is recommended that ADM(IE) consider the priority and potential benefit of this Program against the level of effort required to ensure effective direction and management. The outcome of this consideration should result in a clearly substantiated case to either terminate the Program or renew it in association with improved strategic direction and a governance model that will enable future success. If renewal of CAP is desired, a case substantiating continuation and a plan to improve performance should be made to the Deputy Minister (DM) and the Chief of the Defence Staff (CDS) prior to 1 April 2010.

Annex A—Management Action Plan

CRS Recommendation

1. It is recommended that ADM(IE) consider the priority and potential benefit of this Program against the level of effort required to ensure effective direction and management. The outcome of this consideration should result in a clearly substantiated case to either terminate the Program or renew it in association with improved strategic direction and a governance model that will enable future success. If renewal of CAP is desired, a case substantiating continuation and a plan to improve performance should be made to the DM/CDS prior to 1 April 2010.

Management Action

The following is the proposed Management Action Plan of ADM(IE)/DGRPP to address the above recommendation:

• Obtain approval of the CAP Directive (Policy Instrument).
  • OPI:  ADM(IE)/DGRPP
  • Target Date:  1 April 2010

• Provide appropriate resources, i.e., fill the vacant position of the CAP Advisor. That position reports to the Section Head, Real Property National Programs and is dedicated full time to the program. The job description was revised and has been sent to ADM(HR-Civ) for classification.
  • OPI:  ADM(IE)/DGRPP
  • Target Date:  FY 2010/11

• Re-institute the CAP Working Group where all Level 1s are represented.
  • OPI:  ADM(IE)/DGRPP
  • Target Date:  1 April 2010

• With the support of the CAP Working Group, undertake the following actions:
  1. prepare a survey of all base/wings to identify potential CAP projects: transfer of infrastructure like sewage/water systems, water treatment plants, waste water treatment plans, community oriented projects like daycare centers, arenas, etc.;
  2. based on survey, establish a five-year plan that will include priorities and implementation strategies;
  3. undertake a promotional campaign of CAP via revised written documentation, website update and possibly a tour of bases/wings to explain the Program where good potential for projects have been identified;
  4. identify resources on bases/wings in support of CAP projects to ensure that projects can be efficiently and diligently implemented; and
  5. review/streamline CAP process.
  • OPI:  ADM(IE)/DGRPP
  • Target Date:  FY 2010/11

Annex B—CAP Logic Model (from the 2006 RMAF)

Figure 1. CAP Logic Model. This flowchart depicts the link between CAP activities and CAP outputs and, eventually, CAP immediate and ultimate results.

Text description of this flowchart is available on a separate page.

Annex C—Levels of Expenditure Authority

Annex D—List of Proposed CAP Projects

Annex E—Description of the Three Successful Projects

• Slip Turn Around Construction Project—Kingston:

This project was in collaboration with the City of Kingston where a turn lane was constructed at one of the two entrances to the Base on Hwy 15. The contribution amount was $30,000. The project is completed.

• Belmont Park Water Systems Upgrade:

This project was in collaboration with CRD—Victoria. Canadian Forces Base (CFB) Esquimalt had transferred the Belmont Park water distribution system to CRD in 1999. However, the fire flow capacity of the system did not meet Fire Underwriters Survey standard. CRD confirmed that improvements were necessary to bring the flow capacity of the water system up to a minimum standard. Thus, an additional water trunk line needed to be added to the existing system. Also, in 2005 DND entered into a lease agreement with School District 62 for the use of an elementary school property. The School District advised DND that it would not be permitted to occupy and use the new school until after the water system has been upgraded. DND agreed to make a one-time payment of $450,000 to avoid paying the system development charge to cover the capital costs of this upgrade, which would have amounted to $510,361.01. The project is completed.

• Admirals/Colville Intersection Project—Esquimalt:

This project was in collaboration with the Township of Esquimalt and Public Works and Government Services Canada (PWGSC). This is a very dangerous intersection where roads and a railway go through it. The intersection is mostly used by DND personnel. Thus, this project was initiated in order to improve employee safety and to mitigate the liability of the Crown. DND and PWGSC owned parts and pieces of the intersection. All the land under the street was transferred to the township of Esquimalt for $1. The contribution agreement is for $2.4 million which is used to redesign and reconfigure the intersection by adding traffic lights and marked lanes, as well as constructing curbs. This project is in progress.

Notes

• [1] This is the contribution agreement between TB and DND.
• [2] The number of collaborations partly depends on the interest and ability of the local municipalities and is a factor that is beyond the control of DND.
• [3] Such as sidewalks, curbs, roads, water mains, water treatment facilities and sewers.
• [4] Refer to Annex C for levels of expenditure authority.
• [5] There are different funding sources within the defence portfolio.  Vote 1 funding is used for salaries, operation and maintenance costs.  Vote 5 funding is used for capital procurement and Vote 10 is used for grants and contribution programs.
• [6] Refer to Annex D for a list of projects considered for the CAP program since 2001.
• [7] Successful projects are the ones where a contribution agreement between the parties is signed.  Refer to Annex E for a brief description of these projects.
• [8] Transfer of Water Tower Petawawa.