Evaluation of WD’s Activities Related to Industrial and Technological Benefits Policy-Defence Procurement

December 2018

Executive Summary

Background

In 2014, the Government of Canada established the Industrial and Technological Benefit (ITB) Policy, replacing the Industrial and Regional Benefits (IRB) Policy as Canada's offset mechanism. Firms bidding on eligible defence procurements are evaluated on quality, price, and economic benefits. Innovation, Science and Economic Development (ISED), along with the Regional Development Agencies (RDAs) including WD, evaluates the industrial benefits. RDAs also play an important role in connecting bidders and obligors with regional organizations (industry and post-secondary institutions) and facilitating transactions.

Purpose and Methodology

Covering the period since the introduction of the ITB Policy, the purpose of this evaluation is to review the activities undertaken by WD to support implementation of the ITB Policy with the aim of informing future delivery. WD has not previously undertaken an evaluation of the defence procurement file. Activities and initiatives under the previous IRB policy were included the WD's 2009 Evaluation of Policy, Advocacy and Coordination. The lines of evidence for the evaluation included a review of documents and literature, interviews with 86 key informants, four focus groups, and six case studies.

Conclusions

The major conclusions arising from the evaluation are as follows:

• The ITB Policy is a unique policy instrument that provides WD with significant opportunities to promote development of economic clusters, particularly the aerospace and defence cluster, and promote inclusiveness.

• The activities undertaken by WD in support of the ITB Policy and defence procurement align well with the needs identified. However, further and more targeted assistance will be required to both increase the share of ITB obligations captured by organizations in western Canada and lever those obligations to address key factors constraining further development of the cluster.

• While WD does not have access to data on the success of western Canada in capturing ITB obligations, examples gleaned from key informant interviews and secondary sources illustrate the impact of the ITB Policy and WD's activities.  The activities of WD are an important factor in enabling organizations to capture IRB and ITB obligations.

• The investment made by WD in supporting the ITB Policy and defence procurement is low relative to the potential benefits of increasing the share of ITB obligations captured by western Canada. Even a small increase in the share of the ITB and IRB obligations would be sufficient to justify increased expenditures made by WD.

Recommendations

Based on findings and conclusions, the evaluation provides the following recommendations:

• WD senior management should review defence procurement activities and determine the level of priority that WD will place on increasing the share of ITB obligations captured by western Canada and better leveraging those commitments to promote cluster growth and inclusiveness. Implementing a more proactive, strategic and consistent approach to leveraging the ITB Policy would likely require additional resources.
• The department should strengthen the governance system by clearly defining the relative roles and responsibilities of WD regional offices and Ottawa.
• The department should strengthen delivery of ITB related activities by improving processes through actions such as:
  • Adopting communication, collaboration and work management software that would assist teams in organizing, tracking, and managing work associated with particular procurements.
  • Establishing a portal that would connect various stakeholders, facilitating communication and informing them about new opportunities, developments and events.
  • Developing shared support materials such as seminar and presentation materials and supplier development tools.
  • Providing training for staff new to the file.
• In association with the other RDAs , the department should work with ISED to ensure access to the data needed to assess opportunities, develop strategies and track the results. The department should complement regional data obtained from ISED with other data through project reporting in departmental databases and a periodic online survey of regional representatives who have participated in various activities.

View the print-friendly version: PDF

Table of contents

• Executive Summary
• Introduction
  • Background
  • Purpose and Scope of the Evaluation
  • Evaluation Approach and Methodology
  • Limitations
• Relevance
• Results
  • Portfoio of IRB and ITB Obligations
  • Capturing the Impacts of the ITB Policy in Western Canada
  • Attribution to the Activities of WD
  • Barriers to Accessing ITB and IRB Obligations
• Design and Delivery of WD's Activities
  • Impacts of the 2014 Changes to the Policy
  • Comparison of Roles and Responsibilities
  • Capturing and Sharing Information About Industry Capabilities
  • Contribution Towards Cluster Growth and Inclusiveness
  • Governance Structure
• Efficiency and Economy
  • Comparison of Benefits and Costs
  • Performance Metrics
• Opportunities for Improvement

Introduction

Background

The Department of Western Economic Diversification Canada (WD) was established in 1987 to lessen western Canada's strong economic dependence on its natural resources. Under the Western Economic Diversification Act, WD is mandated to "guide, promote and coordinate the policies and programs, including those related to industrial benefits, of the Government of Canada in relation to the development and diversification of the economy of Western Canada." Since the mid-1970s, the Government of Canada has pursued the use of industrial benefits as part of federal procurement defence and security contracts that are exempt from international trade agreements.

In 2014, the Government of Canada established the Industrial and Technological Benefit (ITB) Policy, replacing the Industrial and Regional Benefits (IRB) Policy as Canada's offset mechanism. The ITB Policy applies to all eligible defence procurements over $100 million and eligible Canadian Coast Guard procurements over $100 million; eligible defence procurements between $20 million and $100 million are reviewed for the use of value propositions.

Firms bidding on procurements are evaluated on quality, price, and economic benefits.  Economic benefits are rated using value propositions, economic commitments that bidders make in their bid that become a contractual commitment for the winning bidder (obligors). The contracting department or agency evaluates the technical aspects. The contracting authority evaluates the financial aspects, and Innovation, Science and Economic Development (ISED), along with the Regional Development Agencies (RDAs) including WD, evaluates the industrial benefits. RDAs also play an important role in connecting bidders and obligors with regional organizations (industry and post-secondary institutions) and facilitating transactions.

Purpose and Scope of the Evaluation

The purpose of this evaluation is to assess, validate, and potentially improve the activities undertaken by WD in support of the ITB Policy and defence procurement. The evaluation focuses on the period from December 2014 to March 2017.

Evaluation Approach and Methodology

Each key finding reported and/or conclusion presented in this report has been triangulated and confirmed from two or more lines of evidence to ensure reliability. The lines of evidence included are summarized in the diagram below.

Limitations

The main limitations and challenges are summarized as follows:

• While ISED has access to extensive data on the economic impacts of the ITB Policy, it does not share that data with the RDAs including WD. Performance data collected by WD focuses on activities and their outputs. To mitigate this challenge, interview and case study guides were adjusted to collect examples of outcomes related to WD's activities.

• Some key informants were familiar with the ITB but unable to comment on specific impacts or outcomes. In these cases, opinions were obtained on the ITB Policy, need for WD's ITB activities, existing levels of collaboration, and potential suggestions regarding design and delivery.

• Potential for respondent bias as some key informants have a vested interest in the ITB Policy.  To address this, a broad cross-section of representatives was interviewed of which a majority were not directly involved in or receiving support from WD.

Relevance

The aerospace and defence industry in western Canada

According to Statistics Canada ^{Footnote 1} , there were 662 companies active in Canada's defence-related industries in 2016. These companies employed 27,000 people, generated $10.1 billion in revenues, invested $400 million in R&D, and contributed about $6.2 billion to GDP (including $3.0 billion in direct GDP). SMEs accounted for over 90% of the firms but only 24% of revenues and 32% of employment. In terms of employment, western Canada accounts for about 20% of employment (approximately 5,400 people), while 38% is based in Ontario, 28% in Quebec, and 14% in Atlantic Canada.

The aerospace industry overall is considerably larger than the defence industry. The Canadian aerospace industry directly generated $27.7 billion in revenues, employed over 87,000 people, and invested over $1.9 billion into R&D in 2016. The direct contribution of the aerospace industry to Canadian GDP was estimated at $12.9 billion in 2016. Western Canada accounted for 44% of Canadian aerospace maintenance, repair and overhaul (MRO) employment (24,500 people) and 36% of aerospace manufacturing employment in 2016 (11,250 people). ^{Footnote 2}

According to Jenkins report on defence procurement, federal government contracts play a critical role in the development of the industry. ^{Footnote 3} The Government of Canada is the largest single customer for defence products and services produced in Canada (accounting for about 28% of sales). The Government of Canada's new defence policy, Strong, Secure, Engaged ^{Footnote 4}, incorporates plans for significant new capital investments. The policy provides $33.8 billion in new capital money over its 20-year lifespan and expands funding for other projects.

Need for WD to assist western canadian industry lever opportunities

There is strong rationale for WD to be actively involved in helping companies and post-secondary institutions capture ITB Policy obligations and levering those benefits to promote economic development.

The evaluation found that:

• The activities are targeted largely at aerospace and defence, which represents an important emerging cluster for western Canada. In 2016, the defence industry employed 5,400 people in western Canada^{Footnote 5} while the aerospace industry employed 35,000. ^{Footnote 6} The global industry is projected to continue to grow because of increasing global defence spending and rising demand for passenger travel.

• Many countries effectively use offset mechanisms to increase the economic impact of their defence expenditures and build local industry. The ability of companies bidding on prime contracts to match their offset strategies with the requirements of country-specific mechanisms has become a key source of competitive advantage.

• Obligors, western Canadian companies, post-secondary institutions, and other stakeholders recognize a strong and continued need for WD to be involved (78% identified a strong need and 15% identified somewhat of a need). Obligors look to WD for guidance in understanding the offset requirements and options, learning about regional capabilities, identifying investment opportunities, and facilitating introductions. In the absence of support from WD, SMEs often lack the capacity and resources needed to learn about government procurements and the ITB Policy, establish relationships with prime contractors, and pursue opportunities. Post-secondary institutions express strong interest in levering the ITB Policy to attract funding for research and education programs but require assistance in becoming more familiar with the Policy to, identifying opportunities. They also expressed interest in working with bidders to develop proposals that are consistent with the value proposition, capabilities of the institution, and objectives and activities of the bidder.

The Western Economic Diversification Act requires WD to guide, promote and coordinate the policies and programs, including those related to industrial benefits. The impact of the ITB Policy in leveraging opportunities is a function of the success of western Canada in capturing the obligations of the obligors, and the impact of the obligations captured.

When asked if there was a need for WD to be involved in assisting industry and post-secondary institutions to lever opportunities related to major defense and security capital projects undertaken by the Government of Canada, most key informants (93%) indicated a strong need or somewhat of a need for WD to be involved. Stakeholders identified that there was a need for WD to:

• Educate industry and other stakeholders on the ITB Policy and about defence procurement processes.
• Create awareness, amongst bidders and obligors, of the capabilities of the industry in western Canada, advocate for the industry and help shape the value proposition.
• Help to connect bidders/obligors with companies and post-secondary institutions

Key informants noted that in the absence of WD support and guidance, most SMEs and industry stakeholders would not be able in position to take advantage of the ITB Policy.

Role of WD

WD's role has, to-date, been one of a convener and business facilitator connecting international aerospace and defence companies that have, or could have, ITB obligations to the industry in western Canada. WD has also played the role of promoter of western industry nationally and internationally at trade shows, conferences, and other events. WD activities undertaken related to the defence procurement priority in 2017-18 included:

• Advocating for western Canada in the implementation of the Defence Procurement Strategy (DPS) including the development and evaluation of value proposition and ITB plans.
• Promoting western Canadian industry through supplier development activities such as supplier requests, business-to-business meetings, industry days, and supplier tours with global defence companies.
• Informing western Canadian stakeholders of Canadian defence procurement opportunities and what is required to identify and successfully pursue participation opportunities in the global supply chains of prime contractors and key partners.
• Identifying strategic investments that support innovation, international competitiveness and growth in an effort to strengthen the western Canadian aerospace, defence and marine sectors.
• Working with partners to support collaborative trade and investment opportunities, including western Canadian participation at international events and trade missions in the aerospace, defence and marine sectors.
• Representing western interests to decision makers in the development of national policies and strategies relating to the Canadian defence procurement process, and the development and sustainment of the aerospace, defence and marine sectors in western Canada. This includes work directly in support of ISED in the analysis and development of specific value proposition criteria, as well as broader defence related policy or program initiatives such as the potential for inclusiveness as an element of value proposition.

WD can lever the knowledge and opportunities gained through ITB into broader opportunities for western Canadian industry, use the value proposition to target priority areas such as investments in R&D, product development and skills training, and link bidders to developments that WD has supported through other departmental programs such as WINN and WDP.

Extent to which WD services respond to needs

WD has a legislative mandate to deliver services related to industrial benefits and focus on supporting economic development and diversification of the region. A review of documents and files indicates that, over the period covered under the evaluation, WD has undertaken a range of initiatives related to ITB and defense procurement including:

• evaluating ITB bids submitted by prime contractors;
• supporting supplier development activities and responding industry requests related to value proposition; and
• organizing and/or participating in exhibitions and trade shows.

Other activities related to management and administration such as participating at advisory committees and briefings or organizing regular telephone calls with provincial counterparts.
The activities undertaken by WD align well with the types of needs identified. Through its defence procurement/ITB team, WD:

• advocates for western Canada in the definition of requirements and development of value propositions;
• promotes western Canadian industry through responding to supplier requests about regional industry capabilities;
• facilitates business-to-business meetings and tours;
• supports events and trade missions;
• informs stakeholders about the ITB policy, procurement process, and opportunities; and
• works with partners from provincial governments, associations and other stakeholders.

Of the 104 activities reported by WD's defense and ITB team from 2014-15 to 2017-18, 33% related to evaluating ITB bids submitted by prime contractors, 33% related to supporting supplier development activities and responding industry requests related to value proposition, and 22% related to organizing and/or participating in exhibitions and trade shows. Other activities related to management and administration such as participating at advisory committees and briefings or organizing regular telephone calls with provincial counterparts.

The activities undertaken by WD align well with the types of needs identified. Through its defence procurement/ITB team, WD advocates for western Canada in the definition of requirements and development of value propositions; promotes western Canadian industry through responding to supplier requests about regional industry capabilities; facilitates business-to-business meetings and tours; supports events and trade missions; informs stakeholders about the ITB policy, procurement process, and opportunities; and works with partners from provincial governments, industry associations and other stakeholders. WD has also made some strategic investments to strengthen the western Canadian aerospace, defence and marine sectors.

When asked if the WD defence procurement activities are responsive to existing needs, 65% of key informants indicated that they are very responsive or somewhat responsive and 22% indicated they are not responsive. Representatives from western Canadian SMEs and post-secondary institutions were more likely to consider the WD's services as somewhat responsive or not responsive.

However, key informants argue that further and more targeted assistance is required to both increase the share of ITB obligations captured by organizations in western Canada and lever those obligations to address some of the key factors constraining further development of the aerospace and defence industry.  Industry growth in western Canada is constrained by factors such as access to supply chains and markets, rates of technology development, transfer and adoption, manufacturing capabilities, access to skilled workers, and access to capital. Offset mechanisms such as the ITB Policy have been used effectively to address these and other types of constraints.  According to Jenkins report on defence procurement, federal government contracts play a critical role in the development of the industry. ^{Footnote 7}

Nevertheless, many key informants noted (and focus group participants agreed) that the extent of the activities undertaken by WD is not sufficient to adequately address the existing needs. In particular, WD defence procurement activities are often not sufficient to create business partnerships and/or long-term relationships between SMEs and prime contractors. Key informants, particularly obligors and SMEs, noted that WD is mostly reactive rather than proactive in its approach to ITB and defense procurement work. Case study reviews demonstrated that industry representatives do not receive communication from WD about upcoming defense and aerospace procurement opportunities and/or events. Those who attend the industry events and workshops related to the ITB Policy and defence procurement do not receive follow-ups on the outcomes of these events.

Results

Portfolio of IRB and ITB Obligations

As of January 2018, the portfolio of current IRB and ITB obligations across Canada included 79 projects representing $32.1 billion in obligations, of which $20.1 billion had been met, $8.8 billion was in progress, and $3.2 billion was yet to be identified (e.g. which represent future work opportunities). ^{Footnote 8} Of the 79 projects, 30 involve obligations of less than $100 million, 41 involve obligations of between $100 million and $1 billion, and eight involve obligations over $1 billion.

The progress made in fulfilling obligations is generally commensurate with the pace of work underway to deliver the goods and services being procured. Between 2011 and 2015, the IRB Policy resulted in $1.75 billion being invested in 375 small and medium-sized enterprises (SMEs). ^{Footnote 9} During the same time frame, the policy contributed to $82 million in research and development at 45 Canadian post-secondary and research institutions.

Capturing the Impacts of the ITB Policy in Western Canada

Aggregate data is not available on the extent to which companies, post-secondary institutions, and others in western Canada have captured IRB and ITB obligations (e.g., data on the value of obligations captured, contract values, types of expenditures, and regional and sector breakdown). While ISED tracks performance against obligations, it does not share that information with the RDAs. ^{Footnote 10} Other sources of this data are not available as most ITB and IRB related business deals are not reported publicly and WD is not in a position to collect and/or report information on the economic outcomes of its ITB related activities.

Key informants provided examples that illustrate the impact of ITB Policy and WD's activities. When asked to describe the types of impacts that the ITB and IRB obligations have generated in western Canada, key informants most frequently identified:

• development of ongoing relationships with prime contractors (identified by 38% of key informants);
• new investments in R&D activities and innovative technologies (36%);
• increased revenues (35%),
• technology development (29%);
• investment (28%);
• participation in global supply chains (24%);
• employment 23%); and
• development of HQP (16%).

In their interviews, key informants provided 13 examples of sales, projects or initiatives involving private sector companies in western Canada that arose from the IRB or ITB Policy. Results from the examples included the development of new relationships and partnerships, the sale of existing products and services, and the generation of new investments, transfer of IP rights, and/or development of new technologies in partnership with the prime contractors. Other results mentioned were the building the capacity of western Canadian companies to supply products and services as part of the prime contractor's global supply chain, and the establishment of a joint venture with a prime contractor to develop a state-of-the-art research facility and undertake R&D activities.

These examples involved a range of obligors and prime contractors including the Thales Group, Seaspan, Lockheed Martin, General Dynamics Mission Systems, Textron Systems, Mercedes-Benz, Honeywell, QinetiQ Air Drones, HD Technology, General Electric, Boeing, McDonnell Douglas, Westland Helicopters, Airbus and Harris Canada. Key informants also highlighted examples involving western Canadian post-secondary institutions and non-profit organizations, including:

• Development of a new laboratory and purchase of a new helicopter by the Saskatchewan Indian Institute of Technologies (SIIT), which increased its capacity to undertake research and teach mechanical and technical skills to students.
• Expanding the activities of the Canadian Light Source synchrotron, the national light source facility at the University of Saskatchewan.
• Increased technical skills among 500 graduates of the University of Victoria (UVIC) who were trained at the Centre for Aerospace Research within the university. The Centre was established with financial support from Boeing and received a number of contracts from other prime contractors such as Bombardier.
• Establishment of the STAR facility at the University of British Columbia Okanagan with financial support from Lockheed Martin, which aims to provide technical solutions to business challenges and support economic development and job creation in western Canada. Avcorp Industry is also working with UBC Okanagan to establishing a Learning Centre for Advanced Composites.
• Establishment of the Vancouver Institute for Visual Analytics (VIVA) at Simon Fraser University with financial support from Boeing, which is focused on development, research and training on the visual analytics.
• Establishment of the Canadian Composites Manufacturing R&D Consortium (CCMRD) involving Boeing, the Composites Innovation Centre, and the NRC Institute for Aerospace Research.
• Establishment of the $50 million Aviation Engine Testing, Research and Development Centre (TRDC) in Winnipeg. The TRDC hosts West Canitest R&D Inc. (WestCaRD), a not for profit organization that partners with GE and Standard Aero.
• Establishment of a high-tech Interaction Lab equipped with 3D printers, virtual reality technology, and a women in trades program at Camosun College (Victoria, BC).
• Establishment of an indigenous in Trades program at the British Columbia Institute of Technology (Burnaby, BC). BCIT also received funding from Lockheed Martin Aeronautics to help design and support creation of 3D simulation technology.

Many of the key informants noted that they are aware of one or more ITB or IRB contracts being awarded to companies and post secondary institutions in western Canada. A common perception from key-informants and focus group participants is that Ontario, Quebec and Atlantic Canada have tended to be more successful in capturing obligations (in nominal terms and relative to the size of the regional industry).

Representatives from industry and post-secondary institutions reported mixed results with respect to the success of their own organizations, with 40% indicating their organization was not at all successful in capturing IRB and ITB obligations, 40% had some success, 10% were very successful, 10% were still uncertain. Those that were very successful were representatives of post secondary institutions. Representatives who were not successful generally fell into three categories:

• Although aware of the Policy and possibly participating in one or more events, they have not had any meaningful interaction with prime contractors to date (apart from perhaps a B2B meeting);
• They have interacted with a prime contractor but the prime either was not able to win the bid, the application was not submitted, or a procurement decision has yet to be made;
• They were part of a successful bid by a prime contractor who eventually purchased the goods or services from another supplier (identified by four companies). Two industry representatives felt that the prime contractors used their companies in the application to win the bid and then replaced them with contractors outside of Canada who could provide the products less expensively.

In addition to the capturing of obligations, key informants identified two additional benefits that can result from the relationships developed with prime contractors because of IRB and ITB activities:

• SMEs develop a better understanding of the defence procurement process and begin to build relationships with prime contractors. Even companies that participate in the ITB bidding process but not get contracts may still benefit from development of those relationships with the prime.
• Representatives of the defense and aerospace industry that reported some success in capturing IRB and ITB obligations levered their initial sales to make further sales with the contractor as well as obtain business from other large contractors.

Attribution of Benefits to the Activities of WD

The activities of WD are only one factor, but an important factor, in enabling organizations in western Canada to capture IRB and ITB obligations. Of the key informants who expressed an opinion, 52% indicated that the impacts generated by IRB and ITB policies in western Canada can be attributed somewhat to activities and programming delivered by WD while 13% indicated that the impacts are very attributable. Case studies demonstrated that many of the regional events and activities (e.g., consultations, B2B sessions, conferences, workshops, etc.) would not have happened or been much smaller in the absence of the support from WD.

During the focus group discussions, most participants agreed that the role played by WD (and other RDAs ) is critical to successful implementation of the ITB Policy. In the absence of support from WD, prime contractors would find it much more difficult to identify regional capabilities and develop their offset strategy. Prime contractors may be less likely to bid on defense contracts in Canada, effecting the success of the procurement process, and less likely to incorporate organizations from western Canada into their bid.

Barriers to Accessing ITB and IRB Obligations

According to key informants and focus group participants, the major barrier preventing industry and post-secondary institutions from capturing a greater share of the IRB and ITB benefits is their limited understanding of the ITB Policy. Despite the progress made by WD, awareness of the ITB and IRB opportunities among stakeholders in western Canada remains low. Many of those familiar with the Policy are unsure how to participate and whom to approach. Those that approach prime contractors often are not equipped with necessary knowledge and skills.

Other constraints that were identified include:

• A shortage of system integrators who are able to serve as the vital link between the large Multinational Enterprises (MNEs) who commonly act as obligors and the small SMEs that characterize the aerospace and defence industry.
• Limited familiarity of bidders with industry capabilities in western Canada, the types of R&D projects undertaken by the western Canadian universities and post-secondary institutions, and the ITB Policy.
• Large contractors tend to be less active in western Canada because of the comparatively smaller size of the sector as well as the distance from Ottawa where most defense procurement activities occur.

A few key informants noted that the defense sector has not been a priority for government in western Canada and, therefore, information about the industry and its capabilities is very limited.

Design and Delivery

Impacts of the 2014 Changes to the Policy

In February 2014, the Government of Canada launched the ITB Policy to replace the IRB Policy. ^{Footnote 11} The objective of the new ITB Policy was to better leverage military procurements to provide economic benefits to Canadian industry. The introduction of value proposition, as part of defense procurement, significantly increased the timing and level of assistance from RDAs requested by prime contractors.  The change in policy has increased the resources that prime contractors invest during the bidding process and provides greater opportunities for Canadian suppliers to be included in the bid. The primary implication, according to key informants and focus group participants, is that WD needs to become more proactive in its approach to educating industry and post-secondary institutions about the ITB Policy and defence procurements, creating awareness of available opportunities, identifying and communicating regional capabilities, and organizing connections between the industry representatives and prime contractors.

Under the new Policy, the level of public reporting on results has increased. A website has been developed by ISED ^{Footnote 12} that identifies the obligors. For each obligor, the website annually reports on the total value of the total obligation, the obligation completed to date, in progress, and to be identified, and the estimated timeframe for each procurement project. However, industry representatives indicated that this level of detail did not give them sufficient guidance regarding potential opportunities that currently exist or will arise in the short, medium and longer-term.  It was also noted that the failure of ISED to share detailed data means that WD and the other RDAs are not in a position to manage their ITB and defence procurement activities. The lack of data makes it more difficult to monitor opportunities, build strong links of communication with industry, and track the results of their activities.

Comparison of Roles and Responsibilities

WD generally takes the lead role in working with provincial governments, industry associations and other federal departments in helping local industry and post-secondary intuitions capture benefits related to the ITB Policy. WD is in a unique position in that it represents western Canada and is familiar the capabilities in each province, is connected to ITB Policy and defence procurement nationally, and can provide a single source of contact for bidders and obligors.  As such, the roles and responsibilities of WD tend to complement rather than duplicate or overlap those of these other organizations. However, various parties identified the need for more consistent and ongoing communication between WD and these other organizations to better coordinate activities.

Capturing and Sharing Information About Industry Capabilities

WD's regional staff identify industry capabilities within their respective regions through various activities including internet searches, accessing third party databases, and undertaking visits to company production sites and offices. Other activities include attending industry events, conferences, and workshops, and speaking with partners such as industry associations and representatives of the provincial governments. Data are commonly entered into an Excel spreadsheet. When requests for information are received from ISED or from prime contractors, WD extracts the necessary information and shares it with them. WD staff identified the lack of a common methodology and database to store and share the information as a major limitation.

Contribution Towards Cluster Growth and Inclusiveness

WD's 2018-2019 Departmental Plan identifies two priorities: Cluster Growth and Inclusiveness. The Plan identifies the need to make strategic investments that accelerate the development and adoption of new technologies and advance selected key western Canadian clusters. For 2018-19, the priority focus for inclusiveness is Indigenous peoples, women, and youth.

The ITB Policy and WD's ITB activities can be important tools for pursuing the new priorities of cluster growth and inclusiveness:

• By leveraging opportunities related to Canada's defence and security projects, the ITB Policy can be very effective in facilitating the development of clusters. The impact will not be limited necessarily to aerospace and defence. ISED recently identified 16 Key Industrial Capabilities (KICs), covering a range of emerging technologies with the potential for rapid growth as well as established capabilities where Canada is globally competitive or where domestic capacity is essential to national security. The emerging technologies include advanced materials, artificial intelligence, cyber resilience, remotely piloted systems and autonomous technologies, and space systems. The Jenkins report recommended the adoption of KICs to better leverage the economic opportunities arising from the planned defence spending. ^{Footnote 13}
• Multipliers can be used as part of the value proposition to attract specific types of investments such as investments in R&D, commercialization, education and skill development needed to promote development of clusters. Specific initiatives associated with ITB obligations can also be targeted at groups that are prioritized under WD's inclusiveness priority. For example, a commitment made to Seaspan under the National Shipbuilding Strategy (NSS is supporting three institutional trades training programs towards indigenous peoples in trades.

There are opportunities to increase the contribution to WD's new priorities by improving coordination and achieving greater synergies between WD's ITB and defence procurement activities and its other programming. WDP funding, leveraged with funding from other sources, can be used to incentivize targeted investments. A review of WD documents and files indicates that, over the period covered by the evaluation, 14 projects related to aerospace and defence procurements were funded by WD including 4 WINN projects ($6.2 million in interest free loans) and 10 WDP projects ($6.4 million in funding). Stakeholders recommended the development of a regional development strategy that would help guide and ITB activities and investment decisions.

Governance Structure

The nature of activities related to the ITB and defence procurement varies somewhat from those of most other activities that WD undertakes. For example, the ITB activities tend to be focused largely on a particular cluster (i.e. aerospace and defence) or small group of clusters. The activities do not directly involve provision of grants and contributions. Defence procurement projects involve multiple regions, with staff in Ottawa playing a key role.

The nature of the activities creates a strong need for coordination across regions.  That can be a challenge give the decentralized governance structure (e.g. WD staff working on the ITB file report regionally), a lack of common tools and coordination mechanisms, and the extent to which staff are dedicated to the ITB Policy file (e.g. staff may work only part-time on the ITB file).

There is no ideal governance structure for delivering the ITB Policy and defence procurement activities.  Moving to a structure where the regionally-based staff reported nationally rather than regionally could ease some of the existing challenges but create challenges with respect to coordinating ITB activities with other activities within the region. Key informants highlighted the importance of adopting mechanisms and tools that can facilitate coordination both within the region and across regions.

Other issues and themes raised during interviews by WD staff and other stakeholders familiar with the governance structure included the need to:

• Develop effective mechanisms for coordinating activities and sharing information across regions.
• More clearly define the relative roles and responsibilities of the regions and Ottawa with respect to collecting and compiling information, staging events and undertaking other activities, and working with obligors. Senior management need to agree on the relative roles and responsibilities.
• Better coordinate ITB and defence procurement activities not only within and across regions but with other key stakeholders including the provincial government, and industry associations and others.
• Provide better training and support for WD staff working on the ITB Policy and defence procurement. It was noted that there had been considerable turnover in some of the positions and support needed to get new staff up to speed.

Efficiency and Economy

Comparison of Benefits and Costs

Activities related to defence procurement and ITB Policy are not established as a separate cost centre within WD, although estimates of FTEs were developed by the Department. The estimated number of FTEs dedicated to the file decreased over the past five years, from 9.9 in 2014-15 to 7.4 in 2017-18 and 7.8 in 2018-19. Approximately two-thirds of the FTEs are based in Ottawa.

Number of Full-time Equivalent Positions Involved in Defence Procurement and ITB Policy Activities By Region

Source: WD Policy and Strategic Direction Sector

O&M expenditures on defence procurement and ITB Policy activities are estimated to have ranged between $1 million and $1.5 million annually over the past five years.

The adequacy of the FTE complement is dependant upon WD's strategy with respect to the ITB Policy and defence procurement.  According to WD staff, existing levels are generally adequate under the current approach where WD is largely reactive.  However, additional resources would be required if WD was to get more actively involved in working with industry and post-secondary institutions to capture obligations and lever economic impacts.

The data required to directly compare the benefits generated by WD's activities to the costs is not available.  However, the investment is small relative to the value of obligations created under the ITB Policy. The value of the ITB and IRB obligations has averaged about $2.5 billion per year.  At that level, a strategy that increased the national share of obligations captured by western Canada by 1% would directly generate $25 million in additional value. Even a small increase in the share of the ITB and IRB obligations captured by companies and post-secondary institutions in western Canada would likely be sufficient to justify expenditures made by WD. Under Canada's new defence policy, Strong, Secure, Engaged ^{Footnote 14}, the rate of capital investment is expected to increase sharply. If this new defence policy is implemented as planned, capital expenditures will increase to about $12 billion annually by 2023 (as compared to $3.1 billion in 2016/17).

Performance Metrics

WD collects and reports data on the types of ITB and defense procurement activities delivered (e.g., participation in meetings, responding to requests for information, support for supplier development events and tours, WD led events, etc.). WD staff consider the existing performance measurement systems to be somewhat effective in reporting on WD's defence procurement and ITB activities.

Performance measurement would be improved by incorporating information on ITB obligations captured by organizations in western Canada and other outcome indicators. The major issue identified by WD staff and other key informants (particularly in the focus groups) is the lack of data on outcomes. It was recognized that tracking outcomes is difficult given that:

• The information and support provided by WD is only one of the factors that contributes to capturing obligations.
• Defence contracts can take many years to negotiate and information on deals is often not disclosed publicly.
• Creating successful relationships between an SME and a prime contractor may not bring economic benefits if the prime contractor does not win the contract from the government.
• The benefits of developing relationships between SMEs and contractors is not necessarily limited to a direct sale between the SME and obligor.

Key informants and focus group participants suggested that WD add outcome related indicators to measure performance of its ITB programming. Towards that end, it was recommended that WD should gain access to the data collected by ISED and conduct annual follow-up surveys with industry representatives. In addition, key informants noted that an effective strategy to promote the ITB Policy would be to collect information and publicize success stories.

Opportunities for Improvement

The evaluation highlights areas, summarized in the diagram below, where WD's ITB Policy and defence procurement activities could be enhanced to better meet the needs of bidders, obligors, regional industry and post-secondary institutions as well as contribute further to the achievement of WD's priorities of cluster growth and inclusiveness.

The first step will be for senior management to determine the level of priority that WD will place on increasing the share of ITB obligations captured by western Canada and better leveraging those commitments to promote cluster growth and inclusiveness.  Implementing a more proactive, strategic and consistent approach to leveraging the ITB Policy would likely require additional resources and strong support from senior management at both the regional and departmental levels and additional resources (i.e. increases in staffing, investment in process improvements, and leveraging existing WD programs towards industry development activities and incentivizing investments in key areas).

To increase the impact of its ITB Policy and defence procurement activities, WD could:

1. Take a more strategic approach to leveraging the ITB Policy by preparing a formal strategy, designing an approach for each procurement opportunity, maintaining more regular contact with key stakeholders, and working to further influence the development of value propositions.

WD's activities would benefit from:

• Preparing a formal development strategy, either independently or in association with other key stakeholders (e.g., provincial governments, industry associations, post-secondary institutions, etc.) involved in promoting cluster development and assisting regional businesses and post-secondary institutions capture ITB obligations. The strategy should provide a vision for development, outline existing supports, describe existing barriers and gaps, and identify priorities and required strategies. It should incorporate a pan-western view and be aligned with WDs new priorities, the recently defined Key Industrial Capabilities for the ITB Policy, and the Government of Canada's strategy on cluster growth.
• Designing an approach for each procurement opportunity, which outlines the associated opportunities for western Canada, identifies potential bidders and key intermediaries, defines the region's capabilities, outlines specific actions, and identifies partners.
• Maintaining regular contact with regional industry, post-secondary institutions, existing obligors and prospective bidders for upcoming procurements. WD could become more active in assisting post-secondary institutions to identify potential R&D, commercialization and educational opportunities associated with existing and upcoming procurements. In addition, WD could develop packages of potential investments that could be promoted to bidders and obligors.
• Working to influence development of value propositions to reflect western Canadian capabilities, opportunities and priorities including investments targeting priority groups.

2. Strengthen the governance system by clearly defining the relative roles and responsibilities of WD regional offices and Ottawa.

In particular, WD's activities would benefit from more clearly defining the relative roles and responsibilities of the regions and Ottawa with respect to collecting and compiling information, staging events and undertaking other activities, and working with obligors. Consideration could able given to establishment an advisory committee to facilitate sharing information and coordinating activities within and across regions. The function of the advisory committee would be to overcome existing challenges associated with coordinate ITB and defence procurement activities not only within and across regions but with other key stakeholders including the provincial government, and industry associations and others. Similar committees have been established by other RDAs .

 3. Strengthen service delivery by establishing a common capabilities database, adopting communication and collaboration software, establishing a shared portal, creating an advisory committee, developing shared seminar, presentation and supplier development tools and resources, and providing training for staff new to the file.

WD's activities would benefit from:

• A common database or CRM, accessible and updatable across offices, that contains information on company characteristics, capabilities, contact information, and interactions with WD.
• Adoption of communication, collaboration and work management software that would assist teams in organizing, tracking, and managing work associated with particular procurements. Using inexpensive off-the-shelf software, team members could define opportunities, agree on responsibilities and target dates, communicate, share information, files and calendars, and report on results.
• A portal that could be accessed by industry associations, provincial governments, bidders, obligors, SMEs, post-secondary institutions and others (similar to one developed by CED).
• Development of shared support materials such as seminar and presentation materials (e.g. on the ITB Policy, defence procurement process and upcoming procurements) and supplier development tools that help prepare SMEs for approaching and working with prime contractors.
• An onboarding or orientation program for staff new to the ITB Policy and defence procurement file.  There had been considerable turnover in positions and training would better enable staff to get up to speed.

4. Use program funding to pursue and further lever opportunities associated with the ITB Policy.

For example, WDP funding could further be used to support various industry development activities (e.g. funding industry associations for marketing, supplier days, supplier development, compiling data on industry capabilities, and developing an industry strategy). Funding could also be used to incentivize obligor investments in key priority areas (e.g. leveraging investments in R&D, demonstration projects, commercialization, business incubators, education and training, etc.). WINN funding could be used to support the development of new technologies, products and processes that would enable the capturing of benefits by companies in western Canada.

5. In association with the other RDAs , work with ISED to ensure access to the data needed to assess opportunities, develop strategies and track the results.

The lack of regional data makes it more difficult for WD to assess the regional impact of the ITB Policy, monitor opportunities and developments, build communication links with industry, and track the results of their activities. WD could complement regional data obtained from ISED on the capture of ITB obligations with other data on the impact of their activities collected through project reporting, data collected through the CRM database, and a periodic online survey of regional representatives who have participated in various activities.