¶ armasuisse Evaluation Model: Critical Analysis

¶ Summary

The Federal Office for Defence Procurement (armasuisse) used the AHP method (Analytic Hierarchy Process) with 79 individual criteria across four main categories for the type evaluation within the Air2030 programme for the first time. The F-35A achieved 336 points with a lead of at least 95 points over the runner-up. [1]

This page analyses the evaluation methodology, explains how the AHP method works, compares it with common evaluation methods used in defence procurement, and documents parliamentary and expert criticism of the evaluation model.

---

¶ 1. The armasuisse Evaluation Model in Detail

¶ 1.1 Overview: Two-Phase Evaluation Procedure

armasuisse conducted the evaluation in two separate phases -- a so-called two-envelope procedure: the technical specialists had no insight into the costs and vice versa. This was intended to ensure an unbiased assessment. [2]

Phase 1: Technical Evaluation (2019-2020)

• Flight tests of all four candidates at Payerne military airfield (April-June 2019)
• Ground tests and simulations
• Technical data collection for all 79 evaluation criteria

Phase 2: Commercial Evaluation (2020-2021)

• First request for proposals (2018)
• Second request for proposals (January 2020) with refined requirements [3]
• Cost proposals and calculation of life-cycle costs over 30 years

¶ 1.2 The Four Evaluation Criteria with Weightings

armasuisse evaluated the candidates based on four main criteria with fixed weightings: [2]

The resulting overall utility was then compared against the procurement and operating costs over 30 years (cost-benefit analysis). [2]

¶ 1.3 Evaluation Result

The F-35A achieved the best overall result with 336 points. The gap to the runner-up was at least 95 points. Notably, the F-35A won not only in effectiveness, but also in product support and cooperation. [1]

---

¶ 2. The AHP Method: How It Works and Scientific Classification

¶ 2.1 Basic Principle of the Analytic Hierarchy Process

The AHP method was developed in the 1970s by Thomas L. Saaty at the University of Pittsburgh and first published in 1980 in the book "The Analytic Hierarchy Process". [4]

Core idea: AHP breaks down complex decisions into a hierarchy:

Level 0: GOAL
         "Best fighter aircraft for Switzerland"
              │
Level 1: MAIN CRITERIA
         Effectiveness (55%) | Product Support (25%) | Cooperation (10%) | Offset (10%)
              │
Level 2: SUB-CRITERIA
         79 individual criteria at armasuisse
              │
Level 3: ALTERNATIVES
         F-35A | Rafale | Eurofighter | Super Hornet

¶ 2.2 The Saaty Scale (1-9)

Unlike classical scoring, AHP works with pairwise comparisons on the so-called Saaty scale: [4]

¶ 2.3 Mathematical Process

1. Construct pairwise comparison matrix: For n criteria, n(n-1)/2 comparisons are required
2. Calculate eigenvector: The principal eigenvector of the matrix yields the weights
3. Consistency check: Calculation of Consistency Index (CI) and Consistency Ratio (CR)
   • CR = CI / RI (Random Index)
   • Acceptance criterion: CR ≤ 0.10 (maximum 10% inconsistency)
   • If CR > 0.10, pairwise comparisons must be revised
4. Aggregation: Local priorities multiplied by weights yield global priorities

¶ 2.4 The Contrast Effect -- Critical AHP Phenomenon

A central problem of the AHP method is the contrast effect: small real differences lead to disproportionately large point differentials.

Thomas Ferber, an AHP expert, warned about this effect as early as 2021 and compared it to a "contrast slider in Photoshop". [5] His assessment: the conventional utility analysis is "fairer because it is more accurate and differentiated".

Representatives of the unsuccessful manufacturers described the AHP evaluation as a "black box". [5]

¶ 2.5 The armasuisse "Transfer Function"

armasuisse was aware of the contrast effect and incorporated a so-called "transfer function" to dampen it. [2] However, the exact workings of this transfer function were not publicly documented -- a point of criticism that makes the evaluation difficult to verify.

---

¶ 3. Comparison with Common Evaluation Methods

With the AHP method, armasuisse used a procedure that is uncommon in international defence procurement. The following table compares AHP with established alternatives:

¶ 3.1 Classical Utility Analysis (NWA/MAUT)

¶ 3.2 Other Methods Overview

PROMETHEE (Preference Ranking Organization Method for Enrichment Evaluation)

• Developed by Jean-Pierre Brans (1982)
• Outranking method with preference functions
• Advantage: Accounts for incomparability
• Usage: Academic studies, rarely commercial procurement

ELECTRE (ELimination Et Choix Traduisant la REalité)

• French outranking method (Bernard Roy, 1960s)
• Allows veto thresholds (knock-out criteria)
• Criticised as a "black box"
• Usage: France, EU projects

Best Value Procurement (US DoD Standard)

• Dominant method in US procurement (FAR 15.101-1)
• Trade-off between technical merit and price
• Wide discretionary margins
• Usage: US DoD, NATO procurement

Total Cost of Ownership (TCO)

• Focus on life-cycle costs (30+ years)
• Considered alongside the AHP utility value at armasuisse
• High forecasting uncertainty

¶ 3.3 Comparison Table

¶ 3.4 Conclusion on Methods Comparison

The AHP method is scientifically sound and offers an advantage over classical utility analysis through its consistency check. However, the contrast effect is a significant problem, which armasuisse attempted to resolve through an undocumented "transfer function".

The classical utility analysis would have been more transparent and would have provoked less criticism. In international defence procurement practice, AHP is an outlier -- Finland, Belgium, Canada, and Germany used different methods.

---

¶ 4. External Plausibility Check by Homburger AG

¶ 4.1 Commissioning and Costs

On 27 April 2021, the DDPS under Federal Councillor Viola Amherd commissioned the law firm Homburger AG verbally to conduct a plausibility check of the evaluation result. The costs for the initial expert report amounted to CHF 550,000. [6]

Total costs of DDPS mandates to Homburger (2021-2025): approximately CHF 2.5 million for over 9 mandates. [6]

¶ 4.2 Result and Disclaimer

Homburger concluded that the ranking of bidders was "plausible". [7]

However, the report contained a far-reaching disclaimer: Homburger excluded any responsibility for the accuracy and completeness of the results. [6]

¶ 4.3 What Was NOT Examined?

The Control Committee-N found that Homburger had not examined essential aspects: [7]

• No legal review of the fixed-price assurances
• No validation of FMS risk allocation
• No independent review of life-cycle costs

The financial review was delegated to Gut Corporate Finance. [6]

¶ 4.4 Review of the Plausibility Check

On 18 June 2021, the Federal Office of Justice (FOJ) was tasked with "plausibilising" the Homburger report -- a plausibility check of the plausibility check. [6]

---

¶ 5. Criticism of the Evaluation Model

¶ 5.1 SFAO Criticisms (Audit Report No. 21410)

The Swiss Federal Audit Office (SFAO) published Audit Report No. 21410 on Air2030 risk management on 8 July 2022, with the following key criticisms: [8]

• "No legal certainty for fixed price" in the sense of a lump sum under Swiss case law
• Uncertainty regarding maintenance costs over the entire service life
• The Homburger brief report was called into question
• Operating costs are based on estimates, not binding commitments

¶ 5.2 Control Committee-N Report (9 September 2022)

The Control Committee of the National Council (GPK-N) formulated five recommendations regarding the procurement process: [7]

1. The Federal Council unnecessarily restricted its room for manoeuvre -- it could not factor in foreign policy aspects into the type selection
2. The further processing of evaluation results by the DDPS was "partially inappropriate"
3. The AHP method was applied for the first time in a significant procurement, which entails "certain risks"
4. It is not comprehensible why armasuisse dispensed with the experience of F-35 operator countries
5. The practice of "closing meetings" should be reviewed

Federal Council's response: One recommendation partially accepted, one rejected, three considered "already fulfilled". [7]

¶ 5.3 Expert Criticism

Weighting issue:

• The high weighting of "effectiveness" (55%) may have favoured the most technologically advanced offering (F-35A with stealth technology)
• Sovereignty risks (ITAR restrictions, FMS dependency) were possibly underweighted within the "cooperation" criterion

Cost comparability:

• FMS costs (government-to-government) and direct purchase costs (Rafale, Eurofighter) were possibly not made methodologically comparable
• FMS is based on estimates, while European offers contained fixed prices

¶ 5.4 Bias Allegations (Secondary Sources)

Unsuccessful manufacturers and critical media raised the following allegations: [9]

• Inconsistencies "towards the end of the evaluation process"
• Late change of payment terms (May 2021) disadvantaged certain bidders
• "Closing meetings" with the agenda item "handling of data" (document destruction allegations)

These allegations were rejected by armasuisse. A definitive assessment is not possible due to the classification of the evaluation report.

---

¶ 6. International Comparison: Evaluation Methods

¶ 6.1 Finland: HX Challenge (2015-2021)

• Methodology: Multi-Criteria Decision Analysis (MCDA) with external review by VTT Technical Research Centre
• Candidates: 5 (F-35A, Rafale, Eurofighter, Super Hornet, Gripen E)
• Transparency: High -- comprehensive public report published
• Parliamentary involvement: Yes
• Result: F-35A (December 2021) [10]

¶ 6.2 Belgium: Fighter Capability Programme (2014-2018)

• Methodology: Cost-efficiency model (TCO-oriented)
• Candidates: Formally 2 (Eurofighter withdrew)
• Transparency: Medium
• External review: None known
• Result: F-35A (October 2018)

¶ 6.3 Canada: Future Fighter Capability Project (2010-2023)

• Methodology: Following an initial sole-source scandal, open scored evaluation with weighted criteria
• Candidates: 3 (F-35A, Gripen E, Super Hornet)
• Transparency: High (after reform)
• External review: Auditor General of Canada
• Result: F-35A (March 2022) [11]

¶ 6.4 Germany: Tornado Replacement (2022)

• Methodology: No open evaluation -- political pre-decision
• Rationale: NATO nuclear sharing requires US certification
• Candidates: Effectively only F-35A
• Result: F-35A (March 2022) [12]

¶ 6.5 Comparison Table

*Eurofighter withdrew its offer

¶ 6.6 Observation

It is noteworthy that all F-35 purchasing countries arrived at the same result despite using different evaluation methods. This can be interpreted as confirmation of the F-35A's technical superiority -- or as an indication that the evaluations were less open-ended than portrayed.

---

¶ 7. Assessment of Methodological Quality

¶ 7.1 Objectivity

Pro:

• Technical tests were conducted by armasuisse experts
• Two-envelope procedure prevented bias

Contra:

• No independent external review of the evaluation model itself
• Criteria weightings were determined internally

¶ 7.2 Reliability

Pro:

• Structured scoring system with defined criteria
• Consistency check (AHP)

Contra:

• Different weightings would have led to different results
• Transfer function not documented

¶ 7.3 Validity

Pro:

• Effectiveness as a core criterion is appropriate for a fighter aircraft

Contra:

• Sovereignty aspects (ITAR, FMS dependency) not adequately represented
• Political risks of US dependency not evaluated

¶ 7.4 Transparency and Democratic Oversight

• The complete evaluation report is classified
• Parliamentary inspection rights (SiK-N, SiK-S, FK-N, FK-S) were restricted
• By comparison: Finland published a comprehensive public report

---

¶ 8. Key Points of Criticism (Synthesis)

¶ 8.1 Structural Problem: The Evaluator Decides

armasuisse was simultaneously the commissioning body, executor, and beneficiary of the evaluation. An independent external review -- as in Finland by VTT or in Canada by the Auditor General -- did not take place.

¶ 8.2 The Weighting Problem

The high weighting of "effectiveness" (55%) favoured a technologically leading offering. The question of whether this was politically intended remains open.

¶ 8.3 The Cost Problem

FMS costs are based on estimates, while European manufacturers offered fixed prices. Whether these were made methodologically comparable is not transparent.

¶ 8.4 The Transparency Problem

The classification of the evaluation report prevents independent verification and impedes democratic oversight.

¶ 8.5 The Methodology Problem

The AHP contrast effect and the undocumented "transfer function" raise questions about traceability.

---

¶ 9. Chronological Timeline

---

¶ 10. Source Directory

[1] VBS Medienmitteilung: Typenentscheid F-35A (30.06.2021)

[2] armasuisse Insights: AHP-Methode

[3] armasuisse: Zweite Offertanfrage (Januar 2020)

[4] Saaty, T.L. (1980): The Analytic Hierarchy Process. McGraw-Hill. / Saaty, T.L. (2008): Decision making with the analytic hierarchy process. Int. J. Services Sciences, Vol. 1, No. 1, pp.83-98.

[5] SRF: Juristisch heikle Bewertungsmethode beim Kampfjet-Kauf

[6] NZZ: Die Anwälte von Homburger spielten beim Kauf der Kampfjets eine umstrittene Rolle

[7] GPK-N Bericht vom 9. September 2022 (PDF)

[8] EFK Prüfbericht Nr. 21410: Air2030 Risikomanagement (PDF)

[9] Defense-Aerospace: How Switzerland Manipulated Data to Favor the F-35

[10] Finnish Ministry of Defence: HX Fighter Programme

[11] Government of Canada: Future Fighter Capability Project

[12] BMVg: Tornado-Nachfolger

[13] GPK-N Medienmitteilung: Inspektion Festpreis (01.07.2025)

---

¶ Verification Protocol

---

Last updated: 6 February 2026