Maciej Buczak

Risk Management & Research

Maciej Buczak

Risk Management & Research

99.9% Quantile, Rarity, Unexpected Situation — Ambiguities of the AMA Method
EN | PL
Diagram sieciowy
W hy was AMA — long regarded as Basel's most precise tool for pricing operational risk — withdrawn from regulation? The answer lies in three concepts the regulator left undefined: the rarity of an event, the interpretation of the 99.9% quantile, and the nature of the "unexpected situation." Filled for years with intuition rather than precision, these gaps led institutions applying the same formula to widely divergent results. It is to these three concepts, and their insufficient definitions, that this article is devoted.
OPERATIONAL.risk MODEL.risk
What Are We Hedging Against?

Paragraph 667 of the Basel II regulation (BCBS, 2004) states: „Given the continuing evolution of analytical approaches for operational risk, the Committee is not specifying the approach or distributional assumptions used to generate the operational risk measure for regulatory capital purposes. However, a bank must be able to demonstrate that its approach captures potentially severe ‘tail’ loss events. Whatever approach is used, a bank must demonstrate that its opera-tional risk measure meets a soundness standard comparable to that of the internal ratings-based approach for credit risk (i.e. comparable to a one year holding period and a 99.9th per-centile confidence interval).”

The above provision addresses three important aspects that must be considered at the out-set, before any further, more detailed assumptions regarding the operational risk model are formulated. The first of these is the problem of defining what, in essence, constitutes an unex-pected situation against which a financial institution should protect itself. The second concerns the level of protection formulated in statistical terms. The third issue arises directly from the two preceding ones. It is the question of how the concept of "rarity" is to be understood. This question will be addressed first.

Download full version of article

Download
Opis zdjęcia
What Is Rarity?

The Basel II regulation (BCBS, 2004) was designed to support the safety of the financial sec-tor. The concept of RWA and capital buffers was aimed at unexpected situations and at ensur-ing the survival of financial institutions under such rare circumstances.

Concept of rarity is inextricably linked to the concept of a typical event — average, everyday. It is precisely the definition of a typical event that determines the perception of what constitutes a rare one.

Rare — meaning what exactly? The regulator formulated the concept of rarity on the time axis. This means that the intention is to protect against a situation occurring no more than once in some given period of time. This is a significant assumption, and one that carries conse-quences. A phenomenon can be described in various ways depending on the axis on which one chooses to place it. This is most clearly visible when a rare event is considered alongside other observations — those that are typical, and therefore not rare — that are placed on the same axis. These will constitute a reference point (sometimes of decisive importance, as will be demonstrated later) for the rare observation. It is thus apparent that the concept of rarity is inextricably linked to the concept of a typical event — average, everyday. It is precisely the definition of a typical event that determines the perception of what constitutes a rare one. Different types of non-rare (typical, average) observations will be placed: (a) on the time axis — where the non-rare observations are years in which the level of the phenomenon is typical and expected; and (b) on the value axis — where the non-rare observations are values of the phenomenon at an average, typical level. One may, for instance, have at one's disposal the following sets of observations: 10 years with typical values, or 300 observations of the phenom-enon with typical values. This represents a fundamental difference in the way of looking at the modeled phenomenon. This distinction affects the perception of what constitutes a rare (ex-treme) observation, on what its interpretation as rare depends, and how the probabilities as-signed to it are to be understood.

Example 1: A Rare Event on Different Axes

Problem: what is the probability of an extreme road traffic loss (total loss, complete destruction of the vehicle). Assume that the considerations concern a single vehicle. The problem can be approached in two ways:

A. Placing the Problem on the Time Axis

In this case, one says that such a loss may occur with respect to a single vehicle once every X years (e.g., once every 10 years). Rarity is here expressed in relation to the observation of pass-ing time: one must wait a long time for a loss of this level of extremity. This extreme loss can be expected once every 10 typical, ordinary years.

B. Placing the Problem on the Phenomenon Axis: Road Traffic Losses

In this case, all losses and breakdowns experienced by the vehicle would be observed. Rarity would be expressed as once every X losses (breakdowns). Under this approach, a total (extreme) loss can be expected, for instance, once every 100 breakdowns — that is, once every 100 ordi-nary, typical events.

The consequences of choosing one of the above axes for describing the phenomenon become clear-ly visible when the reasoning is applied to two different vehicles. Let one of them have an aver-age annual mileage of 50,000 km, and the other 5,000 km. The first vehicle is exposed to a total loss at a rate ten times greater than the second. Additionally, as a result of its intensity of use, it accumulates a large number of ordinary, typical events and breakdowns. The second, by con-trast, has low exposure to total loss, and few observations of ordinary events — these simply had no opportunity to occur in large numbers. The application of approach A or B in these cases changes the way in which the modeled phenomenon is perceived.

The Problem with the 99.9% Quantile

The quantile level (99.9%) proposed by the regulator, consistent with the level established for credit risk, attracted considerable controversy from the outset. This centered on the perception of excessive conservatism. The consequences of adopting such a high level were pointed out: serious problems with the reliability of results at such a distant quantile, as well as very high capital levels corresponding to the 99.9% threshold.

Opis zdjęcia

Behind this perception of the aforementioned quantile as excessively conservative lies a cer-tain deeply rooted intuition associated with its interpretation. This intuition points to the inte-gration of the concept of a distribution quantile with the time horizon defined in the modeling of the phenomenon under study. With respect to the risk models described in (BCBS, 2004), the horizon was explicitly indicated as annual. This meant that the AMA advanced measure-ment model was to be capable of determining: (i) the expected value, which was the annual value of operational losses (that is, the sum of all annual losses); as well as (ii) the level of un-expected losses (99.9%), also understood as the sum of annual losses. With respect to unex-pected loss levels, it can be said that establishing capital buffers (the capital requirement) at the level of the 99.9% quantile reading of the aggregate loss distribution was intended to pro-tect the bank against the total amount of annual losses that would materialize in that excep-tionally adverse, extreme, and rarely occurring year.

The integration of the quantile level con-cept with the study horizon is entirely justified. Such reasoning leads, however, to the com-monly accepted interpretation of the 99.9% quantile as the level of annual losses occurring with a probability of once in 1,000 years. This interpretation was explicitly indicated by the regulator (BCBS, 2011): “Just in limited cases, for example when the biggest data point is demonstrated to be larger or equal in amount to the event that occurs less than once in 1000 years, the use of empirical curves to estimate the tail may be accepted in SBA models.”. The temporal interpretation, much like the 99.9% quantile itself, was borrowed from the conceptual framework formulated in the domain of credit risk and quickly became widespread in the fi-nancial sector. It can be found in academic textbooks such as Jorion (2007), Hull (2018), Laubsch and Ulmer (1999), as well as in numerous publications: Danielsson (2000) and (2003), Reinhart and Rogoff (2009), and many others.

Once in 1,000 Years

Adopting such a widely used interpretation of the 99.9% quantile leads to a line of reasoning in which the view of the quantile as excessively conservative appears quite justified. For how can one reasonably expect to protect against a phenomenon occurring once every 1,000 years? What approach could be taken to at least attempt to approximate a hypothetical situation of such frequency and to determine the expected loss volume associated with it? It would seem that such an approach — supported by the regulator — undermines from the outset the very sense and rationality of the modeling effort undertaken.

Indeed, it is impossible to imagine a situation that is to occur 1,000 years hence — let alone its financial consequences, since by that time the very concepts of finance may no longer be in use. It would be equally difficult for a hypothetical modeler in the year 1000 AD to have pre-dicted a stock market crash in the year 2000. These are temporal distances that preclude any possibility of placing events separated by such intervals on a single axis. One can certainly not speak of any consistency in the causes of events so distant from one another — they do not belong to a homogeneous class, they do not share the iid (independent and identically distrib-uted) properties. Nor does this interpretation follow from the mechanics of the framework — that is, from the way in which the aggregate distribution is constructed (through the combina-tion of frequency and severity distributions), from which the quantile (99.9%) is read. It is therefore not an accurate interpretation.

Once in 1,000 Possible Scenarios

Drawing on the laws of statistics, the only thing that can be said about the 99.9% quantile reading from the aggregate annual loss distribution is that once in every 1,000 such readings from the distribution, a value may occur that corresponds to the 99.9% quantile. It should be noted that temporal distances (e.g., once every 1,000 years) are not in any way embedded in the construction of the aggregate loss distribution. This distribution is constructed by combin-ing the various possibilities of annual loss counts with the corresponding various possibilities of loss amounts for those counts. The aggregate loss distribution therefore represents rather a potential space of possibilities for annual aggregate losses — that is, what the scale (disper-sion) of possible occurrences of annual loss aggregates looks like. It would therefore seem that the most adequate interpretation of the 99.9% quantile is one that says the value at this quan-tile is the second-to-last of the 1,000 worst possible scenarios — a present-day scenario, one that could plausibly materialize within the coming year.

To illustrate the differences in interpretation and their consequences, let us consider an ex-ample in which the same question is approached with different answers arising from different understandings of the quantile.

Example 2: The Difference Between the Temporal and Scenario-Based Interpretation of the Quantile

Question: Is it possible that Poland will become a powerhouse in automobile manufacturing?

A. Answer at the level of the 99.9% quantile understood as once in 1,000 years: this is possible (over such a long period of time, a complete reshuffling of the structure of economic forces in the world is conceivable).

B. Answer at the level of the 99.9% quantile understood as once in 1,000 scenarios: this is not possible (it is self-evident that even under the most favorable circumstances, dominating the global automobile market within the space of a single year is not feasible).The Correct Interpre-tation of the Quantile and the Concept of Rarity

The Correct Interpretation of the Quantile and the Concept of Rarity

The reasoning presented here regarding the correct interpretation of the 99.9% quantile as one in a thousand plausible annual scenarios (here and now — that is, within the coming year) brings greater precision to the concept of rarity that follows from such reasoning. Rarity should in this case be interpreted as an exceptional year in terms of operational loss values. This atypical year is, however, placed on an axis together with other annual periods that could materialize today — not on a linear, historical timeline. Rarity is therefore understood here as the exceptionality of a given scenario of events relative to the entire spectrum of other scenari-os, where the explicitly indicated annual period serves merely as an aggregating formula for the operational risk phenomenon — it is the aggregate of all losses within a scenario, that is, losses that may occur over the course of a year.

The Problem of Defining the Unexpected Loss Situation

The previously cited paragraph 667 of the Basel II regulation (BCBS, 2004) provides no guid-ance as to how the concept of an unexpected loss situation is to be generated for the purposes of its measurement. All that is known is that this situation should encompass potentially se-vere "tail" events. The risk measure should also meet a credibility standard comparable to that defined in the IRB method for credit risk (the 99.9% quantile). This under-definition of the AMA framework allows for considerable dispersion in the understanding of what constitutes an unexpected loss situation.

Referring to the method indicated by the regulator as a point of reference, the IRB formula has a certain predefined shape of the unexpected loss situation. In accordance with the IRB framework, an unexpected loss situation is a deterioration in the quality of all exposures (as measured by the probability of default) in the bank's portfolio. Thus, each exposure in an un-expected loss situation will be described by a higher PD ratio than in the expected loss situa-tion . Exposures differ from one another only in the magnitude of the PD level shift, which depends primarily on the risk class in which the exposure currently resides — that is, in the expected loss situation. Additionally, this method predefines the distribution of the PD varia-ble, along which the aforementioned shifts occur, as a normal distribution.

T he interplay between frequency and severity phenomena was meant to resolve, at the level of each institution applying the method, the ultimate shape of this extreme, dangerous situa-tion. The aggregate loss distribution arising from the combination of these two phenomena was intended to have the shape of that situation embedded in its definition, which in turn was to follow from the relative strength of these two components.

In the AMA method, only the actuarial construction was predefined, which required sepa-rate and independent modeling of two phenomena: the frequency of operational events and the severity of a single loss. The remaining assumptions were to be formulated independently by the bank applying the AMA method, as part of the development of its operational risk model. This constituted a significant complication.

Beyond describing the properties of the processes generating the individual components of the operational risk phenomenon (the selection of frequency and severity distributions), a cru-cial question — one that requires careful conceptual consideration — is the determination of the type of unexpected loss situation:

(a) a situation in which the threat and its scale are determined by a single high-severity event,

(b) a situation in which the threat and its scale are determined by the mass/volume of events, without a single dominant extreme event.

This is a fundamental reflection on the nature of the threat against which protection is sought. Resolving this dilemma requires a deep understanding of the mechanisms generating operational risk within a bank, including knowledge of their properties in atypical situations — which is rarely identical to the expected loss situation. Such an in-depth analysis is indispensa-ble for the correct formulation of the modeled phenomenon.

The Basel framework does not, however, address this question. The grounding of the meth-od in an actuarial foundation — with separate modeling of the two phenomena (frequency, severity) and the concept of their convolution — was intended to remove this dilemma from the path. The interplay between frequency and severity phenomena was meant to resolve, at the level of each institution applying the method, the ultimate shape of this extreme, danger-ous situation. The aggregate loss distribution arising from the combination of these two phe-nomena was intended to have the shape of that situation embedded in its definition, which in turn was to follow from the relative strength of these two components.

Key Takeaways

" The regulator never defined what AMA was actually meant to protect against.
Paragraph 667 of Basel II only required that the model capture "potentially severe tail loss events" and meet a soundness standard comparable to IRB — but it never specified whether the threat should come from a single, extreme event or from a mass accumulation of smaller losses. This left a fundamental conceptual gap for banks to resolve on their own.

" The "once in 1,000 years" interpretation of the 99.9% quantile is wrong — and the article offers the correct alternative.
The widely accepted temporal reading doesn't follow from how the aggregate loss distribution is actually constructed, and leads to absurd conclusions (as the Poland-as-car-manufacturing-powerhouse example shows). The correct interpretation: the 99.9% quantile represents the second-worst of 1,000 plausible annual scenarios — one that could materialize as early as next year, not a thousand years from now.

" The lack of precise definitions produced divergent results across institutions — and ultimately led to the method's withdrawal.
Because all three key elements — rarity, the quantile, and the unexpected situation — were left underdefined, different institutions applying the identical AMA formula arrived at widely different capital levels. This became one of the central reasons for the method's withdrawal under CRR III.

From the article "99.9% Quantile, Rarity, Unexpected Situation — Ambiguities of the AMA Method"

As can be seen, the nature of the threat that was to affect us at the distant quantile was left unspecified. It was instead assumed that knowledge of the independent forces of operation-al risk would translate into the correct shaping of the unexpected loss situation at the 99.9% quantile of the aggregate distribution. The results of such reasoning will be presented in the further part of this article.

It should be noted, however, that already at this level — that of the foundational assump-tions — a high potential for conceptual divergence across market participants emerges. These hypothetically different definitions of the unexpected loss situation could certainly have result-ed in significant differences in capital buffer levels, which was one of the key reasons for the withdrawal of the AMA method in CRR III. ■

Share this article
LinkedIn X
Maciej Buczak

Maciej Buczak

Risk management professional with 8+ years validating financial models in banking, currently teaching risk management and econometrics in Warsaw. His research questions the paradigmatic foundations of financial risk architecture — from VaR to AMA and IRB — drawing on extreme value theory.

References Academic sources cited throughout this article — click to expand the full list

Basel Committee on Banking Supervision (2004), International Convergence of Capital Measurement and Capital Standards: A Revised Framework, Bank for International Settlements. (Basel II)

Bank for International Settlements, Basel Committee on Banking Supervision (2011), Operational Risk – Supervisory Guidelines for the Advanced Measurement Approaches, Basel Committee on Banking Supervision, June 2011.

Danielsson, J. (2000), The Emperor has no Clothes: Limits to Risk Modelling, Financial Markets Group Special Paper No. 126, London School of Economics, London.

Danielsson, J. (2003), On the Feasibility of Risk Based Regulation, CESifo Economic Studies, 49(2), pp. 157-179.

Hull, J.C. (2018), Risk Management and Financial Institutions, Fifth Edition, John Wiley & Sons, Hoboken.

Jorion, P. (2007), Value at Risk: The New Benchmark for Managing Financial Risk, Third Edition.

Laubsch, A.J., Ulmer, A. (1999), Risk Management: A Practical Guide, First Edition, RiskMetrics Group, August 1999.

Reinhart, C.M., Rogoff, K.S. (2009), This Time is Different: Eight Centuries of Financial Folly, Princeton University Press, Princeton.

Link skopiowany do schowka!
Reader Discussion



Share
your thoughts

Tip: Choose the category that best describes your comment. This may help other readers. See more

Filter comments: