In the previous post, we have been discussing conventional approach to the portfolio optimization, where assets' expected returns, variances and covariances were estimated from historical data. Since these parameters affect optimal portfolio allocation, it is important to get their estimates right. This article illustrates how to achieve this goal using Black-Litterman model and the technique of reverse optimization. All examples in this post are build around the case study implemented in Python.
Instability of asset returns
Empirical studies show that expected asset returns explain vast majority of optimal portfolio weightings. However, extrapolation of past returns into the future doesn't work well due to a stochastic nature of financial markets. Nevertheless, our goal is to achieve optimal and stable asset allocation, rather than trying to predict future market returns.
The world of efficient markets
To maximize investor's utility function, Modern Portfolio Theory suggests holding the combination of a risk-free asset and optimal risky portfolio (ORP) lying on a tangency of the efficient frontier and capital market line. Modern Portfolio Theory also suggests that optimal risky portfolio is a market portfolio (e.g. capitalization-weighted index).
If we assume markets are fully efficient and all assets are fairly priced, we don't have any reason to deviate from the market portfolio in our asset allocation. In such case, we don't even need to know equilibrium asset returns nor perform any kind of portfolio optimization, as outlined in the previous article. An optimization based on equilibrium asset returns would lead back to the same market portfolio anyway.
An active investor's view
The different situation is when investor believes the market as a whole is efficient, but has concerns about the performance of specific assets or asset classes due to the possession of material non-public information, resulting for example from superior fundamental analysis.
If investor with distinct views on a performance of few specific securities doesn't want to completely abolish the idea of mean-variance optimization, he may still use quantitative approach to incorporate such view into the MVO framework, further referred to as the Black-Litterman model.
Black-Litterman model
In the previous article, we have been discussing classic Mean-Variance optimization process. The process solves for asset weights which maximize risk-return trade-off of the ORP portfolio, given historical asset returns and covariances:
On the other hand, the general workflow of the Black-Litterman optimization is illustrated on the diagram below.
However, the thing which differs is a way how we are observing expected asset returns, which is one of the inputs into the forward optimizer (g). Previously we have used historical asset returns as a proxy.
Now, as we can see in the diagram, equilibrium asset returns are used instead (d or g).
Equilibrium asset returns (d) are returns implied from the market capitalization weights of individual assets or asset classes (a) and historical asset covariances (b) in a process known as reverse optimization (c).
Forward (h) and reverse (c) optimizations are mutually inverse functions. Therefore, forward-optimizing equilibrium asset returns (d) would yield to the optimal risky portfolio (j) with exactly the same weights as the weights observed from assets' capitalization (a).
However, the beauty of the Black-Litterman model comes with the ability to adjust equilibrium market returns (f) by incorporating views into it and therefore to get optimal risky portfolio (j) reflecting those views. This ORP portfolio may be therefore different to the initial market cap weights (a).
Case Study
We will build upon the case study introduced in the previously published post Mean-Variance portfolio optimization. In that post, we have used forward optimizer to calculate weights of an optimal risky portfolio \(\mathbf(W_{n \times 1})\) given historical returns \(\mathbf(R_{n \times 1})\) and covariances \(\mathbf(C_{n \times n})\) Now, we will use the same case study to illustrate the principle of the Black-Litterman model.
Before doing so, let us define some basic symbols:
| \(\mathbf{R}\) | Vector of historical asset returns |
| \(\pi\) | Vector of implied equilibrium returns |
| \({\pi}'\) | Vector of view-adjusted implied equilibrium returns |
| \(\mathbf{C}\) | Matrix of historical return covariances |
| \(\mathbf{W}\) | Vector of market capitalization weights |
| \(\mathbf{W}'\) | Vector of optimal risky portfolio weights |
| \(\lambda\) | Risk-return trade-off |
| \(\mathbf{Q}\) | Vector of views about asset returns |
| \(\mathbf{P}\) | Matrix linking views to the portfolio assets |
Reverse optimization of equilibrium returns
Equilibrium market returns are those implied by the capitalization weights of market constituents. They represent more reasonable estimates than those derived from historical performance. The another advantage is that equilibrium returns (whether or not adjusted) will yield into the portfolio weights similar than weights of the capitalization weighted index, thus representing more intuitive and investable portfolios.
Equilibrium excess returns \(\pi\) are derived in Black-Litterman using this formula:
$$ \lambda = \frac{r_p - r_f}{\sigma_p^2} $$
$$ \pi=\lambda \cdot \mathbf{C} \cdot \mathbf{W} $$
In our example, corresponding code is represented by the following snippet. Please note that excess returns don't include risk free rate.
# Calculate portfolio historical return and variance mean, var = port_mean_var(W, R, C) # Black-litterman reverse optimization lmb = (mean - rf) / var # Calculate return/risk trade-off Pi = dot(dot(lmb, C), W) # Calculate equilibrium excess returns
As we can see, equilibrium returns implied from capitalization weights represent more conservative estimates than those calculated from historical performance. For example, implied return on Google \(\pi_6\) is "only" 16.5% given its capitalization weight of 11.6%, as opposed to unjustified historical performance \(\mathbf{R}_6\) of 33.2%.
| Name | \(\mathbf{W}\) | \(\pi\) | \(\mathbf{R}\) |
| XOM | 16.0% | 15.6% | 7.3% |
| AAPL | 15.6% | 17.9% | 13.0% |
| MSFT | 11.2% | 15.2% | 17.2% |
| JNJ | 9.7% | 9.9% | 14.4% |
| GE | 9.4% | 17.9% | 14.0% |
| GOOG | 11.6% | 16.5% | 33.2% |
| CVX | 9.2% | 17.3% | 9.2% |
| PG | 8.5% | 9.4% | 11.2% |
| WFC | 8.7% | 21.8% | 26.9% |
Efficient frontier based on historical returns is charted in red, while implied returns are in green:
As we have already mentioned, the whole idea behind the reverse and forward optimization in Black-Litterman model is a process of incorporating custom views to the returns. The good news is that the whole process can be expressed as mathematical formula, while the bad news is that the steps to derive this formula are beyond the scope of this article.
Given the views and links matrices \(\mathbf{Q}\) and \(\mathbf{P}\), we calculate view-adjusted excess returns \({\pi}'\) as follows:
$$ \tau = 0.025 $$ $$ \omega = \tau \cdot \mathbf{P} \cdot \mathbf{C} \cdot \mathbf{P}^T $$ $$ {\pi}' = [(\tau \mathbf{C})^{-1} + \mathbf{P}^T \omega^{-1} \mathbf{P}]^{-1} \cdot [(\tau \mathbf{C})^{-1} \pi + \mathbf{P}^T \omega^{-1} \mathbf{Q})] $$
, where \(\tau\) is scaling factor, \(\omega\) is uncertainty matrix and \({\pi}'\) is vector of view-adjusted equilibrium excess returns.
Please note that rather than specifying uncertainty matrix of views \(\mathbf{P}\) explicitly by the investor, we derive it from the formula above. It's because we believe that volatility of assets represented by covariance matrix \(\mathbf{C}\) is affecting certainty of both returns and views in a similar way.
Going back to our example, we have the vector of nine excess returns \(\pi\), to which we want to apply our views. Let's our views to be as below:
- Microsoft (MSFT) will outperform General Electric (GE) by 2%
- Apple (AAPL) will under-perform Johnson & Johnson (JNJ) by 2%
$$ \mathbf{Q}_{1 \times 2} = \begin{pmatrix}0.02 & 0.02\end{pmatrix} $$ $$ \mathbf{P}_{2 \times 9} = \begin{pmatrix}0 & 0 & 1 & 0 &-1 & 0 & 0 & 0 & 0\\ 0 &-1 & 0 & 1 & 0 & 0 & 0 & 0 & 0\end{pmatrix} $$
, where the first row in a link matrix \(\mathbf{P}\) has a positive sign +1 for MSFT and negative -1 for GE.
By applying these views to an original vector of excess equilibrium returns \(\pi\), we will get the view-adjusted vector \({\pi}'\), later used to forward-optimize optimal risky portfolio weights \(\mathbf{W}'\):
| Name | \(\mathbf{W}\) | \({\pi}\) | \({\pi}'\) | \(\mathbf{W}'\) |
| XOM | 16.0% | 15.6% | 14.9% | 15.3% |
| AAPL | 15.6% | 17.9% | 13.1% | 3.3% |
| MSFT | 11.2% | 15.2% | 15.7% | 22.6% |
| JNJ | 9.7% | 9.9% | 10.1% | 21.9% |
| GE | 9.4% | 17.9% | 16.1% | -0.0% |
| GOOG | 11.6% | 16.5% | 15.2% | 11.6% |
| CVX | 9.2% | 17.3% | 16.4% | 8.8% |
| PG | 8.5% | 9.4% | 9.2% | 8.4% |
| WFC | 8.7% | 21.8% | 20.5% | 8.2% |
These views are incorporated into equilibrium returns \(\pi\) also with respect to the uncertainty matrix \(\omega\). Therefore, for example, the stated view that MSFT will outperform GE by 2% is not fully incorporated into a difference in their implied returns, which is just 16.1% - 15.7% = 0.4%.
Forward-optimization of view-adjusted returns
Once the view-adjusted equilibrium returns are ready, the final step is forward mean-variance optimization of the market weights \(\mathbf{W}'\) with respect to these returns \({\pi}'\) and historical covariances \(\mathbf{C}\), pretty much as described in the previous post: Mean-Variance Portfolio Optimization
Graphically, the efficient frontier constructed using equilibrium returns before and after incorporating views looks as follows.
Black-Litterman model is based on an assumption that asset returns have greatest impact to portfolio weightings in mean-variance optimization. It is therefore attempting to reverse-engineer these returns from index constituents rather than relying on historical data. This results into several advantages including but not limited to:
- Intuitive portfolios with weightings not too much different from benchmark indices.
- Ability to incorporate custom views
- Lower tracking error and reliance on historical data
- Greater stability of efficient frontier and better diversification
