Criticisms of the Adcorp Employment Index

In November 2011 we wrote to Business Day expressing concerns about the claims made by Adcorp in the Adcorp Employment Index (AEI) and the lack of transparency of the methods used in the AEI.

Unfortunately the methods used in the AEI have still not been set out transparently on the AEI website. Instead our understanding of it comes from a series of emails sent by Loane Sharp of Adcorp to the authors setting out the methodology behind the Index. In this paper we set out several criticisms of the methodology used in the Adcorp Employment Index and respond to some of the criticisms Adcorp has made about the Statistics South Africa (Stats SA) employment estimates.

The employment levels estimated by Stats SA and the AEI differ mainly due to the way informal employment is estimated. To estimate informal employment the Adcorp Employment Index uses a crude version of the currency demand method to first estimate the level of unrecorded activity. This method has been roundly criticised by the academic community and international organisations. But in then attempting to estimate non-formal employment Adcorp must go even further by guessing at the level of labour intensity required to produce their already dubious estimate of unrecorded activity.

Adcorp also guesses at the number of "illegal migrants" in South Africa and adds these into the labour force. Since not all data is available every month to estimate this index, Adcorp must forecast some of this, turning the index into a forecast of employment at time, albeit one that is released after time . Our conclusion is that the Adcorp Employment Index does not reliably estimate employment in South Africa.

The Adcorp Employment Index Methodology

The Adcorp Employment Index (AEI) is released by Adcorp every month (see http://www.adcorp.co.za/Industry/Pages/Adcorp'sEmploymentIndex.aspx). It is an estimate of employment in South Africa. Adcorp introduced a new version of the AEI in 2011, which increased the number of people estimated by Adcorp to be employed in South Africa from 12.9 million in December 2010 to 19.1 million in January 2011. This massive increase was due to Adcorp including a new estimate of what we will call in this paper, for reasons outlined below, "non.formal" employment [1].

The February 2011 AEI divides employment into official and unofficial employment, mentioning that the unofficial sector "is not recognized by Statistics SA, [and] numbers 6.19 million people according to Adcorp's estimates. It includes unrecorded and, in some cases, illegal transactions such as employment of unregistered foreigners, evasion of income, payroll and other taxes, and other economic activity in the underground economy" (Adcorp, 2011b).

Almost all of the difference between the employment numbers given by Stats SA and by Adcorp in 2011 stems from Adcorp's new estimate of non-formal employment in the AEI of roughly 8.5 million people. Our main criticism of the 2011 Adcorp Employment Index is of this estimate of non-formal employment. This estimate requires: (i) a method that has been roundly condemned in the academic literature and (ii) an assumption that is almost certainly wrong.

Definitions Matter!

The Statistics South Africa labour force surveys are designed to obtain estimates of all employment undertaken by individuals in the households surveyed. In the reports produced from the results of the labour force surveys Stats SA breaks down employment into formal sector employment, informal sector employment, employment in agriculture, and employment in private households (domestic workers, gardeners etc.).

Informal sector employment is classified as such using the responses of individuals to the Stats SA surveys and is defined as employment which takes place in firms employing less than 5 people who do not pay income tax or employment of "Employers, own account workers and persons helping unpaid in their household business who are not registered for either income tax or value-added tax" (Statistics South Africa, 2010).

In the national accounts measurement and employment literatures the distinctions between notions of unrecorded activity, informal employment and the informal sector and hidden activities are crucial. As noted in an IMF working paper "Informal activities may well be recorded, activities hidden from tax authorities may well be reported in statistics, and illegal activities may well be included (directly or indirectly) in official statistics" (Bloem and Shrestha, 2000).

The AEI mixes up these definitions by attempting to estimate non-formal employment, since in doing this it uses a (bad) estimate of the size of the unrecorded economy, which does not necessarily overlap with the size of the income generated by non-formal employment. For example some of the employment in private households is likely to be part of the unrecorded economy, yet this is included in official employment in the AEI, in addition to the estimates of the non-formal sector based on the size of the unrecorded economy.

Estimating the level of Non-formal Employment

Adcorp uses a simple version of the currency demand method to estimate the size of expenditure in the unrecorded economy and then makes several assumptions that allow it to estimate the amount of non-formal employment that was needed to generate this level of expenditure. Both steps of the method require very dubious assumptions.

Currency demand methods are based on the observation that some economic activity takes place outside the formal sector. Some of this activity is unlikely to be recorded, since it is engaged in by firms that are not registered or paying tax and by individuals working without formal contracts and not paying taxes.

This activity may also include disposing of the proceeds of criminal activities. If this activity is mostly conducted in cash then the difference between gross domestic expenditure and some variant of the cash money supply is assumed to represent unrecorded economic activity. The difference between the amount of cash in circulation and the level of domestic expenditure is then assumed to be the level of unrecorded activity. This is a particularly crude variant of the currency demand methods that have been used since Feige (1979).

Currency demand methods have been criticised by economists at the IMF (Bloem and Shrestha, 2000), academic economists (cf Thomas, 1999) and the OECD (2002). One criticism is that it is not clear what the difference between cash supply and Gross Domestic Expenditure (GDE) actually represents: cash has uses other than in unrecorded activities.

Currency demand methods also require that the frequency with which money is spent per unit of time, what is called the velocity of money, is both constant over time and the same in the recorded and unrecorded economy (Bloem and Shrestha, 2000). The velocity of money actually varies and it is very unlikely that it is the same in both recorded and unrecorded transactions.

The method has its defenders (cf Bhattacharyya, 1999) but this defence is of a much more sophisticated method than Adcorp's. In addition, one of the method's defenders acknowledges that the lack of standard errors to assess confidence is a major weakness of most of the estimates of the size of the unrecorded economy (Bhattacharyya, 1999) - even if all other aspects of Adcorp's method were unproblematic, we would have no sense of the precision of their estimates.

Unfortunately, Adcorp's next step is even more troubling. Adcorp uses an estimate of the size of the unrecorded economy to generate an estimate of the size of non-formal employment, something we have not seen any other studies do. This requires assuming that the labour intensity of expenditure that is unrecorded is 20% more labour intensive than the formal sector. This number is not based on any data, it is simply a guess. It may well be that unrecorded activity is more labour intensive, but whether this is true and by how much is not discussed in the AEI. This second step has been called an "innovation" by Loane Sharp, but we prefer to label it an "imagination".

The final non-formal employment estimated from the currency demand method (around 8.5 million) is included in the AEI in a rather opaque way. In the 2011 AEI Adcorp distinguishes between official and unofficial employment, where official employment is basically a forecast of Stats SA's estimated total employment [2], including formal, informal and agricultural employment, as well as employment in private households. The total level of non-formal employment estimated from the currency demand method is then split in the AEI between official and unofficial employment, so that the "official" employment levels roughly coincide with Stats SA's total employment estimates, and the other 6.5 million goes into the "unofficial" category. This is not at all clear from the presentation of the AEI and contributes to a lack of transparency in the AEI that makes it difficult for journalists and others to easily evaluate Adcorp's claims.

Another problem with currency demand methods in general is that they produce only a total estimate for employment. This total provides no information on the age, gender, industry or occupation of those estimated to be employed using the currency demand method. Thus the 2011 AEI cannot tell us anything about the 8.5 million individuals estimated by the currency demand method to be employed in non-formal employment. This is a problem which Adcorp implicitly recognises in the AEI, since the AEI does include estimates of (what must be) "official" employment by occupation and industry [3], which includes Stats SA's estimate of informal employment, despite Adcorp supposedly believing that the Stats SA informal employment numbers cannot be trusted!

This is another important part of the AEI which is not documented and which is unclear from a first reading of the Index. Having outlined our major concerns with the AEI currency demand method for estimating non-formal employment we now explore several other concerns with the index.

The Use of Forecasting in the Employment Index

One of the supposed benefits of the Adcorp Employment Index is that it is available on a monthly basis, soon after the end of each month. But because the AEI requires forecasts of some of the variables used in the creation of the Index, including Gross Domestic Expenditure and Stats SA's formal sector employment, the index is not really an estimate of employment at time, but a forecast of employment at time, albeit one that is released only after time! Thus claims that "Employment grew at an annual rate of 6.5% in December" should actually be phrased as "employment is forecast to grow at a rate of 6.5% in December [and we are giving you this forecast after the fact, in January]".

Including "Illegal Migrants"

Adcorp's estimate of total employment actually exceeds Stats SA's estimate of the size of the labour force, the number of people working plus those who are willing and able to work, and looking for work. The only reason Adcorp does not then calculate a negative unemployment rate is that 2.5 million "illegal migrants" are added into the labour force estimates of Statistics South Africa by Adcorp. How this figure is obtained or whether it is adjusted over time is unclear. This is not noted anywhere in the AEI as far as we can tell, despite this being an important assumption and criterion for the public to judge the quality of the AEI.

An About Turn on Unemployment

An important result of the large estimates of employment in the 2011 AEI is that Adcorp has effectively done away with the South African unemployment problem. Adcorp believes there are around 19 million employed people in South Africa and, including the 2.5 million "illegal migrants", the total labour force is around 20 million, meaning that the unemployment rate is around 5%.

An irony in this dramatic change is that the Adcorp CEO Richard Pike and Loane Sharp have recently written a book, whose publication was sponsored by Adcorp, in which they argue that SA faces an "impending unemployment crisis" (Pike et al 2010) [4]. It seems that Adcorp's changes to its employment index in 2011 have effectively made this impending crisis vanish.

Responses to Adcorp's criticisms of Stats SA employment numbers

Adcorp's own rationale for its employment index is that Statistics South Africa is underestimating the level of employment and that unemployment simply cannot be as high as Stats SA estimates it is.

It is not clear exactly how Adcorp thinks Stats SA underestimates unofficial employment. One potential reason is that individuals may hide activities that have not been declared to the South African Revenue Services in the belief that the survey data would be passed on to SARS. This could well be true for formal employment activities estimated by Stats SA too, yet Adcorp has no problem using this employment data from Statistics South Africa in the calculation of the AEI.

Loane Sharp has pointed out to us that the Finscope Survey of Small Businesses (Finscope 2010), finds much higher levels of self-employment in small firms than Statistics South Africa. This is taken by Sharp as evidence that the Stats SA employment numbers must be incorrect.

It is puzzling that this survey finds higher levels of self-employment than Statistics South Africa's labour force surveys, but since the survey data and detailed methodology are not in the public domain it is not yet possible to explore how and why Finmark's numbers differ from those of Stats SA's.

In contrast several other surveys conducted by academics from UCT, UKZN and several American and British universities have found similarly high levels of unemployment to Statistics South Africa, including the Project for Statistics of Living Standards and Development in 1993, the KwaZulu-Natal Income Dynamics Study (1998 and 2004), the Cape Area Panel Study (2002.2006) and the National Income Dynamics Study (2008).

A further criticism of Statistics South Africa's informal employment numbers by Loane Sharp (2011) focuses on the high correlation between the level of employment from the Quarterly Employment Survey (which surveys only formal sector firms) and the Quarterly Labour Force Survey, which surveys households and the individuals who make up these households, and which thus should measure both formal and informal employment.

Sharp argues that the high correlation between the two employment estimates suggests the QLFS is only really measuring formal employment. This reveals a lack of understanding of basic economic statistics: two series can be highly correlated over time because of a common trend (for example, the state of the economy) rather than because they are measuring the same thing (Gujarati, 1995).

Questions about whether South Africa has high or low unemployment must be settled with high quality data, rather than on prior beliefs and guesses. We have shown that the Adcorp estimates of employment are most definitely not of high quality, and pointed to several other surveys that confirm the estimates of Statistics South Africa. The Finscope small Business Study obtains substantially higher estimates of self-employment than other surveys, but we are unable to interrogate why this is the case until these data are made publicly available.

Conclusion

Adcorp uses a crude currency demand method to estimate the size of the unrecorded economy, despite this method having been strongly condemned by academics and the international community. Adcorp goes even further by mixing up definitions of informal employment and the unrecorded economy to estimate informal employment, which requires a guess at the labour intensity of the unrecorded economy.

Adcorp also guesses at the number of illegal immigrants, and this helps Adcorp to avoid estimating a negative unemployment rate. Adcorp's estimated unemployment rate is very low, and does away with the impending unemployment crisis outlined in the recent book sponsored by Adcorp. None of Adcorp's estimates have any kind of statistical precision attached to them, and it is not hard to see why.

Unfortunately these issues are not easily identified by the public, journalists or even academic researchers, since Adcorp is very reticent to spell out its methods on its website or elsewhere. This makes it very difficult to evaluate the truth of Adcorp's claims without investing significant amounts of time, a commodity in short supply for the journalists who regularly discuss the Adcorp Employment Index and publicise Adcorp's "findings". A similar lack of transparency underlies Adcorp's calculation of labour productivity, which we will address in a future paper.

All of the survey data available in the post-Apartheid period, except Finmark's small business survey, suggest that employment is lower and unemployment higher than Adcorp claims. So does simply looking around. The Adcorp Employment Index seems to function more as a source of free advertising for Adcorp, rather than a reliable way of estimating employment in South Africa.

Footnotes

[1] Theoretically we define non-formal employment as all employment that is defined by Stats SA as not being either formal employment, agricultural employment or employment in private households, but we use the shorthand "non-formal" in this paper. Part of the problem, as we discuss below, is that the estimation method used in the AEI cannot credibly estimate non-formal employment.

[2] We discuss the use of forecasting in the following section.

[3] The occupational and industry breakdowns do not say which sector they are from, but a quick addition of the employment numbers suggests it must be official employment, ie Stats SA's estimate of formal, agricultural and private household employment.

[4] In addition to a number of dubious claims the book states that 3.5 million Zimbabweans have been absorbed into the South African labour market in the last ten years, whilst the AEI figure is 2.5 million "illegal migrants" (origins unspecified).

References

Adcorp, 2011a. Adcorp Employment Index, January 2011. Downloaded from

http://www.adcorp.co.za/Industry/Pages/AEIFebruary2011.aspx

Adcorp 2011b. Adcorp Employment Index, December 2010. Downloaded from

http://www.adcorp.co.za/Industry/Pages/AEIJanuary2010.aspx

Bhattacharyya, Dilip (1999), On the Economic Rationale of Estimating the Hidden Economy. The Economic Journal, 109, F348.F359.

Bloem, Andriaan and Shrestha, Manik (2000), Exhaustive Measures of GDP and the Unrecorded Economy, IMF Working Paper. Downloaded from

www.oecd.org/dataoecd/40/17/2069676.pdf

Feige, E (1979), How big is the irregular economy? Challenge, vol 12, 5.13.

Finscope (2010), Report on Small Business Survey 2010. Downloaded from

http://www.finscope.co.za/new/pages/Initiatives/Countries/South.

Africa.aspx?randomID=d29cd5ff.c761.4d12.ac33.99f522c0cc44&linkPath=3_1&lID=3_1_11

Gujarati, D, 1995, Basic Econometrics, 3rd edition, McGraw.Hill, New York.

Hussmans, Ralph, (2004) Measuring the informal economy: From employment in the informal sector to informal employment. ILO Working Paper 53, Technical Report, ILO, Washington, DC.

OECD (2002), Measuring the Non.Observed Economy: A Handbook. Downloaded from

www.oecd.org/dataoecd/9/20/1963116.pdf

Pike, Richard, Sharp, Loane and Black, Ted (2010),The New Divide, TerraNova, Johannesburg

Sharp, L (2011), Article from Moneyweb.co.za, February 14, downloaded from

http://www.moneyweb.co.za/mw/view/mw/en/page292681?oid=529778&sn=2009+Detail&pid=292678

Stats SA 2010, Statistical Release P0211, Quarterly Labour Force Survey, Quarter 3 2010.

Thomas, J (1999), Quantifying the Black Economy: "Measurement without theory" yet again? The Economic Journal, 109, F381-F389.

Source: DataFirst, University of Cape Town. Transcribed from the PDF. Please check against the original.

Click here to sign up to receive our free daily headline email newsletter