1. Statistical Reform Promotion Council and EBPM (Evidence Based Policy Making)

The phrase of statistical reform has received increasing attention in recent years. At the 16th Council on Economic and Fiscal Policy held on October 16, 2015, a question was raised regarding the accuracy of economic statistics used to estimate the gross domestic product (GDP). The Council deliberated on the necessity to further develop economic statistics that was aimed at improving the accuracy of GDP statistics. Following continued debate at the Council on Economic and Fiscal Policy on November 4, 2015 and March 24, 2016, the 22nd Council on Economic and Fiscal Policy (held on December 21, 2016) raised the possibility of using private big data or administrative data besides official statistics for EBPM (Evidence Based Policy Making) in Japan, as has already been adopted in western countries, in addition to for refining basic statistics to improve the accuracy of GDP statistics. Based on these developments, the Statistical Reform Promotion Council (Chairman: Yoshihide Suga, Chief Cabinet Secretary) was established in 2017.

The objective of the Statistical Reform Promotion Council is stated as follows: Promoting the establishment of EBPM (Evidence Based Policy Making) throughout the Japanese government and the response to the needs of citizens, etc. from the perspective which exceeds statistical administration departments. The council has convened twice thus far. At its second meeting (held on April 14, 2017), the council released the Midterm Report by the Statistical Reform Promotion Council (hereinafter referred to as the “Midterm Report”). The Midterm Report clearly states that development and reform of various data including statistics are required to construct a system for promoting EBPM. These various data including statistics consist of statistics, statistical microdata and administrative data used for statistical purposes. When considering these actions by the Statistical Reform Promotion Council, greater attention is clearly being placed on statistical reform, particularly the way in which official statistics is created and offered, as well as the way in which government-recorded data is used and applied.

2. Current status for creation/provision of official statistics in Japan

Official statistics in Japan is provided in a variety of forms including both statistical tables (aggregate result tables) which were aggregated from questionnaire information (individual data) obtained from statistical surveys and are disclosed to the general public as open-data, and microdata created based on individual data from questionnaires for which information has been entered. The following schematic drawing shows the way in which official statistics is provided in Japan.^[1]

In addition to being disclosed in printed reports, statistical tables of official statistics in Japan are disclosed through e-Stat, the portal site for official statistics of the Japanese government. This makes it easy to access statistical tables on the internet. Recently, in order to further improve the efficacy of use for statistical data, improvements are being made to the functionality of open-data in official statistics. Specifically, efforts are being made to further improve the potential for use as open-data by 1) developing a usage environment via an API (Application Programming Interface) function and 2) enhancing statistical GIS functions.

In Japan, provision of official microdata is performed based on the Statistics Act (Act No. 53 of 2007). The provision of individual data and the creation/provision of Anonymized microdata (microdata created by applying anonymization techniques to questionnaire information) in Japan has been performed pursuant to the provisions of the articles of the current Statistics Act (Article 33: Provision of Questionnaire Information; Article 35: Production of Anonymized Microdata; Article 36: Provision of Anonymized Microdata). Furthermore, in Japan, it is possible to pay a fee for tailored aggregation of statistical surveys (Article 34: Production of Statistics, etc. by Entrustment)

Here are the legal procedures related to the creation of Anonymized microdata in fundamental statistical surveys. First, in regards to the creation of Anonymized microdata for a fundamental statistical survey, the director of the administrative agency which is the authority concerned issues an inquiry to the Statistics Commission. Next, after the Anonymized Data Sub-Committee of the Statistics Commission reviews the anonymity of Anonymized microdata for the fundamental statistical survey, the details of deliberation at the Anonymized Data Sub-Committee are reported to the Statistics Commission. The Statistics Commission then reports on the validity of methods for creating Anonymized microdata. After that, Anonymized microdata is created and provided at the responsibility of the Statistics Creation Bureau. Currently, Anonymized microdata is provided from seven surveys in Japan, including the Population Census. However, for almost all statistical surveys from which Anonymized microdata is created, only one type of Anonymized microdata is created. Therefore, discussions are currently being held on methods for creating multiple types of Anonymized microdata to meet the needs of users.^[2]

In order to use Anonymized microdata, it is necessary to apply for use of anonymized data based on Guidelines for Creation and Provision of Anonymized Data (created by the Director-General for Policy Planning of the Ministry of Internal Affairs and Communications (in charge of statistical standards)). In addition to mainly restricting the use of Anonymized microdata to academic research and education, the Guidelines for Production and Provision of Anonymized Data also require clarification for the range of users, usage period, usage location, and storage location of anonymized data when applying for use. In this respect, Anonymized microdata in Japan differs from public use files which can be accessed by anyone without restriction on purpose of use.

Application for the use of individual data intended for use in academic research is filed based on “Guidelines on Application of Article 33 of the Statistics Act” (created by the Director-General for Policy Planning of the Ministry of Internal Affairs and Communications (in charge of statistical standards)), which list detailed rules on procedures for the creation and provision of questionnaire information. Under the current Guidelines, when applying to use individual data of government statistics, it is assumed that only the minimum required variables (survey items) for conducting empirical analysis are used. On the other hand, there is a high level of need for exploratory use of groups of variables contained in individual data. In many western countries, researchers with access authorization for individual data are able to use individual data at secure environments such as on-site facilities and through remote access from university laboratories, etc. In these countries, instead of performing advance screenings of aggregate tables created for analysis and of analysis models, examiners perform a confidentiality check for analysis results after the individual data has been analyzed at on-site facilities or through remote access. Based on these conditions, Japan is also currently discussing the possibility of using individual data at on-site facilities in our country.

3. Future use and application of data in Japan

In the future of official statistics, it is most likely that a variety of forms for data provision will be reviewed based on user needs, while paying attention to the level of data anonymity. In foreign countries, a clear distinction is made between the provision (for example, use at onsite facilities and via remote access) of data which is restricted to certain users and intended for academic research, and disclosure of data which can be used in an open format (open-data, public use files, etc.). Forms of provision for official statistics in Japan should be explored while referring to conditions in other countries.

Both in Japan and western countries, there is a high level of social interest in the use and application of administrative data. The European countries of Denmark, Finland, Sweden, and the Netherlands have established statistic creation systems for register bases using administrative data. Other western countries are advancing the provision of administrative data for academic research. For example, England is currently implementing the Administrative Data Research Network project for the provision of administrative data. This has made it possible to access administrative data, as well as administrative data which has been linked with official microdata. ^[3] Japan should refer to such conditions in foreign countries when reviewing the possibilities of using and application of administrative data in our country.