The following are the outputs of the real-time captioning taken during the virtual Fifteenth Annual Meeting of the Internet Governance Forum (IGF), from 2 to 17 November 2020. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid to understanding the proceedings at the event, but should not be treated as an authoritative record.
***
>> Hello, good morning.
>> Good morning, Alexandre.
>> We can start now. You can start, please. I hope you don't have, do you need a interpreter?
>> ALEXANDRE BARBOSA: No.
>> You can start.
>> ALEXANDRE BARBOSA: Good morning. Good afternoon. Good evening, everyone. And thank you for attending this session, the IGF 2020, entitled Ensuring Trusted Data Sharing for Monitoring the SDGs. I'm Alexandre Barbosa from the regional center for studies on the development of the Information Society at the Brazilian network center. It is a great pleasure for me to moderate this panel. I welcome and thank this group of distinguished speakers for having accepted our invitation to speak to you today. It is important to mention that we have in this panel experts renowned for their knowledge and large experience in the statistical data production, and this is a group very much involved in data advocacy both by promoting the use of data for evidence based policymaking, and encouraging the production of reliable and internationally comparable data. So I'm really very happy to be part of this debate, because measurement and data production are very important to CETIC.br as related statistics production for both I would say policymaking and also for monitoring ICT policies and international agenda such as the 2030 Sustainable Development Agenda, SDGs. And before we start our session, let me very quickly announce the rules for our online interaction for this virtual session.
Each speaker will be given 8 minutes, around 8 minutes for their initial remarks. Then I will be receiving questions from participants to the panelists in the chat box, and I will address them at the end of each round of interventions, if time allows. This meeting is being recorded for report writing purpose.
To start our session today, I'd like to share my thoughts with you and also offer some insight from the Brazil's experience in the production of ICT related statistics, and not only Brazil, but I would say that a national statistical system which is not only the national statistical office but the whole systems are under increasing pressure to produce high quality data in a timely manner on a wide range of areas, and I would say often with a link to the resources. Besides policymakers need more and more disaggregated data on multiple social demographic variables, such as gender, age, race, disabilities. We can list many variables, but also disaggregated by variables that are relevant, that make sense to national and local context. This is really important. And I also say I would like to say that this shows how innovation and modernization of national statistical systems are of utmost importance. We need to go through innovation process and also modernization of the systems. I'm quite sure that data sharing is really becoming a important debate among data producers. More and more alternative data sources, and data collection methods, also we can name multistakeholder partnerships and data sharing are part of the innovation process in my opinion. Traditional official data producers may not be able to provide timely reliable disaggregated data that is needed for policy design and SDG monitoring needs. We need to search for these partnerships, and therefore, a new data production ecosystems that bring new data providers and data sharing opportunities must be part of this discussion, and must be really discussed and put into practice by the key stakeholders.
In this session today, we will address the urgently need actions to build a comprehensive data production model that incorporates different sources, data sources, and also data sharing mechanisms to meet policy data needs. We will also discuss here today key opportunities, challenges and alternative data for governance of such new models of public statistics production.
That is to ensure that trusted data sharing and data use is possible. Needless to say to this group, if you are here participants listening to the discussion, it's because you have interest in data production and data use. Needless to say to this group that we are living in a veritable data revolution. More than that data is currently produced faster than it can be used and transformed into information that can really effectively drive this automation that can first digital, the digital economy and of course Sustainable Development. So in my opinion, we data producers will have to move beyond our own boundaries, if we are to establish partnerships that will allow us to shift from using traditional data sources to using alternative data sources and share data.
This is, I think that you all know as well that this is largely due to the fact that the ICT industry and mostly the private sector own very large amount of relevant data that would likely be unavailable to data producers such as national statistical office if it weren't for agreements and partnerships. This new environment of course poses numerous challenges for us data producers, requiring governance models that enable a trusted cross organization data sharing for instance, and in particular when we go to private data sources. In fact, I truly believe that the ticket for the future in terms of official data and statistics production as it was once mentioned by Robert Growth, it is a combination of traditional design data from surveys, data or whatsoever with alternative sources, such as big data.
This is the combination that Robert Growth say that it's the ticket for the future, we have to combine data sources. Assuming that many countries, especially in the global south is still facing major data gaps in the production of official and timely data for SDG monitoring, and also considering that big data has the potential to complement traditional data sources, we could conclude that so called data revolution offers many opportunities, but also imposes the need to include statistical system capacity so as to better respond to new requirements related to SDG measurement and alternative data sources including data from the Telecom and Internet providers, data from social media companies, data from the Web, data from mobile and tracking devices, or private sector transaction data, such as credit card and banking transactions, are really very important and can help us a lot in this process of data productions.
This is just my initial comments, to share with you what I really think and what I really believe that is the future of data production. Without further ado, I would like to introduce our presenters for this session. We have Daniel Ker from the OECD measurement analysis digital economy from the directorate for science technology and innovation, Jaco Toit, chief of universal access to information session, at the communication and information sector at UNESCO in Paris, OECD also our colleague Daniel from Paris in France, we have Mark Uhrbach, chief of digital economy metrics at the Statistics Canada, Dominik Rozkrut, President of the Statistics Poland, Professor Alison Gillwald, Executive Director of Research ICT Africa and Professor is Professor at University of Cape Town in Nelson Mandela department of governance, and Helani Galpaya, Chief Executive Officer at LIRNEasia. Daniel is trying to connect. So maybe I will leave Daniel to be the last to speak, but just to say that we have a very good mix of international organizations, national statistical offices, and also academia and Civil Society organizations. So I will go directly to our colleagues from UNESCO, Jaco, and here we have a representative from international organization that will share with us his view on all these issues that I have mentioned. And I would like to invite Jaco Toit from UNESCO to share his views on the challenge and benefits of data sharing especially in the context of the SDGs, and how we could move into this new and comprehensive data production ecosystems. Jaco, it is my pleasure to give you the floor and thank you again for joining us today.
>> JACO TOIT: Thank you very much, Alexandre, for this opportunity, also to talk about how does a international organization like UNESCO actually produce and share and monitor information on the SDGs.
As you might know that UNESCO's mandate is quite wide and within this mandate, we collect a lot of data on the SDGs, where UNESCO is the custodian of several indicators, within my section on universal access to information, we look specifically at collecting data on access to information, so the number of countries that adopt and implement constitutional and statutory policy guarantees for public access to information, and for that, we have used very traditional ways of collecting data, where we have a national questionnaires that are destined to oversight bodies for access to information. We have institutional questionnaires for public authorities like ministries of environment, of finance and others.
Once we receive this data, we also verified it then with UNESCO databases on countries with access to information and laws. So it is a very traditional way of collecting data, as an institution that is intergovernmental. But when we collect this data, I think we also realize that we need additional sources of data and have to look at other ways of collecting data, specifically within the realm of access to information, we send originally our questionnaires to SDG focal points, but they were not familiar with the national or institutional implementation of access to information legislation, and then we had to look at additional recipients of our questionnaires.
Also when these bodies actually receive our questionnaires, of course we had very grand plans to have full data sets, but people are not familiar with our terminologies or the specialized terminologies that we are using, as it relates to SDGs, and that ended up that people did not provide all the data that we needed, and some cases, incomplete responses because relevant bodies do not necessarily see the importance of collecting this data. And then finally a point that I wanted to mention is data ownership. We collect data from public institutions, but we also encourage Civil Society groups to collect data on access to information, because they can for example produce shadow reports on the question of access to information.
But how do we put this data together, so that we could make sure that the Government institutional information is a source and Civil Society organization is put within the right context. These are open questions that at least we still struggle with.
In order to address these elements, I think we try to look specifically at the guiding principles which is vulnerability, accessibility, interoperability and reusability, as a prism in order to say how do we manage this data so that we benefit from data sharing and minimize some of the risks from the origin of this data.
In terms of fund ability, UNESCO works in cooperation with the inter‑agency and expert groups on SDGs to validate the metadata to ensure that UN statistical standards are followed. I think that we are quite fine with. We also based our questionnaire specifically on implementation for some of the principles related to access to information that we had to interpret with group of stakeholders. We publish our data on dedicated platforms where people can download, visualize and analyze the data, in terms of accessibility, but how do we bring the different data sources together. That is still a open question.
We also, while we collect data on a specific indicator, we want also to look at the interoperability of this data. For example, how does the data that we collect on access to information can be seen through the prism of gender, for example, or through the prism of SDG 9, which is on industrial cooperation. This is something that we haven't succeeded yet in my view that we collect SDG information in our box, but how do we put these different boxes together. I know UNDP through the maps initiative tried to do that, but I think we still are far away of putting this together.
In terms of reusability, we make sure that the data producers, Government, Civil Society or others are very well attributed in terms of the data, so that when we mix it, we can actually see that where it comes from to put the context, as we look at official data.
When we look towards a new comprehensive data production system, I think it's important that when we look at the SDGs that we realize that SDGs are not freestanding, but subject to each other. I think this is a challenge that we have to try to solve and to work more on. We have to realize that different organizations are involved, so while governments or statistical services are our primary try in terms of producing this information, we also need to look out of the box, as you said, to private sector, towards data scraping and other elements and how do we do that.
Lastly I would like to mention also the importance of looking at sharing, to pay attention to open access policy, UNESCO has a open access policy but how do we apply it to open data. How do we move forward in terms of open data, when we look at official data, and I think these are all open questions that we might want to look at. I'll stop there and thank you so much for this opportunity.
>> ALEXANDRE BARBOSA: Thank you very much, Jaco. Maybe we will get back to you in the next round. But I do believe that UNESCO is doing a very important, playing a important role in this new data governance ecosystem production, I would say, because of course access to information is very critical to UNESCO and open data, so and I have been following what UNESCO is doing. I guess that not only UNESCO but also other UN agencies are really very, will play a increasingly important role in terms of providing a debate around these issues that you have mentioned, data vulnerability, accessibility, open data, and also all the things that comes with that, like privacy, which is a very important topic that I hope we can discuss in the next, with the next speakers. So thank you very much, Jaco, for this very important contribution to this debate.
Now I would like to welcome Daniel Ker from OECD that I have already introduced him. He was with difficulties to connect. Welcome, Daniel. I would like to invite you to share and to provide your view on the issue of alternative data sources versus traditionally designed survey data, and the challenge associated with each of this type of data. I know that I have been following also very closely all the work that OECD is doing in terms of this topic of data sharing of having access to alternative data sources.
I really appreciate, Daniel, your willingness to share with us what OECD is doing in this regard. You have the floor for about 8 minutes.
>> DANIEL KER: Thank you, Alexandre. Firstly I'm sorry I was late. I was trying desperately to get in, but not working hard enough it seems. But I'm here now. I'm not sure I'll be able to cover everything OECD is doing in this area, but I'll talk about things from my area and do my best to touch on some of the wider work.
The first thing we need to say is that it's not really about surveys or alternative sources that you mentioned. It is definitely about surveys and complementary sources. They each have their own strengths and limitations that can be complementary to each other. Well designed surveys do a great job of measuring a huge range of economic and social phenomena across different sectors, households, businesses, Government, nonprofits, all the way down to individuals as well. They are well‑established approach and there is a wide literature on how we can ensure the results that we get from surveys are appropriately representative and robust.
Nevertheless, surveys do have shortcomings, and these include cost and the burden of respondents and a trade off between timeliness and representativeness but surveys are not great at measuring some things. That is one area that we can draw on other data sources as a complement.
Data from other sources might be useful to help us address shortcomings and help us provide insights in those areas where surveys don't work so well. We need to design this complementary system between the two. I want to quickly say that my area of the OECD we are concerned with measuring and analyzing the digital economy. There are some areas that could be worked on with information held by key businesses in particular.
E‑commerce for example, surveys do a good job of telling us how many firms engage in e‑commerce, but other sources could help us better understand the volume of e‑commerce revenues or the share that e‑commerce is in monetary terms. There is another area that we are very interested in, international flows of cloud computing services, which are quite abstract and confusing in terms of how they are structured. Online platforms which I'm sure will come up repeatedly, especially platforms allowing people to sell their labor or time or skills, issues related to well‑being, things like exposure to misinformation on line, it would be interesting to get information from social networks on that, and thinking about the SDGs. One example that came to me was there is a SDG indicator, the broadband subscriptions per 100 inhabitants down by speed. It would be great if we can get information from, there will be no businesses must withhold on whether these are business or consumer subscriptions, that is currently not something widely shared and known but should be something the businesses selling the services know. Also real world networks speeds versus what the network says the speed should be, these networks must be monitoring the speeds of traffic through their networks, so should be able to share information on that if they were willing to.
There is many more areas where we could think of synergies like this. But there is certainly a number of key challenges. The first one is understanding what we as statisticians we want and articulating that in a way that makes sense to businesses. When we talk about data sharing, it is really easy to get confused between sharing data sets and making it sound like we want Facebook to hand over their entire data set that underpins their entire business, for example, or sharing statistics derived from those data sets.
In general, our aim as statisticians is to produce summary statistics, but does not need to require the data sets themselves to be shared, although it would be nice if they could be. This to an extent is a question of division of labor, but could be a way of managing concerns around commercial sensitivity, and but at the same time we need to ensure that the processes that we have to create the statistics are robust and they deliver comparability, aggregate ability in the data, etcetera.
But to do that, we can agree the way that the summary statistics are to be calculated, and I will touch upon some examples from the OECD distributed microdata projects that offer a potential model for this in a moment. I think we need to be clear to ourselves and to businesses, are we asking for information about the business itself or are we asking for information about their customers, so in the case of a platform, is that the people or businesses supplying services and using services through the platform rather than the platform itself. There is a blurring of lines there, but we need to be clear that we understand what we are asking for first.
The second key challenge I'd like to highlight is that businesses and statisticians tend to look at the world in different ways. What sounds wise to statisticians like basic statistics can sound like a scary commercially sensitive map of their entire business and how they make their money and where from, or even worse, concepts like GDP have been described to me in workshops that relevant by some businesses that came to us, so we need to find a common language, way of articulating what we are looking for. Some established statistical norms also, such as the idea that you can identify the production of the good or service to a geographic place and its consumption to a geographic place, those don't necessarily apply well to firms, that operate multilateral, multinationally without borders essentially.
But nevertheless, what we can do is try to find ways to encourage firms to share data if we can find a common way of understanding and talking to them. One thing that firms and statisticians increasingly agree on is that data are an asset, but for a business an asset is not something they would normally share for free. We need to find a value proposition that works for the firms, that where they can say this isn't just about me giving something that is great for the statisticians but I don't get anything in return, we need to see a value in that sharing. This could be linked to efficiency, linked to reducing burdens on them, linked to corporate social responsibilities, so the firm feel like they are doing something good, and linked to giving them a real picture of the market that they operate in. There are various things there. Or of course, one could talk about taking legislative action to secure data sharing, but really if I go back to the multinational firms that operate across countries without borders, that is going to require a huge amount of coordination across countries, and furthermore at the moment already the powers, the different statistical agencies have in different countries to request data, varies a lot. You would be starting from a heterogeneous base.
To encourage firms to share data, we need to encourage them to have confidence in data sharing. One way to ensure this or to facilitate this is through legal frameworks. We need a common legal understanding of what is expected by the parties and how that can be forced legally if necessary. One thing that we found at the OECD may help is provision of information through trusted third parties, so businesses are used to disclosing information to their accountants. There may be a conduit through them that can help as well. Those accountants can provide assurance on the legal frameworks.
There is technical aspects, things like how is the data going to be stored and processed. Is it going to be secure, is it going to be up to the right standards. There is a need to bring in the right people on both sides to have those technical conversations and to help businesses understand what can be shared while remaining within the rules on privacy such as GDPR.
It is worth highlighting that the independence of statistical agencies from Government can be a important string to our bow, in many OECD countries national statistic offices are set up at arm's‑length from Government to safeguard the statistics they produce from interference. We can leverage that to say, look, we are not going to have the tax agency coming in the back door and double checking what you said on your tax filing last year. There is a degree of separation between these arms of Government.
A couple other things to highlight, data anonymization that we can use, this can include not only anonymizing subject of the data but also its source. That can be important in markets where, if we are able to get enough firms around the table together to manage disclosiveness in a concentrated market like infrastructure as a service, cloud services, there is only three big players in infrastructure as a service, if you go to one of them and ask where they make their money, they say that is a map of my business. If you go to three of them, you can convince them to talk to each other and provide the same data to a trusted third party, for aggregation. Once they aggregate the data, you have some degree of disclosure control that you can hopefully win them on board with.
Another issue to highlight is statistical limitations, so we need to try and understand what the limitations of the sources could be, other ways to communicate them. Importantly and I'll come on to this briefly, we want to be able to continue production of statistics over time. It is well and good doing it once, but statisticians want to do things over and over. This isn't just analysis.
Finally, one thing for us as statisticians is willingness to take risks. We need to be willing to put in the resources to try to pursue data sharing, despite the uncertainty of whether you will have success. I've seen several times in several organizations the ideas are there, but we stop short at the line of putting resources there, because you don't want to have to go back at the end and report that you didn't get where you hoped. But you need to try and take these risks.
I'll quickly highlight a couple models that have worked for us at the OECD and elsewhere. We had success with several one‑off models. This is great for analysis but not a basis for regular statistics I was talking about. We have used contract instrument such as loan disclosure agreements to get detailed transaction data from Spanish banks, and that allows interesting analysis. We have been approached by businesses we are interested in working with about implanting our analyst into those businesses for a period of time to do analysis and produce statistics that we can take out. But this is one of those examples where there wasn't the resource or the appetite for risk to really embrace that opportunity.
I've mentioned the broadband statistics earlier. We have some OECD statistics gathered through regulators. Regulators have regulatory power, but they tend to operate in markets that have a relatively small number of players. This also shows the benefits of having personal relationships with those players, and some of the other markets that were interested in may not have regulators in the same way, but they have small numbers of players we can try to build relationships with.
I'd like to highlight a success that Eurostat had on this. They had detailed negotiations, managed to secure agreement with four short stay accommodation platforms to regularly share data on books and guest that is could be aggregated into statistics. While we are waiting to see the tangible outcomes of that because these things take time, the agreement is there. It shows the breakthroughs are possible if you put the effort in, and that international organizations can play a important coordinating role and offer potentially a value proposition to some of the businesses as well as on behalf of our member countries.
Basically, by saying, if you have the choice as a business of completing one standardized submission agreed across countries, rather than many heterogeneous submissions done with each individual country, that could offer them some value in terms of time saved efficiency. That is an approach we are trying to make work with reporting on terrorist violence and extremist content at the OECD.
We have had success in bringing together groups of stakeholders from business and the statistics domain around the table, around the common measurement purpose. You can see some of the outcomes of this in the OECD.AI portal observatory portal that have several indicators on it.
Of course, none of this, we shouldn't ignore the fact that there are markets that can access the data, it can be easy to overlook, but there are examples where private enterprises are selling access to data to those producing statistics, including the OECD. But of course, one of the big problems is supply and demand don't necessarily reveal themselves organically in a way that gives a market solution. Related to that, I'd like to highlight two pieces of OECD work related to data sharing.
First, recommendation of the council concerning access to research data from public funding. That says if you are doing, if you are a private company or anyone doing research that was funded by the public the data should be shared as widely as they reasonably can be. The second is a 2019 report called enhancing access to and sharing of data, reconciling risks and benefits for data reuse across societies. This takes a much wider perspective than just the perspective of producing statistics, but many of the points in there are relevant to sharing for statistical purposes including the need to balance the benefits of enhancing data openness with the risks and the need to recognize legitimate private and national and social public interests, when deciding where the balance lies, the need to reinforce trust and empower users to proactive stakeholder engagement and community building, and the examples I touched upon are examples of that. It is putting in the time and the leg work to talk to companies and try and build the community around the shared purpose. Finally, encouraging the precision, provision of data through incentive mechanisms that make sense and hang together.
To draw it all together, it is important to understand that the process is going to be resource intensive, and it requires a lot of dialogue to bear fruit. This requires willingness on both sides, including the willingness to take and manage risks both for the business and for the statistical agencies. We need to build on the successes that there have been to illustrate the value propositions that there can be for businesses and to strengthen those value propositions to make it easy to get groups of compatible companies in a room to talk to each other to talk about the practicalities of data sharing, for statistical purposes, and overall to try and normalize, targeted controlled and appropriate data sharing for statistical purposes.
Hopefully that was useful.
>> ALEXANDRE BARBOSA: Thank you very much, Daniel. It was very interesting. I was saying in my initial remarks that the ticket for the future as said by Robert Gross is this combination of data sources like design data, survey data and organic data sources such as big data. Of course, you brought one very important topic which is the distinction between, what is really data sharing. Does it mean providing access to data sets, or just sharing the statistical results.
Right now, I would like to mention that in Brazil we are working with IDG which is our national statistical office, they have access, real access to the data set from our Mumbai operator in metropolitan area of Rio de Janeiro, and they are doing a model to try to estimate the proportion of the population using the Internet and also the proportion of population covered by 3G and 4G networks.
Those are statistics very easily produced by this type of CDRs from mobile operators. But of course as you said, data set became an asset. So develop proposition and how we are going to have access to this data, all the legal agreements and commercial agreements, it is a very important issue to be discussed.
But now, since we have now heard from UNESCO and OECD, two international organizations, I would like to listen what the national statistical offices are doing and what are their thoughts in terms of this new data production ecosystems, and data sharing. I would like to give the floor to Mark Uhrbach from the Statistics Canada, and maybe Mark, thank you for being with us today. As we all know, Statistics Canada is a best practice reference in terms of data production, statistics production. I would like to hear from you what do you believe that data users such as policymakers maybe are very hard user of data and statistics, what are they requiring in terms of data disaggregation, more complex data set, what are the views of data users when Statistics Canada produce statistics, what are the users' requirements, and what do you think that the COVID‑19 pandemic has forced national statistical offices to look at alternative data sources and also alternative data collection methods, since for many surveys we cannot go into the household or the companies or other establishments to collect data. What are the rules of this new data sources. Mark, you have the floor for about 8 minutes. Thank you.
>> MARK UHRBACH: Great, thank you very much, Alexandre, and thank you very much for the opportunity to speak here today with such a distinguished group of guests many of which I've had the fortune to work with over the past many years, and it's always a challenge to follow Dan. But I should be used to it by now. (chuckles).
A couple points I'd like to make today. The first one will build on what Dan was talking about earlier in terms of, I think we agree that really survey tools that we have used in the past alone aren't going to get us is there in terms of measuring what we want to in the future. As we have talked about, there is lots of advantages to some of the traditional methods that we have used, but we can certainly supplement and improve on those as well.
Going back to your question, Alexandre, although the focus of this session is on the development of data to meet policy design and SDG monitoring needs, the topic is timely within the context of the COVID‑19 pandemic. Throughout the world, here in Canada as well, we have seen the need from policymakers for realtime or near realtime high quality information on a ongoing basis, in order to attempt to try and make the best decisions possible during a dynamic situation.
In many countries, this data has come from the national statistical organization, here in Canada that is Statistics Canada, we have been responsible for providing a lot of that, although we have provided that information in a different fashion than we have done previously. At the beginning of the pandemic, Statistics Canada was able to use some of our traditional tools as statistician using surveys and administrative data that already existed, to show the effects and the potential effects of COVID on segments of the population, and on certain sectors, so looking at the potential of people to telework from home, whether we could identify potential, in sectors of the economy that we expected to be hit relatively hard by this, areas of the service sector of course, restaurants, tourism, things like that.
We were able to produce some of those statistics very quickly. But those tools alone were not sufficient to meet the needs of this unprecedented situation that we found ourselves in. Within a matter of weeks, the agency reacted by supplementing some of the traditional sources with a series of crowdsourcing exercises, Web panels, Web scraping activities, and also mobility and satellite data that were obtained both through public sources and private sources. The combination of the use of these methods allowed the agency to provide policymakers with much better perspective on the effects of the pandemic, on widespread and national level, but then also on a much more granular level as we were able to get into some of the details.
For the first, Statistics Canada, this was a area that we have been working on very much in the last few years, but I think what happened is as a result of the pandemic, it accelerated many of these initiatives, and it provided us with a bit more free license to explore some of these alternative data sources, especially as you say when traditional survey data could not be collected due to the lockdowns that were experienced here in Canada, and allowed us to use some of these sources to measure impacts of COVID, due to some of the shifts in consumer spending and some of the protective measures that were in place that restricted our movement.
Though some of the metrics that we put out, I think kind of in hindsight perhaps were not perfect, the response allowed us as statisticians to learn quickly the potential of some of the other data sources, and provided policymakers with up to date information that they needed to make better social, economic and health decisions, and try and chart a course towards number one, a safe recovery in terms of health, but also in terms of financial recovery for the country.
I think it's clear that we couldn't have done this just by using traditional methods alone, they would have locked some of the detail and timeliness required for the circumstances. So that is not to discount any of our traditional methods that we already have in place. In some cases traditional survey tools will still provide some of the data that we need and the indicators that are necessary, both to monitor the SDGs in this example, but also meet the needs of policymakers. However, for new and somewhat emerging issues related to the digital transformation which is my team's area of focus within the agency, I think we recognize now that this will not be enough. When we look at the e‑commerce transactions, the use of online platforms that Dan referenced earlier, there is a opportunity to use big data such as transaction level data or activity logs, to try and better measure this and these nontraditional privately held potentially big data sources will provide some of the inputs to build the next generation of indicators, as the digital transformation continues to evolve.
In many cases the lessons learned by NSOs through the world since the onset of the pandemic can continue to be built upon as we go forward to further develop comprehensive model that incorporates different data sources and data sharing mechanisms while still maintaining the integrity of the statistical system.
Second, I'd like to talk a little bit about that value proposition for data providers. Dan touched on the OECD's attempts to do this as well. This is something that we have been faced with at Statistics Canada too, while there is that opportunity to partner with businesses who possess valuable data to help measure emerging data challenges, I think we need to demonstrate to them why they should do this, and I think this is particularly true in the case of the digital economy. There is a lot of opportunity there, where digital trails exist throughout, and in many cases the best source of data for measurement are the providers of the services themselves who have built the firms on the information that they gleaned from these big data sources. For us at Statistics Canada and for many national statistical offices, there are two methods to attempt to obtain this data from private firms who often hold what they see as proprietary information quite closely. The first is trying to compelling the firms to provide it under legislation. However, for many firms that hold the data that we are trying to get, related to digitalization, these are often multinational firms that have headquarters outside of the country, and our legislative powers there are limited in terms of providing that information. The second option is to demonstrate this value proposition to firms. If they are not compelled to provide the information, or even for Canadian firms that are compelled to provide some information, what incentive do they have to provide better data to us and to do it on a more timely basis. If we all view data as an assets, firms need to be able to trade on that data that they are giving up, and see the value in doing so. In this case, the value that resides there is helping to answer some of the country's policy questions and issues.
A couple of examples within Canada, we have made progress in terms of partnering with some private firms to provide key information that is used to produce indicators. We do have a partnership with many large retailers to provide scanner data, statistics Canada uses that to build a more representative consumer price index. This data source became key during the pandemic, as a risk mitigation strategy, when prices that are typically collected from the field were unable, we were unable to take, to use that method due to the health restrictions that were in place.
As well, ongoing work that we have now with some of the major Telecom providers could help to unlock some of the new metrics related to broadband and mobile services provision there. Those conversations I think are necessary, as data now is big business, official statistic producers are no longer the only game in town. However, we have a role to play there still, and NSOs can demonstrate the value added through data analysis linkage and analytics, as well as the standards of quality that might be applied, which we apply, higher degree of quality in many cases than some other data providers out there.
The point for us to make in obtaining some of the alternative data sources that we can, as a NSO, we can do more with more information, we can paint the most accurate picture possible for the country, and the alternative to that is really that we can only provide the information, the information that we provide is limited by the information that is provided to us, and that could lead to potentially incorrect decision‑making and policies.
Finally, I'd like to make the point in order to facilitate data sharing, I think as a data steward, we must establish trust with those whose data is provided to us, so to that end, Statistics Canada has created a trust center which attempts to explain to firms and to individuals why their data is collected and how it is used, and put that out there in the transparent fashion. As well, very importantly, it outlines how their data is protected once it comes into the agency, and statistics Canada's accountability framework, and that is all done with the idea of building that level of trust between those that provide information to us and ourselves who put it back out into the world.
I will leave it at that for this discussion. But thank you very much again.
>> ALEXANDRE BARBOSA: Thank you very much, Mark, very interesting. Thank you for sharing all the difficulties and challenge that Statistics Canada is facing, like us or other data producers willing to use alternative data collection methods and data source. It is indeed a big challenge. You gave very good examples. National statistical offices such as yourselves plays a important role in this debate, mainly within the UN statistical system and debate, based on the fundamental principle of official statistics how we are going to discuss this data governance for the production of official statistics. Thank you very much for your contribution. Maybe we get back to you if we have time.
Now, I would like to give the floor to another very relevant national statistical office, Statistics Poland. I would like to invite Dominik Rozkrut. I'm happy to see you here with us. Thank you for your kindness to bring your expertise and experience. Very recently I joined a session with Dominik Rozkrut as a participant in the UN global data, big data for official statistics conference. I could witness all the innovation projects that Statistics Poland has under the leadership of Dominik. Thank you so much. You are invited to share with us your innovations and personal view as the President of Statistics Poland in terms of this new environment of data production.
>> DOMINIK ROZKRUT: Thank you very much, Alexandre. I'm honored by this invitation. I did whatever I could to participate even though I'm in a bad position for other reasons. But I would like to say that I'm obsessed with innovation in our office, and we are doing a lot of innovative projects. We want to be at the top of the world in terms of what we can do in the statistical office, and I think that there is a huge perspective of what we can still achieve together, working internationally together, to tackle the new data sources and to make use of it, and in favor of our societies, because this is our ultimate goal.
There are a few things I would like to mention that, I won't repeat with those that already spoke. I want to talk about the challenges. There is a important point, we can assume there are these new data sources, like administrative data, years back. We had to tackle those sources somehow. But there are things that we have to consider and I'd like to mention three challenges. First challenge in terms of how the official statistics is organized, we are unfortunately, this is a joke and a serious thing at the same time, part of the public authorities, so it conveys a lot of limitations in terms of how we can act.
In our case, many public health authorities currently lack the capacity and skills to fully realize the potential of the data. Even though we would have the access, theoretically, probably won't be able to take advantage of it. We have to prepare for that. There are a few initiatives especially at international level I'd like to mention in a minute.
The second challenge is that the vast majority of data is collected by the private sector, and there exists little economic incentives, yet many barriers like ethical, privacy related, reputation to share data, beyond any small projects and commercial projects as well. This is the second limitation.
The third challenge is there is lack of dedicated professionals who are responsible for sharing of data externally for social benefit. On the side of the companies as well, that we have to train people that will be able to cooperate with the public sector, and with nonprofit organizations as well, to flourish this data ecosystem that would enable us to exchange data and take advantage of the data, and mutually, all together, you know. So these are the three challenges that I would like to say that are very important.
There are a few areas of obligations that turn out really at the moment that we can really imagine that if there would be a way to access the data, will be able quickly to react and producing some interesting statistics, and just for the sake of having it as a complete discussion, I would like to mention mobile network operator data that is pretty common, everybody knows that. There was a fantastic case of API provided by Google and Apple recently, in terms of the pandemic, I think that's, that was marvelous thing. But of course that doesn't stop there. There are so many other applications that we can think about, if we would only be able to access these data. This is very, very difficult.
The second area is financial transactions, online payments. We have always tried to do so and there have been many private corporations in Poland, for example, which were able to provide statistics based on this source at the time of the pandemic, it was pretty meaningful. I think it should go somehow into the mainstream of official statistics, because it really helps us to produce something that is meaningful, timely and granular. That is the second area.
The third area is something easier I would say, intelligence data, the data that we can go govern ourselves, but we have to prepare ourselves towards these applications, we need to train people. It is not as easy as it seems to be. It has to be linked with the other data sources, and there is many methodological issues involved in that. We still also have to, and that's, may be surprising, basic statistical methodology that would allow us to use new data sources which we didn't foresee two years in the past.
The fourth area is counter data, some countries are already using it, but there is major countries around the world not doing that, so that is a role for us to promote it, and satellite imagery. What we are doing, we especially invested recently in satellite imagery. We have been for the first time this year producing estimates on agricultural crops, based on satellite imagery. We incorporate these new metas into broader spectrum of applications. We are very satisfied with that. We are very proud of that as well at the same time. Of course, AIS which is location system statistics and a few others, these are areas we invest gradually in order to build up the teams that are able to cope with the complexities of these sources. But this is doable. I can tell you this is really doable.
We don't have these, it's very important to say, I remember at the beginning of my career when I started my career, both at university and in the bank, the commercial bank, and I was surprised by the fact how not developed it was in terms of digitalization, and in today's terms. On the other hand, that was a big limitation, big barrier in order to do this new matters whatever, and you had to invest a lot into software and licenses, whatever.
Most of the tools that are necessary nowadays are available as open source, and that is a brilliant occasion for all of us, in order to, in terms of taking advantage of that, and really, you don't need to spend money on that. You just need to train people and grant yourself an access which is the main topic of what we are discussing here.
There is one area especially important, forgotten often, is healthcare systems. The big data in terms of pooling together the data from different parts of the healthcare system can really help us in situations like we have now in the pandemics, also in many other serious problems, therapies, healthcare systems, this is the area that can make the biggest difference. But I don't think it's well understood or mentioned enough.
To make a little humorous, I have this kind of saying, talking about accessing these data sources, like yeti, everybody talks about it but no one has really seen him. The question is what to do. Of course the pandemic increased the level of innovation. We all agree, we need across our groups of chief statisticians, I meet with Neil from Canada and other my brilliant colleagues and analysts, try to harass them with a question, what can you do in order to sustain the level of innovation that you had at the moment, because I argue that when the situation comes back to normal, we will come back to be very slow as usual as statistical offices, and but for these questions they don't like me, but on the other hand they agree to have conferences on that. And the next OECD committee meeting will be devoted to that topic in particular. That is very important.
The European Union are trying to do a lot. I'm engaged in these actions. We had a business to Government data sharing expert group, established by the European Commission. I was part of this group. We were working for two years trying to build up some document that would investigate the possibility to create a European data spaces and how to facilitate data sharing, things like that. We have by the way come up with some set of principles in terms of how we should think about accessing the private data, publicly held data by statistical offices. These principles include the principle of minimal data, so we should get access but proportional to what is needed for public good, transparency, privacy, confidentiality, this is the way we achieve the trust. Nondiscrimination, level playing field, that is important, so if one company is obliged to do something, then the others operating similarly have to do the same.
Of course, following existing statistical frameworks, this is one of the things that we can offer as statistical offices to the creation of data spaces. We have the standards, metadata standards, all the classifications, things like that, that are the core of the language of sharing the data so that we can use that together. The issue of data stewardship was already mentioned, and the role as a steward, at the company, we need to train people who can be those data stewards and collaborate and creating collaboratives for data sharing.
Following this, in the EU we have the European statistical system. We have published a position paper on the European data spaces, and arguing for the increased role of the official statistics in creating those European data spaces. I don't want to take too much time on that. But this is extremely important topic for us, up to the degree that we have already the President of the bureau of the European system, so we have established a few months ago a special task force on access to privately held data. We meet very often, almost once a month, we meet in order to push for creation of the legal basis to access the privately held data for the official statistics within European Union.
My ambition would be, to be also one of the leading regions in the world, as it was in terms of the General Data Protection Regulation, to establish some kind of legal framework that will allow us to, for the public good, to access the privately data, privately held data. At the UN level we are working on global working group on big data and official statistics. We had a general meeting today but I didn't participate. I couldn't participate in that for some reasons. But also within this group, I lead a group task force in capacity and training, and we are developing some programs for national statistical institutes, so they can start training their people using the materials that we produce together. There is a lot of investment into this area. I think that is our future. Of course, there is a bigger role than that for the international organizations. I would like to challenge all of the international organizations, not only the European level, but always OECD, what are you doing in order to facilitate that. It is not just a question of producing a fancy report, whatever, but it is a question of achieving the goals actually as quickly as possible.
To summarize my intervention, I'd like to come back to something so basic as fundamental principles of official statistics. They are based on this single fundamental human right to have access to the truth. That is what official statistics are about, we are there to provide people, the individuals with necessary information to take informed decisions. Of course, it doesn't really matter if this is the regular source as we imagine it for the last 50 years, or is it administrative records or are these big data sources, privately held data. We are there. We have our obligations. We have to provide our societies with the necessary information, and times are that we need to use also these new data sources as well. Thank you very much.
>> ALEXANDRE BARBOSA: Thank you very much, Dominik. Congratulations on all these efforts that Statistics Poland is doing especially under the mandate of the global working group on big data for official statistics. I've seen now the work that you have been doing, you and your team, on capacity‑building. This is really very relevant. But one thing that strikes me, listening to you, the culture of data sharing varies from country to country. Maybe private corporations and banks or other type of data providers from the private sectors in Poland may be more open to sharing their data than let's say in Brazil or U.S. or in Canada. This is something that strikes me.
It is interesting to see that there is a cultural issue behind these possibilities of data sharing. Thank you very much. Now it's my great pleasure to invite Professor Alison, who has been doing wonderful work in Africa in general but in South Africa, to the center that she is Executive Director, Research ICT Africa. We have been cooperating in the past years with RICT. I would say that she represents a important organization related to the data production. Professor Alison, could you share with us your experience with both demand and supply side statistics production in African countries, and in your view, what are the main challenges that data producers are facing in Africa accessing alternative data sources. If you could also in this 8 minutes share your thoughts and views on data governance. You have the floor, Professor. Thank you for being with us.
>> ALISON GILLWALD: Thanks for the generous introduction. I'm going to time myself so Helani has time to speak. If I run over time, please stop me, if nothing rings a bell or does anything. I think people have raised points I wanted to raise. I'll try to follow up.
(distorted audio)
I've got a very large dog that is now, now had enough of Zoom for the day, going to try to get rid of it (chuckles).
I wanted to highlight the points made around how COVID has amplified the implications of not having the data. Your question was how much data do we have in order to monitor the progress towards SDGs. In fact, across Africa, across a whole lot of the indicators but particularly across the digital underlying digital ICT indicators, we don't actually know. We don't know what progress we are making other than generally by way of target, but we don't have the data, because it has not been able at the national statistical level.
COVID has highlighted this. We have seen that people, governance don't have the information that people need in order to make decisions. We have seen even that this digital inequality is playing itself out in the data that is available for COVID mitigation. People are not connected, they are unable to use the technologies for contact tracing and the data is not coming up at AI dashboards around this. Many countries, we are resorting to old manual systems because there is such invisibility, such problems with representation in these existing data sets and data methods that you have been referring to.
It's wonderful to know they are there. But they are actually really for large part of the world they are a figment of some hemisphere imagination.
What have we got, and what can we do? I think that it's important to identify some of the limitations with some of the data that we do have, and because there obviously are enormous potential with big data, and ensuring that this data becomes part of the public good, statistical public good available data that is nonrivalrous and nonexcludable. People can use it. It doesn't exclude some company using the data. We need to think about complementary forms of data as new forms of our national public good databases.
The point is that a lot of data we still are dependent on is supply side data, and across Africa, supply side data provided by operators, and often there isn't a regulatory audit to these, regulatory institution audits of these. Often institutions that are responsible for this within the UN, ITU, are dependent on our very small limited number of countries represented, nationally represented surveys that we are doing.
I want to emphasize the importance of this because there is a lot of talk at the moment about how we are going to, public data can be accessed and emphasize the need for us to continue to strive to try and get these nationally representative data sets. I want to make this point particularly in relation to the Internet before moving on to the fact that we need much more data than simply Internet connectivity data which we seem to have settled for in the past.
But the supply side data that we are receiving in Africa, and we have seen this in the figures of mobile phone ownership that are over a hundred percent, in countries that we know everybody does not have a phone, the supply side data cannot disaggregate the data. We cannot even get a simple figure as a unique subscriber from that supply side data. We know that people have multiple SIM cards, etcetera, so we don't even have the most basic information.
Where we do get disaggregated information in various forms, this often is, this masks the various other policy information that you need around inequality. The figures that we are seeing around gender for example, gender parity or gender differences, these are masking the real underlying drivers of inequality. For our work on gender, because it's been touched on but not spoken at length, we are broadly, it highlights the fact that education and income are primary determinants of access to education. It is the fact that women are not getting education, and therefore the associated incomes, that they are unable to participate.
It is still a matter, it is about whether you are in rural areas which still continue to have much lower penetration rates than urban areas, etcetera. This is the only demand side data can you get that in prepaid mobile markets, where people aren't registered, and even the registration can be a wonderful statistical tool. Everybody knows that registration of SIM cards is a farce in most countries.
What we need to do is understand the importance of that. From a research, evidence based policy point of view, it is only through that nationally representative data that we can do the kind of analysis, the modeling we need to do in order to disaggregate the data and as I said, unveil the masks. That is a important thing to keep in mind. We can't abandon those.
I should say in this Forum that the traditional funding, that we don't get statistics offices across Africa, the in‑depth statistics that we need, we might get a annual survey, something like that, but that is about it. We are not using standardized indicators, international indicators, mobile phones, household penetration. We have enormous problems around connections to most basic statistics, but the important part of this is that we have been doing these surveys for over a decade in Africa, the data depended on why the ITU, the World Bank, regularly pleads with us, they can have it before we publicize it, etcetera, and yet we cannot get sustained funding for this public good, public accessible, database all made accessible, and yet we can't get funding from them. Even traditional funding that has come, that has been remarkable over the last years, decades or more, has dried up. That money is now devoted to artificial intelligence and other things, big data, that you cannot actually get a basic survey done.
What is interesting about COVID, we have suddenly had governments saying where is your data, your data is from 2018. Where is the next round of data. It is not there. We don't understand implications of COVID as a result. I want to urge us as a international community to, these are global public goods now, we are talking about global services, global things that we have to govern globally to make effective and we need to share those resources, including looking at the global digital text as a very small portion of that going towards the indicators we need for governments to be able to use this data meaningfully.
The other point I want to make, we need new digital indicators. We are looking at old Internet indicators. We need indicators for the digitalization that we are seeing across our economies and societies. That is what we try to do with the after access survey that we did together with LIRNEasia, and maybe Helani can post that access link for us while we are speaking, but that goes beyond ‑‑ there is my time. It goes beyond the basic indicators to look at microwork, platform workers, mobile, done for a long period of time, Cybersecurity awareness, data governance, all the stuff that we need in order to govern, to be able to share data, to be able to allow data to cross borders, these kinds of governance arrangements that we need for data sharing that aren't even in place yet. We need empirical demand side data to effectively govern these data governance. Thank you very much.
>> ALEXANDRE BARBOSA: Thank you so much, Alison. It is our great pleasure to listen to your bright insights. I will not summarize what you said because I have to run to get Helani, so last but definitely not least, I will give the floor to Helani Galpaya from LIRNEasia, which is a important Think Tank in Asia Pacific, you probably know in terms of the work they do and research they do in terms of policy research on infrastructure industry including ICT industry. Helani is very much involved in the production and also advocating for the need for data for evidence policymaking. Let me ask you to share your experience with us, and give us your insights on issues such as privacy and Civil Society data sources and privacy of data in the context of a new data ecosystem. Please, you have the floor.
>> HELANI GALPAYA: Thank you, Alexandre. Thank you, Alison, for putting yourself on timer. We are a Think Tank. We are outside of Government. We work in south Asia, southeast Asia. We do both traditional survey related data collections, like after access which is useful, over the past 7 years, looking at big data and methods of using new sources of data and predicting certain social science behaviors using these sources of data.
A lot of work is, OECD is like fiction for us, we have over 90 percent penetration of cell phones, that is it. There isn't a lot of things. That is our IoT people walking around. What really is for example negotiate three month old on running basis so historical, access to records from multiple network operators, for example, in Sri Lanka, we have used that data for major things including advising governments on transport planning how it should be, what is the change in nature of the city, in our part of the world, cities are changing so fast, even four year economic census is not going to capture residential versus commercial areas as part of the change.
To capture where people live, people migrate to at a given time, and to understand where diseases are spreading, by combining the people movement data with everything from rainfall and vegetation data, so to understand where the disease spreads. One of the things that we worry about in using this proxy indicator of people is that whether this is really the signal that is seen in the core detail record of a Telecom operator is in fact a person, or there is multiple people. We have signal from a human being, is important to address, given behavior that we know elsewhere. We worry about who is in and who is out, while this is very nice, there is still at least 10 percent who don't show up with the phone signal. How do we collect the data about them, and we worry about that all the time.
We of course worry about the personal information that is represented in this core detail records because it's, nothing if fully anonymized. We assign a unique identifier, and it is at least in theory possible to go back and recreate some proxies for who these people are or where they might live. In parallel we are doing research on how to use these large databases, where you can anonymize them to the extent they are not usable, to answer in these social science research questions we are asking, or you can use techniques like creating synthetic data, and we are doing this on creating synthetic data on call detail records for example, so that you have the same level of confidence and error rates and use ability of data sets, but there is a huge level of privacy preservation that also comes in return, beyond just anonymization.
Another type of research we do is to understand the long term changes in demand for skills and education. For example, going back to UNESCO's, the skill frameworks, transfer of skills, we have data from large job platforms and we are anonymizing them using natural language processing, to understand what are the skills and how has the demand for it changed over short periods like pre and post pandemic but slightly longer periods, therefore what are the indications for vocational policy and so on. In both of these, the way we have been able to access data provides one of biggest problems. It is based on relationships. Because we have had the relationships with many of the Telecom operators and have been operating in these markets for a long time, the job platforms, we are able to negotiate this data. Unfortunately, this is still how the data is shared, and all the examples I heard from this panel is also because of power or relationships that you get the data, right? While we are beneficiary of that, we have been on the other side of this also, because there is no common agreement on sharing even the basic Telecom data. That is a fundamental problem.
What we have is large multinational companies and platforms coming and doing the reverse, not just giving their data to some one party, but coming and negotiating with governments in a way that nobody else can, no other small firm or Think Tank can or society can, and taking Government data, for example, and using it to do very valuable things, like highly sensitive modeling which can save lives because compared to physical model of flooding this alerts people 48 hours before. This is fantastic. We can issue an alert. But why would the Government not make that data available, to insurance provider and research organization in the country. Second, what are implications of all the time relying heavily on a model that is developed on data from India, Bangladesh, Sri Lanka, America and so on and getting results versus developing capacity in other countries because over time you are going to lose capacity and you are going to lose the ability to get models unless you have access to the core of the model and algorithm used by multinational.
The next time the flood hits and the multinational algorithm is wrong, who is going to take accountability for this? We are not addressing a lot of these issues. The whole ecosystem around data governance and sharing is problematic, big players are at a advantage. Relationships are at an advantage. Singapore is doing interesting work looking at small players can come in, contribute with their own data instead of having to make payments to access data. We run into the problem of data provenance, when we talk to a NSO using these new methods, it is a skill issue, they don't know how to do it. It is a different conversation from a stats Canada that we have in many of the south Asia countries, how are we doing, is the model reliable, transform modeling has been done in this way for ten years and so on.
I will stop there. There are regulatory ways of getting people to come to the table, there are incentives which many of the people have talked about, and there are other innovative ways. Thank you.
>> ALEXANDRE BARBOSA: Thank you so much, Helani, because you highlighted the fact that the context, the regional context like we are talking about Asia and southeast Asia is very different from Africa, from Latin America, of course from Europe and global north in general. This pose different challenges in terms of access and data sources, and also in terms of fundings and sustainability. Those are important issues that we have to keep in mind, because we are not talking about the same level of development in all these regions, Latin America and Africa and southeast Asia.
Thank you so much. I'm really sorry that we are running out of time. We have some questions, we had people following here in Zoom as participants, and also YouTube, as if I'm not mistaken, the peak of participants we are in the total Zoom plus YouTube about 45 people which is quite good audience for this topic. If you are here participants, dear participants, it is because you have interest in data and in statistics and data production.
I think that some participants brought some questions that we are not going to have time to reply but I would like at least to acknowledge and to read them. We have five questions, two of them was already replied by Jaco from UNESCO. We have one, could you please share the data governance model used to gain insight for realtime decision‑making. Second, how does data ownership, data stewardship managed in heterogeneous platform with public‑private partnerships, PPP, considering trust, transparency, to obtain a sharing economy. Another question, regarding the organic data sources, accumulated by private entities, often used for private purpose, such as competitive insight, innovation, etcetera, how can we encourage owners to share the data with regional statistical office for the production of public statistics. Next, so far, we have discussed the collection of data to monitor SDGs, but I'd like to be, I'd be interested to know whether the organizations represented here also model this data to predict the development of SDGs fulfillment. That is a difficult question. How can we assure that no one uses national data sharing global level for illegitimate geopolitical goals. I think that we need globally agree that governance, conventional UN framework in this regard. Another one, how, does the African Union has any plan for better data in Africa. Last, what is the UNESCO UIS, what role is UNESCO playing in ICT data collection for monitoring SDGs, so very interesting questions.
But unfortunately, we don't have time to discuss. I think that for me, I could learn a lot from your experience and your insight. Of course, we didn't even touch things like bias of big data, coverage, issues related to coverage. I guess that Alison has mentioned about representativeness of data for evidence based policymaking. This is another issue. Privacy, sustainability, funding. So all these issues are important to be in the debate.
I think that today was just a flavor of what is waiting for us data producers, and key stakeholders in this ecosystems to discuss. I was about to ask one minute round of last comment but I think I will skip that part because it's not possible to give a good message in one minute. I would like to finish by thanking all the speakers. I really appreciate your time, appreciate your being here. And I would like to read what the UN IGF organizer has asked to form a call for voluntary commitments. There is a call for voluntary commitment that has been launched, it is a call for voluntary actions or pledge for, to follow the goals of the Internet Governance Forum and the Secretary‑General roadmap for digital cooperation. This is really relevant, in my opinion. Any stakeholders can make a voluntary commitment of a action to be carried out during the 2021 cycle until next year IGF, 20201, that I really hope that we can be together face‑to‑face in Poland, I hope or beyond, actions that are supportive for achieving or implementing IGF objectives or action areas related to the Secretary‑General's roadmap. Therefore, as moderator of this panel I invite all of you to respond the call for this voluntary commitments. Your commitments can be done either verbally during the session that is finishing right now, and if you prefer, you can submit them through the Forum available for that purpose, which will also be facilitated in our chat box. I guess that Ana Laura has already disseminated this link for you to reply. But I will make sure that we will send a E‑mail to you with the link. Self report and self evaluation of this voluntary commitments will be made possible through the IGF Secretariat Web site, and will be reviewed at the first open consultation. Thank you so much for your commitment. Please stay safe. I send out the best energies to you in this very difficult and very strange moment that we are living. Definitely, data production, data sharing, data governance is going to be a hot topic for us, and we have a responsibility in this debate. Thank you so much and I wish you good day. Thank you.