IGF 2018 - Day 3 - Salle II - WS75 Approaches to a Wicked Problem: Stakeholders Promote Enhanced Coordination and Collaborative, Risk-Based Frameworks of Regional and National Cybersecurity Initiatives

The following are the outputs of the real-time captioning taken during the Thirteenth Annual Meeting of the Internet Governance Forum (IGF) in Paris, France, from 12 to 14 November 2018. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid to understanding the proceedings at the event, but should not be treated as an authoritative record. 

***

 

Thank you very much for being with us today. This workshop will tackle an important issue around cybersecurity and collaborate on regional and national security initiatives. My name is Weiss Weiss Weiss I live in Brussels. So, when the idea of this workshop is really government really struggling around cyberthe security now that more and more objects are connected to the internet. I think it's really a key issue in order to create a trusted environment also for people to be able and use the devices that are connected to the internet.

      I have with me here a great lineup of speakers. So the -- and the -- let's say the panel will focus on two -- on two aspects. From the challenges and capacity building and the development of the national cybersecurity framework. The second is for regional and national and international cybersecurity.

      I will leave the floor to the speakers. Five minutes each for your remarks. Then we'll open up to the audience as well because it's very interesting to have the captions. So I really welcome your questions.

      With that, I'm going to start with Bill Dutton. The floor is yours.

     >> WILLIAM DUTTON: Thanks, chair. We had a conference in February of 2018 where we at the GCOCC in oxford, we brought people in not to focus on the findings of our work, but to actually have people come in and talk about findings around the world from our collaborators. So we were gathering together observations about cybersecurity. And we had a full conference over a full day and I'm going to summarize it in five minutes and try to give you some of the key themes that I thought were most interesting and maybe provocative for the panel.

      One was the concept of cybersecurity being a wicked problem. And that's a nice way of putting it and we capture that in a report and it will be available. But the idea being there's so many actors, so many potential risks, and so many technologies involved and these are all changing as we speak so that dealing with this problem is wicked in the sense that it's -- it's very difficult to resolve. And people have moved away from saying we're going to resolve it. We're going to have cybersecurity. It's going to be resilient and capacity to deal with cybersecurity on an ongoing basis.

      So this wicked problem notion is very useful. And as the chair said, there were -- there were two sort of themes. One around the sense of building cybersecurity capacity. The other was thinking much more regionally and globally. And I will briefly give you some sense of what we meant in the two categories.

      With respect to national cybersecurity capacity building, one of the interesting things is that people were saying that cybersecurity is now not a -- they don't have to sell cybersecurity as much. Everybody recognizes cybersecurity as a big issue. This is a big turning point in their -- in the sense that at all levels, local, regional, and across government and business, cybersecurity is recognized as a central problem.

      The other was there was some consensus on the central -- the central steps, early steps in developing capacity. And those tend to center around developing strategy and also working on the development of a national computer security incident response, assert that -- and development asserts and strategies seem to be one of the most emphasized aspects of capacity building at the national level.

      Another interesting thing to me in this area was people mentioned cybercrime has had what -- one reason why cybersecurity is agreed as a -- as high priority especially in emerging economies as well as developed economies is this process of cybercrime and that it is focused people's attention on the priority that has to be given to cybersecurity capacity. And also it is becoming interesting.

      One of the best examples of collaboration across governments and across sectors in nations. So this sharing of data, information, coordination of police and so forth. So that in the cybercrime area, there seems to be a leading edge in terms of developing capacity for collaboration across jurisdictions, for example. And in data sharing.

      The other was there was certainly this wide spread recognition that -- that a technical focus on cybersecurity is misplaced unless people focus also on the cultural and societal aspects of cybersecurity, things like developing norms that may not be just in terms of major cyberattacks and what -- and what norms should be guiding this. But even among users. And how do we develop norms and cultural mindsets to that prioritized cybersecurity so that people are just consciously thinking about cybersecurity in their everyday information practices.

      And the other was the importance of getting indicators of capability. And this is one of the things we do at the GCSC is the -- is develop indicators of capability. But there was a fairly good consensus that indicators like this are valuable in all countries because they show their methods of indicating progress, their -- they're important for showing and demonstrating impact, and so forth.

      But then the second theme was much more -- I found it was -- there seemed to be a shift from simply developing national cybersecurity capacity to thinking more regionally and globally. And it was almost like we think we know how to do the national capacity building. And people are in progress in that. But we're less adept at doing regional and global collaboration. And the way it came about is a way I recorded it was a sort of a twist, which was acting regionally and globally in order to address -- in order to effect local problems. So, acting globally to be -- and regionally to help locally.

      And there seemed to be this sense that -- of -- and there were great examples of collaboration in east Asia and other parts of the world in which there were regional efforts that really were affecting local -- local issues such as training programs, international support for the costs of cybercapacity building, and simply discussing this regionally. And making this a priority of foreign policy at the regional and global level.

      But, I'll stop there. And hopefully one or more of these themes will be picked up by others and we can build on that in the discussion. Thank you.

     >> Carolyn Weisser. Thank you very much, Bill. I will give the floor to Greg Shannon, who's in the software engineering institute. So the floor is yours.

     >> Thank you, Chair. Glad to be here talking to you this morning. I'm here on behalf of IEEE as well as Carnegie University. In my day job, I'm the scientist for the first search for incident response capability and we are now helping to promulgate throughout the world with the help of the U.S. government in terms of building regional and national capacity throughout the world. This is work that's been going on for three decades now.

      What I want to highlight today is -- this is my fourth IGF and I see a clash of cultures in how we think about capacity and how we think about cybersecurity. On the one hand, we see the U.N. perspective of the nation led multilateral approach. On the other hand we have the internet built up in the multi- stake holder approach. If you want to look for a model of regional and national collaboration, you need look no further than the IETF with the internet engineering task force where they meet three times a year and figure out how to engineer the internet. Since the internet is always on the verge of breaking down because it continues to grow, they always have a very full docket.

      If you look at what makes incident response work effective in this area, what it comes down to is understanding and respecting the agency, the ability of engineers infield to make decisions and respond. I think this is a key point that highlights the difference in cultures. Who has agency to make decisions? Is it only the government? Or is it actually -- people in the field who are putting their hands on the key board, on the cables, and helping to keep connectivity there? Helping to respond to security incidents, understand what the attacks are, doing the forensics, are working with law enforcement. They have the agency.

      So there's really two models here. One is that the respects and encourages that distributed agency and teaches that distributed agency model to our -- from a capacity building point of view. The alternative is one that said, no, there are a set of rules that you must follow. And that is how your job is described and we understand the job that you need to do and you can do that job only. And you have -- you have limited agency.

      Of course, if you give someone agency, it's always in the context of a -- it's grounded in a moral and cultural and ethical context. You know, we have agency, moral agency, you want to have people trust you. So you have to have some sense of accountability. Whether it's your reputation, which is traditionally how the internet has worked, where your reputation as a responder to cybersecurity incidents is based on your performance with your peers. Not necessarily a law there. And your competency. These two alternatives provide -- if you have a distributed agency, you can have it expansive and agility. You can allow people to do experimentation.

      We've never run an internet before on a global scale. So anyone who pretends to know what the solution is is really not being honest about how things are operating and the degree to which experimentation is important. This is agility and experimentation, we get innovation and resilience. And many governments, of course, want ininnovation, many citizens want a resilient infrastructure.

      Take a restricted approach to agency in stove pipes and bureaucracy. And two groups need to be able to interact instead of trusting the -- trusting the members to make good choices. You end up with a bureaucratic approach. This leads to stagnation and brittleness. This is what adversaries want to exploit. It's much easier to destabilize a brittle internet than it is an internet that has a resilience built into it through a distributed agency model.

      I think that's one thing to keep in mind as we try to build global capacity in this area. Thank you very much.

     >> CAROLIN WEISSER: Thank you, Greg. The I move now to David Duren from GAC if you wanted to --

     >> DAVID DUREN: Thank you, chair. In the next five minutes, I would like to describe how what the GAC is and how we are -- how we are developing. The GFC is a neutral and formal platform of private organizations, with governments, country s, and also with international organizations.

      It was designed in 2015. At that time, there were several organizations, countries who were cyber capacity building. But there was a need to bring them together. To create a platform that could bring the parties together to be more efficient, to be able to find each other, to be able to reduce overlap.

      So, so the goal of the GFC in 2015 when it was created was to strengthen international corporation and coordination and cyber capacity building. And in that promise, we started where a lot of organizations are starting, the platforms. With awareness raising.

      So if I look at the GFC in the developments and steps we've taken, I see a process from awareness to implementation and try together within this platform, try to create an ecosystem for cyber capacity building.

      So what we've done, what are sort of key steps that we've taken in 2015 and 2016? First of all, I think it's important to build a personal network, a network where people know each other, where people find each other, so that's what we've facilitated in 2015.

      Also, and this is one of the needs of the global community, is to create some overview on cyber capacity building. So we did it in a different race, bring together people also to -- to challenge them to come up with best practices. The collaboration and the collaboration with the cyber capacity and they are a portal trying to get the noble few on what is going on around the world, what initiatives, what activities and cyber capacity building are going on?

      If you talk about cyber capacity building, you want to be action-oriented. So we have the concept of initiatives or say programs -- where members find each other, work together, and make it possible for their best practice physical for the global community and the GFC community.

      We also produced like products. If you talked about cyber capacity building, it's mostly about practical products, like tool kits, guidelines, how to implement, for example, assurance, or how to deal with the mobility disclosure, or how to develop cyber strategies very practical things, topics, themes, that -- that almost everywhere in the world people are looking for.

      So that was 2015/2016. I think there's a lot of awareness raising for cyber capacity building. Making nobody what is out there, what are the best practices, and those sorts of things.

      In 2017, we developed the so-called communique so a high-level document presented at the conference on cyber space in Delhi in 2017. It was the commitment of cyber capacity building. And it's about awareness raising on a high level, the capacity for cyber capacity building. And it was also a framework that was provided with it with themes, topics, that are important.

      So now I come to the last part. So what's next? So there's some -- what we try to do like a set with the step-by-step, is to create an ecosystem for cyber capacity building. And what we've done in 2018 is the working groups along the lines of the themes of the Delhi communique.

      And these working group s bring together members of the GFC where you can support, and get knowledge from the community and implement it. So this triangle is really important if you move to what's more implementation efforts.

      Another thing is what we developed, what's -- what we try to do in the GFC is some sort of clearinghouse, a place for information, those who bring together needs and support and to create a mechanism or a process to be able to do that, to bring that together.

      And the last part, another key element is to create a CCB cyber capacity building knowledge portal where the global community can find practical products and information about lessons learned and how to implement and also to see, to get on what our initiatives, what are the projects world-wide topics that matter to me.

      So, in a nutshell, what we do is step-by-step creating and facilitating the global community for cyber capacity building and creating this ecosystem. Thank you.

     >> CAROLIN WEISSER: Thank you very much, David. The lady from the energy -- Akvile, she's not here. So the next in line is you if you want to give your remarks, from Microsoft.

     >> Thank you for the opportunity to be here on this panel today. I just want to pick up on some of the comments that my colleagues have started us out with. And focus in particular in the area of national efforts to develop cyber security policies, particularly in the context of critical infrastructure protection. And think about why it's really important in the context of those developments to think about collaboration regionally and globally. And in particular, pick up on some of the comments that colleagues have made related to the importance of technical security not being sufficient, but also weeding through the national efforts to build sort of cultural and social capacities for cyber security risk management as well as the importance of resiliency and the importance of forums like the GFCE for ensuring that there's an opportunity to share best practices in cyber security risk management across borders.

      So in particular from our perspective, we're tracking lots of different national and regional efforts around the world that governments are really making to ensure that there, you know, their local organizations have the cyber security capacity to deal with what's confronting them today. And in particular, we're seeing lots of developments in the context of critical infrastructure protection as connectivity in those critical sectors really increases, the importance of this issue also increases. So government attention in this area is really important.

      Just a few examples of some of the policies that I'm talking about in the EU we have a networking information security directive and in China, the cyber security law that's being implemented that will impact many critical sectors, Singapore, cyber security act, and in Japan, there's the development that's on going of the cyber physical security framework. These are just a few examples. There's lots of others sort of in development legislation around the world that intends to really think about how to ensure that it's critical and -- critical sectors are protected from the cyber security perspective.

      And we're also in lots of national strategies, seeing lots of governments intending to do work in this space. And from our perspective, this is really important. This is a really critical opportunity to spread cyber security capacity across critical sectors. It -- and so we're really excited about the opportunity to engage on these government efforts.

      At the same time, there is a risk that some of these efforts, you know, really introduce compliance-focused approaches that in some ways have resources that can otherwise be focused on cyber security risk management and sort of compliance-focused approaches.

      There's also a risk that some of the developments really fragment across both borders and sectors and that could create new complexities for cyber security risk management. So we think it's an important opportunity but it's important that governments are thinking about, you know, what we've learned thus far. And I think Greg's comments about really ensuring that there is an ability for actors with their fingers on the key board to be agile. And it's important to think about what different governments are doing across sectors to make sure that that sort of fragmentation is managed.

      So a couple of thoughts in particular on the importance of having risk-based governments to be approachable from what governments are doing in this space. From our perspective, this is a couple of the comments that Bill kicked us off with, not just security being important but the social and cultural capacities that need to be developed and knowing what to suggest that some sort of controls focused their purchase, some more prescriptive approaches cannot be more valuable. But it's important that the foundation of cyber security risk management and operational risk management is in place for any organization. Is that sort of the kind of foundational element that enables organizations to really have good conversations that build those sort of cultural and social capacities to really think from an organizational perspective. Are we looking across the entire environment? Are we really processing the risks that we're taking on. Are we really having a mature conversation about how we're managing those risks. Are we detecting threats and incidents? Are we prepared to respond to those incidents. Are we resilient. These are the questions that really come to the surface when the conversation is based on a wholistic approach to risk management.

      From our perspective as a company, we actually used the cyber security framework which was initially developed by the national institutes of standards and technology in the U.S. in collaboration with lots of other governments and industry stakeholders really reflecting that multi- stake holder approach where you have lots of different folks coming together to reflect their expertise and build a best practice in cyber security risk management.

      We've also seen the cyber security framework move to the ISOCIC world which is is exciting. There was ISOISC2003, it's a report building on the report established by the framework and having a wholistic approach to cyber risk management and brings that to a kind of an international context and thinking of how to build on all of the work that's been ongoing for the last decade or more in the space of information security.

      And really I kind of ground that in the holistic approach to risk management that identify, protect, respond, and recover really the five kind of major steps that we see the wholistic risk management program and the cyber security context.

      One or two more comments on why from our perspective, these two approaches are really valuable to think about. One, you know, we think that the cyber security framework really enabled that kind of conversation across an organization. Across business groups as a function of the organization. Kind of across from the -- within the vertical parts of an organization. So kind of the senior leadership of the organization, enabled a really consistent conversation about cyber security risk management which is really critical, again, to building the cultural capacities as an organization to really make sure that the right investments are being made in cyber security.

      The other reason why we think those two reference points in a particular ISOIUC273 are valuable helps to kind of deal with that challenge that I mentioned at the outset that lots of governments do really important work in this space. There are challenges if governments develop really different approaches and having a global ecosystem and having fragmentation across that from regional context and a sector context.

      So making sure there's efforts to look across borders and see what best practices are being developed. And leveraging those at least as a starting point and figuring out where it makes sense to make adjustments for a national context I think is really important. Thank you.

      Thank you, Amanda. I'm going to move to Juan Manuel Wilches is the commissioner, the Comision of Comunicaciones of Colombia.

     >> Juan Manuel Wilches: The stakeholders' approach to cyber security is on the developed and I guess I can say that Colombia is an example that we are still in our infancy, we're still developing our policies.

      We got something done in 2016. I cannot based on collaboration and the work that we did to the OES. And also to the work -- I want to talk about and try to show a little bit and how we came pout with the policy. But I guess it's the first step -- it's one of the first steps I want to continue to work.

      We had a policy in 2011 it was the first policy for the country. It was capacity building. Trying to build up the cyber security environment. And it worked for a couple of years.   But then we found out that we were not collaborating, we were not working together. And the defense was on one side, the intelligence agencies were on the other side. The ICT issue was on another side. We department have -- we didn't have infrastructure or communication. So we started to work on how to -- how to manage -- how to manage it to get to a different stage and we looked for help from the OAS, from experts, internet experts. We need to learn from other countries. We had a lot of support from the UK, the U.S., a lot of countries, France.

      But the thing was we spent almost 2014, 2015, to discuss things. We had a lot of information and the lines. But we're lacking the most important thing for any country, which is to have a strategic vision within the country for the need to implement policy. But it's over -- the whole -- to create the environment. That's not only about defense or about security. It's not only about ICTs or net worths. We need to do something that encompassed the whole thing.

      I want to think in the same line of thought, the whole of government. So I guess from that, we had some help if I can say like that. It pushes forward for the OCD. The process to the organization. And in September of 2015, the recommendation came out. It's December of 2015. You need to make a decision on the next policy and cybersecurity or we're going to gain access to the organization.

      So I guess that made us think as a government just one way, like one focus, one objective. And once we should -- the policy was April, 2016. It was one of the first countries to issue the cybersecurity policy based on the recommendations for the OECD. And I guess he served some example for other countries in the region. We're learning as well as giving to other countries. So I guess we have to have that collaboration. You never start learning about different stuff that you need to implement in this kind of environment because this environment changes so quickly that we need to be updated every day.

      I wanted to highlight some of the most important stuff that built that process. In 2014 and 2015, we worked, we collaborated with OES, we had a lot of experts. We have good recommendations from those experts. But we had a lack of commitment from government stakeholders in the country in Colombia.

      One of the groups was requesting more participation in the discussion that wasn't happening as well as -- they didn't have that and they're criticizing the policy of the policy.

      We got a push for the OECD, for the process. And I guess we didn't come in the objective and did it based on the highest level of government, the minister of ICTs, the presidency. Once we got -- got an objective in this case, we find out that we're limited to coming up with a unique solution to the problem of the policy, like security, defense, social economic objectives, hope all of them combined for just resolution. And I think that's what I mentioned in the beginning.

      So once that highest level of government was agreed on what needed to be done, the work flowed. We created an agenda and established the working groups. Different tables where we had different stakeholders to discuss what we needed to define.

      We came out as a simple policy. It's based on -- four basic principles. The first is to safeguard human rights, the second is to adopt a collaborative approach. It's important to put that in the decision makers and all of the stakeholders. We need to collaborate. It won't work.

      And I think there's a risk management approach. And define different dimensions for that. Some of the speakers before have talked about this. The first one is governance. We need to find a structure -- institutional to decisions about that. So that presidency took the role. They're the ones leading the discussion, because they can -- they can combine the interest of the different stakeholders, the difference, they were to protect, they want to control, they want to match the other stakeholders, human rights can be protected. And the minister of ICTs for the minister of economy. We need that kind of development. So we need to combine all of those objectives in just one.

      We have management of risk in the recommendation. We developed one of the cultures of the citizens. We need to create awareness in the different stakeholders. I guess the analysts have talked about that a lot more. Everyone in the country, what is the security. Why they need to protect starting with their cell phones and how do we can protect ourselves. And just the capacity building, collaborate, continue to work with other countries, continue to as I was saying, give information as well as learn from the other countries. And I guess that's the -- that's the key.

      And I wanted to mention a couple of data statistics that we have. The working definition of how companies need to implement those policies. We had a survey with them. I will find out that we're not still -- we're not doing as well as we would like to. We're not sharing information about incidents, for example. 78% of operators don't send information to the cert. And 64% of them don't coordinate with the cert. All that -- no specified product for the whole for the cert. We'll need to work more on that. It's good. But we continue to work with it. Thank you.

     >> CAROLIN WEISSER. Last but not least, Kerry Ann Barrett. 

     >> Kerry Ann Barrett: We have within us a cyber security program, how we approach capacity building is threefold in that we look at it -- we try to look at it wholistically recognizing that in order to address as Amanda points out the risks, even some of the risk that companies and countries or individuals face, we have to kind of examine, okay, what is happening? One, do persons have the knowledge that they are at risk? So, within our program, we have an entire section dedicated to research. Some of the reports that we produced are some of the first in Latin America actually addressing the threat of cyber security and what's happening.

      We've been publishing reports since 2012. And reports often try to hold together all the experiences of the member states in terms of what they're seeing as threats in the region. How do I approach my risk. Secondly, do I have the capacity to identify and am I at risk? So our program focuses on building the capacity of technical persons within our region. We build the capacity of our legal persons, we look at awareness raising as he points it out in terms of does my citizen, do they know they're at risk and what they can do to protect themselves?

      The third approach that we take in tying all of those things together is looking at capacity building from a more political level. We take that political view from the policy development and establish a framework development perspective. So just follow the thought that capacity building needs to happen, it wouldn't necessarily drill down to the national level and grassroots the citizens themselves. So at the OAS in 2004, we published a Helms Faric cyber security framework. There's a need to develop cybersecurity policies and there is a need to establish national incident response teams and there is a need to raise the awareness level of our citizens in the region.

      So, taking it from that perspective, the program, then, works with our member states to build out national cybersecurity strategies. Why is that important? Is because it gives from the international region and now the national level, the government's attitude and even the mandate to build out capacity within their country.

      So, most of our national cybersecurity strategies, which we have about ten in the region right now, we're kind of proud of that, because in just one year alone, we had four countries publishing strategies. And for us, it's important because there's a political consciousness and a need to build cybersecurity strategy. And each time we do go to the member states to do it, we take it from the whole of government and whole of nation approach. The whole of government and nation looks into the fact in a you do have a need to bring together public-private partnership. We do roundtable discussions. We look at how can we leverage on international ways of experience. We do it in UK, Canada, the USA actually that partner with us to go to the country to bring that best experience into the country as well. And the whole of government, whole of nations take into account, how do we have civil society at the table? For example, Guatemala, Mexico, Colombia, civil society commented critically on the development of the strategy process and what we loved was that the government was transparent with the development because it was transparent on the relevant websites. It allowed public feedback and that fed back to the drafts. For example, in Costa Rica, the process was led by NGO for the comment period. And they actually took notes, fed that back to the government, and the strategy was actually improved in all cases, especially in Guatemala. Especially a critical factor in it. In Mexico as well, the multi-stakeholder had a date dedicated to civil society where we presented the civil society at their individual roundtable all of the strategy issues and they actually commented as well. So I think if you approach the problem from a risk-based approach, you have to look at the vectors and I think in terms of knowing what is tolerable for you, it's not just from a technical perspective. But you have to look at it from a staff perspective.

      If my nation needs to be cybersecurity and my technical persons aren't trained up, the political level, there's no will, the risk is big. And actually it galvanizes because there's no corporation internationally if I can't trust you, then.

      So the perspective, I think, that we would want to add is approaching capacity building from our risk-based approach means looking at capacity building from a whole of government, a whole of nation approach. And a multi- stake holder model should not just think about involving civil society, but involving everyone, public Spector. Civil society and even the end user and citizens that are part of the process.

      I think on that I'll probably close.

     >> CAROLIN WEISSER: Thank you. I heard a lot of Morant quotes from the speakers and highlighting the importance of building social capacity, the fact of approaching the problem from an wholistic standpoint, the importance of involving everyone on the table in a profession, the digital environment is certainly evolving so we need to constantly update each other. And certain ly cyber crimes, cyber security is recognized as an important problem and everyone more or less highlighted the importance of the collaborative approaches.

      Now, I would also just to open up the floor from -- for the audience, questions, comments?

If you also want to react to any comments that the speakers have done, it's really a welcome. So I -- I'm looking around to see if there is a first person who would like to start to break the ice. No? Yeah. Please.

     >> Hi, I'm with the Israeli minister of justice. We're involved in developing Israel's legal framework for cybersecurity. And one issue that keeps recurring and actually Israel hosted a panel last year on this topic, but we're still in the thick of thing, and I would be interested in hearing what the panelists have to say is this tension between on one hand, citizens want the government to take care of things, to be responsible to devise the right incentives, adopt the right standards, etc.

And on the other hand, there's this skepticism or fear that once ite's done that, the government will hold too much power, too much information. And so this inherent tension is still something that I think countries that are trying to implement a good cybersecurity ecosystem are struggling with or may be struggling with. So I'm curious if you've encountered this kind of tension and how you suggest going about resolve it.

     >> CAROLIN WEISSER: Please.

     >> I think one can look at the concept of the voluntary risk-based approach is one model is mentioned earlier. You know, that Microsoft and others in the U.S. have used where the multi- stake holder conversation, you know, to identify what are the priorities. So I think that's one of the real challenges is getting the multi- stake holder loose so you can identify and at least discuss the tensions and have a conclusion that respects those -- those tensions. I think people understand there is a tension certainly in the U.S. We haven't quite worked it all out. But I think that is one approach to consider.

     >> Just to build on that. Part of the cybersecurity capacity building model, what the -- what the -- certainly the one that we looked at from the oxford perspective is it builds in policy on privacy, freedom of expression. And I think it's important for people to realize that capacity building and cybersecurity is not focused on cybersecurity only but how do you balance that with other values. And it has to continually build into a process of capacity building means that you also get -- protect privacy, freedom of expression. And that these values are not forgotten in the process of trying to secure computer systems and so forth.

     >> I think it's part of the transparency of government. That helps as well. If citizens are confident that the government is transparent with the process of digital development and as we pointed out, if human rights is by design and security is by design as the term that's being floated now, I think it's more -- it's critical that the transparency helps.

      So, as I'm building out additional economy and facilitating an environment that an investor can thrive, the government is transparent that I'm considering human rights, I'm considering security.

     >> No, the one thing I would underline that's been brought up in response is I think the process of developing and implementing some of the policies is really important. We talk a lot about the substance of the policy, what it looks like, and if that, of course, is important as well. But from our perspective, if you have a really good process, then you can likely get to a good outcome.

      It's by highlighted, Kerry Ann, you talked about the rope bust process that you've gone through to make sure there's a whole nation, a whole government approach, really, having everyone come to the table. You mentioned about the cybersecurity framework process. That was a year-long process developing that. And that was intensive. There were multiple workshops all over the United States where folks came together and really talked about these issues and what the implications of different approaches might be. There are multiple common opportunities. Even before the first version of the framework was published. And even since, you know, it's been four years since the first version of the framework was published. It was recently updated with the 1.1 this year.

      In between, there have been three or four common opportunities for stakeholders to have a chance to provide input on. How's it going? How is the implementation of this going? What are unexpected issues that have arisen? What are new risks that aren't being considered? So I think just to underline, I think what other colleagues have said, the process can be really important, not only beginning to an effective approach, but also to building trust.

     >> Yes?

     >> I was thinking something that happened in Colombia about three years ago. There was a security issue happening through apps. People were running stuff and the user of defense came up to say we continue to block social networks and the situation needed to be control because it was affecting security in the country

      I guess that's part of the reflection of when you have the effects of issues that you take as minister of defense in terms of how it affects citizenship. I don't say that it was wrong. I guess it was what he was trying to solve because it was his problem. I come from the military -- what Kerry Ann said about thinking about protecting human rights, protecting privacy, what others have said. When all of the different stakeholders are thinking in those terms, you need to consider the whole thing, the whole framework. You cannot do your part and then -- and then don't -- don't get to consider the other points of view. You end up allowing for situation for different stakeholders, you end up having discussions, you end up having workshops and you end up building those like this for the principles, one of the principles human rights need to protect citizens, you need to protect the communication rights, you need to protect the rights. And all of us have to be aware of that, the mission of defense, telecommunications, intelligence, whatever it is, we need to have the principles in place.

     >> Okay? Of course, of course.

     >> I just -- I think there's sort of a tension on the panel about a couple of issues. And I just want to raise that with my friends there. My friend from Colombia.

     >> What is the issue?

     >> One is, I think first of all, this -- I think I totally believe in agency. And also I think fragmentation. It's a real problem. It's really interesting when I talked to our researchers when we come back from country reviews. And one of the process -- first of all, obviously, in the United States, there must be 17 or more agencies charged with cybersecurity. And it is distributed and often fragmented as well. But in smaller countries, there is more of an argument for more centralized for some agency taking a lead. And just two issues with that.

      One, when -- when our researchers would bring stakeholders together from across the country, across the -- they would remark to the researchers -- this is the first time we've been in the same room together. And this -- we haven't been talking. And so that shouldn't happen. I mean, this should be -- this should be -- there should be interaction and discussion across these various fragmented or distributed agents.

      And the other is, I don't think -- is there -- I guess it's a question. Is there necessarily a conflict between enrolling government, enrolling people across government and citizen s with a multi- stake holder model? That is the multi- stake holder model. That -- in other words, you can't -- if you only have the technical community focused on cybersecurity, you don't have the resources, you don't have the buy-in from government. You don't have citizens thinking that they have shared some of the responsibility for this.

      So, the true multi- stake holder model is really a distributed agency. But you have to -- it's often the case that it's not just government control of the process. It's government enrolling government in the process. And you can't enroll government in the process without giving --off know, well, anyway. Am I off base?

     >> I agree. You don't want chaos to come out. You said that was the first time they got together. That's trust building, you have to have the situations. You look at how the can coalitions responded over the years, the various incidents, whether it's -- you know, the whole history of global incident response is about people who form and norm quickly to build trust and respond at a -- at least at a technical level. Quickly to incidents. And so it's without that face-to-face, that trust building capacity. Because at the end of the day, no cybersecurity response, capacity, is going to work in isolation. It's going be connected and you have to have those relationships even if it's somewhat centralized.

     >> You want to speak on this?

     >> Yes, yes. Maybe to add to this. It's not primarily not about having the conversation. It's like really working together, participation, public-private, then you build trust and then you can see it change, like the netherlands, where we come from, 2010-2011. It's like, okay, regulation. It's not like in cyberspace, the government shouldn't -- shouldn't start the regulation, the private sector can do it themselves. Now when the -- when theneteverlands moved on and we had the multi- stake holder approach with participation, with all kinds of projects, building, capacity building together, then also the conversation said what are the rules and responsibilities came up in more depth. And now it's -- I think, in a lot of countries, it's acknowledged that -- that government has a role in regulation, how and how far. That's the second question. They have a responsibility in that.

     >> Thank you. As usual, the lively interesting discussion starts when time is up, unfortunately, but what I would like now, the speakers to really focus on a few minutes which are left is on the way forward. What are your suggestions? What shall we be doing on -- to fiber and maybe to also have the collaborative approach in a fragmentation, or what is it that you're suggesting in general to this approach if you -- if you want to is if you would like to end up with the panel with that suggestion? Who would like to start?

     >> I think one of the things that came out of our discussions of the conference and so forth is to highlight best practice in a variety of areas. That's something almost everyone agrees on. And we can even argue over what is the best practice. But if we can have some case studies of best practice in regional and global collaboration, data sharing, for example, and in international capacity building, that could be a positive role, I think.

     >> Anybody else who would like to go next?

     >> Yes. I think it's important that we talk about the money spent on cybercapacity building is well-spent money, I think that's very important.

     >> I think from our perspective, it's -- I think thinking about multi- stake holder and capacity building more wholistically, we usually think of it as just involving civil society. It's a little more. We want civil society, international players, private sector, everyone around the table, because I think if we only isolate it to one stake holder grouping, we miss out on how large the conversation could be and how much you can leverage on what everyone has to do to improve capacity building.

     >> Well, I just wanted to say -- we're talking about a while ago. But how government shouldn't be the only one that's -- or shouldn't be the only one talking about this. It could be that government is the one that puts everyone together. Gives up the stage or having a discussion. Needs to consider everyone on the table. And I wanted to say civil society, private sector, all of the governmental institutions. The original like limitations, and international experts, people from other countries that could provide some additional knowledge experiences, best practices. And I want to listen to all of that. So it's a very detailed work that needs to be done. And it doesn't take just a month or -- we need to work and we need to do it appropriately. Having everyone on the table.

     >> I'll just agree with my colleagues in closing. I think having governments take next steps in acting as conveners for a really robust multi- stake holder process for thinking through how to really help organizations at the national level and individuals of the national level understand cybersecurity risks and really build effective cybersecurity risk management programs is really important. And then beyond that thinking about some of these regional forums for working on these issues, like OAS, like GFC, I think, are really important for sharing some of those best practices across borders and really having that exchange between government that you talked about. I really like the way you framed it in terms of you took information in, you shared information back out. That should be an ongoing process of continuous improvement.

      The other issue that was brought up earlier is thinking about how to elevate this issue. There's lots of work that needs to happen across all of the stakeholders at the national level really engaging political leaders is also valuable to get the buy-in and the investment in this issue. So I think of other forums which in the next year we can work on this issue, the G-20 being one, thinking about how in the G-20 we can promote effective, interoperable approaches to cyber security risk management across the G-20 governments would be really valuable and other kind of global or regional forums. I think there are lots of places to increase our awareness of these issues and that would be a really valuable next step and really promotes some of the best practices that may be relevant in different circumstances across the board. So thank you.

     >> Bill, you wanted to -- no, sorry.

     >> Part of what we were discussing up here is we have another 25 minutes, I think, so we -- of course, we'll use the time. But kind of a -- to your quest for kind of an observation, kind of a closing. The technical community has provided to cybersecurity through their agency. The technical committee -- the resource committee has failed to provide easy cyber security. If cyber security was effective, if things weren't buggy, we probably wouldn't be here. Part of my vision is that in 100 years, hopefully sooner, we won't have these conversations because we'll have a better understanding of how to incorporate resilience and security.

      So, in the meantime, we need that multi- stake holder approach. We need the engagement. We need the technical community to be humble to recognize they aren't delivering what society now wants, what is resilience and security. I think that's really where that being part of the risk conversation, being part of the multi-stakeholder process is really important. But hopefully in 100 years, we won't be having the same conversation. 

     >> I just -- I totally agree with that. And, yeah, we're sort of beyond the point of fear campaigns and blaming users for cybersecurity problems. It's not all the technical -- but, yes, we need -- we need approaches to security that users can employ reasonably. It's been a priority for us at the cybersecurity center. Which is as the turn comes and people recognize this centrality of security and -- and are pouring more resources into capacity building, of government resources, public resources to capacity building around the world and regionally, it's become more and more imperative that we show it matters, that it works. We need evidence. We need -- and so one of the things that we're doing is -- is trying to work with the field research that we're doing and now 60 some countries and they're aware there's a capacity building model that's being employed by our center. And where we have data about exactly what's going on in terms of capacity building. We're trying to build a -- data sets that will provide evidence-based support that capacity building actually pays off. That it matters.

      And this is -- not any -- you think this is common sense. You pour money into capacity building. You put money in. It works. It will work. But public leaders are -- are obviously and rightfully questioning is this pain -- so we need to build evidence-based support for the -- for the impact of capacity building. And we're working on that. We're developing that. I think that will be something we'll talk about later this afternoon in another session as well.

     >> Carolin Weisser: Since we have other time than what I thought, if there are other questions from the floor, it would be great to comment or to -- yeah? 

     >> BARBARA WANNER: Thank you, I'm Barbara Wanner, I'm one of the co-organizers with Carolin. In multilateral conversations, there have been some countries who preferred or advocated for bringing the creation of cybersecurity frameworks under a governmental entity. What.

      What I'm hearing from all of you thereupon is that you feel that that is an overarching governmental entity. That this sort of top-down approach is not really optimal in terms of enabling a flexfable response. Did I understand you correctly? Thank you.

     >> CAROLIN WEISSER: Anyone who would like to react?

     >> I'm going to jump on -- it's quite the opposite. The conclusion should be. It's not that it should be a government entity and top down. We found that the government has to be outward looking as well. Ultimately the responsibility as the question we had here, the government is responsible for safety and security of its citizen. It's also responsible for economic development and growth and prosperity of its citizens. In doing that, I think cybersecurity is one of the few topics that governments are recognizing they haven't. They're recognizing they don't go alone. So cybersecurity is one of the few topics that have brought together all of the players around the table to make sure we get it right.

      So I would disagree with the conclusion you have and say definitively it's now not top down, but it's more pluralistic, it's whole of government, whole of a nation now coming together. We usually have the term separate, but now there's a need now to merge both to one under cybersecurity.

     >> I want to agree that it really -- you know, given the impact on elections, on public safety, you know, and the fact that you can exploit platforms in new ways that, again, engineers hasn't anticipated. And users are taking platforms in directions that weren't really predictable. I think that we agree that the whole society involvement is important to me is affecting so many different aspects.

     >> I think it characterizes a mix of those approaches, right? You have the top down in the sense of government acting as a convener to bring the conversations together and having the responsibility to do so, as well as the bottom up in the sense that a lot of the recognition that's what's effective and the kind of technical capacity and the best practices in cybersecurity risk management coming from the community. It's a mix of the purchase, I'd say. It's important in the multi- stake holder process.

     >> CAROLIN WEISSER: Yes, please?

     >> Well, one of the things that said -- one of the objectives that's mentioned in the policies that we have is the development of the -- (indiscernible) of the framework. It's not -- I'll give you an example. We came out with the -- we had a decision to do this regarding the standards that need to be applied by telecom operators in terms of what their networks need to have in terms of security. And it was based on ITU series X, I guess, accommodations.

      A lot of participation. But when we discussed that last year, year and a half, we figured out that most of the operators were working themselves on the 27,000ISO signers. And our relation did not -- was not -- did not agree. So what we did, we did a survey. Got that that I mentioned. A lot of them were working with ISO. So when you're working from the legal point of view or the literal point of view, you need to understand to go down and work with the different parties that need to comply with all of this -- these issues. To find out what they're doing. Because we need to understand what they have -- they probably have some developments they have done some things, they have things in place. And you cannot take for granted that what you say in your law or what you say in your relation is going to make you effective that's going to have in terms of cost and -- on solutions or the campaigns.

      So, my guess like with every other thing we've discussed today is that we need to work with the industry, we need to work with the ones that are going to comply with those laws to find what is the best way and how to start with the very complex framework. If you don't have the basics indicating what this is.

      We didn't -- we understood -- we found out that the companies, the telecom companies didn't have the policies based on -- based on the approach. What we start to do is start by the policy based on risk management and start by working on that. We had that for a couple of years. And we do the military in the next few years. And we're going to have to go with them, work with them in the next four years to implement step-by-step-by-step all of the different regulations and standards and is implemented.

      So, the discussion process that you have to work together with them.

     >> I'm sorry, go ahead.

>> I have a question for you. How much in the legal profession, in your experience, how multi-stakeholder oriented are our lawyers familiar with the multi-stakeholder processes.

     >> They are familiar.

>> Excellent. I'm happy to hear that. But I hear what you're saying, there's capacity building there also. In my experience, only as lawyers only as people who are digital natives who become more mature in their career, who become lawyers, that seems to be the answer, unfortunately.

     >> I'm of the minority opinion about these issues. But I think that the -- first of all, I don't think there's as many legal problems as other people do. And I think anything that is unlawful offline is also unlawful on-line. It is not the wild west. Law applies to on-line behavior. And bespoke laws for on-line behavior may be often driven by the success of them, there's activity or we must do some policy or some legal process.

      But generally, the central problem in the cybercrime area is coordination internationally. And -- and implementation of law in a more global sense. And that's why cybercrime is one of the areas in which there's a lot of -- that area is leading some of the efforts of coordination and data sharing and more international level.

      But I think if I can make a plea to legal and regulatory thinking, it would be that we have not yet come up with a regulatory model that works for the internet. Okay? Too many policy makers, politicians, lawmakers, are trying to apply frameworks built for other media to the internet. And it is not a broadcaster. It is not a common carrier. It's not the post office. It's -- the internet is a unique hybrid media. And if people import law and regulation based on a broadcast model to the internet, then they start spending time arresting somebody for tweeting an untoward comment, etc.

So I think the most highest priority has to be careful thinking about regulatory models that really work within the context of the internet. And social media. That are so different from traditional -- other traditional media. We can't just import regulation and models from traditional media to the internet. The question is agreeing with that.

     >> CAROLIN WEISSER: Any other questions from the audience? One thing I wanted to ask you because we were discussing the distribution within agencies. But among them also brought up the fact there are various initiatives at global level from different governments who are wrote out. So my career is like how can we avoid also the -- apart from the complexity but also the creation doubling standards. Not respecting international standards, is there a risk? How can we avoid that? Is there a way we can bring collaborative efforts too this? And if anyone wants to jump in on that?

     >> So, I'm not a lawyer. I'm not a policy maker. But I want too emphasize the notion of experimentation. The European model as well as the U.S. model allows entities to experiment with different regulatory frameworks before things are nationalized and globalized. And I think that's an important thing to remember here is that there's opportunities for experimentation in different regulatory frameworks to be tried out to see what works and what doesn't.

      As you scale up regulations, as the internet scales, there's always new surprises. We're going through that now with social media. The ability -- the flexibility to experiment I think is really crucial. It's part of the -- it's part of the process.

      Thank you.

     >> So I would jump in on -- you mentioned two different kinds of fragmentation at the national level and the development of some of the cybersecurity initiatives, you can have different government agencies having different perspectives and at the global level having different government s bringing different initiatives to the table, different approaches to the table. At the national level, my colleagues have talked a lot about making sure that there's a government convener that really has developed consistent principles that can be applied across government and really ensure the coordination across government.

      I think that same kind of concept can be applied at the international level. The process is quite different, of course. And you have regional and international organizations that hopefully help to facilitate that coordination at the international level.

      I think you mentioned the international standards, I think there is definitely something to be said for the experimentation that happens at the national level before the processes are or approaches are sufficiently mature to make their way to an international standards process and you need to give the ecosystem the time and space to go through the process and different approaches do mature. They move to the international standards context. It seems that that is really international standards are something that help to facilitate, interoperability and consistency on a global basis.

      And I think from my perspective, I think that's one of the inputs from the national perspective in developing approaches to something like cybersecurity risk management and thinking about both having those conversations across governments, leveraging international standards, and really from my perspective as well, recognizing the importance of interoperability and fragmentation or limiting fragmentation, that's an important mindset to really start from.

      So I mentioned one international. It's a report, ISOIC3703 to me is really relevant in the context. It really establishes a process that seems like it can be used consistently across both sectors and different national approaches. We need to recognize governments are different. They're not going to just cut and paste the international standard. They might really have good reasons for making some tweaks or some adaptions for their national context.

      But where there are international standards and they have been sort of sufficiently mature practices to move to space starting from that point and then making the kind of adaptions to the local context seems like an approach that really helps to deliver that fragmentation.

      Yes?

     >> I just want to -- I'm not sure about that. Because the -- it's very hard to experiment in this area. It's really dangerous. I think the -- I know we talked about international standards. I would say international implications of -- so, a judge in a -- in one nation, a judge can make -- have a ruling that has international implications. And the fact that a directive from the EU will not just be an EU directive, it will be something that has implications world-wide. Because of the global internet.

      So that that -- please don't experiment. Please think through these things and we need people who really understand the internet that are -- that are making these policies. And making these judgments, because they have international repercussions that are usually unintended, unexpected, and not good. Did you want to add?

     >> Well, it's just that I think you are perpetually in that state. There's no unicorn out there who -- you know, unicorn oracle that can tell you here's the implication of that policy. Because once you make it, again, the internet is going to continue to grow. I mean, we've got another, you know, 3.5 billion people to add, yet. You know, we've got the IOT tidal wave coming. And, so, it has to be flexible. And I think also we have to admit when we've made a mistake. I mean, in the U.S., we have -- you know, we were trying to control media copyright issues and had the unintended consequence of making it challenging to do cybersecurity research. Legitimately government funded cybersecurity research. It was an unintended consequence. It goes to the training, the policy makers at the time. But you'll always be in that state. I don't -- anyone who understands how the internet really works, they understand some part of it, but, you know, that's why we have these meetings here where we have the collective knowledge to inform the policy choices but ultimately, we've got to be able to admit when we're wrong. That's part of what the experiment is about. Figuring out, oh, that's not quite the right answer. We need change.

     >> We always encourage the multi-stakeholder model is wide because no one has the complete picture. So the constant dialogue is a part of our strategy development, we try to ensure that there is a built-in review process. And while we do training with our governments when they do strategies, we say to them, don't wait until the two years to review. Sometimes they'll review in six months because you thought a good policy position six months ago, the internet and technology has changed so much you have to change the strategy midway, maybe not redo the process but redo the focus of the policy. It's something we encourage that a multi-stakeholder should never be locked into one thinking. It's one format. It has to be all persons speaking all the time. Because everyone has a little bit of a puzzle. That's why we get it wrong sometimes as well. 

     >> Certainly, it's difficult also to calculate all of the unintended consequence with an internet and devices which are, you know, technology which are emerging every day and constantly.

      So, apart from being flexible, are there other advice that you would give to policy makers apart from certainly not experimenting and thinking broadly and consulting technical people and all of the stakeholders? Is there anything else apart from flexibility that you would like to build in a -- in a policy? Any -- any other suggestions?

     >> I'll jump in and say the experimentation is the wrong word. I think there's a recognition that there's going to be some efforts in the national space and I think from this in the national space, it should be ongoing and a continuous process of improvement. That's another prince Pell to think about flexibility. There's recognition that continues, improvement is important as well.

      I think one way to bridge the thinking of this topic is to say that that's why from our perspective having a multi- stake holder approach to developing any of the policies is important because hopefully you are eliminating some of the challenges and the continuous improvement processes really ensuring that there's a way to go back and fix mistakes and think about implications. I -- I think -- I wouldn't necessarily say that the -- that recognizing the word experimentation is not the best word. But it doesn't create the necessity of thinking about international standards? I think what I was trying to recognize is that there could be a national process for developing and thinking about policy and then looking to the international space. And then thinking about how -- the importance of interoperability globally and looking for mechanisms to help enable consistency across different national bridges can help to pull in that fragmentation and international standards is one of the ways perhaps of doing so.

     >> Yes, you can -- yes?

     >> I mean, I recognize that it's important to recognize multiple best practices that will work. But on the other hand, in cyber capacity building, it also helps if there are some best practices that globally recognize that are scalable, because it's an efficient way to get cyber capacity done. Both ways.

     >> You ask for advice for the policy makers. I think the one piece of advice I offer is for them to recognize how agile the internet ecosystem is. You know? If they make a decision today, by tomorrow morning, industry can be responding with changes to their infrastructure, to their policies given the way that they have agile development and continuous delivery, continuous integration of their products. I mean, you know, how many of you on your phone this morning had an update to your app. That's the pace in which the internet is changing. There's no pause. Six months is a very long time. You know, new product companies where you want to see a product in -- you know, the first -- the minimal viable product within two weeks after they put the first money in. So, you know, the pace is something that policy makers really need to appreciate.

      And, you know, in security, it's a -- it's just as big a challenge. And I think your comment about efficient methodologies, that's what industry is figuring out, how to incorporate security officially within methodology so we can deliver the products at this pace with security in it.

     >> And then I think another point, well, I would make is to beg policy makers to slow down on occasions because there are a variety of areas right now where policy is being made on -- out of -- in a state of panic. All right? Literal panic about filter bubbles, ecochambers, disinformation. And I mean certainly I've done research in seven countries. That's fairly robust and I think valid that suggests that this is really wildly exaggerated in terms of its impact.

      And while all these things make sense, you can say, oh, yeah, that's disinformation, that's happening. What's new about the internet having some misspelled words or wrong information or propaganda on it or any media having this? And the impact of this is really not as great as people believe given journalistic coverage and government panic over this.

      So, I would bet anything that we are going to have a raft of legal efforts that are going to be absolutely wrong in terms of content regulation and so forth, that are based upon this panel. And it's just going to be very difficult to claw back from that.

      So it's -- I don't know how we can -- it would be -- it seems silly to ask lawmakers to slow down, because you -- you can take too long to do anything. But, but in this kind of environment where everybody is wringing their hands over fake news or whatever, it's undoubtedly going to happen. And hopefully there'll be -- they'll look for evidence, real evidence of actual impact, not the fact that something exists, but that -- that users are -- are seriously affected by any of these things before they make policy that would be against such principles as privacy and freedom of expression and the things that we really have in the global internet.

     >> We are now coming to the end of the -- of the panel. The time is really up. But if you have one final remark, please don't hesitate to make it. But otherwise I heard very interesting suggestions. But I think it's really a buildup to the way forward, in particular, in sharing best practices in collaborating much more to experiment but not too much and educate to everyone and creating a multi-stakeholder approach to involve the whole of the government, the nation, the citizens as well. And everybody should be speaking and advising how to best approach cybersecurity. So, unless there are final burning comments, I would then thank you very much for this very interesting discussion and thank you, everybody, for being here and contributing to the debate.