Welcome to the United Nations | Department of Economic and Social Affairs

The following are the outputs of the real-time captioning taken during the Tenth Annual Meeting of the Internet Governance Forum (IGF) in João Pessoa, Brazil, from 10 to 13 November 2015. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid to understanding the proceedings at the event, but should not be treated as an authoritative record. 

 

***

 

>> NICOLAS SEIDLER:  Hi, hi everybody.

Good afternoon.

My name is Nicolas Seidler, I'm a policy adviser at the Internet Society.

Welcome to this panel organized by the Internet Society, the OECD and the worldwide forum.

So today we're going to talk about managing security risks.

While taking a step back, when you think about the perspective of an increasingly connected world, there's a lot to be excited about.  Right?  Who doesn't dream of having his fridge connected to the internet and to order milk when your supplies gets low.  Stuff of dreams, right?  And there's not much risk involved if your fridge gets hacked.

Now, take another example.  Think about your future connected car, self‑driving car.  Great to think you will be able to read your favorite book without paying attention to the traffic, but what if your car actually gets hacked and cannot brake anymore?  And actually that's not a Sci‑Fi scenario.  That's something that has happened a few weeks ago.  Some people tried to openly hack into a connected car and managed to disable the braking system.

So that sort of gives a sense of the type of physical damage that could happen when you think about digital security risks.

While the trend is really going upwards for large cyber-attacks, you might remember in 2014 there was a massive theft of critical information from major Korean banks.  You might remember as well the disclosure of e‑mails and personal data following the intrusion in Sony Pictures and Entertainment network.

Finally one of the examples that really made the news a few months ago was the theft of data from over 20 million civil servants following the intrusion in the U.S. of the Office of Personal Management Information Systems, and Mike will talk about it.

>> MIKE:  I was one of them.

>> NICOLAS SEIDLER:  Mike was a victim of it.

It's really hard to measure the financial effect of cyber-attacks.  Some have tried Atlantic council and Zurich securities group has actually estimated the effect of cyber-attacks on the internet to cast the world $30 million by 2015.

Obviously numbers to take with a grain of salt but gives a sense of the scale of those cyber-attacks.

So the goal of these examples is not really to scare you, maybe a little.  But it's really about realizing that security risk is much more than an abstract concept that's old relevant for technical and security experts.  Actually as data is getting across all types of industries, managing security risks becomes a concern of a wide range of stakeholders.

With this panel here we're going to try to change our thinking on security and explore what, you know, a modern approach to security risk could look like.

So let me introduce our distinguished panel.

On my level, Mike Nelson, working on global internet policy issues at Cloudflare, Cloud service provider.

Next to Mike is Laura Berner, cyber security and privacy expert.  She has been leading work related to renewal of security guidelines, which he will talk about.

Next Laurent, sorry, next to Laurent we have Lucy desedera, she's a security analyst at the Brazilian national cert, and she's working on the CGI.BRS training and security awareness team.

Next to Lucy we have Flavia.  Flavia Lefreve is a civil society representative at CIG.BR and a lawyer and representative of civil society in the range of multi stakeholders in Brazil.

Finally, remotely panelist, Aaron Martin.  Aaron is a technology expert specializing in cyber security privacy and digital identity at JP Morgan chase, so we have also a voice that's coming from sort of non‑ICT industry but the banking sector and it's going to be really interesting to hear his views on security risks in that context.

So I'd like to start by maybe asking each of the panelists basically their views to basically highlight what they see as key challenges for managing cyber security risks and how do you propose addressing those issues.

I thought that we could maybe start with Laurent.

>> Laurent:  Good afternoon, everybody.

I work for the OECD secretariat and would like to take this opportunity to briefly introduce our latest recommendation on digital security risk management for economic and social prosperity.  In so doing I will respond to your question, Nicolas.

This recommendation took two and a half years to develop, and it is the third generation of an OECD recommendation in this area.  We started long time ago in pre‑Internet age, 1992, having what we called at the time the security guidelines.  And we have revised them in 2002 and now in 2015.

It was a long process.  One‑year consultation, multi stakeholder consultation of experts and one‑year drafting of this legal instrument but nonbinding.

Over 20 years following the issue from an economic and social perspective, the OECD has not addressed and does not generally address issues related to national security, law enforcement, intelligence, international stability.  We focus on the economic and social prosperity angle.

We are not also a technical organization.  We don't do technical standards, but we want to and need to understand what's happening at the technical level to help government develop better policies.  Our slogan is better policies for Bev lives.

So as I said, over 20 years we have seen two major recent shifts in this area.  The message of this recommendation is actually we have to messages.

The first one is to shift the mindset from a security approach to a security risk management approach.  What does that mean?  Sounds nice but what does it mean?  It's a little more complicated than it sounds.

It means the digital environment, the internet, this ICT digital world is no different from other environments.  It is impossible to create a safe and secure digital environment.  There is always some level of risk with which you have to deal.

If you think about it, there's no environment in the world which is a hundred percent secure.  Why would it be different in the digital environment?

However, having said that, it is possible, and it is needed if you want to use that environment to carry out economic and social activities, to reduce the risk to an acceptable level in light of the economic and social activity that you want to carry out.

And this is not fundamentally new.  ICT security experts are thinking in these terms and have been thinking in these terms for over ten years.  It was actually already in our 2002 recommendation.  But it was not well understood because the terminology in particular isn't clear and difficult to understand.

So the new recommendation clarifies this, makes it much more straightforward, tries to help people understand the difference, that there is no difference between the off line world and digital world when it comes to security.

That's the first message, but there is another one, which is rather new.  This one is rather new.

Security is generally seen as cyber security, to use that word.  The recommendation does not use that term.  Cyber security is not just a technical issue.  It is an economic and social issue.  Of course it is also a technical issue.  But from an OECD perspective and from the perspective of whoever is trying to achieve something using this environment, it is an economic and social issue.

What does that mean?  It means that if you want to reduce the risk to an acceptable level, you should first focus on the economic activity that is using the environment rather than on the infrastructure.  It means that what you really want to focus on is what should I do for my activity to be really carried out successfully.

And the matters that relate to technical aspects of security flow from that.  They are not at the origin.  It also means that you don't want to manage digital risk for security, for that objective of achieving security, but always keep in mind that you are carrying out an economic and social activity.

It also means from the perspective of a business or an organization that you should integrate risk management, security risk management, into the economic decision of carrying out an activity that uses digital technologies.  You are not saying, oh, well, I'm going to use this wonderful environment to get some benefits and some economic opportunities, and I will deal with problems later.  You decide to use the technology.  You have decided how you are going to use it to get the benefits immediately with that decision.  You also decide to manage the security risks, which amounts to saying that security risk management is an economic and social issue.

I will stop there, just making two other points.  I mean, we have seen very concretely in the current landscape that this is an economic and social issue.  Just a number of the examples that Nicolas has just mentioned, thinking about Sony pictures, thinking about Target stores, thinking about the office of personnel management breaches.  We have had the consequences as we know but also that the highest level of these organizations was just fired, which shows it's not just a technical issue.  These people were not there to manage the infrastructure.  They were there to carry out objectives, achieve a mission.  They could not achieve the mission.  They were fired.

The recommendation that will be my last point includes eight principles that help approach this issue and develop policies in organizations as well as in government and for public policies that are grounded in risk management rather than on rigid security, as well as a section with over 30 recommendations for the development of national strategies that are in the sense period.

I will stop there.

>> NICOLAS SEIDLER:  Thanks, Laurent, I think you raised many points we will get back to.  The notion of risk, security not as an absolute goal but risk management.  Economic and social development as what security should put at.

I think we will get back to that.

I'd like to give the floor to Lucy to talk about her perspective.

>> Lucy:  Good afternoon, everyone, thank you for being here.

My name is Lucy Mott, I work for cert.BR, the national cert in Brazil.

Reading the OECD recommendation, I grabbed some points that it's mainly our daily work, and some aspects that I think is really important and that need great attention.

So one of the principles that for me is really really really important and we need a narrative with highest priority is the principle of awareness, skill and empowerment.

I work in the area of security awareness and cert.BR, and with the things that it's basically a consequence of lack of knowledge.

So having people prepared for digital security, for cyber security, it's paramount.

If we think of users and professionals, these people need to understand what digital risk is and how to manage it.  So the end user and the professionals that work daily and nightly in the systems and development of systems, they need to be prepared to understand and to manage risks.  Otherwise we will continue having threats and vulnerabilities growing up and growing up.  And if we think of an internet of things where everything in our life is connected to the Internet, if we don't have a prepared people to develop these systems and this hardware and software, we will continually have vulnerabilities and even more security issues to deal.  And until what point we can deal with that.

And this is also a point that we need also to start thinking of public policies, changing the public policies, particularly in the area of reeducation.

Why?  Because it's time to change the school curriculum.  Since the primary school to the high school to the university.  Because if we don't start teaching people about digital security, we will not have people prepared for the digital life in the future.

And so time for government to start thinking of the education of government policies to a better education system.

Another point that I think is important comes also from the recommendations of the corporations.

As I told you I work in cert.BR.

We see every day we cannot solve problems alone.  We cannot solve problems in isolation.  We always depend on someone else.

Doesn't matter if it is taking down of a web page, a phishing web page or international crime investigation, you always have to involve and cooperate with third parties.

So we need to start cooperate, not start but develop cooperation and trust as we discuss a lot this morning.

One last point that I would like to bring and highlight, it's preparedness and continuity.  It doesn't matter how much we manage risk, how much we try to avoid it, try to bring it into a level of acceptance.  We will always have an instance.  It will be a big incident with a lot of impact, or it can be lower.  Depends on how fast you can detect, treat, and respond to that.

So having the capability of security incident response, it's paramount also to reduce the risk, to reduce the impact, and operate and also in the whole economy.

And that's what I have to say, thank you.

>> NICOLAS SEIDLER:  Thank you, very much.

I think that the notions of preparedness and resilience resonate very well with the notions that you manage a risk and that you don't try to have full security.

Interesting points.  Share responsibility, collaboration to address security, I think that we can get back to that later as well.

Now I'd like to ask Flavia to share her perspective from a civil society angle.

>> FLAVIA LEFREVE:  Thank you for the invitation.  Good afternoon, everyone. I'm one of four representatives of civil society and CGI.br.

We know that the internet is an environment that promotes the social, economic and cultural development.  However, we also know there are many issues on cyber security which challenge governments and enterprises to ensure trust and quality standards on Internet.

Civil society and consumers are also challenged because it is commonly argued that in the name of cyber security, surveillance could be justified.

Our defense instruments on this issue are establishing minimum security parameters in regard to data and limits for governments and companies can use personal data.

We need data protection policies which must be associated with security frameworks, enforcement by public agencies, in order to ensure that they can penalize companies and governments that break these rules.

In Brazil we face difficulties especially with regard to three aspects.  First of all, we don't have today a specific law for data protection.  The process for the addition of law has lasted more than five years and the perspective in that the debate in Congress lasting at least one year in order to reach the publication of the law.

Also, the microview enacted in April 2014 established general principles for data protection such as informed consent and limits on access and use of data for the administrative and policy authorities.

The effect is that critical issues such as data transfer between public and private agents and vice versa and international transfer are not regulated.

Moreover, a draft view on this matter did not set about creating a body with power to oversight and impose penalties for public and private actors that breaks the personal data protection rules and safety standards.

Even in countries where there are already rules regulating extensively in the personal data protection, the supervision and penalties are problematic, especially in view of obstacles resulting from jurisdiction and surveillance of the countries.

Finally, sustainability in the field of internet is associated especially regarding the social and economic rights.  Social rights like human rights, freedom of expression and political freedom, privacy, free flow of information, security and diversity, and economic, labor laws, competitive laws, environmental laws, and consumer rights.

Thank you.

>> NICOLAS SEIDLER:  Thank you very much.

I think that, yeah, you raised a few interesting points.  One is related to the relation between companies and users, obviously, more and more ICT actors deal with vast amounts of data, so by default they have responsibility.  And also the notion security should not be at odds with human rights.

I'd like to try to ask our remote panelist if we can connect him to share his views, Aaron Martin.

It's good?  Aaron, can you hear us?

Can you hear us, Aaron?

>> AARON MARTIN:  I can hear you.

>> NICOLAS SEIDLER:  Great, great.

Aaron, we were having a first round on the panel to basically ask panelists to share their perspectives on what they see as the key cyber security challenges related to managing risk.

Could you share your views with us?  Thanks.

>> AARON MARTIN:  With pleasure.

Again, sorry I couldn't be there.  I'm stuck in New York where it's quite cold and gray today.

I would much prefer to be in Brazil with you.

Thanks for organizing the events, and thanks for the invitation to participate.

I'm Aaron Martin from JP Morgan Chase's global security participate ships and global strategy team.

Today I want to stress I'm speaking in a personal capacity.

I'd like to sort of touch on three points in my opening remarks.

First is the importance of cyber security for the financial sector.  The second is the importance of collaboration across the sector and beyond.  And the third is the need for increased policy coordination in the area of cyber security.

On the first point, cyber security has become a top priority for the financial industry.  And it's a priority because we have to ensure the security of sensitive information and customer and client data, as well as official reliable operations given the high risks of attack.

Cyber security has become a board room issue across the sector and will remain so for some time.

Second, the financial services sector recognizes that in order to protect itself, collaboration is essential.

The sector doesn't view the defense of critical infrastructure as a competitive issue.  And I think this is actually very important.  It's understood that team work is required to get cyber security right, and in particular the FSI sec, which is the Financial Services Information sharing and analysis center, is the primary means for industry collaboration.

I'm happy to share more about how the sector collaborates during the discussion.

My third point in the opening remarks is about policy approaches in the area of cyber security.

As regulatory approaches and policy frameworks for cyber security proliferate, there's a potential for diffusion of efforts to synchronize the sector's approach to doing security.

While the industry is pushing for increased policy coordination, both at the national and international levels, in order to better harmonize and better coordinate, which would allow the sector to focus on prioritizing efforts its efforts in cyber security and uplifting its cyber defenses across the sector.

Those are my opening remarks, and I'm happy to engage in the discussion later today.

>> NICOLAS SEIDLER:  Thank you, Aaron.

Again, it's great to have a view on this panel that also comes from an industry which focuses not ICTs but has integrated ICTs into the activities.

Last but not least, Mike, could you share your views with us.

>> MIKE:  Thank you very much for organizing this event.  I really think this is an important event because we do bring together different perspectives, sort of a multistakeholder view.

I'm here for a couple of reasons.  One is that I have been working on internet issues since before we had a commercial internet.  I was working at the White House in the Clinton administration, and I have to say that back then we didn't have quite as many worries about cyber security and were not nearly as many panels and conferences on the topic, though we were thinking about it even back then.

The other reason I'm here is, obviously, because Cloudflare is a web security company.  I have been working there since the start of the year, working on global public policy, and security is one of the most important issues we're working on.

I thought what I'd do in just a few minutes is point to a few things we haven't really touched on, and talk to some of the misperceptions around cyber security.

Because of the news media reports about confidential and private information being stolen, much of the focus in our discussion is about that.  It's about hackers getting at the data and stealing it.  But there are other things going on that we have to pay attention to as well.

One is the possibility of malicious hackers getting into databases and altering them.  In some ways that could be as serious or more serious than taking the data and publicly disclosing it.  Imagine if hospital records are altered.  Recently we had the president of Estonia come and visit us at Cloud fare and he pointed out the fact he has an A positive blood type is not really sensitive information, but if he went to the hospital and somebody had changed that so that he was now O negative, that could have deadly consequences.

Likewise, if somebody was to get into the customer information at JP Morgan or some other bank or a major stock market and start altering the data, that could lead to a huge loss of confidence in our banking institutions and in our markets.

So we have to pay attention to the integrity of data, not just the confidentiality of data.

And the other thing I want to point out is that in the discussion about cyber security, we need to focus a lot on reliability and how malicious attacks could undermine services, knock them off‑line.  Data might not be compromised, but if you knock off major e commerce site at the height of the Christmas season, that could amount to millions of dollars an hour.

Our company, Cloudflare, is in the business of protecting four million websites from distributed denial of service attacks.

We do that for free with our basic service, and it's really important to small businesses who are often subject to attacks.  This is the new protection racket, sort of cyber extortion.  And you'll have small organized gangs attacking 50 or 100 small websites every day, just small businesses, florist, craft shop, bookstore, and they knock them off line just by sending millions of messages using zombie nets and bot nets.

Our service protects them for free, and they don't have to end up paying these extortionists 50 or $100 a month for a promise that they won't do it again.

This is happening.  It's a growth business, unfortunately.

So let's make sure we don't just focus on stolen data, stolen information.

The other thing is let's focus a lot on the word sustainable.  Too often security solutions are just too expensive.  They add too much burden.  And they don't keep up with the technological threats that the malicious hackers are deploying.  So I think we always have to be understanding that the simple answer, which is let's make some standards, is not sufficient and often is counterproductive if we're telling people to deploy systems and use technologies that are five or ten years out of date.

So in the later discussion I hope we'll get into some of the policy recommendations.  It's an area I have spent a lot of time on.  Prior to joining Cloudflare I was at Microsoft and actually spent a whole day teaching a class on what governments can do to create national cyber security solutions and to foster an environment that will foster better cyber security practices.

So I look forward to a further discussion, particularly delving into the question of what is to go done.

Thanks.

>> NICOLAS SEIDLER:  Thanks, Mike.  I think that we got a lot of interesting elements in the first round that we are going to get back to.

I'm going to follow up now with another question, but I would like you in the audience to think if you have any questions.  We'll differently take them from the audience.

Mike, I think it's going to be very interesting to get back to the notion of SMEs, which also deal with data as well as being victims of attacks.

First one I'd like to get back to, I think a few of you mentioned that security should not be an end in itself but rather a means to protect economic and social development or human rights.

So I would just like to get a bit further on that.  From your perspective, what does cyber security mean in this broader economic and social context?  And follow‑up question, if the objective of cyber security is to promote economic and social development, how does that affect how policies and strategies are being implemented?  Do you think a change is needed in policies?

Anybody want to jump in?

Mike.

>> MIKE:  I'll start with that.  Laurent mentioned this colossal number of $90 billion this one in possible damage.  Oh, you did.  I'm sorry.

I wanted to say that's probably an underestimate if you count the damage done by all the excited applications that won't be deployed because people don't trust the system.

Not only are we looking at the direct damages, we have to think about the indirect damage that might be done because people aren't buying the service because they don't trust it.  It's very hard to measure trust.  It's very hard to assess the damage done because of underinvestment and lack of market, but that could be an equally huge number and do a lot of damage to the global economy.

One thing that I've learned over the years, though, that we have to be doing more of is fostering information sharing, mostly between companies.

Aaron can talk at length about this, as can many of my colleagues at Cloudflare.

We're very involved in managing the traffic that goes over the web.  We actually handle about five percent of all the world's web requests every day.  And in a year or two it will probably be ten percent.  So we see the effects of a new virus.  We have to react very quickly if it turns out that some key component of the infrastructure is being attacked by a new method like the heart bleed virus, which we helped defeat working with Google and a couple other companies.

So if we can spend some time talking about information sharing, and particularly company to company and company to customer infrastructure sharing, think we can develop some interesting insights.

In Washington DC, the U.S. Congress just passed a Cyber Security Information Act, CSIA, and they are making the final revisions on the bill before sending to the website and President Obama.

Unfortunately, that focuses entirely on companies communicating to government.  That's important, but the real action and the really rapid action is happening when companies talk to companies.  And so we should look at how that can help.

One thing that is standing in the way of information sharing between companies and between cyber security experts is a recently concluded treaty called the basanar arrangement that was designed to protect dissidents and political opponents in countries with dictators and authoritarian governments.  The concern was that these people were getting snooping tools and learning how to hack into e‑mail systems and mobile phones and using that technology to find dissidents and political opponents and arrest them or worse.

So this treaty was developed to block the export of these tools by companies that develop them.

Unfortunately, was incredibly broad and the result was they were not only stopping export to the bad people abusing these technologies, they were also blocking the collaboration needed by the cyber security experts to find vulnerabilities and fix those vulnerabilities.

For every bad guy who gets some of these snooper tools, there's probably 90 or 95 or 99 good guys who are using these tools to test systems, to find vulnerabilities and fix those vulnerabilities.

So this was a disaster, and it has exactly the opposite impact of the new cyber security information sharing act in the U.S., and probably much more negative impact than the positive impact of that bill.

.

The last point I wanted to make, I think another thing we need to focus is on the enabling environment.  When you say those two words in the IGF meeting usually you're talking about the enabling environment for investment in infrastructure development.  About you we also need an enabling environment for information sharing and for investment by companies and organizations in good cyber security.

The most important thing that we have done in this area in the last ten years to create an enabling environment in the United States is pass legislation on security breach notification.

If you are a big company and somehow your machines are compromised, a malicious hacker steals your data, you have to notify your customers.  That can cost millions of dollars, even tens of millions of dollars because of these state laws that require this disclosure.

That provides a huge motivation for companies to prevent those kinds of disclosures.

.

Not only is it embarrassing, it's also very expensive to handle the aftermath because of these regulatory requirements.

They didn't tell people what to do.  Didn't tell them how to protect the data.

They just told them there would be very large consequences if they lost their customers' data.  And that has really motivated a lot of improved cyber security practices.

Thanks very much, Nicolas.

>> NICOLAS SEIDLER:  Thanks, Mike.  I'd like to jump on one thing you mentioned, the importance of information sharing.

I would just like to ask Lucy, because when you mention the notion of information sharing must be central and do you have any best practices there to share?

>> Lucy:  Yeah, this is part of our daily basis work we have.  We share information and we build like a community that we share information, of course, that are basically principles that we need to keep that is the confidentiality of that information.  So when we establish some cooperation in information sharing, we don't, we almost, we never distribute this information for other purpose.

And basically we have cooperation with different entities from the market, from the government.  We also work in cooperation with United States government in some campaigns.  And one key point in information sharing I would like to tell again is trust.  And trust is only built when you do what you say you were going to do.

So basically if you say I will not share this information with someone that is outside, someone that doesn't have the right to see that information, you have to accomplish that.  And it's only for the purpose of, for example, doing some notification that will send anonymous information.  So before sending the information to somewhere, someone that has to deal with that information would sanitize what's not necessary to deal with that incident.

So here is another key point that is don't share the disclosure or don't grab what is not necessary.  It is also a principle for most companies who grab information from users.  Okay, since we are going into ask something, let's ask all the other information.  And that's a risk because since you have that information, that information is personal, and in case of a data break, you will have big consequences.

So key points.  Don't get more than you need.  Sanitize what you don't need to share.  And build trust.  Accomplish what you have done in regard to that.

>> NICOLAS SEIDLER:  Thanks, and it's funny you mention trust.  I actually remember being on another panel, and the person was actually emphasizing the importance of B to B networks.

>> (Chuckles).

>> NICOLAS SEIDLER:  The importance of personal relationship and trust between individuals.

Laurent, would you like to jump in on the link to social economics.

>> Lucy:  From the previous comments, trust is not established by degrees.  You establish among people and doing the right thing.

>> NICOLAS SEIDLER:  Makes sense.

Laurent.

.

>> Yes, thank you, Nicolas.

Jumping on the question, I think you used the term and it's actually the title of this workshop, sustainable development.  It's interesting to link, rather unusual to link cyber security and sustainable development, which for us means social and economic development.

I mention we don't using the term cyber security.  It would be interesting to think about that.

When I say we, it's OECD members and actually the stakeholders agreed, okay, we are going to give up on the media visibility that using this term brings us necessarily because it has become a buzz word, to use the actual term that should be used, talking about risk management.

We tried to avoid saying that cyber, or suggesting that cyber is somehow different than the rest of the economy.  It is exactly the same.  And using this term is bringing the wrong culture.  It sounds like you need someone who has some superpower or some super knowledge or super something to address a problem that is super special.

And if you go to a SME with that mindset, they may just have actually one, actually the CEO of the company might be the only one person who can take some time, a little bit of time to think about this, and the problem will not be solved.

You have to address this as an issue which is a real life issue, which is highly completely interrelated, inherent to the fact of using the digital environment to do something.

And security has the same problem.  If you using it alone with a prefix like cyber, it means there is a solution, you can be secure, it's possible to be secure.  That's not the message either.  You can never be secure, as I said earlier.

The notion of sustainable development is also very important here because, okay, you are going to have a product that integrates digital components or you are going to have a service or you are going to integrate some digital elements in the way your business works, and you will make some benefits with that.

If you want this to be sustainable, it means that it will, as I understand, it will work for a sufficient number of years or time or you are integrating something core to your business.  It should be sufficiently long‑standing.

Well, if you don't address the security dimension of the digital aspects you integrate to your business, it's just not sustainable.  I mean, this is very obvious when you think about it.

But if you don't see that part of the problem when you think about the opportunities that the digital components will bring to your business, if you just don't see that it also carries some security risks that can just harm your business, if you don't see that at the moment when you make the decision to use the digital component, it's not sustainable.  You are in the situation.

It's really a matter of culture and awareness, which is in our recommendation, the first principle.  Without awareness you don't achieve anything.  It's really a matter of skills of course as Lucy just said.  If you are aware there's an issue but don't know how to address, we are not making progress.

And I wanted also to not respond but build on what Flavia has said and on what Mike has said.  You need it enabling conditions.  As we see in the OECD, you need the framework conditions.  And they are not just about cyber security policies as such or digital security risk policies.  They are also about things like privacy protection because you cannot have one without the other.

Again it's not sustainable if you are going to use security technologies that end up harming privacy.  You have may have some benefits in the short‑term, but in the long run your company will be harmed from a reputational perspective, for example, or it will be caught not being in line with the law and having again a reputational and financial damage.

I think it's very important to address the whole issue from that perspective and think in these terms.  And then turn to the security experts, the I.T. experts, those who actually can better understand the technical dimension of this and work with them in cooperation.  Again the theme of cooperation is absolutely essential by all of this.  Nothing can be achieved by one player along.  That is pre‑internet thinking.  It's gone, has been gone 20 years now but takes a lot of time to realize it.  Thank you.

>> NICOLAS SEIDLER:  Thank you, Laurent.

Flavia, then going to Aaron, and I'd like to open the floor for a few questions.

>> FLAVIA LEFREVE:  Considering the dynamic of the market and the dangers in the internet, I believe that regulation is not (audio breaking up) but is an important tool to protect the cyber environment.  But regarding human rights required standards of safety and quality and build accountant system and investments in education, as said Lucy, and involvement in the public organizations as leading cooperation between all agents, public agents and private agents also.

>> NICOLAS SEIDLER:  Thank you very much.  Aaron, did you want to jump in on this one?

I guess you can hear me?  Marvels of the internet.

Okay, we can see you.

Can we hear you?

>> AARON MARTIN:  Yes, actually would like to respond to a couple of points that Mike made.  One in his opening remarks and the second in his response to your first question.

I think his point about the risks being sort of broader than just stolen data I think is a very, very important point.  I think increasingly, despite the headlines, despite what you read in the news, the sorts of cyber security risks that are of significant worry to our sector is to do with the loss of availability of systems, the sort of underreliability of systems.  And I think this is something that often gets underexplored because it's not as high profile in the news.

I think we should spend some time to discuss that at length.

The second point is with respect to information sharing.  So as I mentioned in my opening remarks, the financial services sector I think does a really good job of sharing among organizations within the sector.  I think there's actually a lot of room for improvement in terms of sharing outside the sector, both with government and also other sectors, other industry sectors, including sectors like Mike's and Cloudflare.

.

Also there's increasing recognition that attacks on exchanges internal to the bank may still have a big impact on financial institutions themselves.  So we won't be a recipient of attack but still impacted by the loss of availability of certain core utilities and exchanges.  I think those are issues that require a lot more attention from policy and decision makers within industry to address.

>> NICOLAS SEIDLER:  Go ahead, Mike.

>> MIKE:  Just wanted to add one more thing.  Another place where we tend to put too little attention is in data in transit.  Since all these hacker attacks and all the news reports are about data stolen from a database sitting somewhere, we tend not to spend as much time thinking about the security of our networks.

I'm very proud of the fact that last September Cloudflare announced that we would give away SSL encryption to every customer who wanted it.  And in three weeks we doubled the number of websites that support HTTPS, the websites that have the lock on them.

That was an example of where we did something to make it very easy and sustainable for anybody to just click a few buttons and have this service.  Because in order to make cyber security work, it has to be easy.

Yesterday we did something even more important to make sure that this internet infrastructure works effectively and reliably.  We announced that any of our customers, including our free customers, can have DNS sec, which is the next generation of domain name system, to make sure that the domain name you see is really associated with the person you think it's associated with.

This is a standard that's been developed over the years.  It's been available.  The Internet Engineering Task Force did a great job with it.  It just didn't roll out.  People were not making it available.  It wasn't easy enough.

And so Cloudflare and other companies are working together to make this possible and simple and free, and we're eager to talk to as many domain registrars as possible so that this can be available to anybody and everybody.

>> NICOLAS SEIDLER:  Thanks Mike.

Are there any questions from the audience to the panel?

Maurice, please.

Oh, you don't have a mic.

.

>> AUDIENCE:  Hi, Mohit, Internet Society ambassador.

I have a few comments and a question.

One comment, the inclusion of risk management in economic and social decision making.  When it comes to OECD, it kind of makes an impact, particularly for the governments, because then they take it seriously, and kind of cascade it down.

Also one important point, what has been raised, is the data sharing between company to company.

There has been immense efforts which company and organizations have been taking.  So it's important that the sharing happens.  So the security becomes inclusive.

These are the two points which I really want to comment.

But I want to come to the next point where I want to talk about information sharing between consumer and the organization.

So while this might have an impact on economy possibly because what happens is when there's access data shared between consumer to the organization, they can place their product better which kind of has a positive impact on economy.  But at the end it also affects their privacy and trust, which on a longer run might impact economy.

So what is the panel view on that?  And how can probably this be taken into longer run to see how this can be addressed?

>> NICOLAS SEIDLER:  Thanks a lot.  Yeah, that was very interesting, this idea of not only business to business info sharing but customer to businesses.  So how do you see that working in practice, and can it be done while protecting people's privacy?  I guess that's Mohit's point.

>> MIKE:  I'll just say I vehemently agree that transparency ask a huge part of the puzzle here.  I highlighted the security breach notification because that was a way in which we could learn more about the attacks that have happened and provide this incentive that would push companies to make sure that they don't have to pay all this money in case there is a breach.

Another type of transparency, and this is something that's starting to happen, is that companies are building in monitoring tools so that if there is some kind of attack on a system, that reports back.  So I think that's also going to be part of the long‑term solution here, is if we have, particularly if ISPs can install software so that every user of their service has an automatic tool that will wave a flag when there's an attack or a virus that's taken over the machine, that would really help.

I mean the distributed defense is what we need here.  It's a distributed network, so we need a distributed defense.

Great comments.

>> NICOLAS SEIDLER:  Thanks.  I'd like to get back.  We talked about, unless someone else wants to jump in on this one.

But we mentioned before SMEs, and I'd like to get your point of view on what are the challenges for businesses and especially SMEs to adopt a culture of managing security risks.

I mean, in the internet world it's often that, you know, permissionless innovation, you can start a business really quickly but might also very quickly get into the situation where you have to deal with vast amounts of data.

So how is that integrated?

>> MIKE:  One of the big challenges for small and medium size businesses is hiring the kind of talent that they need to manage their systems.

If you're a really hot‑shot cyber security wizard, you probably don't want to go work for some small business with 40 people or 80 people.  You're not going to learn from other people because you're the only person there working in that area.

And so these small companies often have a real challenge getting the technical talent they need, and that's why it's so helpful that we're now seeing a lot more cloud based security services.  Cloudflare is just one of them.  But it's so much more complicated to buy boxes, install software, make sure it's all properly configured.

If you can start routing your traffic through cloud services or using cloud services for your basic computing and data storage needs, the odds are those systems are going to be a lot more secure.

If you're Amazon or Microsoft or IBM, you can hire the very very best computer engineers and cyber security wizard s, and the services they provide are enabling millions of small ambitions all around the world.

So I think the development and deployment of the cloud is a really positive development when it comes to our overall cyber security, both in the developed countries and especially in the developing world.

>> NICOLAS SEIDLER:  Thanks.  Laurent.

>> Yes, the question of SMEs is a very important one.  And during the whole process for developing the recommendation, it came up several times as well as the question of individuals.

But just how can individuals manage risk.  We tend to think about risk management as something extremely complicated, and it can be extremely complicated.  And ISO standards, and you can have the issue, want to have the full‑fledged risk management, yes, it can be a real big machine.  But we also manage risk each time we cross the street, each time we do anything we manage risk.  We just don't like break down each of the steps and have like a framework and document it, et cetera, but that's what we do.  We read and write, so we assess the risk and we take a decision, we treat the risk, we reduce it to the acceptable level but perhaps not crossing it there but going to the light, you know, where there are crossing lights.

That's the same approach.  Going to SMEs, they have I would say a significant, a severe awareness issue.  But pretty much the whole society has an awareness issue.  The awareness of the problem, the awareness issue is getting a little bit better because we see media talking more and more about all of this.  So people are aware, increasingly aware that there is an issue, but it doesn't solve the skills problem.

They don't know what to do, and here SMEs face a capability, capacity issue.  They don't have the resources as Mike said, and it's becoming a real public policy challenge.  Because Aaron mentioned, the SMEs become very much a part of the GDP and jobs in countries, so that's one aspect, but also a part of ecosystems broader than themselves.  When a multinational firm or a critical infrastructure sector has big actors, they often rely on SMEs, which can become the weakest point in the whole value chain, and that can be disastrous for also the big actors and the society at large.

So it's a public policy issue.  We started to work on this.  We did a study.  We going to organize a meeting on the digital economy in Cancun Mexico in 2016, in June 2016, in which there will be a session, a panel on trust and security and privacy, and SMEs is one of the angles that we want to address.

We have done a little study that is not yet out, and we interviewed a number of SMEs, a small number.  Starting to work on it.  What we see is they often think for example they are too small to be really a target.  These types of behaviors are really, really a problem.

We are thinking about looking at how governments can help because that's what the OECD is doing, trying to help governments.  There are all sorts of incentives that can be put in place.  There are areas where governments can help market to develop and thinking about, for example, the insurance market.  There are a lot of reflections on whether trying to remove obstacles to the development of insurance market can help not just covering risk and solving the problem by that, but fostering better risk management practices in SMEs and in other organizations.

This is one example, but there are many others, so it's really a key issue.

Thank you.

>> NICOLAS SEIDLER:  Thanks, Laurent.

So that was interesting to have the SME perspective.  And building on that, I was wondering, do you think that if we think about bigger fishes, big actors in banking, trade, infrastructure, do you feel that they are integrating that notion of security risk management?  Or, you know, are we a long ways off?

Typically we were discussing earlier at the IGF you rarely see, again, banking actors or people from industry that are not ICT companies per se but have an ICT components to them.

I would love to have Aaron's perspective as well on this one.  But if you guys on the panel have a perspective on this, please.

>> MIKE:  I wanted to say, that's why we have to use the technology we're using for Aaron to get these people outside the narrow circle of internet governance specialists to be part of the discussion.

I'm a huge advocate for doing a lot more remote participants and a lot more remote sessions.

>> AARON MARTIN:  The comment Laurent made about SMEs and sort of how best to manage their cyber security risks, this is actually a very important initiative and one that we have taken up at JP Morgan Chase.

We call it third party oversights, and very often those third parties will be small and medium size enterprises.  And they are very quickly realizing that cyber security risks are real and that there are sort of best practices and model approaches that exist that could be adopted.

And what we're doing is working with them to ensure that they are not just aware of those practices and sort of model frameworks, but also that they adopt them, that they are sort of audited on a regular basis so that we aren't exposed to additional risk through those relationships.

Now, to the second point about the involvement of the financial services sector and organizations like multinational banks like JP Morgan Chase, in discussions like this I think this is something that the sector is increasingly aware of.  There's an organization called the Financial Services Sector Coordinating council, also known as the FSSC, which has a number of working groups which cover a range of different cyber security and sort of critical infrastructure topics.

And one of those groups is focused on internet governance issues as they relate to the financial services sector.

And that is largely around issues of DMS and new domain names, but increasingly we are broadening the scope to look at other policy issues such as how to work with organizations like ICAN and forum like the IGF to raise awareness of cyber-crime issues, the role of banks in sort of influencing policy around cyber-crime, and so forth.

I think these are great questions and there certainly needs to be a continued dialogue in terms of working with the internet governance bodies to increase participation from other parts of industry.

Thanks.

>> NICOLAS SEIDLER:  Thanks, Aaron.

I think a note to ourselves to try next year to have people from the automobile industry, public administration, et cetera.

Are there any other questions from the audience?  Please, I think the mic, yeah.

.

>> AUDIENCE:  Can you hear me?  Oh.

Okay, so what's the man's name who is virtual?

>> NICOLAS SEIDLER:  Aaron Martin.

>> AUDIENCE:  Aaron, I'd like your card after this, please, and we can exchange.

I'm Sally long.  I'm with the open group.  I have mentioned a couple times in other sessions.

We have do have standard for product and supply chain security and accreditation program that goes with it.

One of the questions, the members in this organization, Microsoft, Wawa, Cisco, Juniper, DOD, IBM.

And so they pooled their best practices and created a standard of that for cyber security and supply chain risk.  And actually the reason, one of the little known reasons that they got started was to avoid legislation, to avoid having government say you're going to do it this way.  Instead they got together and said umm, here's what's practical and here's what's achievable.  And they came out with this to avoid legislation.

I guess my question is, I do believe standards are part of sustainability for the internet, but you also mentioned policy as a way of sustainability.  And I know at least in the U.S. the big vendors really want to avoid legislation.  And in fact even in my forum, they have a program wherever a piece of legislation is on the edge, they kind of want to do a collective response to it or response to it not to be regulated.

So how do you get around that?

>> NICOLAS SEIDLER:  Thanks for that excellent question.

Panel?

>> MIKE:  That is the hardest question of all actually.

>> NICOLAS SEIDLER:  Flavia, Lucy and Mike.

>> FLAVIA LEFREVE:  I think I understand the concerns today about the power of the governments defining rules and making intrusion in the business development.  But I think it's necessary to construct minimum standard security models and frameworks to define terms of responsibility in a perspective of consumers' rights.  Because as Mike said, the investments in security are really big.

And I think the companies don't will be spontaneously the necessary investments.

Then I think it's a necessary definition even in most stakeholder regulatory process, but definition of minimum terms of responsibility of the companies to guarantee the security.

>> NICOLAS SEIDLER:  Thanks, Flavia.

Lucy.

>> I think another point of view, another way of dealing with the situation also it's the market regulation.  Have some examples where the market came in with possible solutions.  So we are trying now doing some cooperation together with the Federation of Industries in Sao Paolo because we have seen a lot of attacks come from security cameras.  So you have cameras that were supposed to be used to secure your home, your company, and in fact they are being used to attack your insides.

So there's a problem, a problem because this camera is not being produced with the good standards of security, they are not being sold with the good practice of security.

So let's start with the one who produced these cameras and by the Federation of Industries, we can reach a lot of producers of these cameras in the state that we are based on.

So this is a possibility that we could reach some better security without using the government regulation.  Especially because I think there are very very very few government officers that understand security and in point that can make good regulations and good laws.

So if sometimes it's good, yes, but it has to be at a minimum so that you don't undermine the whole business.  So this is a point.

>> NICOLAS SEIDLER:  Thanks.

Mike.

>> MIKE:  There's a very important paper that Jeff Houston with AP growth wrote necessary called the Internet of Stupid Things, really well done.  His point was a lot of companies that make consumer products, whether an alarm clock on a smoke detector or a thermostat, they don't come from a culture of cyber security.  So they weren't building in effective means to protect the data.  And it's really been strange that some of the worst cases of poor design have been for security devices.

One of the first fines that the Federal Trade Commission issued against a company for inadequate security was for a baby monitor.  This baby monitor was broadcasting pictures of your baby to all your neighbors.

Another one that just came to light was a system that police departments used for scanning license plates, mostly for parking tickets.  They would just drive up and down the street and the camera would take pictures of license plates on cars and send that data back to headquarters.  Unfortunately, they were not encrypting that.  They put it out on a Wi-Fi network broadcasting that anybody within a hundred feet.  Real privacy problem there.  Just because the people designing the system didn't think that was a risk.

The most appalling example, I think, was the drones flying over Afghanistan were sending the drone videos being used to monitor where the terrorists and the mujahadin were in the open, no encryption, no security.  The people that the U.S. Army was fighting were actually watching who the U.S. was watching.  That gives you a lot of advantage if you're trying to evade detection.

So there's a lot of need to make sure that not only in the computer industry but across the board we have more intelligence design and more thought, more auditing.  So I think we will not have a shortage of jobs for cyber security students who are getting their degrees now.

It's a hot job tip.

>> NICOLAS SEIDLER:  Listen to Mike or the young fellows in the room.

Laurent.

>> Yes, on the issue of regulation, I always react to this.  I agree with what was said, but the use of regulation I have noticed working on many countries, there is a cultural dimension to that as well.  Some countries are very keen on regulation just as a matter of culture.  That's the way they see the world.  Others are just, ah, it's really the last resort.

And business, what really strikes me is that businesses in the first category of countries actually when you ask them, well, don't you have feel that this regulation would be a problem for you, in some cases they agree, but in some others they say in this country if there is no regulation, nothing will ever happen.

So I'm just saying it because it goes to my second point.  So there is regulation of actually the ICT technology, but I think given that this technology is everywhere, across sectors, all stages of the value chain now, the regulation is going to follow the track of the sector regulation.  In other words, when we have driverless scanners, it's not an ICT problem, not a cyber security problem, it's a transportation issue in which there is a digital component.

And regulation will come through that, which is a huge, again, awareness and skills challenge for those who regulate these areas which have nothing to do originally with ICT, to understand the challenges related to digital risk.  And I could go with health and with energy and it's all of our society.

So yeah, a huge challenge.

>> NICOLAS SEIDLER:  Thanks, Laurent.

You wanted to follow up?

>> MIKE:  Let me add to that.  So glad you brought up the supply chain issue because that was an example of where the proposed legislation would have been far worse than the disease.  I mean, it would have killed the patient.

In some case companies had to certify only native born Americans with security clearances were touching the code or programming the chips.  It was a really strange mindset.  It was almost like a cold war thinking that only the U.S. is okay and we have to isolate ourselves from everybody else.

The good news in regulation is that when you look at a 2 x 2 matrix where the vertical is the amount of regulation and the horizontal is the amount of enforcement, you find a lot of countries that pass a lot of regulation but then forget to enforce it.  Then you have a lot of countries like the U.S. that tend not to regulate very much but they do enforce what they regulate.

And we have only had a few countries like the former Soviet Union, North Korea, that actually regulate a lot and enforce it.  We also have a few failed states down in the other corner where there's no regulation and no enforcement and total chaos.

The problem for a global company, particularly a U.S. one who is used to the light‑touch regulation in most sectors in the U.S. but the strong effective enforcement, is that they assume they are going to have to deal with regulations around the world.  And in this area in particular, it could be impossible to build a system that would abide by all the different regulatory requirements that you would see from developed countries that you're trying to sell into.

So we do have to make sure that we're not telling companies to build the impossible, to meet all sorts of criteria and standards that are conflicting with each other.

>> NICOLAS SEIDLER:  Thanks, Mike.

A question from the audience here, and then I have a question from a remote participant.

>> MIKE:  We're in trouble now.  This guy actually should be on the panel.

>> AUDIENCE:  Andy Purdy from wawa.

Following up on Mike's point.  One of the graft developed in the United States has been in the cyber security framework.  Regulators or those going to buy products could using the framework which is a processed based analysis where an organization uses this framework to analyze their risk.

And part of the message I think OECD could carry to the leaders of organizations both governmental and private, organizations need to make a commitment to cyber security and privacy.  They need to create and to incorporate it into their enterprise risk management framework.

They should use something like this framework set a set of requirements and have internal measurements to determine whether or not they are meeting those requirements.

The other thing I think the OECD could send a strong message and perhaps Aaron and JP Morgan chase is on the demand said.

The greatest challenge for companies like mine goes across all the sector.  Folks need to start asking questions and work together within sector and nationally and regionally, what are the questions that should be asked of or required from the suppliers of ICT.

It doesn't have to start with regulation and doesn't have to start with requirements.  It can be give preferential treatment.  But the buyers of ICT are not using their power.  They are not asking suppliers to raise the bar.  And that's an opportunity we all have to make a difference.

>> NICOLAS SEIDLER:  Thank you.

Laurent, would you like to respond?

>> If you have not done so, I encourage you to read our recommendation.

That's precisely the idea.  The recommendation targets not the people on the buying side who actually buy the ICT equipment.

It targets the economic leaders.  Those who are actually responsible for achieving the mission of the organization and who make the decision to use ICTs to get some competitiveness, some gains of some kind, they should understand this is their responsibility to manage digital risk.  Once they understand, they work with the ICT experts to manage the risk.

It's a completely different dialogue.  It's a completely different situation than the most often one which is it's a technical problem, techies will solve it and work with whatever ICT provider to solve that issue and we will be secure, which is perhaps not in large ICT firms, perhaps not in those companies which have a risk culture that is really rooted in their core in some sectors it's the case, but with a large majority of companies in the world it is that situation.

And that's what we're trying to change with this.  There is the ICT, I would say there is the more operational side of the problem where the cyber security framework is absolutely an important tool.  But if the business decision making side does not understand that this is a risk they should take responsibility for managing, they should take, not fully delegate to a technical manager, it's not going to happen.

In other words, the recommendation bridges the business side and the ICT side.

That's the objective.

>> NICOLAS SEIDLER:  Thank you, Laurent.

We're actually close to the end.

I'd like to just read out the question from a remote participant, from Bill, asking when are we going to see robust statistics on cyber security to help policy makers solve this mess.

Open question.  I'd like to give actually all panelists, yep, when are we going to see robust statistics on cyber security to help policy makers solve this mess.

>> MIKE:  I'm a geophysicist by training.  In our field in many situations, if you are within 50 percent, that was pretty good.  In cyber security quite often the numbers are even worse than that when you're trying to make estimates of how much damage occurs in a particular incident or how much damage will be done in coming years.

I really don't think that statistics are the big issue, though.  We know what we need to do to do better.  This isn't a lot of cost benefit analysis because the costs are so much higher than, the benefits are so much higher than the costs needed to ameliorate them.  So while I would like to see a lot better statistics in a lot of areas, this is not an area where I think we have a real crisis in our understanding of the issue.

I would say that Cloudflare does have a lot of information on where attacks come from, and we are trying to make those available to the OECD and other organizations so that they can use our data to understand which countries are doing the best job of using DNS sec, SSL, other technologies to protect their infrastructure.  Conversely, which ones are not doing a lot of job and have a lot of bot nets and attacks going on.  That kind of report card can be very helpful.

The other thing I wanted to do in my closing remarks is do what professors do.  I have been teaching at Georgetown about seven years, and I like to give out reading assignments.  So if you haven't already read it, there's a wonderful book by Alan Friedman and Peter singer called cyber War and Cyber Security, what everyone needs to know.

If you're an expert in the field, don't bother to read it.  But if you have somebody in your company, I had it backwards, cyber Security and Cyber War, what everyone needs to know by Peter singer and Alan Friedman.

Really nice overview of what the issues are, what technologies are helpful, and where the threats are.

After you have read that, if you want to get really scared, you can read a book by Peter singer, one of the authors of this book.  It's a fiction book called Ghost Ship, and it's the story of cyber war in the future when the entire U.S. Navy is disabled by a cyber attack and actually have is to bring out battle ships from World War II that don't have any clips in them to go do battle with the adversary.

Both very good.

The other book I would recommend is more broad, and called Tubes.  It's a travel log for the internet.  The author explored all the different parts of the internet from the domain name system to the Facebook data centers to Amazon and the wires in between.  It's a really fun book.  Again, it's not very technical but will give you a much deeper understanding of how this funny thing called the internet actually works.

>> NICOLAS SEIDLER:  Thank you very much, Mike.

Laurent, any final remarks?

>> Yes, quickly to say I strangely disagree with Mike on the evidence.  We need evidence.  As an OECD official I cannot live without a graph.  And we have very, very limited ability to produce robust evidence in that space.

There is that evidence.  It's not as robust as it is in other areas.  We need evidence in that space as in other places for sustainable policies.

And it's also a need, for example, for the development of insurance if we want this, if we see this as something that could help.  They definitely need metrics.  So it's a challenge for everyone.

We're working on it, but it is slow, extremely complicated, and really requires to bring together multi disciplinary players, players from various disciplines to make progress.

So it's a long exercise.  Anyone who can help, who thinks they can help, is welcome to come to us.

On the reading assignments, I noted the references, but there's a lighter one, not a book.  OECD the recommendation.  You can find it at slash DSRM, digital security risk management.

>> NICOLAS SEIDLER:  We need the GIF.

>> MIKE:  Think we can agree we don't want paralysis by analysis.  The threat is there, we quantified enough to know there is a threat that needs to be addressed.

I do agree insurance companies could be helpful in gathering this data because they are motivated.

>> NICOLAS SEIDLER:  Thanks.

Lucy.

>> My final points, and also getting back to the question, I mean statistics may mean nothing if people are not capable of understanding it.

We have statistics regarding Brazil, we produce the statistics.  But if people are not capable of understanding what it means and what the impact of that is on business, government, society, it maybe means nothing.

Well, anyway, the point is we need to educate people to understand security risk, then we will have a better way of talking and think security.  Because what we have seen today, most of company and government doesn't think of security.  They think security, we'll get to this later.  And later will be possibly too much late and much more costly.  Yeah.  And more expensive.

>> NICOLAS SEIDLER:  Thank you, Lucy.

Flavia.

>> FLAVIA LEFREVE:  Thank you.  As Lucy said, CJVR do researchers, have ten years.  More continue years we have robust data about cyber security.  But unfortunately we don't have involvement of the government to use this information to at least initialize a process, a regulatory process.  Because in Brazil we have multistakeholder regulatory process in CGI.br.

And without the minimum guarantees, even having information to base the public politician about the cyber security.

That's it.

>> NICOLAS SEIDLER:  Thank you, Flavia.

Last but not least, Aaron from New York, any concluding remarks?

I think we always have a small delay.

Go ahead.

>> AARON MARTIN:  To the point about the cyberer security framework, excellent point.  I think it is fast becoming the international standard for organizations interested in doing better cyber security risk management.

I think the OECD digital security risk framework and this cyber security framework are quite compatible.  The OECD fits into this framework just about the level at which it is addressing an issue.

To the point about regulatory harmonization, we very much agree that cyber security regulation is important.  Let's not create a million different regulations which become impossible to implement.

Finally, to the point about metrics, actually to speak Flavia's point, I think there is a point especially with Gorge of government policy making.

I encourage OECD and it's partners to encourage work in this area.

>> NICOLAS SEIDLER:  Thanks, Aaron.  On that hope, I hope you enjoyed the discussion.  I certainly did.  We'll also leave the discussion with a few books to read.

So I think that's not the last discussion.

Please join me in thanking the panel.

(Applause).

(End of session).