IGF 2021 – Day 2 – WS #233 Not Espionage as Usual

The following are the outputs of the captioning taken during an IGF virtual intervention. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid, but should not be treated as an authoritative record.

***

 

>> We all live in a digital world.  We all need it to be open and safe.  We all want to trust.

     >> And to be trusted. 

     >> We all despise control.

     >> And desire freedom.

     >> We are all united. 

     >> STEPHANE DUGUIN: I think we are live, I'm not sure.  Let's say we are.  Good morning, good afternoon, good evening, everyone, depending on where you are.  It is a pleasure to moderate and host this session in IGF.  This session is on espionage, it is Not Espionage as Usual.  You can imagine the questions we will have.  We will discuss this topic this morning. 

     I would like to welcome our panelists, Talita de Souza Dias.  Hello, Talita.  Kaja Ciglic. Hi, Kaja.  Serge Droz.  I see you there.  Just some comments on the set up of the session.  So it is a 90‑minute session with an intro from the panelists, and then what we would really welcome is the discussion, question, discussion, question, so please, everyone.  Not hesitate to use the chat.

     But here I am playing French and being very rude.  I forgot to introduce myself.  I'm Stephane Duguin, I'm the Chief Executive Officer of the Cyber Peace Institute.  It is a pleasure to have this session today.

     The topic as I mentioned is Not Espionage as Usual.  We are in this time where the long‑lasting thinking of what is espionage.  How harmful it is, how harmless it is, that is the question.  The context of sophisticated cyber‑attack from State and non‑State actors are clearly linked to espionage operations and are not harmless, in the sense of what they are, yeah.

     What they entail as of the moment of the attack and what is happening postattack or during the attack impact on who, which population, impact infrastructure, the impact on trust, impact on the whole digital construct.  I mean traditional attacks that cause harm have been considered unacceptable.

     We say that the attacks are impacting network system, attacking, disrupting, destroying.  And Government espionage activities are permissible or considered permissible.  When it comes to cyber operations, tactics employed and the scale of the activity can make this distinction less clear and recently attacks just hit the news. 

     In the recent past we can use the SolarWinds attack as an example.

      In terms of what this meant as an attack on the whole chain of the activity who was impacted, what process were, impacted, and what did it cost in the trust to the supply chain when it comes to the happening of the attack. 

     The question is here, how should we understand boundaries of state behavior in cyberspace when it comes to espionage activities?  How does this impact the operations when it comes to the ICT supply chain?  I'm excited to hear the thought of Kaja, Talita and Serge in these matters.  I would like to open the floor.

     If each of you can give five‑minute opening statement, and then we'll jump into open conversation.  Talita, we'll start with listening to your statements.  Thank you very much.

     >> Talita de Souza Dias:  Thank you for the introduction.  I'm a junior scientist at research of Oxford and a research fellow at WEOG that studies these precisely especially on what we call intersections of cyberspace.  It is a pleasure to be here, because the I.T. supply chain attacks is one of the topics that we discussed during this Oxford process on international protections in cyberspace.  This is really right up my alley, our alley, it is a pleasure to have the activity to discuss this issue, which is so divisive, as we talked about during our introduction.

      What I want to talk about in the five to seven minutes, I want to make four points to the question, in response to the question that is posed per the session.  These points arise out of the workshop that we held as part of the Oxford process on precisely this issue of the I.T. supply chain attacks using the SolarWinds attack as an example of this attack.  And other conclusions that my team have sort of like reached when assessing the issue from an international law perspective.  We're in the midst of producing an article precisely on this question of SolarWinds hack and I.T. supply chains more broadly.  I want to talk about the conclusions reached during our research.

      The first point I want to make is obviously Stephane, you already raised this point [static]

is the I.T. supply chain attack pose unique challenges because of the pervasiveness, the insidiousness of these types of attacks.  So even if the main sort of like goal is to just extract information to obtain data, to do espionage, they might have indiscriminate effects in the supply chain, for example, update systems as in the case of the SolarWinds attacks.

      There are unique challenges that the attacks bring to the table.  It is important to understand the specificity of the types of operations when dealing with them from an international law perspective.  One main issue that these kinds of attacks might pose is that they might, as you mentioned, I think, Stephane, so far, is they might undermine the trust in some systems that are regarded as essential for the operation of the Internet.  Such as the update system as I already mentioned [mic shuffling]

this is an issue from the international perspective.

      The second point I wanted to make is quite well agreed upon.  And this is something that we have reiterated many times during our conversations on these types of attacks, supply chain attacks and other cyber operations, it is that international law applies to information, communication, technologies, cyberspace, if you will.  This includes I.T. supply chain attacks.  The international law is not regulated.  International law applies as a whole to all kinds of operations including I.T. supply chain attacks.  Including SolarWinds.  The tricky thing is what about espionage.  We talk about the attack and what about espionage.  That is the main topic for this discussion today.  This brings me to the third point, that is [mic shuffling]

espionage is per severed per ‑‑ perceived as permissible in international law.  [Mic shuffling]

it doesn't mean that other rules of international law beyond those that apply to espionage don't apply.  Quite the contrary.  So depending on the target dates, the effects or means or methods deployed in a certain operation, including a certain supply chain attack, different law international laws apply.

      That brings me to the fourth point [mic shuffling]

[static]

that apply to the I.T. supply chain attack like the SolarWinds hack.  These don't necessarily cover I.T. supply chain specifically.  There are rules that attack supply chains or digital supply chains but aren't specific rules that prohibit these kinds of attacks, indiscriminate I.T. supply chain attack.  But there are a number of more international laws that cover a number of different aspects of the operations because they protect individuals, persons and objects that are involved in these supply chains.  These I.T. supply chains.  To give examples, people talk about the rule of sovereignty in cyberspace.  States have control over their territory and their cyber infrastructure.  Some States, even over their data.  Any trespass says over their digital environment could amount to issues.  That is one view.  Another relevant rule and perhaps the more relevant rule for these operations is rules regarding due diligence. 

      Because in operations like the SolarWinds hack, it is unclear who the perpetrator is.  We don't know whether a state actor was actually directly involved.  We know that it came from Russia.  And we know some non‑State actors were behind it, but we don't know exactly what was the link between these actors and Russia. 

      So rules that require due diligence, that is irrelevant.  Irrelevant if it is a state that perpetrates the attack or not.  What matters is whether the state from which the operation emanated failed to take action to prevent these kinds of attacks.  This is pertinent here.  There is not just one rule of due diligence, there are a number of due diligence obligations.

      So for example, there are rules that require States to prevent transboundary harm, regardless of whether the operation is illegal or not.  That is very pertinent to the issue of I.T. supply chains and espionage.  Even if espionage is not illegal and supply chains attacks are not illegal, if they cause transboundary harm, they're covered by this rule.  This is to give an overview of the framework that applies.  You have human rights laws that apply in particular the right to privacy.  Which is pertinent, like SolarWinds, private communications were breached.  Human rights laws are relevant.

      Just to finish up, to conclude, even though we have the patchwork of rules that applies, we still need more study, more research into how exactly the rules apply to cover these distinctive kinds of operations.  So that's it for me, I guess.  And I'm keen to discuss the issue from an international perspective in the Q&A.  Thanks. 

     >> STEPHANE DUGUIN: Thank you, Talita.  A lot of points to review afterwards.  Thank you for the opening.  Kaja, I will turn to you for the next address.

     >> Kaja Ciglic:  Thank you.  Thank you for the remarks Talita, as well.  I think SolarWinds was attributed to the Russia intelligence service.  So we do aware ‑‑ the Governments think they know who did it.  So that just as a clarification.  I would say ‑‑ I think talk about two points.  I think try and explain a little bit about why, from an industry perspective espionage like this is problematic.  And why it is not ‑‑ cannot be seen as the same as traditional forms of espionage.  And then talk a little bit also ‑‑ Talita covered a lot of the international law applications.  Talk ‑‑ just highlight a few normative agreements that have been made in this space as well.  And that could be leveraged, should be probably leveraged to more clarity and to implement sort of some guardrails in this space. 

      First of all, I think from attack like SolarWinds, I think both Stephane and Talita talked about [static]

sorry.  There is sound.  Talked about the challenges that relate to tampering with some of the critical functions to keep the Internet safe.  The software update mechanism, which is one of the tools we have to push out security updates.  So if there is vulnerability in the system, whether it is Microsoft or anyone else's, there is not much that a company can do, beyond issuing a patch. 

      To get that system to be effectively "fixed," for lack of a better word.  And if people ‑‑ people already distrust the software update mechanism a fair amount.  There is fears that it might break something when you update.  There are concerns about you know just what the kind of information do I give to the company when I press "update."  So does it send information back?  All of these things already percolate at the back of people's minds, organizations and individuals.

      If they're consistently the software update mechanism are also used to attach effectively malicious software to them by, you know, SolarWinds is one.  But WannaCry were similar examples.  I think that makes it ‑‑ that really properly undermines our ability to secure the online environment and seeds further mistrust on the online ecosystem.

      The other thing I would highlight is in the traditional offline world, espionage is a person followed a person.  And to try and get information.

      Here, because the products and services used are used across millions, billions of people sometimes, if there is a vulnerability that is inserted it affects everybody that uses it.  Even if it doesn't ‑‑ this was the case in SolarWinds.

      Not all of the entities that were compromised were actually spied on, but that does not mean that ‑‑ but that still meant that all of the companies that have discovered the vulnerability had to go through the process of figuring out whether they were compromised, figuring out if they weren't, fixing it, figuring out what it meant with customers and others downstream.  The impacts are much broader ‑‑ I'm characterizing here ‑‑ there were people in Government that were the actual targets of the attack.

      So the implication on the security of the Private Sector ecosystem are immense, both in terms of again, trust, but also financial. 

      I think, you know, it is important that as Talita was saying, we use some of the international legal frameworks that we have to try and limit as much as we can some of these issues of professionality and assessment was highlighted by Governments in the attribution of the attack.

      I think there is definitely something in there to think about.  You know, I'm being realistic here.  I would love for this not to be a practice that is acceptable.  But ... I'm also well aware that it is going to be difficult for the intelligence communities to be like, we're just not ‑‑ we're just going to stop doing some of this.

      But having guardrails, I think, would be really, really important.  I think in terms of the other ‑‑ in terms of the two other things that in addition to international law I wanted to highlight ‑‑ maybe three.

      So Microsoft has A, pushed forward a suggestion in the UNDDE but in the upcoming open‑ended Working Group, the two processes of the United Nations, the deal, the rules of the road for cybersecurity actions, to get States to agree to not tamper with the update mechanism.  We'll see how that goes.

      The other norm that was proposed, proposed a couple years back by the global Commission on the stability of cyberspace that also looked at the space was the norm to avoid tampering, which called on state and non‑State actors to not tamper with products and services, either in development or production or allowed them to be tampered with ‑‑ this is important ‑‑ if doing so may substantially impair the stability of cyberspace.  There is something in there as well.  I think particularly the qualifier that allows for a little bit of the wiggle room that I feel realistically we need to acknowledge. 

      Then the third one was actually a norm that has been agreed, in the UNGGE, in 2015, the Group of Governmental experts, Governments that came together to agree on the 11 norms, voluntary, to act responsibly in this space. 

      Which was the norm that States should take steps to ensure supply chains security and also means seeking to prevent the proliferation of ICT and use of harmful hidden functions, which I think is something that we can also discuss.

      I will also say and then I will finish this, you know, the 11GGE norms were voluntary and adopted by a small number of States and since including this year have been agreed by all states in the General Assembly Resolution.  So I think the steps that we need to as a community encourage States to do is to not just ‑‑ this is right.  Research, and see how these things are applied but actually put pressure on Governments to start implementing them and actually doing things that they agreed they will do. 

      That will, by itself, clarify some of the uncertainty.  I will start there.  Also, happy to discuss some of these later.

     >> STEPHANE DUGUIN: Thank you, Kaja.  To remind the audience, the chat is open, don't hesitate to post your questions, comments, Serge, would you like to complement.

     >> Serge:  I'm on first, they are the guys that clean up the mess after such things happen.  It will be a fairly boring panel because we're all really critical of espionage and the operations.  We are not big fans of let's hack the supply chains here.  I think that is good.  Let me add to this on a technical level, there are issues with the espionage that we see today.  First of all, not only do they create problems with trust that was rightly pointed out.  They open up holes that are misused by others.  For example, exchange disability, onsite ‑‑ whatever state actor was done, all the criminals jumped on the train and created a lot more work.

      For instance, responders kept installing ransomware on victims that hadn't patched.

      I think what may have started as kind of a common espionage thing turned into a big party for criminals to add to this ‑‑ I don't know, I'm not a lawyer, but an international law point of view, like question the proportionality would be given here.  I mean, we create a lot of damage.  This has a side effect.  The side effect is that responders during the attacks once discovered are totally overworked.  We need to focus on remediating these types of things, diverting our energy and focus from other areas where we should be working too. 

      So I think that is something that we really have to keep in mind.  The second thing, that's probably more with espionage operations that kind of are not on this really, really big stage.  Like let's talk about using tools like the NSO tools, Pegasus tools, stuff like this.  The problem is the tools are really affordable.  It is not the state has a problem because yes, the States always spy on each other.  But suddenly all kinds of people and organizations are spying on Civil Society and normal citizens get really cool tools.  If you look in all James Bond he has Q that develops tools no one else has.  That is not the case in cyber.  Once the tools are out of the box, everyone has them, not just James Bond. 

      That really is a challenge.  I think the reason for this is that while in the old days, the physical world espionage was quite expensive.  You need big satellite things and people on the ground.  That put a high price.  Espionage operations were quite expensive.  Now they have become really cheap.  And that is a game‑changer.

      So maybe rather than discussing about okay, espionage is kind of legal and have to fix this and that.  I think States are kind of now taking advantage of having something for really cheap.  And at the same time creating rules because you are really fuzzy and don't know what they mean in cyberspace.  And also attribution.  You can discuss if it is hard or not.  But many of the tasks or physical attribution methods.  We always discuss about well, maybe we can continue doing this.  We have to fix it here a little bit.

      Essentially, we're putting Band‑Aids on something.  I think maybe it is the time that we start rethinking of what is okay or whatnot.

      I kind of agree with the previous speakers, we shouldn't be dreaming, but I think it may make sense to occasionally say take a step back, are we discussing Band‑Aid policy or are we making cyberspace a better place.  Food for thought.

      I'm not 100% sure about all the answers for the points, but I leave it at that. 

     >> STEPHANE DUGUIN: Thank you, thank you very much Talita, Kaja and Serge.  I took some notes on what was said so far in order to kick start the second part of the panel with the open conversation between the panelists. 

      It is clear that we are not as ascribing a threat that is invasive, insidious and unique.  I like what you mention Kaja in terms of we go way beyond having targeted or convincing the proportional espionage activity coming from the history of what is happening in the offline, let's say.  The tools, the capacity, know‑how, investing into research that will target million.  The impact is on billions of users.

      Interesting point that all of you share, I would like to put this back is it undermines the trusted systems, the impact is the trusted systems and the example of the software update mechanism that could not be trusted when they are sometimes the only place that people are going to get trust in making sure that what they use is secure is quite critical.

      Serge you mention also that it opens doors to criminals.  Not because any actors is managing a big campaign to spy and intrude and ear drop and steal data.  It is the Internet is all connected.  When you open doors, when the invest into vulnerability, you open doors also for the ones that are looking for the doors to be open meaning criminals.  Then doing that, the excellent point, the capacity for everyone to defend themselves.  The first responders are overbooked and start to be problem with the capacity in terms of first responder.  Make a simple example of COVID, you have a wave of COVID, a wave of infection, and then you can't manage anymore.  Except in this case, it is a disease engineered by humans.

      Usefulness of the legal instrument, interestingly what you mentioned Talita and Kaja, what we are looking at in the proportionality of the sovereignty and due diligence, and beyond the fact ‑‑ I will be curious about your thought.  Beyond the fact that this first exists, are they operationalized?  What is there to put in place, from the legal instrument to the norms, Kaja was mentioning before. 

      And what are we seeking?  I will just also close this with Serge, in fact, are we looking for Band‑Aid or looking to sort the issue?  What should be the ambition?  I would like to give back the floor with an opening question to the three of you.  Something not mentioned so far is we should not look at this as a state's own controlled and forced capacity and market.  I mean, all of this is indirecting and in fact relying upon an industry‑led market which is developing and setting the tools.  Serge with that, there is a reason that there is a study looking into at least 224 companies in the world that are involved in espionage.  The more the companies the ‑‑ the mean the less the price. 

      It is simple demand and supply.  The less the price, the more the companies, the more that are learning the skills to spy and the tools are accessible by a lot of entities.

      I'm curious about the panel views on this open vulnerabilities market that is underlying this threat.  Maybe that is an opening question.  I am seeking what is in the chat to continue.  Anyone want to take it? 

[Overlapping conversations].

     >> Serge:  I will take it.  It relates ‑‑ what you asked is about companies and organizations, kind of what I call a gray market that aids and makes it possible, that is related to the question but the person in the chat who says the suppliers level the field to make it ‑‑ they make the tech accessible to players that don't usually have this.  That's true.  The blocking of this stuff is hard.  There is the certain initial investment, high initial investment to make before you can become a successful cyber spy.  And having private companies that sell this big‑style kind of makes this stuff accessible to others.

      So this is now a really good question.  If you really insist on the point that espionage is totally okay, maybe we should insist on the point that tools should be available to everyone.  I understand that companies and organizations can say we don't have resources superpowers have.

      Looking at the damage the tools do and damage the operations do, you cannot wonder if this is really the way forward.  If you compare it to the other like weapons of mass destruction, like nukes or something that would be officially five countries are supposed to have, it is actually a little more but the general consensus, if the people don't really know how to handle these types of things.

      The nuclear arms we really understand what responsible behavior and the space is, you don't use it.

      So if you work on the premise, okay, this should in principle be available to everybody that is responsible, however you define "responsible," then you have to come up with an understanding of how to use this.  This is hard in cyberspace, that is where we are failing.  I think today, one of the rules of the challenges is that no one really agrees on responsible use.  And all we have is the mandates we mentioned.

      Personally, I'm really very critical.  I completely accept the fact that smaller Nations have a disadvantage.  That should lead to the more powerful Nations really taking the responsibilities serious. 

     >> STEPHANE DUGUIN: Thank you, Serge.  I wanted to add something.  There is recent coverage in the investigation run by journalists in the EU.  That is a part of the EU countries are procuring, by the way, with taxpayers' money these tools.  Sometimes the EU countries are targeted by the same tools.  It shows you how the situation is it is not what state you are, and by the end of the day it is a market no one controls.  And no one should underestimate the financial capacity of criminal Groups.

      At the end of the day, it is the capacity to buy what is developed.  Even if this market only States are going to buy it, it is a foolish dream, big foolish dream.  Talita, please.

     >> Talita de Souza Dias:  Thank you.  Thank you Serge for your remarks.  You mention the issue of proportionality.  The operation wasn't proportionate to the traditional aims of espionage.  I think this is relevant when you are thinking about international human rights law and the right to privacy, for example.

So for any state to limit or non‑State actor for that matter because non‑State actors also have responsibilities to respect human rights law, right? 

      There has to be a legal basis.  There has to be legitimate aim.  There has to be necessity and proportionality, in any interference or right to privacy.

      If you talk about operations that will affect or have as a side effect interference of privacy, private entities, individuals ultimately, these operations, they must, must be targeted.  

      So for example, the indiscriminate nature of supply chains attacks or SolarWinds, for example, would go directly against this requirement of necessity and proportionality in the human rights law, which means any surveillance type of operation must be targeted.  It must be reasoned. 

      So for example, both surveillance, we all know, it is blatantly illegal in the international human rights law precisely because it is indiscriminate, that is something to bear in mind.  The fact that an operation is not targeted to a specific individual or state actor, and it is not justified on some basis, for example, for national security reasons, that means it raises a red flag that the operation is contrary to the right to privacy under international human rights law.

      The second point is related to that and these technology, Pegasus, et cetera.  For example, if you read through the report from the Special Rapporteur on the right to freedom of opinion and expression on surveillance and human rights, basically the Special Rapporteur proposes a moratorium for the technologies until we find a way to make sure that the technologies can only be used for aims that are legitimate, necessary and proportionate. 

      So if these technologies can be used for both good and bad aims, then they should not be sold.  Until we find a way to control the damage that they might ‑‑ that they might give rise to.  So this is really interesting that you mentioned the pervasiveness of the technologies and the way in which they're being employed.  And from an international human rights law perspective, at least the technologies should not be on the market until we find a way to control them, right.

      And on espionage, you mentioned espionage is something that, you know, most people frown upon, at least in the cyber community because it is bad.  Actually, I wanted to talk a little bit about the other side of the coin.  Actually there are important advantages or there are good uses of espionage for peace and security.  So States have done it for industries, because for example, it avoids conflict.  So if States know what other States are doing, it gives them a little bit more confidence in what to expect in terms of, you know, confrontation, nonconfrontation.  So if States have more information that means that they're less likely to be hostile towards other States.  That's a good ‑‑ a benefit of espionage, but obviously, espionage can be used for a number of different purposes.  What we have seen now is that it is being used in an indiscriminate manner.  Perhaps the way forward is not so much to prohibit espionage as a whole, but maybe, just establish some rules of the road, you know, some boundaries, you know, some red lines beyond which espionage should not go.  To try to contain it to the traditional uses that it had before. 

      And Stephane, you mention the operationalization of international law, that is a problem, right?  In any area of international law it is difficult.  It is difficult to enforce international law because we don't have an international police force.  We don't have an international executive.  That is tricky.  It is symbolic.  It serves as a deterrent as a preventive tool to help guide behavior of States.  But I think that one thing that would contribute to the operationalization of international law on that front.  On this sort of like educative, didactic function of international law is to help build legal capacity of States, especially less‑developed States.  I've done a lot of capacity building exercises, especially with Latin America countries.  What I hear constantly, constantly is they lack legal capacity.  They don't know what rules of international law apply.  I think it is important to invest on that.  Thank you.

     >> Kaja Ciglic:  I have lots of notes on different topics.  If I start.  Maybe on the point that Talita mentioned about the good use of espionage.  Microsoft issued the Microsoft annual defense report.  It looks at the previous year and the attacks on the Microsoft systems we have been seeing.  The trends type of things about cyber crime, nation‑state and the different trends whether it is uptake in ransomware or attacks on IoT.  If you look at nation‑state sponsored attacks, the vast majority of them are for espionage purposes.  They're not destructive attacks.  They're attacks to try and find information. 

      Whether they ‑‑ the attribution we made ... Monday?  Yesterday?  I'm sorry.  On Monday, Seattle time.  Against a threat actor that is coming from China was very clear that the targets were largely Government ‑‑ Minister of Foreign Affairs.  Few nonprofits dealing with foreign affairs.  And across a particular Region of the world, basically looking to find information that could only really be used by a nation‑state.  Similarly, I think this was last year ...

      We have seen a really big spike by actors associated with Iran, ahead of conferences dealing with the nuclear deal, for example.

Both at Government and think tanks associated with Governments involved in the negotiations.

      So yes, this is exactly why nation‑states are using them.  However, because the way the online ecosystem is set up, that means the implications themselves like with SolarWinds on a much broader set of enterprises. 

      And are hardly ‑‑ can hardly be called proportionate.  I think that is where sort of the point, as well, earlier on, that was about, you know, whether the actors or cyber entities kind of level the field.  Yes, they do.  I think I'm not sure that is a good thing.  I also would say cyber weapons, you know, with quotation marks ‑‑ by themselves, level the field.  Right?  I think if you compare it to investments, think tanks, planes, all of this is cheaper.  As a country you invest in the vulnerability, it is cheaper than building an army, a physical one.

      So yes, there is a level ‑‑ there is definitely a point there about leveling the field.  But I would also say as you level the field, you are in danger of destroying the field in the process.

      You know, the way Microsoft is engaging in this conversation has been this fear of endless escalation without ‑‑ you know, what that means, not ‑‑ you know not just necessarily a Government response, response to a Government and kind of a system is down or something like that.  But unintended consequence.

We have seen this to an extent with not only WannaCry, but how the systems are very connected and things can spread very quickly.  And unintentional damage is something that is ‑‑ that can have dramatic consequences for our economies, not just human rights, and things like that.  For economies, not just the ICT sector, but because we are more or less relying on technologies for all of us.  Hospitals will go down, the Governments won't be able to operate necessarily because their systems won't work

      So that's where I think Serge's point about responsible use is exactly the right one.  I would say, some of the bigger countries, if we are looking at some of the more malicious than others, that have invested in this space have also invested in understanding the implications and invested in developing frameworks that guide the use of the technologies for their intelligence services.  You know, the U.S. example of vulnerability ‑‑ you know, they're not perfect.  We need to do more.  But the example of the vulnerability equities process, which is a process that looks at, you know, they find a software vulnerability and decide what to do with it.  You know, do they report it responsibly to the vendor?  So the company whose product has a vulnerability?  Or do they store it so they can use it at a later date to ‑‑ to pursue an intelligence objectives?  So having the process to have the discussions about the implications of the economy, the implications on Civil Rights, I think they're critical.  I think we need to have more of that.

      This is why also the sort of the cyber mercenary’s angle concerns me a little more because what Stephane said ‑‑ not only do the intelligence or the services that are traditionally acted in this space use them, we see the law enforcement use them increasingly.  And again, not necessarily the same frameworks.  And then again, Stephane said this will be used by cyber criminals because it is not like they don't have money. 

      So it is the potential of spreading it.  So I think it would be important to ‑‑ and also Stephane, it was an interesting point.  You said, you know, Europeans buy them and this is definitely where I think that we as a multistakeholder Civil Society industry have the opportunity to use our voice to put pressure on our Governments, particularly democratic Governments to lead by example.  Not only European by it, but a lot of the companies come out of Europe.

      But making sure we're vocal about how we disagree with this type of behavior.  And on the industry side, find ways to regulate is the wrong word.  But impact that behavior, whether it is, you know, we saw WhatsApp sue.  And we so Apple do similar.  Amazon and Microsoft put MSO off the Cloud ecosystem making sure that the technology cannot be again as effectively as powerful.

      I think that there are things that we can do collectively to put pressure on Governments to try and get to that end place.  But again, I think the end place will be difficult to achieve. 

      Maybe just one last point on the earlier question at the beginning, where some of the discussions are happening.

      So at the U.N., there is an open‑ended Working Group, which is a Group consistent of States that is starting next week.  I put the link in.  You know, it is a fairly limited participation for the multistakeholder community, but there will be consultations and there will be a bit ‑‑ always there is an ability to reach out to your Government and put some of the points across.  So I would encourage you to check that as well. 

     >> STEPHANE DUGUIN: Thank you, thank you very much Kaja.  I was looking at the chat but I would like to answer what has been mentioned by the three of you to put additional points before going to the chat. 

      Um ... you mentioned the legal action.  You look at it from international law, international instruments, and norms of international behavior.  Indeed, there is some national initiative, on the basis of national law when it comes to legal election.  It is high in the news with NSO and different companies that are suing NSO basically.  Something else, I'm not sure the Group knows about this.  The company that is under the investigation for being complicit in active torture.  Because technology was used to facilitate to civilians that led to torture.  Looking into that.  Interestingly enough, it looks like the measures taken by States or state actors to espionage the world and it is more complicated.  And countermeasures by Civil Society.  Can you imagine the responsibility is to protect people?  Today the takedown of infrastructure.  The forensic analysis of phones, organization of class action is managed by industries in society to protect people.  Kind of a strange situation, to say the least.

      Another question, which is linked to the comment that an attendee put in the chat.  I hope I pronounce your name right.  Where is the information?  Who is responsible for the information security?  That is the discussion now.  The supply chains attack.  You attack all of the weak link in the supply chains to get the data and certainly the responsibility behind that is complicated to look into.

      I have a question to the Groups about this.  I strongly believe you can only policy what you can measure.  Is there a need to share with the audience a source of information about, you know, the scale of the attack on supply chains, the occurrences, what they mean in terms of violation of international law?  Is there a body of work there?  Your views and maybe some pointers, if there is any. 

     >> Talita de Souza Dias:  Is the question is there a body of work of international law in the context of ICT.

     >> STEPHANE DUGUIN: Do we have a compendium of information of the supply chains attacks that happen over the years and this is what it means in terms of attributed, yes, no, an analysis of what is meant by international law violation or bodies ‑‑

     >> Talita de Souza Dias:  The only consolidation or compendium we have on this topic because it is so niche, right?  And so recent, is the report I'm holding right now that we did as part of the process, which is a cooperation between our institute and Microsoft.  We produced this wonderful report of the workshop that we held on this topic, right after SolarWinds happened.  It consolidates all the discussions we had among international lawyers, different views on espionage, different rules that apply to the supply chains attacks.  It is not yesterday published, but it will be published soon.  Another compendium or consolidation of views we have on this topic is an article we're putting together on SolarWinds which I mentioned, so we're in the process of publishing that, too.

      So what I would say is just tune in into our website and wait for this to be published because they will be there soon.  That is the only thing on I.T. supply chains we have right now.  Obviously, there were blog posts right after SolarWinds happens and the blog is there.  Blogs on justice security, agile talk.  A few posts on the topic from an international perspective, but there is nothing really systematic about it yet. 

     >> STEPHANE DUGUIN: Thank you, Talita.  Serge on your side when it comes to the community of the first responder is there any statistics or data to look into the scale of it or progression of the threat? 

     >> Serge DROZ:  That is a really good question.  It is at the heart of it because there is little data and very little solid research on this which allows States to really wiggle their way through all the opinions.  Because the opinions are whatever, opinions, not facts.

      There is a lot of published kind of reports, mostly by security companies like Kaja mentioned a couple that describe the attacks.  I guess people doing statistics.  I think they said it before, relations, systematically collect state driven operations.  Things like that.

      Now, turns out that all of the reports, they're targeted towards the clientele that the organizations bring out.  That is research done at system net, and it leaves out Civil Society, which is still a big part. 

      I think right now, we have some idea of what's going on in terms of malicious operations.  But it is a bias picture.  It is bias towards people that can actually afford expensive security companies.  And it really leaves out the poorest of the poorest.  These are the ones that really need this.

      An example of this bias is really we all complain about NSO.  I don't want to defend this company.  I think their business case is certainly not one that I would subscribe to, but NSO is just one of these.  It is not just NSO and not just the Israeli high‑tech Council.  Stephane, you referred to this early on.  There is high confidence of 80 companies selling this stuff to shady entities, that is only what is based on available information.  I think it is important that we start focusing on these type of things.  So do we get a good picture?  That is also important so we can actually get an idea of the damages we do.  It is not just SolarWinds or the exchange thing that really, it is a lot of small ones add up to.  I think a lot of the first responders have a gut feeling of how much is going on.  But I think we really lack serious research, academic research here that really is found on scientific principles.  Again, comparing to the nuclear stuff, we have a network of satellites that monitor X‑ray radiation, these are scientific facts no one can argue about.  I think we need it. 

     >> Kaja Ciglic:  I will add another lens, almost.  I agree with a lot of it.  I agree, there needs to be more research done in this space.  Maybe Stephane, you can talk about what you guys are doing, too.  But I think it's one of those things, you know, like ‑‑ this again is Microsoft data looking at Microsoft network.  So exactly like Serge said, it is skewed to what ‑‑ to a particular set of information in the world.

      But we have ‑‑ you know, all we have seen are increase over the past year in terms of the number of actors.  Associated with nation‑state.  The level of activity as well as their sophistication of what they're doing.  Sort of the P in the APT, I think is becoming more and more persistent.

      And this is, I think where SolarWinds is a good example of also ‑‑ once again, why this is so problematic.  It is not problematic just for ‑‑ "just" is the wrong framework.

      It is not problematic solely for the human rights activist.  And the people who are often under threat and particularly sort of from some of the tools used by ‑‑ developed by cyber mercenaries.  But also problematic for companies with large resources.  No other sphere of warfare, again, quotation marks.  Is a private business expected to withstand on a daily basis attacks from either Government or Government‑sponsored actors that, that sometimes take years to develop?  Have lots ‑‑ on the offensive capability, have lots of actors associated, individuals associated with it, lots of money poured in it.  I think that is also where the disparity, I think, even for the really big companies.  I think that is something to keep in mind.  As sort of more and more of these take place.

      You know, some of ‑‑ the ones that make the news are a very small proportion of what is actually going on.  You know, also the ones that actually succeed are a very small proportion of what is going on.

      But you still have to defend on a daily basis. 

     >> STEPHANE DUGUIN: Very, very true.  Just to give an idea, we are providing free support to NGOs that are targeted or potential targets of cyber‑attack.  Espionage, data, trying to understand as an actor, who are the NGO helping, this is data that can be interesting from a geopolitical agenda point of view.

      And we see that the capacity for these NGOs everywhere in the world is very low.  Versus what is mentioned, the amount of money put in the attack.  That is problematic.  In the face of this, the real companies ‑‑ I put in the chat the reports from the Council.  They're sending these to anyone that can bite.

      It is just escalating the flex to an unprecedented level.

      Thanks for sharing the information Talita and Serge what is available in terms of compendium of knowledge on the threat.  Looking forward to the report, Talita. 

      But really, it is at least ‑‑ if I do a parallel with healthcare, there has been discussion about the past years that healthcare is unique, shouldn't attack healthcare, at the end of the day, there is nothing just to document the level of attack on healthcare.  And document all of the attack level it is increasing over time.  You can see the activities of States, the Statesmen, the agreement, and the parallel if it helps healthcare to be less attack, yes or no.

      We need something similar for this problematic.  But okay.  Easier ask than done.

      Looking at the chat.  Please welcoming question.  Otherwise I have additional one.  It is on the remediation after the attacks.  So when these kinds of attacks are happening and documenting like let's take SolarWinds example, what is happening afterwards to ‑‑ in order to help the numerous numbers of victims? 

      Because one in 75, it is like there was an agenda to spy on a specific entity.  Because the spying activity there are vulnerabilities created everywhere, opening doors to criminals to just invest in this and start attacking, this creates victims.  What is done for the victims afterwards?  What is the recourse?  What other capacity is available? 

      I don't know if you have some thoughts about this? 

Serge do you want to start? 

     >> Serge:  Can you rephrase the question?  I'm not sure I understood it.

     >> STEPHANE DUGUIN: My bad.  We're looking sometimes at the cases as if the day when they become public and almost attributed, then we enough to something else.  Then starts the very ‑‑ the real problem for the victims.  People identifying that they are victims or maybe they were stolen, their system was patched or cleaned.  How do you see that there is specific support past the big attacks like SolarWinds, to help the victims recover, get better, not being attacked anymore?

     >> Serge:  That is an important question.  It has a two‑part answer.  The first is this, one an attack is public that is when we really start to work and it eats up resources, again, you make a comparison to the pandemic.  Initially a lot of mature organizations they kind of start fixing their systems and cleaning them up.

      There is a lot of smaller organizations that just don't grasp the importance of what this is about and that they should do this.  Which kind of is mind‑boggling to me.  Maybe that is because I'm living in the security‑aware bubble.  A lot of companies, a year after say the exchange they haven't patched their system.  That makes it really hard for, in particular national C CERTs to follow up and convince them.

      In fact, what often happens ‑‑ again, this is something that we need to address ‑‑ is if the Government C cert or team detects that you have a problem.  The response is how do you know this?  Have you been hacking me?  The anger is directed towards the wrong people. 

      This goes all into the question of victim notification, which really is a difficult one.  And right now, we're facing this with another Group.  Sometimes we can actually anticipate an oncoming ransomware operation.  But we find it really hard to inform the victims.  Law enforcement says it is not our job.  Suite level agencies don't do it as a matter of principle.  That is hard.  Maybe we solve this by becoming better in information sharing.  There is the second part.

      That is some of the damage you can never undo.  If your data is stolen, it is stolen.  You can't get it back.  It is not like an expensive painting.  If you catch the thief you get the painting back.  If you catch the data thief, the data is still out there. 

      And this really is a problem.  Obviously, right now, I could say I don't really care about state secrets, but I do care about personal data.  I do care about like this case where attacks hack a mental hospital.  I mean, these patients have a hard time as it is.  So this information now is public.  It will never be unpublic.  It will always be out there.  That is something we can never undo.

      Again, if you go back and say what are the costs of these operations?  It is so much collateral damage.  It is internal collateral damage that is not taken into account.  Then again, if today I look at the discussions about when is espionage legitimate?  I mean, I'm talking there is illegitimate use.  We talk about this tiny space here to adjust something.  I think rather than focusing on where exactly does this line between bad and not bad run?  Let's map out the part where we really know it is bad.  That is big enough.  We have all this later talk on the fine line and the 80% toward the dangerous territory, I'm already super happy.

     >> Kaja Ciglic:  I would agree.  I think to build on what Serge said, I don't remember who it was.  It was one of the Government representatives that recently said on how do you help people?  How do you help organizations?  They recently said ransomware is good for this.  Obviously, it is terrible as an epidemic and as a crime.  But in comparison to the attacks we have been talking about over the past hour or so, sort of the espionage stuff that affects reality sort of a very, you know, like ‑‑ it is not as dramatic as an impact as is like our hospital shutdown by ransomware. 

      It has brought the awareness of the implications of cyber‑attack much more to the front of individuals' and organizations' consciousness.

      There is more of an awareness of okay, maybe we should do some basic things to secure the system ahead of the attack taking place.

      I think this is the bit that has been really, I think, boring for like 15 or so years that ‑‑ the tools that protect everybody from the vast majority of the attacks ‑‑ obviously not the most persistent.  Not the most sophisticated.  I think they will get in, whatever you do.  But they have been the same for the past decade, right?  So have been things like patch and patch quickly.  Don't wait like Serge said, a year.

      Updates, you know have updated systems.  Ensure that you use ‑‑ you know, strong passwords, multifactor authentication.  You know, that are really tools available out there.  Simple.

      Sometimes ‑‑ simple.  It is that one extra step a lot of people are like ... it is a step too far.  That really help prevent things like the data of individuals being spread and out there.

      And the reason why I say this, both in terms of the preventative measure and also sort of the patching, the post measure, almost is because there are ‑‑ you know, there are ‑‑ there's ‑‑ there's little that we can all do.  You know, as a platform, we can share patches, we can share information.  But the end of the day, it is a responsibility of the individual or the organizations to actually implement some of those.  We can't force them.  So making sure that those things are connected and implemented more frequently, I think will be important.

      The other thing that I wanted to also say is to, you know, what I'd love to see is for ‑‑ whether it is a small Group or States, whether ‑‑ this is either on the cyber mercenaries or more broadly on the espionage.  Small Groups of States or small Groups of Private Sector actors to start like stepping up and walking the walk.  And saying, you know, not just saying, okay, we disagree with this practice, but then going a step further and implementing it.  You know, whether ‑‑ you know, whether it is the cybersecurity attack accord and we will not conduct offensive operations or for Governments to say we will not, which I think the incoming German Coalition is saying they will not buy the Private Sector offensive actor tools.

      I think they're not a long‑term solution, not a solution for everything, but they start to create the normative environment of what is good and bad behavior and hopefully over time we can become much clearer about you really shouldn't do that.  Doesn't matter where you are in the world. 

     >> Serge:  Maybe to add to this.  Most of the States paint themselves as helping to improve security because we really important out people with weak security and that's what they get for not working with us.  For most of the ransomware give you a fairly high quality security report after you have paid up. 

      But that brings me to another thing.  I think when you say it is the responsibility of everyone to kind of be prepared.  I would argue that is probably true for big players, big companies, but the vast majority of victims by now for ransomware for example, are small and medium enterprises that frankly just can't do it.

      And so they move part of the stuff to the Cloud because the big Cloud providers can do it, can help them.  That may be part of a solution.  You can't move a hospital device into the Cloud, that is not how it works.

      So I think we have a lot of questions here.  And I think ‑‑ I'm not sure who mentioned this before, mostly what States do is actually espionage.  I kind of start wondering if this really is true or if we include kind of state tolerated cyber operations and moving away from the espionage things.

      I find it is hard to believe that these huge ransomware Groups that claim to have hundreds of millions in profit can do so out of the state not closing both of the rights.  So I think maybe that is something that we need to address.  Because it is not only that they're criminals and the criminal oh, that is not really part of international law, blah, blah, blah.

      But it is also about arming of the people.  If you have criminals with hundreds of millions of dollars, you have a problem.  They is literally buy whatever they want. 

     >> STEPHANE DUGUIN: Serge, I'm going to Talita, to your last point here.  That is very problematic, that is the case.  It is not a scenario that we imagine this is the case.  We have today criminal Groups ‑‑ they don't have to be cyber criminals, but criminal Groups that are having income of small countries.  Please, Talita.

     >> Talita de Souza Dias:  Thank you, Stephane, thank you, Serge, thank you, Kaja for all of this.  I want to raise three points brought up in the previous discussion.

      So on data gathering, some of you mentioned healthcare and attacks against the healthcare sector, hospitals, things like that.  And so just one important thing to bear in mind when we talk about any kind of attack against the healthcare sector or a research facility is that whether you are talking about data gathering, whether you are talk about ransomware.  Even the slightest penetration into that network can be really compromising.  Especially when you are talking about research facilities.  Right?  Because for example, if you are doing clinical trials, even tampering with ‑‑ just invading the system might tamper with the data, with the confidentiality of results.

      So that might mean that the whole research is compromised.  And the same goes for example, patient records.  If you have the minimal intrusion into a database containing patient records, then, you know, you might just compromise the whole thing.  You have to build it all back.

      So the impact of the attacks on the healthcare sector is significant.  One thing that we learned when we were kind of like discussing these attacks against the healthcare sector, during the pandemic from an expert, a cybersecurity expert handling these issues within the University of Oxford.  Cyber-attacks against Oxford from Russia.  What he would say what keeps me asleep at night is the fact that they might just tamper with the data.  And all of our research on the AstraZeneca vaccine, for example, might just go to waste.  So it is important to treat the attacks differently.

      Because you have harm that might arise from minimal, minimal kinds of effects or operations.  So this is really important to treat those types of attacks differently.  In a way ‑‑ it is almost as if these kinds of targets are off limits, right?  When you talk about rules of war, States cannot attack hospitals.  And something similar kind of like happens in peacetime, when you are applying existing rules of international law.

      So on ransomware, ransomware is ‑‑ as Kaja said is more complicated and more problematic than I.T. supply chains attacks.  Because ransomware is almost per se illegal under international law, right?  Because the whole purpose of ransomware is to coerce the victim.  And that, in itself, that in itself might breach a number of rules of international law.  The coercion aspect, right? 

      The fact that you are threatening to delete someone's data or threatening to release someone's data.  So ransomware is almost inherently legal in international law.  This is something that we also discuss as part of the Oxford process.

      There is agreement in the academic community that ransomware is almost per say illegal.

      The other point I wanted to raise is when it comes to both ransomware and I.T. supply chains attacks and other cyber operations, for that matter, we must not only think about the obligations of the perpetrator state, the state that perpetrates the attacks or the States from where these operations emanate.  Like the whole state if you will.

      We must also think about operations of ‑‑ obligations of the victim state.  Because the victim state has obligations to protect their own citizens, right?  Under international law.  As Kaja was saying, simple things like adopting cybersecurity measures, you know, just monitoring their networks.  Simple things would kind of like allow these States to discharge their obligations to protect their own citizens and other private entities within their territories.

      So it is important to focus on the obligations of victim States as well.  The problem is, though, that we don't have enough guidance on how these states can discharge the duties, at least on international.  We have the norms of responsible state behavior.  But again, the norms are quite general, quite abstract, don't provide specific guidance.

      So I mean, the latest GG report that we had earlier this year talks about some measures of due diligence.  So for example, information sharing, investigation, which is good, but, you know, it is not enough.  So I'm a big ‑‑ I'm a big advocate for technical standards in this area.  So we do need more specific technical standards.  We need to bring the legal, the policy and the tech community together to put together specific technical standards that States and their cybersecurity teams must put in place to prevent these kinds of attacks.  This is something I'm a big believer in.  This is something we currently don't have. 

      We have institutions that deal with technical standards, so we have the ISO, we have the International Telecommunication Union.  But these fora, they're underused.  They have so much potential.  And we could do so much more with them. 

      So that's just a few thoughts on what has been discussed. 

     >> STEPHANE DUGUIN: Thank you Talita.  On the last point in terms of the recent report, there are indeed some measures, not perfect, to say the least, but at minimum, some of them can be measured.  That's also, I guess, the responsibility of Civil Society and other entities, this is some institutes that are measuring what steps we're doing out of what we're supposed to do.  We talked about what is measured.  There is an organization to reduce the space for criminal Groups.

      What does that mean?  A simple way to measure this is see cases open against criminal Groups, the number of times the requests were opened, the number of ‑‑ yeah, the scale of the judicial response when it comes to an investigation on cyber criminals.  So there are points for sure we're looking into to overtime, see what States discuss and decide as the rules of the road is helping in any way the community.  If not, what kind of recommendation can we give.

      A question in the chat.  Can we have a Special Rapporteur so all countries and Groups to report the civilians as a violation of human rights and digital rights and intervention in internal affairs of other countries?  So I'm not the one to appoint Special Rapporteur at the U.N. level.  We can discuss about it.  I don't know if ‑‑

     >> Talita de Souza Dias:  I can comment on this one.  There are already some Special Rapporteurs dealing with these issues.  I already mentioned the Special Rapporteur on freedom of opinion and expression.  It was David Kay before.  And under his mandate.  There was a specific report on surveillance and human rights where he reported these incidents of mass surveillance.

      So there has been action at the U.N. level at the U.N. Human Rights Council level on these issues.  But I think that's perhaps having a Special Rapporteur to deal with human rights in the digital age, more broadly, would perhaps be helpful.  What we have is different Rapporteurs dealing with specific human rights, right? 

      So that's the structure of the U.N. sort of like Human Rights Council. 

      But we don't have one Rapporteur just for human rights in the digital age broadly considered.  So perhaps that would help to kind of like understand these issues that are happening from a more holistic, integrated perspective.  Sometimes we cannot separate these different operations.  So sometimes they happen ‑‑ they're interlinked.  So for example, supply chains attacks are often connected with ransomware as was said earlier.

      Sometimes it is hard to disconnect the rights at play.  So for example, if there is a violation or interference with the right to privacy almost automatically it will impact the right to freedom of expression.  That will have a chilling effect.  For example, use of Pegasus against journalists.  If they suspected they were surveilled they couldn't freely speak out and do their jobs.  The holistic valuation of the rights in the digital age is helpful.

      Not within the U.N. but the ILC so the International Law Commission there are rumors that perhaps one day there will be a report on international law in cyberspace, including issues of human rights.  Fingers crossed, this will happen when the new members that have been elected this year take ‑‑ start their mandate in 2023, we might see this kind of discussion. 

     >> Kaja Ciglic:  Building on it, I would say so, you know, there are other ways to also drive accountability.  I think some of it has also been mentioned in the chat.  I think partly the concern that I have is, you know, States already make the information public when they want to.  It obviously would be great if they do it more often.

      I'm not sure a central Rapporteur would be a mechanism that would encourage them to do so. 

      There is lots of organizations in the Civil Society in particular that track some of these attacks already as well.

      I think a better ‑‑ I mean a better way forward, I think is to A, encourage Government to attribute attacks when they happen.  And also attribute them in ‑‑ attribute them A, to nation‑states when they happen in terms of nation‑states and prosecute criminals when it is criminal.

      No matter where they are in the world.  I think the other thing is too, when they attribute it to nation‑states to make sure they actually reference the laws the norms that were violated in the process.  Because I think that helps build the how this is applied that did we talked about in the beginning of this conversation.

      And in addition, I do think efforts like the citizen lab, and efforts by the private industry whether it is our own or other security researchers that make information available to collect and track some of the actors.  Don't attribute with the same powers that States have, but are able to point to specific actors, point to specific behaviors that are challenging.  And does drive greater transparency and understanding of what is going on.  So I would encourage more of that and more of sharing of those type of information amongst different Groups as well. 

     >> STEPHANE DUGUIN: Thank you.  Looking at the chat.  So this question was taken.

     >> Talita de Souza Dias:  Can I mention.  I forgot to mention the Special Rapporteur on privacy, who just took up her mandate in August of this year.  So that's Professor ‑‑ what is the name?  Dr. Brian from Paraguay.  One of the mandates is mass surveillance.  We have two Special Rapporteurs dealing with this head on.  The Rapporteur on freedom of expression and Rapporteur on privacy. 

     >> STEPHANE DUGUIN: Thank you very much Talita.  Looking at the chat, I think we addressed the question.  We have three minutes left, three speakers, so simple math, one minute each for your closing statement and what would be the messages to the audience.  Serge, starting with you. 

     >> Serge:  Thank you, Stephane.  Talita you have restored my faith in international law.  We have everything on the table we need to.  But I maintain States enjoy the fog we have in all of cyberspace and the international law in cyberspace.  I say that to say we should move forward and clear up the fog at least partially.  If we don't miss that, we can start to define the parts. 

     >> STEPHANE DUGUIN: Thank you, Serge.  Kaja? 

     >> Kaja Ciglic:  I think that we are out of time anyway, but like I said, I think there is definitely more to be done in terms of implementation and when it comes to implementation and holding actors across the board accountable for what they said they would be doing. 

     >> Talita de Souza Dias:  For me, I say what keeps me asleep at night is the danger of these kinds of operations affecting operational technologies.  So technologies that have a connection with the physical devices.  Like power plants.  That scares me a lot.  That is what scared me the most about SolarWinds is that window was left open.

      I think we need to focus more on prevention rather than remedying these kinds of attacks.  We need to think ahead.  We need to be worrying everyday about these things.  I believe ‑‑ I'm a true believer in the world of international law.  Not so much as an enforcement mechanism, but as an educative or educational tool to guide behavior.

      But we do need more guidance.  We need to bring different experts together to put together specific guidance in how international law and the norms of responsible state behavior must be implemented.  That is what I say. 

     >> STEPHANE DUGUIN: Thank you very much to all speakers.  To offer the fact that Cyber Peace Institute, we are available to be engaged with anyone with the data, knowledge on these topics.  This is something that we want to investigate resources to track, to trace, to analyze with collaboration and system level is mentioned.  It is mentioned the amazing work that is being done in this field.  This is known because of their work.

      This is the initiative that must be respected and used.  Thanks, again, Kaja, Talita, Serge.  I think we are on time, which is quite amazing.  And I learned a lot.  I hope it was interesting for everyone.  Enjoy your day, everyone.  Stay safe.  I heard there is a bug on there.  See you.  Bye