IGF 2016 - Day 3 - Room 1 - WS108: Empowerment through Quality Online Education

 

The following are the outputs of the real-time captioning taken during the Eleventh Annual Meeting of the Internet Governance Forum (IGF) in Jalisco, Mexico, from 5 to 9 December 2016. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid to understanding the proceedings at the event, but should not be treated as an authoritative record. 

***

>> MIKHAIL KOMAROV:  Your colleagues thank you for coming.  We will start in a few minutes.  So we can start.  Great, perfect. 

Yes, dear colleagues, thank you for coming.  For the last workshop I'm in for today.  Workshop on empowerment online education.  First of all I would like to say a few words about the reason we got this workshop.  Last year this is the anniversary of participation of my team from high school participating in the IGF and we are happy with that and it's our fourth year of organizing workshops, the IGF and we would also like to thank the community supporting that activity. 

So this workshop is a result of our long‑term collaboration, first of all with the panelists, with their relevance of the topic.  We started with mobile applications and so last year we discussed quality of online content and educational content and this year we are going further discussing empowerment for quality online education but also we will discuss connection to the local labor market, which is important top topic. 

I just would like to say a few words about co-organizers and panelists.  So if you don't mind, I won't spend time introducing your positions but I would like to introduce you. 

So first of all, co‑organizers of the workshop, it's Sarah Kaiden and Cristina.  And Ines.  And then here we have Olga Cavalli representing Argentina we have Cristina from Mexico actually representing and quite interest in the approach of criteria online.  We have Patrick Ryan from Google here.  My name is Mikhail Komarov and we have Andrey Shcherbovich.  Can we show the slides, please?  These are questions which we would like to discuss today with the participation of the audience as well.  Which policies should be developed ‑‑ I am quite interested in actually last question, all of them are quite important the terms of relevance and open resources and ranking to the local labor market.  But I don't want to spend time discussing my personal opinion.  I will have some time afterwards but I would like first to give the floor to Sarah Kaiden.  She's now in Africa.  And I would like actually to give the floor and to get your opinion and her answers on questions we raise together.  If you have connection with remote participant.  Sarah? 

>> SARAH KAIDEN:  Hello, can you hear me? 

>> MIKHAIL KOMAROV:  Yes, we can hear you.  Can you please describe your experience and your ideas actually on questions to be put to the workshop? 

>> SARAH KAIDEN:  Hello, everyone.  Good evening.  This is Sarah Kaiden.  I'm from Uganda.  I work at Christian university.  I want to thank everyone for joining us for this session today.  I believe it will be interesting as we talk about open education repositories.  It is 2:00 a.m. in Uganda but I'm happy that I participate in this session and thank you for facilitating this.  I wanted to on the issue of policies and quality ranking.  I would like to start by saying that quality control should be a collective effort.  So it should not be responsibility of anyone in particular, because there are so many people involved.  It's not a particular role.  It's not a particular team.  So in the spirit of multistakeholderism, I am talking about collaboration among different stakeholders, starting from researchers who put this piece of work together to ensure that the work is of good quality, that the citations are right, that references have been made correctly, that papers have been published using the proper guidelines, the people who review the papers.  Just make sure that the proper publication guidelines are followed to the detail.  Publishing houses who publish the content that's put in the repositories, so you don't just get quality from anywhere and put it anywhere.  You need to have some kind of guidelines to know this is the content we'll accept.  Maybe it should have been peer reviewed or it should have been reviewed by someone else.  We have institutional review boards who also have a huge role to play in this.  So it's a collective effort, you have researchers, technical communities or maybe social communities the software for the content and should be able to scale so that as the more content grows, you can still have the system running.  It should be stable, it should be secure.  No one should be able to hack into it.  As we develop policies, we need to look at all the stakeholders who are involved and like you can see, it's really a wide range.  So that's it for me for now but I'll be able to make contributions later on.  Thank you. 

>> MIKHAIL KOMAROV:  Thank you very much, Sarah.  I would like to add in terms of policies we discussed last year, policies for quality control of online education content and actually as a result of the previous year, we agreed that it should be collective responsibility.  So business set it's not necessarily to control the quality because a civil society will decide what's good and what's bad and they will rank good or bad content.  So that's what we agreed and what we agreed last year. 

I would like to Patrick to share his opinion on technologies, how technologies actually is changing their life from a business perspective from a private sector position. 

>> PATRICK RYAN:  Thank you.  I'm going to break the rules a little bit and stand up because I want to be able to see what's happening here.  I'm a recovering lawyer and a recovering professor and we like to hear ourselves talk and I want to be able to see what's going on in order to be able to speak.  I'll move through this very quickly.  I know our time is tight. 

So what we bring up the presentation, the perspective that I want to share with you is some of the thinking that we have at Google put into this in developing our own online platform.  Although I'm hoping to present this in a way that is not Google specific.  I'm not trying to sell you products and hold me accountable, this is not intend to be a Google‑specific thing but we believe other software providers should be involved in.  This is a lot of people trying to hack this problem.  The main problem is this, and I'll go into details in just a minute.  Teachers don't have time to teach.  If you look at the kind of work that they do, 30%, a little more than 30% of their time is spent teaching.  The rest is spent doing a lot of other tasks.  How can technology help with that?  How can it help with content, select it and deliver and lighten that burden so the teaching effectiveness is greater. 

When we think about this we realize you've got existing products.  This is our case, we have our suite of products any other company has a suite of products and you want to understand how people are using them now, and tie them in and really bring something together specific for educators.  They are a unique bunch, not necessarily the best computer users.  It needs to be simple.  We obviously know things are changing but if you look outside, the world is very different.  Even ten years ago we didn't have as much electronics as we do today.  In classrooms aren't changing.  This picture may not be fully reflective, but in 1940 everybody is sitting in a stadium style like we are right now.  And now people are doing the same thing.  There may be some more commuters computers, but for the most part it's the same kind of interaction.  Outside of that there's a world that's changing and developing and adapting.  So we talked to a lot of teachers and what I wanted to attribute today is some kind of experience.  There will be some quotes here that we won't be able to go into too much detail on because of time.  Lesson planning takes almost two hours.  Grading and evaluation takes almost two hours, printing takes almost two hours and there's a lot of printing that takes place.  E‑mail takes about an hour and a half every day.  E‑mail related to students and talking to teachers and admins takes hours every day.  Communicating with parents, this is the case for elementary education, communicating with parents is a big part of it and assignments are still for the most part around the world, for the most part handed in by paper.  Teachers have to put the whole thing together by themselves.  IT companies like ours, we're not alone Microsoft and other companies are looking at ways to integrate in order to make this a better solution for educators and for students and that's a really good thing.  This is a great example because this is a photo that we took of somebody trying to explain how to in a second grade class how to get Google docs into Google drive and it's a five or six steps and it's really complicated and that's a bad experience.  You have to add now to the time that teachers are already taking talking to students, talking to parents, collaborating now they have to explain how to use the tools.  This is not their strength.  We need to make that easier.  We think generally open technology can.  And this is something that a lot of different people are trying to crack and the way that we're looking at it is by taking up software platform, thinking about how the devices are that interact and then the question of content.  The point here I think for that point is that a lot of content stuff, which has been a very important point of our topics in the past, is driven a lot by the feedback loops.  And there's basically three things that we think teachers need in order to have more time.  There's really just three things.  The first is to improve their communications.  Make their communications more effective.  To be able to make announcements and to be able to have one to many and exchanges that can cut out a lot of that regular time and make it more effective.  When you tell one student something you need to tell other students, sometimes it's a matter of having a platform where you can have that conversation all together.  The second thing is to collect and create assignments.  This is really difficult both for students and for teachers but to be able to have that done in a uniform way and to remove the pain from having to explain what you need to do in order to save things.  And finally, being organized.  I mean I have difficulties being organized in life anyway.  But I would be nothing without electronics.  In the school world we need to organize things a little bit differently offering some tools and thoughts about how to do that we think is important.  The way we did this, we started with a test.  We started with one classroom and trying to see how they learn and we talked to students and got the feedback from what theory experience was.  And the feedback is the most important thing.  This has been an ongoing theme for all of our workshops here.  Right, Mikhail?  We can talk about who gives the feedback, there's a lot of places for it but feedback is really important and it's an iterative thing and it's something that has to fin and it's something that involves technology.  The way people are consuming the information today, maybe by a combination of PDFs and videos and other formats, but it you will is run by a software platform, each university, each school has its own team that does it.  So we have a number of learnings that we've taken from our experiences here about the flexibility, the assignments needing to be simple to use, the teachers needing to use them.  These are all basic things.  And there is a number of things now that everybody wants is more.  Which is good.  When you get feedback from people, the problem is they give you feedback.  So we need to integrate it into the things that we're looking at is the things to be able to integrate more.  All we're asking for is better, tighter integration make it easier so we're working on a lot of those things.  And I'll just run through some of these things quickly.  I'll be able to leave some of these slides behind but we have a lot of quotes that we have from teachers about how fabulous an online enablement can be.  This isn't necessarily tied to the type of product that we offer, although it was done in that context.  And feedback from students is similarly very strong.  The challenges getting from the first launch of a technology, getting people to adopt it and getting people to use it, and our journey started small with one and we just kept on multiplying it to multiple teachers and rolling it out you've got a whole team before you know it.  So I've got a lot of quotes in here.  I'm going to leave these slides behind so we can run through it and I think we're tight on time but thank you for listening. 

>> MIKHAIL KOMAROV:  Thank you very much, Patrick.  I think you totally covered your position in terms of benefit of open education resources and technologies. 

I would like to go also through some key I would say core issues of rights for education and problem 23 will cover the topic.  Just give us a small historical review of human educational rights issue.  After this I hope we will continue with questions. 

>> ANDREY SHCHERBOVICH:  Okay, thank you very much.  At first I would like to refer to the universal declaration of human rights.  This declaration promotes education for everyone.  And this declaration also that promotes that this education should be involved human rights education.  Education about human rights.  So also the provision.  That education should be directed for the development of personalities respect rights of fundamental freedoms.  Tolerance, avoid discrimination issues and other things. 

So we have not enough documents from international level, which providing human rights and freedoms for the Internet users, but I would like to refer to recommendation of the council of Europe that it is guide to human rights for the user.  And the educational right is also one of the key points for this recommendation.  Online access to educational culture of scientific of other content in its own languages that's important that have to preserve multilingual atmosphere within the Internet. 

Also there is issue of media literature, information literature for that we need to understand how to use technology and use technology to promote education in fear of human rates. 

Now I would like to refer to the charter of rights and principles for the Internet.  The authoring principles of the Internet.  Everyone has a right to education.  Everyone has right to be educated about the Internet and to use the Internet for education.  This also involve virtual learning, multimedia issues, publications access to textbooks and other materials on free or local basis without any kind of discrimination.  For everyone.  It should be ‑‑ also it's proclaimed everyone should be educated about the Internet.  It's also raising awareness for human rights.  Also digital literacy. 

Well, I have some problems which I would like just to mention.  It's about recognition of online education as commodity a public good, recognition of academic degree received by the Internet and about some standardization of online learning.  And now I would like shortly some best practices.  First of all United Nations cyber school bus.  It's a good example of human rights education on the Internet.  We started at 1996.  Here as well we can see the amnesty international online courts on human rights.  This is a platform for online education about human rights issues.  And the last but not least to have a center in Russia which also provide online platform for human rights education.  It's used by education, for promotion rights to education for everyone. 

I think that's all my presentations.  Thank you very much. 

[ Applause ]

>> MIKHAIL KOMAROV:  Thank you.  Andrey also presented some interesting problems I would say which are also mentioned in our questions which we hope to discuss with the audience. 

So I would like to actually to ask all Olga, what do you think in terms of quality of education with the help of resources and other types of way to provide education to the people of a civil society.  So what do you think about the quality?  Do you think open education resources lower the quality or does it increase the quality?  What do you think? 

>> OLGA CAVALLI:  Well, thank you first for inviting me.  It's this how many years?  The fifth anniversary? 

>> MIKHAIL KOMAROV yeah, I would say fifth. 

>> OLGA CAVALLI:  I think I have participated.  Sometimes it overlaps with other activities, thank you very much and I always do my best to join your nice initiative. 

I was thinking about before I came to my workshop my experience as a university teacher for many years, 20 years so far and talking about policy and any kind of regulation.  And I found that some programs, for example, at the national university level in Argentina they are kind of fixed.  You have to go through a very strict process of national authority in saying yes or no to the program.  So I wonder if that is even worse than having some feedback from the community in for example some online training courses.  So I would say that it is not one size fit all.  There is I think that every online tool or online content could benefit from the feedback from the community.  From both, from students, for those who use the content, and also from colleagues.  In my university we have feedback from the students.  And it's funny because of course those teachers that are more perhaps not so strict they get the best comments and those that are more strict they got horrible comments.  So it's not only about students.  It's also about other colleagues revising the content.  But I would not be much in favor of regulations or strict rules that that may diminish the value of the content with the pass of time.  This is one comment. 

You make a very interesting point about labor market.  I would like to stress the fact that in Argentina ‑‑ Argentina was the first country to propose the concept of tele work.  And when we had that, it was request from our former ministry of labor at that time, and we were kind of a frayed of proposing that in 2005 and we did that and it was well‑received by the community that day.  So now it's part of the world Summit of information document.  What I mean with that is since then, Argentina has developed a series of online training for some certifications, not necessarily strict university teaching course, but some trainings to have some, how can I say in English?  Skills or of different things to do.  But you have some certifications that could be managed online.  So I think that that is very useful.  And about the global south.  Global south has the Latin American perspective of global south is maybe a little bit different from other global south in Africa.  What we have in Latin‑America is an extreme unbalanced distribution of everything, which is the most balanced distribution all over the world.  So if you go to Mexico, you have areas of the city that are like Manhattan.  You have high connectivity, online training, computers, nice houses, nice cars.  Not far away from there you have neighborhoods that are not connected or people have no access to the basic parts of the infrastructure needed for education.  So that challenge of Latin‑America is building an infrastructure that is more equal.  Balancing those inequalities.  And that means infrastructure for access, infrastructure for access energy, roads, intimate.  Then once you have the infrastructure and you have the access, then you will have our in balance in relation with content.  Then you will have the challenge of the language.  I teach, when I teach technology in my university.  I'm an engineer.  When I bring papers in English, it's like I brought something from hell.  Now why English?  No we speak Spanish.  Well, guys, we are learning technology.  You must skill to have those documents.  And that is a big discussion.  So content in other languages is a problem.  We organize the school of Internet governance every year and since day zero I insisted in having translation.  Simultaneous translation in English and Spanish.  They told me it's extremely expensive, you're wasting your money.  People will understand English.  And no, that doesn't happen.  So language is a very, very big barrier and in Latin‑America you have several languages though you think it's only two, Portuguese and Spanish but you have the Caribbean.  I would say language access, I would not favor any regulation and I would favor feedback from the community, but not only from students but also from colleagues in having some mixture of feedback to know if the training and the contents are fine. 

And finally, and of course I can comment on other things.  There is kind of a fantasy that online training can be done without the need of time.  That is a fantasy.  You need the same time or maybe more time to do the online training and also the training that you get in the classroom.  So I will stop here and maybe I can comment further then.  Thank you very much.

>> MIKHAIL KOMAROV:  Thank you very much for the interesting comments.  I guess we will continue with discussing these questions.  I just wanted to come back to the initial question in terms of education versus the quality of education with the help of open education resources and just briefly describe set of criteria which were developed in Russia this year as the initiative of institute of Internet development to assess open education resources in online courses.  So we will also probably get some feedback in the information about Mexico with the same experience, I guess.  So could you please show that slide?  Great. 

So these are our questions.  And yeah.  And these are criteria.  So I didn't actually ‑‑ I didn't describe them in details, but what I would like to say in the world when we have different types of educational resources available different levels, so we'll starting from public schools, going to university education of different levels, professional certificates and some skills training or lifelong education in general, different programs, so in Russia we got an initiative to actually somehow to rank the courses in order to help students to choose courses.  And so we were focused as a proposal of another organizer, she proposed to focus on hotel star ranking.  We use that ranking in order to choose a hotel.  You think it's not appropriate.  It is appropriate.  Why?  Because we know how actually each course can get the star and for what actually, right?  So we know for the hotels if they got 3 stars or 4 stars, so what they should have, what are the requirements for that.  And so we got a set of at least of criteria available so we started with quality of service itself, some internal characteristics of their educational resource.  So if you have individual study plan what about opportunity to have contact with the lecturer or author of the course and some other characteristic.  Another set of criteria, it's a level or quality of author of their massive open online course or open indication resource.  So organization of their author, set of publication, and so on.  Quality of software platform.  More I would say technical issues of the platform, like ergonomics, security, reliability and so on, if the support mobile, if it doesn't and so on.  Quality of content but while I'm talking about quality of content I'm just talking about, you know, some basic measurements, like if it's unique content or not.  What about the connection to the labor market.  I mean, if you have industry in the country where that knowledge might be applied or what about practical issues of the content, right?  And so on. 

And the last set of criteria service availability.  Service availability in general we are talking about online course or open indication resources.  In terms of pricing, they are not free.  If you need a certificate, you have to pay probably.  If you don't need a certificate you don't have to pay if you want to pass some test.  So in terms of requirements, basic requirements in order to understand the content and so on.  And as a set of criteria there.  So we haven't began a process of discussing that community in Russia.  But the general approach is quite similar to the hotel star ranking whether you know for 3 star and of course you know there's feedback from the students, feedback from other experts.  From other lecturers or authors if you're talking about education.  And we know that for 2, 3, 4 stars you know that service, that open line course, that open indication resource should actually be sufficient and should be ‑‑ should satisfy those requirements which are somehow indicated.  And so then students can easily choose what does he want and what's the difference between two courses.  The problem is he didn't have different level of authors but they center great feedback from the audience.  So they will spend their time, I appreciate you said about timing.  Because if you are talking about online education it does mean we can actually we are able to get highly professional or highly skilled in a very short period of time.  Should spend more time in order to deeply understand the knowledge in the content.  That's what I wanted to share with you in terms of set of criteria and actually I would like to get position of Cristina in terms of Mexico.  I heard you've got special online platform in Mexico. 

>> Yes, thank you. 

>> CRISTINA CARDENAS:  I'm Cristina I'm from Mexico.  And I want to explain you first why we work from this department and we want together in consistent to make the public policy works.  We have been delivering more than 2 million devices during the last two years and right now we have stopped to think about what we have done.  We devices to students no more than three or four months and have content that they don't need anymore and the content was charged in the devices that we deliver.  So it was a problem because we never think to have the content in a platform to share to all the students to different levels and we create content only for the fifth grade because that was public policy at that time.  Delivering devices in fifth grade. 

So what we are doing right now ‑‑ so we decide that first we need to have a goal to know what we want to do.  And to select the content and to know what we are going to evaluate and to know the whole policy we need to have first a goal that everybody understands and they know why we have technology at schools.  So we are based on the framework ‑‑ we want to know the kids how to collaborate, how to communicate, how to solve a problem with technology.  So doing that we want to have all the public policy related to that and we also have the idea that technology is not only for a grade, it's for all grades and for all subjects and that's why we are going to work with different companies.  For example, if we want that teachers just the content that we are going to select, we need to tell them and we need to train them and we need to let them know why we select that content or when do they need to use it or which are the short examples that ‑‑ the shore moments they can use technology.  As we mention do it online doesn't mean less time.  In fact needs more time, needs more training for the teachers to use it.  And in many schools as we already observed during the last three years that we run pilots before changing the whole public policy in the federal government, we observe teachers don't use it more than three hours per week.  So we need to help them to understand why we delivering devices, why we are giving them content.  So teachers training is one of the components.  Another important thing is that they need content as we have mentioned.  And we were working only in the fifth grade and we were creating the content in the ministry of education.  And while we were working on these pilot programs with the industry and civil society and academics and international organizations, we realized that there are many free content available and that everybody is doing something related with technology, education, content, and we decide that we stop production of content and we are gathering content.  And doing that we have the same problems that we've mentioned before, the criterias for selecting the content and that is now our work.  We are not creating, we are curing contents, electing and classifying the content for all the grades, not only the fifth grade that have a device, because we also observe that during the last years, the students that use technology are not the kids who are having the devices delivered by the government, but they are using through the smart phones or the computers that they have at home.  So education is not only in the school but we also saw that the teachers want to use technology most of the time they train, not in the school but outside the school. 

We also have another section I will mention later we create that encourage teachers and parents kids to use technology and many of the fields that we were working in the pilot programs also show us that they have a lot of initiatives for encourage the teachers to use technology.  For example, there is the price for the best teacher using technology of Microsoft.  Or Google.  Google has a lot of many prizes for teachers or students for best practices.  And while I was invited to these practices, I saw that most of the teachers that go there from Mexico were from private schools.  And it is because many of the teachers who understand that they are many practices like that needs to go to the side, first they need to know that there are prizes.  They need to go to the site and they need to go to the special programs, then they go to the classification for the subject they are teaching.  So there are so many clicks before they are arriving that there is a prize that it becomes so difficult to find them.  And we are also selecting here some of the initiatives and we are like in a calendar showing to the teachers and make it so it's more easy to find them.  Just like a race, whoever runs knows if you're looking for a race, then there are many sites that help you to find every race for a weekend and it's easy to find.  We want do the same for prices and other things.  And then we put connectivity because it's not less important.  In many of the schools we don't have connectivity, we don't have equipment and in that we are doing another public policy that it's a long process but we are renting equipment.  We don't want to worry for the replacement of the equipment.  We want that the school selection consider connectivity, not because it's fundamental for learning with the use of technology, but for monetary and for content and for tracking what are the teachers are doing and the training that we provide them whether it's good enough or not and also we want the teachers to start to create and share experience and work in that field. 

So this is how the platform looks.  It's very simple.  We launch it last month.  You can get inside ‑‑ it's prende.mx. 

We have this section, the yellow one is for teachers.  The green one is for content and the other one is for the initiatives that I mentioned before.  The one in yellow it's for teachers but we can find it's for teachers to understand what are the digital skills that we want to develop with them, certifications as we mention before, what are certifications that exist from different ‑‑ we have some tools that the teachers can use.  It's a one‑top site for the teachers to find tools to teach and to learn how to implement technology.  So this is a step by step and different levels and the teachers can use it at any time at any hour.  Of course we need to guide them more.  Some of the teachers will need more help, but the main goal for this section of the platform is to create nets.  We observe that many teachers work better when they have a communication among them.  So we are running a pilot program right now with Facebook and we want that the teachers start to communicate and share experience of how they use technology in a simple way.  And we are only maybe ‑‑ we are only intervening ‑‑ we are only participating as a moderators to know that the content that they upload is not incorrect, or not proper for the group that we want to have. 

So general speaking, the section is work for teachers and how to use technology. 

The other side is the side of resources, open resources.  We have for every participant in the project, we ask them that they gave us the rights to have them upload and the students can also download the content.  And most of these content has been donated.  We haven't made any peso for the content.  We already have 2,000 upload and that you have an example, one of the biggest companies of telecommunications in the company translated 5,000 contents in Spanish from Khan academy.  So we are going to have them in their site but with the link in our site.  That they can look for it.  And it's Telefonica, it's Google, it's Microsoft, so we have many resources and we are very slow uploading them because we need to have these agreement, legal agreement and that's why we are slow.  But we consider that with the fears that have arrived to our office, we will have a huge platform content and our problem will be curating the content more than worry to create the content. 

Just to have some data right now.  The problem is not having the platform upload.  The problem right now that we are having is that we need to reach the teachers.  That they understand what the platform is.  And this is another session.  Thank you very much. 

>> MIKHAIL KOMAROV:  Thank you very much. 

[ Applause ]

Thank you very much for such a detailed description.  And also the teachers should be trained for using technology and how much time and who's going to pay for that but that's a question also.  I would like to come back to our questions actually.  In order to describe the author, at your platform you also don't have a link to the local labor market.  I mean you've got companies and private business translating the content but not telling where that content should be applied, right?  It is quite interesting.  The question why a question of linkage, a question of connection between educational resources and can you please show the questions?  And labor market.  Yes.  You know, if the government supports open education resources or national platforms.  So then if our students attend different let's see courses, but government knows that knowledge wouldn't be applied in the local labor market, what the benefit of the government supporting that activity.  From the global perspective, and from the globalization, let's say approach, we understand that there's grade for people being able to get education in different areas of different level, different content.  And applied wherever you want.  But for the perspective of government, for instance, it's a big question.  Because if the government structure cannot apply the knowledge you got in your country, then actually obviously they supporting you leaving the country, right?  And that's an interesting question.  I know that that question wasn't raised so I hope that the question of connectivity of education resources will be raised somewhere at their global level, right?  Because it's a global question.  But I would like to first of all ask if you have questions from remote participants or remote speaker.  I don't know if Sarah would like to add something.  Otherwise I would like us to spend more time than was planned because we started 15 minutes late.  And so to spend 10 more minutes just to discuss those questions and to ask the audience what do they think about the actions.  So if you have any questions from remote participants?  No.  Okay. 

So please, yes?  And also, could you please name yourself and the organization and could you please show the questions, if possible. 

>> AUDIENCE:  My name is ‑‑ I'm from Brazil.  And I work with ‑‑ [ indiscernible ] because in Brazil we realize me and my friends have a lot of opportunities of online learning but we don't not know how to find this.  So we got this ‑‑ [ indiscernible ] and we and scholarships and also from online learning and we started a goal by Facebook and now we created a website and I coordinate 75 ambassadors.  Because we want them to actually reach people and to talk to them and to make people understand that we can use online learning.  Because in Brazil we still have a lot of prejudice with that.  People use Internet just for social media.  Because it's not very common seeing that you're learning something online it's quite hard.  So actually why the lecture ask about the quality rank on the Internet is pretty much how do you think we can get enough to work with that flexible in the action that will work with my friends, having support from schools, government and things like that.  Because we started by ourselves because we have opportunities of scholarships and we want to share that.  But if you go to places in Brazil to talk about that is really difficult because online learning is really hard in a country where people just go to a school that not have chairs.  So you know, people don't get access to Internet.  Myself I have access to Internet when I was like 14 years old and got a computer so much later.  So is it still like that.  So what I want to ask you is how can we work Internet into countries like that and how can we use this platform and make this accessible to everyone.

>> MIKHAIL KOMAROV:  Thank you very much for your question.  First I would like to address the question to Olga. 

>> OLGA CAVALLI:  I fully agree with you.  And yes.  There are schools without chairs, without electricity, and sometimes without a roof.  It's not a joke. 

So there's a big imbalance in our countries from very wealthy areas.  Some parts of the cities, rural areas or parts of the cities that are not rural are urban but at the same time have a big lack of infrastructure.  That is a big problem and it's a big challenge.  My suggestion, and we have been thinking about and we are thinking about doing a project starting in 2017 with a chapter where I'm a president in Argentina is trying to work with the government and some universities in building some small cases of not very expensive and sustainable connectivity for especially rural schools, starting with the school as the focal point of the community.  And we are working with the government because there are some university to see which is the best technology solution, perhaps there are different options.  We mainly think about wireless solutions and build a small case and try to explain it and share the knowledge with and replicate it in some other local places.  And we think about the school as this place where the community gathers and so I'm fully optimistic of our young people.  I think that kids and young people will grab that very quickly.  Much more than teachers which is a challenge also.  Because sometimes teachers don't know what to do and they feel threatened by the technology.  So this is what we are thinking about.  And the government we talked to the government and this is very, very premature.  We have no study, but it's an idea.  And if it works, I can share that with you.  With a nice group of young people.  And then when it's ‑‑ I think content is available.  Then we can gather some good training courses and also I would like to share with you that the ministry in Argentina is training online programmers.  We have a big lack of ICT training professionals not only university level but sometimes certification level.  So the industry is not growing much because we don't have people to work in it.  So these are two ideas that I wanted to share with you. 

>> MIKHAIL KOMAROV:  Thank you.  Patrick? 

>> PATRICK RYAN:  Well, there were two questions.  One was feedback and the other is how do you get this to people who don't have electricity at some level.  And that second question is one that I'm very passionate about because I'm not going to talk about it now because I'm so passionate about it.  It's so rich of a topic.  It's an important thing and there are people like the U.S. in very rich places thinking about how to get the Internet to people who don't have it and there's a problem there and we need to fix it.  One thing I want to talk about that I think is really interesting is the idea of institutionalizing feedback and I want to give one example that I think is just fabulous.  Everybody here is I'm sure taken an Uber at one point or another or a Lyft.  For them their content is their driver.  And getting' valuation from feedback is paramount.  They need to know from everybody whether drivers are good and bad.  The way they do that is by installing an annoying program that bugs me, because I have to rate or everybody who takes a driver I don't do it ever I run to the thing but it won't let me log back in until I rate.  I've got to stop and think about it.  I'm usually pretty good at rating people.  But I've had a few bad apples.  In Singapore took me on a 45 minute tour.  The bottom line is institutionalizing feedback is really important.  We don't do that in an educational context.  When you give students an option for feedback.  It's hit and miss.  Institutionalizing that forcing them to have feedback I think is one of the most important things for us to do.

>> MIKHAIL KOMAROV:  Thank you very much.  Probably we will go to another question because Cristina described very detailed how system should work in Mexico.  I don't know if the description answered your questions, but after the session if you don't mind.  I would like to get questions or comments from the audience if you have any other questions and your opinion about topics we had discussed.  Any questions?  Comments? 

>> AUDIENCE:  High quality courses so what is budget a high level, university level committee to evaluate the courses.  And then it's our ‑‑ our university have a similar high quality courses presented.  Thank you. 

>> MIKHAIL KOMAROV:  Thank you very much.  I guess universities you know also from Russian perspective, our university we also have to encourage teachers first of all to create some the content.  Because it takes much more time to create online course than just order course because you have to use those technologies.  So you had also another question or comment? 

[Away from microphone].

>> AUDIENCE:  Well, this kind of question changed my life.  I began to study learning my high school and now I am online and the university in Guadalajara online.  I've been this about the benefit.  I think it's for adults is another opportunity.  I have a classmate from ‑‑ how you say?  Other places.  And it's changed our lives.  Just want to say it's a benefit.  Thank you. 

>> MIKHAIL KOMAROV:  If you don't have any other questions, so I will probably just conclude with what we discussed.  I don't know if you have any questions from remote participants?  No.  Okay. 

All right.  So today you know, the key issues of this workshop was continuation of the topic of empowerment through quality online education.  The thing is that we got a presentation from Andrey in each country education and online education might be different in terms of regulations.  And actually even though we have United Nations description of online education opportunities for all the people to get the education through the Internet, in each country we have different regulations.  And different policies.  And actually different descriptions of online education.  Somewhere it's distant learning, somewhere we are talking about online courses and so on.  So what we wanted to raise here, that even if we have such different world, in terms of different descriptions and definitions of online education, and different conditions, right, for that, I appreciate the thing about imbalance in terms of infrastructure, depending on the regions, right?  So we anyway should give the opportunity to everyone to get their education through online services.  But we should also somehow guide them how they should choose actually proper content or proper courses.  And that's why we have those questions and topics about whether it should be measured somehow, whether they should be some criteria proposed and so on.  In Russia as I said we proposed instead of criteria next year some of the courses will be deleted even from course areas as well.  We will see the feedback which really should be institutional.  In terms of evaluation of the approach.  And but you know, the question of connection between labor market and education, in traditional education systems, we have business we have companies, we have private sector participating as a partner of the educational process.  The education, the online education we have private sector funding development of content, development of different courses.  But sometimes we still don't have the connection between the content, between the course and applications in local labor market.  And that's another question probably to the global organizations, how it should be done, how actually, how it should be implemented, right?  And so what I also wanted to say that I think a great example of educational platform for Mexico where government it all started creating content itself and then also connected different communities, right?  It's a good example how the first step anyway should be introduced.  Because it's going not from the community which sometimes as we said should be dedicated how to use open line resources, how teachers should create the content, but also it should be introduced by the government wishing to expand, wishing to give the right of education to everybody even through online services. 

And I also you know wanted to again to emphasize that languages and content and different languages is also a big problem.  And so we should also think about the same evaluation or criteria but involving local communities to evaluator rank the courses and so on.  As we have with the hotel stars ranking.  And I think afterwards if you can think about that approach, if it would help you to choose courses or not, so and probably to share your opinion after even after the seminar.  And, yeah.  So thank you very much.  And we will have the report afterwards, the IGF website and I hope that we will have more focused discussion on one of those questions.  Probably next year the workshop next workshop or next session.  I support it.  So thank you very much. 

>> AUDIENCE:  One moment.  Tomorrow there will be a whole delighted session on conflicts and Internet governance it will be in the launch time. 

>> MIKHAIL KOMAROV:  Thank you very much. 

[ Applause ]

[ Session Concluded at 6:15 ]