Tackling the 3D Simulation League: an interview with Klaus Dorer and Stefan Glaser

A screenshot from the brand new simulator that will probably be trialled for a particular problem at RoboCup2025.

The annual RoboCup occasion, the place groups collect from throughout the globe to participate in competitions throughout numerous leagues, will this 12 months happen in Brazil, from 15-21 July. In advance of kick-off, we spoke to 2 members of the RoboCup Soccer 3D Simulation League: Executive Committee Member Klaus Dorer, and Stefan Glaser, who’s on the Maintenance Committee and who has been lately growing a brand new simulator for the League.

Could begin by simply giving us a fast introduction to the Simulation League?
Klaus Dorer: There are two Simulation Leagues in Soccer: the 2D Simulation League and the 3D Simulation League. The 2D Simulation League, because the identify suggests, is a flat league the place the gamers and ball are simulated with simplified physics and the principle focus is on staff technique. The 3D Simulation League is far nearer to actual robots; it simulates 11 versus 11 Nao robots. The degree of management is like with actual robots, the place you progress every motor of the legs and the arms and so forth to attain motion.

I perceive that you’ve been engaged on a brand new simulator for the 3D League. What was the concept behind this new simulator?
Klaus: The goal is to carry us nearer to the {hardware} leagues in order that the simulator will be extra helpful. The present simulator that we use within the 3D Simulation League is named SimSpark. It was created within the early 2000s with the goal of constructing it doable to play 11 vs 11 gamers. With the {hardware} constraints of that point, there needed to be some compromises on the physics to have the ability to simulate 22 gamers on the similar time. So the simulation is bodily considerably sensible, however not within the sense that it’s simple to transpose it to an actual Nao robotic.

Stefan Glaser: The thought for growing a brand new simulator has been round for a couple of years. SimSpark is a really highly effective simulation framework. The base framework is area impartial (not soccer particular) and particular simulations are realized by way of plugins. It helps a number of physics engines within the backend and supplies a versatile scripting interface for configuration and variations of the simulation. However, all this flexibility comes with the value of complexity. In addition to that, SimSpark makes use of customized robotic mannequin specs and communication protocols, limiting the quantity of obtainable robotic fashions and requiring groups to develop customized communication layers just for speaking with SimSpark. As a results of this, SimSpark has not been broadly adopted within the RoboCup neighborhood.

With the brand new simulator, I want to handle these two main points: complexity and standardization. In the ML neighborhood, the MuJoCo physics engine has turn into a very fashionable alternative for studying environments after Google DeepMind acquired it and launched it open supply. Its requirements for world and robotic mannequin specs are broadly adopted locally and there exist a whole lot of ready-to-use robotic mannequin specs for all kinds of digital in addition to real-world robots. In the center of final 12 months, they (MuJoCo) added a function which lets you manipulate the world illustration throughout simulation (including and eradicating objects to / from the simulation whereas preserving the simulation state). This is one important requirement we now have within the simulation league, the place we begin with an empty discipline after which the brokers join on demand and type the groups. When this function has been added, I made a decision to make a step ahead and attempt to implement a brand new simulator for the 3D Simulation League primarily based on MuJoCo. Initially, I wished to begin improvement in C/C++ to attain most efficiency, however then determined to begin in Python to scale back complexity and make it extra accessible for different builders. I began improvement on Easter Monday so it’s not even three months outdated!

I feel it could be helpful to elucidate slightly bit extra concerning the setup of our league and the necessities of the simulator. If we take the FIFA sport (in your favourite gaming machine) for instance, there may be one simulation occurring which simulates 22 gamers and the choice making is a part of the simulation having full entry to the state of the world. In the 3D Simulation League we now have two groups with 11 robots on the sector, however we even have 22 particular person agent softwares that are related to the simulation server, every controlling one single robotic. Each related agent solely receives sensor data associated to their robotic within the simulation. They are additionally solely allowed to speak by way of the server – there isn’t any direct communication between the brokers allowed in Simulation League. So we now have a common setup the place the simulation server has to have the ability to settle for as much as 22 connections and handle the state of affairs there. This performance has been the most important focus for me for the final couple of months and this half is already working effectively. Teams can join their brokers, which can obtain sensor data and might actuate joints of the robotic within the simulation and so forth. They are additionally in a position to choose totally different robotic fashions in the event that they like.

An illustration of the simulator set-up.

Presumably the brand new simulator has a greater illustration of the physics of an actual robotic.
Klaus: Exactly. For instance, how the motors are managed is now a bit totally different and far nearer to actual robots. So after I did my first experiments, I noticed the robotic collapse and I assumed it was precisely how an actual robotic would collapse! In SimSpark we additionally had falling robots however the motor management within the new simulator is totally different. Now you possibly can management the motors by pace, by power, by place, which is far more versatile – it’s nearer to what we all know from actual robots.

I feel that, at the least initially, it will likely be harder for the Simulation League groups to get the robots to do what they need them to do, as a result of it’s extra sensible. For instance, in SimSpark the bottom contact was far more forgiving. So if you happen to step arduous on the bottom, you don’t fall instantly with a SimSpark robotic however with a MuJoCo robotic this will probably be far more sensible. Indeed, in actual robots floor contact is considerably much less forgiving.

I had a query concerning the imaginative and prescient side – how do the person brokers “see” the place of the opposite brokers on the sector?
Stefan: We simulate a digital imaginative and prescient pipeline on the server facet. You have a restricted discipline of view of ±60° horizontally and vertically. Within that discipline of view you’ll detect the top, the arms, the ft of different gamers, or the ball, for instance, or totally different options of the sector. Similar to widespread real-world imaginative and prescient pipelines, every detection consists of a label, a course vector and the gap data. The data has some noise on it like actual robots have, too, however groups don’t have to course of digital camera photographs. They get the detections instantly from the simulation server.

We’ve beforehand had a dialogue about shifting in direction of getting digital camera photographs of the simulation to combine into the imaginative and prescient pipeline on the agent facet. This was by no means actually sensible in SimSpark with the implementation we had there. However, it ought to be doable with MuJoCo. However, for the primary model, I used the identical approach the standard simulator handled the imaginative and prescient. This implies that groups don’t want to coach a imaginative and prescient mannequin, and don’t have to deal with digital camera photographs to get began. This reduces the load considerably and in addition shifts the main focus of the issue in direction of movement and resolution making.

Will the simulator be used at RoboCup 2025?
Stefan: We plan to have a problem with a brand new simulator and I’ll attempt to present some demo video games. At the second it’s probably not in a state the place you possibly can play a complete competitors.

Klaus: That’s normally how we proceed with new simulators. We wouldn’t transfer from one to the opposite with none intermediate step. We can have a problem this 12 months at RoboCup 2025 with the brand new MuJoCo simulator the place every taking part staff will attempt to train the robotic to kick so far as doable. So, we is not going to be taking part in a complete sport, we received’t have a number of robots, only a single robotic stepping in entrance of the ball and kicking the ball. That’s the technical problem for this 12 months. Teams will get an thought of how the simulator works, and we’ll get an thought of what must be modified within the simulator to proceed.

This new problem will probably be voluntary, so we aren’t certain what number of groups will take part. Our staff (MagmaOffenburg) will definitely participate. It will probably be attention-grabbing to see how effectively the groups carry out as a result of nobody is aware of how far an excellent kick is on this simulator. It’s a bit like in Formula One when the principles change and nobody is aware of which staff would be the main staff.

Do you might have an thought of how a lot adaptation groups should make if and once you transfer to the brand new simulator for the total matches?
Stefan: As a long-term member of 3D Simulation League, I do know the outdated simulator SimSpark fairly effectively, and know the protocols concerned and the way the processes work. So the primary model of the brand new simulator is designed to make use of the identical fundamental protocol, the identical sensor data, and so forth. The thought is that the groups can use the brand new simulator with minimal effort in adapting their present agent software program. So they need to be capable of get began fairly quick.

Although, when designing a brand new platform, I want to take the chance to make a step ahead when it comes to protocols, as a result of I additionally need to combine different Leagues within the long-term. They normally produce other management mechanisms, and so they don’t use the identical protocol that’s distinguished in 3D Simulation. Therefore there must be some flexibility sooner or later. But for the primary model, the concept was to get the Simulation League prepared with minimal effort.

Klaus: The large thought is that this isn’t simply used within the 3D Simulation league, but additionally as a helpful simulator for the Humanoid League and in addition for the Standard Platform League (SPL). So if that seems to be true, then it will likely be utterly profitable. For the Kick Challenge this 12 months, for instance, we use a T1 robotic that may be a Humanoid League robotic.

Could you say one thing about this simulation to actual world (Sim2Real) side?
Stefan: We’d prefer it to be doable for the motions and behaviors within the simulator to be ported to actual robots. From my viewpoint, it could be helpful the opposite approach spherical too.

We, as a Simulation League, normally develop for the Simulation League and subsequently want to get the behaviors operating on an actual robotic. But the {hardware} groups normally have the same concern once they need to take a look at high-level resolution making. They may need two to 5 robots on the sector, and in the event that they need to play a high-level decision-making match and practice in that regard, they all the time should deploy a whole lot of robots. If additionally they need to have an opponent, they should double the quantity of robots with a purpose to play a sport to see how the technique would prove. The Sim2Real side can be attention-grabbing for these groups, as a result of they need to be capable of take what they deployed on the actual robotic and it also needs to work within the simulation. They can then use the simulation to coach high-level abilities like staff play, participant positioning and so forth, which is a difficult side for the actual robotic leagues like SPL or the Humanoid Leagues.

Klaus: And the explanation we all know it is because we now have a staff within the Simulation League and we now have a staff within the Humanoid League. So that’s another excuse why we’re eager to carry this stuff nearer collectively.

How does the refereeing work within the Simulation League?
Klaus: A pleasant factor about Simulation Leagues is that there’s a program which is aware of the actual state of the world so we are able to construct within the referee contained in the simulator and it’ll not fail. For issues like offside, whether or not the ball handed the purpose line, that’s fail protected. All the referee selections are taken by the system itself. We have a human referee however they by no means have to intervene. However, there are conditions the place we wish synthetic intelligence to play a task. This will not be at present the case in SimSpark as a result of the principles are all arduous coded. We have a whole lot of fouls which can be debatable. For instance, there are various fouls that groups agree mustn’t have been a foul, and different fouls that aren’t known as that ought to have been. It can be a pleasant AI studying activity to get some conditions judged by human referees after which practice an AI mannequin to higher decide the principles for what’s a foul and what isn’t a foul. But that is at present not the case.

Stefan: On the brand new simulator I’m not that far into the event that I’ve carried out the automated referee but. I’ve some fundamental algorithm which progress the sport as such, however judging fouls and deciding on particular conditions will not be but carried out within the new simulator.

What are the subsequent steps for growing the simulator?
Stefan: One of the subsequent main steps will probably be to refine the physics simulation. For occasion, despite the fact that there exists a ball within the simulation, it’s not but rather well refined. There are a whole lot of physics parameters which we now have to resolve on to replicate the actual world nearly as good as doable. This will doubtless require a sequence of experiments with a purpose to get to the right values for numerous points. In this side I’m hoping for some engagement of the neighborhood, as it’s a nice analysis alternative and I personally would like the neighborhood to resolve on a generally accepted parameter set primarily based on a degree of proof that I can’t simply present all on my own. So in case somebody is all for refining the physics of the simulation such that it greatest displays the actual world, you’re welcome to hitch!

Another main subsequent step would be the improvement of the automated referee of the soccer simulation, deciding on fouls, dealing with misbehaving brokers and so forth. In the primary model, foul circumstances will doubtless be judged by an skilled system particularly designed for this objective. The simulation league has developed a set of foul situation specs which I plan to adapt. In a second step, I want to combine and assist the event of AI primarily based foul detection fashions. But yeah, one step after the opposite.

What are you notably wanting ahead to at RoboCup2025?
Klaus: Well, with our staff we now have been vice world champion seven occasions in a row. This 12 months we’re actually hoping to make it to world champion. We are very skilled in getting losses in finals and this 12 months we’re wanting ahead to altering that, from a staff perspective.

Stefan: I’m going to Brazil with a purpose to promote the simulator, not only for the Simulation League, but additionally throughout the boundaries for the Humanoid Leagues and the SPL Leagues. I feel that this simulator is a good probability to carry individuals from all of the leagues collectively. I’m notably within the particular necessities of all of the groups of the totally different leagues. This understanding will assist me tailor the brand new simulator in direction of their wants. This is considered one of my main highlights for this 12 months, I’d say.

You can discover out extra concerning the new simulator on the undertaking webpage, and from the documentation.

Klaus Dorer is professor for synthetic intelligence, autonomous programs and software program engineering at Offenburg University, Germany. He can be a member of the Institute for Machine Learning and Analytics IMLA. He has been staff chief of the RoboCup simulation league groups magmaFreiburg (since 1999), residing programs, magmaFurtwangen and is now staff chief of magmaOffenburg since 2009. Since 2014, he has additionally been a part of the humanoid grownup measurement league staff Sweaty.

Stefan Glaser is educating assistant for synthetic intelligence and clever autonomous programs on the Offenburg University, Germany. He has been a part of the RoboCup simulation league staff magmaOffenburg since 2009 and the RoboCup humanoid grownup measurement league staff Sweaty since 2014.