quadcopter simulators

[knmcguire]

Here just a quick preview of the ignition gazebo simulator with 5 quadcopters. I must say that it aint as slow as it was with gazebo, but I haven't had these fly fully. Ignition has now an plugin for multi copter control, which seems to be new, or at least based on the ROS1 RotorS package.

So the first step is to get one of these to fly, then multiple ones, and also see if we can get the crazyflie model in https://app.ignitionrobotics.org/fuel/models, so it's even easier to include them in ignition.

I still expect it to be pretty slow for big swarms, so perhaps we can also put in some kind of position only based on the dynamic script that @whoenig wrote. But small steps for now :)

I'll try to share my process in this thread.

[knmcguire]

Little update, the last picture was made within a native ubuntu install. Unfortunately the windows binary install are not up to par yet: https://answers.gazebosim.org//question/27539/fortress-windows-binary-install/

Very unfortunate... so that means that I'll hang around on ubuntu more often afterall

[whoenig]

Ahh, very sad, but good that you got a detailed response. I feel the whole Windows support of ROS2 is very half-baked at best at the moment. I couldn't even find a way to enable continuous integration for Windows (or Mac)...

[knmcguire]

yeah.... and even WSL2 (xserver) ain't the full solution here as it is not supporting the OGRE2 rendering. the first version of OGRE works but it looks awful. Old gazebo works but I don't want to go there anymore as it will disappear eventually (and I want to use the multicopter plugin)

It's sad since graphics support on windows is way more advanced... so frustrating!

[knmcguire]

The ignition msg of actuators is not fully supported for the ros_ign_bridge yet out of the box, as can be seen in the readme of the repo. I've tried to build the actuator msg itself based on the original mav_com repo, but no luck yet.

I've started this thread on answers.gazebo.org, but they are pretty responsive so hopefully it's solvable!

[knmcguire]

Apparently it is because the ros2 branch of ros_ign_bridge commented out the mav_msgs part: https://github.com/ignitionrobotics/ros_ign/blob/5d9cfef3d58948a77b962d231e281c88ea88e188/ros_ign_bridge/include/ros_ign_bridge/convert.hpp#L341-L354

I just have to build it from source!

[knmcguire]

Perhaps I should give Webots a go, as they do have ROS2 support, and hopefully better windows support. They seem to have something going for the mavic pro, so let's see!

Shall I keep those results in this discussion topic or make a new one?

[whoenig]

We can keep this thread open for all simulation for now. Another good simulator is https://www.coppeliarobotics.com/, which also has ROS2 Windows support.

[knmcguire]

Sim CF from: https://github.com/wuwushrek/

video:
https://user-images.githubusercontent.com/11061260/141525308-762ab1c9-0b3c-48c6-91ff-02e55bc30cd4.mp4

This is gazebo 11 with Noetic

This is with the crazyflie firmware in the loop. Only 4 for now, if I try to fly 7 as the example shows, the cf_handler gazebo plugin crashes. But, the realtime factor during this flight stays around 0.95 which is pretty good!

Currently I forked a branch to my github account: https://github.com/knmcguire/sim_cf to make some fixes since the original repository is no longer actively monitored.

[knmcguire]

I like the different propeller color option :)

[whoenig]

Cool! Just need that for ROS2:-)

[knmcguire]

yeah I hope that the gazebo plugins written for this are not to far of what is required for ROS2. Also here they kind of use their own type of 'server' that goes around the ros frame work if they are controlling more than 1 crazyflies.

[knmcguire]

Mavic in Webots with ROS2

Runs pretty fast even with a complicated environment, and it seems that the driver plugin is easy enough, which is used to determine it's dynamics.

The only problem is that they don't support URDF, so if we want to make a crazyflie model, we probably have to make an URDF first and use their conversion tool.

[whoenig]

Looks nice! There is a (complicated) URDF Model of the Crazyflie in https://github.com/gsilano/CrazyS/blob/master/rotors_description/urdf/crazyflie2.xacro.

[knmcguire]

Ah great! Are we able to use this with consent of the author? The repository has apache 2.0 on there so I'm not sure if it falls under that too.

SimCF also has an urdf file, and no licence or active monitoring, and I have some dea meshes lying around myself of a crazyflie design, so it shouldn't take long to make one yourselves. Especially if one of the goals would be to get a crazyflie model in ignition fuel model system, we should probably make one ourselves, or we should contact the author.

[gsilano]

Feel free to take whatever you need ;-)

[knmcguire]

So I decided to go and make my own urdf, especially since for ignition gazebo not all things are the same and I do want to make it compatible. Also the model can be much easier now due to the newly intergrated plugins (so xacro doesnt have to be used). Moreover, this should be a low poly version so better for rendering.

I've started a repo for this here: https://github.com/knmcguire/crazyflie_models. Currently the props are not spinning and none of the quadcopter plugins are intergrated, but just slowely building it up from there.

[knmcguire]

ah yeah ofcourse! let me update it

[knmcguire]

there are some instructions now! the model is still extremely simple though, so don't expect flight. But I'm planning to connect all the plugins next (after seperating the body and the rotors)

[knmcguire]

Now with flight :)

Screencast.from.2021-11-17_15.26.36.mp4

[whoenig]

wow - amazing!

[knmcguire]

Implemented the motor control plugin but it requires some tuning as it doesnt take off... but it's nice that those plugins exist so we can atleast fly it before doing sitl or something like that :)

[knmcguire]

So I managed to have the crazyflie sort of flying with velocity controller, which is according to the doc based on the lee controller from the rotorS project. but it's quite difficult to tune it properly... so it's still unstable.

ign_gazebo_controller.mp4

[knmcguire]

But perhaps it's better to have a simple physics model as crazyswarm simulator as a plugin to gazebo or something similar. Do you know where I find these physics equations in the python sim you build, @whoenig ?

[whoenig]

I E-Mailed it! There is nothing really secret about it, but I just don't happen to have a repo for it, yet.

[gsilano]

Yeah, that is Lee's geometric controller. A student from UniMore (i.e., Stefano Grossi) started working on it a while a go. I was in the process of integrating his code in my CrazyS but, you know, when you don't have a project on this find some spare time is complicated.

This is the link to his repo (in Italian only :( )

[knmcguire]

Here a video of regular gazebo using the 'actor' framework, which are regular 'animations' not affected by the phyics engine. This might be interesting if we want to show bigger swarms while maintaining faster simulation.

gazebo_actor.mp4

I've uploaded an example for this on https://github.com/knmcguire/crazyflie_models

[whoenig]

This is cool! Will ignition get this feature back, though?

[knmcguire]

i would think so! For now, I found out now that we can perhaps kind of trick it with setworldposecmd, and turning off the physics. So currently looking at an example where I can use /cmdvel and convert it in a plugin for the velocity of the drone itself. So let's see!

[knmcguire]

Got something beginning to work with the new plugin with ignition gazebo

[knmcguire]

Saw on twitter that somebody else also started on a simulator: https://github.com/malintha/multi_uav_simulator. Seems to be in rviz! So it's probably without an physics engine then.

[malintha]

Sure! It wasn't straightforward for the crazyflie as it was for the quadrotor model in the original paper. These are the tuning and the model parameters for my implementation. https://github.com/malintha/multi_uav_simulator/blob/crazyflie/config/cf_config.yaml

I suspect Lee's controller might work better on robots with large moments of inertia (but I could be wrong).

[malintha]

@whoenig @knmcguire I recently added trajectory optimization with receding horizon planning to the simulator. Now you can publish a goal from the command line to the drones and they can track it. Currently, I am planning to extend it with higher order control inputs like velocity/acceleration. Feel free to give it a try/open a discussion. I'm all ears to any of your thoughts. :)

[knmcguire]

Ah great ! I saw it indeed on twitter that you released that. For now other projects has but the simulation work a little bit on hold, but I hope I have some time these weeks to focus on it again. I'll let you know once I've tried it 😄

[knmcguire]

I've tried the receding horizon, the visualization of the trajectory is very nice! it is not yet implemented on the crazyflie branch right?

Problem only is that there is something wrong with rendering the MAV models on my end but I'll start an issue for that on your repo:)

[malintha]

Much thanks for trying it out! :) Yeah, the crazyflie branch is a bit behind for now. I'm thinking to merge it with master and let the user to bind a drone model of choice to the controller. So we don't need to maintain separate branches for the models. Also, that might make it easy to extend with other dynamical models.

[knmcguire]

Another possibility is Airsim: https://microsoft.github.io/AirSim/ . Maybe not in the immediate future but definitely one to remember. Maybe I should try out the experimental Unity plugin?

[knmcguire]

I got this hint from @noahzn btw who commented on the crazyflie models repo: knmcguire/crazyflie_models#3

[knmcguire]

I've started with a low poly crazyflie model from scratch (instead of converting cad files and simplifying it, which made the models messy. These should should be much more suitable for simulation. The meshes can now be found here on the crazyflie bitcraze account: https://github.com/bitcraze/crazyflie_simulation

[uhvardhan]

Hi all,

I have been following this discussion for a bit.

I also needed a simulator where I can simulate large swarms(For now 10 crazyflie will do - but in the future maybe around 50-100 crazyflies). I found this : https://github.com/utiasDSL/gym-pybullet-drones. I just started working on this.

Also, let me know if I can help you with anything. I am more than happy to learn and help. Crazyswarm has made things a lot easier for me and I am more than happy to spend my time on similar projects.

Thanks

[knmcguire]

Ah awesome, that fits totally in the list! I didnt know that the dynamics system lab also had their own simulator now. Thanks for sharing!

To find a simulator suitable for massive swarms is difficult, but it should not be impossible. I've played around with Unity to make a swarm collection game and hit about a hundred easily but that is without physics. But what I remember from Gazebo and the crazyS and CF_sim simulation, is that got very slow with more than 5-6.

I used argos sim in my phd days before but was forced to use the ground robots only as the drones didn't had collision models yet intergrated at that time. It seems that they are not super active anymore.

Currently I'm working with the https://github.com/bitcraze/crazyflie_simulation repository with both Webots and Gazebo, so you can look at that.

[malintha]

This is really cool. Apparently, the underlying physics engine, Bullet uses GPU acceleration as well for dynamics [1]. Nvidia's Isaac Gym also has a similar architecture for boosting swarm physics, and has examples of hundreds of robots interacting in complex physics environments. Not sure how this would stand up to that. But they don't have a dedicated drone swarm code yet, AFAIK. But there are some really cool examples in the main page though [2].

[1] Paper for this: https://arxiv.org/pdf/2103.02142.pdf
[2] Isaac Gym: https://developer.nvidia.com/isaac-gym

[uhvardhan]

@malintha Thank you for sharing Isaac Gym - it is really interesting. I am going to try it out.

@knmcguire I have been following your repository: https://github.com/bitcraze/crazyflie_simulation. It's a nice setup you have got there.

I do have a very dumb question(I am still trying to wrap my head around different kinds of simulators): What is the major advantage of using simulators with physics?

I am going to continue to work on PyBullet - probably because I am most comfortable in Python. Will regularly update here.

[malintha]

@uhvardhan On top of my head, I can think of several advantages of using physics-based simulations.

• They'll let you close the "reality-gap" in research. Instead of using numerical simulations with point robots, they let you validate and design algorithms that can be deployed in real-world. Kinematic models are also good, but they are still far from reality.

• They are essential for designing controllers for robots. We cannot design low-level controllers without observing gravity, torques etc.

• Nowadays these are mostly targeted for RL audiences to directly train policies on real-world like environments. For example, you can train a NN to decide how much control to apply given the state, while accounting for dynamics related complexities, e.g.: apply break if you observe going downhill and accelerating.

Isaac Gym's aim is also to allow people to train on high-fidelity environments expecting zero-shot transfer. This is a recently popularized paper on performing acrobatic drone maneuvers with zero-shot sim2real transfer. https://arxiv.org/pdf/2006.05768.pdf. They've used Gazebo to train the policies.

[knmcguire]

@malintha great answer, I agree indeed as well. Although I would say that it even would help with the high level application level as well. I had to make an wall following state machine for a crazyflie for multiranger once, and it really helped to have some more realistic quadcopter dynamics to make it really robust. I could almost use the exact same code for onboard the STM for that.

Also from the Bitcraze point of view, we also would like to use the same simulator to test the lower level controllers and estimators of the firmware as well, to add to our nightly build automatic test lab for instance: https://github.com/bitcraze/crazyflie-testing

But yes, ofcourse, if you would like to simulate a very large swarm more close to 50-100s, the question of if you really would need it becomes more important.

[knmcguire]

So currently I have a sort of flying crazyfie in webots, in attitude mode with fixed heigth in https://github.com/bitcraze/crazyflie_simulation, where you can control the roll and pitch with the keyboard arrow keys. It is using very easy control for that.

I tried to separate the controller and started to implement it in the Ign Gazebo SDF that I also made, but unfortunately it seems that webots and ign gazebo do not match up.... so that requires a bit more investigation.

I think the first few steps that I will do, is to make a velocity controller for the Webots crazyflie model, make it react on a /cmd_vel ros command, and perhaps also make a python wrapper for the control. So stay tuned!

[knmcguire]

This was a great review paper I found about robotics simulators: https://ieeexplore.ieee.org/document/9386154

It even has the aerial robotics simulator table overview.

Nothing for swarming though, but I think we can take a look at the simulators used for machine learning, as they often require multiple robots to be simulated in parallel doing the same task.

[knmcguire]

it seems that this discussion thread is getting pretty long and perhaps not about crazyswarm specific. We are trying out github discussions ourselves locally on the crazyflie-simulation repository, so I advise us to continue our simulation related discussions there!

[diplodocuslongus]

Hi,
The page at crazyflie-simulation can't be found, could you please update the link? All of this is very interesting and I'd like to follow along.

[knmcguire]

ah yes sorry, we have converted to organization wide discussions now. You can go here for any crazyflie simulation related questions: https://github.com/orgs/bitcraze/discussions