Bitcraze
Crazyflie Simulation using UnrealEngine5 #779
Replies: 11 comments 7 replies
-
|
Nice! |
Beta Was this translation helpful? Give feedback.
-
|
hi @simondlevy. I haven't had the time to look at the simulation itself but the video looks very cool! Yes, unreal engine would be great for these kinds of vision processing. I've been working with Webots myself, which is perhaps more on the interaction and physics sides, but the rendering is not as good. Would you be interested to write a guest blogpost once you are done with your implementation? You can send us an email at contact_at_bitcraze.io |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, @knmcguire , it'd be an honor to write a blog post on the simulation! As I'm working toward a journal deadline this summer (describing a Crazyflie deck I'm building to support the vision sensor I mentioned), I probably won't be able to have the full-on simulator going for a few months. I'll look forward to getting in touch when I have something ready -- especially since there's a related conference coming up soon after that! In addition to the vision-processing feature, another helpful feature of the simulator is that instead of the built-in physics engine in UE5, I use a custom multirotor dynamics model, based on this classic article. The colleagues with whom I built the simulator had tried using a general-purpose simulator (Gazebo, similar I suspect to Webots), but they found it inadequate for getting the physics accurate enough to be run by an actual flight-control algorithm. |
Beta Was this translation helpful? Give feedback.
-
|
A little more progress ;^) |
Beta Was this translation helpful? Give feedback.
-
|
very cool! you are going quickly with your development 😄 |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, @knmcguire! Seeing what you've done with Webots + Crazyflie, however, I'm actually now considering looking into Webots for my own work. The main issues, I think are (a) how fast an update rate is possible for the physics and the controller, and (b) the fidelity of the simulated camera images. With UnrealEngine (i.e., threaded C++) I'm getting on the order of 10^5 Hz for the physics and 10^3 Hz for the control; i.e., the control is within the same order of magnitude as you'd get with a typical racing quadcopter (e.g., 8kHz for Betaflight). And of course the UE image quality is determined by the map/level you're using (I typically just purchase a pre-built environment, which provides extremely realistic images). But for machine-vision research (esp. the kind of optical-flow work that interests me), the super-high-fidelity of of a game engine like UE may be unnecessary -- and it's pretty amazing what's possible with Webots at this point, too! Finally, a huge advantage of Webots is that (like Gazebo), it runs straightforwardly on Linux. Although Epic Games provides a Linux release for the UE5 editor, I've never really been able to get it working consistently. Having got your Crazyflie simulation (apartment wall-following) working, I'm going to spend some time this week seeing if I can get a sense of how fast the updates are. More soon ... |
Beta Was this translation helpful? Give feedback.
-
|
Well, that was easy! ;^) Thanks to your very clear and elegant Python code, it only took a minute to instrument the control loop by adding a print() statement for 1/dt. As shown in the lower-left corner of this screenshot, I'm seeing only around 3x10^1 Hz updates. I think it is still worth looking into how fast this can be sped up, though -- perhaps by trying your C control code instead of the Python. |
Beta Was this translation helpful? Give feedback.
-
|
You could try indeed to make a python binding of the c code of the wallfollowing code. I did that a while ago with this branch: https://github.com/bitcraze/crazyflie-simulation/tree/wallfollowing2. But for the sack of a lecture I had to give I converted it fully to python as well. |
Beta Was this translation helpful? Give feedback.
-
|
As a quick test of the speed Webots provides with C instead of Python, I tried the built-in DJI Mavic example and got 1.25x10^2 Hz. This is the expected order-of-magnitude speedup one expects switching from Python to C, but still considerably slower than I can get by running the PID controller on a C thread in my Python-extension code. Webots is certainly a delight to work with, though -- very fast compile and load! |
Beta Was this translation helpful? Give feedback.
-
|
Yeah! it's really nice to work with I think. quite easy to implement new stuff |
Beta Was this translation helpful? Give feedback.
-
|
Looking at the PID-loop specs for Crazyflie, I see that the attitude rate (gyro-based) PID runs at just 500 Hz -- much slower than the 8kHz of Betaflight (FPV racing firmware), but obviously good enough to control the Crazyflie. This makes me lean even further in the direction of switching to Webots. Given that current models of flight control for insects use a 0.002s time constant, Crazyflie control rate seems biologically realistic, too. |
Beta Was this translation helpful? Give feedback.
-
|
Given the progress I've been making using Webots, I'm going to close this discussion and start a new one about the new deck I've been designing (and simulating). |
Beta Was this translation helpful? Give feedback.
-
|
Very cool! and welcome to the dark side 😄 (theme of webots) |
Beta Was this translation helpful? Give feedback.
-
|
Hi! I just reopened this because if you close it, it will remove the discussion from the list show & tell list apparently. I think it is nice if show&tell discussions are retained but I can lock the conversation if you'd like? |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, @knmcguire; I appreciate your doing this! As I am working right now connecting the Python client GUI to Webots, I think it's worthwhile to keep this discussion open after all. |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Awesome! I'll be going on a holiday soon but once I'm back id love to take a closer look at this for sure😁 |
Beta Was this translation helpful? Give feedback.
-
I've been enjoying Crazyflie so much that I wanted to add it to a photorealistic, general-purpose multirotor simulator that I've been working on for the past several years. As this video, shows, I've so far just managed to connect the Crazyflie client GUI to the sim and do a simple takeoff maneuver, the pieces are now in place to begin the real work of incorporating the Crazyflie flight-control algorithms into the simulator.
Because of the ability of the simulator to do real-time machine vision on the video captured from the sim, I think that this approach could support prototyping of vision-based sensors like the AI Deck, or the Dynamic Vision Sensor for which I'm currently attempting to design my own custom deck.
Beta Was this translation helpful? Give feedback.
All reactions