MAVProxy crashes after reinstallation, when adding new MAVLink message

[retepretep]

Hi!
I added a new MAVLink message CSMAG10 to ardupilotmega.xml. Handling this with ArduCopter works, it also did work with mavproxy until a few weeks ago. I followed the steps from Section 6 in http://ardupilot.org/dev/docs/code-overview-adding-a-new-mavlink-message.html.
After uninstalling pymavlink /ardupilot/modules/mavlink/pymavlink$ sudo pip2 uninstall pymavlink and installing the updated version including the new MAVLink message ardupilot/modules/mavlink/pymavlink$ sudo python2 setup.py install --user

I tried to open mavproxy again:
/ardupilot$ sim_vehicle.py -v ArduCopter

and the following error has been thrown
Waiting for heartbeat from tcp:127.0.0.1:5760 MAV> Exception in thread main_loop: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 754, in run self.__target(*self.__args, **self.__kwargs) File "/home/crowdsweeper/.local/bin/mavproxy.py", line 830, in main_loop master.wait_heartbeat(timeout=0.1) TypeError: wait_heartbeat() got an unexpected keyword argument 'timeout'

According to other forum entries ( #543 , #222 ) this seems to be a version compatibility error of pymavlink and mavproxy. I tried installing it manually with
ardupilot/modules/mavlink/pymavlink$ sudo pip2 install mavproxy --user. After this step I could run SITL in MAVProxy again (`sim_vehicle.py'), but my new MAVLink message wasn't handled.

How can I reinstall pymavlink with avoiding this timeout error?
versions of my installed modules after the last run of setup.py are:
pymavlink: 2.3.3
mavproxy: 1.8.6
python: 2.7.12

[stephendade]

Do a pip list after uninstalling pyamvlink to confirm that it's really uninstalled. Sometimes (if you've got multiple installations) pip gets confused about what it's uninstalling.

Otherwise try building/installing from Ardupilot's mavlink git repo (https://github.com/ArduPilot/mavlink)

[retepretep]

Thanks @stephendade, pip list did help. For some reason another pymavlink version had still been installed. I took a look at pymavlink's setup.py and some of the files that generate the code, such as ardupilot/modules/mavlink/pymavlink/dialects/v10/ardupilotmega.py, as I was trying to find out, how to add my new message to mavproxy, without manually adding a file.
I did update pymavlink to the newest version 2.3.5 and tried it again, doing the same procedure as I explained above:

1. uninstalling pymavlink (and now making sure by checking with pip list)
2. using setup.py

However, now there was a new strange error, I haven't encountered before:
[...]
running install_lib running build_py Using message definitions from /home/crowdsweeper/git/ardupilot-Copter_3_6_5-CS/ardupilot/modules/mavlink/pymavlink/../message_definitions Building /home/crowdsweeper/git/ardupilot-Copter_3_6_5-CS/ardupilot/modules/mavlink/pymavlink/../message_definitions/v1.0/test.xml for protocol 1.0 Building /home/crowdsweeper/git/ardupilot-Copter_3_6_5-CS/ardupilot/modules/mavlink/pymavlink/../message_definitions/v1.0/python_array_test.xml for protocol 1.0 ERROR: /home/crowdsweeper/git/ardupilot-Copter_3_6_5-CS/ardupilot/modules/mavlink/pymavlink/dialects/v10/common.xml:3532:0:ERROR:SCHEMASV:SCHEMAV_ELEMENT_CONTENT: Element 'field': This element is not expected. Expected is one of ( deprecated, wip, description ). /home/crowdsweeper/git/ardupilot-Copter_3_6_5-CS/ardupilot/modules/mavlink/pymavlink/dialects/v10/common.xml:3543:0:ERROR:SCHEMASV:SCHEMAV_ELEMENT_CONTENT: Element 'field': This element is not expected. Expected is one of ( deprecated, wip, description ).
[...]
this error happened in some other lines of common.xml aswell.

I checked common.xml whether there has been any recent change that might cause this error, but couldn't find anything in the line numbers of the error message.
I also did git checkout for commits of pymavlink 2.3.4 and 2.3.3 and still got this error of setup.py.

Because I couldn't even execute setup.py, I didn't yet build mavlink from the repo, as you suggested.

Do you know how I can fix this issue or what might be the cause?

Best Regards,
Peter

[stephendade]

There's someone else with the same issue as you here: ArduPilot/APWeb#16

Looks like you still need to update your git?

[retepretep]

Thanks for linking this @stephendade.
I did as @shortstheory suggested: git submodule update --init --recursive. This changed setup.py back to using pymavlink 2.2.10. So now i could run setup.py, but when running /ardupilot$ sim_vehicle.py -v ArduCopter, mavproxy crashed again, complaining about got an unexpected keyword argument 'timeout'.

About the git update:
I use a modified version of ArduCopter, based on the 3.6.5-release - is pymavlink 2.2.10 incompatible to ArduCopter 3.6.5? Do I have to merge it with the Copter 3.6.8-release?

[stephendade]

hmm, maybe try the mavlink direct from the Ardupilot repo: https://github.com/ArduPilot/mavlink rather than the submodule inside Ardupilot

[retepretep]

That did the trick! Now I have a pymavlink/setup.py with version 2.3.5 and I can run it without crashing, so the pymavlink issue seems to be resolved.

However compilation of ArduCopter fails with the following error:
[419/520] Compiling libraries/GCS_MAVLink/GCS_ServoRelay.cpp ../../libraries/GCS_MAVLink/GCS_Common.cpp: In member function 'void GCS_MAVLINK::send_distance_sensor(const AP_RangeFinder_Backend*, uint8_t) const': ../../libraries/GCS_MAVLink/GCS_Common.cpp:308:10: error: too few arguments to function 'void mavlink_msg_distance_sensor_send(mavlink_channel_t, uint32_t, uint16_t, uint16_t, uint16_t, uint8_t, uint8_t, uint8_t, uint8_t, float, float, const float*)' 0); // Measurement covariance in centimeters, 0 for unknown / invalid readings ^ compilation terminated due to -Wfatal-errors.
Appearently the function call of mavlink_msg_distance_sensor_send in GCS_MAVLINK::send_distance_sensor(const AP_RangeFinder_Backend *sensor, const uint8_t instance) in libraries/GCS_MAVLink/GCS_Common.cpp is different in Copter 3.6.8 than in my Copter 3.6.5-fork. Copter 3.6.8 seems to use MAVLink2 with the extensions, which are not used in Copter 3.6.5.
So I probably won't get around merging my Copter-3.6.5-fork with the Copter-3.6.8 release, unless there is a way to switch using the MAVLink2-extensions off.

Until I fix the extension compatibility issues, I can't confirm that MAVProxy works again on my system, but I'm confident that the new setup.py fixed the error in mavproxy.

In case others also face the problem with mavproxy, here are my steps to get pymavlink/setup.py running in pymavlink 2.3.5:

1. ardupilot/modules/mavlink$ sudo pip2 uninstall pymavlink
2. doublecheck whether pymavlink has been properly uninstalled with pip list
3. (remove ardupilot/modules/mavlink/pymavlink directory - probably not necessary)
4. cd to mavlink and
   4.1. (do ardupilot/modules/mavlink$ git checkout message_definitions/v1.0/ard upilotmega.xml in case you added some MAVLink messages, like I did, store them somewhere else temporarily)
   4.2. ardupilot/modules/mavlink$ git pull https://github.com/ArduPilot/mavlink.git master
   as suggested by @stephendade
5. ardupilot/modules/mavlink$ git submodule update --init --recursive
   also reinstalls pymavlink, which is a submodule of mavlink repo
6. (replace ardupilot/modules/mavlink/message_definitions/v1.0/ardupilotmega.xml with your modified version, if applicable)

you can check if it is the right pymavlink/setup.py file, by checking the file's first 20 lines: the version should be written in a string: version = '2.3.5'.

7. ardupilot/modules/mavlink/pymavlink$ sudo python2 setup.py install --user

Now pymavlink 2.3.5, lxml 4.3.3 and mavproxy 1.8.6 are installed.

[stephendade]

umm ... perhaps I didn't explain this well.

The mavlink in ardupilot/modules/mavlink is only used for compiling Ardupilot (it's never actually installed, just the header files are built). It's separate to what you'd install on your pc/laptop as part of a GCS.

The mavlink in https://github.com/ArduPilot/mavlink is the one you should be installing on your laptop. It should be downloaded to a completely separate folder independant of ardupilot and built/installed on your pc/laptop.

For further info:
The mavlink in https://github.com/ArduPilot/mavlink and ardupilot/modules/mavlink are not always the same version. The https://github.com/ArduPilot/mavlink is always the latest version. The ardupilot module mavlink is sometimes a few versions behind (but regularly updated) as ardupilot sometimes needs to be changed/patched to support changes to mavlink.

[retepretep]

Now I understand. I really thought mavproxy GCS is provided as part of ardupilot/modules/mavlink. I will install a separate mavproxy in another folder next week, thanks for the tip!

[retepretep]

It was just so convenient for debugging, using it like /ardupilot$ sim_vehicle.py -v ArduCopter