-
Notifications
You must be signed in to change notification settings - Fork 147
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fixes #43 #49
fixes #43 #49
Conversation
…umber of calls to the led to fix audio problems
…umber of calls to the led to fix audio problems
What was "nice" with the previous behavior was that the blinking was happening in sync with the audio playback. Maybe we could simply blink every XX samples? Also we didn't see clipping issue with previous releases, I wonder if this is a regression with the PIO latency with recent Android Things developer preview. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I appreciate that you've taken up this project yourself in a better way than just removing the code. I've also added Proppy to get some of this thoughts on this change, and a few comments in general.
@Override | ||
public void onAudioSample(ByteBuffer audioSample) { | ||
if (mLed != null) { | ||
if ("END_OF_UTTERANCE".equals(eventType.toString())) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm pretty sure this is a constant
|
||
@Override | ||
public void onAudioSample(ByteBuffer audioSample) { | ||
if (mLed != null) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we keep this in, just in case there is no connected LED?
} catch (IOException e) { | ||
Log.w(TAG, "error toggling LED:", e); | ||
e.printStackTrace(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we keep the current error message?
try { | ||
mLed.setValue(true); | ||
} catch (IOException e) { | ||
e.printStackTrace(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's keep the previous error log
} catch (IOException e) { | ||
Log.e(TAG, "error turning off LED:", e); | ||
} | ||
try { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's keep the current null checker
mBlinking = false; | ||
try { | ||
mLed.setValue(false); | ||
sleep(150+mRandom.nextInt(100)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if random blinking delays are ideal.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It makes it feel a bit more dynamic, IMHO if the interval is always the same it feels too static
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you provide a short video/gif of each one as a comparison? I'm not convinced that the random blinking is better. Consistency is not bad. This project is meant to be a sample and not a highly polished and complex project.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just changed it to a constant
mRandom = new Random(); | ||
} | ||
|
||
public void setBlinking(boolean b) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if doing a setBlinking(boolean)
would be the right structure. Instead, doing something like blink()
handle the logic in the class would be better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. I originally used the boolean on the way that if you passed false it stopped, but I couldn't find an "end of speech" trigger, so I changed the behaviour but did not rename the method.
mLed.setValue(true); | ||
sleep(150+mRandom.nextInt(100)); | ||
} catch (IOException e) { | ||
e.printStackTrace(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we do the Log.e(TAG, <message>, e)
instead?
} catch (IOException e) { | ||
e.printStackTrace(); | ||
} catch (InterruptedException e) { | ||
e.printStackTrace(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we do the Log.e(TAG, <message>, e)
instead?
build.gradle
Outdated
@@ -19,7 +19,7 @@ buildscript { | |||
jcenter() | |||
} | |||
dependencies { | |||
classpath 'com.android.tools.build:gradle:2.3.0' | |||
classpath 'com.android.tools.build:gradle:2.3.1' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any specific reason for this change?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nothing special besides Android Studio prompting for it.
@proppy maybe something has changed on the latest release, but on 5.1 it is definitely an issue. I can't see a way to make it sync with the playback while solving this problem. |
Other possible solution is to create a layer on the communication with the LED that only changes it if the previous call was longer than X ms ago. |
I think I addressed all the comments @Fleker, let me know if there is any other request |
} catch (IOException e) { | ||
Log.w(TAG, "error toggling LED:", e); | ||
} | ||
Log.e(TAG, "onAudioSample"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you change this to not an error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sure, I'll make it info like the other ones
@@ -192,9 +206,9 @@ public void onConversationFinished() { | |||
Log.i(TAG, "assistant conversation finished"); | |||
if (mLed != null) { | |||
try { | |||
mLed.setValue(false); | |||
mLed.setValue(true); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we turn off the LED in this case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since the LED is configured as ACTIVE_LOW, true is actually off (I said that as a note on the description) I find it confusing, but I did not want to change the original behaviour.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The ACTIVE_LOW
/ ACTIVE_HIGH
are set so that setting the value to true/false does what is expected. In my case, the LEDs are ACTIVE_LOW
, but that's where the value should be changed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In my case led.setValue(false) does turn it on, I'm not sure if that is a configuration problem, a hardware difference or something that changed on the OS since this project was originally made
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually the blinking is following this pattern, first on, then off, so at the end of each blink, it stays off. Let me know if you want me to revert all of them and change the active mode.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would revert the changes so that true
=> on
and false
=> off
.
In my experience, the LED is ACTIVE_LOW
, and I don't know if I want to change that if that's how the VoiceHat really is, but we should probably change it there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be helpful if someone else could verify the LED state on different AIY voice kits and Android Things versions. Until then, I'm not inclined to change anything.
IMHO opening the LED with the correct mode should be a responsibility of the voiceHAT driver.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, after one more test / reading of the code, onButtonEvent the LED is off when pressed and turns on when released (pressed == false)
The code is mLed.setValue(pressed)
And that is code I have not touched.
Unless it does not work as I described for you, then we clearly have some inconsistency on hardware / operating system.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mLed.setValue(pressed);
makes sense using the true/on setup.
I do agree that the LED should be set up through the VoiceHat. Until that change is made and published to the driver, we should refrain from making this workaround.
} catch (IOException e) { | ||
Log.e(TAG, "error turning off LED:", e); | ||
Log.w(TAG, "error turning off LED:", e); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think keeping the level as error is preferable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sure, I missed that change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you make the change?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry I missed that one. Fixed now
import java.util.Random; | ||
|
||
public class LedBlinkThread extends Thread { | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Typically we don't have an empty line
|
||
public class LedBlinkThread extends Thread { | ||
|
||
private static final String TAG = "LedBlinkingThread"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Typically we use LedBlinkThread.class.getSimpleName()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good point. I changed it.
mBlinking = false; | ||
try { | ||
mLed.setValue(false); | ||
sleep(150+mRandom.nextInt(100)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you provide a short video/gif of each one as a comparison? I'm not convinced that the random blinking is better. Consistency is not bad. This project is meant to be a sample and not a highly polished and complex project.
} catch (IOException e) { | ||
Log.e(TAG, "error turning off LED:", e); | ||
Log.w(TAG, "error turning off LED:", e); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you make the change?
@@ -192,9 +206,9 @@ public void onConversationFinished() { | |||
Log.i(TAG, "assistant conversation finished"); | |||
if (mLed != null) { | |||
try { | |||
mLed.setValue(false); | |||
mLed.setValue(true); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would revert the changes so that true
=> on
and false
=> off
.
In my experience, the LED is ACTIVE_LOW
, and I don't know if I want to change that if that's how the VoiceHat really is, but we should probably change it there.
Maybe we could do that in a follow up change? And keep this PR simple by just removing the blinking logic (without replacing it with something else). It would fix the immediate issue (#43) and gives up more time to design a replacement. What do you think? |
The LED flashing on audio received causes too many calls to GPIO and that produces audio clipping.
As a solution, I have created a thread that starts one blink each time setBlinking (true) is called, so if several blinks are requested, the are considered as "do one more blink"
To make it a bit more dynamic, the duration of the blink is pseudo-random.
Note that the LED Is opened as ACTIVE_LOW, and therefore false means turn on and true means turns off. You may want to change that.