Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixes #43 #49

Closed
wants to merge 7 commits into from
Closed

fixes #43 #49

wants to merge 7 commits into from

Conversation

plattysoft
Copy link

The LED flashing on audio received causes too many calls to GPIO and that produces audio clipping.

As a solution, I have created a thread that starts one blink each time setBlinking (true) is called, so if several blinks are requested, the are considered as "do one more blink"

To make it a bit more dynamic, the duration of the blink is pseudo-random.

Note that the LED Is opened as ACTIVE_LOW, and therefore false means turn on and true means turns off. You may want to change that.

@plattysoft plattysoft changed the title fies #43 fixes #43 Nov 12, 2017
@Fleker Fleker requested review from Fleker and proppy November 13, 2017 01:36
@proppy
Copy link
Collaborator

proppy commented Nov 13, 2017

What was "nice" with the previous behavior was that the blinking was happening in sync with the audio playback.

Maybe we could simply blink every XX samples?

Also we didn't see clipping issue with previous releases, I wonder if this is a regression with the PIO latency with recent Android Things developer preview.

Copy link
Collaborator

@Fleker Fleker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate that you've taken up this project yourself in a better way than just removing the code. I've also added Proppy to get some of this thoughts on this change, and a few comments in general.

@Override
public void onAudioSample(ByteBuffer audioSample) {
if (mLed != null) {
if ("END_OF_UTTERANCE".equals(eventType.toString())) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure this is a constant


@Override
public void onAudioSample(ByteBuffer audioSample) {
if (mLed != null) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep this in, just in case there is no connected LED?

} catch (IOException e) {
Log.w(TAG, "error toggling LED:", e);
e.printStackTrace();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep the current error message?

try {
mLed.setValue(true);
} catch (IOException e) {
e.printStackTrace();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep the previous error log

} catch (IOException e) {
Log.e(TAG, "error turning off LED:", e);
}
try {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep the current null checker

mBlinking = false;
try {
mLed.setValue(false);
sleep(150+mRandom.nextInt(100));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if random blinking delays are ideal.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes it feel a bit more dynamic, IMHO if the interval is always the same it feels too static

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide a short video/gif of each one as a comparison? I'm not convinced that the random blinking is better. Consistency is not bad. This project is meant to be a sample and not a highly polished and complex project.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just changed it to a constant

mRandom = new Random();
}

public void setBlinking(boolean b) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if doing a setBlinking(boolean) would be the right structure. Instead, doing something like blink() handle the logic in the class would be better.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I originally used the boolean on the way that if you passed false it stopped, but I couldn't find an "end of speech" trigger, so I changed the behaviour but did not rename the method.

mLed.setValue(true);
sleep(150+mRandom.nextInt(100));
} catch (IOException e) {
e.printStackTrace();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we do the Log.e(TAG, <message>, e) instead?

} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we do the Log.e(TAG, <message>, e) instead?

build.gradle Outdated
@@ -19,7 +19,7 @@ buildscript {
jcenter()
}
dependencies {
classpath 'com.android.tools.build:gradle:2.3.0'
classpath 'com.android.tools.build:gradle:2.3.1'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any specific reason for this change?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing special besides Android Studio prompting for it.

@plattysoft
Copy link
Author

@proppy maybe something has changed on the latest release, but on 5.1 it is definitely an issue.

I can't see a way to make it sync with the playback while solving this problem.

@plattysoft
Copy link
Author

Other possible solution is to create a layer on the communication with the LED that only changes it if the previous call was longer than X ms ago.

@plattysoft
Copy link
Author

I think I addressed all the comments @Fleker, let me know if there is any other request

} catch (IOException e) {
Log.w(TAG, "error toggling LED:", e);
}
Log.e(TAG, "onAudioSample");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you change this to not an error?

Copy link
Author

@plattysoft plattysoft Nov 14, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, I'll make it info like the other ones

@@ -192,9 +206,9 @@ public void onConversationFinished() {
Log.i(TAG, "assistant conversation finished");
if (mLed != null) {
try {
mLed.setValue(false);
mLed.setValue(true);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we turn off the LED in this case?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the LED is configured as ACTIVE_LOW, true is actually off (I said that as a note on the description) I find it confusing, but I did not want to change the original behaviour.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ACTIVE_LOW / ACTIVE_HIGH are set so that setting the value to true/false does what is expected. In my case, the LEDs are ACTIVE_LOW, but that's where the value should be changed.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my case led.setValue(false) does turn it on, I'm not sure if that is a configuration problem, a hardware difference or something that changed on the OS since this project was originally made

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually the blinking is following this pattern, first on, then off, so at the end of each blink, it stays off. Let me know if you want me to revert all of them and change the active mode.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would revert the changes so that true => on and false => off.

In my experience, the LED is ACTIVE_LOW, and I don't know if I want to change that if that's how the VoiceHat really is, but we should probably change it there.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be helpful if someone else could verify the LED state on different AIY voice kits and Android Things versions. Until then, I'm not inclined to change anything.

IMHO opening the LED with the correct mode should be a responsibility of the voiceHAT driver.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, after one more test / reading of the code, onButtonEvent the LED is off when pressed and turns on when released (pressed == false)

The code is mLed.setValue(pressed)

And that is code I have not touched.

Unless it does not work as I described for you, then we clearly have some inconsistency on hardware / operating system.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mLed.setValue(pressed); makes sense using the true/on setup.

I do agree that the LED should be set up through the VoiceHat. Until that change is made and published to the driver, we should refrain from making this workaround.

} catch (IOException e) {
Log.e(TAG, "error turning off LED:", e);
Log.w(TAG, "error turning off LED:", e);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think keeping the level as error is preferable.

Copy link
Author

@plattysoft plattysoft Nov 14, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, I missed that change.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you make the change?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I missed that one. Fixed now

import java.util.Random;

public class LedBlinkThread extends Thread {

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typically we don't have an empty line


public class LedBlinkThread extends Thread {

private static final String TAG = "LedBlinkingThread";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typically we use LedBlinkThread.class.getSimpleName()

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point. I changed it.

mBlinking = false;
try {
mLed.setValue(false);
sleep(150+mRandom.nextInt(100));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide a short video/gif of each one as a comparison? I'm not convinced that the random blinking is better. Consistency is not bad. This project is meant to be a sample and not a highly polished and complex project.

} catch (IOException e) {
Log.e(TAG, "error turning off LED:", e);
Log.w(TAG, "error turning off LED:", e);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you make the change?

@@ -192,9 +206,9 @@ public void onConversationFinished() {
Log.i(TAG, "assistant conversation finished");
if (mLed != null) {
try {
mLed.setValue(false);
mLed.setValue(true);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would revert the changes so that true => on and false => off.

In my experience, the LED is ACTIVE_LOW, and I don't know if I want to change that if that's how the VoiceHat really is, but we should probably change it there.

@proppy
Copy link
Collaborator

proppy commented Nov 15, 2017

Other possible solution is to create a layer on the communication with the LED that only changes it if the previous call was longer than X ms ago.

Maybe we could do that in a follow up change? And keep this PR simple by just removing the blinking logic (without replacing it with something else).

It would fix the immediate issue (#43) and gives up more time to design a replacement.

What do you think?

@plattysoft plattysoft closed this Dec 3, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants