Make it possible to give microphone & camera permissions for Livebook Desktop on Mac OS

[mat-hek]

Hello, we have a small demo livebook that uses the microphone via OS API. Unfortunately, it doesn't work on Livebook desktop, unless you configure a custom runtime. I suppose it's because of permissions, but I cannot allow Livebook to use the microphone - it doesn't show up in that setting. I believe that would be the case for the camera too.

Would it be possible to make these permissions configurable for Livebook desktop?

[josevalim]

Livebook Desktop works as a regular page in your browser. There is nothing exclusive to it. We have both Kino.Input.audio and Kino.Input.image which work for me from the desktop. Do they work for you?

[josevalim]

Compiling, shipping and using that via another NIF doesn't look easy - correct me if I'm wrong.

You may be able to compile it a a Swift program and ship with the executables, and then shell out to it. So no need for the NIF integration.

[mat-hek]

You may be able to compile it a a Swift program and ship with the executables

Okay, I guess I'd have to ship it for Intel and ARM and then run the right executable?

[josevalim]

I think you can ship a single executable with both. @wojtekmach will know better.

[wojtekmach]

I'm not sure if a Swift executable would work when shelling out from Livebook Desktop runtime. I think the executable might be treated as a separate program and need to have Info.plist with entitlements, be code signed and notarised, etc.

Regarding the NIF route, here's a proof of concept:

Mix.install([
  {:objc, github: "wojtekmach/objc"}
])

defmodule HelloObjc.MixProject do
  use Mix.Project

  def project do
    [
      app: :hello_objc,
      version: "1.0.0"
    ]
  end
end

defmodule Main do
  def main do
    HelloObjC.hello()
  end
end

defmodule HelloObjC do
  use ObjC, compile: "-lobjc -framework AVFoundation"

  defobjc(:hello, 0, ~S"""
  // c_src/darwin.m or something like that
  #import "AVFoundation/AVFoundation.h"

  void request_permission() {
      if ([AVCaptureDevice authorizationStatusForMediaType:AVMediaTypeAudio] == AVAuthorizationStatusAuthorized) {
          printf("mic authorized\n");
      }
      else {
          printf("mic not authorized\n");
      }

      [AVCaptureDevice requestAccessForMediaType:AVMediaTypeAudio completionHandler:^(BOOL granted) {
          if (granted) {
              printf("mic granted\n");
          }
          else {
              printf("mic not granted\n");
          }
      }];
  }

  // c_src/linux.c
  // void request_permission() {}

  // c_src/foo_nif.c
  static ERL_NIF_TERM hello(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[])
  {
      request_permission();
      return enif_make_string(env, "Hello world!", ERL_NIF_LATIN1);
  }
  """)
end

Main.main()

This uses https://github.com/wojtekmach/objc, a small library which just makes easier to experiment with NIFs inline.

The full compile command that you'd put into your Makefile is this:

clang -o hello.so c_src/hello.m -I$OTP_ROOT/usr/include -lobjc -framework AVFoundation

[mat-hek]

Thanks, that helps a lot. It doesn't work for me though :P It returns hello world, but the popup won't show up and the permissions are not granted. Does it work for you?

[wojtekmach]

It does, I wouldn't have sent the snippet otherwise. :) Does it print "mic authorized" or "mic not authorized" at all? It seems the popup is only ever shown once per app. When I was testing this snippet in the terminal the app was considered iTerm2. On subsequent runs, no popup. To control the setting I had to change it manually in System Preferences.

Running this command reset the permissions for all apps asking for Microphone access:

$ tccutil reset Microphone

Apparently you can do it per app too:

$ tccutil reset Microphone BUNDLE_ID

(https://apple.stackexchange.com/a/360610)

The Bundle ID for Livebook is dev.livebook.Livebook.

[mat-hek]

Thanks for the build, still getting the same error though ¯\_(ツ)_/¯

They were copied on my machine though not at the exact same place in Info.plist

you're right, I haven't noticed, but for some reason they are not considered

If you could paste a livebook I'm happy to test stuff.

I'm running the exact same livebook as you :D

[wojtekmach]

I see maybe the system does not see the latest version of the .app somehow.

Remove /Applications/Livebook.app if you have it.

Try running:

/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister /path/to/Livebook.app

and maybe reset the permissions if you haven't, tccutil reset Microphone, and lmk.

I didn't notarise that Livebook.app.zip, if you run into any quarantine issues before unzipping do:

$ xattr -l Livebook.app.zip
$ xattr -d X Livebook.app.zip # X is the quarantine one, something like com.apple.quarantine maybe

[mat-hek]

I'll try. Anyway, it seems there's something wrong with my mac or I'm missing something. I marked the PR #2177 as ready for review and I'll try to add your snippet to my lib. Are you sure that running it as an external script won't work? That way it would at least not crash the BEAM in cases like mine.

[wojtekmach]

Ok, let us know how the testing goes. I don't think we need the entitlements, just the Info.plist the changes but I will test with and without.

If you are able to test it on another Mac, that'd be appreciated. With Parallels free trial you can easily create a Mac guest VM so that's a pretty easy way to test stuff.

I'd rather not merge this unless this works for you.

Are you sure that running it as an external script won't work?

No I am not, if you explore that route definitely let us know how it goes.

[mat-hek]

Just tested on my friend's Mac and it worked ;)

[wojtekmach]

Awesome, thanks @mat-hek!

[mat-hek]

Hi, didn't find time to create a proper library for requesting permissions, so for now I merged that into our plugin. It seems it works, even on my laptop 😄 Thanks everyone for your help ;)

[wojtekmach]

Great thanks for the heads up!